diff --git a/.gitignore b/.gitignore index e604c99de..18508f822 100644 --- a/.gitignore +++ b/.gitignore @@ -14,8 +14,12 @@ # Auto-generated docs Markdown + llms.txt index + skills discovery index # (regenerated on every build by scripts/generate-static-md.js, - - +# generate-llms-txt.js and generate-skill-index.js via prestart/prebuild). +# SKILL.md and OpenAPI.md are hand-written, so keep them tracked. +/static/docs/*.md +!/static/docs/SKILL.md +!/static/docs/OpenAPI.md +/static/docs/llms.txt /static/.well-known/ # Claude diff --git a/static/docs/Device-reservation.md b/static/docs/Device-reservation.md deleted file mode 100644 index 31d564e4c..000000000 --- a/static/docs/Device-reservation.md +++ /dev/null @@ -1,77 +0,0 @@ -# Device Reservation on Private Device Cloud - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -**TestMu AI Device Reservation** enables users to reserve private cloud devices in **advance**, ensuring uninterrupted access during critical testing windows. By scheduling devices ahead of time, teams can **prevent conflicts**, eliminate last-minute delays, and run both manual and automated tests with greater reliability. This becomes particularly valuable for time-sensitive releases, large-scale regression cycles, and continuous integration pipelines, where guaranteed device availability is essential. - -Beyond preventing scheduling conflicts, Device Reservation also helps organizations **optimize resource utilization**, enforce fair usage policies, and gain clearer visibility into device allocation across teams, leading to more efficient and predictable testing operations. - -> Device Reservation feature is currently in **closed beta** and continuously improving based on user feedback. please reach out via **24×7 chat** or email us at **support@testmuai.com** to enable it for your organization and try it out. - -## Use Cases - -- **Release-Day Reliability**: Reserve devices in advance to guarantee uninterrupted testing during production releases or hotfix rollouts. -- **Parallel Test Execution**: Secure multiple devices for running automated test suites in parallel, speeding up regression cycles. -- **Cross-Team Coordination**: Schedule device bookings across QA, Dev, and Ops teams to align on shared testing timelines. -- **Peak Hour Management**: Prevent conflicts during high-demand periods by ensuring fair access and availability through reservations. -- **Optimized Resource Utilization**: Track and manage reservations to avoid idle devices and maximize the ROI of your private cloud setup. - -## Device Reservation Workflow - -### Accessing the App Live Dashboard -1. Open the [App Live Dashboard](https://applive.lambdatest.com/app). -2. Enable the toggle for Private Devices (**Available only for private device customers**). -3. The User will see two buttons: - - **My reservations** – View all ongoing and upcoming reservations with filter options. - - **Go to Reservations** – Open the calendar view to explore and manage bookings. - -First-time visitors will see an empty calendar and must create a reservation to begin scheduling devices. - -### Admin Privilege - -Admins have additional controls to manage device reservations, accessible inside **Go to Reservations** under the settings button (as shown below): - -- **Advance Booking Window** – Define how far in advance members can schedule reservations. -- **Max Reservation Duration** – Set the maximum duration allowed for a reservation (1–48 hours). -- **Active Reservations per Member** – Limit the number of active reservations per member (1–99). -- **Devices per Reservation** – Define how many devices can be booked in one reservation. - -### Calendar View and Permissions - -The **calendar** view differs for members and admins: - -- **Members:** - - The member view shows a calendar with reservation blocks for devices. - - `Blue blocks` represent the member’s own reservations, which can be edited or deleted. - - `Grey blocks` represent reservations made by other team members. Members cannot edit or delete these. - -- **Admins:** - - Admins can view and manage all reservations on the calendar. - - They have full control to create, edit, or delete any reservation. - -### Creating a Reservation - -To create a reservation, click **Reserve Device** from the calendar view. Then choose the type of reservation: - -- **Immediate Reservation** – Reserve a device instantly for 1 to 12 hours. - -- **Future Reservation** – Schedule a reservation for a later date/time within the allowed scheduling window. - -Select **one or multiple devices** based on availability and confirm the reservation. - -### Viewing and Editing Reservations - -After adding reservation slots, the user can view them under **My Reservations** and make edits or cancellations if required. - -### Session Expiry and Extensions - -These rules apply to both admins and members, except admins can still override reservations. - -- In the last **15 minutes** of a reservation, a pop-up appears notifying the user that the session is about to end. -- If no pre-booked reservation exists for the device, the user will have the option to extend the session. -- If another booking is already scheduled for the device, the extension option will still appear, but the request will **fail**. -- Once the reserved time ends, the device session is automatically closed and released. - -- Reservation cannot be edited after the slot has been started. -- If an **admin** deletes any reservation, it is **cancelled** immediately. -- In automation, reservations cannot be extended automatically in the last 15 minutes. Extensions must be handled manually. diff --git a/static/docs/accelq-integration.md b/static/docs/accelq-integration.md deleted file mode 100644 index af1a56eaf..000000000 --- a/static/docs/accelq-integration.md +++ /dev/null @@ -1,43 +0,0 @@ -# ACCELQ Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -ACCELQ is an AI Native Codeless automation testing and management platform. It enables you automate web, mobile, API, database, and packaged applications. - -By integrating TestMu AI with ACCELQ, you can perform web and mobile app automation on an online device farm of 3000+ real browsers, devices, and OS combinations, and accelerate your end-to-end digital testing efforts. - -## Prerequisites - -1. Make sure you have a TestMu AI account. If you don't have an account yet, sign up for free. - -2. To integrate TestMu AI with ACCELQ, you will need a TestMu AI username and access key. - -3. Install the ACCELQ Agent Dashboard. - -## Integrating TestMu AI From ACCELQ Agent Dashboard - -1. Open your ACCELQ Agent Dashboard. - -2. Once you are in the target agent, click **Edit Configuration**. - -3. You need to use the external device provider for web, mobile, or both platforms. For example, if you wish to perform web browser automation, select **Use external device provider for Web**. - -4. Now enter your TestMu AI username and access key and save/restart the ACCELQ Agent. - -### Executing The Tests - -1. Once the agent is operable, you can choose browser, device and OS combinations from the list that TestMu AI support. - -2. From the Run modal, select the agent from the AGENT/HOST section. After that, choose browser and device from the corresponding sections. - -## Capturing Views For Mobile Automation - -1. From the **Record View** dialog box, click **Cloud Provider**. - -2. Select *TestMu AI* from the **Provider** dropdown. Enter your TestMu AI username and access key, and click **Authenticate**. - -3. Once the authentication is successful, you can select the real devices that is fetched from the TestMu AI platform. - -4. In the **App Url**, provider your TestMu AI App URL. - -5. Now click **Start Capture** and it will launch your test session on the selected device. diff --git a/static/docs/accessibility-ai-issue-detection-agent.md b/static/docs/accessibility-ai-issue-detection-agent.md deleted file mode 100644 index 71a2fb45a..000000000 --- a/static/docs/accessibility-ai-issue-detection-agent.md +++ /dev/null @@ -1,28 +0,0 @@ -# AI Issue Detection Agent - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -AI-assisted issue analysis can help teams interpret findings faster, understand likely patterns, and move more quickly into triage. - -The guidance below describes where AI-assisted analysis fits in the Accessibility reporting workflow. It also shows how to pair automated interpretation with issue detail and human review so triage stays grounded and defensible. - -## Use this when - -Use AI-assisted issue interpretation when you want faster triage support, not as a replacement for rule-level validation or remediation review. - -## Important note - -AI issue analysis should support prioritization and understanding. It should not be treated as a substitute for manual validation, engineering review, or compliance sign-off. - -## How to use it in a review (onboarding) - -1. Open **[Issue Summary](/support/docs/accessibility-testing-dashboard-issue-summary/)** or **[All Issues](/support/docs/accessibility-testing-dashboard-all-issues/)** after a scan completes. -2. Select an issue or cluster where the UI exposes **AI-assisted** analysis (wording may appear as summary, explanation, or suggested next step). -3. Read the AI output as **hypothesis support**, verify against the DOM, design intent, and WCAG rule text in the **[rule repositories](/support/docs/accessibility-web-rule-repository/)** where applicable. -4. If the suggestion is wrong, capture that feedback through your usual QA process so the team does not over-trust automation. -5. Pair AI hints with the **[Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/)** for engineering-ready fix order. - -## Related docs - -- [Issue Summary](/support/docs/accessibility-testing-dashboard-issue-summary/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-android-rule-fixed-orientation-lock.md b/static/docs/accessibility-android-rule-fixed-orientation-lock.md deleted file mode 100644 index 5889062ee..000000000 --- a/static/docs/accessibility-android-rule-fixed-orientation-lock.md +++ /dev/null @@ -1,35 +0,0 @@ -# Fixed Orientation Lock - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Android activities must support both portrait and landscape orientations unless a specific orientation is essential to the functionality. - -**WCAG Reference** -**Maps to:** WCAG 1.3.4 Orientation | **Applies to:** WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.1 | **Level:** AA | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/orientation.html) - -## What this rule checks - -The scanner flags activities with `android:screenOrientation` set to a fixed value (`portrait`, `landscape`, `sensorPortrait`, `sensorLandscape`) in the manifest, preventing device rotation. - -## Why it matters - -Users with motor disabilities may mount their device in a fixed orientation (e.g., attached to a wheelchair). Users with low vision may prefer landscape to display larger text. Locking orientation removes this choice and can make the app physically uncomfortable or impossible to use. - -## Common failure patterns - -- `android:screenOrientation="portrait"` set globally on all activities -- orientation locked programmatically with `setRequestedOrientation()` for non-essential reasons -- splash screens or onboarding locked to portrait that propagate the lock to subsequent screens - -## Remediation guidance - -- remove `android:screenOrientation` from the manifest (defaults to user-controlled rotation) -- only lock orientation when the content genuinely requires it (e.g., a camera viewfinder) -- test the app in both orientations to confirm layouts adapt correctly -- document any essential orientation requirement with a clear justification - -## Related docs - -- [Android Rule Repository](/support/docs/accessibility-android-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-android-rule-inaccessible-text-focus.md b/static/docs/accessibility-android-rule-inaccessible-text-focus.md deleted file mode 100644 index aaa6dac60..000000000 --- a/static/docs/accessibility-android-rule-inaccessible-text-focus.md +++ /dev/null @@ -1,36 +0,0 @@ -# Inaccessible Text Focus - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Meaningful text content in Android apps must be focusable by assistive technology so users can read it sequentially. - -**WCAG Reference** -**Maps to:** WCAG 4.1.2 Name, Role, Value | **Applies to:** WCAG 2.0, WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.0 | **Level:** A | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/name-role-value.html) - -## What this rule checks - -The scanner flags `TextView` and other text-bearing elements that are not reachable through TalkBack navigation because they are marked as not important for accessibility or are hidden behind non-focusable containers. - -## Why it matters - -If meaningful text cannot receive TalkBack focus, blind and low-vision users miss the information entirely. This is especially problematic for status messages, instructions, and error text. - -## Common failure patterns - -- `TextView` elements inside containers with `importantForAccessibility="noHideDescendants"` -- dynamically added text views that do not inherit accessibility properties -- text elements positioned off-screen (used for visual spacing) that are still rendered but not focusable -- `TextView` elements with `focusable="false"` that contain meaningful information - -## Remediation guidance - -- ensure text elements containing meaningful content have `importantForAccessibility="yes"` -- do not suppress descendant accessibility on containers that hold readable text -- remove decorative text from the accessibility tree with `importantForAccessibility="no"` (only when truly decorative) -- test by swiping through the screen with TalkBack and confirming all meaningful text is announced - -## Related docs - -- [Android Rule Repository](/support/docs/accessibility-android-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-android-rule-insufficient-color-ratio.md b/static/docs/accessibility-android-rule-insufficient-color-ratio.md deleted file mode 100644 index 4133af367..000000000 --- a/static/docs/accessibility-android-rule-insufficient-color-ratio.md +++ /dev/null @@ -1,36 +0,0 @@ -# Insufficient Color Ratio - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Text and meaningful visual elements in Android apps must maintain sufficient contrast against their backgrounds for readability. - -**WCAG Reference** -**Maps to:** WCAG 1.4.3 Contrast (Minimum) | **Applies to:** WCAG 2.0, WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.0 | **Level:** AA | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/contrast-minimum.html) - -## What this rule checks - -The scanner measures the contrast ratio between foreground text or icon colors and their background, flagging pairs that fall below the 4.5:1 ratio for normal text or 3:1 for large text. - -## Why it matters - -Users with low vision, color-vision deficiencies, or those using devices in bright sunlight depend on adequate contrast to read text and distinguish UI elements. Low contrast makes content illegible. - -## Common failure patterns - -- light gray text on white backgrounds in settings or list items -- placeholder text in input fields with very low contrast -- status indicators using pale colors on light backgrounds -- disabled-state styling that falls below minimum ratios while the element remains interactive - -## Remediation guidance - -- verify text contrast against backgrounds using a color contrast analyzer -- use Material Design color tokens that are pre-validated for contrast compliance -- ensure all text states (enabled, disabled, error, hint) meet minimum ratios -- test on multiple devices and brightness levels to catch theme-specific failures - -## Related docs - -- [Android Rule Repository](/support/docs/accessibility-android-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-android-rule-insufficient-target-spacing.md b/static/docs/accessibility-android-rule-insufficient-target-spacing.md deleted file mode 100644 index 40a92aa03..000000000 --- a/static/docs/accessibility-android-rule-insufficient-target-spacing.md +++ /dev/null @@ -1,36 +0,0 @@ -# Insufficient Target Spacing - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Adjacent interactive Android elements must have adequate spacing between them to prevent accidental activation of neighboring controls. - -**WCAG Reference** -**Maps to:** WCAG 2.5.5 Target Size (Enhanced) | **Applies to:** WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.1 | **Level:** AAA | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/target-size-enhanced.html) - -## What this rule checks - -The scanner measures the gap between adjacent clickable or focusable elements and flags pairs that are too close together, increasing the risk of mis-taps. - -## Why it matters - -Even when individual touch targets meet the minimum size, placing them too close together makes it difficult for users with motor impairments to tap one without accidentally hitting the other. Sufficient spacing reduces errors and improves usability. - -## Common failure patterns - -- toolbar buttons placed edge-to-edge with no gap -- action buttons in list items (edit, delete, share) packed tightly together -- bottom navigation icons with minimal spacing on small screens -- floating action buttons positioned near other interactive elements - -## Remediation guidance - -- add at least 8dp of spacing between adjacent interactive elements -- use `android:layout_margin` or Compose `Modifier.padding` to create clear separation -- consider grouping related actions into an overflow menu to reduce density -- test on the smallest supported screen size to confirm spacing remains adequate - -## Related docs - -- [Android Rule Repository](/support/docs/accessibility-android-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-android-rule-interactive-role-undefined.md b/static/docs/accessibility-android-rule-interactive-role-undefined.md deleted file mode 100644 index a3a6a8506..000000000 --- a/static/docs/accessibility-android-rule-interactive-role-undefined.md +++ /dev/null @@ -1,35 +0,0 @@ -# Interactive Role Undefined - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Interactive Android elements must expose a meaningful accessibility role so TalkBack can announce what the element is and how to interact with it. - -**WCAG Reference** -**Maps to:** WCAG 4.1.2 Name, Role, Value | **Applies to:** WCAG 2.0, WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.0 | **Level:** A | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/name-role-value.html) - -## What this rule checks - -The scanner flags interactive elements (buttons, switches, checkboxes, sliders) that do not expose a recognized accessibility role to the Android accessibility framework. - -## Why it matters - -TalkBack announces both the accessible name and the role ("Button", "Switch", "Checkbox") to tell users what kind of control they are interacting with. Without a role, users hear the name but cannot tell whether to tap, swipe, or double-tap. - -## Common failure patterns - -- custom clickable `View` elements with no `accessibilityClassName` override -- `LinearLayout` or `FrameLayout` used as a button with an `onClickListener` but no role -- Jetpack Compose elements using `clickable` without a `role` parameter in `Modifier.semantics` - -## Remediation guidance - -- use native Android widgets (`Button`, `Switch`, `CheckBox`) whenever possible -- for custom views, override `getAccessibilityClassName()` to return the appropriate widget class name -- in Jetpack Compose, set `role = Role.Button` (or the appropriate role) inside `Modifier.semantics { }` -- test with TalkBack to confirm the announced role matches the element's behavior - -## Related docs - -- [Android Rule Repository](/support/docs/accessibility-android-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-android-rule-mismatched-label-text.md b/static/docs/accessibility-android-rule-mismatched-label-text.md deleted file mode 100644 index 8253f4275..000000000 --- a/static/docs/accessibility-android-rule-mismatched-label-text.md +++ /dev/null @@ -1,35 +0,0 @@ -# Mismatched Label Text - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The accessible name of an Android element must contain its visible text so that voice-access users can activate controls by speaking what they see. - -**WCAG Reference** -**Maps to:** WCAG 2.5.3 Label in Name | **Applies to:** WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.1 | **Level:** A | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/label-in-name.html) - -## What this rule checks - -The scanner compares the visible text of interactive elements with their computed accessible name (from `contentDescription` or labeling relationships) and flags mismatches where the visible text is not included in the accessible name. - -## Why it matters - -Voice Access users say "tap [visible text]" to interact with controls. If the accessible name differs from the visible text, the voice command fails and users cannot activate the control. - -## Common failure patterns - -- a button labeled "Submit" visually but with `contentDescription` set to "Send data" -- a `contentDescription` that abbreviates or paraphrases the visible text -- programmatic label updates that fall out of sync with the visible UI - -## Remediation guidance - -- ensure the `contentDescription` contains the visible text (it can be longer but must include the visible words) -- prefer not setting `contentDescription` on elements that already have visible text, letting the framework use the text directly -- audit dynamic content where visible text and accessible text are set independently -- test with Voice Access to confirm controls respond to their visible label - -## Related docs - -- [Android Rule Repository](/support/docs/accessibility-android-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-android-rule-misplaced-field-label.md b/static/docs/accessibility-android-rule-misplaced-field-label.md deleted file mode 100644 index f2fba03b3..000000000 --- a/static/docs/accessibility-android-rule-misplaced-field-label.md +++ /dev/null @@ -1,35 +0,0 @@ -# Misplaced Field Label - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Form field labels in Android apps must be positioned correctly relative to their associated input so assistive technologies can pair them accurately. - -**WCAG Reference** -**Maps to:** WCAG 3.3.2 Labels or Instructions | **Applies to:** WCAG 2.0, WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.0 | **Level:** A | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/labels-or-instructions.html) - -## What this rule checks - -The scanner verifies that labels associated with form fields through `labelFor` or proximity are positioned in a way that creates a clear visual and programmatic association. - -## Why it matters - -If a label appears far from its field or near the wrong field, TalkBack may announce the wrong label for a control. This causes confusion and can lead users to enter data in the wrong field. - -## Common failure patterns - -- a `TextView` label placed below the input field instead of above or beside it -- labels that are visually near the correct field but programmatically linked to a different field via `labelFor` -- dynamic layouts where field reordering breaks label-field proximity - -## Remediation guidance - -- place labels immediately above or to the start of their associated input field -- use `android:labelFor` on the label `TextView` pointing to the correct input `id` -- in Jetpack Compose, use `Modifier.semantics { contentDescription = "..." }` or explicit label composables -- test with TalkBack to confirm the announced label matches the visual label for each field - -## Related docs - -- [Android Rule Repository](/support/docs/accessibility-android-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-android-rule-missing-field-label.md b/static/docs/accessibility-android-rule-missing-field-label.md deleted file mode 100644 index 222a8ecd0..000000000 --- a/static/docs/accessibility-android-rule-missing-field-label.md +++ /dev/null @@ -1,34 +0,0 @@ -# Missing Field Label - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Input fields need labels that clearly describe the expected input. - -**WCAG Reference** -**Maps to:** WCAG 3.3.2 Labels or Instructions | **Applies to:** WCAG 2.0, WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.0 | **Level:** A | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/labels-or-instructions.html) - -## What this rule checks - -This rule checks whether Android input controls expose clear labels that describe the purpose of the field to assistive technologies. - -## Why it matters - -Users need to know what information a field expects before they can complete forms accurately with TalkBack or other assistive tools. - -## Common failure patterns - -- editable fields with placeholder-only guidance -- inputs without associated labels in custom layouts -- reused generic labels that do not identify the field clearly - -## Remediation guidance - -- provide a visible and accessible label for each field -- connect the label and field programmatically when the UI framework supports it -- avoid relying on hints alone when they disappear after input starts - -## Related docs - -- [Android Rule Repository](/support/docs/accessibility-android-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-android-rule-missing-image-alt.md b/static/docs/accessibility-android-rule-missing-image-alt.md deleted file mode 100644 index cd5ebb724..000000000 --- a/static/docs/accessibility-android-rule-missing-image-alt.md +++ /dev/null @@ -1,34 +0,0 @@ -# Missing Image Alt - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Meaningful images in Android experiences need an accessible description. - -**WCAG Reference** -**Maps to:** WCAG 1.1.1 Non-text Content | **Applies to:** WCAG 2.0, WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.0 | **Level:** A | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/non-text-content.html) - -## What this rule checks - -This rule checks whether meaningful Android images expose a useful accessibility description for TalkBack and related assistive technologies. - -## Why it matters - -If meaningful imagery is silent to assistive technology, users can miss content, state, or action context that is visible on screen. - -## Common failure patterns - -- product or profile images with no `contentDescription` -- icon buttons with visual meaning but no accessible label -- decorative images announced unnecessarily - -## Remediation guidance - -- add a clear `contentDescription` to meaningful images -- use an empty description for decorative content when appropriate -- verify that image-based controls expose both purpose and action - -## Related docs - -- [Android Rule Repository](/support/docs/accessibility-android-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-android-rule-missing-input-value.md b/static/docs/accessibility-android-rule-missing-input-value.md deleted file mode 100644 index 813f822d0..000000000 --- a/static/docs/accessibility-android-rule-missing-input-value.md +++ /dev/null @@ -1,35 +0,0 @@ -# Missing Input Value - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Android input fields must expose their current value to assistive technologies so TalkBack users can verify what they have entered. - -**WCAG Reference** -**Maps to:** WCAG 4.1.2 Name, Role, Value | **Applies to:** WCAG 2.0, WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.0 | **Level:** A | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/name-role-value.html) - -## What this rule checks - -The scanner verifies that `EditText`, `AutoCompleteTextView`, and custom input elements expose their current text value through the accessibility framework. - -## Why it matters - -TalkBack users cannot see the screen to verify their input. If the current value is not announced, users cannot confirm what they typed, leading to errors in forms, searches, and text entry. - -## Common failure patterns - -- custom input views that render text visually but do not expose it through `AccessibilityNodeInfo` -- masked fields (passwords, PINs) that hide the value without providing a character count or confirmation mechanism -- programmatically set text that does not trigger an accessibility event - -## Remediation guidance - -- use standard `EditText` or `TextInputLayout` which expose values automatically -- for custom inputs, populate `AccessibilityNodeInfo.setText()` with the current value -- fire `TYPE_VIEW_TEXT_CHANGED` events when the value changes programmatically -- test with TalkBack by focusing the input field and confirming the current value is announced - -## Related docs - -- [Android Rule Repository](/support/docs/accessibility-android-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-android-rule-missing-screen-title.md b/static/docs/accessibility-android-rule-missing-screen-title.md deleted file mode 100644 index b1303c40f..000000000 --- a/static/docs/accessibility-android-rule-missing-screen-title.md +++ /dev/null @@ -1,36 +0,0 @@ -# Missing Screen Title - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Each screen or activity in an Android app must have a descriptive title so users can identify where they are in the app. - -**WCAG Reference** -**Maps to:** WCAG 2.4.2 Page Titled | **Applies to:** WCAG 2.0, WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.0 | **Level:** A | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/page-titled.html) - -## What this rule checks - -The scanner verifies that each `Activity` or navigable screen exposes a title through the `android:label` attribute, toolbar title, or accessibility pane title. - -## Why it matters - -TalkBack announces the screen title when a user navigates to a new screen. Without a title, users cannot tell which screen they are on, making navigation through an app disorienting and slow. - -## Common failure patterns - -- activities with no `android:label` in the manifest -- fragments used as top-level destinations without setting an accessibility pane title -- dynamic screen titles that are set programmatically but omit the TalkBack announcement -- toolbar titles that are set to empty strings or the app name on every screen - -## Remediation guidance - -- set `android:label` on every `` in `AndroidManifest.xml` -- for fragment-based navigation, call `ViewCompat.setAccessibilityPaneTitle(view, "Screen Name")` -- ensure toolbar titles reflect the specific screen, not just the app name -- test by navigating between screens with TalkBack to confirm announcements - -## Related docs - -- [Android Rule Repository](/support/docs/accessibility-android-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-android-rule-missing-view-accessibility.md b/static/docs/accessibility-android-rule-missing-view-accessibility.md deleted file mode 100644 index 47591aba9..000000000 --- a/static/docs/accessibility-android-rule-missing-view-accessibility.md +++ /dev/null @@ -1,35 +0,0 @@ -# Missing View Accessibility - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Meaningful Android views must be accessible to assistive technologies rather than being invisible to the accessibility framework. - -**WCAG Reference** -**Maps to:** WCAG 4.1.2 Name, Role, Value | **Applies to:** WCAG 2.0, WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.0 | **Level:** A | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/name-role-value.html) - -## What this rule checks - -The scanner flags views that carry meaningful content or functionality but are marked as `importantForAccessibility="no"` or have `focusable="false"` when they should be focusable. - -## Why it matters - -When a view is excluded from the accessibility tree, TalkBack users cannot perceive or interact with it. If the view carries meaningful information or actions, those users lose access entirely. - -## Common failure patterns - -- setting `android:importantForAccessibility="no"` on views that contain meaningful text or actions -- container layouts that suppress child accessibility without individual evaluation -- custom views that do not call `sendAccessibilityEvent` after state changes - -## Remediation guidance - -- set `importantForAccessibility="yes"` on views that carry meaningful content -- only mark truly decorative elements as `importantForAccessibility="no"` -- ensure custom views implement `AccessibilityDelegate` or override accessibility methods -- test with TalkBack to confirm all meaningful elements are announced - -## Related docs - -- [Android Rule Repository](/support/docs/accessibility-android-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-android-rule-nested-control-issues.md b/static/docs/accessibility-android-rule-nested-control-issues.md deleted file mode 100644 index 0292d936a..000000000 --- a/static/docs/accessibility-android-rule-nested-control-issues.md +++ /dev/null @@ -1,35 +0,0 @@ -# Nested Control Issues - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Interactive Android elements should not be nested inside other interactive elements, as this creates ambiguous focus behavior for assistive technologies. - -**WCAG Reference** -**Maps to:** WCAG 4.1.2 Name, Role, Value | **Applies to:** WCAG 2.0, WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.0 | **Level:** A | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/name-role-value.html) - -## What this rule checks - -The scanner flags interactive views (buttons, switches, checkboxes) that are nested within other interactive containers (clickable layouts, other buttons), creating a confusing accessibility tree. - -## Why it matters - -When TalkBack encounters nested interactive elements, it may focus on the outer container and announce a combined label that obscures the inner controls, or it may skip the inner controls entirely. Users cannot predictably reach or activate individual nested actions. - -## Common failure patterns - -- a clickable `LinearLayout` that contains a `Button` and a `CheckBox` as children -- a list item with an `onClickListener` on the row and separate clickable actions inside it -- nested `RecyclerView` items with overlapping click targets - -## Remediation guidance - -- flatten the interactive hierarchy: either the container or its children should be interactive, not both -- use `android:importantForAccessibility="no"` on the container if individual child controls should receive focus -- for list items with multiple actions, use `AccessibilityAction` to expose additional actions through TalkBack's actions menu -- test by swiping through the screen with TalkBack to confirm each control is individually reachable - -## Related docs - -- [Android Rule Repository](/support/docs/accessibility-android-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-android-rule-non-accessible-interaction.md b/static/docs/accessibility-android-rule-non-accessible-interaction.md deleted file mode 100644 index 322213179..000000000 --- a/static/docs/accessibility-android-rule-non-accessible-interaction.md +++ /dev/null @@ -1,36 +0,0 @@ -# Non-accessible Interaction - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Interactive elements in Android apps must be operable through assistive technology gestures, not just direct touch. - -**WCAG Reference** -**Maps to:** WCAG 4.1.2 Name, Role, Value | **Applies to:** WCAG 2.0, WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.0 | **Level:** A | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/name-role-value.html) - -## What this rule checks - -The scanner identifies elements that respond to touch events but are not reachable or activatable through TalkBack gestures (double-tap to activate, swipe to navigate). - -## Why it matters - -TalkBack users interact with the screen through gestures: swiping to move focus and double-tapping to activate. If an element responds only to direct touch coordinates, TalkBack users cannot use it. - -## Common failure patterns - -- custom views that intercept `onTouchEvent` without implementing accessibility actions -- gesture-based interactions (long press, swipe) with no accessibility alternative -- views that use `OnTouchListener` without a corresponding `OnClickListener` -- canvas-drawn interactive elements with no accessibility node info - -## Remediation guidance - -- implement `OnClickListener` alongside any touch listeners -- add `AccessibilityAction` entries for custom gestures (long press, swipe) -- for canvas-drawn elements, create virtual child accessibility nodes using `ExploreByTouchHelper` -- test with TalkBack to confirm every interactive element can be reached by swiping and activated by double-tapping - -## Related docs - -- [Android Rule Repository](/support/docs/accessibility-android-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-android-rule-repository.md b/static/docs/accessibility-android-rule-repository.md deleted file mode 100644 index 89b6081b5..000000000 --- a/static/docs/accessibility-android-rule-repository.md +++ /dev/null @@ -1,52 +0,0 @@ -# Android Rule Repository - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Use this repository to browse Android accessibility rules and remediation guidance. - -This repository organizes Android app accessibility rules so common mobile findings map clearly to WCAG and practical remediation. Use it when a report points to a specific Android rule and you need more than a high-level checklist to plan fixes and retests. - -## When to use this - -Use this page when your Accessibility App or Native App Automation report points to a specific Android rule and you need deeper context than the summary checklist. - -## Priority Android rule pages - -| Rule page | WCAG | Use it for | -| --- | --- | --- | -| [Missing Image Alt](/support/docs/accessibility-android-rule-missing-image-alt/) | 1.1.1 | Meaningful images without accessible descriptions | -| [Missing Field Label](/support/docs/accessibility-android-rule-missing-field-label/) | 3.3.2 | Inputs without clear accessible labels | - -## Quick Reference - -| Rule Name | WCAG | Level | Impact | Description | Deep dive | -|-----------|------|-------|--------|-------------|-----------| -| Interactive Role Undefined | 4.1.2 | A | Critical | Interactive container elements (ViewGroup, FrameLayout, LinearLayout, etc.) that are clickable but lack a **role** for screen readers. A label (contentDesc) provides the **Name** only; WCAG 4.1.2 requires both Name and Role. Add a semantic child widget (Button, Switch, CheckBox, etc.) so the role is programmatically determinable, or ensure the container is not the sole interactive focus. | - | -| Missing Image Alt | 1.1.1 | A | Critical | Images lack alternative text descriptions that screen readers can announce to users. Add `android:contentDescription` to meaningful images or set to empty string for decorative images to ensure proper accessibility support. | [Open rule](/support/docs/accessibility-android-rule-missing-image-alt/) | -| Missing View Accessibility | 4.1.2 | A | Serious | Interactive elements like buttons, clickable views, or custom controls lack proper accessibility labels that describe their purpose. Ensure all interactive Views have clear `android:contentDescription` or associated labels for screen readers. | - | -| Unlabeled Checkbox Element | 4.1.2 | A | Serious | Checkbox controls are missing accessible names that describe their purpose or current state. Provide descriptive labels using `android:text`, `android:contentDescription`, or associated TextView labels so users understand what they're selecting. | - | -| Insufficient Color Ratio | 1.4.3 | AA | Serious | Text elements don't meet minimum contrast ratio requirements making them difficult to read for users with visual impairments. Maintain 4.5:1 ratio for normal text and 3:1 for large text by adjusting colors or background. | - | -| Missing Field Label | 3.3.2 | A | Serious | Input fields like EditText lack descriptive labels that identify their purpose to users. Provide clear labels using `android:hint`, associated TextView with `android:labelFor`, or TextInputLayout to describe expected input. | [Open rule](/support/docs/accessibility-android-rule-missing-field-label/) | -| Non-accessible Interaction | 4.1.2 | A | Serious | Custom interactive elements, buttons, or gesture-based controls cannot be accessed or activated through assistive technology. Ensure all interactive elements have proper focus handling, role definition, and descriptive labels. | - | -| Missing Screen Title | 2.4.2 | A | Serious | Activities or screens lack descriptive titles that help users understand their current location in the app. Set meaningful titles using `setTitle()` or `supportActionBar?.title` for proper navigation context. | - | -| Unlabeled Toggle Control | 4.1.2 | A | Serious | Switch, toggle, or similar controls are missing accessible names that explain what they control and their current state. Provide descriptive labels that clearly indicate what the toggle affects (e.g., "Wi-Fi enabled"). | - | -| Mismatched Label Text | 2.5.3 | A | Serious | The visible text label on a control differs from the programmatically accessible name, causing confusion for screen reader users. Ensure the accessible name includes or matches the visible text exactly. | - | -| Missing Input Value | 4.1.2 | A | Moderate | Text fields are missing programmatic values that assistive technology can read, making it difficult for users to understand current input state. Ensure EditText values are properly exposed and announced by screen readers. | - | -| Inaccessible Text Focus | 4.1.2 | A | Moderate | Text elements that receive focus lack proper accessibility properties, confusing screen reader users about their purpose. Either provide clear interactive roles and descriptions or remove focus capability for non-interactive text. | - | -| Misplaced Field Label | 3.3.2 | A | Moderate | Form labels are not positioned correctly relative to their controls or lack proper programmatic association. Ensure labels appear visually before controls and use `android:labelFor` for proper screen reader announcement order. | - | -| Nested Control Issues | 4.1.2 | A | Moderate | Interactive elements are incorrectly nested inside other interactive elements, creating focus traps and navigation confusion. Separate interactive elements into distinct, non-nested components to ensure proper accessibility navigation. | - | -| Unnamed Nested Element | 4.1.2 | A | Moderate | Elements within containers lack their own accessible names when they should be independently accessible. Provide distinct labels for nested interactive elements or mark decorative elements as non-focusable. | - | -| Fixed Orientation Lock | 1.3.4 | AA | Moderate | App restricts viewing to only portrait or landscape orientation without accessibility justification. Support both orientations or provide alternative access methods for users who cannot rotate their devices due to physical constraints. | - | -| Undersized Touch Target | 2.5.5 | AAA | Moderate | Interactive elements are smaller than the recommended minimum touch target size, making them difficult to activate for users with motor impairments. Ensure all touch targets are at least 48dp x 48dp. | - | -| Insufficient Target Spacing | 2.5.5 | AAA | Moderate | Interactive elements are placed too close together without adequate spacing, increasing risk of accidental activation. Provide sufficient spacing between adjacent touch targets to prevent targeting errors for users with limited dexterity. | - | - -## How to use this repository - -1. Open the rule page that matches the finding in your report. -2. Review the remediation guidance for Android UI components. -3. Apply the fix and retest the affected screen in App Scanner or Native App Automation. - -## Related docs - -- [Android checklist](/support/docs/accessibility-android-what-we-cover/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-android-rule-undersized-touch-target.md b/static/docs/accessibility-android-rule-undersized-touch-target.md deleted file mode 100644 index 257a05290..000000000 --- a/static/docs/accessibility-android-rule-undersized-touch-target.md +++ /dev/null @@ -1,36 +0,0 @@ -# Undersized Touch Target - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Interactive Android elements must meet a minimum touch target size of 48x48 dp to ensure users with motor impairments can tap them reliably. - -**WCAG Reference** -**Maps to:** WCAG 2.5.5 Target Size (Enhanced) | **Applies to:** WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.1 | **Level:** AAA | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/target-size-enhanced.html) - -## What this rule checks - -The scanner measures the rendered size of clickable and focusable elements and flags those smaller than 48x48 dp (the Android accessibility guideline minimum). - -## Why it matters - -Users with motor impairments, tremors, or limited dexterity need adequately sized targets to tap accurately. Small targets cause frequent mis-taps, frustration, and can make features practically unreachable. - -## Common failure patterns - -- icon buttons (close, favorite, menu dots) rendered at 24x24 dp without additional touch padding -- inline text links with no minimum height enforcement -- compact toolbar actions sized to the icon dimensions only -- list item action buttons (delete, edit) with minimal padding - -## Remediation guidance - -- set `android:minWidth` and `android:minHeight` to at least 48dp on all interactive elements -- use `android:padding` or `TouchDelegate` to expand the touch area without changing the visual size -- in Jetpack Compose, use `Modifier.sizeIn(minWidth = 48.dp, minHeight = 48.dp)` -- Material Design components enforce this by default; prefer using them over custom implementations - -## Related docs - -- [Android Rule Repository](/support/docs/accessibility-android-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-android-rule-unlabeled-checkbox-element.md b/static/docs/accessibility-android-rule-unlabeled-checkbox-element.md deleted file mode 100644 index 622d2489c..000000000 --- a/static/docs/accessibility-android-rule-unlabeled-checkbox-element.md +++ /dev/null @@ -1,35 +0,0 @@ -# Unlabeled Checkbox Element - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Android checkbox elements must have an accessible label that describes what the checkbox controls. - -**WCAG Reference** -**Maps to:** WCAG 4.1.2 Name, Role, Value | **Applies to:** WCAG 2.0, WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.0 | **Level:** A | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/name-role-value.html) - -## What this rule checks - -The scanner identifies `CheckBox` and checkbox-like views that have no `contentDescription`, no associated ``, and no text content, leaving TalkBack with nothing to announce beyond "Checkbox, not checked." - -## Why it matters - -Without a label, TalkBack users hear "Checkbox" but cannot tell what option they are selecting or deselecting. This makes forms and settings screens unusable for blind users. - -## Common failure patterns - -- `CheckBox` views with no text and no `contentDescription` -- custom checkbox implementations that do not set accessibility text -- programmatically created checkboxes that skip label assignment - -## Remediation guidance - -- set the `android:text` attribute on the `CheckBox` element -- if visual text is provided by a separate `TextView`, use `android:labelFor` on the label pointing to the checkbox -- for icon-only checkboxes, set `contentDescription` to describe the purpose -- test with TalkBack and confirm the announced text clearly identifies the option - -## Related docs - -- [Android Rule Repository](/support/docs/accessibility-android-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-android-rule-unlabeled-toggle-control.md b/static/docs/accessibility-android-rule-unlabeled-toggle-control.md deleted file mode 100644 index f56ac1210..000000000 --- a/static/docs/accessibility-android-rule-unlabeled-toggle-control.md +++ /dev/null @@ -1,35 +0,0 @@ -# Unlabeled Toggle Control - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Android toggle controls (switches, toggle buttons) must have labels that describe what setting they control. - -**WCAG Reference** -**Maps to:** WCAG 4.1.2 Name, Role, Value | **Applies to:** WCAG 2.0, WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.0 | **Level:** A | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/name-role-value.html) - -## What this rule checks - -The scanner flags `Switch`, `ToggleButton`, and `SwitchCompat` elements that have no accessible label, leaving TalkBack to announce only the state ("On" / "Off") without context. - -## Why it matters - -Hearing "Switch, off" without knowing what the switch controls is useless. TalkBack users need both the label and the state to understand the setting and decide whether to change it. - -## Common failure patterns - -- `Switch` elements with no `android:text` and no `contentDescription` -- toggle controls where the label is a separate `TextView` not programmatically linked -- custom toggle implementations that do not expose accessibility text - -## Remediation guidance - -- set `android:text` on the toggle control to describe the setting -- if the label is a separate view, use `android:labelFor` on the label pointing to the toggle -- for Jetpack Compose, set a description in `Modifier.semantics { contentDescription = "..." }` -- test with TalkBack to confirm the full announcement includes both the label and the state - -## Related docs - -- [Android Rule Repository](/support/docs/accessibility-android-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-android-rule-unnamed-nested-element.md b/static/docs/accessibility-android-rule-unnamed-nested-element.md deleted file mode 100644 index 04d05c508..000000000 --- a/static/docs/accessibility-android-rule-unnamed-nested-element.md +++ /dev/null @@ -1,35 +0,0 @@ -# Unnamed Nested Element - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Nested Android elements that carry meaningful content must have their own accessible name rather than inheriting an ambiguous or empty label from a parent container. - -**WCAG Reference** -**Maps to:** WCAG 4.1.2 Name, Role, Value | **Applies to:** WCAG 2.0, WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.0 | **Level:** A | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/name-role-value.html) - -## What this rule checks - -The scanner identifies nested views (images, icons, text) inside a focusable parent container that do not have their own accessible label, causing their meaning to be lost in TalkBack's aggregated announcement of the parent. - -## Why it matters - -When a focusable container aggregates all child text into a single announcement, individual elements lose context. Users cannot distinguish between a product image, a price label, and an add-to-cart button if they are all merged into one announcement. - -## Common failure patterns - -- image icons inside a clickable row with no individual `contentDescription` -- status badges (unread count, priority indicator) that are visually meaningful but unnamed -- grouped elements where only the parent has an accessible name, hiding the details of children - -## Remediation guidance - -- add `contentDescription` to each meaningful child element within a group -- if the parent should aggregate children, compose the parent's `contentDescription` to include all relevant child information -- use `android:importantForAccessibility="no"` only on truly decorative children -- test with TalkBack at different granularity levels (character, word, element) to confirm all meaningful content is announced - -## Related docs - -- [Android Rule Repository](/support/docs/accessibility-android-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-android-rules.md b/static/docs/accessibility-android-rules.md deleted file mode 100644 index 5db850a33..000000000 --- a/static/docs/accessibility-android-rules.md +++ /dev/null @@ -1,34 +0,0 @@ -# Android Accessibility Rules - Quick Reference - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -These Android rule summaries are a quick reference for findings that appear in Accessibility App Scanner and Native App Automation reports. Use the repository pages for deeper remediation guidance when a finding needs rule-level detail. - -| Rule Name | WCAG | Level | Impact | Description | Deep dive | -|-----------|------|-------|--------|-------------|-----------| -| Interactive Role Undefined | 4.1.2 | A | Critical | Interactive container elements (ViewGroup, FrameLayout, LinearLayout, etc.) that are clickable but lack a **role** for screen readers. A label (contentDesc) provides the **Name** only; WCAG 4.1.2 requires both Name and Role. Add a semantic child widget (Button, Switch, CheckBox, etc.) so the role is programmatically determinable, or ensure the container is not the sole interactive focus. | - | -| Missing Image Alt | 1.1.1 | A | Critical | Images lack alternative text descriptions that screen readers can announce to users. Add `android:contentDescription` to meaningful images or set to empty string for decorative images to ensure proper accessibility support. | [Open rule](/support/docs/accessibility-android-rule-missing-image-alt/) | -| Missing View Accessibility | 4.1.2 | A | Serious | Interactive elements like buttons, clickable views, or custom controls lack proper accessibility labels that describe their purpose. Ensure all interactive Views have clear `android:contentDescription` or associated labels for screen readers. | - | -| Unlabeled Checkbox Element | 4.1.2 | A | Serious | Checkbox controls are missing accessible names that describe their purpose or current state. Provide descriptive labels using `android:text`, `android:contentDescription`, or associated TextView labels so users understand what they're selecting. | - | -| Insufficient Color Ratio | 1.4.3 | AA | Serious | Text elements don't meet minimum contrast ratio requirements making them difficult to read for users with visual impairments. Maintain 4.5:1 ratio for normal text and 3:1 for large text by adjusting colors or background. | - | -| Missing Field Label | 3.3.2 | A | Serious | Input fields like EditText lack descriptive labels that identify their purpose to users. Provide clear labels using `android:hint`, associated TextView with `android:labelFor`, or TextInputLayout to describe expected input. | [Open rule](/support/docs/accessibility-android-rule-missing-field-label/) | -| Non-accessible Interaction | 4.1.2 | A | Serious | Custom interactive elements, buttons, or gesture-based controls cannot be accessed or activated through assistive technology. Ensure all interactive elements have proper focus handling, role definition, and descriptive labels. | - | -| Missing Screen Title | 2.4.2 | A | Serious | Activities or screens lack descriptive titles that help users understand their current location in the app. Set meaningful titles using `setTitle()` or `supportActionBar?.title` for proper navigation context. | - | -| Unlabeled Toggle Control | 4.1.2 | A | Serious | Switch, toggle, or similar controls are missing accessible names that explain what they control and their current state. Provide descriptive labels that clearly indicate what the toggle affects (e.g., "Wi-Fi enabled"). | - | -| Mismatched Label Text | 2.5.3 | A | Serious | The visible text label on a control differs from the programmatically accessible name, causing confusion for screen reader users. Ensure the accessible name includes or matches the visible text exactly. | - | -| Missing Input Value | 4.1.2 | A | Moderate | Text fields are missing programmatic values that assistive technology can read, making it difficult for users to understand current input state. Ensure EditText values are properly exposed and announced by screen readers. | - | -| Inaccessible Text Focus | 4.1.2 | A | Moderate | Text elements that receive focus lack proper accessibility properties, confusing screen reader users about their purpose. Either provide clear interactive roles and descriptions or remove focus capability for non-interactive text. | - | -| Misplaced Field Label | 3.3.2 | A | Moderate | Form labels are not positioned correctly relative to their controls or lack proper programmatic association. Ensure labels appear visually before controls and use `android:labelFor` for proper screen reader announcement order. | - | -| Nested Control Issues | 4.1.2 | A | Moderate | Interactive elements are incorrectly nested inside other interactive elements, creating focus traps and navigation confusion. Separate interactive elements into distinct, non-nested components to ensure proper accessibility navigation. | - | -| Unnamed Nested Element | 4.1.2 | A | Moderate | Elements within containers lack their own accessible names when they should be independently accessible. Provide distinct labels for nested interactive elements or mark decorative elements as non-focusable. | - | -| Fixed Orientation Lock | 1.3.4 | AA | Moderate | App restricts viewing to only portrait or landscape orientation without accessibility justification. Support both orientations or provide alternative access methods for users who cannot rotate their devices due to physical constraints. | - | -| Undersized Touch Target | 2.5.5 | AAA | Moderate | Interactive elements are smaller than the recommended minimum touch target size, making them difficult to activate for users with motor impairments. Ensure all touch targets are at least 48dp x 48dp. | - | -| Insufficient Target Spacing | 2.5.5 | AAA | Moderate | Interactive elements are placed too close together without adequate spacing, increasing risk of accidental activation. Provide sufficient spacing between adjacent touch targets to prevent targeting errors for users with limited dexterity. | - | - -> We are continuously expanding our App Accessibility guidelines. Visit this page for the latest updates and new requirements. - -## Related docs - -- [Android Rule Repository](/support/docs/accessibility-android-rule-repository/) -- [Android Coverage](/support/docs/accessibility-android-what-we-cover/) -- [Native App Automation Appium (Overview)](/support/docs/accessibility-native-app-automation-test/) diff --git a/static/docs/accessibility-android-what-we-cover.md b/static/docs/accessibility-android-what-we-cover.md deleted file mode 100644 index eb9d81a4d..000000000 --- a/static/docs/accessibility-android-what-we-cover.md +++ /dev/null @@ -1,47 +0,0 @@ -# Android Accessibility Checklist - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Use this page to see **which rules App Scanner / automation can report** (with links to rule-level guidance) and a **manual test checklist** for areas best validated by people. For remediation detail, use the [Android Rule Repository](/support/docs/accessibility-android-rule-repository/). - -**Other platforms:** [Web checklist](/support/docs/accessibility-web-what-we-cover/) · [iOS checklist](/support/docs/accessibility-ios-what-we-cover/) - -## Supported (rules in this product) - -| Rule | WCAG (maps) | Doc | -|------|-------------|-----| -| Interactive Role Undefined | 4.1.2 | [Rule →](/support/docs/accessibility-android-rule-interactive-role-undefined/) | -| Missing Image Alt | 1.1.1 | [Rule →](/support/docs/accessibility-android-rule-missing-image-alt/) | -| Missing View Accessibility | 4.1.2 | [Rule →](/support/docs/accessibility-android-rule-missing-view-accessibility/) | -| Unlabeled Checkbox Element | 4.1.2 | [Rule →](/support/docs/accessibility-android-rule-unlabeled-checkbox-element/) | -| Insufficient Color Ratio | 1.4.3 | [Rule →](/support/docs/accessibility-android-rule-insufficient-color-ratio/) | -| Missing Field Label | 3.3.2 | [Rule →](/support/docs/accessibility-android-rule-missing-field-label/) | -| Non-accessible Interaction | 4.1.2 | [Rule →](/support/docs/accessibility-android-rule-non-accessible-interaction/) | -| Missing Screen Title | 2.4.2 | [Rule →](/support/docs/accessibility-android-rule-missing-screen-title/) | -| Unlabeled Toggle Control | 4.1.2 | [Rule →](/support/docs/accessibility-android-rule-unlabeled-toggle-control/) | -| Mismatched Label Text | 2.5.3 | [Rule →](/support/docs/accessibility-android-rule-mismatched-label-text/) | -| Missing Input Value | 4.1.2 | [Rule →](/support/docs/accessibility-android-rule-missing-input-value/) | -| Inaccessible Text Focus | 4.1.2 | [Rule →](/support/docs/accessibility-android-rule-inaccessible-text-focus/) | -| Misplaced Field Label | 3.3.2 | [Rule →](/support/docs/accessibility-android-rule-misplaced-field-label/) | -| Nested Control Issues | 4.1.2 | [Rule →](/support/docs/accessibility-android-rule-nested-control-issues/) | -| Unnamed Nested Element | 4.1.2 | [Rule →](/support/docs/accessibility-android-rule-unnamed-nested-element/) | -| Fixed Orientation Lock | 1.3.4 | [Rule →](/support/docs/accessibility-android-rule-fixed-orientation-lock/) | -| Undersized Touch Target | 2.5.5 | [Rule →](/support/docs/accessibility-android-rule-undersized-touch-target/) | -| Insufficient Target Spacing | 2.5.5 | [Rule →](/support/docs/accessibility-android-rule-insufficient-target-spacing/) | - -## Manual test checklist - -Reserve time for **TalkBack**, **Switch Access**, **font scaling**, and **real devices** even when automated rules pass. Use [Understanding WCAG 2.2](https://www.w3.org/WAI/WCAG22/Understanding/) and Google's Material Design accessibility guidance with this checklist. - -| Area | Manual verification focus | -|------|---------------------------| -| **1.2.x Time-based media** | In-app media (captions, descriptions), validate with representative assets. | -| **1.3.2–1.3.3 · 1.4.x beyond contrast** | Sequence, sensory-only instructions, resize/reflow/hover content: interactive and visual review. | -| **2.x keyboard / focus / timing (full)** | Full flows with TalkBack; session and motion behavior, validate beyond rule snapshots. | -| **3.x language, errors, help** | Error suggestions, legal safeguards: product and compliance review. | -| **AAA criteria** | Optional stretch goals, plan when AAA is explicitly in scope. | - -## Related docs - -- [Android Rule Repository](/support/docs/accessibility-android-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-android-what-we-do-not-cover.md b/static/docs/accessibility-android-what-we-do-not-cover.md deleted file mode 100644 index a744ed043..000000000 --- a/static/docs/accessibility-android-what-we-do-not-cover.md +++ /dev/null @@ -1,3 +0,0 @@ -# Android Accessibility Manual Checklist - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). diff --git a/static/docs/accessibility-app-scanner-scan-configurations.md b/static/docs/accessibility-app-scanner-scan-configurations.md deleted file mode 100644 index 66b4e8693..000000000 --- a/static/docs/accessibility-app-scanner-scan-configurations.md +++ /dev/null @@ -1,118 +0,0 @@ -# Scan Configurations for Mobile App Accessibility (Manual) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Scan Configurations let users decide exactly which accessibility rules run on a **manual** mobile app scan before it starts. Instead of always running the full built-in rule set, users choose a WCAG conformance target, turn rule groups on or off, and hand-pick individual rules. TestMu AI remembers each user's choices and pre-fills them on the next scan. - -This brings mobile app accessibility to **parity with web accessibility**, where configurable scans already existed. - -## When to use this - -Use Scan Configurations when users want to scope a manual scan to only the relevant checks - for example, running a focused **WCAG 2.1 AA** pass, excluding AI-powered rules, or disabling individual rules that do not apply to a screen. Because rules outside the selection are skipped **before** they run on the device, narrower configurations also make scans faster. - -## How to configure a scan - -1. **Start a mobile app accessibility scan** through the Manual flow. -2. **Open the scan configuration panel.** It loads the **rules catalog** for the selected platform, grouped by category. The first time it is opened, it shows the **system defaults**. After that, it shows the user's **last-used settings**. -3. **Choose the WCAG version and level** (for example, WCAG 2.1 AA). The rule list automatically narrows to the rules included by that selection. -4. **Toggle the required rule groups** Best Practice, Beta, AI Detection -5. **Fine-tune individual rules.** Within each category, switch specific rules on or off. Anything left on is **Enabled**; anything switched off is **Disabled**. -6. **Run the scan.** Only the selected rules are evaluated on the device, and the report contains results for exactly those rules. - -{/* IMAGE PLACEHOLDER: App Scanner home screen with the Scan Settings summary (original screenshot, compressed to under 100 KB). Save the file at the path below, then uncomment. - -*/} - -## Configuration Options Details - -{/* IMAGE PLACEHOLDER: scan configuration panel (original screenshot). Save the file at the path below, then uncomment. - -*/} - -### WCAG version and level (with inheritance) - -Users select a single WCAG target such as `wcag2a`, `wcag21aa`, or `wcag22aaa`. The selection is **cumulative** higher versions and levels include the lower ones: - -- A higher **version** includes the lower ones: choosing **2.1** also brings in all **2.0** rules; choosing **2.2** brings in **2.0 + 2.1**. -- A higher **level** includes the lower ones: choosing **AA** also brings in all **A** rules; choosing **AAA** brings in **A + AA + AAA**. - -> **Example:** Selecting **WCAG 2.1 AA** runs every rule whose success criterion is in WCAG **2.0 or 2.1** at level **A or AA**. WCAG 2.2 rules and AAA-only rules are **not** included until the version or level is raised. - -### Rule groups - -Some rules carry a special tag in addition to their WCAG criterion. Each tag has its own toggle, and a rule runs only if **both** its WCAG criterion is in range **and** any tag it carries is switched on. - -| Toggle | What it controls | Default | -|---|---|---| -| **Best Practice** | Recommendations that go beyond strict WCAG conformance (for example, punctuation or emoji hygiene in labels). | ON | -| **Beta** | Newer rules still being refined (for example, traversal-order and certain contrast or touch-target checks). | ON | -| **AI-powered** | Rules evaluated by AI Detention Agent (for example, *Image in Text* detection). | OFF | - -A rule tagged "Best Practice" will **not** run if the Best Practice toggle is off, even if its WCAG criterion is in range. The same applies to Beta and AI-powered rules. If an expected rule does not run, check its group toggle. - -### Individual rules and categories - -The catalog is grouped into categories so that coverage can be reasoned about at a glance: - -- Accessibility Labels -- Accessible Elements -- Accessible Images -- Color Contrast -- Content Structure -- Display Orientation -- Focus and Navigation -- Input Purpose -- Readable Text and Layout -- Touch Target Size and Spacing - -Within any category, individual rules can be switched on or off. **An explicit selection always wins:** if a user disables a rule, it will not run even if the WCAG or group settings would otherwise include it. - -## Settings are remembered per user - -Each user's scan configuration is persisted on their last saved scan. - -- Every time a user runs a scan, the configuration is saved as that user's **last-used settings**, scoped to **the user account and the platform** (Android and iOS are remembered separately). -- The next time the user opens the scan configuration panel, it is **pre-filled with the configuration from the most recent scan** on that platform, so the setup does not have to be repeated. -- Changing settings and running a new scan **updates** the user's last-used settings to the newest run. -- This is **per user and per platform**: each user's Android and iOS setups are tracked independently, and teammates' setups do not affect one another. - -> In short: whatever a user configured on the most recent scan becomes the starting point for the next one. - -## Defaults for first-time users - -For users who have never run a configured mobile scan before (no saved settings yet), the panel starts from these **system defaults**. After the first scan, these are replaced by the user's last-used settings. - -| Setting | Default value | -|---|---| -| WCAG version and level | **WCAG 2.1 AA** | -| Best Practice | **ON** | -| Beta | **ON** | -| AI-powered | **OFF** | -| Individual rule selection | None pre-excluded (everything in range is enabled) | - -## What to expect in results - -- **Scoped results.** The report contains violations only for the enabled rules. Rules turned off (directly, or through WCAG or group settings) do not appear and do not affect the accessibility score for that scan. -- **Faster scans.** Because disallowed rules are skipped **before** they run on the device, narrowing the configuration generally makes scans quicker. -- **Configuration recorded with the test.** The WCAG version, group toggles, and the enabled or disabled rule lists are stored alongside the scan, so the team can always see how a given result was produced. The exact rules applied to a scan are also visible in the report (see [Scan Configurations via Capabilities (Automation)](/support/docs/accessibility-automation-scan-configurations/#what-to-expect-in-results)). - -## FAQ - -**Why did a Best Practice rule not run under WCAG 2.1 AA?** -A check runs only if its WCAG criterion is in range **and** every tag it carries is enabled. If the Best Practice toggle is off, Best Practice rules are skipped regardless of WCAG level. Turn the relevant group toggle on. - -**Does every scan need to be reconfigured?** -No. The last-used configuration is pre-filled automatically (per platform). Users adjust only what they need and run. The scan must be saved for the configuration to persist. - -**Do configuration changes affect teammates?** -No. Settings are saved per user and per platform. - -**Why are AI-powered rules off by default?** -They are opt-in by design: AI rules invoke AI evaluation and surface items for manual verification rather than automatic pass/fail. Users enable them when that depth is required. - -## Related docs - -- [Accessibility App Scanner (Overview)](/support/docs/accessibility-app-scanner/) -- [Scan Configurations via Capabilities (Automation)](/support/docs/accessibility-automation-scan-configurations/) -- [Tag Support for Accessibility Scans](/support/docs/accessibility-tag-support/) -- [Navigating the Dashboard](/support/docs/accessibility-testing-navigating-dashboard/) diff --git a/static/docs/accessibility-app-scanner.md b/static/docs/accessibility-app-scanner.md deleted file mode 100644 index ffcc818be..000000000 --- a/static/docs/accessibility-app-scanner.md +++ /dev/null @@ -1,44 +0,0 @@ -# Accessibility App Scanner (Overview) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Accessibility App Scanner is the **manual** mobile app workflow for reviewing accessibility issues on **real Android or iOS devices**, screen by screen, without writing Appium code. - -## When to use this - -Use App Scanner when you want to **inspect** Android or iOS app screens interactively and validate findings as you move through the app, ideal for exploratory passes, design reviews, or reproducing issues filed by users. - -## Prerequisites - -- Access to **Accessibility** and **App Scanner** for your organization -- A **build** of the app you are authorized to test (store build, enterprise IPA/APK, or uploaded artifact per your workflow) -- A **device profile** that matches your audience (OS version, screen size) - -## Onboarding: first manual scan - -1. **Upload or select** the application package in the App Scanner flow (follow in-product prompts for accepted formats). -2. **Pick a real device** pool that matches your target users (OS + locale). -3. **Launch the session** and wait until the app is installed and foregrounded. -4. **Navigate critical journeys** (onboarding, login, checkout, settings). After each major screen stabilizes, trigger the **scan** action the UI provides. -5. **Review issues** in the side panel or overlay, open each item to see rule text, element context, and suggested remediation where available. -6. **Save or end** the session so results appear under Accessibility **Reports** for dashboard analysis ([Navigating the Dashboard](/support/docs/accessibility-testing-navigating-dashboard/)). -7. Optional: run a **[Screen Reader](/support/docs/screen-reader-on-accessibility/)** pass on the same build for manual confirmation. - -## Typical workflow (summary) - -1. Upload or select the app. -2. Start the device session. -3. Scan important screens manually. -4. Review the issues. -5. Save the report and move into dashboard analysis if needed. - -## Product boundary - -App Scanner is a manual mobile-app workflow. If you are running Appium code, use [Native App Automation Appium (Overview)](/support/docs/accessibility-native-app-automation-test/). If you are authoring the flow in KaneAI, use [Mobile App Accessibility Testing](/support/docs/kaneai-mobile-app-accessibility/). - -## Related docs - -- [Scan Configurations for Mobile App Accessibility (Manual)](/support/docs/accessibility-app-scanner-scan-configurations/) -- [Native App Automation Appium (Overview)](/support/docs/accessibility-native-app-automation-test/) -- [Mobile App Accessibility Testing](/support/docs/kaneai-mobile-app-accessibility/) -- [Tag Support for Accessibility Scans](/support/docs/accessibility-tag-support/) diff --git a/static/docs/accessibility-appium-testng.md b/static/docs/accessibility-appium-testng.md deleted file mode 100644 index 0ebf59670..000000000 --- a/static/docs/accessibility-appium-testng.md +++ /dev/null @@ -1,59 +0,0 @@ -# Appium TestNG - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Use this guide when **Appium** drives Android or iOS tests and **TestNG** is your runner. Accessibility still depends on **session capabilities** plus **`lambda-accessibility-scan`** at stable UI states. - -## Prerequisites - -- Appium Java client (or matching stack) + TestNG -- App build reachable by the grid -- Accessibility entitlement - -## Onboarding path - -### 1. Configure capabilities in `@BeforeClass` / `@BeforeMethod` - -Illustrative Java pattern (adapt platform names, device, and app paths to your suite): - -```java -UiAutomator2Options options = new UiAutomator2Options(); -options.setDeviceName("Pixel.*"); -options.setApp("lt://APP_ID"); // or storage URL per your setup -options.setCapability("accessibility", true); -// options.setCapability("accessibility.autoscan", true); // optional -AppiumDriver driver = new AndroidDriver(new URL("https://mobile-hub.lambdatest.com/wd/hub"), options); -``` - -Use the **official capability set** your account documentation lists for the current Appium version; the critical addition is `"accessibility": true`. - -### 2. Call the scan hook after navigation - -```java -((JavascriptExecutor) driver).executeScript("lambda-accessibility-scan"); -``` - -Place this **after** explicit waits for loading spinners, animations, or lazy content. - -### 3. Run TestNG - -```bash -mvn test -``` - -### 4. Review reports - -Open **[Navigating the Dashboard](/support/docs/accessibility-testing-navigating-dashboard/)** and locate the session’s accessibility results. - -## Troubleshooting - -| Symptom | What to check | -|--------|----------------| -| Hook throws | Driver must be a session where accessibility capability was set; verify spelling `lambda-accessibility-scan`. | -| Empty report | Hook never called and autoscan off; or page never reached stable state. | - -## Related docs - -- [Native App Automation Appium (Overview)](/support/docs/accessibility-native-app-automation-test/) -- [Appium WebdriverIO](/support/docs/accessibility-appium-webdriverio/) -- [Tag Support for Accessibility Scans](/support/docs/accessibility-tag-support/) diff --git a/static/docs/accessibility-appium-webdriverio.md b/static/docs/accessibility-appium-webdriverio.md deleted file mode 100644 index 22138700d..000000000 --- a/static/docs/accessibility-appium-webdriverio.md +++ /dev/null @@ -1,61 +0,0 @@ -# Appium WebdriverIO - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Use this guide when **WebdriverIO** orchestrates **Appium** tests for Android or iOS. Accessibility is enabled on the **remote options** object, and you call **`lambda-accessibility-scan`** from async tests or hooks. - -## Prerequisites - -- WebdriverIO + `@wdio/appium-service` (or your chosen runner) configured for TestMu AI -- App artifact available to the grid -- Accessibility entitlement - -## Onboarding path - -### 1. Set capabilities in `wdio.conf.ts` (or `.js`) - -```ts -export const config = { -capabilities: [{ -platformName: 'Android', -'appium:deviceName': 'Pixel.*', -'appium:app': 'lt://APP_ID', -'accessibility': true, -// 'accessibility.autoscan': true, -}], -// host/user/key per your standard WDIO LambdaTest preset -}; -``` - -Match keys to your Appium server version (`appium:` prefix for W3C caps). - -### 2. Call the hook after screens load - -```ts -await browser.execute('lambda-accessibility-scan'); -``` - -Use WDIO’s **`waitUntil`** patterns before executing the hook so dynamic content is present. - -### 3. Run the suite - -```bash -npx wdio run wdio.conf.ts -``` - -### 4. Review Accessibility results - -Same dashboard path as other mobile automation ([overview](/support/docs/accessibility-native-app-automation-test/)). - -## Troubleshooting - -| Symptom | What to check | -|--------|----------------| -| `execute` undefined | Ensure the session is still active and you are not using a mocked driver in unit tests. | -| No report | Missing `accessibility: true` or hook never awaited. | - -## Related docs - -- [Native App Automation Appium (Overview)](/support/docs/accessibility-native-app-automation-test/) -- [Appium TestNG](/support/docs/accessibility-appium-testng/) -- [Tag Support for Accessibility Scans](/support/docs/accessibility-tag-support/) diff --git a/static/docs/accessibility-automation-scan-configurations.md b/static/docs/accessibility-automation-scan-configurations.md deleted file mode 100644 index a461307df..000000000 --- a/static/docs/accessibility-automation-scan-configurations.md +++ /dev/null @@ -1,115 +0,0 @@ -# Scan Configurations via Capabilities (Automation) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -In automation there is no scan-configuration panel. The scan scope is supplied through each test's **capabilities**, so every test carries its own configuration and the effective rule set is derived from these values. This is the automation counterpart to the manual [Scan Configurations](/support/docs/accessibility-app-scanner-scan-configurations/) panel. - -Per-rule enable/disable picking is a **Manual-only** feature and is not used in automation. In automation, the rule set is computed from the WCAG version plus the group toggles. - -## When to use this - -Use these capabilities when users **already run Appium** against TestMu AI real devices and want each accessibility scan scoped to a specific WCAG target and set of rule groups, without opening a UI. See [Native App Automation Appium (Overview)](/support/docs/accessibility-native-app-automation-test/) for the surrounding test setup and the `lambda-accessibility-scan` hook. - -## Prerequisites - -- An Appium test project targeting TestMu AI **real devices** (Android or iOS). -- `LT_USERNAME` / `LT_ACCESS_KEY` available to the process. -- Accessibility enabled on the session via the `accessibility` master capability. - -## Capabilities reference - -| Capability | Type | Allowed values | Default (RD App Automation) | -|---|---|---|---| -| `accessibility` | boolean | `true` / `false` | - (master toggle, must be `true` to enable a11y scanning) | -| `accessibility.wcagVersion` | string | `wcag2a`, `wcag2aa`, `wcag2aaa`, `wcag21a`, `wcag21aa`, `wcag21aaa`, `wcag22a`, `wcag22aa`, `wcag22aaa` | `wcag21aa` | -| `accessibility.bestPractice` | boolean | `true` / `false` | `true` | -| `accessibility.betaRules` | boolean | `true` / `false` | `true` | -| `accessibility.aiEnabled` | boolean | `true` / `false` | `false` | - -Notes: - -- Defaults apply only when `accessibility` is `true` and the specific capability is omitted. -- `accessibility.aiEnabled` is the same AI toggle used elsewhere in accessibility; it is reused here. -- A backend capability `accessibility.needsReview` also exists but is not part of the standard automation scan config (defaults off). - -## How the effective rule set is computed - -The rule set for a test is derived from the WCAG version and the group toggles, using the same logic as the manual panel: - -- **WCAG inheritance.** A higher version or level includes the lower ones. Selecting `wcag21aa` runs every rule whose success criterion is in WCAG **2.0 or 2.1** at level **A or AA**. WCAG 2.2 rules and AAA-only rules are not included until the version or level is raised. -- **Group combination.** Some rules carry a Best Practice, Beta, or AI tag. A rule runs only if **both** its WCAG criterion is in range **and** every tag it carries is switched on. For example, a Best Practice rule is skipped when `accessibility.bestPractice` is `false`, even if its WCAG criterion is in range. - -## Example: setting capabilities - -Enable accessibility, target WCAG 2.1 AA, keep Best Practice and Beta rules on, and leave AI rules off. - -```json -{ -"accessibility": true, -"accessibility.wcagVersion": "wcag21aa", -"accessibility.bestPractice": true, -"accessibility.betaRules": true, -"accessibility.aiEnabled": false -} -``` - -```java -MutableCapabilities caps = new MutableCapabilities(); -caps.setCapability("accessibility", true); -caps.setCapability("accessibility.wcagVersion", "wcag21aa"); -caps.setCapability("accessibility.bestPractice", true); -caps.setCapability("accessibility.betaRules", true); -caps.setCapability("accessibility.aiEnabled", false); -``` - -```python -caps = { -"accessibility": True, -"accessibility.wcagVersion": "wcag21aa", -"accessibility.bestPractice": True, -"accessibility.betaRules": True, -"accessibility.aiEnabled": False, -} -``` - -```javascript -const capabilities = { -"accessibility": true, -"accessibility.wcagVersion": "wcag21aa", -"accessibility.bestPractice": true, -"accessibility.betaRules": true, -"accessibility.aiEnabled": false, -}; -``` - -With these capabilities in place, the scan is triggered at each stable screen: - -```java -driver.executeScript("lambda-accessibility-scan"); -``` - -Only the rules in the effective set are evaluated, and the report for that build contains results for exactly those rules. - -## What to expect in results - -- **Scoped results.** Each scan reports violations only for the rules in the test's effective set. Rules outside the WCAG range or behind an off group toggle do not appear and do not affect the accessibility score for that scan. -- **Configuration recorded with the test.** The WCAG version and group toggles are stored alongside the scan, so the team can always see how a given result was produced. -- **Applied rules are visible in the report.** The report header shows the applied configuration as tags (for example, **WCAG 2.1 AA**, **Best Practices**, **Beta Rules**), and the **Applied Settings** panel lists every rule that was evaluated, grouped by category and searchable, so the exact selected rules can be confirmed for any scan. - -The report shows the applied WCAG target and group tags, and the **Applied Settings** panel lists the selected rules by category: - -{/* IMAGE PLACEHOLDER: report Applied Settings panel showing the rules evaluated for the scan. Save the screenshot at the path below, then uncomment. - -*/} - -## Product boundary - -This page covers configuring scans through **capabilities in automation**. For hand-picking individual rules and reusing last-used settings in the **manual** App Scanner flow, see [Scan Configurations (Manual)](/support/docs/accessibility-app-scanner-scan-configurations/). - -## Related docs - -- [Native App Automation Appium (Overview)](/support/docs/accessibility-native-app-automation-test/) -- [Scan Configurations (Manual)](/support/docs/accessibility-app-scanner-scan-configurations/) -- [Appium TestNG](/support/docs/accessibility-appium-testng/) -- [Appium WebdriverIO](/support/docs/accessibility-appium-webdriverio/) -- [Tag Support for Accessibility Scans](/support/docs/accessibility-tag-support/) diff --git a/static/docs/accessibility-automation-settings.md b/static/docs/accessibility-automation-settings.md deleted file mode 100644 index 39662bf2a..000000000 --- a/static/docs/accessibility-automation-settings.md +++ /dev/null @@ -1,56 +0,0 @@ -# Configure Accessibility Automation - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This document details the configuration options available for your automated accessibility tests, ensuring comprehensive and efficient assessments. - -To enable the accessibility testing within your automated test suite, set the accessibility: true in your configuration file. You can also define other settings capabilities as described below. - -```java -"accessibility" : true, // Enable accessibility testing -"accessibility.wcagVersion": "wcag21a", // Specify WCAG version (e.g., WCAG 2.1 Level A) -"accessibility.bestPractice": false, // Exclude best practice issues from results -"accessibility.needsReview": true // Include issues that need review -``` - -## Key Configurations Options - -By configuring these options effectively, you can tailor your accessibility tests to achieve a balance between thoroughness and efficiency, ensuring your web applications are inclusive for all users. - -### 1. Enable Accessibility Checks - -- **Purpose:** Activate accessibility testing within your automated test suite. This allows you to identify and address accessibility violations that might hinder usability for users with disabilities. -- **Implementation:** Set the `accessibility` property to `true` within your configuration file. - -```bash -accessibility : true -``` - -### 2. WCAG Version - -- **Purpose:** Define the specific Web Content Accessibility Guidelines (WCAG) version your tests should evaluate against. WCAG defines internationally recognized standards for web accessibility. -- **Options:** Common options include WCAG 2.0, WCAG 2.1 Level A, or WCAG 2.1 Level AAA. Each level represents increasing accessibility requirements. -- **Implementation:** Specify the desired WCAG version using the wcagVersion property within your configuration file. - -```bash -accessibility.wcagVersion: 'wcag21a' -``` - -### 3. Best Practices Checks - -- **Purpose:** Include or exclude checks that go beyond the defined WCAG standards but are considered good practices for optimal usability. -- **Default Value:** The default setting is false, focusing strictly on WCAG violations. -- **Implementation:** Enable best practice checks by setting bestPractice to true in your configuration file. - -```bash -accessibility.bestPractice: false -``` - -### 4. Needs Review - -- **Purpose:** Flag potential accessibility issues that might require human evaluation for definitive assessment. -- **Implementation:** Enable needs review checks by setting needsReview to true in your configuration file. This ensures potentially ambiguous issues get flagged for manual review. - -```bash -accessibility.needsReview: true -``` diff --git a/static/docs/accessibility-automation-test.md b/static/docs/accessibility-automation-test.md deleted file mode 100644 index e520d482c..000000000 --- a/static/docs/accessibility-automation-test.md +++ /dev/null @@ -1,96 +0,0 @@ -# Automating Accessibility Testing with Selenium - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This document walks you through the process of evaluating the accessibility of your website through the execution of automated tests using TestMu AI's Accessibility Tool. - -> Compatible only with Chrome and Edge browser versions >= 90. - -## Prerequisites - -- Your [TestMu AI Username and Access key](/support/docs/using-environment-variables-for-authentication-credentials/) -- Setup your local machine as per your testing framework. - -## Step-by-Step Guide to Trigger Your Test - -### Step 1: Setup Your Test Suite - -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**sample repo** -Download or Clone the code sample from the TestMu AI GitHub repository to run your tests. - - View on GitHub - -If you are using your own project, make sure you update the **Hub endpoint** in your tests file. By setting up the Hub endpoint, you establish the communication channel between your tests and the browser nodes, enabling effective test distribution and execution. - -Configure the desired capabilities based on your test requirements. For example: - -```java -DesiredCapabilities capabilities = new DesiredCapabilities(); -capabilities.setCapability("browserName", "chrome"); -capabilities.setCapability("version", "70.0"); -capabilities.setCapability("platform", "win10"); # If this cap isn't specified, it will just get the any available one -capabilities.setCapability("build", "LambdaTestSampleApp"); -capabilities.setCapability("name", "LambdaTestJavaSample"); -``` - -> You can generate capabilities for your test requirements with the help of our inbuilt 🔗 Capabilities Generator. - -### Step 2: Establish User Authentication - -Now, you need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in the [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). - -Run the below mentioned commands in your terminal to setup the CLI and the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -### Step 3: Configure the Necessary Capabilities - -To enable the accessibility testing within your automated test suite, set `accessibility: true` in your configuration file. - -There are two primary ways to run accessibility tests: - -#### 1. On-Demand Scans (via Hooks) -For precise control over which pages are scanned, you can trigger scans manually at specific points in your test execution. This is the recommended approach to reduce test execution time and focus only on relevant pages. - -To use this, simply enable accessibility in your capabilities: -```java -capability.setCapability("accessibility", true); // Enable accessibility testing -``` - -Then, trigger the scan directly within your test script when the desired page is fully loaded: -```java -// Execute the LambdaTest accessibility scan hook -driver.executeScript("lambda-accessibility-scan"); -``` -*Note: If you do not execute the hook in your script when using this method, no accessibility reports will be generated.* - -#### 2. Continuous Auto-Scanning -If you want the accessibility scanner to run automatically on every single page navigation throughout the entire test session without writing manual hooks, you can pass the `accessibility.autoscan` capability: - -```java -capability.setCapability("accessibility", true); // Enable accessibility testing -capability.setCapability("accessibility.autoscan", true); // Automatically scan all pages -``` - -#### Advanced Capabilities -You can also define other settings capabilities to refine your scan rules as described below: - -```java -capability.setCapability("accessibility.wcagVersion", "wcag21aa"); // Specify WCAG version (e.g., WCAG 2.1 Level AA) -capability.setCapability("accessibility.bestPractice", false); // Exclude best practice issues from results -capability.setCapability("accessibility.needsReview", true); // Include issues that need review -``` - -### Step 4: Execute and Monitor your Test - -Now execute your tests and visit the [Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard). Click on the Accessibility tab and check the report generated. - -```bash -mvn test -``` diff --git a/static/docs/accessibility-automation.md b/static/docs/accessibility-automation.md deleted file mode 100644 index 20db21413..000000000 --- a/static/docs/accessibility-automation.md +++ /dev/null @@ -1,34 +0,0 @@ -# Accessibility Automation (Overview) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Accessibility Automation lets teams run accessibility checks as part of automated web test execution. - -This overview introduces the Accessibility Automation workflow and supported execution model for running checks alongside automated web tests. You will see how framework-based runs relate to DevTools, Web Scanner, and Test Scheduling, and what to expect from grid execution, settings, and reporting. - -## When to use this - -Use this workflow when your team already runs automated web tests and wants repeatable Accessibility checks in the same execution path. - -## What this includes - -- grid-based automation execution -- supported framework integrations -- automation settings such as WCAG version, best practices, and needs review -- reporting through the Accessibility dashboard -- optional CI/CD and HyperExecute orchestration (see **[HyperExecute integration: Selenium accessibility](/support/docs/selenium-hyperexecute-accessibility-tests/)** when running on HyperExecute with Selenium) - -## Product boundary - -Accessibility Automation is not the same as DevTools, Web Scanner, or Test Scheduling. It is specifically for framework-driven execution. - -## Known limitation - -For Playwright Accessibility Automation, use Chrome. `pw-chromium` is currently not supported for accessibility report generation because the required extension does not load reliably in bundled Chromium sessions. - -## Related docs - -- [Selenium](/support/docs/accessibility-automation-test/) -- [HyperExecute integration: Selenium accessibility](/support/docs/selenium-hyperexecute-accessibility-tests/) -- [Configure Accessibility Automation](/support/docs/accessibility-automation-settings/) -- [CI/CD Integration Guide](/support/docs/accessibility-cicd-integration-guide/) diff --git a/static/docs/accessibility-capture-screenshot.md b/static/docs/accessibility-capture-screenshot.md deleted file mode 100644 index 821817a6e..000000000 --- a/static/docs/accessibility-capture-screenshot.md +++ /dev/null @@ -1,59 +0,0 @@ -# Capture Screenshot for Accessibility Testing - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Automatically capture visual evidence of accessibility issues during automated testing. Screenshots include red bounding boxes that highlight the exact location of accessibility violations, providing immediate visual context for debugging and issue resolution. - -## Enable Capture Screenshot - -### Manual Testing - -1. Navigate to **Accessibility Tool** → **Settings** → **Other Settings** -2. Enable **Capture Screenshots** -3. Run your accessibility scan to capture annotated screenshots - -### Automation Testing - -Add the `accessibility.captureScreenshot: true` capability to your test configuration: - -```javascript -'LT:Options': { -"accessibility": true, -"accessibility.wcagversion": "wcag21aa", -"accessibility.bestPractice": true, -"accessibility.needsReview": false, -"accessibility.captureScreenshot": true, -"accessibility.passedTestCases": false -} -``` - -## Understanding Screenshot Capture - -### Visual Annotations - -Screenshots include visual indicators that help identify accessibility issues: - -- **Red Bounding Boxes**: Highlight elements with accessibility violations -- **Full Page Coverage**: Captures entire webpage content, not just visible viewport -- **Issue Mapping**: Each highlighted area corresponds to specific violations in the report - -> This feature is available across all supported testing frameworks and in Accessibility Devtools - -## Results and Dashboard - -### Accessibility Dashboard - -When enabled, screenshots appear in the Accessibility tab of your Automation Dashboard: - -- **Image View Section**: Dedicated area for viewing captured screenshots -- **Thumbnail Gallery**: Quick overview of all captured screenshots from the test session -- **Issue Correlation**: Click on violations to see corresponding highlighted areas on screenshots - -## When to Enable - -**Enable screenshot capture when:** - -- Debugging complex accessibility issues that require visual context -- Creating comprehensive accessibility documentation -- Working with teams that need visual evidence for issue understanding -- Conducting accessibility audits or compliance reviews diff --git a/static/docs/accessibility-choosing-the-right-tool.md b/static/docs/accessibility-choosing-the-right-tool.md deleted file mode 100644 index 187cb3b1b..000000000 --- a/static/docs/accessibility-choosing-the-right-tool.md +++ /dev/null @@ -1,191 +0,0 @@ -# Choosing the Right Accessibility Tool - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI **Accessibility Testing** spans browser DevTools, web automation, scheduled and Web Scanner scans, native mobile (manual and Appium), reports, checklists, and optional AI/MCP workflows. Use this page as a **router**: match your situation to a starting doc, then follow the linked guides for setup and onboarding. - -If you are new to the product, read **[Getting Started with Accessibility Testing](/support/docs/accessibility-testing/)** first. It lists every major path in one place. - -## Quick decision table - -| I need to… | Start here | -|------------|----------------| -| Manually scan a website in the browser | [Accessibility DevTools (Overview)](/support/docs/accessibility-devtools/) | -| Run accessibility inside Selenium/Cypress/Playwright/etc. | [Accessibility Automation (Overview)](/support/docs/accessibility-automation/) → your [framework doc](#use-automation-when) | -| Schedule recurring scans inside Accessibility (sitemap/CSV/crawler) | [Test Scheduling - Sitemap (Overview)](/support/docs/accessibility-test-scheduling/) | -| Run scans from the **Web Scanner** product | [Getting Started with Web Scanner](/support/docs/web-scanner-getting-started/) → [Starting an Accessibility Scan with Web Scanner](/support/docs/web-scanner-accessibility-scan/) | -| Manually test a **native app** on a real device | [Accessibility App Scanner (Overview)](/support/docs/accessibility-app-scanner/) | -| Get accessibility from **Appium** code | [Native App Automation Appium (Overview)](/support/docs/accessibility-native-app-automation-test/) | -| Author mobile flows in **KaneAI** with scans | [KaneAI Mobile App Testing](/support/docs/kaneai-mobile-app-accessibility/) | -| Read dashboards, exports, and triage | [Navigating the Dashboard](/support/docs/accessibility-testing-navigating-dashboard/) | -| See what automation covers vs manual gaps | [Web](/support/docs/accessibility-web-what-we-cover/) · [iOS](/support/docs/accessibility-ios-what-we-cover/) · [Android](/support/docs/accessibility-android-what-we-cover/) checklists | -| Use **AI** or MCP for analysis | [Accessibility MCP Server](/support/docs/accessibility-mcp-server/) (accessibility-only) vs [TestMu AI MCP Server](/support/docs/testmu-mcp-server/) (platform-wide) | - -**Scheduling vs Web Scanner:** [Test Scheduling](/support/docs/accessibility-test-scheduling/) is the **Accessibility-native** recurring flow. [Web Scanner](/support/docs/web-scanner-getting-started/) is a **separate product** for URL-based scans. If you are unsure, compare [Accessibility DevTools (Overview)](/support/docs/accessibility-devtools/) (product boundary) and [Test Scheduling product boundary](/support/docs/accessibility-test-scheduling/#product-boundary). - -## Use DevTools when - -- you want **manual** or browser-assisted scanning on a live page -- you need quick investigation without writing test code -- you want scan types such as [quick scan](/support/docs/accessibility-testing-run-quick-scan/), [full page](/support/docs/accessibility-testing-full-page-scanner/), [partial page](/support/docs/accessibility-testing-partial-page-scanner/), [multi-page](/support/docs/accessibility-testing-multi-page-scanner/), [workflow](/support/docs/accessibility-testing-workflow-scanner/), or [keyboard](/support/docs/accessibility-keyboard-scan/) scan -- you need to tune DevTools behavior, see [DevTools Settings](/support/docs/accessibility-devtools-settings/) and [Update DevTools](/support/docs/accessibility-update-devtools-extension/) - -**Start:** [Accessibility DevTools (Overview)](/support/docs/accessibility-devtools/) → [Install Toolkit](/support/docs/accessibility-testing-install-devtools/) if you have not installed the extension yet. - -## Use Automation when - -- you want accessibility checks inside **automated** test runs (CI/CD, nightly builds) -- you need **regression** coverage tied to the same suite as functional tests -- you use a supported **web** stack on the grid - -**Hub doc:** [Automating Accessibility Testing with Selenium](/support/docs/accessibility-automation-test/) (Chrome/Edge, capabilities, `lambda-accessibility-scan` hook vs `accessibility.autoscan`). - -**Framework-specific entry points:** - -- [Selenium](/support/docs/accessibility-automation-test/) (Java-oriented hub; same capability model for other JVM languages where applicable) -- [Cypress v10](/support/docs/cypress-v10-accessibility-test/) · [Cypress v9 (Legacy)](/support/docs/cypress-v9-accessibility-test/) -- [Playwright](/support/docs/playwright-accessibility-test/) -- [TestNG](/support/docs/accessibility-testng-test/) · [JUnit 5](/support/docs/accessibility-junit5-test/) -- [NUnit (C#)](/support/docs/accessibility-nunit-csharp-test/) -- [Robot Framework](/support/docs/accessibility-robot-framework-test/) -- [Cucumber (Java)](/support/docs/accessibility-cucumber-java-test/) -- [HyperExecute integration: Selenium accessibility](/support/docs/selenium-hyperexecute-accessibility-tests/) - -**Configuration and pipeline:** - -- [Configure Accessibility Automation](/support/docs/accessibility-automation-settings/) (WCAG version, best practice, needs review; listed as “Automation Settings” in the sidebar in some releases) -- [CI/CD Integration Guide](/support/docs/accessibility-cicd-integration-guide/) - -## Use Test Scheduling when - -- you want **recurring** site scans without opening DevTools each time -- you need **sitemap**, **CSV**, or **crawler**-driven URL discovery -- you want the **Accessibility-native** scheduling surface, not the Web Scanner app - -**Start:** [Test Scheduling - Sitemap (Overview)](/support/docs/accessibility-test-scheduling/). - -**Common next steps:** - -- [Sitemap Extraction & URL Import](/support/docs/accessibility-sitemap-extraction-url-import/) -- [Create Scheduled Scan](/support/docs/accessibility-test-scheduling-scan/) · [Edit Scheduled Scan](/support/docs/accessibility-test-scheduling-edit/) -- [Crawler](/support/docs/accessibility-test-crawler/) - -**Advanced URL grouping:** If hash-based routes should split issues, enable [Fragment Identifier](/support/docs/accessibility-fragment-identifier/) in DevTools settings (web URL grouping context). - -## Use App Scanner (Manual) when - -- you are validating **native Android or iOS** screens interactively -- you want to inspect issues **screen by screen** on real devices without Appium - -**Start:** [Accessibility App Scanner (Overview)](/support/docs/accessibility-app-scanner/). - -**Contrast with automation:** [Native App Automation Appium (Overview)](/support/docs/accessibility-native-app-automation-test/) · [Appium TestNG](/support/docs/accessibility-appium-testng/) · [Appium WebdriverIO](/support/docs/accessibility-appium-webdriverio/). - -## Use Native App Automation when - -- you already run **Appium** for mobile functional tests -- you want **`lambda-accessibility-scan`** checkpoints and dashboard reports from those runs - -**Start:** [Native App Automation Appium (Overview)](/support/docs/accessibility-native-app-automation-test/). - -**Tags:** [Tag Support for Accessibility Scans](/support/docs/accessibility-tag-support/) when you need scan metadata across runs. - -## Use Web Scanner when - -- you are already inside the **Web Scanner** product -- you want **URL-based** accessibility scanning from that workflow (wizard, scheduling tab, etc.) -- you do **not** need DevTools or framework automation to start - -**Start:** [Getting Started with Web Scanner](/support/docs/web-scanner-getting-started/) → [Starting an Accessibility Scan with Web Scanner](/support/docs/web-scanner-accessibility-scan/). - -**Also useful:** [Adding URLs](/support/docs/web-scanner-adding-urls/) · [Scheduling Options](/support/docs/web-scanner-scheduling-options/) · [Advanced Features](/support/docs/web-scanner-advanced-features/). - -## Use KaneAI when - -- you are **authoring** a mobile test in KaneAI (not maintaining raw Appium projects) -- you want to insert accessibility scan steps inside the **authored** flow - -**Start:** [KaneAI Mobile App Testing](/support/docs/kaneai-mobile-app-accessibility/). - -## Use Reports when - -- you need the **dashboard**, issue breakdowns, exports, or ticketing handoff after any scan type - -**Core flow:** [Navigating the Dashboard](/support/docs/accessibility-testing-navigating-dashboard/) → [Issue Summary](/support/docs/accessibility-testing-dashboard-issue-summary/) → [All Issues](/support/docs/accessibility-testing-dashboard-all-issues/) (reports may also surface **[Accessibility Score](/support/docs/accessibility-web-score/)** when enabled). - -**Sharing and tracking:** - -- [Exporting & Sharing Reports](/support/docs/accessibility-exporting-sharing-reports/) -- [Integrations (Jira / Slack)](/support/docs/accessibility-report-integrations/) -- [Bug Report](/support/docs/accessibility-report-bug/) -- [Passed Test Cases](/support/docs/accessibility-passed-test-cases/) (fuller pass/fail story when enabled) - -## Use Features (product options) when - -- you need **hide/restore**, **AI issue detection**, **PDF** scans, **screenshots**, **tags**, **fragment identifiers**, **analytics widgets**, or deep **remediation** guidance alongside reports - -In the sidebar, **Features** is grouped as **Web**, **Mobile**, and **Common** (same order as below). - -### Web - -- [Hide and Restore Issues](/support/docs/accessibility-hide-restore-issues/) -- [AI Issue Detection Agent](/support/docs/accessibility-ai-issue-detection-agent/) -- [Capture Screenshot](/support/docs/accessibility-capture-screenshot/) -- [Accessibility Score](/support/docs/accessibility-web-score/) -- [Fragment Identifier](/support/docs/accessibility-fragment-identifier/) -- [PDF Accessibility Scanning](/support/docs/accessibility-pdf-accessibility-scanning/) - -### Mobile - -- [Tag Support](/support/docs/accessibility-tag-support/) - -### Common - -- [Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) -- [Analytics Widgets](/support/docs/analytics-modules-accessibility-widgets/) - -## Use Screen Reader testing when - -- you must validate behavior with **assistive technology** (not only automated rules) - -**Hub:** [Screen Reader Overview](/support/docs/screen-reader-on-accessibility/). - -**By platform:** - -- [NVDA on Windows](/support/docs/accessibility-nvda-windows/) -- [JAWS on Windows](/support/docs/accessibility-jaws-windows/) -- [VoiceOver on macOS](/support/docs/accessibility-voiceover-macos/) -- [TalkBack on Android](/support/docs/accessibility-talkback-android/) - -Pair screen reader sessions with [Keyboard Scan](/support/docs/accessibility-keyboard-scan/) for keyboard-only coverage on web. - -## Use Checklists and Rule References when - -- you need to know **which WCAG-aligned rules** automation covers **per platform** -- you want the **manual test checklist** (beyond automated rules) in one glance -- you need **rule-level** remediation text - -**Checklists:** [Web](/support/docs/accessibility-web-what-we-cover/) · [iOS](/support/docs/accessibility-ios-what-we-cover/) · [Android](/support/docs/accessibility-android-what-we-cover/). - -**Rule repositories:** [Web](/support/docs/accessibility-web-rule-repository/) · [Android](/support/docs/accessibility-android-rule-repository/) · [iOS](/support/docs/accessibility-ios-rule-repository/). - -**Compliance framing (not legal advice):** [Accessibility Compliance Guide](/support/docs/accessibility-compliance-guide/) · [VPAT and ACR evidence (customer-owned templates)](/support/docs/accessibility-vpat-report-generation/) (TestMu AI does **not** provide VPAT or ACR reports). - -## Use Accessibility MCP Server when - -- you want **AI-assisted** accessibility analysis through an **MCP-compatible** client, scoped to **Accessibility** workflows - -**Doc:** [Accessibility MCP Server](/support/docs/accessibility-mcp-server/). - -**Not the same as** installing the full multi-tool MCP stack: use [Introducing TestMu AI MCP Server](/support/docs/testmu-mcp-server/) for platform-wide MCP setup, then return here for accessibility-specific usage. - -## Related docs - -- [Getting Started with Accessibility Testing](/support/docs/accessibility-testing/) -- [Accessibility FAQs](/support/docs/accessibility-faq/) -- [Supported WCAG Versions & Browsers](/support/docs/accessibility-supported-wcag-browsers/) -- [Starting an Accessibility Scan with Web Scanner](/support/docs/web-scanner-accessibility-scan/) -- [Mobile App Accessibility Testing (KaneAI)](/support/docs/kaneai-mobile-app-accessibility/) -- [Screen Reader Overview](/support/docs/screen-reader-on-accessibility/) -- [Web checklist](/support/docs/accessibility-web-what-we-cover/) · [iOS checklist](/support/docs/accessibility-ios-what-we-cover/) · [Android checklist](/support/docs/accessibility-android-what-we-cover/) -- [Accessibility MCP Server](/support/docs/accessibility-mcp-server/) · [TestMu AI MCP Server](/support/docs/testmu-mcp-server/) diff --git a/static/docs/accessibility-cicd-integration-guide.md b/static/docs/accessibility-cicd-integration-guide.md deleted file mode 100644 index ee817916a..000000000 --- a/static/docs/accessibility-cicd-integration-guide.md +++ /dev/null @@ -1,66 +0,0 @@ -# CI/CD Integration Guide - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Use this guide when you want **Accessibility Automation** (Selenium capabilities + hooks or auto-scan) to run on **every PR, nightly, or release branch**, not only from a laptop. - -## Typical CI/CD use cases - -- run accessibility checks on pull requests or nightly jobs -- compare Accessibility results across builds -- keep automated accessibility validation close to release workflows - -## High-level flow - -1. Configure the test framework with Accessibility capabilities. -2. Run the tests from CI/CD. -3. Review the resulting report in the Accessibility dashboard. -4. Export or escalate findings if needed. - -## Onboarding checklist - -1. **Store secrets** as `LT_USERNAME` and `LT_ACCESS_KEY` (or your platform’s secret manager) and inject them into the job, never commit keys to the repo ([credentials guide](/support/docs/using-environment-variables-for-authentication-credentials/)). -2. **Reuse the same command** you already use locally (`mvn test`, `dotnet test`, `npm test`, etc.) so CI exercises the identical driver bootstrap that sets `accessibility: true`. -3. **Start with a smoke slice** (one suite or `@Tag`) so pipeline time stays predictable; expand coverage after reports look correct. -4. **Open the Automation dashboard** after the first green CI run and confirm the **Accessibility** tab populates; bookmark the filtered view for your team. -5. **Optional gating:** treat critical/serious new issues like other quality gates, document who can override flakes. - -## Example: GitHub Actions (pattern) - -```yaml -name: accessibility-smoke -on: [pull_request] -jobs: -a11y: -runs-on: ubuntu-latest -steps: -- uses: actions/checkout@v4 -- uses: actions/setup-java@v4 -with: -distribution: temurin -java-version: "17" -- name: Run Selenium + Accessibility smoke -env: -LT_USERNAME: ${{ secrets.LT_USERNAME }} -LT_ACCESS_KEY: ${{ secrets.LT_ACCESS_KEY }} -run: mvn -q -Dgroups=a11y test -``` - -Adapt `run` to your build tool; the important part is that the invoked tests set Accessibility capabilities per [Selenium](/support/docs/accessibility-automation-test/) and [settings](/support/docs/accessibility-automation-settings/). - -## Example: Jenkins (pattern) - -- Bind **Credentials** to environment variables `LT_USERNAME` / `LT_ACCESS_KEY`. -- Use a **freestyle** or **pipeline** step that checks out the repo and runs the same Maven/Gradle/npm command as above. -- Archive HTML or logs if you need offline evidence; primary reports still live on the TestMu AI dashboard. - -## Where to review results - -- **Automation** build → **Accessibility** tab for the session ([Selenium flow](/support/docs/accessibility-automation-test/)). -- For **HyperExecute**, see [HyperExecute integration: Selenium accessibility automation](/support/docs/selenium-hyperexecute-accessibility-tests/). - -## Related docs - -- [Accessibility Automation (Overview)](/support/docs/accessibility-automation/) -- [Configure Accessibility Automation](/support/docs/accessibility-automation-settings/) -- [HyperExecute integration: Selenium accessibility automation](/support/docs/selenium-hyperexecute-accessibility-tests/) diff --git a/static/docs/accessibility-compliance-guide.md b/static/docs/accessibility-compliance-guide.md deleted file mode 100644 index 1db11cccf..000000000 --- a/static/docs/accessibility-compliance-guide.md +++ /dev/null @@ -1,31 +0,0 @@ -# Accessibility Compliance Guide (ADA / WCAG / EAA / 508) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This page explains how common accessibility compliance frameworks relate to Accessibility Testing outputs. - -Teams working under ADA, WCAG, Section 508, or the EAA still need a practical bridge between legal frameworks and day-to-day testing signals. This guide explains how Accessibility Testing reports, manual verification, and rule-level remediation guidance fit into those broader compliance conversations without standing in for legal certification. Use it when stakeholders ask how the product supports their framework language at a high level. - -## Common frameworks - -- **WCAG** provides technical accessibility guidance and success criteria. -- **ADA** often relies on WCAG-aligned expectations in practice for digital accessibility. -- **Section 508** applies accessibility requirements in regulated contexts. -- **EAA** introduces accessibility obligations for many digital products and services in Europe. - -## Important note - -Accessibility Testing supports compliance workflows, but automated results alone should not be treated as final legal certification. - -## Onboarding stakeholders (what to say in week one) - -1. **WCAG** is the technical baseline, map automated findings to success criteria using **[Web](/support/docs/accessibility-web-what-we-cover/)**, **[iOS](/support/docs/accessibility-ios-what-we-cover/)**, and **[Android](/support/docs/accessibility-android-what-we-cover/)** checklists plus rule repositories. -2. **ADA / EAA / 508 conversations** still require **manual evidence** (keyboard-only paths, screen readers, policy docs). Use Accessibility outputs as **inputs**, not the final legal position. -3. Establish a **definition of done**: e.g., “no open critical/serious automated issues on core journeys + documented manual matrix.” -4. For procurement or VPAT-style evidence work, pair this guide with **[VPAT and ACR evidence (customer-owned templates)](/support/docs/accessibility-vpat-report-generation/)** (TestMu AI does **not** provide VPAT or ACR reports) and **[Supported WCAG Versions & Browsers](/support/docs/accessibility-supported-wcag-browsers/)**. - -## Related docs - -- [Supported WCAG Versions & Browsers](/support/docs/accessibility-supported-wcag-browsers/) -- [VPAT and ACR evidence (customer-owned templates)](/support/docs/accessibility-vpat-report-generation/) -- [Web Accessibility Coverage](/support/docs/accessibility-web-what-we-cover/) diff --git a/static/docs/accessibility-cucumber-java-test.md b/static/docs/accessibility-cucumber-java-test.md deleted file mode 100644 index a28840de2..000000000 --- a/static/docs/accessibility-cucumber-java-test.md +++ /dev/null @@ -1,62 +0,0 @@ -# Cucumber (Java) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Cucumber scenarios still execute **ordinary Selenium** code under the hood. Enable Accessibility once on the **shared WebDriver** (usually in a **@Before** hook), then call **`lambda-accessibility-scan`** from step definitions or a small helper when a screen is stable. Behavior matches [Selenium Accessibility Automation](/support/docs/accessibility-automation-test/). - -> **Browsers:** Chrome or Edge, supported versions only. - -## Prerequisites - -- Cucumber JVM + a DI or PicoContainer (or Spring) setup that exposes a singleton `WebDriver` -- TestMu AI grid URL and credentials -- Accessibility entitlement - -## Onboarding path - -### 1. Create the driver with Accessibility in `@Before` - -```java -@Before(order = 0) -public void setUp() { -MutableCapabilities caps = new MutableCapabilities(); -caps.setCapability("browserName", "chrome"); -caps.setCapability("accessibility", true); -// caps.setCapability("accessibility.autoscan", true); // optional -driver = new RemoteWebDriver(new URL(HUB_URL), caps); -// inject driver into step defs -} -``` - -### 2. Scan after key navigations - -In a step that represents “user is on dashboard” (or after `Given/When` that loads a URL): - -```java -driver.executeScript("lambda-accessibility-scan"); -``` - -Avoid calling the hook on every tiny interaction; align it with **logical pages** or states. - -### 3. Optional: auto-scan for exploratory flows - -If scenarios bounce across many URLs and you want full coverage without per-step hooks, enable `accessibility.autoscan` in `@Before` instead. - -### 4. Run Cucumber and open reports - -```bash -mvn test -Dcucumber.filter.tags="@smoke" -``` - -Dashboard → session → **Accessibility** tab. - -### 5. CI alignment - -Tag scenarios that should run Accessibility (`@a11y`) so pipelines stay fast; keep capability setup in the shared hook so tagged and untagged runs behave predictably. - -## Related docs - -- [Selenium](/support/docs/accessibility-automation-test/) -- [Configure Accessibility Automation](/support/docs/accessibility-automation-settings/) -- [JUnit 5](/support/docs/accessibility-junit5-test/) (often the JUnit Platform runner behind Cucumber) -- [CI/CD Integration Guide](/support/docs/accessibility-cicd-integration-guide/) diff --git a/static/docs/accessibility-devtools-settings.md b/static/docs/accessibility-devtools-settings.md deleted file mode 100644 index 9cd94e351..000000000 --- a/static/docs/accessibility-devtools-settings.md +++ /dev/null @@ -1,40 +0,0 @@ -# Configure Accessibility DevTools - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This page covers the settings available in the Accessibility DevTools browser extension. These settings apply when you run scans manually through the DevTools panel. - -If you are configuring settings for automated test execution (Selenium, Cypress, Playwright, etc.), see [Configure Accessibility Automation](/support/docs/accessibility-automation-settings/) instead. - -## Access DevTools settings - -Click on your **profile icon** and then **Settings**. - -## WCAG version - -Select which **WCAG version** your DevTools scans should check against. Options include WCAG 2.0, WCAG 2.1 A, WCAG 2.1 AA, WCAG 2.1 AAA, WCAG 2.2 A, and WCAG 2.2 AA. The recommended version is **WCAG 2.1 AA**. - -## Review Issues - -Some issues require manual review before they can be confirmed as true violations. When this setting is enabled, the scan results include a **Needs Review** category. - -To review an issue: - -1. Click on the issue to expand it. -2. Select the element you want to evaluate. -3. Click **Yes** to confirm it as an issue, or leave it unconfirmed. -4. Use **Undo** to reverse a confirmation. - -## Best Practices - -Enable this setting to include issues that go beyond WCAG requirements but improve overall user experience. Best practice issues are reported separately from WCAG violations. - -## Autosave Reports - -This option is enabled by default. When active, scan results are automatically saved to the Accessibility dashboard. Toggle it off if you want to review results without persisting them. - -## Related docs - -- [Configure Accessibility Automation](/support/docs/accessibility-automation-settings/) -- [Run Your Quick Scan](/support/docs/accessibility-testing-run-quick-scan/) -- [Navigating the Dashboard](/support/docs/accessibility-testing-navigating-dashboard/) diff --git a/static/docs/accessibility-devtools.md b/static/docs/accessibility-devtools.md deleted file mode 100644 index efc87dcc5..000000000 --- a/static/docs/accessibility-devtools.md +++ /dev/null @@ -1,41 +0,0 @@ -# Accessibility DevTools (Overview) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Accessibility DevTools is the browser-based workflow for manual and assisted Accessibility testing on websites and web apps. - -Use this overview when you need to decide whether DevTools is the right surface, what the browser-based toolkit includes, and how it differs from Automation, Web Scanner, and Test Scheduling. It frames manual and assisted browser sessions in the context of the broader Accessibility product. - -## When to use this - -Use DevTools when you want to inspect a page directly in the browser, validate issues quickly, or run scan types that are tied to an interactive browsing session. - -## What you can do in DevTools - -- install the toolkit -- run quick, full-page, partial-page, multi-page, workflow, and keyboard scans -- configure WCAG, best-practice, and needs-review settings -- review issues in an interactive session before moving to reports or remediation - -## Product boundary - -DevTools is not the same as: - -- **Accessibility Automation**, which runs through test frameworks -- **Test Scheduling** (Accessibility), which runs recurring web scans inside the Accessibility product -- **Web Scanner**, the separate Web Scanner product for URL lists and scans, different navigation tree from Accessibility Test Scheduling - -## Typical workflow - -1. Install the toolkit. -2. Open the target page. -3. Select the right scan type. -4. Review the findings. -5. Move into the dashboard or remediation flow if needed. - -## Related docs - -- [Install TestMu AI Accessibility Toolkit](/support/docs/accessibility-testing-install-devtools/) -- [Run Your Quick Scan](/support/docs/accessibility-testing-run-quick-scan/) -- [Keyboard Scan](/support/docs/accessibility-keyboard-scan/) -- [Configure Accessibility DevTools](/support/docs/accessibility-devtools-settings/) diff --git a/static/docs/accessibility-exporting-sharing-reports.md b/static/docs/accessibility-exporting-sharing-reports.md deleted file mode 100644 index a0f5d6b09..000000000 --- a/static/docs/accessibility-exporting-sharing-reports.md +++ /dev/null @@ -1,32 +0,0 @@ -# Exporting & Sharing Reports - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Use this page to understand how Accessibility results can be shared across stakeholders and exported for downstream workflows. - -The following sections outline common reasons teams export Accessibility results and which downstream workflows those exports support. They also explain how sharing reflects the active state of the report you are viewing, including filters and visibility choices. - -## Common use cases - -- sharing findings with engineering or QA -- sending audit evidence to accessibility stakeholders -- reviewing results outside the dashboard -- exporting the current report state after filters or hide/restore decisions - -## Important behavior - -Report exports reflect the active issue state for the current report view. In scheduled aggregated reporting, historical consistency can still limit how past aggregate views are reflected. - -## How to export (onboarding) - -1. Open the report from **[Navigating the Dashboard](/support/docs/accessibility-testing-navigating-dashboard/)**. -2. Apply any **filters** or **[hide/restore](/support/docs/accessibility-hide-restore-issues/)** decisions first. Exports represent the **current** visible state. -3. Use the report’s **export** or **share** action (label varies by surface) and pick the format your stakeholder needs (spreadsheet, PDF, or packaged evidence where offered). -4. Store exports with **build metadata** (date, commit SHA, scan type) so audit trails stay traceable. -5. For recurring programs, pair exports with **[Integrations](/support/docs/accessibility-report-integrations/)** so issues also land in Jira or Slack automatically. - -## Related docs - -- [Hide and Restore Issues](/support/docs/accessibility-hide-restore-issues/) -- [Bug Report](/support/docs/accessibility-report-bug/) -- [Integrations (JIRA / Slack / GitHub)](/support/docs/accessibility-report-integrations/) diff --git a/static/docs/accessibility-faq.md b/static/docs/accessibility-faq.md deleted file mode 100644 index 3628dc172..000000000 --- a/static/docs/accessibility-faq.md +++ /dev/null @@ -1,81 +0,0 @@ -# Accessibility FAQs - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## What's the latest WCAG version supported? -We adhere to **WCAG 2.2 AA**, the most recent version of the Web Content Accessibility Guidelines. In addition to this, we support the following versions: WCAG 2.0 A, WCAG 2.0 AA, WCAG 2.1 A, WCAG 2.1 AA, WCAG 2.1 AAA, and WCAG 2.2 A. - -## Do you have screen reader support? -Yes, we provide screen reader support: - -- **Windows:** JAWS and NVDA. -- **macOS:** VoiceOver. -- **Android:** TalkBack. - -See [Screen Reader Overview](/support/docs/screen-reader-on-accessibility/) for setup and usage details. - -## Which browsers are supported? -Our accessibility testing tools are optimized for **Chromium-based** browsers, **version 90 and above**. This includes **Google Chrome, Microsoft Edge, and other Chromium-based browsers**. Using the latest version ensures the best compatibility and performance with our tools. - -## Do I need to add explicit wait time to scan for accessibility? -No, you don't need to add explicit wait times. Our advanced tools are designed with built-in intelligence to detect when DOM elements are loaded automatically. They then immediately initiate accessibility scans for these elements. This feature ensures a seamless and efficient testing process without manual intervention for timing. - -## How can I enable accessibility for my account/organization? -Accessibility testing is a premium feature that requires activation. To enable it, contact our sales team through our [website](https://www.testmuai.com/accessibility-automation) or your account manager. They will provide you with pricing details and plan options. Once you've chosen a plan, our team will activate the feature for your account. - -## How many accessibility automation scans does one organization get? -Although there is limitations on the number of parallel tests or threads you can run but there are no restrictions on the number of pages you can scan for thorough accessibility testing. - -## Why is it showing a sub-version (e.g. 1.41.3) even though I have selected WCAG 2.1 AA? -WCAG 2.1 AA is the overall accessibility standard you've selected for testing. The sub-versions (like 1.4.1 or 1.1.1) are specific guidelines within that standard. These sub-versions represent individual success criteria that fall under WCAG 2.1 AA. While you test against the broader WCAG 2.1 AA standard, the results show compliance with these more specific guidelines to provide detailed insights into your site's accessibility. - -## Is it possible to scan private or non-public websites? -Yes, our system is fully capable of scanning private or non-public websites. - -- **DevTools:** DevTools runs on your local system, it can directly scan your locally hosted projects. -- **Accessibility Automation:** The same way you use a tunnel to access any websites hosted locally, we can do the same for Accessibility Automation. - -## On what basis are the different statuses (e.g., critical, minor) assigned? -Our system uses a proprietary algorithm to assign severity statuses to accessibility issues. These statuses (such as critical, major, minor) are based on several factors: - -- Impact on user experience for people with disabilities -- Frequency of the issue across the site -- Complexity of the fix required -- Legal compliance implications -This nuanced approach helps you prioritize your accessibility improvements effectively. - -## What frameworks are supported for web automation? -Accessibility Automation supports Selenium, Cypress (v9 and v10+), Playwright, WebdriverIO, TestNG, JUnit 5, NUnit (C#), Robot Framework, and Cucumber (Java). All of these can also run on HyperExecute. See [Accessibility Automation](/support/docs/accessibility-automation/) for the full list. - -## Can I share the dashboard link with anyone? -Yes, you can share dashboard links with anyone within your organization. However, for security reasons, access is limited to members of your organization to protect potentially sensitive information about your web applications. - -## What is the accessibility score given by TestMu AI and how should it be interpreted? -The TestMu AI accessibility score is a proprietary metric designed to give you a quick overview of your site's accessibility. It's calculated based on: -- Number and severity of issues detected -- Coverage of WCAG success criteria -- Overall accessibility of key user flows -> A higher score indicates better accessibility, but it's important to note that this score should be used as a general guide rather than a definitive measure of compliance. Always review the detailed reports alongside the score for a comprehensive understanding of your site's accessibility. - -## How many parallel automated accessibility tests can I run? -The number of parallel automated accessibility tests you can run depends on the number of parallels included in your automation plan. For example, if your plan includes 2 parallels, you can run 2 concurrent tests. - -## Can I run tests on mobile apps? -Yes. Mobile app accessibility testing is supported through: - -- **App Scanner (Manual):** Interactive screen-by-screen scanning on real devices. See [Accessibility App Scanner](/support/docs/accessibility-app-scanner/). -- **Native App Automation:** Automated accessibility checks via Appium during test execution. See [Native App Automation](/support/docs/accessibility-native-app-automation-test/). -- **KaneAI:** Insert accessibility scan instructions into KaneAI-authored mobile test flows. See [KaneAI Mobile Accessibility](/support/docs/kaneai-mobile-app-accessibility/). - -Mobile web browser accessibility testing through automation is not currently supported. - -## Do you support accessibility testing for PDFs? -Yes. PDF Accessibility Scanning is available as part of the Accessibility product. See [PDF Accessibility Scanning](/support/docs/accessibility-pdf-accessibility-scanning/) for details on supported capabilities and how to use this feature. - -## How many pages can be scanned for accessibility in a workflow scan? -There is currently no limitation on the number of pages that can be scanned in a workflow scan. This allows you to: - -- Test entire websites or large web applications comprehensively -- Create custom workflows that cover all critical user journeys -- Conduct thorough accessibility audits without artificial constraints -However, for very large sites, we recommend breaking scans into logical segments for more manageable analysis and faster results. diff --git a/static/docs/accessibility-fragment-identifier.md b/static/docs/accessibility-fragment-identifier.md deleted file mode 100644 index b41aad306..000000000 --- a/static/docs/accessibility-fragment-identifier.md +++ /dev/null @@ -1,54 +0,0 @@ -# Fragment Identifier for Accessibility Testing - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The **Fragment Identifier** option lets you treat URLs that differ only by the fragment (the part after `#`) as **separate entities** for accessibility issue tracking and management. That gives you finer-grained audits when anchors, in-page sections, or hash-based routes represent meaningfully different UI states. - -This is an **advanced** setting. It applies where issues are grouped by **web page URL** (for example, Accessibility DevTools and related web reporting). Native mobile app sessions are not keyed on HTTP URL fragments in the same way. - -**Default:** The feature is **disabled**. When it is off, URLs that share the same base path but use different fragments are treated as **one** URL for issue tracking. - -## Enabling Fragment Identifier - -1. Open **Accessibility DevTools** → **Settings**. -2. Open the **Other Settings** section. -3. Find **Fragment Identifier**. -4. Turn the toggle **on**. - -## How fragment identifiers work - -The fragment is the portion of a URL after the `#` symbol. It often points to a section, anchor, or client-side route state on the same document. - -### When the feature is disabled (default) - -You scan or track: - -- `https://example.com/page#section1` -- `https://example.com/page#section2` -- `https://example.com/page#section3` - -**Result:** Issues roll up under a single target, for example `https://example.com/page`. - -### When the feature is enabled - -The same three URLs are tracked as **distinct** targets: - -- `https://example.com/page#section1` -- `https://example.com/page#section2` -- `https://example.com/page#section3` - -**Result:** Issues are categorized and reported **per full URL**, including the fragment. That supports clearer ownership and trends when hash fragments reflect real differences in content or flow. - -## Benefits - -- **Granular issue tracking**: See and triage issues for specific sections or hash states instead of one blended bucket. -- **Better SPA support**: Hash-based routing and deep links are easier to reason about in the dashboard and reports. -- **Clearer workflow**: Assign and manage work per fragment when that matches how your app is structured. -- **Richer reporting**: Reports can reflect section- or state-specific findings where the base URL alone would hide variation. - -## Related docs - -- [Accessibility DevTools (Overview)](/support/docs/accessibility-devtools/) -- [Test Scheduling - Sitemap (Overview)](/support/docs/accessibility-test-scheduling/) (when hash-based URLs affect scheduled or aggregated reports) -- [Navigating the Dashboard](/support/docs/accessibility-testing-navigating-dashboard/) -- [Hide and Restore Issues](/support/docs/accessibility-hide-restore-issues/) diff --git a/static/docs/accessibility-getting-started-quick-setup.md b/static/docs/accessibility-getting-started-quick-setup.md deleted file mode 100644 index 63f3683a9..000000000 --- a/static/docs/accessibility-getting-started-quick-setup.md +++ /dev/null @@ -1,3 +0,0 @@ -# Getting Started (Quick Setup) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). diff --git a/static/docs/accessibility-hide-restore-issues.md b/static/docs/accessibility-hide-restore-issues.md deleted file mode 100644 index afff8c3fa..000000000 --- a/static/docs/accessibility-hide-restore-issues.md +++ /dev/null @@ -1,202 +0,0 @@ -# Hide and Restore Issues - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Use **Hide and Restore Issues** to manage known, accepted, or intentionally excluded findings without deleting them from the report hierarchy. The feature lets you hide and later restore issues at multiple hierarchy levels, capture a reason during the action, and, for supported scheduled web scan flows, apply the same preference to future scans as well. - -The sections that follow describe how hide and restore work for supported **web** scans: supported hierarchy levels (URL through element), capture of hide reasons, optional **Hide for future scans**, the **Hidden Issues** tab, how restore follows stored preferences, and why **Save** is required. You will also see how issue counts, charts, graphs, and exports reflect active issues after you hide or restore findings. - -This feature is currently supported only for **web scans**. It is **not supported for app scans**. - -## Supported hierarchy levels - -You can hide and restore issues at all supported levels in the report hierarchy: - -- **URL** -- **Rule** -- **Element group** -- **Element** - -Each hierarchy level has its own action entry point and confirmation modal so users can review the exact scope of the hide or restore action before applying it. - -## Understand the report hierarchy - -The report can be explored from top to bottom across the issue hierarchy. This is the same structure used by the hide and restore feature. - -When you apply a hide or restore action, the effect depends on the level you selected in this hierarchy. - -## How hiding works - -### Step 1: Open the issue hierarchy and locate the target - -Open the report and navigate to the issue location you want to manage. You can start from a URL, then drill down into its rules, element groups, and individual elements. - -### Step 2: Click the hide action - -Use the hide icon from the relevant level in the hierarchy. - -### Step 3: Choose the level you want to hide - -You can hide from any supported level depending on how broad or narrow the action should be. - -**Hide at URL level** - -Use this when all findings under a specific URL should be hidden together. - -**Hide at rule level** - -Use this when the hide action should apply to one rule and the issues grouped under that rule. - -**Hide at element group level** - -Use this when the issue set should be hidden only for a specific group of similar elements. - -**Hide at element level** - -Use this when only a single element-level issue should be hidden. - -### Step 4: Select a reason in the hide modal - -When you hide an issue or issue set, the modal lets you choose a reason for hiding it. - -The selected reason is shown later inside the issue as a **reason tag**. - -If a reason is chosen at a higher hierarchy level, that reason is mapped to all associated issues under that hierarchy. - -### Step 5: Choose whether it should apply to future scans - -For **Web Scanner Accessibility scans** and **Accessibility scheduled scans**, the hide modal also lets you choose whether the issue should be hidden for **future scans**. - -Use this option when: - -- the issue is already known and accepted across scheduled runs -- the same hide preference should continue in future executions -- the team wants scheduled reports to stay aligned with an approved hide decision - -The **Hide for future scans** option is available only for supported **Web Scanner Accessibility scans** and **Accessibility scheduled scans**. It is not available for app scans. - -### Step 6: Confirm the hide action - -After you confirm the modal, the selected issue set is removed from the active report state and is moved into the **Hidden Issues** flow for that report. - -## Hidden Issues tab - -After issues are hidden, you can access them from the **Hidden Issues** tab. - -Use this tab to: - -- review which issues are currently hidden -- inspect the stored reason tag -- understand whether the issue is hidden only for the current scan or also for future scans -- restore issues from the same hierarchy levels used during hiding - -## How restore works - -### Step 1: Open the restore action - -Inside the **Hidden Issues** tab, use the restore icon for the hierarchy you want to restore. - -### Step 2: Review the restore modal - -The restore modal reflects the stored hide preference and keeps restore behavior consistent with the original hide action. - -**Restore for current scan only** - -If none of the issues in the selected set were hidden for future scans, the restore action applies to the current scan only. - -**Restore for future scans too** - -If the selected hierarchy, or any child inside that hierarchy, was hidden for future scans, the restore modal preserves that future-scoped behavior. - -### Restore behavior across hierarchy levels - -Restore works at the same supported levels used for hiding: - -- **URL** -- **Rule** -- **Element group** -- **Element** - -The restore action uses the stored **hide preference** to stay consistent with the way the issue was hidden. - -That means: - -- if the selected hierarchy was hidden for **future scans**, the restore action also restores it with that same future-scoped behavior -- if any child item inside the selected hierarchy is also marked **for future scans**, restoring the higher hierarchy respects that future preference as well -- if no issue in that set is marked for future scans, the restore action applies only to the **current scan** - -This keeps hide and restore behavior aligned across URL, rule, element group, and element levels and gives clearer feedback when future-scoped preferences exist anywhere inside the selected hierarchy. - -## Save your changes - -Hide and restore actions change the active report state immediately in the current session, but you still need to click **Save** to persist the preference. - -When the save request is processing, the interface shows the in-progress state. - -### Before you click Save - -Before saving: - -- the report can still reflect the temporary hide or restore state for the current session -- exports can still use that temporary session state - -### After you click Save - -After saving: - -- the hide or restore preference is committed -- the report state remains aligned with the saved hide settings -- future-scan behavior, where supported, follows the saved preference - -## How stats, graphs, and exports are updated - -After issues are hidden or restored, the report updates based on **active issues only** across: - -- visible issue counts -- stats in the report -- charts and graphs -- export data - -This helps teams focus on the currently active issue set rather than the full unfiltered list. - -### Exception for aggregated scheduled reports - -For **aggregated reports** in scheduled tests, updates are reflected only from the **consequent runs**. This keeps historical scheduled reporting audit-consistent while still applying the hide or restore preference going forward. - -## Best practices - -- Hide issues only when the team has a clear and documented reason. -- Use the reason field consistently so reviewers understand why the issue is hidden. -- Hide at the smallest useful hierarchy level unless a broader action is genuinely intended. -- Use **Hide for future scans** only when the same treatment should continue across scheduled web scans. -- Always click **Save** after review so other users see the intended state. - -## Troubleshooting - -### I do not see the future-scan option - -The **Hide for future scans** option is available only for supported **Web Scanner Accessibility scans** and **Accessibility scheduled scans**. - -### My report changed even before I clicked Save - -The report can still reflect the temporary state in the current session before the preference is saved. - -### The reason appears on more issues than I selected - -If you hide from a higher hierarchy such as URL, rule, or element group, the selected reason is mapped to the associated issues under that hierarchy. - -### Restore is affecting future scans too - -Restore follows the stored hide preference. If the selected hierarchy, or any child item inside it, was hidden for future scans, the restore flow keeps that future-scoped behavior consistent. - -### Why do my aggregated scheduled reports not fully change right away - -Aggregated scheduled reporting preserves historical audit consistency. Updates are reflected from consequent runs rather than rewriting older aggregated history. - -## Related docs - -- [Navigating the Dashboard](/support/docs/accessibility-testing-navigating-dashboard/) -- [All Issues](/support/docs/accessibility-testing-dashboard-all-issues/) -- [Exporting & Sharing Reports](/support/docs/accessibility-exporting-sharing-reports/) -- [Starting an Accessibility Scan with Web Scanner](/support/docs/web-scanner-accessibility-scan/) -- [Test Scheduling - Sitemap (Overview)](/support/docs/accessibility-test-scheduling/) diff --git a/static/docs/accessibility-ios-rule-accessibility-label-not-punctuated.md b/static/docs/accessibility-ios-rule-accessibility-label-not-punctuated.md deleted file mode 100644 index 7ea924687..000000000 --- a/static/docs/accessibility-ios-rule-accessibility-label-not-punctuated.md +++ /dev/null @@ -1,35 +0,0 @@ -# Accessibility Label Not Punctuated - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Accessibility labels on iOS elements should end with appropriate punctuation so VoiceOver produces natural pauses and intonation when reading them. - -**WCAG Reference** -**Maps to:** WCAG 3.3.2 Labels or Instructions | **Applies to:** WCAG 2.0, WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.0 | **Level:** A | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/labels-or-instructions.html) - -## What this rule checks - -The scanner flags accessibility labels that do not end with proper punctuation (period, comma, question mark, or exclamation mark), which affects VoiceOver's speech cadence and clarity. - -## Why it matters - -VoiceOver uses punctuation to determine pacing and intonation. A label that ends abruptly without punctuation causes VoiceOver to run the label into the next announcement without a pause, making it harder for users to parse individual elements. - -## Common failure patterns - -- labels like "Add to cart" with no trailing period -- descriptive labels on images that read as sentences but lack ending punctuation -- multi-word labels that describe status or state without punctuation - -## Remediation guidance - -- end accessibility labels with a period when they form a complete phrase or sentence -- use punctuation consistent with the label's tone: period for statements, question mark for questions -- keep labels concise: a label should describe the element, not be a paragraph -- test with VoiceOver to confirm the speech rhythm sounds natural between consecutive elements - -## Related docs - -- [iOS Rule Repository](/support/docs/accessibility-ios-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-ios-rule-accessibility-role-definition.md b/static/docs/accessibility-ios-rule-accessibility-role-definition.md deleted file mode 100644 index 63a010d11..000000000 --- a/static/docs/accessibility-ios-rule-accessibility-role-definition.md +++ /dev/null @@ -1,37 +0,0 @@ -# Accessibility Role Definition - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Interactive iOS elements must expose the correct accessibility traits so VoiceOver announces their role (button, link, switch, header) accurately. - -**WCAG Reference** -**Maps to:** WCAG 4.1.2 Name, Role, Value | **Applies to:** WCAG 2.0, WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.0 | **Level:** A | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/name-role-value.html) - -## What this rule checks - -The scanner verifies that interactive elements have appropriate `accessibilityTraits` set, such as `.button`, `.link`, `.header`, `.adjustable`, or `.selected`, matching their actual behavior. - -## Why it matters - -VoiceOver uses traits to tell users what kind of element they are interacting with and how to use it. Without the correct trait, users may not know they can tap, swipe, or adjust a control. - -## Common failure patterns - -- custom interactive views with no `accessibilityTraits` set (defaults to `.none`) -- links styled as buttons but missing `.link` trait -- section headers in lists without `.header` trait -- slider-like controls missing `.adjustable` trait - -## Remediation guidance - -- set `accessibilityTraits` on every interactive element to match its behavior -- use `.button` for tappable controls, `.link` for navigation links, `.header` for section headings -- for adjustable controls (sliders, steppers), set `.adjustable` and implement `accessibilityIncrement()` and `accessibilityDecrement()` -- in SwiftUI, use `.accessibilityAddTraits()` to add the appropriate traits -- test with VoiceOver and confirm the announced trait matches the element's behavior - -## Related docs - -- [iOS Rule Repository](/support/docs/accessibility-ios-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-ios-rule-assistive-technology-access.md b/static/docs/accessibility-ios-rule-assistive-technology-access.md deleted file mode 100644 index f8b7b756b..000000000 --- a/static/docs/accessibility-ios-rule-assistive-technology-access.md +++ /dev/null @@ -1,36 +0,0 @@ -# Assistive Technology Access - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Interactive iOS elements must be accessible to VoiceOver and other assistive technologies, not just direct touch interaction. - -**WCAG Reference** -**Maps to:** WCAG 4.1.2 Name, Role, Value | **Applies to:** WCAG 2.0, WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.0 | **Level:** A | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/name-role-value.html) - -## What this rule checks - -The scanner flags interactive elements that have `isAccessibilityElement` set to `false` or are hidden from the accessibility tree while still being visually present and functional. - -## Why it matters - -When an interactive element is excluded from the accessibility tree, VoiceOver users cannot perceive or activate it. This effectively locks those users out of features that are available to sighted users. - -## Common failure patterns - -- custom views with `isAccessibilityElement = false` that contain interactive content -- `UIView` subclasses that implement touch handling but do not expose accessibility properties -- elements hidden behind other views that are accessible via direct touch but not VoiceOver navigation -- container views that suppress child accessibility without evaluating each child - -## Remediation guidance - -- set `isAccessibilityElement = true` on all meaningful interactive elements -- for custom container views, either make children individually accessible or provide a container-level label that summarizes the content -- implement `UIAccessibility` protocols on custom views (label, traits, value, hint) -- test with VoiceOver to confirm every interactive element is reachable by swiping - -## Related docs - -- [iOS Rule Repository](/support/docs/accessibility-ios-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-ios-rule-button-element-capitalisation-check.md b/static/docs/accessibility-ios-rule-button-element-capitalisation-check.md deleted file mode 100644 index 79cb73498..000000000 --- a/static/docs/accessibility-ios-rule-button-element-capitalisation-check.md +++ /dev/null @@ -1,36 +0,0 @@ -# Button Element Capitalisation Check - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Button labels in iOS apps should follow standard capitalisation conventions to ensure VoiceOver reads them naturally and consistently. - -**WCAG Reference** -**Maps to:** WCAG 3.1.6 Pronunciation | **Applies to:** WCAG 2.0, WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.0 | **Level:** AAA | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/pronunciation.html) - -## What this rule checks - -The scanner flags button labels that use all-uppercase text (e.g., "SUBMIT"), all-lowercase text (e.g., "submit"), or inconsistent capitalisation that affects VoiceOver pronunciation. - -## Why it matters - -VoiceOver may spell out all-uppercase labels letter by letter (e.g., "S-U-B-M-I-T" instead of "Submit") depending on the speech engine. Inconsistent capitalisation creates an unpredictable listening experience. - -## Common failure patterns - -- button titles set to "DELETE" or "CANCEL" in all caps -- labels that use lowercase for stylistic reasons ("learn more") -- inconsistent capitalisation within the same screen ("Submit" on one button, "SAVE" on another) -- uppercase applied through code (`uppercased()`) rather than styling - -## Remediation guidance - -- use title case or sentence case for button labels ("Submit Order", "Learn more") -- if visual uppercase is desired, apply it through `UIButton.Configuration` text transform or CSS `text-transform` rather than changing the actual text string -- ensure the accessibility label uses natural capitalisation even if the visual display is uppercase -- test with VoiceOver to confirm buttons are spoken as words, not spelled out - -## Related docs - -- [iOS Rule Repository](/support/docs/accessibility-ios-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-ios-rule-color-contrast-issues.md b/static/docs/accessibility-ios-rule-color-contrast-issues.md deleted file mode 100644 index 3b9fb447f..000000000 --- a/static/docs/accessibility-ios-rule-color-contrast-issues.md +++ /dev/null @@ -1,36 +0,0 @@ -# Color Contrast Issues - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Text and meaningful visual elements in iOS apps must maintain sufficient contrast against their backgrounds to remain readable for all users. - -**WCAG Reference** -**Maps to:** WCAG 1.4.3 Contrast (Minimum) | **Applies to:** WCAG 2.0, WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.0 | **Level:** AA | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/contrast-minimum.html) - -## What this rule checks - -The scanner measures the contrast ratio between foreground text or icon colors and their backgrounds, flagging pairs that fall below 4.5:1 for normal text or 3:1 for large text. - -## Why it matters - -Users with low vision, color-vision deficiencies, or those using devices in bright outdoor conditions depend on adequate contrast. Low contrast makes text illegible and UI elements indistinguishable. - -## Common failure patterns - -- light gray text on white backgrounds in settings or list cells -- placeholder text in `UITextField` with very low contrast -- status indicators using pale colors on light backgrounds -- custom themes or dark mode implementations where contrast ratios were not validated - -## Remediation guidance - -- verify text contrast against backgrounds using a color contrast analyzer or Xcode Accessibility Inspector -- use Dynamic Colors or semantic color assets that adjust automatically for light and dark modes -- ensure all text states (enabled, disabled, placeholder) meet minimum contrast ratios -- test with Increase Contrast enabled in iOS Accessibility settings - -## Related docs - -- [iOS Rule Repository](/support/docs/accessibility-ios-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-ios-rule-duplicate-accessibility-label.md b/static/docs/accessibility-ios-rule-duplicate-accessibility-label.md deleted file mode 100644 index 999831191..000000000 --- a/static/docs/accessibility-ios-rule-duplicate-accessibility-label.md +++ /dev/null @@ -1,37 +0,0 @@ -# Duplicate Accessibility Label - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Interactive iOS elements within the same screen must have unique accessibility labels to avoid confusion when multiple controls sound identical to VoiceOver. - -**WCAG Reference** -**Maps to:** WCAG 4.1.2 Name, Role, Value | **Applies to:** WCAG 2.0, WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.0 | **Level:** A | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/name-role-value.html) - -## What this rule checks - -The scanner identifies multiple interactive elements on the same screen that share the same accessibility label, making them indistinguishable to VoiceOver users. - -## Why it matters - -When VoiceOver announces two buttons as "Edit" without any additional context, users cannot tell which one to activate. Unique labels enable users to confidently choose the correct control. - -## Common failure patterns - -- list cells each containing an "Edit" button or "Delete" button with identical labels -- multiple "Close" buttons in a screen with several dismissible panels -- icon buttons that all share a generic label like "Action" or "More" -- navigation items on different sections of a tabbed interface with the same label - -## Remediation guidance - -- include context in the label: "Edit shipping address" vs. "Edit billing address" -- for list items, append identifying information: "Delete order #1234" -- use `accessibilityLabel` with specific context rather than generic action words -- in SwiftUI, use `.accessibilityLabel("specific description")` to override default labels -- test by having VoiceOver list all actions on the screen and confirming each is distinguishable - -## Related docs - -- [iOS Rule Repository](/support/docs/accessibility-ios-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-ios-rule-dynamic-type-support.md b/static/docs/accessibility-ios-rule-dynamic-type-support.md deleted file mode 100644 index 238126687..000000000 --- a/static/docs/accessibility-ios-rule-dynamic-type-support.md +++ /dev/null @@ -1,37 +0,0 @@ -# Dynamic Type Support - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -iOS apps must support Dynamic Type so text scales according to the user's preferred text size setting. - -**WCAG Reference** -**Maps to:** WCAG 1.4.4 Resize Text | **Applies to:** WCAG 2.0, WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.0 | **Level:** AA | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/resize-text.html) - -## What this rule checks - -The scanner verifies that text elements use scalable font styles (either system text styles or custom fonts registered with `UIFontMetrics`) and respond to the user's Dynamic Type preference. - -## Why it matters - -Users with low vision set a larger preferred text size in iOS Settings. If an app uses fixed font sizes, text remains small regardless of the user's preference, defeating the purpose of the system-wide accessibility feature. - -## Common failure patterns - -- hard-coded font sizes (e.g., `UIFont.systemFont(ofSize: 14)`) instead of text styles -- custom fonts that are not registered with `UIFontMetrics` for scaling -- layouts that break at large text sizes because they use fixed dimensions -- text that scales but containers do not, causing clipping - -## Remediation guidance - -- use `UIFont.preferredFont(forTextStyle:)` or register custom fonts with `UIFontMetrics` -- set `adjustsFontForContentSizeCategory = true` on labels and text views -- use Auto Layout with constraints that accommodate text growth -- test with all Dynamic Type sizes, including the five Accessibility sizes beyond the standard range -- in SwiftUI, use `.font(.body)` or other text styles which scale automatically - -## Related docs - -- [iOS Rule Repository](/support/docs/accessibility-ios-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-ios-rule-missing-accessibility-labels.md b/static/docs/accessibility-ios-rule-missing-accessibility-labels.md deleted file mode 100644 index 2bd807d9e..000000000 --- a/static/docs/accessibility-ios-rule-missing-accessibility-labels.md +++ /dev/null @@ -1,34 +0,0 @@ -# Missing Accessibility Labels - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Interactive iOS elements need accessible names that assistive technologies can announce. - -**WCAG Reference** -**Maps to:** WCAG 4.1.2 Name, Role, Value | **Applies to:** WCAG 2.0, WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.0 | **Level:** A | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/name-role-value.html) - -## What this rule checks - -This rule checks whether meaningful iOS controls and interactive elements expose accessible names through labels and related accessibility properties. - -## Why it matters - -VoiceOver users depend on clear accessible names to understand what each control does and to move through the interface confidently. - -## Common failure patterns - -- custom buttons with no `accessibilityLabel` -- image-based controls with no accessible name -- duplicate labels that make multiple controls sound identical - -## Remediation guidance - -- add accurate labels for meaningful interactive elements -- make labels action-oriented when the element performs an action -- verify the spoken output in VoiceOver after implementation - -## Related docs - -- [iOS Rule Repository](/support/docs/accessibility-ios-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-ios-rule-missing-button-element-label.md b/static/docs/accessibility-ios-rule-missing-button-element-label.md deleted file mode 100644 index f03740783..000000000 --- a/static/docs/accessibility-ios-rule-missing-button-element-label.md +++ /dev/null @@ -1,36 +0,0 @@ -# Missing Button Element Label - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -iOS button elements must have an accessibility label that describes the action they perform. - -**WCAG Reference** -**Maps to:** WCAG 1.3.1 Info and Relationships | **Applies to:** WCAG 2.0, WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.0 | **Level:** A | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/info-and-relationships.html) - -## What this rule checks - -The scanner flags `UIButton` and `Button` elements that have no title text, no accessibility label, and rely only on an image or icon without an accessible name for VoiceOver. - -## Why it matters - -VoiceOver users hear "Button" with no additional context when a button lacks a label. They cannot tell what the button does, making it impossible to confidently interact with the interface. - -## Common failure patterns - -- icon-only buttons (close X, hamburger menu, share icon) with no accessibility label -- buttons with an image but no title text -- custom button subclasses that override drawing but do not set accessibility properties -- system bar button items using custom images without labels - -## Remediation guidance - -- set `accessibilityLabel` on every button that does not have visible title text -- for icon-only buttons, describe the action: "Close", "Open menu", "Share" -- in SwiftUI, use `.accessibilityLabel("action description")` on `Button` views -- if the button already has title text, VoiceOver uses it automatically, no additional label needed - -## Related docs - -- [iOS Rule Repository](/support/docs/accessibility-ios-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-ios-rule-missing-checkbox-element-label.md b/static/docs/accessibility-ios-rule-missing-checkbox-element-label.md deleted file mode 100644 index cd2a9b39d..000000000 --- a/static/docs/accessibility-ios-rule-missing-checkbox-element-label.md +++ /dev/null @@ -1,35 +0,0 @@ -# Missing Checkbox Element Label - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Checkbox-style controls in iOS apps must have an accessibility label that describes the option they control. - -**WCAG Reference** -**Maps to:** WCAG 1.3.1 Info and Relationships | **Applies to:** WCAG 2.0, WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.0 | **Level:** A | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/info-and-relationships.html) - -## What this rule checks - -The scanner flags toggle-like or checkbox-pattern elements that have no accessibility label, leaving VoiceOver to announce only the state (selected/not selected) without context. - -## Why it matters - -Hearing "not selected" without knowing what option is being referred to is useless. VoiceOver users need both the label and the state to understand the setting and decide whether to change it. - -## Common failure patterns - -- custom checkbox views built with images and tap gestures but no accessibility label -- settings toggles where the label is a separate `UILabel` not programmatically associated -- list cells with selection indicators but no per-cell accessibility description - -## Remediation guidance - -- set `accessibilityLabel` on the checkbox element to describe the option -- group the checkbox and its label into a single accessibility element using `shouldGroupAccessibilityChildren` -- in SwiftUI, use `.accessibilityLabel("Option description")` on toggle or checkbox views -- test with VoiceOver to confirm the full announcement includes both the label and the state - -## Related docs - -- [iOS Rule Repository](/support/docs/accessibility-ios-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-ios-rule-missing-editable-element-label.md b/static/docs/accessibility-ios-rule-missing-editable-element-label.md deleted file mode 100644 index 562c07435..000000000 --- a/static/docs/accessibility-ios-rule-missing-editable-element-label.md +++ /dev/null @@ -1,37 +0,0 @@ -# Missing Editable Element Label - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Editable iOS elements (text fields, text views, search bars) must have an accessibility label that describes what input is expected. - -**WCAG Reference** -**Maps to:** WCAG 1.3.1 Info and Relationships | **Applies to:** WCAG 2.0, WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.0 | **Level:** A | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/info-and-relationships.html) - -## What this rule checks - -The scanner flags `UITextField`, `UITextView`, `UISearchBar`, and similar editable controls that have no accessibility label and no associated placeholder text that VoiceOver can use as a fallback. - -## Why it matters - -When a text field has no label, VoiceOver announces "Text field" with no indication of what the user should type. This makes forms, search interfaces, and data entry screens unusable for blind users. - -## Common failure patterns - -- text fields that rely on visual placeholder text but have no accessibility label -- search bars where the placeholder disappears on focus and no label remains -- multi-line text views (comments, notes) with no accessible description -- form fields where the label is a separate view not programmatically linked to the input - -## Remediation guidance - -- set `accessibilityLabel` on every editable element to describe the expected input -- use `UITextField`'s placeholder text as a fallback, but provide a proper label for clarity -- group the label and input into a single accessibility element if appropriate -- in SwiftUI, use `.accessibilityLabel("Field description")` on `TextField` views -- test with VoiceOver to confirm each field announces a meaningful description before the user types - -## Related docs - -- [iOS Rule Repository](/support/docs/accessibility-ios-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-ios-rule-missing-image-element-label.md b/static/docs/accessibility-ios-rule-missing-image-element-label.md deleted file mode 100644 index 66009ebeb..000000000 --- a/static/docs/accessibility-ios-rule-missing-image-element-label.md +++ /dev/null @@ -1,36 +0,0 @@ -# Missing Image Element Label - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Meaningful `UIImageView` and `Image` elements in iOS apps must have an accessibility label that describes their content or purpose. - -**WCAG Reference** -**Maps to:** WCAG 1.1.1 Non-text Content | **Applies to:** WCAG 2.0, WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.0 | **Level:** A | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/non-text-content.html) - -## What this rule checks - -The scanner identifies image elements that are visible on screen and carry meaningful content but have no `accessibilityLabel`, causing VoiceOver to either skip them or announce the image file name. - -## Why it matters - -VoiceOver users cannot see images. Without a label, they miss the information the image conveys, whether it is a product photo, an avatar, a status icon, or a chart. - -## Common failure patterns - -- product images in a shopping app with no accessibility label -- user avatars or profile pictures with no descriptive text -- status icons (success checkmark, error X) with no label -- chart or graph images with no text alternative - -## Remediation guidance - -- set `accessibilityLabel` on every meaningful `UIImageView` to describe what the image shows -- for decorative images, set `isAccessibilityElement = false` to hide them from VoiceOver -- in SwiftUI, use `.accessibilityLabel("description")` on `Image` views -- keep labels descriptive but concise (e.g., "Product photo: Blue running shoes" rather than "image_shoe_blue_v2.png") - -## Related docs - -- [iOS Rule Repository](/support/docs/accessibility-ios-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-ios-rule-missing-switch-element-label.md b/static/docs/accessibility-ios-rule-missing-switch-element-label.md deleted file mode 100644 index 2bdfd5096..000000000 --- a/static/docs/accessibility-ios-rule-missing-switch-element-label.md +++ /dev/null @@ -1,35 +0,0 @@ -# Missing Switch Element Label - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -iOS switch controls must have an accessibility label that describes the setting they toggle. - -**WCAG Reference** -**Maps to:** WCAG 1.3.1 Info and Relationships | **Applies to:** WCAG 2.0, WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.0 | **Level:** A | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/info-and-relationships.html) - -## What this rule checks - -The scanner flags `UISwitch` and `Toggle` (SwiftUI) elements that have no accessibility label, leaving VoiceOver to announce only "Switch, on" or "Switch, off" without context. - -## Why it matters - -Hearing "Switch, off" tells the user the state but not what the switch controls. Without a label, users cannot make informed decisions about whether to toggle the switch. - -## Common failure patterns - -- `UISwitch` elements in table cells where the label text is in a separate `UILabel` not programmatically linked -- settings screens where switches are visually paired with labels but not associated in the accessibility tree -- custom switch implementations that do not expose an accessible name - -## Remediation guidance - -- set `accessibilityLabel` on the `UISwitch` to describe the setting (e.g., "Dark mode", "Push notifications") -- use `UITableViewCell`'s built-in `textLabel` as the accessibility label for the cell when the switch is the cell's accessory view -- in SwiftUI, the `Toggle("Label text")` initializer automatically provides a label. Ensure it is always populated -- test with VoiceOver to confirm the full announcement includes both the setting name and the state - -## Related docs - -- [iOS Rule Repository](/support/docs/accessibility-ios-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-ios-rule-repository.md b/static/docs/accessibility-ios-rule-repository.md deleted file mode 100644 index 6b0df53de..000000000 --- a/static/docs/accessibility-ios-rule-repository.md +++ /dev/null @@ -1,49 +0,0 @@ -# iOS Rule Repository - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Use this repository to browse iOS accessibility rules and remediation guidance. - -This repository groups iOS accessibility rules (VoiceOver, labeling, touch targets, and related mobile requirements) with pointers into deeper rule pages. Open it when an App Scanner or Native App Automation report cites an iOS finding and you want WCAG-aligned context and remediation next steps. - -## When to use this - -Use this page when your Accessibility App or Native App Automation report points to an iOS issue and you need rule-level explanation or remediation guidance. - -## Priority iOS rule pages - -| Rule page | WCAG | Use it for | -| --- | --- | --- | -| [Missing Accessibility Labels](/support/docs/accessibility-ios-rule-missing-accessibility-labels/) | 4.1.2 | Controls or meaningful content without accessible names | -| [Touch Target Sizing](/support/docs/accessibility-ios-rule-touch-target-sizing/) | 2.5.5 | Controls that are too small or too tightly spaced | - -## Quick Reference - -| Rule Name | WCAG | Level | Impact | Description | Deep dive | -|-----------|------|-------|--------|-------------|-----------| -| Missing Accessibility Labels | 4.1.2 | A | Serious | Interactive UI elements lack proper accessibility labels, hints, or descriptions that VoiceOver can announce to users. Set `accessibilityLabel` for meaningful descriptions and `accessibilityHint` for usage guidance. In SwiftUI use `.accessibilityLabel()` and `.accessibilityHint()`. For images, provide descriptive text. For buttons, ensure labels describe the action, not just visual appearance. | [Open rule](/support/docs/accessibility-ios-rule-missing-accessibility-labels/) | -| Color Contrast Issues | 1.4.3 | AA | Serious | Text and background color combinations fail to meet WCAG minimum contrast ratios for visual accessibility. Use Apple's Color Contrast Calculator or online tools to verify 4.5:1 ratio for normal text, 3:1 for large text. Implement system colors that adapt to accessibility settings and test in high contrast mode. Consider using semantic colors like `UIColor.label` and `UIColor.systemBackground`. | - | -| Touch Target Sizing | 2.5.5 | AAA | Moderate | Touch target areas are smaller than Apple's recommended 44pt minimum size or lack adequate spacing from adjacent elements. Ensure interactive elements are at least 44x44 points by increasing button frame size or adding transparent padding. Use constraints to maintain minimum spacing between adjacent touchable elements. In SwiftUI, use `.frame(minWidth: 44, minHeight: 44)` and test on actual devices. | [Open rule](/support/docs/accessibility-ios-rule-touch-target-sizing/) | -| Assistive Technology Access | 4.1.2 | A | Serious | UI components cannot be properly detected, focused, or activated by VoiceOver and other assistive technologies. Set `isAccessibilityElement = true` for custom views, ensure proper view hierarchy, and avoid blocking accessibility elements. Implement `accessibilityActivate()` for custom interactions and use `accessibilityElements` array to define focus order. Test navigation with VoiceOver gestures. | - | -| Text Truncation Issues | 1.4.4 | AA | Serious | Text content becomes truncated or cut off when users increase font sizes through iOS Dynamic Type settings. Use `adjustsFontForContentSizeCategory = true` on text elements and implement flexible layouts with priority constraints. Use `UIFont.preferredFont(forTextStyle:)` for scalable system fonts. Test with largest accessibility text sizes and avoid fixed height constraints on text containers. | - | -| Accessibility Role Definition | 4.1.2 | A | Moderate | UI elements lack appropriate accessibility traits that define their role, state, or behavior for assistive technology interaction. Set correct `accessibilityTraits` (.button, .link, .header, .selected, .disabled) and combine traits when needed. Update traits dynamically based on state changes. In SwiftUI, use `.accessibilityAddTraits()` and `.accessibilityRemoveTraits()` to ensure custom controls communicate their purpose clearly. | - | -| Dynamic Type Support | 1.4.4 | AA | Serious | App interface fails to properly scale or adapt when users enable larger text sizes in iOS accessibility settings. Enable Dynamic Type support using `traitCollectionDidChange` to respond to size changes and use Auto Layout with flexible constraints. Implement `adjustsFontForContentSizeCategory` on text elements. Test with Settings > Accessibility > Display & Text Size > Larger Text and design layouts that reflow gracefully. | - | -| Accessibility Label Not Punctuated | 3.3.2 | A | Critical | Accessibility labels are missing sentence-ending punctuation. End accessibility labels with a period, exclamation mark, or question mark so VoiceOver inserts a natural pause between elements. Without punctuation, the screen reader reads consecutive labels as a single run-on phrase, making it harder for users to distinguish where one element ends and the next begins. | - | -| Missing Image Element Label | 1.1.1 | A | Critical | An accessibility label is an attribute assigned to UIImageView or UIButton elements that convey information graphically. This label gives a textual description of the graphic, making it accessible to users relying on screen readers. Add the `accessibilityLabel` property to all non-decorative UIImageView and UIButton elements containing images. For decorative images, set `isAccessibilityElement = false` to prevent unnecessary screen reader announcements. | - | -| Missing Button Element Label | 1.3.1 | A | Critical | Button elements within your app must be properly labeled and fully accessible to users relying on assistive technologies like screen readers. Add `accessibilityLabel` or button title text to all buttons so screen reader users understand what action each button performs. Ensure labels describe the action (e.g., "Submit form", "Close dialog") rather than just the visual appearance. | - | -| Button Element Capitalisation Check | 3.1.6 | AAA | Minor | Button accessibility labels should begin with an uppercase letter and follow proper capitalization standards to improve readability and screen reader pronunciation. Use sentence case (e.g., "Save changes") instead of all caps or inconsistent capitalization. Avoid ALL CAPS as it may cause screen readers to read each letter individually. Follow iOS Human Interface Guidelines for consistent text formatting. | - | -| Missing Checkbox Element Label | 1.3.1 | A | Critical | Checkbox elements within your app must be properly labeled and fully accessible to users relying on assistive technologies like screen readers. Add `accessibilityLabel` to checkbox elements to describe what option is being selected or deselected. Combine with appropriate `accessibilityTraits` (.button, .selected) and update traits dynamically when state changes to communicate current selection status. | - | -| Missing Editable Element Label | 1.3.1 | A | Critical | Editable elements such as UITextField or UITextView should have both name and value available to screen readers. Add `accessibilityLabel` or associate with a UILabel to describe what information should be entered in text fields. Use `placeholder` text for additional guidance but never rely on it alone for accessibility. Ensure labels remain visible when fields are focused or filled. | - | -| Missing Switch Element Label | 1.3.1 | A | Critical | Switch controls (UISwitch) in an app must have both their name and value available to screen readers. Add `accessibilityLabel` to switch controls to describe what setting or feature is being toggled on or off (e.g., "Enable notifications", "Dark mode"). The switch state (On/Off) is automatically announced by VoiceOver, so focus the label on describing the setting being controlled. | - | -| Duplicate Accessibility Label | 4.1.2 | A | Critical | Multiple UI elements on the same screen share the same accessibility label, causing confusion for screen reader users who cannot distinguish between different controls. Ensure each interactive element has a unique `accessibilityLabel` to prevent confusion. For similar elements (like multiple "Delete" buttons), add context such as "Delete photo 1", "Delete photo 2" or reference the item being acted upon. | - | - -## How to use this repository - -1. Open the rule page that best matches the report finding. -2. Review the rule intent and the likely iOS implementation pattern involved. -3. Apply the fix and rerun the affected scan flow to confirm improvement. - -## Related docs - -- [iOS checklist](/support/docs/accessibility-ios-what-we-cover/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-ios-rule-text-truncation-issues.md b/static/docs/accessibility-ios-rule-text-truncation-issues.md deleted file mode 100644 index 31b41c0bb..000000000 --- a/static/docs/accessibility-ios-rule-text-truncation-issues.md +++ /dev/null @@ -1,36 +0,0 @@ -# Text Truncation Issues - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Text content in iOS apps must not be truncated in ways that hide meaningful information from users, especially when Dynamic Type increases font sizes. - -**WCAG Reference** -**Maps to:** WCAG 1.4.4 Resize Text | **Applies to:** WCAG 2.0, WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.0 | **Level:** AA | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/resize-text.html) - -## What this rule checks - -The scanner identifies text elements that are clipped or truncated (ending with "...") when the content contains meaningful information that is lost, particularly when the user has increased the text size through Dynamic Type settings. - -## Why it matters - -Users with low vision rely on Dynamic Type to increase text size. If the app truncates text instead of expanding or wrapping, these users lose access to important information like button labels, instructions, or data values. - -## Common failure patterns - -- labels with a fixed `numberOfLines = 1` that truncate long content -- table cells with fixed heights that clip multi-line text at larger Dynamic Type sizes -- navigation bar titles that truncate when text size increases -- buttons with fixed widths that clip label text - -## Remediation guidance - -- use `numberOfLines = 0` with Auto Layout constraints to allow text to wrap -- set minimum cell heights using Auto Layout rather than fixed values -- use `adjustsFontForContentSizeCategory = true` on text elements -- test with the largest Dynamic Type size (Accessibility sizes enabled) to confirm no meaningful content is lost - -## Related docs - -- [iOS Rule Repository](/support/docs/accessibility-ios-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-ios-rule-touch-target-sizing.md b/static/docs/accessibility-ios-rule-touch-target-sizing.md deleted file mode 100644 index dc9172e7e..000000000 --- a/static/docs/accessibility-ios-rule-touch-target-sizing.md +++ /dev/null @@ -1,34 +0,0 @@ -# Touch Target Sizing - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Touch targets should be large enough for reliable interaction. - -**WCAG Reference** -**Maps to:** WCAG 2.5.5 Target Size (Enhanced) | **Applies to:** WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.1 | **Level:** AAA | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/target-size-enhanced.html) - -## What this rule checks - -This rule checks whether iOS controls provide enough touch area and spacing for reliable interaction. - -## Why it matters - -Small or tightly packed targets make interaction harder for users with motor impairments and for anyone using assistive touch patterns. - -## Common failure patterns - -- icons or text buttons smaller than the recommended touch area -- adjacent controls with almost no spacing between them -- visually large controls whose tappable region is much smaller than the rendered component - -## Remediation guidance - -- increase the tappable region of small controls -- add spacing between adjacent actions -- verify that custom components preserve touch size across device sizes - -## Related docs - -- [iOS Rule Repository](/support/docs/accessibility-ios-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-ios-rules.md b/static/docs/accessibility-ios-rules.md deleted file mode 100644 index 852f01170..000000000 --- a/static/docs/accessibility-ios-rules.md +++ /dev/null @@ -1,31 +0,0 @@ -# iOS Accessibility Rules - Quick Reference - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -These iOS rule summaries are a quick reference for findings that appear in Accessibility App Scanner and Native App Automation reports. Use the repository pages for deeper remediation guidance when a finding needs rule-level detail. - -| Rule Name | WCAG | Level | Impact | Description | Deep dive | -|-------------------------------------|--------|-------|---------|-------------|-----------| -| Missing Accessibility Labels | 4.1.2 | A | Serious | Interactive UI elements lack proper accessibility labels, hints, or descriptions that VoiceOver can announce to users. Set `accessibilityLabel` for meaningful descriptions and `accessibilityHint` for usage guidance. In SwiftUI use `.accessibilityLabel()` and `.accessibilityHint()`. For images, provide descriptive text. For buttons, ensure labels describe the action, not just visual appearance. | [Open rule](/support/docs/accessibility-ios-rule-missing-accessibility-labels/) | -| Color Contrast Issues | 1.4.3 | AA | Serious | Text and background color combinations fail to meet WCAG minimum contrast ratios for visual accessibility. Use Apple's Color Contrast Calculator or online tools to verify 4.5:1 ratio for normal text, 3:1 for large text. Implement system colors that adapt to accessibility settings and test in high contrast mode. Consider using semantic colors like `UIColor.label` and `UIColor.systemBackground`. | - | -| Touch Target Sizing | 2.5.5 | AAA | Moderate | Touch target areas are smaller than Apple's recommended 44pt minimum size or lack adequate spacing from adjacent elements. Ensure interactive elements are at least 44x44 points by increasing button frame size or adding transparent padding. Use constraints to maintain minimum spacing between adjacent touchable elements. In SwiftUI, use `.frame(minWidth: 44, minHeight: 44)` and test on actual devices. | [Open rule](/support/docs/accessibility-ios-rule-touch-target-sizing/) | -| Assistive Technology Access | 4.1.2 | A | Serious | UI components cannot be properly detected, focused, or activated by VoiceOver and other assistive technologies. Set `isAccessibilityElement = true` for custom views, ensure proper view hierarchy, and avoid blocking accessibility elements. Implement `accessibilityActivate()` for custom interactions and use `accessibilityElements` array to define focus order. Test navigation with VoiceOver gestures. | - | -| Text Truncation Issues | 1.4.4 | AA | Serious | Text content becomes truncated or cut off when users increase font sizes through iOS Dynamic Type settings. Use `adjustsFontForContentSizeCategory = true` on text elements and implement flexible layouts with priority constraints. Use `UIFont.preferredFont(forTextStyle:)` for scalable system fonts. Test with largest accessibility text sizes and avoid fixed height constraints on text containers. | - | -| Accessibility Role Definition | 4.1.2 | A | Moderate | UI elements lack appropriate accessibility traits that define their role, state, or behavior for assistive technology interaction. Set correct `accessibilityTraits` (.button, .link, .header, .selected, .disabled) and combine traits when needed. Update traits dynamically based on state changes. In SwiftUI, use `.accessibilityAddTraits()` and `.accessibilityRemoveTraits()` to ensure custom controls communicate their purpose clearly. | - | -| Dynamic Type Support | 1.4.4 | AA | Serious | App interface fails to properly scale or adapt when users enable larger text sizes in iOS accessibility settings. Enable Dynamic Type support using `traitCollectionDidChange` to respond to size changes and use Auto Layout with flexible constraints. Implement `adjustsFontForContentSizeCategory` on text elements. Test with Settings > Accessibility > Display & Text Size > Larger Text and design layouts that reflow gracefully. | - | -| Accessibility Label Not Punctuated | 3.3.2 | A | Critical | Accessibility labels are missing sentence-ending punctuation. End accessibility labels with a period, exclamation mark, or question mark so VoiceOver inserts a natural pause between elements. Without punctuation, the screen reader reads consecutive labels as a single run-on phrase, making it harder for users to distinguish where one element ends and the next begins. | - | -| Missing Image Element Label | 1.1.1 | A | Critical | An accessibility label is an attribute assigned to UIImageView or UIButton elements that convey information graphically. This label gives a textual description of the graphic, making it accessible to users relying on screen readers. Add the `accessibilityLabel` property to all non-decorative UIImageView and UIButton elements containing images. For decorative images, set `isAccessibilityElement = false` to prevent unnecessary screen reader announcements. | - | -| Missing Button Element Label | 1.3.1 | A | Critical | Button elements within your app must be properly labeled and fully accessible to users relying on assistive technologies like screen readers. Add `accessibilityLabel` or button title text to all buttons so screen reader users understand what action each button performs. Ensure labels describe the action (e.g., "Submit form", "Close dialog") rather than just the visual appearance. | - | -| Button Element Capitalisation Check | 3.1.6 | AAA | Minor | Button accessibility labels should begin with an uppercase letter and follow proper capitalization standards to improve readability and screen reader pronunciation. Use sentence case (e.g., "Save changes") instead of all caps or inconsistent capitalization. Avoid ALL CAPS as it may cause screen readers to read each letter individually. Follow iOS Human Interface Guidelines for consistent text formatting. | - | -| Missing Checkbox Element Label | 1.3.1 | A | Critical | Checkbox elements within your app must be properly labeled and fully accessible to users relying on assistive technologies like screen readers. Add `accessibilityLabel` to checkbox elements to describe what option is being selected or deselected. Combine with appropriate `accessibilityTraits` (.button, .selected) and update traits dynamically when state changes to communicate current selection status. | - | -| Missing Editable Element Label | 1.3.1 | A | Critical | Editable elements such as UITextField or UITextView should have both name and value available to screen readers. Add `accessibilityLabel` or associate with a UILabel to describe what information should be entered in text fields. Use `placeholder` text for additional guidance but never rely on it alone for accessibility. Ensure labels remain visible when fields are focused or filled. | - | -| Missing Switch Element Label | 1.3.1 | A | Critical | Switch controls (UISwitch) in an app must have both their name and value available to screen readers. Add `accessibilityLabel` to switch controls to describe what setting or feature is being toggled on or off (e.g., "Enable notifications", "Dark mode"). The switch state (On/Off) is automatically announced by VoiceOver, so focus the label on describing the setting being controlled. | - | -| Duplicate Accessibility Label | 4.1.2 | A | Critical | Multiple UI elements on the same screen share the same accessibility label, causing confusion for screen reader users who cannot distinguish between different controls. Ensure each interactive element has a unique `accessibilityLabel` to prevent confusion. For similar elements (like multiple "Delete" buttons), add context such as "Delete photo 1", "Delete photo 2" or reference the item being acted upon. | - | - -> We are continuously expanding our App Accessibility guidelines. Visit this page for the latest updates and new requirements. - -## Related docs - -- [iOS Rule Repository](/support/docs/accessibility-ios-rule-repository/) -- [iOS Coverage](/support/docs/accessibility-ios-what-we-cover/) -- [Native App Automation Appium (Overview)](/support/docs/accessibility-native-app-automation-test/) diff --git a/static/docs/accessibility-ios-what-we-cover.md b/static/docs/accessibility-ios-what-we-cover.md deleted file mode 100644 index 5c46b2087..000000000 --- a/static/docs/accessibility-ios-what-we-cover.md +++ /dev/null @@ -1,44 +0,0 @@ -# iOS Accessibility Checklist - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Use this page to see **which rules App Scanner / automation can report** (with links to rule-level guidance) and a **manual test checklist** for areas best validated by people. For remediation detail, use the [iOS Rule Repository](/support/docs/accessibility-ios-rule-repository/). - -**Other platforms:** [Web checklist](/support/docs/accessibility-web-what-we-cover/) · [Android checklist](/support/docs/accessibility-android-what-we-cover/) - -## Supported (rules in this product) - -| Rule | WCAG (maps) | Doc | -|------|-------------|-----| -| Missing Accessibility Labels | 4.1.2 | [Rule →](/support/docs/accessibility-ios-rule-missing-accessibility-labels/) | -| Color Contrast Issues | 1.4.3 | [Rule →](/support/docs/accessibility-ios-rule-color-contrast-issues/) | -| Touch Target Sizing | 2.5.5 | [Rule →](/support/docs/accessibility-ios-rule-touch-target-sizing/) | -| Assistive Technology Access | 4.1.2 | [Rule →](/support/docs/accessibility-ios-rule-assistive-technology-access/) | -| Text Truncation Issues | 1.4.4 | [Rule →](/support/docs/accessibility-ios-rule-text-truncation-issues/) | -| Accessibility Role Definition | 4.1.2 | [Rule →](/support/docs/accessibility-ios-rule-accessibility-role-definition/) | -| Dynamic Type Support | 1.4.4 | [Rule →](/support/docs/accessibility-ios-rule-dynamic-type-support/) | -| Accessibility Label Not Punctuated | 3.3.2 | [Rule →](/support/docs/accessibility-ios-rule-accessibility-label-not-punctuated/) | -| Missing Image Element Label | 1.1.1 | [Rule →](/support/docs/accessibility-ios-rule-missing-image-element-label/) | -| Missing Button Element Label | 1.3.1 | [Rule →](/support/docs/accessibility-ios-rule-missing-button-element-label/) | -| Button Element Capitalisation Check | 3.1.6 | [Rule →](/support/docs/accessibility-ios-rule-button-element-capitalisation-check/) | -| Missing Checkbox Element Label | 1.3.1 | [Rule →](/support/docs/accessibility-ios-rule-missing-checkbox-element-label/) | -| Missing Editable Element Label | 1.3.1 | [Rule →](/support/docs/accessibility-ios-rule-missing-editable-element-label/) | -| Missing Switch Element Label | 1.3.1 | [Rule →](/support/docs/accessibility-ios-rule-missing-switch-element-label/) | -| Duplicate Accessibility Label | 4.1.2 | [Rule →](/support/docs/accessibility-ios-rule-duplicate-accessibility-label/) | - -## Manual test checklist - -Plan extra time for **VoiceOver**, **Switch Control**, **device rotation**, and real-world flows (media, authentication, motion) even when automation is green. Use [Understanding WCAG 2.2](https://www.w3.org/WAI/WCAG22/Understanding/) and Apple’s Human Interface Guidelines alongside this list. - -| Area | Manual verification focus | -|------|---------------------------| -| **1.2.x Time-based media** | In-app video/audio (captions, descriptions, live captions): validate with real content. | -| **1.3.2–1.3.5 structure & input purpose** | Meaningful sequence, sensory-only instructions, orientation lock justification, `UITextContentType` / WebView `lang`: design and assistive-tech pass. | -| **2.x keyboard / focus / timing (full)** | Complete tasks with VoiceOver and hardware keyboard; session timeouts and motion: exercise beyond rule snapshots. | -| **3.x language, errors, help** | Error suggestion quality, legal safeguards, help consistency: product and compliance review. | -| **AAA criteria** | Optional stretch goals: schedule only when AAA is explicitly in scope. | - -## Related docs - -- [iOS Rule Repository](/support/docs/accessibility-ios-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-ios-what-we-do-not-cover.md b/static/docs/accessibility-ios-what-we-do-not-cover.md deleted file mode 100644 index daee5b681..000000000 --- a/static/docs/accessibility-ios-what-we-do-not-cover.md +++ /dev/null @@ -1,3 +0,0 @@ -# iOS Accessibility Manual Checklist - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). diff --git a/static/docs/accessibility-issue-remediation-guide.md b/static/docs/accessibility-issue-remediation-guide.md deleted file mode 100644 index ed1c7f3d8..000000000 --- a/static/docs/accessibility-issue-remediation-guide.md +++ /dev/null @@ -1,39 +0,0 @@ -# Accessibility Issue Remediation Guide - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This guide helps teams move from issue discovery to remediation planning. - -You will learn how to translate report findings into an ordered remediation path that matches user impact. You will also see when to step from summary views into rule-level detail so fixes stay aligned with the underlying WCAG expectations. - -## Recommended remediation order - -1. Fix critical blockers on core user journeys first. -2. Resolve serious issues that affect navigation, interaction, or major content access. -3. Use rule repository and checklist pages for rule-specific explanation. -4. Re-run the affected workflow and compare the updated report. - -## How to use this guide - -- start from Issue Summary to understand the report shape -- move into All Issues for affected elements and issue detail -- use rule repository pages for deeper remediation guidance -- save or export updated reports after validation - -## Onboarding sprint (1–2 week template) - -| Day | Focus | -|-----|--------| -| 1 | Baseline scan on **top 5 URLs**; export snapshot ([exporting](/support/docs/accessibility-exporting-sharing-reports/)). | -| 2–3 | Fix **critical** keyboard and name/role issues using rule pages. | -| 4–5 | Address **serious** contrast and form labeling issues; re-scan. | -| 6 | Manual pass with **[Screen Reader](/support/docs/screen-reader-on-accessibility/)** guidance on primary flows. | -| 7 | Review **needs-review** items; hide accepted risk with rationale ([hide/restore](/support/docs/accessibility-hide-restore-issues/)). | - -Adjust cadence to your release train; keep evidence in ticketing tools via **[integrations](/support/docs/accessibility-report-integrations/)**. - -## Related docs - -- [Web Rule Repository](/support/docs/accessibility-web-rule-repository/) -- [Android Rule Repository](/support/docs/accessibility-android-rule-repository/) -- [iOS Rule Repository](/support/docs/accessibility-ios-rule-repository/) diff --git a/static/docs/accessibility-jaws-windows.md b/static/docs/accessibility-jaws-windows.md deleted file mode 100644 index 1f171aaf7..000000000 --- a/static/docs/accessibility-jaws-windows.md +++ /dev/null @@ -1,35 +0,0 @@ -# JAWS on Windows - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Use this page as the Windows-specific reference for **JAWS** when your accessibility program standardizes on Freedom Scientific tooling alongside or instead of NVDA. - -## When to use this - -Use JAWS when your team needs to validate important desktop screen-reader flows on Windows and compare behavior across assistive technologies. - -## Onboarding: start a productive session - -1. Launch **JAWS** before the browser if your IT policy requires it; otherwise JAWS attaches to the active window. -2. Open **Chrome or Edge** (match your automated scan browsers where possible). -3. Press **Insert + Space** to enter **Forms mode** when interacting with native controls, and return to virtual cursor for reading-heavy pages (JAWS announces mode changes). -4. Use **Insert + F6** for headings list, **Insert + F7** for links list, fast ways to audit IA after a scan. -5. Document findings with **speech history** (Insert + Space, then H) when filing bugs. - -## What to validate - -- screen-reader announcement quality -- control labels and state changes -- table, form, and dialog behavior -- keyboard-only task completion - -## Pairing with Accessibility reports - -1. Pick the **top five** automated issues from **[All Issues](/support/docs/accessibility-testing-dashboard-all-issues/)**. -2. Reproduce each with JAWS active; note whether the issue is **real-world blocking** or a false positive. -3. Update tickets with **JAWS-specific** notes (version, synthesizer) for engineering. - -## Related docs - -- [Screen Reader](/support/docs/screen-reader-on-accessibility/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-junit5-test.md b/static/docs/accessibility-junit5-test.md deleted file mode 100644 index 7b5143c7a..000000000 --- a/static/docs/accessibility-junit5-test.md +++ /dev/null @@ -1,68 +0,0 @@ -# JUnit 5 - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Use this guide when your **Selenium** tests use **JUnit 5 (Jupiter)**. Accessibility is still driven entirely by **grid capabilities** and the **`lambda-accessibility-scan`** hook (or **auto-scan**), identical to the [Selenium Accessibility Automation](/support/docs/accessibility-automation-test/) flow. Only the test lifecycle annotations differ. - -> **Browsers:** Use **Chrome or Edge** with supported versions for Accessibility Automation. - -## Prerequisites - -- TestMu AI credentials as **environment variables** ([guide](/support/docs/using-environment-variables-for-authentication-credentials/)) -- JUnit 5 on the classpath (`junit-jupiter-api`, etc.) driving Selenium -- Accessibility enabled for your account - -## Onboarding path - -### 1. Centralize driver creation - -Use `@BeforeEach` or `@BeforeAll` (with a static WebDriver if you share one per class) to build `MutableCapabilities` / `ChromeOptions` once. Every test method should inherit the same Accessibility flags. - -### 2. Enable Accessibility on the session - -```java -capabilities.setCapability("accessibility", true); -capabilities.setCapability("accessibility.wcagVersion", "wcag21aa"); // optional -``` - -See [Configure Accessibility Automation](/support/docs/accessibility-automation-settings/) for all flags. - -### 3. Trigger scans - -**Hook after stable UI:** - -```java -driver.executeScript("lambda-accessibility-scan"); -``` - -**Or auto-scan every navigation:** - -```java -capabilities.setCapability("accessibility.autoscan", true); -``` - -### 4. Run and verify - -```bash -mvn test -``` - -Open the [Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard) → session → **Accessibility** tab. - -### 5. Scale to the full suite - -Add `@Tag("a11y")` or parallel config if you want selective Accessibility runs in CI; keep capability setup shared so behavior stays consistent. - -## Troubleshooting - -| Symptom | What to check | -|--------|----------------| -| No report | `accessibility: true` plus hook **or** `accessibility.autoscan`. | -| Multiple scans on one test | You called the hook more than once; reduce to one stable checkpoint per logical page. | - -## Related docs - -- [Selenium](/support/docs/accessibility-automation-test/) -- [Configure Accessibility Automation](/support/docs/accessibility-automation-settings/) -- [TestNG](/support/docs/accessibility-testng-test/) -- [CI/CD Integration Guide](/support/docs/accessibility-cicd-integration-guide/) diff --git a/static/docs/accessibility-keyboard-scan.md b/static/docs/accessibility-keyboard-scan.md deleted file mode 100644 index 1ea40e94b..000000000 --- a/static/docs/accessibility-keyboard-scan.md +++ /dev/null @@ -1,83 +0,0 @@ -# Keyboard Accessibility Scan (Beta) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Overview - -The Assistive Keyboard Accessibility Test combines automated tab simulation with guided manual verification to identify keyboard accessibility issues. It helps ensure all interactive elements on your web page are accessible via keyboard navigation. - -## Running a Keyboard Test - -**Step 1: Initiate the Test** - -1. Navigate to the web page you want to test -2. Open the Accessibility Devtools from your chrome devtools option -3. Under Assistive Tests, select Keyboard Scan - -**Step 2: Select Test Scope** - -Choose how much of the page you want to test: - -- **Full Page**: Tests the entire page for keyboard accessibility -- **Partial Page**: Allows you to select a specific section or component - - After selecting this option, click and drag to highlight the area you want to test - - Or click on a container element to test that specific section - -> **Tip**: For complex pages, start with partial page tests on individual components before running a full page test. - -**Step 3: Review Tab Simulation Results** - -The tool will automatically: - -1. Simulate pressing the Tab key through your selected scope -2. Identify all elements that receive keyboard focus -3. Display a list of found tabbable elements -4. Show the total count of tabbable items - -The results will be displayed both as : A numbered list in the tool interface and Visual highlights on the actual page elements. - -**Step 4: Identify Missing Interactive Elements** - -The tool will ask: "Are there any interactive elements within the scope that should be reachable via the Tab key but were not included in the list above?" - -If you select "**No**": - -- The manual verification is complete -- No manual issues are logged -- The test proceeds to generate the final report - -If you select "**Yes**": - -- You'll see the prompt: "Please click on the interactive element(s) that were missed in the tab order" -- Click on any interactive elements (buttons, links, form fields, etc.) that should be keyboard accessible but weren't in the tab list. -- Each element you select will be logged as an accessibility violation. -- Click Confirm Selection when you've identified all missing elements - -**Step 5: Review Automated Findings** - -While you're performing the manual check, the tool runs our accessibility automated engine in the background to detect automated keyboard accessibility issues such as: - -- Illogical focus order -- Nested interactive elements -- Scrollable regions without keyboard access -- Focus indicators missing or unclear - -You can inspect individual issues directly on the page or use the highlight option to visualize them: - -## Frequently Asked Questions - -**Q: Can this tool test keyboard navigation within iframes?** - -A: The tool attempts to test iframe content when possible, but cross-origin iframes may have limitations due to security restrictions. - -**Q: Why didn't the tool detect my custom dropdown menu?** - -A: Complex widgets with dynamic content may require manual identification. Use the "Yes" option in Step 4 to mark these elements. - -**Q: How is this different from a regular axe-core scan?** - -A: This test combines axe-core's automated checks with guided manual testing to catch issues that automated tools alone might miss, particularly custom interactive elements. - -**Q: Can I save and share test results?** - -A: Yes, test results can be exported and shared with your team. Look for the export options in the report view. diff --git a/static/docs/accessibility-mcp-server.md b/static/docs/accessibility-mcp-server.md deleted file mode 100644 index d4f4d80a8..000000000 --- a/static/docs/accessibility-mcp-server.md +++ /dev/null @@ -1,28 +0,0 @@ -# Getting Started with Accessibility MCP Tool - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Accessibility MCP Tool is part of the [TestMu AI MCP Server](/support/docs/testmu-mcp-server/). It runs accessibility checks on publicly hosted URLs and local React applications, and returns a detailed report with remediation guidance. - -## Key Features - -- Identifies accessibility issues in your application. -- Generates an accessibility report you can share with your team. -- Provides remediation guidance for a11y issues in locally hosted applications. - -## Capabilities - -1. **getAccessibilityReport** - Fetches a detailed accessibility report for a public URL. You can view the report in your browser to analyze the issues. -2. **buildLocalAppForAnalysis** - Builds and serves your local React application via TestMu AI and identifies accessibility issues. -3. **analyseAppViaTunnel** - Tests a local app already running via TestMu AI tunnel for accessibility failures. - -## How to Invoke - -Once your MCP client is connected, use natural language to interact with the Accessibility tool. Example prompts: - -- "Run an accessibility report on `https://example.com`." -- "Build my local React app and check it for accessibility issues." -- "I have a local app running on port 3000 via TestMu AI tunnel, check it for a11y failures." -- "Show me the accessibility violations on the homepage and how to fix them." - -Your AI client will route the request to the appropriate capability and return the report or remediation guidance inline. diff --git a/static/docs/accessibility-native-app-automation-test.md b/static/docs/accessibility-native-app-automation-test.md deleted file mode 100644 index 0eafa68fe..000000000 --- a/static/docs/accessibility-native-app-automation-test.md +++ /dev/null @@ -1,48 +0,0 @@ -# Native App Automation Appium (Overview) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Native App Automation uses **Appium** with the **`lambda-accessibility-scan`** hook to generate accessibility results during **Android** or **iOS** test runs on the TestMu AI grid. - -## When to use this - -Use this page when your team **already runs Appium** for functional tests and wants accessibility checks in the **same execution path** with deterministic checkpoints. - -## Prerequisites - -- Appium client and test project targeting TestMu AI **real devices** or emulators per your subscription -- `LT_USERNAME` / `LT_ACCESS_KEY` available to the process -- Accessibility enabled on the mobile session (see framework guides below for capability examples) - -## Onboarding: first automated mobile accessibility run - -1. **Reuse a green Appium suite** without accessibility first: confirm install, locators, and hub connectivity. -2. **Add accessibility capability** on the session (exact key names align with your Appium server version; mirror the examples in [Appium TestNG](/support/docs/accessibility-appium-testng/) or [Appium WebdriverIO](/support/docs/accessibility-appium-webdriverio/)). -3. After each **stable screen** (post-navigation waits), call: - -```java -driver.executeScript("lambda-accessibility-scan"); -``` - -4. **Run a single test method** to validate a report appears under Accessibility for that build. -5. Open **[Navigating the Dashboard](/support/docs/accessibility-testing-navigating-dashboard/)** and locate the session’s accessibility report. -6. Expand coverage to full suites only after reports look stable. - -## Typical workflow (summary) - -1. Upload the app and configure the Appium session. -2. Enable Accessibility in the session capabilities. -3. Trigger `lambda-accessibility-scan` at important checkpoints. -4. Review the report in the dashboard. - -## Product boundary - -This page is for direct Appium-based automation. If you are authoring the flow in KaneAI, use [Mobile App Accessibility Testing](/support/docs/kaneai-mobile-app-accessibility/). If you want manual screen-by-screen testing, use [Accessibility App Scanner (Overview)](/support/docs/accessibility-app-scanner/). - -## Related docs - -- [Scan Configurations via Capabilities (Automation)](/support/docs/accessibility-automation-scan-configurations/) -- [Appium TestNG](/support/docs/accessibility-appium-testng/) -- [Appium WebdriverIO](/support/docs/accessibility-appium-webdriverio/) -- [Accessibility App Scanner (Overview)](/support/docs/accessibility-app-scanner/) -- [Tag Support for Accessibility Scans](/support/docs/accessibility-tag-support/) diff --git a/static/docs/accessibility-nunit-csharp-test.md b/static/docs/accessibility-nunit-csharp-test.md deleted file mode 100644 index c9cd72cd5..000000000 --- a/static/docs/accessibility-nunit-csharp-test.md +++ /dev/null @@ -1,65 +0,0 @@ -# NUnit (C#) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Use this guide for **Selenium C#** tests executed with **NUnit**. Accessibility is enabled on the **RemoteWebDriver** session using the same capability keys as Java ([reference](/support/docs/accessibility-automation-settings/)); NUnit only controls **how and when** tests run. - -> **Browsers:** Use **Chrome or Edge** with supported versions. - -## Prerequisites - -- `LT_USERNAME` / `LT_ACCESS_KEY` (or your chosen secret injection in CI) -- Selenium 4 `RemoteWebDriver` pointing at the TestMu AI hub -- NUnit 3+ test project - -## Onboarding path - -### 1. Set capabilities in your driver factory - -Typical pattern in `[SetUp]` or a one-time fixture: - -```csharp -var options = new ChromeOptions(); -options.BrowserVersion = "latest"; -options.AddAdditionalOption("accessibility", true); -options.AddAdditionalOption("accessibility.wcagVersion", "wcag21aa"); // optional -// options.AddAdditionalOption("accessibility.autoscan", true); // optional -var driver = new RemoteWebDriver(new Uri("https://hub.lambdatest.com/wd/hub"), options.ToCapabilities()); -``` - -Exact hub URL and capability names should match your existing grid configuration; only the **`accessibility*`** keys are specific to this feature. - -### 2. Invoke the scan hook (if not using auto-scan) - -After navigation and waits: - -```csharp -((IJavaScriptExecutor)driver).ExecuteScript("lambda-accessibility-scan"); -``` - -Without **either** the hook **or** `accessibility.autoscan`, you will not get Accessibility reports. - -### 3. Run NUnit - -```bash -dotnet test -``` - -or Visual Studio Test Explorer. - -### 4. Open the Accessibility report - -Automation Dashboard → build → **Accessibility** tab (same as [Selenium guide](/support/docs/accessibility-automation-test/)). - -## Troubleshooting - -| Symptom | What to check | -|--------|----------------| -| Capability ignored | Selenium 4 requires `AddAdditionalOption` on `ChromeOptions` (or equivalent) for vendor-specific keys. Verify spelling. | -| No report | Confirm hook or autoscan as above. | - -## Related docs - -- [Selenium](/support/docs/accessibility-automation-test/) -- [Configure Accessibility Automation](/support/docs/accessibility-automation-settings/) -- [CI/CD Integration Guide](/support/docs/accessibility-cicd-integration-guide/) diff --git a/static/docs/accessibility-nvda-windows.md b/static/docs/accessibility-nvda-windows.md deleted file mode 100644 index 6eb3c231f..000000000 --- a/static/docs/accessibility-nvda-windows.md +++ /dev/null @@ -1,40 +0,0 @@ -# NVDA on Windows - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Use this page as the Windows-specific reference for **NVDA** when validating **web** accessibility after DevTools scans, automation, or scheduling. - -## When to use this - -Use NVDA on Windows when you want to understand how screen-reader users experience page structure, labels, navigation, and interactive controls in desktop web workflows. - -## Onboarding: install and basic usage - -1. Download **NVDA** from [https://www.nvaccess.org/](https://www.nvaccess.org/) and install on Windows 10/11. -2. Start NVDA with **Ctrl + Alt + N** (default) or from the Start menu. -3. Learn **single-key navigation**: `H` headings, `F` form fields, `B` buttons, `K` links (NVDA browse mode). -4. Open your target URL in **Chrome or Edge** (match what you scan in Accessibility Automation). -5. Use **Tab / Shift+Tab** to follow focus order; listen for **name, role, and state** announcements. -6. Toggle **speech viewer** (NVDA menu → Tools) if you need to capture text for bug tickets. - -## What to validate - -- page titles and headings -- link and button names -- form labels and instructions -- focus order and keyboard navigation -- state announcements for dynamic UI - -## Suggested first session (30 minutes) - -| Step | Action | -|------|--------| -| 1 | Open homepage; confirm title and landmark announcements. | -| 2 | Tab through primary nav; every focusable control should speak a sensible name. | -| 3 | Open a form; each control should expose label + instructions + errors. | -| 4 | Trigger a live region or toast; confirm polite/assertive behavior matches design. | - -## Related docs - -- [Screen Reader](/support/docs/screen-reader-on-accessibility/) -- [Keyboard Scan](/support/docs/accessibility-keyboard-scan/) diff --git a/static/docs/accessibility-passed-test-cases.md b/static/docs/accessibility-passed-test-cases.md deleted file mode 100644 index 6a8702b57..000000000 --- a/static/docs/accessibility-passed-test-cases.md +++ /dev/null @@ -1,32 +0,0 @@ -# Passed Test Cases - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Passed Test Cases lets teams include successful accessibility checks in a report instead of showing only failures and incomplete items. - -Passed checks deserve attention alongside violations because they show what already works and make trends easier to spot across runs. The sections that follow explain why that balance matters, when to turn the feature on, and how it improves conversations with auditors and stakeholders. - -## When to use this - -Use this feature when you want a fuller view of Accessibility progress and not just a list of violations. - -## Why it matters - -Passed checks help teams: - -- show what is already working well -- compare results across runs -- provide more balanced review material for audits and stakeholders - -## How to enable and read Passed Test Cases (onboarding) - -1. Open an Accessibility **report settings** or **scan configuration** surface where passed checks are exposed (wording may appear as “Passed tests”, “Include passing checks”, or similar, UI varies by scan type). -2. Turn the option **on** before the next run if you want auditors to see green checks alongside violations. -3. After the run, open **[Issue Summary](/support/docs/accessibility-testing-dashboard-issue-summary/)** and look for passed-rule summaries or toggles that reveal passing rules. -4. Pair passes with **[Accessibility Score](/support/docs/accessibility-web-score/)** trends so leadership sees improvement, not only open defects. -5. Export the augmented report when sharing outside the dashboard ([exporting](/support/docs/accessibility-exporting-sharing-reports/)). - -## Related docs - -- [Accessibility Score](/support/docs/accessibility-web-score/) -- [Exporting & Sharing Reports](/support/docs/accessibility-exporting-sharing-reports/) diff --git a/static/docs/accessibility-pdf-accessibility-scanning.md b/static/docs/accessibility-pdf-accessibility-scanning.md deleted file mode 100644 index 35501c8aa..000000000 --- a/static/docs/accessibility-pdf-accessibility-scanning.md +++ /dev/null @@ -1,28 +0,0 @@ -# PDF Accessibility Scanning - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Use this page as the entry point for PDF-related Accessibility guidance and to understand how PDF workflows differ from standard web or mobile Accessibility testing. - -As you read on, you will see where document scanning sits in the broader Accessibility documentation. You will also learn what to verify before treating PDF results as equivalent to DOM-based web testing, because document flows follow different constraints than page markup. - -## When to use this - -Use this page when your team needs to evaluate document accessibility separately from standard DOM-based web accessibility flows. - -## Important note - -PDF accessibility should be treated as its own document-focused workflow. Do not assume that web Accessibility findings automatically cover PDF content. - -## Onboarding checklist (PDF workflow) - -1. **Confirm scope** with legal or design: which PDFs are in scope (marketing, invoices, user manuals)? -2. **Upload or target** the PDF through the product flow your account supports for document scanning (menus vary by release). -3. **Choose WCAG / PDF-UA expectations** aligned with your policy. Note that automated checks still require **manual reading order and tag inspection** for high-stakes filings. -4. **Review findings** in the same dashboard patterns as web reports where applicable; export evidence for audits ([exporting](/support/docs/accessibility-exporting-sharing-reports/)). -5. **Track remediation** in your CMS or document owner tool; re-upload after fixes and re-scan. - -## Related docs - -- [Accessibility Compliance Guide (ADA / WCAG / EAA / 508)](/support/docs/accessibility-compliance-guide/) -- [Web Accessibility Coverage](/support/docs/accessibility-web-what-we-cover/) diff --git a/static/docs/accessibility-report-bug.md b/static/docs/accessibility-report-bug.md deleted file mode 100644 index f6dbde6d8..000000000 --- a/static/docs/accessibility-report-bug.md +++ /dev/null @@ -1,34 +0,0 @@ -# Bug Report - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Use Bug Report to convert Accessibility findings into actionable engineering work. - -Bug Report turns confirmed accessibility findings into engineering work items inside the issue trackers your team already trusts. This page explains when raising a bug from a report is appropriate and how that action extends remediation beyond the dashboard into delivery rituals. Use it when you want a repeatable handoff from accessibility evidence to tracked fixes. - -## Typical flow - -1. Open the report. -2. Review the issue or issue group. -3. Use the report action to create a bug. -4. Route it into the team's issue-management workflow. - -## Best use cases - -- confirmed issues on real user flows -- high-severity findings that need engineering follow-up -- tracked remediation work tied to releases or audits - -## Onboarding: file a bug from a report - -1. Open **[All Issues](/support/docs/accessibility-testing-dashboard-all-issues/)** and select the finding you want tracked. -2. Choose **Report bug** (or equivalent) from the row or detail actions. -3. Confirm the **title** auto-fills from the rule or element; edit so engineers recognize the user impact. -4. Attach **screenshots or HAR** if the dialog offers uploads (speeds reproduction). -5. Pick the **integration project** (Jira/Azure/etc.) if connected via **[Integrations](/support/docs/accessibility-report-integrations/)**; otherwise copy the deep link into your tracker manually. -6. After the engineering fix ships, **re-run** the scan and link the new build to the same ticket for closure. - -## Related docs - -- [Integrations (JIRA / Slack / GitHub)](/support/docs/accessibility-report-integrations/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-report-integrations.md b/static/docs/accessibility-report-integrations.md deleted file mode 100644 index 177c4b356..000000000 --- a/static/docs/accessibility-report-integrations.md +++ /dev/null @@ -1,32 +0,0 @@ -# Integrations (JIRA / Slack) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Accessibility reporting often needs to move beyond the dashboard into issue tracking and team collaboration workflows. - -This guide explains when to use shared platform integrations alongside Accessibility reporting and how they fit into remediation workflows such as ticketing and team notifications. It focuses on how those capabilities support accessibility outcomes rather than duplicating full integration setup documentation elsewhere. - -## Common integration scenarios - -- create engineering tickets from report findings -- share report outcomes with QA or product teams -- connect Accessibility triage to an existing bug-management workflow - -## Important note - -Many integrations are shared platform capabilities. This page exists to explain how they are used in the Accessibility context rather than to duplicate full integration setup docs. - -## Onboarding: wire Jira or Slack for Accessibility - -1. Complete the **global** integration setup first. See **[Jira Integration](/support/docs/jira-integration/)** or **[Slack Integration](/support/docs/slack-integration/)** for credentials, OAuth, and workspace allowlists. -2. In the **Accessibility** report or workspace settings (exact menu depends on UI version), choose the integration you want to **receive** new issues or notifications. -3. Map **severity or tags** to Jira priorities if the UI offers mapping; otherwise agree a default priority in your triage playbook. -4. Send a **test ticket** from a non-production report to confirm fields populate correctly. -5. Train reviewers to use **[Bug Report](/support/docs/accessibility-report-bug/)** for one-off escalations versus integration rules for systematic routing. - -## Related docs - -- [Bug Report](/support/docs/accessibility-report-bug/) -- [Exporting & Sharing Reports](/support/docs/accessibility-exporting-sharing-reports/) -- [Jira Integration](/support/docs/jira-integration/) -- [Slack Integration](/support/docs/slack-integration/) diff --git a/static/docs/accessibility-robot-framework-test.md b/static/docs/accessibility-robot-framework-test.md deleted file mode 100644 index 3c3a97075..000000000 --- a/static/docs/accessibility-robot-framework-test.md +++ /dev/null @@ -1,71 +0,0 @@ -# Robot Framework - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Robot Framework sits **above** Selenium: Accessibility is still configured on the **underlying browser session** (desired capabilities passed into `Open Browser` or your library’s remote configuration). This page is the onboarding path for teams using **SeleniumLibrary** (or equivalent) against the TestMu AI grid. - -> **Browsers:** Use **Chrome or Edge** with supported versions for Accessibility Automation. - -## Prerequisites - -- Robot Framework + SeleniumLibrary (or compatible library) installed -- Remote URL and credentials for TestMu AI -- Accessibility enabled for your workspace - -## Onboarding path - -### 1. Encode capabilities as variables - -Define suite or global variables so every test uses the same grid options: - -```robot -*** Variables *** -${LT_OPTIONS} {"accessibility": true, "accessibility.wcagVersion": "wcag21aa"} -``` - -Exact syntax depends on how you merge JSON into capabilities for your `Open Browser` keyword. Some teams use a **custom keyword** that builds the options dict in Python and passes it to `Create Dictionary` / `Evaluate`. - -### 2. Open Browser with Accessibility on - -Pass the merged capabilities into `Open Browser` (or your wrapper) so the remote session includes `"accessibility": true`. Match the pattern you already use for `browserName`, `platformName`, and auth. - -### 3. On-demand scan with Execute Javascript - -After the page is ready: - -```robot -Execute Javascript return document.readyState -Execute Javascript lambda-accessibility-scan -``` - -Use the second line only when you are **not** using `accessibility.autoscan`. - -### 4. Auto-scan alternative - -If you prefer scans on every navigation without Robot keywords: - -```robot -# In capabilities JSON / dict -accessibility.autoscan ${True} -``` - -### 5. Run the suite and review reports - -```bash -robot --outputdir results tests/ -``` - -Then open the [Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard) for the session and the **Accessibility** tab. - -## Troubleshooting - -| Symptom | What to check | -|--------|----------------| -| No report | Robot must forward `accessibility` to the remote session; inspect the session capabilities in the dashboard. | -| Hook errors | Ensure `Execute Javascript` runs in browser context and spelling is exactly `lambda-accessibility-scan`. | - -## Related docs - -- [Accessibility Automation (Overview)](/support/docs/accessibility-automation/) -- [Selenium](/support/docs/accessibility-automation-test/) -- [Configure Accessibility Automation](/support/docs/accessibility-automation-settings/) diff --git a/static/docs/accessibility-rules-checklist.md b/static/docs/accessibility-rules-checklist.md deleted file mode 100644 index 979deee8f..000000000 --- a/static/docs/accessibility-rules-checklist.md +++ /dev/null @@ -1,3 +0,0 @@ -# Accessibility Rules & Checklist - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). diff --git a/static/docs/accessibility-settings-ios.md b/static/docs/accessibility-settings-ios.md deleted file mode 100644 index 558ecd377..000000000 --- a/static/docs/accessibility-settings-ios.md +++ /dev/null @@ -1,41 +0,0 @@ -# Accessibility Testing on iOS Devices - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Accessibility testing ensures that mobile applications and web content are accessible to all users, including those with disabilities. On iOS devices, this involves testing various accessibility features like text size adjustments, contrast settings, and motion reduction. - -This document provides an overview of the accessibility features available on iOS devices within TestMu AI's Real Devices Cloud. With support for key features like text size adjustments, contrast settings, motion reduction, and more, you can ensure your applications are optimized for users with diverse accessibility needs. - -## Accessibility Settings -iOS devices offer a range of accessibility features that can be enabled or disabled during setup and customized through the settings app, as outlined in [Apple’s documentation](https://support.apple.com/en-in/guide/iphone/iph3e2e4367/ios). - -Both Real Devices App and Browser testing now support these features to enable accessibility testing of your websites and applications. - -## Accessing iOS accessibility settings - -**Step 1 :** Login to your TestMu AI account and navigate to **App Testing/Browser Testing** under Real Devices from the sidebar. - -**Step 2 :** Select the device-browser combination to start your test session. - -**Step 3 :** On the testing toolbar, click on the **iOS Settings** > **Accessibility**. - -- On the **iOS Settings** > **Accessibility** prompt, select **Display and Text Size**. -- Now, you can enable or disable the required display settings by clicking on the checkboxes, and use the slider to adjust the text size. -- Click on the Update button. A prompt will appear on the dashboard stating "**Accessibility Settings updated successfully**". - -#### Key Features: -- **Increase Contrast :** Enabling this option enhances the contrast of UI elements against the background, making text and interface components easier to see. -- **Smart Invert :** This option reverses colors on the display, except for media and images. It's designed for users who prefer darker backgrounds without inverting every color on the screen. -- **Larger Accessibility Settings :** When enabled, this option adjusts interface elements to be larger and more accessible without affecting the core layout of the device’s UI. -- **Text Size Adjustment :** A slider is provided to adjust the size of the text throughout the system. Users can move the slider to the right for larger text and to the left for smaller text. This feature benefits users who need larger fonts for readability across apps and system interfaces. - -- On the **iOS Settings** > **Accessibility** prompt, select **Reduce Motion**. -- Toggle the button to minimize dynamic animations and motion effect. -- Click on the Update button. A prompt will appear on the dashboard stating "**Accessibility Settings updated successfully**". - -#### Key Features: -- **Reduce Motion of User Interface :** This toggle reduces animations throughout the system, including the parallax effect of icons on the Home Screen and in-app transitions. -- **Parallax Effect Adjustment :** Disabling parallax can create a more stable and comfortable experience, particularly for users prone to motion sickness. - -## Conclusion -These accessibility settings allow iOS users to customize their device’s interface to match individual needs, improving usability and comfort. Whether adjusting display, text size or reducing motion. diff --git a/static/docs/accessibility-sitemap-extraction-url-import.md b/static/docs/accessibility-sitemap-extraction-url-import.md deleted file mode 100644 index 75bd9875a..000000000 --- a/static/docs/accessibility-sitemap-extraction-url-import.md +++ /dev/null @@ -1,35 +0,0 @@ -# Sitemap Extraction & URL Import - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Use sitemap extraction and URL import to build the page list for scheduled Accessibility scans efficiently. - -## When to use this - -Use this page when you want to scan many URLs and do not want to add them one by one. - -## Common input methods - -- manual URL entry -- CSV upload -- sitemap extraction - -## Onboarding: seed URLs for a new schedule - -1. Open **[Test Scheduling](/support/docs/accessibility-test-scheduling/)** and start **[Create Scheduled Scan](/support/docs/accessibility-test-scheduling-scan/)** (or edit an existing schedule). -2. Choose **manual**, **CSV**, or **sitemap** import depending on where your canonical URL list lives. -3. For **sitemap**, supply the sitemap URL (often `https://example.com/sitemap.xml`). Wait for extraction to finish, then **review** the list: remove admin-only, duplicate, or non-HTML URLs. -4. For **CSV**, align columns with the template your team uses (path, locale, or environment columns if applicable). Validate in a spreadsheet before upload. -5. For **manual** entry, paste critical journeys first (login, checkout, settings), then broaden coverage. -6. Save the URL set, then continue with authentication and schedule steps in the scheduling wizard. - -## Best practices - -- include core user-journey pages first -- review extracted URLs before scheduling large recurring scans -- exclude pages that are not meaningful for Accessibility validation - -## Related docs - -- [Test Scheduling - Sitemap (Overview)](/support/docs/accessibility-test-scheduling/) -- [Steps to Schedule an Accessibility Scan](/support/docs/accessibility-test-scheduling-scan/) diff --git a/static/docs/accessibility-supported-wcag-browsers.md b/static/docs/accessibility-supported-wcag-browsers.md deleted file mode 100644 index 3ccd6af54..000000000 --- a/static/docs/accessibility-supported-wcag-browsers.md +++ /dev/null @@ -1,30 +0,0 @@ -# Supported WCAG Versions & Browsers - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Use this page as a reference for supported WCAG version selection and browser-related considerations across Accessibility workflows. - -## Supported WCAG coverage - -Current Accessibility docs reference support for these WCAG targets: - -- WCAG 2.0 A -- WCAG 2.0 AA -- WCAG 2.1 A -- WCAG 2.1 AA -- WCAG 2.1 AAA -- WCAG 2.2 A -- WCAG 2.2 AA - -## Browser considerations - -Accessibility web workflows are optimized for Chromium-based browsers. Using current supported browser versions improves compatibility for DevTools and automation-related scans. - -## Important caveat - -For Playwright Accessibility Automation, Chrome should be used instead of `pw-chromium` because the required extension does not load reliably in bundled Chromium sessions. - -## Related docs - -- [Accessibility Automation (Overview)](/support/docs/accessibility-automation/) -- [Accessibility FAQ](/support/docs/accessibility-faq/) diff --git a/static/docs/accessibility-tag-support.md b/static/docs/accessibility-tag-support.md deleted file mode 100644 index b05f5da6a..000000000 --- a/static/docs/accessibility-tag-support.md +++ /dev/null @@ -1,154 +0,0 @@ -# Tag Support for Accessibility Scans - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Use tags to organize accessibility scans on mobile devices by screen, flow, or activity. This helps you label scans with names like `Login Screen`, `Cart Screen`, or `Profile Screen` so your team can quickly understand what each scan represents. - -You can use tags in two ways: - -- **From the dashboard UI** after a scan has already been created. -- **During automation** when you trigger `lambda-accessibility-scan` in your mobile accessibility automation test. - -## Before You Start - -Make sure you have: - -- Access to TestMu AI Accessibility for mobile-device testing. -- An accessibility scan result from a mobile-device session available in the dashboard, or -- An Appium test with accessibility enabled in any supported Appium flow for mobile accessibility automation. - -Tag support is available for accessibility scans that run on mobile devices. Tags are optional. If you do not add a tag, your scans continue to work as usual. - -## What Tags Help You Do - -- Label scans with meaningful names instead of relying only on generic view names. -- Group related issues by screen or activity. -- Filter the dashboard to focus on scans with selected tags. -- Keep reports easier to review across multi-screen app journeys. - -## Option 1: Add Tags from the Dashboard - -Use this flow when the scan is already available in the dashboard and you want to label it afterward. - -### Step 1: Open the tag action for a view - -In the issue view, open the scan or view you want to label and click the tag icon. - -### Step 2: Assign one or more tags - -In the **Assign Tags** modal, search for an existing tag or create a new one, then save your selection. - -You can assign multiple tags to the same scan from the dashboard. - -### Step 3: Review tagged issues - -Once saved, the selected tags appear with the affected issue details. - -### Step 4: Filter the dashboard by tag - -Use the **Tags** filter to show only the issues or views that match the selected tags. - -### Step 5: Use tags in Mobile View - -The same tags are also available when you review the test in **Mobile View**. - -### Step 6: Switch how issues are grouped - -In the **All Issues** tab, you can group the results in two ways: - -- **Group by: Views** to review issues screen by screen. -- **Group by: Issue Types** to review issues by accessibility problem type. - -**Group by: Views** - -**Group by: Issue Types** - -## Option 2: Add Tags During Automation - -Use this flow when you want to attach tags at the exact moment you trigger an accessibility scan during automation on a mobile device. - -This is useful when your test moves across multiple screens and you want each scan to carry its own label from the beginning. - -### Basic Syntax - -Trigger the scan with `lambda-accessibility-scan` and pass one or more tags. - -Tag support is available in all supported Appium flows for mobile accessibility automation. The exact method name depends on your client library, but the payload pattern stays the same: call `lambda-accessibility-scan` and pass a `tags` list. - -```java -((JavascriptExecutor) driver).executeScript( -"lambda-accessibility-scan", -new HashMap() {{ -put("tags", Arrays.asList("Login Screen", "Smoke")); -}} -); -``` - -```python -driver.execute_script( -"lambda-accessibility-scan", -{ -"tags": ["Login Screen", "Smoke"] -} -) -``` - -```javascript -await driver.execute("lambda-accessibility-scan", [{ -tags: ["Login Screen", "Smoke"] -}]); -``` - -### Example: Tag scans for multiple screens - -You can call the scan multiple times in the same test and use different tags for each scan. - -```java -((JavascriptExecutor) driver).executeScript( -"lambda-accessibility-scan", -new HashMap() {{ -put("tags", Arrays.asList("Login Screen", "Smoke")); -}} -); - -((JavascriptExecutor) driver).executeScript( -"lambda-accessibility-scan", -new HashMap() {{ -put("tags", Arrays.asList("Cart Screen", "Checkout Flow")); -}} -); - -((JavascriptExecutor) driver).executeScript( -"lambda-accessibility-scan", -new HashMap() {{ -put("tags", Arrays.asList("Profile Screen")); -}} -); -``` - -## Best Practices - -- Use short, meaningful names such as `Login Screen`, `Search`, `Checkout`, or `Settings`. -- Keep naming consistent across runs so filters and comparisons are easier to use. -- Use one naming style across your team, for example screen names or feature-flow names. -- Keep each tag within **50 characters**. -- Use only letters, numbers, spaces, hyphens, and underscores. -- Keep each scan to **20 tags** or fewer. -- Add tags during automation when you want scans to be labeled from the start, and use the dashboard flow when you want to organize existing scans afterward. - -## Troubleshooting - -- Tag support is currently available only for accessibility scans that run on **mobile devices**. -- You can use it while testing **mobile apps** and **web apps on mobile devices**. -- Tags help you organize scans, but they do not change scan behavior or issue detection. -- If you do not add tags during automation, you can still assign them later from the dashboard. -- During automation, each `lambda-accessibility-scan` call can include one or more tags in all supported Appium flows. -- From the dashboard, you can assign multiple tags to the same scan. -- If tags do not appear where expected, confirm the scan was run on a mobile-device session and that accessibility scanning is enabled for that workflow. - -## Related Docs - -- [Native App Accessibility Automation](/support/docs/accessibility-native-app-automation-test/) -- [Accessibility App Scanner](/support/docs/accessibility-app-scanner/) -- [Navigating Dashboard](/support/docs/accessibility-testing-navigating-dashboard/) -- [All Issues](/support/docs/accessibility-testing-dashboard-all-issues/) diff --git a/static/docs/accessibility-talkback-android.md b/static/docs/accessibility-talkback-android.md deleted file mode 100644 index 18121ddc0..000000000 --- a/static/docs/accessibility-talkback-android.md +++ /dev/null @@ -1,37 +0,0 @@ -# TalkBack on Android - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Use this page when validating **native or hybrid Android** experiences with **TalkBack**, especially after **App Scanner** or **Appium** accessibility runs. - -## When to use this - -Use TalkBack when validating Android accessibility behavior on mobile devices, especially for focus movement, labels, gestures, and control states. - -## Onboarding: turn TalkBack on safely - -1. On the device, open **Settings → Accessibility → TalkBack** (path varies by OEM) and toggle **On**. -2. Complete the **tutorial** gesture set (swipe right then up for default navigation) in a safe test build first. -3. Enable **Developer options → Show layout bounds** only if your policy allows, helps compare focus rectangles with visuals. -4. Open your app under test; swipe **right** to move to the next focusable element, **double-tap** to activate. -5. Use **local context menu** (swipe up then right) to jump by headings or controls when available. - -## What to validate - -- accessible labels on controls and media -- gesture and focus movement between interactive elements -- state changes for toggles, selections, and dynamic content -- screen-by-screen mobile usability - -## Short manual matrix (pair with App Scanner) - -| Flow | TalkBack checks | -|------|-----------------| -| Onboarding | Each carousel slide exposes a name; buttons not only icons. | -| Forms | Error text is reachable and associated with fields. | -| Media | Video controls expose play/pause state. | - -## Related docs - -- [Screen Reader](/support/docs/screen-reader-on-accessibility/) -- [Accessibility App Scanner (Overview)](/support/docs/accessibility-app-scanner/) diff --git a/static/docs/accessibility-test-crawler.md b/static/docs/accessibility-test-crawler.md deleted file mode 100644 index 55416d9a2..000000000 --- a/static/docs/accessibility-test-crawler.md +++ /dev/null @@ -1,51 +0,0 @@ -# Accessibility Crawler - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Why TestMu AI Crawler is Helpful for Accessibility Testing - -In accessibility testing, ensuring that all relevant URLs across a website are properly tested is crucial. Manual URL entry can be cumbersome and inefficient, especially for large websites with many pages. This is where TestMu AI’s URL Crawler comes into play, offering an automated and streamlined solution for accessibility testing. - -With the TestMu AI Crawler, you can: - -- **Automatically extract URLs** from your site based on set parameters like crawl depth and maximum pages to be crawled. -- **Scan large websites efficiently** without having to manually input each URL, saving time and resources. -- **Crawl dynamically** generated content and sub-pages that might otherwise be missed in traditional testing methods. -- **Ensure thorough coverage** by including/excluding specific URL patterns, which helps in targeting exactly the content you need to test for accessibility compliance. - -By automating the URL discovery process, the TestMu AI Crawler allows you to focus on accessibility issues without missing any pages that might contain potential accessibility barriers. - -## Step 1: Access the URL Crawler Feature -- Open the TestMu AI Accessibility Testing Dashboard. -- Select the “Crawler” option from the “Add URLs” section. - -## Step 2: Configure Crawler Settings -- **Base URL:** Enter the domain from which you want to start crawling. This is the root URL for the crawler. -- **Crawl Depth:** Define how deep the crawler should go to fetch additional pages. For example, a depth of 2 will crawl the base URL and its immediate sub-pages. -- **Max Pages:** Set a limit on the maximum number of pages the crawler should scan. This is useful to prevent unnecessary resource consumption. - -## Step 3: Define URL Patterns (Optional) -You can fine-tune your URL crawling by including or excluding certain URLs based on patterns. - -- Enter regular expressions (regex) to include only certain URLs. For example, you can filter by page types, such as `.*\.html` to include only `.html` pages. -- Multiple patterns can be entered, separated by commas. - -- Similarly, enter regex patterns to exclude specific URLs. For instance, you can use `.*\.pdf` to exclude PDF files from being crawled. -- Multiple exclude patterns can be entered, separated by commas. - -## Step 4: Analyze the Scan Report -Once the scan is complete, you can analyze the scan report, which includes: -- **Total Issues:** A graph summarizing the detected accessibility issues. -- **Severity Trends:** Categorized by severity (Critical, Serious, Moderate, Minor). -- **Pages Scanned:** View details of each URL crawled, along with a breakdown of issues per URL. - -You can drill down into individual URLs for a more detailed analysis of accessibility issues. - -> Note: The crawler functionality is available only when creating a new scheduled scan, not for scans that have already been scheduled. - -## Related docs - -- [Test Scheduling - Sitemap (Overview)](/support/docs/accessibility-test-scheduling/) -- [Login & Authentication for Scheduled Scans](/support/docs/accessibility-test-scheduling-login-authentication/) -- [Create Scheduled Scan](/support/docs/accessibility-test-scheduling-scan/) -- [Sitemap Extraction & URL Import](/support/docs/accessibility-sitemap-extraction-url-import/) diff --git a/static/docs/accessibility-test-scheduling-edit.md b/static/docs/accessibility-test-scheduling-edit.md deleted file mode 100644 index 6098a5d7d..000000000 --- a/static/docs/accessibility-test-scheduling-edit.md +++ /dev/null @@ -1,25 +0,0 @@ -# Edit an Accessibility Scan - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The Edit Report feature in Accessibility Scheduling allows you to modify scan settings for your scheduled accessibility tests. This document provides a step-by-step guide to update the scan name, recurrence details, and time for accessibility reports. - -## Accessing the Edit Report Feature -- Navigate to the Accessibility section on your TestMu AI dashboard. -- Select the report you want to edit from the Scheduled Tests list. -- Click on the Edit button located at the top right corner of the report. - -## Editing the Scan Details -Once you click on the Edit button, a dialog box will appear with the following options: - -### 1. Scan Name -- You can update the name of the scan to better reflect its purpose. -- **Example :** Change the name from `Monday, Tuesday, Friday Report for YT Platform` to a more descriptive title if needed. - -### 2. Recurring Details -- **Modify the days** on which the scan will run: Select the days from the dropdown (e.g., Monday, Tuesday, Friday). -- **Update the time** for the recurring scan: Specify the time (e.g., 12:00 PM) and time zone (e.g., GMT +05:30 Asia/Calcutta). - -After making the necessary updates, click Save changes to apply the modifications. If you wish to discard changes, click Cancel. - -> Once the changes are saved, the updated scan name and schedule will reflect in the Scheduled Tests section. The next scan on date and time will be automatically adjusted based on the new recurrence settings. diff --git a/static/docs/accessibility-test-scheduling-login-authentication.md b/static/docs/accessibility-test-scheduling-login-authentication.md deleted file mode 100644 index 263e24bd9..000000000 --- a/static/docs/accessibility-test-scheduling-login-authentication.md +++ /dev/null @@ -1,149 +0,0 @@ -# Login & Authentication for Scheduled Scans - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -**Login configurations** are saved profiles the scanner runs **before** a scheduled accessibility test. You define credentials and selectors once, then **select** a profile from the **login modal** whenever you create or edit a scan. The same modal opens from the **scheduler** (advanced options on the URL step) or from **Login configurations** on the dashboard, so you can manage profiles without starting a new scan. - -Treat credentials as sensitive: prefer dedicated QA or read-only accounts, rotate passwords on your usual cadence, and follow your organization’s access and secrets policy. - -## When to use a login configuration - -Use a saved login when: - -- **Target URLs** redirect to a sign-in page or hide content until the user is authenticated. -- **Recurring runs** should reuse the same sign-in flow without re-entering details in the wizard each time. -- Your app uses **HTTP Basic authentication**, a **single-page HTML form**, or a **username-then-password** flow that maps to **Basic**, **Form**, or **Multi-page** below. - -If the site is not reachable from the public internet, combine login with **local testing / tunnel** options in the schedule wizard where your plan supports them. - -## Open the login modal - -### From the scheduler (create or edit) - -1. In **Create scan** or **Edit scan**, go to the step where you **add URLs** (manual URLs, CSV, sitemap, or crawler, depending on your setup). -2. Expand **Advanced options**. -3. Under login settings, use **Add** next to **Add login configurations** to open the modal. - -### From the dashboard - -1. Open **Login configurations** from the Accessibility (or Web Scanner scheduling) area of the dashboard, as labeled in your product. -2. The **login modal** opens. You can review existing profiles or add a new one. - -## Create a configuration - -In the modal: - -1. Choose **New configuration** (or equivalent) to open the editor. -2. Enter a **configuration name** you will recognize in the list (for example `Staging Microsoft login`). -3. Select **Authentication type**: - - **Basic**: HTTP Basic auth (no HTML form). - - **Form**: Username and password on **one** page. - - **Multi-page**: Username step, then **Next**, then password (same or different URL). - -4. Fill the fields for that type (see tables below) and **Save**. - -The new profile appears in the list immediately. To use it on a schedule, return to the scan wizard, open **Add login configurations** again, and **select** that profile before saving the scan. - -Confirm every CSS selector in Chrome (or Edge) DevTools on the real login page: inspect the control, copy a stable selector (`id`, `name`, `data-testid`, or a short path), then paste it into the form. - -## Field reference - -### Basic authentication - -Use when the server responds with **HTTP Basic** authentication (browser-style username/password challenge), not when you only have a custom HTML login page. In that case use **Form** or **Multi-page**. - -| Field | Required | Description | -|--------|----------|-------------| -| Login Page URL | Yes | Entry URL for the protected experience (for example `https://www.yourwebsite.com/login`). | -| Username | Yes | Username sent with Basic auth. | -| Password | Yes | Password sent with Basic auth. | - -### Form authentication - -Use when, after opening the **Login Page URL**, the username field, password field, and submit control are all on the **same** document. - -| Field | Required | Description | -|--------|----------|-------------| -| Login Page URL | Yes | URL where the sign-in form appears. | -| Username | Yes | Text entered into the username field. | -| Username CSS Selector | Yes | CSS selector for the username input. | -| Password | Yes | Text entered into the password field. | -| Password CSS Selector | Yes | CSS selector for the password input. | -| Login Button CSS Selector | Yes | CSS selector for the control that submits the form (for example **Sign in**). | -| Post-login URL | No | If set, helps verify navigation to a URL that indicates a successful login. | - -### Multi-page authentication - -Use for **sequential** flows: enter the username, click **Next** (or equivalent), then enter the password, common with many enterprise identity providers. - -#### Step 1: Login or username page - -| Field | Required | Description | -|--------|----------|-------------| -| Login Page URL | Yes | First page of the flow (email or username step). | -| Username | Yes | Value entered on step 1. | -| Username CSS Selector | Yes | Selector for the username or email field. | -| Next Button CSS Selector | Yes | Selector for the control that advances to the password step. | - -#### Step 2: Password page - -| Field | Required | Description | -|--------|----------|-------------| -| Password Page URL | No | Use when the password step loads at a **different** URL; leave empty if only the DOM changes on the same URL. | -| Password | Yes | Password value. | -| Password CSS Selector | Yes | Selector for the password field. | -| Login Button CSS Selector | Yes | Selector for the control that completes sign-in. | - -#### Step 3: After login - -| Field | Required | Description | -|--------|----------|-------------| -| Post-login URL | No | Same idea as **Form** authentication: optional URL used to confirm the browser reached a logged-in state. | - -## Attach a configuration to a schedule - -1. In **Create** or **Edit scan**, on the URL step, expand **Advanced options** and open **Add login configurations**. -2. In the modal, pick the **saved configuration** for this scan. -3. Save the scan. Scheduled runs use that profile for authentication. - -You can reuse one configuration on many scans or keep separate profiles per environment (for example `Prod read-only` vs `Staging QA`). - -## Edit or remove configurations - -Open **Login configurations** from the dashboard (or the same modal from a scan), select the profile you need, and use the product actions to **edit** or **delete** it. Updating a profile affects any future runs that reference it; confirm downstream scans if you change selectors or URLs. - -## Best practices - -- **Stable selectors**: Prefer `id`, `name`, `data-testid`, or short attribute-based selectors over long positional paths. -- **Least privilege**: Use automation or QA accounts, not personal admin accounts. -- **Post-login URL**: Point at a route that only appears after a successful login so failures are easier to interpret. - -## Limitations - -These are difficult or unsupported with Basic / Form / Multi-page alone: - -| Situation | Why | -|-----------|-----| -| **OTP, SMS, authenticator, CAPTCHA** | Needs human interaction or a different integration model. | -| **OAuth-only** web flows | No password fields to drive with selectors. | -| **Intermittent risk / “Verify it’s you” prompts** | Runs may succeed or fail unpredictably. | - -Use staging environments with simplified auth, or other supported access paths, when possible. - -## Troubleshooting - -| Symptom | What to check | -|--------|----------------| -| Scan fails on the first URL | Correct profile selected; **Login Page URL** is the real start of the flow. | -| Timeout or “element not found” | Selectors, spelling, or UI that loads late. Verify in DevTools with throttling if needed. | -| Multi-page fails after username | **Next Button CSS Selector** targets the visible control; add **Password Page URL** if the path changes. | -| Works locally, fails on schedule | Cookie banners, geo restrictions, or MFA triggered for automated sessions. | - -## Related docs - -- [Steps to Schedule an Accessibility Scan](/support/docs/accessibility-test-scheduling-scan/) -- [Test Scheduling - Sitemap (Overview)](/support/docs/accessibility-test-scheduling/) -- [Edit an Accessibility Scan](/support/docs/accessibility-test-scheduling-edit/) -- [Crawler](/support/docs/accessibility-test-crawler/) -- [Sitemap Extraction & URL Import](/support/docs/accessibility-sitemap-extraction-url-import/) -- [Web Scanner Getting Started](/support/docs/web-scanner-getting-started/) diff --git a/static/docs/accessibility-test-scheduling-scan.md b/static/docs/accessibility-test-scheduling-scan.md deleted file mode 100644 index 69cf679c6..000000000 --- a/static/docs/accessibility-test-scheduling-scan.md +++ /dev/null @@ -1,50 +0,0 @@ -# Steps to Schedule an Accessibility Scan - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Step 1: Access the Scheduled Scan Feature -- Open the Accessibility Testing Dashboard. -- Click the “Scheduled Scan” option from the menu. - -## Step 2: Configure the Scan -- Enter the Scan Name for identification. -- Select the desired WCAG version for compliance. - -## Step 3. Add Pages for the Scan - -- Enter your domain URL or XML sitemap URL -- Click on the Fetch Sitemap button to extract all the URLs - -- In the Add Pages field, input individual URLs. -- Click Add for each URL. - -- Prepare a CSV file with the list of URLs. -- Click Upload CSV to import the file. - -### Advanced Options -- **Local Testing Setup (Tunnel) :** If scanning locally hosted URLs, add a Local Testing Setup/Tunnel to access non-public environments. -- **Login Configuration :** For pages behind authentication, use **Add login configurations** to open the login modal, pick a saved profile, or create one (**Basic**, **Form**, or **Multi-page**). You can also manage profiles from the dashboard **Login configurations** entry. See **[Login & Authentication for Scheduled Scans](/support/docs/accessibility-test-scheduling-login-authentication/)**. -- **Review Issues :** Enable Needs Review for areas requiring manual inspection. -- **Best Practices Issues :** Check Best Practices to include suggestions for better accessibility. - -## Step 4. Create the Scan -- Select between a Recurring scan (Recommended) or a Quick scan which allow you to perform an immediate scan of the provided URLs without scheduling. -- Specify the frequency (e.g., daily, weekly) and timing details. - -Once all details are configured, click Schedule Scan button. - -The **Run Initial Scan** checkbox, adds flexibility to the scheduling feature. If enabled: -- The scan will run immediately after configuration, regardless of the chosen recurring schedule. -- This is useful for verifying the setup and ensuring that the scan configuration works as intended. - -## Step 5: Access Reports -- Navigate to the Website Scanner section on the dashboard. -- Select the desired report from the Scan Summary list. - -## Step 6: Analyze the Scan Report -After selecting a scan, A detailed scan report will open, as shown in the provided screenshot. The report will display the following key components: -- **Total Issues :** A graph summarizing the trend of detected issues over the last few runs. -- **Severity Trend :** A chart categorizing issues by severity (e.g., Critical, Serious, Moderate, Minor). -- **Pages Scanned :** A list of scanned pages with their corresponding issue counts categorized by severity. - -You can drill down into individual page URLs to view the number of issues found on that specific page using filters provided. From the scan report page, you can also edit, share and stop recurring scan if its no longer required. diff --git a/static/docs/accessibility-test-scheduling.md b/static/docs/accessibility-test-scheduling.md deleted file mode 100644 index ddf3b4dc5..000000000 --- a/static/docs/accessibility-test-scheduling.md +++ /dev/null @@ -1,41 +0,0 @@ -# Test Scheduling - Sitemap (Overview) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Test Scheduling helps teams run recurring Accessibility scans on websites and web apps without manually starting a new scan every time. - -When you want recurring Accessibility scans outside of Web Scanner, this overview explains how the native scheduling flow fits the product, how it differs from Web Scanner, and where to go for setup, URL import, authentication, and recurring schedules. Use it to orient sitemap-based or crawler-driven workflows and ongoing visibility into accessibility drift. - -## Use this when - -- you want recurring scans -- you want to import or discover many URLs at once -- you need scheduled visibility into accessibility drift over time - -## Product boundary - -This page is for the Accessibility-native scheduling flow. If you use the separate **Web Scanner** product instead, start from [Getting Started with Web Scanner](/support/docs/web-scanner-getting-started/) or [Starting an Accessibility Scan with Web Scanner](/support/docs/web-scanner-accessibility-scan/). - -## Typical workflow - -1. Create the scheduled scan. -2. Add URLs manually, through CSV, or through sitemap extraction. -3. Add **saved login configurations** (or tunnel / local access) if required. See [Login & Authentication for Scheduled Scans](/support/docs/accessibility-test-scheduling-login-authentication/). -4. Set the schedule. -5. Review the resulting reports. - -## Onboarding: schedule your first recurring scan - -1. Read **[Steps to Schedule an Accessibility Scan](/support/docs/accessibility-test-scheduling-scan/)** end-to-end once. Menus use the same vocabulary as the live product. -2. Seed URLs using **[Sitemap Extraction & URL Import](/support/docs/accessibility-sitemap-extraction-url-import/)** when you have more than a handful of pages. -3. If pages sit behind login, open **Add login configurations** in Advanced options or use dashboard **Login configurations**, then attach a profile. See **[Login & Authentication for Scheduled Scans](/support/docs/accessibility-test-scheduling-login-authentication/)** (Basic, Form, Multi-page, or tunnel per your environment) before widening scope. -4. Start with a **weekly** cadence on a **subset** of URLs; promote to daily only after noise is manageable. -5. After the first run, open **[Navigating the Dashboard](/support/docs/accessibility-testing-navigating-dashboard/)** and confirm aggregates match expectations. -6. Optional: enable **[Fragment Identifier](/support/docs/accessibility-fragment-identifier/)** in DevTools settings when hash routes should split reporting for SPA teams (web URL grouping only). - -## Related docs - -- [Login & Authentication for Scheduled Scans](/support/docs/accessibility-test-scheduling-login-authentication/) -- [Steps to Schedule an Accessibility Scan](/support/docs/accessibility-test-scheduling-scan/) -- [Sitemap Extraction & URL Import](/support/docs/accessibility-sitemap-extraction-url-import/) -- [Crawler](/support/docs/accessibility-test-crawler/) diff --git a/static/docs/accessibility-testing-dashboard-all-issues.md b/static/docs/accessibility-testing-dashboard-all-issues.md deleted file mode 100644 index 718f6ce79..000000000 --- a/static/docs/accessibility-testing-dashboard-all-issues.md +++ /dev/null @@ -1,34 +0,0 @@ -# All Issues - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The All Issues view lists every issue found in a report and helps teams move from high-level summary to detailed triage. - -The rest of this page walks through using All Issues to inspect findings and interpret grouping. It also explains how to decide whether to fix, hide, or escalate each item so your triage flow stays consistent across reviewers. - -## What you can do here - -- inspect issue-level detail -- review needs-review findings -- understand which guideline each issue maps to -- switch grouping based on the report structure -- take actions such as hide, restore, export, or report a bug - -## When to use this - -Use this view when you need issue-level depth rather than summary statistics. - -## How to triage in All Issues (onboarding) - -1. Open a report from the dashboard, then choose **All Issues**. -2. **Sort or filter** by severity, rule, URL, or state (available controls depend on scan type). -3. Click an issue to read the **element context**, selector hints, and remediation text where the product provides it. -4. For noisy or accepted exceptions, use **[Hide and Restore Issues](/support/docs/accessibility-hide-restore-issues/)** so future runs respect your decision. Document the reason for auditors. -5. For engineering work, use **[Bug Report](/support/docs/accessibility-report-bug/)** or your **[Jira / Slack](/support/docs/accessibility-report-integrations/)** integration so the finding carries screenshots and deep links. -6. Re-run the same URL or flow after a fix and **compare** issue counts or use **[Passed Test Cases](/support/docs/accessibility-passed-test-cases/)** if enabled for a fuller before/after story. - -## Related docs - -- [Issue Summary](/support/docs/accessibility-testing-dashboard-issue-summary/) -- [Hide and Restore Issues](/support/docs/accessibility-hide-restore-issues/) -- [Bug Report](/support/docs/accessibility-report-bug/) diff --git a/static/docs/accessibility-testing-dashboard-issue-summary.md b/static/docs/accessibility-testing-dashboard-issue-summary.md deleted file mode 100644 index d7a6ebc18..000000000 --- a/static/docs/accessibility-testing-dashboard-issue-summary.md +++ /dev/null @@ -1,34 +0,0 @@ -# Issue Summary - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Issue Summary gives you the high-level breakdown of a completed Accessibility run before you move into issue-level triage. - -Read on for the summary metrics surfaced after each run. Each signal is meant to help you prioritize remediation before you drill into individual issues, so you can align fixes with severity and coverage first. - -## What you can review - -- total issues -- issue types -- affected element groups -- best-practice findings -- needs-review findings -- affected WCAG guidelines - -## When to use this - -Use this view when you want to understand report shape and severity before drilling into individual issues. - -## How to use Issue Summary (onboarding) - -1. From **[Navigating the Dashboard](/support/docs/accessibility-testing-navigating-dashboard/)**, open a completed report. -2. Select **Issue Summary** (or the equivalent tab/panel name in the UI). -3. Read **severity totals** first. Critical/serious items usually block users or compliance narratives. -4. Scan **guideline or rule groupings** to see which WCAG themes dominate (contrast, keyboard, forms, ARIA, etc.). -5. Use **needs-review** and **best-practice** counts to decide how much manual validation you owe before filing engineering tickets. -6. When a bucket looks actionable, jump to **[All Issues](/support/docs/accessibility-testing-dashboard-all-issues/)** filtered or grouped by that bucket, or open the **[Web Rule Repository](/support/docs/accessibility-web-rule-repository/)** entry for the rule text. - -## Related docs - -- [All Issues](/support/docs/accessibility-testing-dashboard-all-issues/) -- [Accessibility Score](/support/docs/accessibility-web-score/) diff --git a/static/docs/accessibility-testing-full-page-scanner.md b/static/docs/accessibility-testing-full-page-scanner.md deleted file mode 100644 index c4ddb9443..000000000 --- a/static/docs/accessibility-testing-full-page-scanner.md +++ /dev/null @@ -1,29 +0,0 @@ -# Full Page Scanner - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Full Page Scanner is used to analyze the accessibility of entire web pages at once. It goes beyond individual element checks and provides a comprehensive overview of potential accessibility issues across the entire page content. - -## Functionalities of Full Page Scanner - -- **Scans Multiple elements :** It analyzes various aspects of your webpage, including HTML/CSS code, images, forms, multimedia, and interactive elements. -- **WCAG Compliance Checks :** It identifies potential violations against relevant WCAG guidelines (e.g., missing alt text, insufficient color contrast, keyboard navigation issues). -- **Automated Issue Detection :** It automatically highlights problems and provides detailed descriptions, pinpointing them directly on the scanned page. -- **Prioritization Options:** It prioritizes issues based on severity and potential impact, helping you focus on the most critical ones first. -- **Report Integration:** Results from a Full Page Scan are saved to the Accessibility dashboard for review, export, and integration with issue trackers. - -## Execute the Full Page Scanner for Your Website - -### Prerequisite - -- You have to [setup the Accessibility DevTools](/support/docs/accessibility-testing-install-devtools) in your browser. - -### Trigger the Accessibility DevTool - -- Go to the **Inspect** panel >> **TestMu AI Accessibility DevTools** of your required website. -- Click on the **Full Page Scan** button to start the scanning for **Accessibility Issue** for that particular page. - -### Review Your Issues - -- This will list down all of the issues after scanning your complete webpage. -- You can click on those issues to check with which particular element it is causing issue. diff --git a/static/docs/accessibility-testing-install-devtools.md b/static/docs/accessibility-testing-install-devtools.md deleted file mode 100644 index a663e036e..000000000 --- a/static/docs/accessibility-testing-install-devtools.md +++ /dev/null @@ -1,13 +0,0 @@ -# Install TestMu AI Accessibility Toolkit - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -**Step 1:** Go to the Accessibility Dashboard and click on the [**Download Plugin**](https://chromewebstore.google.com/detail/lambdatest-accessibility/mmbbmjhbidfflcbiffppojapgonepmab) button. - -**Step 2:** This will redirect you to the **Chrome Web Store** from where you can download the extension. - -- Click on **Add to Chrome** button to install the extension and pin the extension. - -**Step 3:** Go to the **Inspect** panel of your desired website. Switch to the **Accessibility DevTools** panel. - -- Login to your TestMu AI account. diff --git a/static/docs/accessibility-testing-multi-page-scanner.md b/static/docs/accessibility-testing-multi-page-scanner.md deleted file mode 100644 index bbed6bf25..000000000 --- a/static/docs/accessibility-testing-multi-page-scanner.md +++ /dev/null @@ -1,39 +0,0 @@ -# Multi Page Scanner - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The Multi Page Scanner analyzes accessibility across multiple webpages within a website or application, providing a comprehensive overview of potential issues. - -You can use Multi Page Scanner for: - -- Testing large websites or applications with multiple interconnected pages. -- Identifying common accessibility patterns and prioritizing fixes that impact the entire user experience. -- Combining with Full Page and Partial Page scans for targeted analysis of specific sections or pages. - -## Functionalities of Multi Page Scanner - -- Crawls your website based on user-defined settings like starting URL, crawl depth, and exclusion patterns. -- Performs accessibility checks on each scanned page using the same principles as the Full Page Scanner. -- Generates a consolidated report highlighting issues across all scanned pages, prioritizing them based on severity and impact. - -## Execute the Multi Page Scanner - -### Prerequisite - -- You have to [setup the Accessibility DevTools](/support/docs/accessibility-testing-install-devtools) in your browser. - -### Trigger the Accessibility DevTool - -- Go to the **Inspect** panel >> **TestMu AI Accessibility DevTools** of your required website. -- Click on the **Multi Page Scan** button to start the scanning for **Accessibility Issue** for that particular page. - -### Scan the Multiple Webpages - -- Simply add all the URLs of the webpages that you want to scan. -- Click on the **Start Scan** button. - -### Review Your Issues - -- This will list down all of the issues after scanning all the listed webpages. - -- You can also filter out your issues list by selecting any specific page. diff --git a/static/docs/accessibility-testing-navigating-dashboard.md b/static/docs/accessibility-testing-navigating-dashboard.md deleted file mode 100644 index 889f2125f..000000000 --- a/static/docs/accessibility-testing-navigating-dashboard.md +++ /dev/null @@ -1,39 +0,0 @@ -# Navigating the Dashboard - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The Accessibility dashboard is the main place to review completed scans, open detailed reports, and act on findings. - -Continue reading to understand the dashboard's purpose and how to navigate between reports. You will also see where to access issue-level views, exports, and supporting actions that close the loop after a scan completes. - -## What you can do here - -- open completed tests -- filter reports by scan type, user, and WCAG-related dimensions -- review issue summary and all issues -- export and share reports -- open related actions such as hide/restore and bug filing - -## Onboarding: first time in the dashboard - -1. **Sign in** to TestMu AI and open **Accessibility** from the product navigation (exact placement can vary by account layout). -2. **Locate the report list** for your workspace: completed DevTools sessions, automation builds, scheduled scans, or Web Scanner imports should each appear as rows or cards once processing has finished. -3. **Pick a report** by name, time, or URL. If nothing appears, confirm the upstream run finished (Automation dashboard for Selenium, or your scan wizard for scheduling/Web Scanner). -4. **Scan the header** for counts, severity mix, and (if shown) **Accessibility Score**—use this as a triage compass before drilling down. -5. Open **[Issue Summary](/support/docs/accessibility-testing-dashboard-issue-summary/)** for grouped insight, then **[All Issues](/support/docs/accessibility-testing-dashboard-all-issues/)** for line-item work. -6. Use **filters** (date, user, scan type, WCAG-related fields where available) to narrow large histories. -7. When you are ready to share evidence, use **[Exporting & Sharing Reports](/support/docs/accessibility-exporting-sharing-reports/)** or integrations from **[Integrations](/support/docs/accessibility-report-integrations/)**. - -## Typical dashboard workflow - -1. Open the completed report. -2. Review high-level stats and issue counts. -3. Move into Issue Summary or All Issues. -4. Take actions such as export, hide/restore, or bug reporting. - -## Related docs - -- [Issue Summary](/support/docs/accessibility-testing-dashboard-issue-summary/) -- [All Issues](/support/docs/accessibility-testing-dashboard-all-issues/) -- [Hide and Restore Issues](/support/docs/accessibility-hide-restore-issues/) -- [Exporting & Sharing Reports](/support/docs/accessibility-exporting-sharing-reports/) diff --git a/static/docs/accessibility-testing-partial-page-scanner.md b/static/docs/accessibility-testing-partial-page-scanner.md deleted file mode 100644 index 76823b6b0..000000000 --- a/static/docs/accessibility-testing-partial-page-scanner.md +++ /dev/null @@ -1,40 +0,0 @@ -# Partial Page Scanner - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -While the [Full Page Scanner](/support/docs/accessibility-testing-full-page-scanner/) analyzes an entire webpage, the Partial Page Scanner provides a more focused approach to accessibility evaluation. - -It analyzes specific regions or elements within a webpage instead of the entire page. It is Ideal for: - -- Testing specific components like forms, interactive elements, or embedded content. -- Investigating reported accessibility issues on particular sections of a webpage. -- Evaluating accessibility changes made to a specific area. - -## Functionalities of Partial Page Scanner - -- **Manual Selection :** Choose the specific area you want to scan by highlighting it on the webpage. -- **Targeted Analysis :** Focuses on the selected area and identifies potential accessibility issues within that region. -- **WCAG Compliance Checks :** Similar to the Full Page Scanner, it checks for violations against relevant WCAG guidelines within the chosen area. -- **Detailed Reporting :** Provides a report highlighting identified issues, descriptions, and WCAG references, specific to the scanned region. - -## Execute the Partial Page Scanner for Your Website - -### Prerequisite - -- You have to [setup the Accessibility DevTools](/support/docs/accessibility-testing-install-devtools) in your browser. - -### Trigger the Accessibility DevTool - -- Go to the **Inspect** panel >> **TestMu AI Accessibility DevTools** of your required website. -- Click on the **Partial Page Scan** button to start the scanning for **Accessibility Issue** for that particular section of the webpage. - -### Select Your Desired Section of the Webpage - -- To start the scan, you will have to select a particular section of your webpage for which you want the analysis report. -- You can select any particular section for scanning via two ways: - - **Element Selection :** You can select that particular element via naigating to that tag from the right panel. - - **Mouse Selection :** You will have to toggle the **Mouse Selection** to **true** and select that particular element by clicking on the webpage directly. - -- Click on **Start Scan** button. - - - It will scan that particular section and will list down the issues with that section (if found any). diff --git a/static/docs/accessibility-testing-run-quick-scan.md b/static/docs/accessibility-testing-run-quick-scan.md deleted file mode 100644 index 9fae1732a..000000000 --- a/static/docs/accessibility-testing-run-quick-scan.md +++ /dev/null @@ -1,27 +0,0 @@ -# Run Your Quick Scan - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Quick Scan is the fastest way to check a single page for accessibility issues using the DevTools extension. It scans the full page and returns a list of issues grouped by severity. - -Quick Scan performs the same check as a Full Page Scan. If you want details on what the scan covers and how issues are categorized, see [Full Page Scan](/support/docs/accessibility-testing-full-page-scanner/). - -## Prerequisite - -- You have [installed the Accessibility DevTools](/support/docs/accessibility-testing-install-devtools) in your browser. - -## Run the scan - -1. Open the target webpage. -2. Open **DevTools** (Inspect panel) and select the **TestMu AI Accessibility DevTools** tab. -3. Click the **Full Page Scan** button. - -## Review the results - -The scan returns a list of issues found on the page. Click any issue to see which element is affected and where it appears in the DOM. - -## Next steps - -- To scan only part of the page, see [Partial Page Scan](/support/docs/accessibility-testing-partial-page-scanner/). -- To scan across multiple pages, see [Multi-Page Scan](/support/docs/accessibility-testing-multi-page-scanner/). -- To configure WCAG version, best practices, and needs review, see [Configure Accessibility DevTools](/support/docs/accessibility-devtools-settings/). diff --git a/static/docs/accessibility-testing-workflow-scanner.md b/static/docs/accessibility-testing-workflow-scanner.md deleted file mode 100644 index 3cf9369b4..000000000 --- a/static/docs/accessibility-testing-workflow-scanner.md +++ /dev/null @@ -1,27 +0,0 @@ -# Workflow Scanner - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Workflow Scan allows you to record multiple real-time interactions and page loads within a user journey on a website and then analyze them for accessibility issues. This helps ensure that users with disabilities can easily navigate and interact with your website across different scenarios. - -## Functionalities of Workflow Scan: - -- **Identify real-world accessibility issues :** Goes beyond static page analysis and identifies problems users might encounter during interaction. -- **Test complex user flows :** Ensures accessibility throughout navigation, forms, and interactive elements. -- **Save time and effort :** Tests multiple pages at once without needing individual scans. -- **Prioritize issues based on usage :** Focuses on problems encountered in typical user journeys. - -## Execute the Workflow Scanner - -### Prerequisite - -- You have to [setup the Accessibility DevTools](/support/docs/accessibility-testing-install-devtools) in your browser. - -### Trigger the Accessibility DevTool - -- Go to the **Inspect** panel >> **TestMu AI Accessibility DevTools** of your required website. -- Click on the **Workflow Scan** button to start the scanning for **Accessibility Issue** for that particular page. - -### Review Your Issues - -- This will list down all of the issues after scanning all of your webpages that you have searched for. diff --git a/static/docs/accessibility-testing.md b/static/docs/accessibility-testing.md deleted file mode 100644 index e69d530fa..000000000 --- a/static/docs/accessibility-testing.md +++ /dev/null @@ -1,65 +0,0 @@ -# Getting Started with Accessibility Testing - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI Accessibility Testing helps teams find, understand, and fix accessibility issues across websites, web apps, and mobile apps. - -## What Accessibility Testing covers - -- **Manual Testing (DevTools)** -- browser-based scanning and assisted testing for websites and web apps. -- **Automation** -- automated accessibility checks in supported test frameworks (Selenium, Cypress, Playwright, and more). -- **Test Scheduling** -- recurring site scans driven by sitemaps, crawlers, or manual URL lists inside the Accessibility product (not the same doc set as the standalone Web Scanner product). -- **Web Scanner** -- the separate **Web Scanner** product in the docs sidebar, where you add URLs and run accessibility scans on that surface. Use Accessibility **Test Scheduling** when your workflow is native to Accessibility scheduling, not Web Scanner. -- **Mobile App Testing** -- manual and automated accessibility testing for native Android and iOS apps, including KaneAI-authored flows. -- **Reports** -- dashboard, issue views, [exporting and sharing](/support/docs/accessibility-exporting-sharing-reports/), passed tests, and bug reporting (issue-tracker connections live under **External integrations** in the sidebar, not inside Reports). -- **Integrations** (sidebar) -- **Integrations with TestMu AI products** (Web Scanner, KaneAI, HyperExecute with Selenium, and related surfaces) and **External integrations** (CI/CD, Jira/Slack) for how Accessibility connects to the rest of your toolchain. -- **Features** (sidebar: **Web**, then **Mobile**, then **Common**) -- **Web**: hide/restore, AI issue detection, capture screenshot, web score, fragment identifier, and PDF scanning; **Mobile**: scan tags; **Common**: remediation guidance and analytics widgets. -- **Checklists** -- coverage summaries, WCAG rule repositories, and manual verification guidance. -- **Accessibility MCP Server** -- AI-assisted accessibility analysis only. For installing and configuring the **platform-wide** TestMu AI MCP Server (all tools), use [TestMu AI MCP Server](/support/docs/testmu-mcp-server/) in the main docs sidebar. - -## Choose your path - -| If you want to... | Start here | -|-----|-----| -| Test a website manually | [Manual Testing (DevTools)](/support/docs/accessibility-devtools/) | -| Run your first browser scan quickly | [Run a Quick Scan](/support/docs/accessibility-testing-run-quick-scan/) | -| Run automated web tests | [Automation](/support/docs/accessibility-automation/) | -| Schedule recurring scans | [Test Scheduling](/support/docs/accessibility-test-scheduling/) | -| Use Web Scanner for scans | [Web Scanner](/support/docs/web-scanner-getting-started/) | -| Test a mobile app manually | [Manual App Scanner](/support/docs/accessibility-app-scanner/) | -| Run mobile automation | [Appium Automation](/support/docs/accessibility-native-app-automation-test/) | -| Author mobile flows in KaneAI | [KaneAI Mobile App Testing](/support/docs/kaneai-mobile-app-accessibility/) | -| View reports and issues | [Reports](/support/docs/accessibility-testing-navigating-dashboard/) | -| Connect CI/CD or Jira/Slack | [CI/CD Integration Guide](/support/docs/accessibility-cicd-integration-guide/) · [Integrations (Jira / Slack)](/support/docs/accessibility-report-integrations/) | -| Look up supported rules and gaps | [Web](/support/docs/accessibility-web-what-we-cover/) · [iOS](/support/docs/accessibility-ios-what-we-cover/) · [Android](/support/docs/accessibility-android-what-we-cover/) | -| Use AI-assisted analysis | [Accessibility MCP Server](/support/docs/accessibility-mcp-server/) | - -Not sure which tool fits? See [Choose Your Tool](/support/docs/accessibility-choosing-the-right-tool/) for a detailed comparison. - -## Quick setup - -Use this section when you want the fastest path from "I need to test accessibility" to "I have my first report." - -### Prerequisites - -- Access to the Accessibility feature for your account -- A website, web app, or mobile app you are authorized to test -- The correct browser, device, framework, or scan surface for your use case - -### First-run checklist - -1. Pick the right product surface from the table above. -2. Confirm whether your flow is manual, automated, scheduled, or integrated through Web Scanner or KaneAI. -3. Run the scan. -4. Open the report in the [dashboard](/support/docs/accessibility-testing-navigating-dashboard/). -5. Review issue counts, severity, and next actions. - -### Common mistakes to avoid - -- Using DevTools when you actually need scheduled or automated execution. -- Using Web Scanner docs for the Accessibility-native scheduling surface. -- Using KaneAI guidance when your team is running direct Appium automation. - -## Accessibility standards - -Accessibility Testing helps you work toward standards such as WCAG, ADA, EAA, and Section 508. Automated testing covers many rules, but it does not replace manual verification for every accessibility requirement. See the [Web](/support/docs/accessibility-web-what-we-cover/), [iOS](/support/docs/accessibility-ios-what-we-cover/), and [Android](/support/docs/accessibility-android-what-we-cover/) checklists for supported rules plus each platform’s **manual test checklist**, with links into each rule repository. diff --git a/static/docs/accessibility-testng-test.md b/static/docs/accessibility-testng-test.md deleted file mode 100644 index 1fe6942ce..000000000 --- a/static/docs/accessibility-testng-test.md +++ /dev/null @@ -1,91 +0,0 @@ -# TestNG - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Use this guide when your **Selenium** web tests run with **TestNG** and you want **Accessibility Automation** in the same sessions. The runner does not change how Accessibility works on the grid: you still enable it with **capabilities**, then either use **on-demand hooks** or **auto-scan**. Details match the main [Selenium Accessibility Automation](/support/docs/accessibility-automation-test/) doc; this page adds a **TestNG-shaped onboarding path**. - -> **Browsers:** Accessibility Automation is supported on **Chrome and Edge** (recent versions). Set `browserName` accordingly. - -## Prerequisites - -- TestMu AI **username and access key** ([environment variables](/support/docs/using-environment-variables-for-authentication-credentials/)) -- A **Selenium + TestNG** project already hitting the TestMu AI grid (hub URL and capabilities) -- **Accessibility** entitlement for your organization -- Access to the **Automation** dashboard and the **Accessibility** tab for the session build - -## Onboarding path (first successful run) - -### 1. Confirm your TestNG entry point - -Most teams initialize the driver in a **Base test class** (`@BeforeMethod` / `@BeforeClass`) or a small factory. You only need **one** place where `DesiredCapabilities` or `MutableCapabilities` is built so every test method gets Accessibility enabled. - -### 2. Add Accessibility capabilities - -Enable Accessibility and optional WCAG / best-practice flags (full reference: [Configure Accessibility Automation](/support/docs/accessibility-automation-settings/)). - -**Minimal enable:** - -```java -capabilities.setCapability("accessibility", true); -``` - -**Optional tuning (example):** - -```java -capabilities.setCapability("accessibility", true); -capabilities.setCapability("accessibility.wcagVersion", "wcag21aa"); -capabilities.setCapability("accessibility.bestPractice", false); -capabilities.setCapability("accessibility.needsReview", true); -``` - -### 3. Choose scan mode - -**A. On-demand (recommended for most suites)** -After navigation and when the page is stable, call: - -```java -driver.executeScript("lambda-accessibility-scan"); -``` - -If you enable `accessibility` but **never** call this hook (and do not use auto-scan), **no Accessibility report** is produced for that navigation. - -**B. Continuous auto-scan** -Scan on every navigation without hooks: - -```java -capabilities.setCapability("accessibility", true); -capabilities.setCapability("accessibility.autoscan", true); -``` - -### 4. Wire TestNG suite XML (optional but typical) - -Point your `testng.xml` (or Gradle/Maven TestNG config) at the packages or classes that use the shared base class so every included test inherits the same driver setup. Run a **single** `@Test` first to validate capabilities before scaling the suite. - -### 5. Execute and open the report - -```bash -mvn test -``` - -(or your TestNG CLI / IDE run configuration.) - -Then open the [Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard), select the session, and use the **Accessibility** tab for the generated report (same flow as the [Selenium guide](/support/docs/accessibility-automation-test/)). - -## What gets reported - -Reports use the **standard Accessibility automation pipeline**: issues, severity, WCAG mapping, and the same dashboard views as other Selenium-based runs. - -## Troubleshooting - -| Symptom | What to check | -|--------|----------------| -| No Accessibility tab / empty report | Confirm `accessibility: true` and that you either call `lambda-accessibility-scan` or set `accessibility.autoscan`. | -| Wrong WCAG level | Adjust `accessibility.wcagVersion` in [settings reference](/support/docs/accessibility-automation-settings/). | -| Flaky scans | Prefer **hooks** after explicit waits rather than autoscan on heavy SPAs. | - -## Related docs - -- [Selenium](/support/docs/accessibility-automation-test/) -- [Configure Accessibility Automation](/support/docs/accessibility-automation-settings/) -- [JUnit 5](/support/docs/accessibility-junit5-test/) (same model, different runner) -- [CI/CD Integration Guide](/support/docs/accessibility-cicd-integration-guide/) diff --git a/static/docs/accessibility-update-devtools-extension.md b/static/docs/accessibility-update-devtools-extension.md deleted file mode 100644 index 581fdaabf..000000000 --- a/static/docs/accessibility-update-devtools-extension.md +++ /dev/null @@ -1,28 +0,0 @@ -# Update your Accessibility DevTools Chrome Extension - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Keeping your TestMu AI Accessibility DevTools Chrome extension updated ensures you have the latest features, improvements, and security patches. Follow the detailed steps below to update your extension. - -## Step-by-Step Guide to Update the TestMu AI Accessibility DevTools Chrome Extension - -### Step 1: Open the Extensions Page -- Launch the Google Chrome browser on your computer. -- In the address bar, type `chrome://extensions/` and press Enter. -- This will open the Extensions page where all your installed Chrome extensions are listed. - -### Step 2: Enable Developer Mode -- On the Extensions page, look for the **Developer mode** toggle located at the top right corner of the page. -- Click on the **Developer mode** toggle to turn it on. When enabled, you will see additional options such as **Load unpacked, Pack extension, and Update**. - -### Step 3: Update Your Extensions -- Once the **Developer mode** is enabled, you will see an **Update** button appear at the top left corner of the Extensions page. -- Click on the **Update** button. - -### Step 4: Wait for the Update Process -- After clicking the **Update** button, Chrome will start checking for updates for all your installed extensions, including the TestMu AI Accessibility DevTools extension. - -This process may take a few moments. Please wait until you see a confirmation message or the progress indicator stops. - -### Step 5: Confirmation -Once the update process is complete, you will see a message confirming that your extensions have been updated. diff --git a/static/docs/accessibility-voiceover-macos.md b/static/docs/accessibility-voiceover-macos.md deleted file mode 100644 index a7c502bfb..000000000 --- a/static/docs/accessibility-voiceover-macos.md +++ /dev/null @@ -1,38 +0,0 @@ -# VoiceOver on macOS - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Use this page as the macOS-specific reference for **VoiceOver** when validating **Safari** or **Chrome** experiences after automated Accessibility findings. - -## When to use this - -Use VoiceOver on macOS when your team needs to understand how desktop macOS users experience navigation, labels, landmarks, and interactive content. - -## Onboarding: enable VoiceOver and browse the web - -1. Open **System Settings → Accessibility → VoiceOver** (names vary slightly by macOS version) and turn VoiceOver **On**, or press **Command + F5**. -2. Accept the quick-start tutorial the first time. It teaches the **VO** modifier (Control + Option by default). -3. Open **Safari**, load your URL, and use **VO + Right Arrow** to move by rotor setting (start with **DOM** or **auto** mode for web). -4. Practice the **rotor** (VO + U) to jump by headings, links, or form controls. -5. Use **Tab** and **Shift+Tab** alongside VoiceOver to confirm focus order matches visual order. - -## What to validate - -- heading and landmark structure -- control names and hints -- form labels and error messaging -- keyboard and screen-reader navigation consistency - -## 20-minute smoke checklist - -| Step | Action | -|------|--------| -| 1 | Landmarks: VO + U → Landmarks; ensure main/nav are sensible. | -| 2 | Headings: navigate by headings; levels should not skip arbitrarily. | -| 3 | Buttons/links: every interactive element speaks a name (not just “button”). | -| 4 | Forms: errors associate with fields; required state is announced. | - -## Related docs - -- [Screen Reader](/support/docs/screen-reader-on-accessibility/) -- [Accessibility DevTools (Overview)](/support/docs/accessibility-devtools/) diff --git a/static/docs/accessibility-vpat-report-generation.md b/static/docs/accessibility-vpat-report-generation.md deleted file mode 100644 index 93138bb3a..000000000 --- a/static/docs/accessibility-vpat-report-generation.md +++ /dev/null @@ -1,126 +0,0 @@ -# VPAT and ACR evidence (customer-owned templates) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -**TestMu AI does not provide VPAT or ACR deliverables** -**TestMu AI does not issue, host, or ship an official VPAT® document or Accessibility Conformance Report (ACR)** as a product output. There is **no** “download VPAT” or “generate VPAT” action in the platform that replaces your organization’s own procurement or accessibility documentation. - -What you **do** get from **Accessibility Testing** are **testing artifacts** (issues, severities, URLs or screens, exports, integrations) that **you** may choose to cite while **your team** completes the official **ITI VPAT** (or equivalent) and assigns **Conformance** levels. This page is only a **workflow guide** for that evidence work. - -A **Voluntary Product Accessibility Template (VPAT®)** is a structured way to document how a product meets accessibility criteria (commonly WCAG 2.x and/or Section 508 / EN 301 549, depending on the template version you use). Buyers and compliance teams use it in **procurement** and **audit** conversations. - -**TestMu AI Accessibility Testing** can speed up **evidence collection** (issue lists, severities, URLs or screens, remediation notes, and exports), but it does **not** auto-generate a finished VPAT or ACR. Filling the official template, assigning **Conformance** levels per criterion, and signing off for your organization remain **human** steps, often with accessibility specialists and legal review. - -This page walks **end to end** from scoping a release to packaging outputs that **you** can reference when drafting VPAT-style tables **outside** the product. It is **not legal advice**; treat it as a practical bridge between product workflows and VPAT preparation. - -## What a VPAT is (and is not) - -| VPAT is | VPAT is not | -| --- | --- | -| A standardized **product disclosure** of accessibility characteristics | A certificate that your product is “fully accessible” | -| A table-oriented document aligned to **specific criteria** (WCAG success criteria, 508 clauses, etc.) | A single PDF export from any one tool | -| Something **your organization** completes and stands behind | Something automated scans alone can complete | -| | An official VPAT or ACR **delivered by TestMu AI** (the platform does not provide these reports) | - -Official templates and guidance are maintained by **ITI (Information Technology Industry Council)**. Your team chooses the right edition (e.g. WCAG 2.x, Section 508, EN 301 549) for the markets and contracts you care about. - -## What Accessibility Testing contributes - -Use the product as an **evidence engine** alongside manual testing and policy review: - -- **Automated and semi-automated findings** tied to pages, components, or builds (web and mobile, depending on your setup). -- **Coverage signals** (what was scanned, when, and under which WCAG target) to support your “evaluation methods used” narrative. -- **Triage and remediation context**: issue detail, hide/restore for noise, bug export, so engineering and accessibility leads can align before you freeze wording in the VPAT. -- **Exports and integrations** for stakeholders who work outside the dashboard ([Exporting & Sharing Reports](/support/docs/accessibility-exporting-sharing-reports/), [Integrations (Jira / Slack)](/support/docs/accessibility-report-integrations/)). - -What Accessibility Testing **does not** replace: - -- **Full manual conformance evaluation** for every success criterion (keyboard-only journeys, zoom/reflow, screen reader behavior, cognitive and multimedia checks where applicable). -- **Authoritative Conformance** labels (Supports / Partially Supports / Does Not Support / Not Applicable) in the VPAT. Those are your judgments, informed by evidence. -- **Legal or contractual interpretation** of ADA, EAA, Section 508, or customer-specific accessibility addenda. - -For framework language at a high level, see the **[Accessibility Compliance Guide](/support/docs/accessibility-compliance-guide/)**. - -## End-to-end workflow - -Follow these phases in order; later phases assume earlier decisions are documented (scope, WCAG level, build or release ID). - -### Phase 1: Lock scope and standards - -1. **Define the product boundary** for this VPAT cycle: web app only, mobile apps, admin vs customer surfaces, embedded third-party widgets, etc. -2. **Pick the VPAT template edition** and the **WCAG conformance target** (e.g. 2.1 AA) your procurement or policy requires. Confirm what your Accessibility projects are configured to use. See **[Supported WCAG Versions & Browsers](/support/docs/accessibility-supported-wcag-browsers/)**. -3. **List representative environments**: browsers, OS versions, assistive technologies you will cite in “evaluation methods used.” Align with how you actually test (for example **[Manual Testing (DevTools)](/support/docs/accessibility-devtools/)** and **[Assistive technology (manual)](/support/docs/screen-reader-on-accessibility/)** guides). -4. **Record a traceability ID** (release version, sprint, or build) and keep the same ID on every export and screenshot filename so auditors can follow the thread. - -### Phase 2: Baseline automated coverage - -1. **Run scans** that match your VPAT scope: DevTools sessions for targeted UX, **[Automation](/support/docs/accessibility-automation/)** for regression suites (including the **[HyperExecute integration: Selenium accessibility](/support/docs/selenium-hyperexecute-accessibility-tests/)** guide where you use that integration), **[Test Scheduling](/support/docs/accessibility-test-scheduling/)** or **[Web Scanner](/support/docs/web-scanner-getting-started/)** for broader URL coverage where applicable. -2. Open completed work in the dashboard using **[Navigating the Dashboard](/support/docs/accessibility-testing-navigating-dashboard/)**; use **[Issue Summary](/support/docs/accessibility-testing-dashboard-issue-summary/)** and **[All Issues](/support/docs/accessibility-testing-dashboard-all-issues/)** for a severity- and criteria-oriented view. -3. Capture **what was in scope** for each run (URLs, app package, scan type). You will reuse this wording in VPAT “remarks” or evaluation-method sections. - -### Phase 3: Manual verification and gaps - -VPAT readers expect evidence beyond automation. - -1. Walk **core user journeys** keyboard-only and with assistive technologies relevant to your audience; use your checklist docs as a matrix: **[Web](/support/docs/accessibility-web-what-we-cover/)**, **[iOS](/support/docs/accessibility-ios-what-we-cover/)**, **[Android](/support/docs/accessibility-android-what-we-cover/)** (including **manual test checklist** sections where present). -2. Log **pass/fail/needs retest** per journey or per WCAG theme, not only per automated issue ID. -3. For anything **Partially Supports** or **Does Not Support**, collect **screenshots, short repro steps, and dates**. The same artifacts you would attach in a bug or audit response. - -### Phase 4: Triage, remediate, re-scan - -1. Use **[Hide and Restore Issues](/support/docs/accessibility-hide-restore-issues/)** only for agreed false positives or noise; document the rationale so your VPAT story matches the report. -2. Route fixes through engineering; use **[Bug Report](/support/docs/accessibility-report-bug/)** or **[Integrations](/support/docs/accessibility-report-integrations/)** if issues should live in your tracker. -3. **Re-run** the same scan types after fixes and keep **before/after** exports if procurement asks for remediation history. - -### Phase 5: Package evidence for VPAT tables - -1. **Export** the report state you want frozen for this VPAT version ([Exporting & Sharing Reports](/support/docs/accessibility-exporting-sharing-reports/)). Exports reflect **filters and hide/restore** at export time. Export intentionally. -2. Build an **evidence index**: map each major VPAT section or WCAG theme to one or more attachments (export file name, dashboard link if permitted internally, ticket IDs). -3. **Draft table rows** in the official template: for each criterion, add **Conformance**, **Remarks**, and pointers to evidence. Prefer concise remarks that quote scan type, date, and scope (“WCAG 2.1 AA DevTools scan on v2.3.1, 2026-04-10, customer checkout only”). -4. For **Accessibility Score** or similar summaries, use them only as **supporting context**, not as a substitute for criterion-by-criterion statements—see **[Accessibility Score](/support/docs/accessibility-web-score/)**. - -### Phase 6: Internal review and publication - -1. Have an **accessibility SME** verify that table wording matches evidence and that no criterion is marked **Supports** without documented manual or automated coverage. -2. Route through **legal or procurement** per your company policy before sharing externally. -3. Version **your organization’s** VPAT or ACR file (e.g. `VPAT_ProductName_2026-04_v1.pdf`) and store it with the underlying **TestMu AI export** bundle for traceability. - -## Mapping product outputs to VPAT-style rows - -You will not find a one-to-one button from “issue type” to “VPAT row.” Instead: - -- **Rule or issue categories** in the dashboard often **align to WCAG success criteria** or platform checks; use your **[rule repository](/support/docs/accessibility-web-rule-repository/)** (web) or platform equivalents to justify which criterion each finding relates to. -- **“Supports”** usually requires both absence of blocking defects **and** documented manual checks for that criterion’s intent. -- **“Partially Supports”** is common when automation passes but assistive technology or keyboard behavior still has gaps, or when only part of the product was evaluated. -- **“Not Applicable”** needs a short justification (e.g. “product has no video; criterion x.x.x not applicable”). - -If stakeholders conflate **score** or **issue count** with **legal compliance**, point them to the compliance guide and this page together. - -## Evidence checklist (audit binder) - -Before you call the VPAT draft “ready for review,” confirm you have: - -- [ ] Locked **product version**, **WCAG/508 edition**, and **evaluation dates** -- [ ] **Inventory of scans** (types, environments, URLs or app builds) -- [ ] **Exported reports** matching the frozen dashboard state for that version -- [ ] **Manual test matrix** or checklist results for critical journeys -- [ ] **Assistive technology** notes where relevant (product, version, OS) -- [ ] **Remediation log** for any open **Partially Supports** / **Does Not Support** items -- [ ] **Internal sign-off** list (accessibility lead, product owner, legal if required) - -## Common pitfalls - -- **Treating a single export as the VPAT**: Buyers expect the ITI-style tables and your org’s responses, not only a scan PDF. -- **Overselling automation**: Many WCAG criteria need manual judgment. Say what was actually tested. -- **Ignoring scope drift**: A VPAT tied to “v3.0” must not silently reuse v2.9 exports without a documented delta. -- **Hiding issues without documentation**: Align hide/restore decisions with what you are willing to defend in remarks. - -## Related docs - -- [Accessibility Compliance Guide (ADA / WCAG / EAA / 508)](/support/docs/accessibility-compliance-guide/) -- [Supported WCAG Versions & Browsers](/support/docs/accessibility-supported-wcag-browsers/) -- [Choosing the Right Tool](/support/docs/accessibility-choosing-the-right-tool/) -- [Navigating the Dashboard](/support/docs/accessibility-testing-navigating-dashboard/) -- [Exporting & Sharing Reports](/support/docs/accessibility-exporting-sharing-reports/) -- [Accessibility Score](/support/docs/accessibility-web-score/) -- [Screen reader testing overview](/support/docs/screen-reader-on-accessibility/) diff --git a/static/docs/accessibility-web-rule-1-1-1-non-text-content.md b/static/docs/accessibility-web-rule-1-1-1-non-text-content.md deleted file mode 100644 index 401f3b48c..000000000 --- a/static/docs/accessibility-web-rule-1-1-1-non-text-content.md +++ /dev/null @@ -1,34 +0,0 @@ -# Non-text Content (1.1.1) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Meaningful images and non-text elements need accessible text alternatives. - -**WCAG Reference** -**Applies to:** WCAG 2.0, WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.0 | **Level:** A | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/non-text-content.html) - -## What this rule checks - -This rule checks whether meaningful non-text content exposes an accessible text alternative that explains its purpose or information. - -## Why it matters - -Screen-reader users and other assistive technology users rely on accessible names or descriptions to understand images, icons, and other non-text elements. - -## Common failure patterns - -- meaningful images with no alt text -- icon-only controls with no accessible name -- charts or visual indicators with no text alternative - -## Remediation guidance - -- add accurate alt text for meaningful content -- mark decorative content as decorative so it is ignored correctly -- ensure icon-only controls expose an accessible name through the appropriate attribute or label - -## Related docs - -- [Web Rule Repository](/support/docs/accessibility-web-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-web-rule-1-3-1-info-and-relationships.md b/static/docs/accessibility-web-rule-1-3-1-info-and-relationships.md deleted file mode 100644 index c21bbc492..000000000 --- a/static/docs/accessibility-web-rule-1-3-1-info-and-relationships.md +++ /dev/null @@ -1,36 +0,0 @@ -# Info and Relationships (1.3.1) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Content structure conveyed visually through layout, formatting, or styling must also be available programmatically so assistive technologies can interpret and present it correctly. - -**WCAG Reference** -**Applies to:** WCAG 2.0, WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.0 | **Level:** A | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/info-and-relationships.html) - -## What this rule checks - -The scanner verifies that headings, lists, tables, form groups, and landmark regions use correct semantic HTML rather than relying on visual appearance alone. - -## Why it matters - -Screen-reader users navigate by structure: headings, lists, and landmarks act as signposts. When structure is only visual (e.g., bold text instead of a heading element), assistive technology cannot expose it, and users lose the ability to skim, jump, and orient themselves. - -## Common failure patterns - -- text styled to look like a heading but using `` or `` instead of ``–`` -- data presented in a grid layout but not using ``, ``, and `` -- form fields grouped visually but missing `` and `` -- navigation and content areas without landmark roles (``, ``, ``) - -## Remediation guidance - -- use native semantic elements for headings, lists, tables, and form groups -- add ARIA landmarks only when native elements are not available -- verify heading levels follow a logical nesting order without skipping levels -- test with a screen reader to confirm the announced structure matches the visual layout - -## Related docs - -- [Web Rule Repository](/support/docs/accessibility-web-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-web-rule-1-3-4-orientation.md b/static/docs/accessibility-web-rule-1-3-4-orientation.md deleted file mode 100644 index de44a0234..000000000 --- a/static/docs/accessibility-web-rule-1-3-4-orientation.md +++ /dev/null @@ -1,35 +0,0 @@ -# Orientation (1.3.4) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Content must not restrict its view and operation to a single display orientation (portrait or landscape) unless a specific orientation is essential. - -**WCAG Reference** -**Applies to:** WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.1 | **Level:** AA | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/orientation.html) - -## What this rule checks - -The scanner detects CSS or viewport meta configurations that lock the page to a single orientation, preventing users from rotating their device. - -## Why it matters - -Users with motor disabilities may mount their device in a fixed orientation. Users with low vision may prefer landscape for wider text columns. Locking orientation can make content inaccessible or uncomfortable to use. - -## Common failure patterns - -- CSS `@media (orientation: portrait)` rules that hide or break content in landscape -- JavaScript that forces a redirect or overlay when the device is rotated -- viewport meta tags that disable user scaling combined with fixed-orientation assumptions - -## Remediation guidance - -- support both portrait and landscape orientations through responsive design -- remove JavaScript orientation locks unless the content genuinely requires a specific orientation (e.g., a piano keyboard app) -- test the layout in both orientations on real devices -- if a specific orientation is essential, clearly document why - -## Related docs - -- [Web Rule Repository](/support/docs/accessibility-web-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-web-rule-1-4-1-use-of-color.md b/static/docs/accessibility-web-rule-1-4-1-use-of-color.md deleted file mode 100644 index 488aef6c8..000000000 --- a/static/docs/accessibility-web-rule-1-4-1-use-of-color.md +++ /dev/null @@ -1,36 +0,0 @@ -# Use of Color (1.4.1) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Color must not be the only means of conveying information, indicating an action, prompting a response, or distinguishing a visual element. - -**WCAG Reference** -**Applies to:** WCAG 2.0, WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.0 | **Level:** A | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/use-of-color.html) - -## What this rule checks - -The scanner flags instances where meaning is communicated exclusively through color without a secondary visual indicator such as text, icons, patterns, or underlines. - -## Why it matters - -Users with color-vision deficiency, low vision, or monochrome displays cannot distinguish elements when the only differentiator is color. Adding a secondary cue ensures everyone perceives the intended meaning. - -## Common failure patterns - -- required form fields marked only with red text -- chart series differentiated only by color with no patterns or labels -- links within body text that are distinguished only by color (no underline or icon) -- status indicators (success/error) using only green/red with no text or icon - -## Remediation guidance - -- pair color with a text label, icon, pattern, or border change -- underline links or add a visible icon to distinguish them from surrounding text -- use patterns, hatching, or data labels in charts alongside color -- test the interface with a color-blindness simulator to confirm the secondary cue is visible - -## Related docs - -- [Web Rule Repository](/support/docs/accessibility-web-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-web-rule-1-4-10-reflow.md b/static/docs/accessibility-web-rule-1-4-10-reflow.md deleted file mode 100644 index d59bcea4f..000000000 --- a/static/docs/accessibility-web-rule-1-4-10-reflow.md +++ /dev/null @@ -1,36 +0,0 @@ -# Reflow (1.4.10) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Content must reflow to fit within a viewport width of 320 CSS pixels without requiring horizontal scrolling, except for content that requires two-dimensional layout (data tables, toolbars, maps). - -**WCAG Reference** -**Applies to:** WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.1 | **Level:** AA | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/reflow.html) - -## What this rule checks - -The scanner identifies content that causes horizontal scrolling at 320px viewport width (equivalent to 400% zoom on a 1280px desktop display). - -## Why it matters - -Users with low vision who zoom to 400% or more should not have to scroll horizontally to read each line of text. Horizontal scrolling makes reading extremely difficult and slow. - -## Common failure patterns - -- fixed-width layouts or containers that do not respond to viewport changes -- wide data tables presented as the primary reading content without a scrollable wrapper -- images or embedded media with fixed pixel widths that overflow the viewport -- multi-column layouts that do not collapse to a single column at narrow widths - -## Remediation guidance - -- use responsive CSS (relative units, `max-width`, flexbox/grid) to allow single-column reflow -- wrap wide tables in a scrollable container so the rest of the page still reflows -- set images to `max-width: 100%` so they scale within their container -- test at 320px viewport width (or 400% zoom) and fix any horizontal overflow - -## Related docs - -- [Web Rule Repository](/support/docs/accessibility-web-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-web-rule-1-4-11-non-text-contrast.md b/static/docs/accessibility-web-rule-1-4-11-non-text-contrast.md deleted file mode 100644 index 1210dc376..000000000 --- a/static/docs/accessibility-web-rule-1-4-11-non-text-contrast.md +++ /dev/null @@ -1,36 +0,0 @@ -# Non-text Contrast (1.4.11) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Visual elements needed to identify UI components and graphical objects must have a contrast ratio of at least 3:1 against adjacent colors. - -**WCAG Reference** -**Applies to:** WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.1 | **Level:** AA | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/non-text-contrast.html) - -## What this rule checks - -The scanner checks the contrast of interactive component boundaries (input borders, button outlines, focus indicators) and meaningful graphical objects (icons, chart segments) against their backgrounds. - -## Why it matters - -Users with low vision need sufficient contrast to perceive form field boundaries, button edges, icon shapes, and chart data. Without adequate contrast, these elements blend into the background and become invisible. - -## Common failure patterns - -- input fields with light gray borders on a white background (below 3:1) -- icon-only buttons where the icon color is too close to the background -- chart segments that rely on low-contrast color fills to convey data -- focus indicators with insufficient contrast against the surrounding area - -## Remediation guidance - -- ensure input and button borders meet the 3:1 ratio against adjacent background colors -- verify icon contrast against both light and dark backgrounds -- add borders or patterns to chart segments alongside color fills -- test all interactive states (default, hover, focus, active) for contrast compliance - -## Related docs - -- [Web Rule Repository](/support/docs/accessibility-web-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-web-rule-1-4-12-text-spacing.md b/static/docs/accessibility-web-rule-1-4-12-text-spacing.md deleted file mode 100644 index e7c82e316..000000000 --- a/static/docs/accessibility-web-rule-1-4-12-text-spacing.md +++ /dev/null @@ -1,36 +0,0 @@ -# Text Spacing (1.4.12) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Content must remain readable and functional when users override text spacing properties (line height, paragraph spacing, letter spacing, and word spacing) up to specified thresholds. - -**WCAG Reference** -**Applies to:** WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.1 | **Level:** AA | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/text-spacing.html) - -## What this rule checks - -The scanner tests whether content is clipped, truncated, or overlapping when text spacing is increased to WCAG-specified thresholds: line height 1.5x font size, paragraph spacing 2x font size, letter spacing 0.12x font size, word spacing 0.16x font size. - -## Why it matters - -Users with dyslexia and other reading disabilities use custom text spacing to improve readability. If a page clips or overlaps text when spacing is increased, those users lose access to the content. - -## Common failure patterns - -- fixed-height containers that clip text when line height increases -- `overflow: hidden` on text containers that truncates content with increased spacing -- absolutely positioned labels that overlap adjacent content when letter spacing grows -- tooltip or badge text that overflows its container - -## Remediation guidance - -- avoid fixed heights on text containers; use `min-height` or auto-height -- do not use `overflow: hidden` on elements that contain user-facing text -- test with a text-spacing bookmarklet that applies the WCAG thresholds -- ensure layouts flex and reflow when text occupies more space - -## Related docs - -- [Web Rule Repository](/support/docs/accessibility-web-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-web-rule-1-4-2-audio-control.md b/static/docs/accessibility-web-rule-1-4-2-audio-control.md deleted file mode 100644 index ad77a15d7..000000000 --- a/static/docs/accessibility-web-rule-1-4-2-audio-control.md +++ /dev/null @@ -1,34 +0,0 @@ -# Audio Control (1.4.2) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Audio that plays automatically for more than three seconds must provide a mechanism to pause, stop, or control the volume independently of the system volume. - -**WCAG Reference** -**Applies to:** WCAG 2.0, WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.0 | **Level:** A | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/audio-control.html) - -## What this rule checks - -The scanner identifies pages where audio or video with sound starts automatically and runs beyond three seconds without an accessible pause or volume control. - -## Why it matters - -Screen-reader users hear both the assistive technology output and the page audio simultaneously. Autoplaying audio can drown out the screen reader, making the page unusable until the user can locate and silence the media. - -## Common failure patterns - -- background music or ambient audio that starts on page load with no pause button -- video banners set to autoplay with sound enabled -- audio players that autoplay and are positioned below the fold, requiring scroll to reach controls - -## Remediation guidance - -- never autoplay audio; if autoplay is required, start muted and let the user opt in -- provide a visible, keyboard-accessible pause or mute control near the top of the page -- ensure the control is the first or one of the first focusable elements so screen-reader users reach it quickly - -## Related docs - -- [Web Rule Repository](/support/docs/accessibility-web-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-web-rule-1-4-3-contrast-minimum.md b/static/docs/accessibility-web-rule-1-4-3-contrast-minimum.md deleted file mode 100644 index 553322f11..000000000 --- a/static/docs/accessibility-web-rule-1-4-3-contrast-minimum.md +++ /dev/null @@ -1,34 +0,0 @@ -# Contrast (Minimum) (1.4.3) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Text and important visual elements need sufficient contrast to remain readable. - -**WCAG Reference** -**Applies to:** WCAG 2.0, WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.0 | **Level:** AA | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/contrast-minimum.html) - -## What this rule checks - -This rule checks whether text and important UI elements meet the minimum contrast requirements needed for readable, usable interfaces. - -## Why it matters - -Low contrast makes content difficult to read for users with low vision, color-vision differences, or difficult viewing conditions. - -## Common failure patterns - -- light gray text on a white background -- status text that uses color with weak contrast -- disabled-looking controls that are still interactive - -## Remediation guidance - -- increase the contrast between foreground and background colors -- verify contrast after hover, focus, and active-state styling is applied -- avoid relying on color tokens that look correct in one theme but fail in another - -## Related docs - -- [Web Rule Repository](/support/docs/accessibility-web-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-web-rule-2-1-1-keyboard.md b/static/docs/accessibility-web-rule-2-1-1-keyboard.md deleted file mode 100644 index b89dd17e3..000000000 --- a/static/docs/accessibility-web-rule-2-1-1-keyboard.md +++ /dev/null @@ -1,36 +0,0 @@ -# Keyboard (2.1.1) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -All interactive functionality must be operable through a keyboard interface without requiring specific timing for individual keystrokes. - -**WCAG Reference** -**Applies to:** WCAG 2.0, WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.0 | **Level:** A | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/keyboard.html) - -## What this rule checks - -The scanner verifies that interactive elements (links, buttons, form controls, and custom widgets) are reachable and operable using keyboard alone: Tab, Shift+Tab, Enter, Space, and arrow keys as appropriate. - -## Why it matters - -Users who cannot use a mouse, including people who rely on switch devices, voice control, or screen readers, depend entirely on keyboard access. If a control cannot be reached or activated by keyboard, those users are blocked. - -## Common failure patterns - -- custom dropdowns or modals that respond only to mouse clicks -- drag-and-drop interfaces with no keyboard alternative -- `` or `` elements used as buttons without `tabindex` or keyboard event handlers -- interactive components that require hover to reveal sub-menus - -## Remediation guidance - -- use native interactive elements (``, ``, ``) whenever possible -- add `tabindex="0"` and keyboard event listeners to custom interactive widgets -- provide keyboard-equivalent actions for drag-and-drop, swipe, and hover interactions -- test the entire flow using only the keyboard (Tab, Enter, Space, Escape, arrows) - -## Related docs - -- [Web Rule Repository](/support/docs/accessibility-web-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-web-rule-2-1-2-no-keyboard-trap.md b/static/docs/accessibility-web-rule-2-1-2-no-keyboard-trap.md deleted file mode 100644 index 7c018632e..000000000 --- a/static/docs/accessibility-web-rule-2-1-2-no-keyboard-trap.md +++ /dev/null @@ -1,36 +0,0 @@ -# No Keyboard Trap (2.1.2) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -If keyboard focus can be moved to a component on the page, focus must also be able to move away from that component using standard keyboard navigation. - -**WCAG Reference** -**Applies to:** WCAG 2.0, WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.0 | **Level:** A | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/no-keyboard-trap.html) - -## What this rule checks - -The scanner checks for components that capture keyboard focus and prevent the user from tabbing or escaping out using standard keys. - -## Why it matters - -A keyboard trap strands users inside a component with no way to continue navigating the page. This is one of the most severe accessibility barriers because it renders everything outside the trap completely unreachable. - -## Common failure patterns - -- modal dialogs that do not return focus on close or lack an Escape key handler -- embedded media players or third-party widgets that consume all key events -- custom rich-text editors that intercept Tab without an exit mechanism -- infinite tab loops inside components with no documented exit key - -## Remediation guidance - -- ensure Escape closes modals and returns focus to the trigger element -- implement focus trapping in modals correctly: trap while open, release on close -- allow Tab to exit custom editors (provide a documented key combination or a skip link) -- test by tabbing into and then out of every interactive region on the page - -## Related docs - -- [Web Rule Repository](/support/docs/accessibility-web-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-web-rule-2-1-4-character-key-shortcuts.md b/static/docs/accessibility-web-rule-2-1-4-character-key-shortcuts.md deleted file mode 100644 index 5d3a237a4..000000000 --- a/static/docs/accessibility-web-rule-2-1-4-character-key-shortcuts.md +++ /dev/null @@ -1,35 +0,0 @@ -# Character Key Shortcuts (2.1.4) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -If a keyboard shortcut uses only a single printable character (letter, number, punctuation, or symbol), the user must be able to turn it off, remap it, or it must only be active when the relevant component has focus. - -**WCAG Reference** -**Applies to:** WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.1 | **Level:** A | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/character-key-shortcuts.html) - -## What this rule checks - -The scanner identifies keyboard shortcuts bound to single character keys that are active globally on the page rather than scoped to a focused component. - -## Why it matters - -Voice-input users dictate text that produces character keystrokes. If a page binds actions to single letters (e.g., pressing "s" to search), voice dictation can accidentally trigger those shortcuts, causing unexpected behavior. - -## Common failure patterns - -- global shortcuts like "s" for search, "j/k" for navigation, or "?" for help -- single-character shortcuts that remain active even when focus is inside a text field -- shortcuts documented only in a help modal, with no way to disable or remap them - -## Remediation guidance - -- require a modifier key (Ctrl, Alt, Shift) for global shortcuts -- scope single-character shortcuts so they only work when the relevant component is focused -- provide a settings UI to disable or remap character shortcuts -- if modifier keys are not feasible, ensure the shortcut deactivates when a text input has focus - -## Related docs - -- [Web Rule Repository](/support/docs/accessibility-web-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-web-rule-2-2-1-timing-adjustable.md b/static/docs/accessibility-web-rule-2-2-1-timing-adjustable.md deleted file mode 100644 index 0dac0167a..000000000 --- a/static/docs/accessibility-web-rule-2-2-1-timing-adjustable.md +++ /dev/null @@ -1,36 +0,0 @@ -# Timing Adjustable (2.2.1) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -When a page enforces a time limit, users must be able to turn off, adjust, or extend the time before it expires. - -**WCAG Reference** -**Applies to:** WCAG 2.0, WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.0 | **Level:** A | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/timing-adjustable.html) - -## What this rule checks - -The scanner identifies session timeouts, auto-redirects, and auto-updating content that impose a fixed time limit without user control. - -## Why it matters - -Users with cognitive disabilities, motor impairments, or users who rely on assistive technology often need more time to read, understand, and interact with content. Unexpected timeouts can cause data loss and frustration. - -## Common failure patterns - -- session timeouts that log the user out without warning or extension options -- auto-advancing carousels or slideshows with no pause control -- form pages that expire and discard input after a fixed duration -- auto-refreshing dashboards that interrupt screen-reader output - -## Remediation guidance - -- provide a warning at least 20 seconds before a timeout and allow the user to extend it -- allow users to disable or increase time limits in account or session settings -- add pause, stop, and resume controls to auto-advancing content -- for real-time exceptions (auctions, live events), clearly document the constraint - -## Related docs - -- [Web Rule Repository](/support/docs/accessibility-web-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-web-rule-2-2-2-pause-stop-hide.md b/static/docs/accessibility-web-rule-2-2-2-pause-stop-hide.md deleted file mode 100644 index 787b6721b..000000000 --- a/static/docs/accessibility-web-rule-2-2-2-pause-stop-hide.md +++ /dev/null @@ -1,36 +0,0 @@ -# Pause, Stop, Hide (2.2.2) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Moving, blinking, scrolling, or auto-updating content must include a mechanism to pause, stop, or hide it. - -**WCAG Reference** -**Applies to:** WCAG 2.0, WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.0 | **Level:** A | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/pause-stop-hide.html) - -## What this rule checks - -The scanner identifies animations, auto-scrolling regions, and auto-updating feeds that run for more than five seconds without user controls. - -## Why it matters - -Motion on screen can be distracting or physically harmful for users with attention disorders, vestibular conditions, or cognitive disabilities. Continuous updates can also interfere with screen-reader announcements. - -## Common failure patterns - -- auto-scrolling news tickers or marquees with no pause button -- animated hero banners that loop indefinitely -- live data feeds that push updates into the DOM without user consent -- background video that plays continuously with no stop control - -## Remediation guidance - -- add visible pause, stop, or hide controls for any auto-moving content -- respect the `prefers-reduced-motion` media query to minimize or remove animation -- for live feeds, allow the user to opt in to updates rather than pushing them automatically -- ensure controls are keyboard accessible and announced by screen readers - -## Related docs - -- [Web Rule Repository](/support/docs/accessibility-web-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-web-rule-2-3-1-three-flashes.md b/static/docs/accessibility-web-rule-2-3-1-three-flashes.md deleted file mode 100644 index 59323eb47..000000000 --- a/static/docs/accessibility-web-rule-2-3-1-three-flashes.md +++ /dev/null @@ -1,36 +0,0 @@ -# Three Flashes or Below Threshold (2.3.1) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Pages must not contain anything that flashes more than three times per second, unless the flash is below general flash and red flash thresholds. - -**WCAG Reference** -**Applies to:** WCAG 2.0, WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.0 | **Level:** A | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/three-flashes-or-below-threshold.html) - -## What this rule checks - -The scanner flags content that produces rapid flashing (more than three flashes per second) in a large enough area to exceed safety thresholds. - -## Why it matters - -Rapid flashing can trigger seizures in people with photosensitive epilepsy. This is a critical safety requirement, not just a usability concern. - -## Common failure patterns - -- animated GIFs or videos with rapid light-to-dark transitions -- strobe effects in promotional content or loading animations -- rapid color cycling in game-like or interactive elements -- auto-playing video ads with flashing transitions - -## Remediation guidance - -- limit flashing to three or fewer per second in any region of the page -- use the Photosensitive Epilepsy Analysis Tool (PEAT) to test video and animation content -- provide a warning before content that contains known flashing and allow the user to skip it -- respect `prefers-reduced-motion` to suppress flashing animations - -## Related docs - -- [Web Rule Repository](/support/docs/accessibility-web-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-web-rule-2-4-1-bypass-blocks.md b/static/docs/accessibility-web-rule-2-4-1-bypass-blocks.md deleted file mode 100644 index 8d64f6475..000000000 --- a/static/docs/accessibility-web-rule-2-4-1-bypass-blocks.md +++ /dev/null @@ -1,36 +0,0 @@ -# Bypass Blocks (2.4.1) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Pages must provide a mechanism to skip past repeated blocks of content that appear on multiple pages, such as navigation menus and headers. - -**WCAG Reference** -**Applies to:** WCAG 2.0, WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.0 | **Level:** A | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/bypass-blocks.html) - -## What this rule checks - -The scanner verifies the presence of skip-navigation links, landmark regions, or heading structures that allow users to bypass repeated content and jump directly to the main content area. - -## Why it matters - -Keyboard and screen-reader users must tab or listen through every repeated element on every page load. Without a bypass mechanism, navigating a site becomes extremely tedious and time-consuming. - -## Common failure patterns - -- no "skip to main content" link at the top of the page -- missing `` landmark so screen readers cannot jump to primary content -- skip links that exist in the DOM but are permanently hidden and never become visible on focus -- pages with no heading structure, removing another way to skip ahead - -## Remediation guidance - -- add a "Skip to main content" link as the first focusable element, visible on focus -- use `` to wrap primary content and `` for navigation blocks -- ensure heading levels provide meaningful structure users can navigate by -- test by pressing Tab immediately after page load to confirm the skip link appears and works - -## Related docs - -- [Web Rule Repository](/support/docs/accessibility-web-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-web-rule-2-4-11-focus-not-obscured.md b/static/docs/accessibility-web-rule-2-4-11-focus-not-obscured.md deleted file mode 100644 index 695600a9f..000000000 --- a/static/docs/accessibility-web-rule-2-4-11-focus-not-obscured.md +++ /dev/null @@ -1,36 +0,0 @@ -# Focus Not Obscured (Minimum) (2.4.11) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -When a user interface component receives keyboard focus, it must not be entirely hidden by author-created content such as sticky headers, footers, or overlays. - -**WCAG Reference** -**Applies to:** WCAG 2.2 -**Introduced in:** WCAG 2.2 | **Level:** AA | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/focus-not-obscured-minimum.html) - -## What this rule checks - -The scanner verifies that focused elements are not fully concealed behind sticky navigation bars, cookie banners, chat widgets, or other fixed-position elements. - -## Why it matters - -Keyboard users need to see which element has focus to understand where they are on the page. If the focused element is hidden behind a sticky header or modal backdrop, the user loses orientation and cannot confirm their next action. - -## Common failure patterns - -- focused items scrolled behind a sticky navigation header -- cookie consent banners that cover the bottom of the viewport where focused elements appear -- chat widgets or help bubbles that overlay interactive elements -- modal backdrops that remain visible and cover page content behind them - -## Remediation guidance - -- add scroll padding (e.g., `scroll-padding-top`) to account for sticky header height -- ensure cookie banners and chat widgets do not overlap focusable page content -- implement `scrollIntoView` with appropriate offsets when managing focus programmatically -- test by tabbing through the entire page and confirming every focused element is at least partially visible - -## Related docs - -- [Web Rule Repository](/support/docs/accessibility-web-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-web-rule-2-4-2-page-titled.md b/static/docs/accessibility-web-rule-2-4-2-page-titled.md deleted file mode 100644 index 0b1643f0e..000000000 --- a/static/docs/accessibility-web-rule-2-4-2-page-titled.md +++ /dev/null @@ -1,36 +0,0 @@ -# Page Titled (2.4.2) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Every web page must have a descriptive title that identifies its topic or purpose. - -**WCAG Reference** -**Applies to:** WCAG 2.0, WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.0 | **Level:** A | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/page-titled.html) - -## What this rule checks - -The scanner verifies that each page has a non-empty `` element and that the title text meaningfully describes the page content. - -## Why it matters - -The page title is the first thing a screen reader announces when a page loads. It also appears in browser tabs, bookmarks, and search results. A missing or generic title forces users to explore the page to understand where they are. - -## Common failure patterns - -- pages with an empty `` element -- all pages sharing the same generic title like "Home" or the site name -- SPA routes that do not update the document title when the view changes -- titles that describe the site but not the specific page (e.g., "Acme Corp" on every page) - -## Remediation guidance - -- write titles in the format "Page Purpose - Site Name" (e.g., "Order History - Acme Store") -- update `document.title` on route changes in single-page applications -- include key differentiators when pages are similar (e.g., "Edit Profile" vs. "View Profile") -- keep titles concise but specific enough to identify the page without seeing it - -## Related docs - -- [Web Rule Repository](/support/docs/accessibility-web-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-web-rule-2-4-3-focus-order.md b/static/docs/accessibility-web-rule-2-4-3-focus-order.md deleted file mode 100644 index 03cef9438..000000000 --- a/static/docs/accessibility-web-rule-2-4-3-focus-order.md +++ /dev/null @@ -1,36 +0,0 @@ -# Focus Order (2.4.3) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -When a page can be navigated sequentially, interactive components must receive focus in an order that preserves meaning and operability. - -**WCAG Reference** -**Applies to:** WCAG 2.0, WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.0 | **Level:** A | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/focus-order.html) - -## What this rule checks - -The scanner verifies that the tab order of interactive elements follows a logical reading sequence and does not jump unexpectedly between unrelated sections. - -## Why it matters - -Keyboard users experience a page as a linear sequence of focusable elements. An illogical focus order makes it difficult to understand context, complete multi-step forms, or predict where focus will move next. - -## Common failure patterns - -- positive `tabindex` values that override the natural DOM order -- visually reordered layouts (CSS Grid/Flexbox `order`) where DOM order does not match visual order -- dynamically injected content that receives focus before the user reaches the trigger -- modals or drawers that do not move focus into themselves on open - -## Remediation guidance - -- rely on DOM source order rather than positive `tabindex` values -- ensure CSS visual reordering matches the underlying DOM sequence -- move focus into modals, dialogs, and drawers when they open, and return it when they close -- test with keyboard-only navigation to confirm the sequence feels natural - -## Related docs - -- [Web Rule Repository](/support/docs/accessibility-web-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-web-rule-2-4-4-link-purpose.md b/static/docs/accessibility-web-rule-2-4-4-link-purpose.md deleted file mode 100644 index 5d2ea63f2..000000000 --- a/static/docs/accessibility-web-rule-2-4-4-link-purpose.md +++ /dev/null @@ -1,36 +0,0 @@ -# Link Purpose in Context (2.4.4) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The purpose of each link must be determinable from the link text alone or from the link text combined with its programmatically determined context. - -**WCAG Reference** -**Applies to:** WCAG 2.0, WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.0 | **Level:** A | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/link-purpose-in-context.html) - -## What this rule checks - -The scanner flags links whose text is generic (e.g., "click here", "read more", "learn more") without surrounding context that clarifies the destination or action. - -## Why it matters - -Screen-reader users often navigate by listing all links on a page. If every link says "Read more," the list is useless. Clear link text lets users decide which link to follow without reading the surrounding paragraph. - -## Common failure patterns - -- repeated "Click here" or "Read more" links across a page -- links that use a URL as the visible link text -- icon-only links (e.g., a pencil icon) with no accessible name -- links whose purpose changes based on visual position but whose text is identical - -## Remediation guidance - -- write link text that describes the destination or action (e.g., "View order #1234" instead of "Click here") -- use `aria-label` or `aria-labelledby` to provide context when the visible text must remain short -- add `aria-label` to icon-only links that describes the action -- when multiple links share text, add visually hidden context (e.g., "Read more about pricing") - -## Related docs - -- [Web Rule Repository](/support/docs/accessibility-web-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-web-rule-2-4-7-focus-visible.md b/static/docs/accessibility-web-rule-2-4-7-focus-visible.md deleted file mode 100644 index 85c4394d6..000000000 --- a/static/docs/accessibility-web-rule-2-4-7-focus-visible.md +++ /dev/null @@ -1,34 +0,0 @@ -# Focus Visible (2.4.7) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Keyboard users need a visible focus indicator while moving through the interface. - -**WCAG Reference** -**Applies to:** WCAG 2.0, WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.0 | **Level:** AA | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/focus-visible.html) - -## What this rule checks - -This rule checks whether users can see where keyboard focus currently is as they move through interactive elements. - -## Why it matters - -Without a visible focus state, keyboard users can lose track of where they are on the page and may be unable to complete key tasks. - -## Common failure patterns - -- focus outlines removed without a replacement style -- custom components with hidden or clipped focus styling -- insufficient contrast between the focus ring and surrounding UI - -## Remediation guidance - -- keep the default focus indicator or replace it with an equally visible custom style -- test focus styling on links, buttons, form fields, menus, and dialogs -- verify focus visibility in light and dark themes - -## Related docs - -- [Web Rule Repository](/support/docs/accessibility-web-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-web-rule-2-5-1-pointer-gestures.md b/static/docs/accessibility-web-rule-2-5-1-pointer-gestures.md deleted file mode 100644 index f9961f2bf..000000000 --- a/static/docs/accessibility-web-rule-2-5-1-pointer-gestures.md +++ /dev/null @@ -1,36 +0,0 @@ -# Pointer Gestures (2.5.1) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Functionality that uses multi-point or path-based gestures must also be operable with a single-pointer action without a path-based gesture, unless the gesture is essential. - -**WCAG Reference** -**Applies to:** WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.1 | **Level:** A | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/pointer-gestures.html) - -## What this rule checks - -The scanner identifies interactions that require multi-touch gestures (pinch, two-finger swipe) or precise path-based gestures (drawing a shape, swiping in a specific direction) without a single-click or single-tap alternative. - -## Why it matters - -Users with motor disabilities may not be able to perform multi-finger gestures or trace precise paths. Providing a single-pointer alternative ensures these users can still access the functionality. - -## Common failure patterns - -- pinch-to-zoom as the only way to resize a map or image (no +/- buttons) -- swipe-to-delete with no alternative button or menu option -- drawing-based interactions (signatures, annotations) with no keyboard or button fallback -- two-finger scroll required to navigate within an embedded region - -## Remediation guidance - -- add button controls for zoom, pan, and delete alongside gesture support -- provide single-tap or single-click alternatives for every multi-point gesture -- for path-based input, offer an alternative method (e.g., typed input, button sequences) -- test with a single mouse click/tap only to confirm all features remain accessible - -## Related docs - -- [Web Rule Repository](/support/docs/accessibility-web-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-web-rule-2-5-2-pointer-cancellation.md b/static/docs/accessibility-web-rule-2-5-2-pointer-cancellation.md deleted file mode 100644 index d52893bb0..000000000 --- a/static/docs/accessibility-web-rule-2-5-2-pointer-cancellation.md +++ /dev/null @@ -1,35 +0,0 @@ -# Pointer Cancellation (2.5.2) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -For single-pointer actions, at least one of the following must be true: the down-event does not trigger the action, the action is triggered on the up-event and can be aborted, the up-event reverses any outcome of the down-event, or completing the action on the down-event is essential. - -**WCAG Reference** -**Applies to:** WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.1 | **Level:** A | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/pointer-cancellation.html) - -## What this rule checks - -The scanner verifies that interactive elements do not fire irreversible actions on `mousedown` or `touchstart` alone, allowing users to abort by moving the pointer away before releasing. - -## Why it matters - -Users with motor impairments frequently trigger accidental clicks. Firing on the up-event (click/mouseup/touchend) allows them to move the pointer off the target before releasing, cancelling the action. - -## Common failure patterns - -- destructive actions (delete, submit, purchase) triggered on `mousedown` -- drag-and-drop that commits on `pointerdown` with no undo -- custom buttons that respond to `touchstart` instead of `click` - -## Remediation guidance - -- use `click` events rather than `mousedown` or `touchstart` for actions -- if down-event activation is needed, provide an undo or confirmation step -- implement drag cancellation by allowing users to drop outside the target zone -- test by pressing down on a button, dragging away, and releasing to confirm no action occurs - -## Related docs - -- [Web Rule Repository](/support/docs/accessibility-web-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-web-rule-2-5-3-label-in-name.md b/static/docs/accessibility-web-rule-2-5-3-label-in-name.md deleted file mode 100644 index ba324f72a..000000000 --- a/static/docs/accessibility-web-rule-2-5-3-label-in-name.md +++ /dev/null @@ -1,35 +0,0 @@ -# Label in Name (2.5.3) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -When a UI component has a visible text label, the accessible name must contain the visible text so that voice-input users can activate the control by speaking its visible label. - -**WCAG Reference** -**Applies to:** WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.1 | **Level:** A | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/label-in-name.html) - -## What this rule checks - -The scanner compares the visible label text with the computed accessible name and flags mismatches where the visible text is not included in the accessible name. - -## Why it matters - -Voice-input users say "click [visible label]" to interact with controls. If the accessible name differs from the visible label, the voice command fails and the user cannot activate the control. - -## Common failure patterns - -- a button that shows "Submit" visually but has `aria-label="Send form data"` -- an input labeled "Email" visually but `aria-labelledby` points to a hidden element with different text -- icon buttons with visible tooltip text that does not match the `aria-label` - -## Remediation guidance - -- ensure the accessible name starts with or contains the visible label text -- prefer visible `` elements over `aria-label` when visible text is available -- when `aria-label` is needed, include the visible text within it (e.g., `aria-label="Submit order form"` for a button labeled "Submit") -- test with voice-input software to confirm controls can be activated by speaking their visible label - -## Related docs - -- [Web Rule Repository](/support/docs/accessibility-web-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-web-rule-2-5-4-motion-actuation.md b/static/docs/accessibility-web-rule-2-5-4-motion-actuation.md deleted file mode 100644 index b1b6d6f8e..000000000 --- a/static/docs/accessibility-web-rule-2-5-4-motion-actuation.md +++ /dev/null @@ -1,35 +0,0 @@ -# Motion Actuation (2.5.4) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Functionality triggered by device motion (shaking, tilting) or user motion (gestures captured by a camera) must also be operable through a UI control, and the motion response can be disabled to prevent accidental activation. - -**WCAG Reference** -**Applies to:** WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.1 | **Level:** A | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/motion-actuation.html) - -## What this rule checks - -The scanner identifies interactions driven by device sensors (accelerometer, gyroscope, camera) that do not offer an alternative button or control and cannot be disabled. - -## Why it matters - -Users who have their device mounted in a fixed position, users with tremors, or users who cannot perform specific physical motions need an alternative way to trigger the same functionality. - -## Common failure patterns - -- "shake to undo" with no on-screen undo button -- tilt-to-scroll with no scroll controls or keyboard alternative -- camera-based gesture recognition with no button fallback - -## Remediation guidance - -- provide an on-screen button or control for every motion-triggered action -- allow users to disable motion-based activation in settings -- if motion is essential to the functionality (e.g., a step counter), document the exception -- test with the device in a fixed position to confirm all functionality is reachable through UI controls - -## Related docs - -- [Web Rule Repository](/support/docs/accessibility-web-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-web-rule-2-5-7-dragging-movements.md b/static/docs/accessibility-web-rule-2-5-7-dragging-movements.md deleted file mode 100644 index b6ac2de16..000000000 --- a/static/docs/accessibility-web-rule-2-5-7-dragging-movements.md +++ /dev/null @@ -1,36 +0,0 @@ -# Dragging Movements (2.5.7) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Functionality that uses a dragging movement must also be achievable through a single-pointer action without dragging, unless dragging is essential or the functionality is determined by the user agent. - -**WCAG Reference** -**Applies to:** WCAG 2.2 -**Introduced in:** WCAG 2.2 | **Level:** AA | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/dragging-movements.html) - -## What this rule checks - -The scanner flags drag-and-drop interactions, slider controls, and reorderable lists that do not offer an alternative single-click or keyboard-based method. - -## Why it matters - -Users with motor disabilities may not be able to hold a pointer button while moving the cursor. A single-pointer alternative (click source, click target) or keyboard method ensures these users can still accomplish the task. - -## Common failure patterns - -- kanban boards that require drag-and-drop to move cards between columns -- sliders that can only be adjusted by dragging the thumb -- file upload areas that only accept drag-and-drop with no browse button -- sortable lists with no move-up/move-down buttons - -## Remediation guidance - -- add move-up/move-down buttons or a "Move to" menu for reorderable lists -- support click-on-source then click-on-destination as an alternative to drag-and-drop -- ensure sliders can be adjusted with arrow keys, or provide a text input alternative -- always include a file browse button alongside drag-and-drop upload zones - -## Related docs - -- [Web Rule Repository](/support/docs/accessibility-web-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-web-rule-2-5-8-target-size.md b/static/docs/accessibility-web-rule-2-5-8-target-size.md deleted file mode 100644 index 05857bdf6..000000000 --- a/static/docs/accessibility-web-rule-2-5-8-target-size.md +++ /dev/null @@ -1,36 +0,0 @@ -# Target Size (Minimum) (2.5.8) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Interactive targets must be at least 24 by 24 CSS pixels in size, with exceptions for inline links, targets whose size is determined by the user agent, and targets with sufficient spacing. - -**WCAG Reference** -**Applies to:** WCAG 2.2 -**Introduced in:** WCAG 2.2 | **Level:** AA | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/target-size-minimum.html) - -## What this rule checks - -The scanner measures the rendered size of interactive elements (buttons, links, form controls) and flags those smaller than 24x24 CSS pixels that do not meet any exception criteria. - -## Why it matters - -Users with motor impairments, tremors, or limited dexterity need adequately sized targets to tap or click accurately. Small targets lead to frequent mis-taps and frustration. - -## Common failure patterns - -- icon buttons (close, favorite, share) rendered at 16x16 pixels with no padding -- densely packed toolbar buttons with no spacing between them -- small checkbox or radio button hit areas that rely on the browser default size -- action links in compact table rows with minimal padding - -## Remediation guidance - -- ensure clickable areas are at least 24x24 CSS pixels (44x44 is recommended for touch interfaces) -- add padding or transparent borders to increase the hit area without changing the visual size -- use `min-width` and `min-height` on interactive elements -- increase spacing between adjacent small targets so users can tap accurately - -## Related docs - -- [Web Rule Repository](/support/docs/accessibility-web-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-web-rule-3-2-1-on-focus.md b/static/docs/accessibility-web-rule-3-2-1-on-focus.md deleted file mode 100644 index fd7f49ba3..000000000 --- a/static/docs/accessibility-web-rule-3-2-1-on-focus.md +++ /dev/null @@ -1,35 +0,0 @@ -# On Focus (3.2.1) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Receiving focus on a component must not trigger an unexpected change of context such as a page navigation, form submission, or modal opening. - -**WCAG Reference** -**Applies to:** WCAG 2.0, WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.0 | **Level:** A | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/on-focus.html) - -## What this rule checks - -The scanner identifies elements that initiate a context change (navigation, submit, new window) when they receive keyboard focus rather than when they are explicitly activated. - -## Why it matters - -Keyboard and screen-reader users move focus to explore the page before deciding to act. If focus alone triggers an action, users lose control and may be taken to an unexpected page or state. - -## Common failure patterns - -- dropdown selects that navigate to a new page as soon as an option receives focus -- links or buttons that auto-submit a form on focus -- components that open a new browser window or tab when tabbed into - -## Remediation guidance - -- trigger context changes only on explicit activation (click, Enter, Space) -- for select menus that act as navigation, add a separate "Go" button -- never attach `onfocus` handlers that cause navigation, submission, or window changes -- test by tabbing through all interactive elements without pressing Enter - -## Related docs - -- [Web Rule Repository](/support/docs/accessibility-web-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-web-rule-3-2-2-on-input.md b/static/docs/accessibility-web-rule-3-2-2-on-input.md deleted file mode 100644 index b5bd6a147..000000000 --- a/static/docs/accessibility-web-rule-3-2-2-on-input.md +++ /dev/null @@ -1,36 +0,0 @@ -# On Input (3.2.2) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Changing the setting of a form control or UI component must not automatically cause an unexpected change of context unless the user has been informed beforehand. - -**WCAG Reference** -**Applies to:** WCAG 2.0, WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.0 | **Level:** A | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/on-input.html) - -## What this rule checks - -The scanner flags form controls that trigger navigation, submission, or DOM restructuring on value change without prior notice to the user. - -## Why it matters - -Users expect to review and confirm their choices before committing. Automatic context changes on input can disorient screen-reader users and cause data loss for users who accidentally select the wrong option. - -## Common failure patterns - -- radio buttons that submit a form or navigate to a new page on selection -- checkboxes that reload the page when toggled -- text fields that trigger search navigation on every keystroke without warning -- dropdown menus that navigate immediately on selection without a submit action - -## Remediation guidance - -- separate the selection action from the submission action (add a "Submit" or "Apply" button) -- if auto-submission is intentional, inform the user in advance (e.g., "Selecting a value will update the page") -- use ARIA live regions to announce dynamic content changes without a full context switch -- test all form controls by changing values and confirming no unexpected navigation occurs - -## Related docs - -- [Web Rule Repository](/support/docs/accessibility-web-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-web-rule-3-2-6-consistent-help.md b/static/docs/accessibility-web-rule-3-2-6-consistent-help.md deleted file mode 100644 index 5b5e4bf3d..000000000 --- a/static/docs/accessibility-web-rule-3-2-6-consistent-help.md +++ /dev/null @@ -1,36 +0,0 @@ -# Consistent Help (3.2.6) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -If a set of web pages provides help mechanisms (contact information, chat, FAQ links, self-help options), they must appear in the same relative order on each page. - -**WCAG Reference** -**Applies to:** WCAG 2.2 -**Introduced in:** WCAG 2.2 | **Level:** A | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/consistent-help.html) - -## What this rule checks - -The scanner verifies that help-related UI elements maintain consistent placement and ordering across pages within the same site. - -## Why it matters - -Users with cognitive disabilities rely on predictable patterns to find help. If a help link or chat button moves to a different position on every page, users must search for it each time, increasing cognitive load and frustration. - -## Common failure patterns - -- a "Contact Support" link in the footer on some pages and in the header on others -- a help chat widget that appears on the right on some pages and the left on others -- FAQ links included on product pages but missing from checkout pages -- help navigation items in a different order across different sections of the site - -## Remediation guidance - -- place help mechanisms in the same relative position across all pages (e.g., always last in the footer) -- use a shared layout component to ensure consistent ordering automatically -- if multiple help options exist, keep their order fixed (e.g., FAQ, then Chat, then Email) -- audit help element placement across page templates to confirm consistency - -## Related docs - -- [Web Rule Repository](/support/docs/accessibility-web-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-web-rule-3-3-1-error-identification.md b/static/docs/accessibility-web-rule-3-3-1-error-identification.md deleted file mode 100644 index 2723fc8bb..000000000 --- a/static/docs/accessibility-web-rule-3-3-1-error-identification.md +++ /dev/null @@ -1,36 +0,0 @@ -# Error Identification (3.3.1) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -When an input error is automatically detected, the item in error must be identified and the error described to the user in text. - -**WCAG Reference** -**Applies to:** WCAG 2.0, WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.0 | **Level:** A | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/error-identification.html) - -## What this rule checks - -The scanner verifies that form validation errors are communicated through visible text messages associated with the relevant field, not only through color or icon changes. - -## Why it matters - -Users who cannot perceive color, see icons, or understand positional cues need explicit text to know which field has an error and what the error is. Without text, they cannot correct the problem. - -## Common failure patterns - -- form fields highlighted in red with no error message text -- a generic "There are errors on this page" banner with no indication of which fields need attention -- error messages displayed visually but not programmatically associated with the field (no `aria-describedby`) -- inline validation that clears the message before the screen reader can announce it - -## Remediation guidance - -- display a text error message adjacent to or within the erroneous field -- associate the error message with the field using `aria-describedby` or `aria-errormessage` -- set `aria-invalid="true"` on the field when an error is detected -- move focus to the first error or provide a summary with links to each erroneous field - -## Related docs - -- [Web Rule Repository](/support/docs/accessibility-web-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-web-rule-3-3-2-labels-or-instructions.md b/static/docs/accessibility-web-rule-3-3-2-labels-or-instructions.md deleted file mode 100644 index fcf1d8c68..000000000 --- a/static/docs/accessibility-web-rule-3-3-2-labels-or-instructions.md +++ /dev/null @@ -1,36 +0,0 @@ -# Labels or Instructions (3.3.2) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Form fields and interactive controls must have labels or instructions that describe the expected input. - -**WCAG Reference** -**Applies to:** WCAG 2.0, WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.0 | **Level:** A | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/labels-or-instructions.html) - -## What this rule checks - -The scanner verifies that form inputs have visible labels programmatically associated with the control, and that fields requiring specific formats include instructions. - -## Why it matters - -Without labels, screen-reader users hear only "edit text" or "combo box" with no indication of what information is expected. Sighted users benefit from persistent labels too, especially when revisiting a partially completed form. - -## Common failure patterns - -- input fields that rely on placeholder text as the only label (disappears on focus) -- labels that exist visually but are not associated with the input (`for`/`id` mismatch) -- date or phone fields with no format hint (e.g., "MM/DD/YYYY") -- groups of related fields (address, payment) with no group label or `` - -## Remediation guidance - -- add a visible `` element with a matching `for` attribute for every form control -- include format instructions or examples near the field (e.g., "Enter date as MM/DD/YYYY") -- use `` and `` for logically grouped fields -- never use placeholder as a substitute for a visible label - -## Related docs - -- [Web Rule Repository](/support/docs/accessibility-web-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-web-rule-3-3-7-redundant-entry.md b/static/docs/accessibility-web-rule-3-3-7-redundant-entry.md deleted file mode 100644 index 3b8b8248c..000000000 --- a/static/docs/accessibility-web-rule-3-3-7-redundant-entry.md +++ /dev/null @@ -1,36 +0,0 @@ -# Redundant Entry (3.3.7) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Information previously entered by or provided to the user in the same process must be auto-populated or available for selection, unless re-entering is essential for security or the data is no longer valid. - -**WCAG Reference** -**Applies to:** WCAG 2.2 -**Introduced in:** WCAG 2.2 | **Level:** A | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/redundant-entry.html) - -## What this rule checks - -The scanner identifies multi-step flows where users are asked to re-enter information they have already provided in a previous step (e.g., name, address, email) without auto-fill or a selection option. - -## Why it matters - -Re-entering the same information is difficult for users with cognitive disabilities, motor impairments, or short-term memory challenges. It increases error rates and abandonment in multi-step workflows. - -## Common failure patterns - -- checkout forms that ask for the shipping address and then require the billing address to be typed again from scratch -- multi-page forms that do not pre-fill fields from earlier pages -- account creation flows that ask for the email on step 1 and again on step 3 -- support ticket forms that require re-entering contact details already available from the user profile - -## Remediation guidance - -- auto-populate fields with information provided in earlier steps of the same process -- offer a "same as shipping" checkbox for billing address forms -- use session storage or form state management to carry data across steps -- if security requires re-entry (e.g., password confirmation), document the exception - -## Related docs - -- [Web Rule Repository](/support/docs/accessibility-web-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-web-rule-3-3-8-accessible-authentication.md b/static/docs/accessibility-web-rule-3-3-8-accessible-authentication.md deleted file mode 100644 index b921dadf5..000000000 --- a/static/docs/accessibility-web-rule-3-3-8-accessible-authentication.md +++ /dev/null @@ -1,36 +0,0 @@ -# Accessible Authentication (Minimum) (3.3.8) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Authentication steps must not require a cognitive function test (such as remembering a password, solving a puzzle, or recognizing images) unless an alternative method, an assistance mechanism, or an object-recognition exception applies. - -**WCAG Reference** -**Applies to:** WCAG 2.2 -**Introduced in:** WCAG 2.2 | **Level:** AA | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/accessible-authentication-minimum.html) - -## What this rule checks - -The scanner flags login flows that rely solely on cognitive challenges (CAPTCHAs, image puzzles, memory-based knowledge questions) without offering an accessible alternative. - -## Why it matters - -Users with cognitive disabilities may not be able to solve puzzles, remember complex passwords, or complete CAPTCHAs. Authentication barriers lock these users out of services entirely. - -## Common failure patterns - -- image-based CAPTCHAs with no audio or alternative verification option -- security questions that rely on memory ("What was your first pet's name?") -- login flows that block password managers from auto-filling credentials -- two-factor authentication that requires memorizing a code without allowing paste - -## Remediation guidance - -- allow password managers to auto-fill login fields (do not use `autocomplete="off"` on authentication fields) -- provide an accessible CAPTCHA alternative (audio CAPTCHA, email verification, or WebAuthn) -- support passwordless authentication (magic links, biometrics, passkeys) -- allow pasting into verification code fields for users who receive codes via email or authenticator apps - -## Related docs - -- [Web Rule Repository](/support/docs/accessibility-web-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-web-rule-4-1-1-parsing.md b/static/docs/accessibility-web-rule-4-1-1-parsing.md deleted file mode 100644 index 6256d0a74..000000000 --- a/static/docs/accessibility-web-rule-4-1-1-parsing.md +++ /dev/null @@ -1,38 +0,0 @@ -# Parsing (4.1.1) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -HTML markup must be well-formed: elements must have complete start and end tags, be nested correctly, not contain duplicate attributes, and use unique IDs. - -**WCAG Reference** -**Applies to:** WCAG 2.0, WCAG 2.1 (obsolete in WCAG 2.2) -**Introduced in:** WCAG 2.0 | **Level:** A | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/parsing.html) - -WCAG 2.2 considers this criterion obsolete because modern browsers and assistive technologies handle parsing errors more gracefully than older ones. It may still appear in scan results when testing against WCAG 2.0 or 2.1. - -## What this rule checks - -The scanner validates HTML for duplicate IDs, malformed nesting, unclosed elements, and duplicate attributes that could confuse assistive technology parsers. - -## Why it matters - -Duplicate IDs break `aria-labelledby`, `aria-describedby`, and `` associations, causing assistive technology to reference the wrong element. Malformed markup can also cause screen readers to skip or misinterpret content. - -## Common failure patterns - -- multiple elements sharing the same `id` attribute -- unclosed ``, ``, or `` elements that shift the DOM tree -- duplicate attributes on the same element -- improperly nested interactive elements (e.g., `` inside ``) - -## Remediation guidance - -- run an HTML validator to catch structural issues -- ensure every `id` is unique within the page -- fix unclosed or misnested elements -- avoid nesting interactive elements inside each other - -## Related docs - -- [Web Rule Repository](/support/docs/accessibility-web-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-web-rule-4-1-2-name-role-value.md b/static/docs/accessibility-web-rule-4-1-2-name-role-value.md deleted file mode 100644 index 17a094cdc..000000000 --- a/static/docs/accessibility-web-rule-4-1-2-name-role-value.md +++ /dev/null @@ -1,36 +0,0 @@ -# Name, Role, Value (4.1.2) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -All user interface components must expose their name, role, states, properties, and values to assistive technologies, and notification of changes must be available programmatically. - -**WCAG Reference** -**Applies to:** WCAG 2.0, WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.0 | **Level:** A | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/name-role-value.html) - -## What this rule checks - -The scanner verifies that interactive elements have an accessible name, an appropriate ARIA or native role, and that state changes (expanded, checked, selected, disabled) are communicated programmatically. - -## Why it matters - -Assistive technologies rely on the accessibility tree to describe controls to users. If a custom widget lacks a role, name, or state information, screen-reader users cannot tell what the control is, what it does, or what state it is in. - -## Common failure patterns - -- custom toggle switches built with `` that have no `role="switch"` or `aria-checked` -- accordion headers that do not expose `aria-expanded` -- custom dropdowns that do not announce the selected value -- buttons with no accessible name (empty text, no `aria-label`) - -## Remediation guidance - -- use native HTML elements whenever possible (``, ``, ``) -- for custom widgets, add the appropriate ARIA role and maintain state attributes (`aria-expanded`, `aria-checked`, `aria-selected`) -- ensure every interactive element has an accessible name through visible text, `aria-label`, or `aria-labelledby` -- update ARIA states dynamically when the user interacts with the control - -## Related docs - -- [Web Rule Repository](/support/docs/accessibility-web-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-web-rule-4-1-3-status-messages.md b/static/docs/accessibility-web-rule-4-1-3-status-messages.md deleted file mode 100644 index 9d8317151..000000000 --- a/static/docs/accessibility-web-rule-4-1-3-status-messages.md +++ /dev/null @@ -1,36 +0,0 @@ -# Status Messages (4.1.3) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Status messages that provide information about the success or results of an action, the waiting state of an application, or the progress of a process must be programmatically determinable through role or properties so they can be presented to the user by assistive technologies without receiving focus. - -**WCAG Reference** -**Applies to:** WCAG 2.1, WCAG 2.2 -**Introduced in:** WCAG 2.1 | **Level:** AA | [Read the official specification →](https://www.w3.org/WAI/WCAG22/Understanding/status-messages.html) - -## What this rule checks - -The scanner verifies that dynamic status messages use ARIA live regions (`role="status"`, `role="alert"`, `aria-live`) so screen readers announce them without moving focus away from the current task. - -## Why it matters - -Sighted users see toast notifications, search result counts, and progress indicators update on screen. Screen-reader users miss these updates entirely unless the content is announced through a live region. - -## Common failure patterns - -- "3 results found" text appears on screen after a search but is not in a live region -- success/error banners injected into the DOM without `role="alert"` or `role="status"` -- loading spinners with no text announcement for screen readers -- cart item count updates that are only visual - -## Remediation guidance - -- use `role="status"` for non-urgent updates (result counts, save confirmations) -- use `role="alert"` for urgent messages (errors, warnings) -- ensure the live region exists in the DOM before the content is injected (add content to an existing container rather than injecting the container) -- avoid overusing alerts, since frequent announcements interrupt the user's workflow - -## Related docs - -- [Web Rule Repository](/support/docs/accessibility-web-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-web-rule-repository.md b/static/docs/accessibility-web-rule-repository.md deleted file mode 100644 index 5582e4d55..000000000 --- a/static/docs/accessibility-web-rule-repository.md +++ /dev/null @@ -1,31 +0,0 @@ -# Web Rule Repository - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Use this repository to browse web accessibility rules, their WCAG mappings, and related remediation guidance. - -This repository links web-focused rule pages that explain what each check covers, why it matters for users, and how to begin remediation. Reach for it when you already know you are fixing a web accessibility issue and want a rule-level reference instead of a broad coverage summary alone. - -## When to use this - -Use this page when you already know you are working on a web accessibility issue and want a rule-level reference instead of a broad checklist page. - -## Priority web rule pages - -| Rule page | WCAG | Use it for | -| --- | --- | --- | -| [Non-text Content (1.1.1)](/support/docs/accessibility-web-rule-1-1-1-non-text-content/) | 1.1.1 | Missing or weak text alternatives for meaningful non-text content | -| [Contrast (Minimum) (1.4.3)](/support/docs/accessibility-web-rule-1-4-3-contrast-minimum/) | 1.4.3 | Low contrast text and visual elements | -| [Focus Visible (2.4.7)](/support/docs/accessibility-web-rule-2-4-7-focus-visible/) | 2.4.7 | Missing or unclear keyboard focus indicators | - -## How to use this repository - -1. Start from the relevant rule page for the issue you are seeing. -2. Confirm whether the issue is primarily automated, manual, or needs both types of verification. -3. Fix the issue in the application code or design system. -4. Re-run the relevant Accessibility workflow and compare the updated report. - -## Related docs - -- [Web checklist](/support/docs/accessibility-web-what-we-cover/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-web-score.md b/static/docs/accessibility-web-score.md deleted file mode 100644 index d90bc9ce7..000000000 --- a/static/docs/accessibility-web-score.md +++ /dev/null @@ -1,320 +0,0 @@ -# Accessibility Score - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The **Accessibility Score** is a single number from **0 to 100** that represents the accessibility health of a scanned page or screen. Instead of only reading long reports, the score gives you a **clear, actionable signal** for where the product stands relative to the issues found in that scan. Use it to **track progress over time**, **compare releases**, and **show improvement trends** to stakeholders—**together** with issue detail and any manual testing your program requires. - -**Not legal or WCAG certification** -The score is **not** the same as WCAG conformance sign-off, VPAT completion, or legal accessibility certification. A score of 100 means **no automated findings were detected**—automated tools catch roughly 30–40% of WCAG issues, so a manual audit is still required for full conformance. Always interpret the score with **[Issue Summary](/support/docs/accessibility-testing-dashboard-issue-summary/)**, **[All Issues](/support/docs/accessibility-testing-dashboard-all-issues/)**, and your own manual coverage ([compliance guide](/support/docs/accessibility-compliance-guide/)). - -## Available across all accessibility products - -The score now uses the **same scoring model across every accessibility testing flow**, so results are directly comparable across platforms and test types: - -| Flow | Platform | -| --- | --- | -| Web Automated (Selenium, Playwright, Cypress) | Web | -| Web Manual (Accessibility DevTools) | Web | -| Web Scheduled (Test Scheduling) | Web | -| Native Android (Real Device) | Mobile | -| Native iOS (Real Device) | Mobile | - -The math is identical across platforms—only the rules that decide which elements are "meaningful" differ (DOM for web, `AccessibilityNodeInfo` for Android, `XCUIElement` for iOS). - -**Updated scoring model** -The Accessibility Score now uses an improved **density-adjusted** model. Earlier scores could be inflated by decorative and structural elements that trivially passed; the current model focuses on **meaningful, functional elements** only, so the number reflects the real state of the page. Scores are generally lower but more honest—a page with critical issues will not show as "Excellent." Tests run before this model was enabled continue to show the existing accessibility level as a fallback. - -## Who needs this? - -The Accessibility Score helps: - -- **Product managers** who need to quantify accessibility health and show progress to leadership -- **QA engineers** tracking improvements across sprints and releases -- **Accessibility teams** monitoring problem areas and regression risk -- **Developers** who want quick feedback on whether a change **improved or hurt** accessibility before merge - -If you ship digital products for real users, the score is a useful **signal**—not the only bar for "done." - -## Score bands - -Every score maps to a label and color band for an at-a-glance read of accessibility health: - -| Score range | Label | What it means | -| --- | --- | --- | -| **90–100** | **Excellent** | Few or no automated findings. In great shape from an automated-testing perspective; keep monitoring each release. | -| **70–89** | **Good** | Some issues found, but manageable in scope. Review and plan remediation in upcoming sprints. | -| **50–69** | **Needs Work** | Significant issues that likely affect users of assistive technologies. Prioritize fixing these. | -| **1–49** | **Poor** | Major accessibility barriers. Users with disabilities will face serious difficulty. Immediate remediation recommended. | -| **0** | **Keyboard scan** | A keyboard-only scan—no visual score is assigned. | - -Exact cutoffs are **guidance**; your product's risk tolerance may differ. - -## Where to find it - -Open any **completed** accessibility test in your TestMu AI **Accessibility** flow. You'll see: - -- The **overall score** for the run, its **label**, and a **color indicator** on the test report. -- A **per-page (web) or per-screen (mobile) score** below the overall score, so you can prioritize which pages to fix first. -- The same scores on **scheduled scans**, **[shared reports](/support/docs/accessibility-exporting-sharing-reports/)**, and **merged reports**. - -Start from **[Navigating the Dashboard](/support/docs/accessibility-testing-navigating-dashboard/)** if you are unsure how reports are listed. - -**Severity is a separate item from the score** -In the report, **Severity** (the accessibility level—Critical, Serious, Moderate, Minor) is a **distinct item** shown alongside the Accessibility Score, not part of the score number itself. Severity classifies **individual issues** by how badly they affect users; the **score** is the single aggregate, density-adjusted number for the whole page or screen. Severity *feeds into* how the score is weighted (see [Step 3](#step-3--calculate-deductions)), but the two are displayed and used independently—so a report can show a high-severity issue **and** its own separate score. In the API these map to the separate `accessibility_level` and `accessibility_score` fields. - -## How the score is calculated - -The Accessibility Score reflects **both** how many issues were found and **how severe** they are, adjusted for the **size** of the page or screen. It is a **density-adjusted deduction model**: it starts at a perfect **100** and deducts points based on the number, severity, and concentration of confirmed issues. The methodology is a **proprietary TestMu AI metric** designed for **trend and prioritization**—it is not a public WCAG scoring rubric. The same four steps run for every product. - -### Step 1 — Filter - -Not all issues count toward the score. The following are **excluded**: - -| Excluded from score | Reason | -| --- | --- | -| Best-practice issues | Recommendations, not WCAG requirements | -| Needs-review issues | Require manual verification—not confirmed failures | -| AI-generated rule findings | Inferred by AI, not confirmed by the rule engine | - -Only issues with a confirmed severity—**Critical**, **Serious**, **Moderate**, or **Minor**—contribute. - -### Step 2 — Deduplicate - -Issues are grouped by their **signature**: the rule that failed, the type of element it failed on, and the page or screen where it occurred. If 50 buttons on the same page fail the same contrast rule, they count as **one signature with 50 instances**, not 50 separate issues—so a single CSS bug can't disproportionately tank the score. - -### Step 3 — Calculate deductions - -Issues are weighted by severity: **Critical** issues have the largest impact, followed by **Serious**, **Moderate**, and **Minor**. When the same issue appears on multiple elements, each additional occurrence adds to the penalty, but with **diminishing returns** so one repeated pattern can't dominate the score. - -### Step 4 — Apply density adjustment - -This is what makes the score **page-size aware**. The same 2 critical issues have very different impact on a 12-element login form versus a 2000-element dashboard. Using the [scored element count](#scored-element-count) as the denominator: - -```text -Density = Total issue instances / Scored elements on the page -``` - -Higher density means issues are more concentrated, and deductions are multiplied accordingly—so small critical flows (login, checkout, payment) are correctly flagged as high priority while large pages with a few scattered issues are not over-penalized. - -### Final score - -The deductions are combined into the final score: - -```text -Score = max(1, floor(100 × e^(−AdjustedDeductions / 133))) -``` - -The minimum graded score is **1**. A score of **0** is reserved exclusively for **keyboard-only scans**. - -### Scored element count - -Density depends on the **scored element count**: the number of elements on a page or screen that are meaningful from an accessibility perspective (buttons, inputs, links, headings, images, and so on). Decorative, structural, and hidden elements are **excluded** so they don't inflate the score. - -A page with 1000 scored elements and 5 issues is in far better shape than one with 10 scored elements and 5 issues, and the score reflects that. - -## Per-product nuances - -The scoring math—filtering, dedup, severity weighting, density, and bands—is **identical everywhere**. The only thing that differs by product is **which elements are treated as "meaningful"** and a few flow-specific behaviors. Everything below lives in this single doc so you never have to look in more than one place. - -### Web (Automated, Manual/DevTools, Scheduled) - -Elements are evaluated from the **DOM**. Not counted toward the scored element count: - -- Structural containers without an ARIA role (``, ``) -- Script, style, and metadata tags (``, ``, ``, ``) -- Formatting elements (``, ``, ``) -- Decorative images (``—an empty alt means intentionally decorative) -- Hidden elements (`display:none`, `visibility:hidden`, `aria-hidden="true"`) -- Elements with `role="presentation"` or `role="none"` -- Table structure elements (``, ``, ``) -- Container frames (``—content inside is scored separately) -- ``, ``, `` without an accessible name - -**Flow notes:** all three web flows use the same rules; **Scheduled** scans additionally track the score run-over-run over time, and a **keyboard-only scan** reports **0** (Keyboard scan) rather than a graded score. - -### Native Android (Real Device) - -Elements are evaluated from `AccessibilityNodeInfo`. Not counted: - -- Layout containers (`LinearLayout`, `ConstraintLayout`, `FrameLayout`, `RelativeLayout`, `ViewGroup`) -- Scroll containers (`ScrollView`, `RecyclerView`, `NestedScrollView`) -- Spacing elements (`Space`) -- Decorative `ImageView` without a `contentDescription` -- Non-displayed elements - -### Native iOS (Real Device) - -Elements are evaluated from `XCUIElement`. Not counted: - -- Container types (`XCUIElementTypeWindow`, `XCUIElementTypeOther`, `XCUIElementTypeGroup`) -- Scroll containers (`XCUIElementTypeScrollView`, `XCUIElementTypeTable`, `XCUIElementTypeCollectionView`) -- System elements (`XCUIElementTypeStatusBar`, `XCUIElementTypeKeyboard`) -- Decorative `XCUIElementTypeImage` without a label -- Non-visible elements - -### Always counted (overrides) - -Regardless of platform, an element is **always scored**—even if it would otherwise be excluded—when it is genuinely interactive or explicitly exposed to assistive tech: - -- It has an explicit ARIA role (e.g. ``) -- It is an `` **without** an `alt` attribute (the missing alt *is* the accessibility issue) -- It is clickable or editable (Android: `isClickable`, `isLongClickable`) -- It has `isImportantForAccessibility=true` (Android) - -Because these rules only change the **denominator**, scores stay directly comparable across web and mobile. - -## Worked examples - -These show how the same model plays out on each product. - -### Web — E-commerce checkout page - -**Setup:** 200 total elements, but only **80 are scored** (the rest are decorative wrappers, icons, and structural ``s). The scan found 1 critical missing label on the payment input, 8 serious button-contrast issues, and 3 serious link-contrast issues—**3 signatures, 12 instances**, affecting ~15% of scored elements. - -| Scoring model | Score | Label | -| --- | --- | --- | -| Old ratio-based | 95 | Excellent | -| **Density-adjusted** | **55** | **Needs Work** | - -The old score was inflated by 120 decorative elements that trivially passed. The new score correctly flags a checkout page where a customer cannot complete a purchase because the payment input has no label for screen readers. - -### Native mobile — App login screen - -**Setup:** 12 total elements, **all 12 scored** (no decorative elements on a login screen). The scan found 3 critical missing labels on the 3 input fields and 2 serious button-contrast issues—**2 signatures, 5 instances**, affecting ~42% of scored elements. - -| Scoring model | Score | Label | -| --- | --- | --- | -| Old ratio-based | 62 | Needs Work | -| **Density-adjusted** | **43** | **Poor** | - -42% of the login screen is critically broken—users literally cannot identify the form fields. This should be the first thing the team fixes; no user can log in without accessible input labels. - -### Web — Marketing landing page - -**Setup:** 2500 total elements, but only **400 are scored** (the rest are decorative SVGs, background images, and layout containers). The scan found 3 serious missing alt texts on hero images and 5 moderate heading-order issues—**2 signatures, 8 instances**, affecting ~2% of scored elements. - -| Scoring model | Score | Label | -| --- | --- | --- | -| Old ratio-based | 99 | Excellent | -| **Density-adjusted** | **84** | **Good** | - -The issues are real (3 hero images are invisible to screen readers) but dispersed across a large page. Fix them next sprint—no need to panic. - -### Same issues, different scores - -Density is why the **same 2 critical issues** produce different scores by page size: - -| Page | Scored elements | Density | Score | Label | -| --- | --- | --- | --- | --- | -| Login form | 10 | 20% | **54** | Needs Work | -| Settings page | 50 | 4% | **69** | Needs Work | -| Product catalog | 200 | 1% | **72** | Good | -| Large dashboard | 1000 | 0.2% | **74** | Good | - -## What impacts your score - -- **Severity distribution** — More weight on critical/serious issues lowers the score faster than many minor-only findings. -- **Issue density** — More issues **per scored element** lowers the score; a small critical flow is penalized more than a large page with the same issues. -- **Scored element count** — Larger, meaningful DOMs/screens change the density denominator so the score stays fair across page sizes. - -## When the score updates - -The score is not static. It recomputes automatically when the underlying issue data changes: - -- **Hiding an issue** (e.g. marking a known exception) recomputes the score from the remaining visible issues—the dashboard updates in real time. See **[Hide and Restore Issues](/support/docs/accessibility-hide-restore-issues/)**. -- **Restoring a hidden issue** recomputes again and the score decreases to include it. -- **Toggling "Needs Review"** updates issue counts but **does not change the score**, because needs-review items are excluded from scoring. - -## Improving your score - -Focus on **high-impact** changes: - -1. **Fix critical issues first** — They carry the most weight and usually block real users. -2. **Reduce density** — Systematic patterns (wrong component-library defaults, shared header issues) hurt more than one-off edge cases. -3. **Prioritize by page** — Use per-page/per-screen scores to fix the lowest-scoring critical flows (login, checkout, registration) first. -4. **Re-scan after meaningful fixes** — Compare score **and** issue lists run-over-run; use **[Exporting & Sharing Reports](/support/docs/accessibility-exporting-sharing-reports/)** for audit trails. - -For fix order and rule context, use the **[Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/)** and platform checklists: [Web](/support/docs/accessibility-web-what-we-cover/) · [iOS](/support/docs/accessibility-ios-what-we-cover/) · [Android](/support/docs/accessibility-android-what-we-cover/). - -## How to use the score in reviews (short checklist) - -1. Open the report from **[Navigating the Dashboard](/support/docs/accessibility-testing-navigating-dashboard/)** and note the score **vs** the previous comparable build. -2. Open **[Issue Summary](/support/docs/accessibility-testing-dashboard-issue-summary/)** to see **which severities and rules** moved. -3. Only then decide if the score alone is enough for a stakeholder update—or if you need **[Passed Test Cases](/support/docs/accessibility-passed-test-cases/)** and manual notes. - -## API response - -When the score is enabled, these fields appear in accessibility test API responses for **every product**: - -| Field | Type | Description | -| --- | --- | --- | -| `accessibility_score` | integer | The score from 0 to 100. 0 means a keyboard-only scan; 1–100 is the density-adjusted score. | -| `score_label` | string | `Excellent`, `Good`, `Needs Work`, `Poor`, or `Keyboard scan`. | -| `scored_element_count` | integer | Meaningful elements evaluated after excluding decorative/structural elements. | - -The fields appear at **two levels**: the **test level** (aggregate across all pages/screens) and the **per-scan level** (each URL on web, each screen on mobile). - -```json -{ -"test_info": { "test_id": "AUT_abc123", "status": "completed" }, -"accessibility_score": 72, -"score_label": "Good", -"scored_element_count": 1627, -"scan_info": [ -{ -"page_url": "https://example.com/login", -"issue_count": 5, -"accessibility_score": 43, -"score_label": "Poor", -"scan_id": "AUT_abc123_1" -}, -{ -"page_url": "https://example.com/dashboard", -"issue_count": 8, -"accessibility_score": 88, -"score_label": "Good", -"scan_id": "AUT_abc123_2" -} -] -} -``` - -If a test ran before the score was enabled, or `scored_element_count` is unavailable, these fields are omitted. The existing `accessibility_level` (Critical / Serious / Moderate / Minor) is still returned and can be used as a fallback. - -## FAQ - -**What does a score of 100 mean?** -No automated issues were detected by the scanning engine. It does **not** mean full WCAG conformance—automated tools catch roughly 30–40% of WCAG issues. A manual audit is still recommended. - -**Why is my score different from before?** -The old ratio-based score counted decorative and structural elements as "passing," which inflated results. The density-adjusted model only weighs meaningful, functional elements—so scores are generally lower but more honest. - -**Why don't I see a score on my test?** -The score requires `scored_element_count`, collected by newer versions of the scanning extensions and SDKs. Tests run before the feature was enabled show the existing accessibility level instead. - -**Does hiding issues change the score?** -Yes. Hiding an issue recomputes the score from the remaining visible issues, and the dashboard updates in real time. Restoring a hidden issue recomputes again. - -**Is the score the same across web and mobile?** -Yes. The formula is identical everywhere; only the exclusion rules that decide which elements count differ (web uses DOM rules, Android uses `AccessibilityNodeInfo`, iOS uses `XCUIElement`). - -**Does toggling "Needs Review" change the score?** -No. Needs-review items are excluded from scoring because they require manual verification. Toggling updates issue counts but not the score. - -**What is the minimum score?** -1. A score of 0 is reserved exclusively for keyboard-only scans. - -## Remember - -- The Accessibility Score is a **TestMu AI proprietary** metric; it is **unique to this platform** and is meant for **product and engineering prioritization**, not as a substitute for legal analysis. -- It does **not** replace manual testing, screen reader validation ([overview](/support/docs/screen-reader-on-accessibility/)), or documented compliance processes. - -## Related docs - -- [Issue Summary](/support/docs/accessibility-testing-dashboard-issue-summary/) -- [All Issues](/support/docs/accessibility-testing-dashboard-all-issues/) -- [Passed Test Cases](/support/docs/accessibility-passed-test-cases/) -- [Hide and Restore Issues](/support/docs/accessibility-hide-restore-issues/) -- [Exporting & Sharing Reports](/support/docs/accessibility-exporting-sharing-reports/) -- [Accessibility Compliance Guide (ADA / WCAG / EAA / 508)](/support/docs/accessibility-compliance-guide/) -- [VPAT and ACR evidence (customer-owned templates)](/support/docs/accessibility-vpat-report-generation/) diff --git a/static/docs/accessibility-web-what-we-cover.md b/static/docs/accessibility-web-what-we-cover.md deleted file mode 100644 index 97eb99e13..000000000 --- a/static/docs/accessibility-web-what-we-cover.md +++ /dev/null @@ -1,72 +0,0 @@ -# Web Accessibility Checklist - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Use this page to see **which WCAG-aligned checks the product supports** (with links to rule-level guidance) and a **manual test checklist** for areas that benefit from human review. For deeper remediation after a scan, use the [Web Rule Repository](/support/docs/accessibility-web-rule-repository/). - -**Other platforms:** [iOS checklist](/support/docs/accessibility-ios-what-we-cover/) · [Android checklist](/support/docs/accessibility-android-what-we-cover/) - -## Supported (rules in this product) - -Each row links to the rule doc for that check. Configuration may limit which WCAG version or level runs in your project. - -| WCAG | Rule | Doc | -|------|------|-----| -| 1.1.1 | Non-text Content | [Rule →](/support/docs/accessibility-web-rule-1-1-1-non-text-content/) | -| 1.3.1 | Info and Relationships | [Rule →](/support/docs/accessibility-web-rule-1-3-1-info-and-relationships/) | -| 1.3.4 | Orientation | [Rule →](/support/docs/accessibility-web-rule-1-3-4-orientation/) | -| 1.4.1 | Use of Color | [Rule →](/support/docs/accessibility-web-rule-1-4-1-use-of-color/) | -| 1.4.2 | Audio Control | [Rule →](/support/docs/accessibility-web-rule-1-4-2-audio-control/) | -| 1.4.3 | Contrast (Minimum) | [Rule →](/support/docs/accessibility-web-rule-1-4-3-contrast-minimum/) | -| 1.4.10 | Reflow | [Rule →](/support/docs/accessibility-web-rule-1-4-10-reflow/) | -| 1.4.11 | Non-text Contrast | [Rule →](/support/docs/accessibility-web-rule-1-4-11-non-text-contrast/) | -| 1.4.12 | Text Spacing | [Rule →](/support/docs/accessibility-web-rule-1-4-12-text-spacing/) | -| 2.1.1 | Keyboard | [Rule →](/support/docs/accessibility-web-rule-2-1-1-keyboard/) | -| 2.1.2 | No Keyboard Trap | [Rule →](/support/docs/accessibility-web-rule-2-1-2-no-keyboard-trap/) | -| 2.1.4 | Character Key Shortcuts | [Rule →](/support/docs/accessibility-web-rule-2-1-4-character-key-shortcuts/) | -| 2.2.1 | Timing Adjustable | [Rule →](/support/docs/accessibility-web-rule-2-2-1-timing-adjustable/) | -| 2.2.2 | Pause, Stop, Hide | [Rule →](/support/docs/accessibility-web-rule-2-2-2-pause-stop-hide/) | -| 2.3.1 | Three Flashes or Below Threshold | [Rule →](/support/docs/accessibility-web-rule-2-3-1-three-flashes/) | -| 2.4.1 | Bypass Blocks | [Rule →](/support/docs/accessibility-web-rule-2-4-1-bypass-blocks/) | -| 2.4.2 | Page Titled | [Rule →](/support/docs/accessibility-web-rule-2-4-2-page-titled/) | -| 2.4.3 | Focus Order | [Rule →](/support/docs/accessibility-web-rule-2-4-3-focus-order/) | -| 2.4.4 | Link Purpose (In Context) | [Rule →](/support/docs/accessibility-web-rule-2-4-4-link-purpose/) | -| 2.4.7 | Focus Visible | [Rule →](/support/docs/accessibility-web-rule-2-4-7-focus-visible/) | -| 2.4.11 | Focus Not Obscured (Minimum) | [Rule →](/support/docs/accessibility-web-rule-2-4-11-focus-not-obscured/) | -| 2.5.1 | Pointer Gestures | [Rule →](/support/docs/accessibility-web-rule-2-5-1-pointer-gestures/) | -| 2.5.2 | Pointer Cancellation | [Rule →](/support/docs/accessibility-web-rule-2-5-2-pointer-cancellation/) | -| 2.5.3 | Label in Name | [Rule →](/support/docs/accessibility-web-rule-2-5-3-label-in-name/) | -| 2.5.4 | Motion Actuation | [Rule →](/support/docs/accessibility-web-rule-2-5-4-motion-actuation/) | -| 2.5.7 | Dragging Movements | [Rule →](/support/docs/accessibility-web-rule-2-5-7-dragging-movements/) | -| 2.5.8 | Target Size (Minimum) | [Rule →](/support/docs/accessibility-web-rule-2-5-8-target-size/) | -| 3.2.1 | On Focus | [Rule →](/support/docs/accessibility-web-rule-3-2-1-on-focus/) | -| 3.2.2 | On Input | [Rule →](/support/docs/accessibility-web-rule-3-2-2-on-input/) | -| 3.2.6 | Consistent Help | [Rule →](/support/docs/accessibility-web-rule-3-2-6-consistent-help/) | -| 3.3.1 | Error Identification | [Rule →](/support/docs/accessibility-web-rule-3-3-1-error-identification/) | -| 3.3.2 | Labels or Instructions | [Rule →](/support/docs/accessibility-web-rule-3-3-2-labels-or-instructions/) | -| 3.3.7 | Redundant Entry | [Rule →](/support/docs/accessibility-web-rule-3-3-7-redundant-entry/) | -| 3.3.8 | Accessible Authentication (Minimum) | [Rule →](/support/docs/accessibility-web-rule-3-3-8-accessible-authentication/) | -| 4.1.1 | Parsing (obsolete in WCAG 2.2) | [Rule →](/support/docs/accessibility-web-rule-4-1-1-parsing/) | -| 4.1.2 | Name, Role, Value | [Rule →](/support/docs/accessibility-web-rule-4-1-2-name-role-value/) | -| 4.1.3 | Status Messages | [Rule →](/support/docs/accessibility-web-rule-4-1-3-status-messages/) | - -## Manual test checklist - -Use this checklist for **quality beyond automated rules**: keyboard journeys, media, language, and judgment-heavy success criteria. Pair it with [WCAG 2.2 Understanding](https://www.w3.org/WAI/WCAG22/Understanding/) and your own test plan. - -| Area | Manual verification focus | -|------|---------------------------| -| **1.2.x Time-based media** | Captions, audio description, transcripts, sign language. Exercise with real media workflows and tooling. | -| **1.3.2 Meaningful sequence · 1.3.3 Sensory characteristics · 1.3.5 Identify input purpose** | Reading order, instructions, autofill semantics. Design and assisted-tech review. | -| **1.4.4 Resize text · 1.4.5 Images of text · 1.4.13 Content on hover or focus** | Zoom, images-of-text, hover layers. Interactive and visual testing. | -| **Full keyboard operability & real focus behavior** | Automated rules support you; still walk critical paths with keyboard only and assistive technologies. | -| **2.4.5–2.4.6, 2.4.8–2.4.10** | Multiple ways, headings/labels (extended), location. Document expectations and verify in session. | -| **2.2.3–2.2.6 · 2.3.2–2.3.3** | Extended timing and motion. Validate against your UX and policy. | -| **3.1.x Language of parts, unusual words, reading level** | Localization and plain language. Content and engineering review. | -| **3.3.3–3.3.6 Error suggestion / prevention** | Error helpfulness and legal safeguards. Review with design and compliance. | -| **AAA success criteria** | Optional stretch goals. Plan explicitly if AAA is in scope; default automation targets A/AA-style coverage. | - -## Related docs - -- [Web Rule Repository](/support/docs/accessibility-web-rule-repository/) -- [Accessibility Issue Remediation Guide](/support/docs/accessibility-issue-remediation-guide/) diff --git a/static/docs/accessibility-web-what-we-do-not-cover.md b/static/docs/accessibility-web-what-we-do-not-cover.md deleted file mode 100644 index 86c98edfe..000000000 --- a/static/docs/accessibility-web-what-we-do-not-cover.md +++ /dev/null @@ -1,3 +0,0 @@ -# Web Accessibility Manual Checklist - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). diff --git a/static/docs/account-management.md b/static/docs/account-management.md deleted file mode 100644 index 45acefe64..000000000 --- a/static/docs/account-management.md +++ /dev/null @@ -1,36 +0,0 @@ -# Account Management - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - - Manage Subscriptions - Learn how to manage your account subscription for the TestMu AI. - - Service Accounts - Learn how to Monitor and Report Test Execution at the user level. - - Sub Organizations - Learn how to distribute concurrency to different sub-organizations . - - Support Access - Learn how to take access to your TestMu AI account and log in on your behalf from the backend. - - Team Management - Learn how to invite your team members to your organizational TestMu AI Account. - - Multi Factor Authentication - Learn how to increase the account security by enabling Multi Factor Authentication. - - Deleting TestMu AI Account - This documentation will guide you through the steps to delete your TestMu AI account - - Group Management - Learn how to perform the test automation by dividing the concurrency among the groups. - - Audit Logs - Learn how to Audit your Logs as well as of your Organization Team Members. - - Password and Access Key Expiration Policy - Learn how to manage your and yours organization member Password and Access Key Expiration Policy. - - Tunnel Security - Learn how to manage your Tunnel Setup and Securities. diff --git a/static/docs/adb-commands-support.md b/static/docs/adb-commands-support.md deleted file mode 100644 index 443123ae8..000000000 --- a/static/docs/adb-commands-support.md +++ /dev/null @@ -1,326 +0,0 @@ -# ADB Commands Support - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Android Debug Bridge (adb) is a versatile command-line tool that lets users communicate with a device. The adb command facilitates a variety of device actions, such as installing and debugging apps. - -TestMu AI provides support for limited commands which can be executed in your app automation test scripts via javascript executors. The following command with the below mentioned parameters need to be used to execute adb command with TestMu AI real device cloud. - -```python -driver.execute_script("lambda-adb", params) -``` - -## Supported Commands - -> **NOTE :** All these commands are supported on both Real and Virtual Devices except for the [**adb shell dumpsys**](/support/docs/adb-commands-support/#adb-shell-command) command, which is not supported for Virtual Device. - -### Swipe -**adb shell input swipe** - - The command is used to generate a swipe gesture by defining the coordinates of starting and ending point of the swipe. The following is a Python sample of using the adb swipe command with TestMu AI executor. - -```python -params = {"command":"input-swipe", "startX":200,"startY":900,"endX": 200,"endY":300} -result = driver.execute_script("lambda-adb",params) -``` - -### Clipboard - -**adb shell input sendKeys** - - The command is used to sends text as if typed at the keyboard in the real devices. The following is a Python sample of using the adb sendKeys command with TestMu AI executor. - -```python -params = {"command":"input-text", "text":"this is my text"} -result = driver.execute_script("lambda-adb",params) -``` - -### Home Button - -- **Navigate to the home screen** - -The command is used to navigate to the home screen of the device while running an app automation test script. The following is a Python sample of using the adb command to navigate to the home screen with TestMu AI executor. - -```python -params = {"command":"keyevent", "keycode":3} -result = driver.execute_script("lambda-adb",params) -``` - -### Enable/Disable auto rotate - -- **enableAutoRotate** - - The command is used to auto rotate the screen of the device while running an app automation test script. The following is a Python sample using the adb command to allow auto rotation with TestMu AI executor. - -```python -params = {"command":"autorotate", "enableAutoRotate": True} -result = driver.execute_script("lambda-adb",params) -``` - -### ADB Shell Command - -TestMu AI allows execution of ADB shell commands during automated test runs. - -#### ADB Shell Command - Public - -- **adb shell dumpsys** - - The `adb shell dumpsys` command is used to obtain detailed information about installed packages on device. When you run this command, it provides a list of information for each package installed on the device. The output includes various details about each package.The following is a Python sample using the adb command: - -```python -params = {"command": "shell", "text": "dumpsys package "} -result = driver.execute_script("lambda-adb",params) -``` - Example - -```python -params = {"command": "shell", "text": "dumpsys package dumpsys input_method"} -result = driver.execute_script("lambda-adb",params) -``` - -- **adb shell getprop** - - The command `adb shell getprop` is used to retrieve system properties from an Android device. When executed, it provides a list of key-value pairs representing various system settings and configurations. These properties include information about the device's build, hardware, and other system-related details. The output can be useful for debugging, development or understanding the device's current state. The following is a Python sample using the adb command: - -```python -params = {"command": "shell", "text": "getprop"} -result = driver.execute_script("lambda-adb",params) -``` - Example - -```python -params = {"command": "shell", "text": "getprop ro.build.version.security_patch"} -result = driver.execute_script("lambda-adb",params) -``` - -- **adb shell cat /proc/version** - - The `adb shell cat /proc/version` command outputs detailed information about the Linux kernel version running on the Android device, including the build date and compiler details. This information is essential for debugging compatibility issues and understanding the device’s operating system internals.The following is a Python sample using the adb command: - -```python -params = {"command": "shell", "text": "cat /proc/version"} -result = driver.execute_script("lambda-adb",params) -``` - -#### ADB Shell Command - Private - -- **adb shell pm** - - The `adb shell pm` command enables management of Android apps programmatically. It supports installing, uninstalling, clearing data, and querying installed packages. The following is a Python sample using the adb command: - -```python -params = {"command": "shell", "text": "pm list packages"} -result = driver.execute_script("lambda-adb",params) -``` - -- **adb shell rm** - - The `adb shell rm` command is used to delete files or directories from the Android device's filesystem directly from your terminal or during automated test execution. It helps in clearing residual data, removing temporary test files, or resetting the test environment by deleting specific logs, APKs, screenshots, or other generated data. The following is a Python sample using the adb command: - -```python -params = {"command": "shell", "text": "rm /sdcard/Download/tempfile.txt"} -result = driver.execute_script("lambda-adb",params) -``` - -- **adb shell mkdir** - - The `adb shell mkdir` command is used to create new directories on an Android device’s filesystem. It is especially useful when preparing the device environment before automated test runs, ensuring that the required folder structure exists for storing screenshots, logs, or other test-related files. The following is a Python sample using the adb command: - -```python -params = {"command": "shell", "text": "mkdir /sdcard/TestResults"} -result = driver.execute_script("lambda-adb",params) -``` - -- **adb shell screencap** - - The `adb shell screencap` command captures the current screen on a real device and saves it as an image file. It’s useful for debugging UI issues or validating visual test results during automation. The following is a Python sample using the adb command: - -```python -params = {"command": "shell", "text": "screencap /sdcard/screen.png"} -result = driver.execute_script("lambda-adb",params) -``` - -- **adb shell content** - - The `adb shell content` command interacts with Android’s content providers to query, insert, update, or delete shared data like contacts, SMS, or calendar entries. It's commonly used in tests to read data, insert test entries, or reset content to a known state. The following is a Python sample using the adb command: - -```python -params = {"command": "shell", "text": "content query --uri content://contacts/phones"} -result = driver.execute_script("lambda-adb",params) -``` - -- **adb shell am** - - The `adb shell am command` uses the Activity Manager to control app components on a real device. It’s useful for launching activities, restarting apps, or sending broadcast intents, commonly done during tests to simulate user actions or reset app state. The following is a Python sample using the adb command: - -```python -params = {"command": "shell", "text": "am start -n com.example/.MainActivity"} -result = driver.execute_script("lambda-adb",params) -``` -- **adb shell dumpsys** - - The `adb shell dumpsys` command is used to obtain detailed information about installed packages on device. When you run this command, it provides a list of information for each package installed on the device. The output includes various details about each package.The following is a Python sample using the adb command: - -```python -params = {"command": "shell", "text": "dumpsys package "} -result = driver.execute_script("lambda-adb",params) -``` - Example - -```python -params = {"command": "shell", "text": "dumpsys package dumpsys input_method"} -result = driver.execute_script("lambda-adb",params) -``` - -- **adb shell getprop** - - The `adb shell getprop` command retrieves system-level properties in the form of key-value pairs. These properties include internal runtime details such as device model, manufacturer, OS version, network state, security patch level, and debug flags. The following is a Python sample using the adb command: - -```python -params = {"command": "shell", "text": "getprop"} -result = driver.execute_script("lambda-adb",params) -``` - -- **adb shell setprop** - - The `adb shell setprop` command sets system properties on a real device, allowing you to modify device behavior. It’s often used to change debug flags, adjust logging levels, or simulate different configurations and network conditions during testing. The following is a Python sample using the adb command: - -```python -params = {"command": "shell", "text": "setprop debug.test true"} -result = driver.execute_script("lambda-adb",params) -``` -- **adb shell cat /proc/version** - - The `adb shell cat /proc/version` command outputs detailed information about the Linux kernel version running on the Android device, including the build date and compiler details. This information is essential for debugging compatibility issues and understanding the device’s operating system internals.The following is a Python sample using the adb command: - -```python -params = {"command": "shell", "text": "cat /proc/version"} -result = driver.execute_script("lambda-adb",params) -``` - -- **adb shell ls** - - The `adb shell ls` command lists files and directories within a specified folder on the Android device. It is commonly used during automated tests to verify the presence and contents of files created, modified, or downloaded by the app. This helps ensure test artifacts are correctly generated and stored on the device. - - #### These are the `ls` commands that can are enabled: - - - `ls /sdcard/Download`: Lists all files and directories inside the Downloads folder. Typically used to check if test downloads or app-generated files exist, and to validate proper file creation or cleanup. - - - `ls /sdcard/Pictures`: Lists all files in the Pictures directory, which usually stores photos and screenshots. Useful for confirming screenshots are saved correctly or for accessing images created during tests. - - - `ls /sdcard/Movies`: Lists media files in the Movies folder. Commonly used to validate recorded or downloaded videos and manage video artifacts from test executions. - -```python -# List files in the Downloads folder -params = {"command": "shell", "text": "ls /sdcard/Download"} -result = driver.execute_script("lambda-adb", params) -# List files in the Pictures folder -params = {"command": "shell", "text": "ls /sdcard/Pictures"} -result = driver.execute_script("lambda-adb", params) -# List files in the Movies folder -params = {"command": "shell", "text": "ls /sdcard/Movies"} -result = driver.execute_script("lambda-adb", params) -``` - -- **adb shell cat** - - The `adb shell cat` command in Android's shell outputs a file’s contents to the console, allowing quick access to text or binary data on the device. Short for “concatenate,” it’s a common Linux command used to read files. In automation, cat helps inspect logs, reports, images, or videos without manual device access. Text files show readable content, while binary files output raw data that may need special handling. - - #### These are the cat commands that can are enabled: - - `cat /sdcard/Download/`: Outputs the contents of a file in the Downloads folder. Typically used to read logs, reports, or downloaded test artifacts. - - - `cat /sdcard/Pictures/`: Outputs the raw binary data of an image file in the Pictures folder, useful for verifying screenshots or photos. - - - `cat /sdcard/Movies/`: Outputs raw binary content of video files in the Movies folder, helpful for validating recorded or downloaded videos. - -```python -params = {"command": "shell", "text": "cat /proc/version"} -result = driver.execute_script("lambda-adb", params) -# Read a log or report from Downloads -params = {"command": "shell", "text": "cat /sdcard/Download/test_log.txt"} -result = driver.execute_script("lambda-adb", params) -# Read an image file (raw binary) from Pictures -params = {"command": "shell", "text": "cat /sdcard/Pictures/screenshot.png"} -result = driver.execute_script("lambda-adb", params) -# Read a video file (raw binary) from Movies -params = {"command": "shell", "text": "cat /sdcard/Movies/test_video.mp4"} -result = driver.execute_script("lambda-adb", params) -``` - -### Additional ADB Shell Commands - -- **adb shell ping -c 4 YOUR_URL** - - This command is used to test the network connectivity between the Android device and a specified host, such as `google.com`. When executed it sends four ICMP request packets to the host and wait for responses. The output includes details such as packet transmission time, success rate and round-trip time, which helps diagnose network connectivity and latency issues. The following is a Python sample using the adb command: - - > **Note :** The **-c** is required in this command -```python -params = {"command": "shell", "text": "ping -c 4 google.com"} -result = driver.execute_script("lambda-adb",params) -``` - -### Enable/Disable Notification - -- **enableNotification** - - These commands enable or disable your app notifications on the device based on the value provided for `enableNotification`. **True** is used to enable notifications, while **False** is used to disable them. The following is a Python sample using the adb command with TestMu AI executor: - -```python -params = {"command": "enable-notification", "enableNotification":True/False} -result = driver.execute_script("lambda-adb",params) -``` - - - These commands are compatible with Android versions 13 and above. - - When you disable app notification permissions using the command, the app go into the background. This behavior occurs due to limitations with ADB, and you need to reactivate the app through test script. - -### Enable/Disable Battery Optimization - -- **disableBatteryOptimization** - - This command is used to enable or disable battery optimization for your app on the device. The following is a Python sample using the adb command to disable battery optimization with TestMu AI executor: - -```python -params = {"command":"disable-battery-optimization", "disableBatteryOptimization":True/False} -result = driver.execute_script("lambda-adb",params) -``` - -### Fixed-to-User Rotation - -- **fixedToUserRotation** - - This command serves to lock the screen rotation in alignment with the app's default behavior and user-defined settings. Below is a Python example utilizing the adb command to enforce fixed screen rotation with the TestMu AI executor: - -```python -params = {"command":"fixed-to-user-rotation", "fixedToUserRotation":True} -result = driver.execute_script("lambda-adb",params) -``` - -### Disabling Animations on Android Devices -Disabling animations can enhance test execution speed by preventing unnecessary visual effects. The following commands are used to disable different types of animations: - -- `adb shell settings put global animator_duration_scale 0`: Disables property animation effects, such as object transitions and transformations. -- `adb shell settings put global transition_animation_scale 0`: Disables window transition animations, making UI transitions instant. -- `adb shell settings put global window_animation_scale 0`: Disables activity window animations, eliminating fade-in or zoom-in effects. - - These commands set the respective global animation scales to 0, effectively disabling animations. - -```java -Map params = Map.of("command", "shell", "text", "settings get global animator_duration_scale"); -androidDriver.executeScript("lambda-adb", params); -``` - -### Enabling Automatic Timezone and Automatic Time Using ADB -By default, Automatic Timezone and Automatic Time settings are enabled on the device. These settings ensure that the device updates its time and timezone automatically based on network-provided information. If needed, you can manually enable these settings using ADB shell commands. To enable these settings via ADB, run the following commands: - -- `settings put global auto_time_zone 1`: Enables Automatic Timezone, allowing the device to adjust its timezone based on network data. -- `settings put global auto_time 1`: Enables Automatic Time, ensuring that the device syncs time automatically from the network. - -These commands help maintain accurate time and timezone settings without requiring manual intervention. If you're using Python with TestMu AI’s App Automation, you can execute these commands using the `execute_script` function: - -```python -# Define ADB commands for enabling automatic time and timezone -params_timezone = {"command": "shell", "text": "settings put global auto_time_zone 1"} -params_time = {"command": "shell", "text": "settings put global auto_time 1"} - -# Execute commands using LambdaTest's ADB execution method -result_timezone = driver.execute_script("lambda-adb", params_timezone) -result_time = driver.execute_script("lambda-adb", params_time) -``` -This approach is particularly useful when testing applications that rely on correct time settings, ensuring they function properly across different time zones. diff --git a/static/docs/advanced-tunnel-features.md b/static/docs/advanced-tunnel-features.md deleted file mode 100644 index 300ecb22f..000000000 --- a/static/docs/advanced-tunnel-features.md +++ /dev/null @@ -1,139 +0,0 @@ -# Advanced TestMu AI Tunnel Features - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -**TestMu AI Tunnel** feature allows you to test your **private server URLs** or **locally hosted web-apps** or **websites** on **3000+** real browsers through TestMu AI. However, sometimes corporate firewalls and proxy settings may have restricted you to leverage the TestMu AI Tunnel binary. Not anymore though, as we’ve come up with a new binary for TestMu AI Tunnel. TestMu AI Tunnel follows various protocols such as **Web Socket, TCP** etc. to help you establish a secure and unique tunnel connection between your system and TestMu AI cloud servers. - -You can download the **TestMu AI Tunnel binary** that will help you establish a secure connection through corporate firewalls between your computer and [LambdaTest](https://www.testmuai.com/) cloud servers for a testing locally hosted website or web-applications. You can test plain **HTML, CSS, PHP, Python** or other similar web files saved on your local system, over combinations of operating systems, browsers, and screen resolutions that are available on TestMu AI. - -| Download Links | -| ---------------------------------------------------------------------------------- | -| [Windows](https://downloads.lambdatest.com/tunnel/v3/windows/64bit/LT_Windows.zip) | -| [macOS](https://downloads.lambdatest.com/tunnel/v3/mac/64bit/LT_Mac.zip) | -| [Linux](https://downloads.lambdatest.com/tunnel/v3/linux/64bit/LT_Linux.zip) | -| [FreeBSD](https://downloads.lambdatest.com/tunnel/v3/freebsd/64bit/LT_Freebsd.zip) | - -## Executing TestMu AI Tunnel for Client Connection -After you download the zip file for your operating system, extract it in a folder and open you command line there. Once you have your terminal routed to the correct directory where the TestMu AI Tunnel binary file is placed, you need to execute the below command. - -```bash title="Format" -LT --user {Your Registered Email ID} --key {Your LambdaTest Access Key} --tunnelName {any random string} -``` - -So for example, if your details are as below: - -| Parameters | Values | -| ------------------------- | ---------------------- | -| **Email** | example@lambdatest.com | -| **TestMu AI Access Key** | 123asd123 | -| **Tunnel Name** | SampleTunnel | - -Then your command would be: - -```bash -LT --user example@lambdatest.com --key 123asd123 --tunnelName SampleTunnel -``` - -Once you execute the command, you will successfully establish a client connection using the **TestMu AI Tunnel**. You will find the command logs to state, that you are now ready to test. - -## What Makes The New TestMu AI Tunnel Binary Special? -Well, other than the fact that you can now establish a connection through your corporate firewalls with the new TestMu AI Tunnel binary. Here are a couple things to top it off - - -### Auto Update Functionality -Earlier, for every version update in out TestMu AI Tunnel, you were compelled to download the latest binary from our platform and over write it over the outdated version in your computer. Well, now this new binary will take care of that. Every time you execute this binary, it will check for the **latest** version available and will update itself **automatically**, in case it gets outdated. - -### Default Port 443 -Now, by default, everything will run over the **port 443** to ensure a secure web browser communication through **http protocol** over **TLS/SSL**. - -### Leverage `.lt.yaml` file -With this new TestMu AI Tunnel binary, you can declare your TestMu AI authentication credentials in a **YAML** file configuration and keep it in the same directory as the **LT binary file**. That way, you won’t have to pass the environment variables in the cmd every time you wish to configure the TestMu AI Tunnel. Once you specify these variables in the `.lt.yaml file`, you will just have to execute the binary file through cmd **LT.exe** and the YAML file will automatically configure a secure TestMu AI Tunnel connection by auto detecting the variables specified in the YAML file. - -Here is an example of the `.lt.yaml` file. - -```yaml -User: salmank -Key: 123456789abcdefghijklmnopqrstuv -TunnelName: LambdaTest -``` - -> **Note:** You will need to replace this file with your credentials and it has to be named exactly "`.lt.yaml`". Once you specify the proxy information as environment variable, it gets auto detected. - -Similarly, you can go ahead and pass any other variables by just specifying them in the YAML file. For example, if you wish to have verbose variable passed on for detailed logs while the binary is being configured. You will add the verbose flag in your YAML file: - -```yaml -User: salmank -Key: 123456789abcdefghijklmnopqrstuv -TunnelName: LambdaTest -Verbose: True -``` - -Now, when you trigger the binary file through cmd. You will have your verbose logs populated automatically, without you having to specify the variable every time. - -### Local Testing By MITM (Man-In-The-Middle) -The **MITM(Man-in-the-middle)** mode enables you to test websites using self-signed certificates on your local system or internal network. It happens very often that you may try to test a website on the localhost which may not have valid SSL certificates before the website is made live. In such cases, you may receive the below error. - -You can test such websites by leveraging the **MITM** mode. The command will look like this: - -`LT --user salmank@lambdatest.com --key 123456789abcdefghijklmnopqrstuv --tunnelName LambdaTest --mitm` - -Here is a screenshot of the same website that was throwing an error earlier but can now be tested by running the **MITM** mode. - -### Using The Tunnel InfoAPIs -By using the tunnel **Info APIs**, you can fetch the current status of the tunnel and can use it to stop the tunnel. You can fetch the current tunnel status using the Info API on the tunnel. Suppose the **InfoAPI** is available on the host over port **8000**, then use the below command to infuse the **InfoAPI** in the tunnel. - -```bash -LT --user salmank@lambdatest.com --key 123456789abcdefghijklmnopqrstuv --tunnelName LambdaTest --infoAPIPort 8000 -``` - -In order to fetch the current tunnel status, execute the below command: - -```bash -curl http://127.0.0.1:8000/api/v1.0/info -``` - -To stop the current tunnel, execute the below command: - -```bash -curl -X DELETE http://127.0.0.1:8000/api/v1.0/stop -``` - -### Tunnel Logs -The --log-level flag in the TestMu AI CLI is used to specify the desired log level for tunnel logs. This feature enables users to control the verbosity of logs generated during tunnel operations, making it easier to debug or monitor activities as needed. - -```bash ---log-level YOUR_LOG_LEVEL -``` - -#### Supported Log Levels: -- **info :** Provides informational messages about the general operation of the tunnel. -- **warn :** Highlights potential issues that might not immediately affect functionality but require attention. -- **debug :** Outputs detailed logs, including diagnostic information, for troubleshooting purposes. -- **error :** Displays error messages when something goes wrong in the tunnel operation. -- **fatal :** Shows critical issues that cause the tunnel to terminate unexpectedly. - -## AllowHost In Tunnel -With the latest tunnel binary release we have introduced the capability to provide the domains which will be resolved from tunnel binary and the rest of the urls will be resolved from our servers. This can speed up the execution of test as the latency will be reduced by transferring data from the tunnel client. You may also use this to restrict the traffic flowing from user’s network. - -**Usage:** - -```bash -–allowHosts -``` - -```bash title="Example" -LT --user --key –allowHosts google.com,apple.com,amazon.com -``` - -**Explanation:** -When this flag is used only requests for provided domains will be routed via tunnel and resolved from the user's network. Requests for domains other than mentioned will be resolved from Lambdatest’s network. - -## Tunnel Arguments -You can find all the arguments for TestMu AI Tunnel by running the below command in your command line: - -```bash -LT --help -``` - -> **Note:** For all modifiers/arguments, refer to the list of [TestMu AI Tunnel Modifiers](/docs/lambda-tunnel-modifiers/). - -That was all you need to know for configuring a client connection through TestMu AI Tunnel. In case you have any questions, feel free to share them with us through our **24/7 chat support** or drop us an email to [support@testmuai.com](mailto:support@testmuai.com). Happy testing! 🙂 diff --git a/static/docs/agent-features-and-metrics.md b/static/docs/agent-features-and-metrics.md deleted file mode 100644 index 6c911c9d3..000000000 --- a/static/docs/agent-features-and-metrics.md +++ /dev/null @@ -1,610 +0,0 @@ -# Agent Features & Metrics - Customer Reference Guide - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Agent Types Overview - -The platform supports **5 agent types**, each designed for a specific testing scenario: - -| Agent Type | Primary Use Case | Key Differentiator | -|-----------|-----------------|-------------------| -| **Chat** | Test text-based chatbot agents | Multi-turn text conversation evaluation with 9 quality metrics | -| **Voice** | Test chatbot agents via audio conversations | Same as Chat, but conversations happen as audio (WAV) instead of text | -| **Phone Caller Inbound** | Test voice agents that receive calls | **Pre-evaluation** (live simulated calls) + **Post-evaluation** (production recording analysis) | -| **Phone Caller Outbound** | Test voice agents that make calls | **Pre-evaluation** (live outbound calls) + **Post-evaluation** (production recording analysis) | -| **Image Analyzer** | Validate AI-generated images | Image quality scoring against prompts and brand guidelines | - -## Chat Agent - -#### 📄 Workflow-Based Test Generation - -Connect your knowledge sources and let the platform auto-generate test scenarios, no manual scripting needed. - -- **Document Upload**: Upload knowledge base documents (PDF, text files) that your chatbot is expected to understand -- **Source Integrations**: Connect Confluence, JIRA, or GitHub as knowledge sources -- **AI Test Generation**: Automatically generates test scenarios from uploaded documents -- **Real-time Progress**: Live streaming of test generation progress - -#### 🎭 Scenario Management - -Build and manage the exact conversations you want to test, manually or via AI. - -- **Manual Scenario Creation**: Create test scenarios with title, description, and expected behavior -- **AI-Generated Scenarios**: Auto-generate scenarios from uploaded knowledge sources -- **Validation Criteria**: Define custom pass/fail criteria per scenario (e.g., "Agent must mention return policy", "Agent should not hallucinate product prices") -- **Special Instructions**: Add specific instructions to guide scenario execution -- **Persona Assignment**: Assign user personas to scenarios (e.g., "frustrated customer", "first-time user") -- **Test Profile Association**: Link test data profiles to scenarios for data-driven testing -- **Scenario Deletion**: Remove scenarios that are no longer needed - -#### 🗂️ Test Suites - -Group related scenarios together and track results over time. - -- **Suite Creation**: Group multiple scenarios into test suites -- **Test Profile Selection**: Assign a test data profile to the entire suite -- **Run History**: View all past runs with status, score, and timestamps -- **Status Filtering**: Filter results by Passed, Failed, In Progress - -#### 🔌 Endpoint Profiles - -Configure how the platform connects to your agent's API, supporting everything from simple REST calls to multi-phase auth flows. - -- **Postman Collection Import**: Upload Postman collections (with optional environment files) to configure your agent's API. Supports nested folders, collection variables, and environment variable substitution -- **Manual Configuration**: Define endpoints via JSON with URL, method, headers, and body -- **Multi-Phase Execution**: Configure suite_setup (login/auth), scenario_setup (session creation), and chat (conversation) phases -- **Variable Management**: Define static values, auto-generated values (UUID, mobile number, timestamp, email), and extracted values from API responses -- **Retry & Caching**: Configure retry-on-failure and cache suite setup results to speed up execution -- **Test Endpoint**: Dry-run your endpoint profile to verify connectivity before running full evaluations -- **Import/Export**: Export profiles as JSON and import across projects -- **Default Profile**: Mark one profile as the default for quick evaluation runs - -#### 🗃️ Test Profiles (Test Data) - -Create reusable data sets to power data-driven testing across multiple scenarios. - -- **Custom Key-Value Data**: Create reusable test data profiles with typed fields (string, number, boolean, email, URL, JSON) -- **Default Profile**: Mark one profile as the default -- **Import/Export**: Share test profiles across projects via JSON export/import -- **Data Injection**: Test data is injected at runtime for data-driven scenario execution - -#### 🧪 Playground - -Interactively test your agent configuration before running a full evaluation suite. - -- **Interactive Chat**: Send messages to your configured agent endpoint in real-time -- **Multi-Turn Conversations**: Maintain conversation context across multiple messages -- **Connection Testing**: Test connectivity via cURL command verification -- **Schema Analysis**: Automatic detection of request/response schema from your endpoint - -#### ⚡ Evaluation Execution - -Run evaluations at scale with real-time feedback on your agent's quality. - -- **Metric Selection**: Choose which quality metrics to evaluate (or run all) -- **Endpoint Profile Selection**: Pick which endpoint profile to evaluate against -- **HyperExecute Integration**: Run evaluations at scale using LambdaTest's HyperExecute infrastructure with optional tunnel configuration for testing agents behind firewalls or private networks -- **Real-Time Streaming**: Live progress updates during evaluation via Server-Sent Events - -#### 🎚️ Metric Threshold Configuration - -Define exactly what "passing" means for your project, then enforce it automatically. - -- **Per-Metric Thresholds**: Set minimum acceptable score (0.0–1.0) for each metric -- **Higher/Lower is Better**: Configure directionality for each metric -- **Named Configurations**: Create named threshold configs (e.g., "Strict", "Default") -- **Active/Inactive Toggle**: Enable or disable threshold configurations - -#### 🚦 Go-Live Assessment - -Get a clear, defensible production-readiness verdict before you ship. - -**Production Readiness Verdicts** -| Verdict | Meaning | -|---------|---------| -| 🟢 **GREEN** | Ready for production | -| 🟡 **YELLOW** | Ready with caveats | -| 🔴 **RED** | Not ready | - -- **Overall Score**: Weighted composite score (0–100) -- **Confidence Level**: Based on number of evaluations run - - HIGH: 100+ evaluations - - MEDIUM: 50–99 evaluations - - LOW: 20–49 evaluations - - VERY_LOW: Fewer than 20 evaluations -- **Dimension Scores**: Functional Completeness, Quality Standards, Risk Profile, Operational Readiness (each weighted 25%) -- **Scenario Coverage**: Matrix showing well-tested vs. untested scenarios -- **Risk Assessment**: AI-powered failure pattern analysis with prioritized action items -- **Validation Criteria Summary**: Aggregated compliance rate across all criteria -- **AI Insights**: Actionable recommendations for improvement - -#### 🗓️ Scheduled Runs - -Automate ongoing regression coverage without manual intervention. - -- **Cron-Based Scheduling**: Schedule evaluations with preset frequencies (Hourly, Daily, Weekdays, Weekly, Monthly) or custom cron expressions -- **Timezone Support**: Full IANA timezone selection -- **Pause/Resume**: Temporarily pause and resume scheduled runs -- **Run History**: Track all scheduled execution results - -Chat agents are evaluated on **9 quality metrics**, each scored on a **0–100% scale**. - -| # | Metric | What It Measures | -|---|--------|-----------------| -| 1 | **Bias Detection** | Detects biased, discriminatory, or unfair responses | -| 2 | **Hallucination Detection** | Identifies false, fabricated, or unsupported information | -| 3 | **Completeness** | Whether the agent fully addresses the user's question | -| 4 | **Context Awareness** | How well the agent understands and maintains conversation context | -| 5 | **Response Quality** | Overall quality, clarity, and helpfulness of responses | -| 6 | **Conversation Flow** | Natural flow and coherence across multi-turn conversations | -| 7 | **User Satisfaction** | Estimated end-user satisfaction based on conversation quality | -| 8 | **File Handling Quality** | Quality of file upload/download interactions (if applicable) | -| 9 | **File Generation Accuracy** | Accuracy of generated files against requirements (if applicable) | - -**Every evaluation result includes:** - -- Overall Score (weighted average of all metrics) -- Per-metric score with pass/fail badge (based on configured thresholds) -- Per-metric detailed analysis text -- Multi-turn conversation transcript (User/Agent exchanges) -- Strengths identified in the conversation -- Areas for improvement -- Actionable recommendations -- Validation criteria results (Pass/Fail/Unable to Verify per criterion with evidence and confidence level) - -## Voice Agent - -The Voice agent is functionally **identical to the Chat Agent**, with one key difference: conversations happen as **audio (WAV)** instead of text. The platform conducts voice-based conversations with your agent and evaluates the audio interaction using the same quality metrics. - -All features from the Chat Agent are available, including: - -- Workflow-based test generation with document upload and source integrations -- Scenario management with AI generation, validation criteria, personas, and special instructions -- Test suites with test profile selection and run history -- Endpoint profiles with Postman collection import -- Test profiles for data-driven testing -- Playground for interactive testing -- Evaluation execution with metric selection and HyperExecute integration -- Metric threshold configuration -- Go-Live assessment with production readiness verdict -- Scheduled runs - -**What's Different from Chat** -- Conversations with the agent are conducted via **audio (WAV)** instead of text messages -- The agent's voice responses are captured and transcribed for evaluation -- Evaluation is performed on the full audio conversation transcript - -Same **9 quality metrics** as the Chat agent: - -| # | Metric | -|---|--------| -| 1 | Bias Detection | -| 2 | Hallucination Detection | -| 3 | Completeness | -| 4 | Context Awareness | -| 5 | Response Quality | -| 6 | Conversation Flow | -| 7 | User Satisfaction | -| 8 | File Handling Quality | -| 9 | File Generation Accuracy | - -## Phone Caller Inbound Agent - -**Two Evaluation Modes** -| Mode | What It Does | -|------|-------------| -| **Pre-evaluation** | The platform simulates customers calling your voice agent with live test calls, then evaluates the resulting conversations | -| **Post-evaluation** | Upload your production call recordings and transcripts from real customer interactions for evaluation on the platform | - -#### 📞 Phone Number Management - -Register and manage the phone numbers your voice agent answers on. - -- **Add Phone Numbers**: Register your agent's phone numbers with country code selection (20+ countries supported) -- **Default Phone Number**: Set a default number for quick test execution -- **Phone Number Display**: Masked display for security with country flag identification -- **Edit & Delete**: Update phone number details or remove numbers no longer in use - -#### 🎭 Scenario Management - -Generate realistic inbound call scenarios at scale with AI or build them manually. - -- **AI Scenario Generation**: Generate up to 20 inbound test scenarios with configurable personas, languages, and special instructions -- **Manual Scenario Creation**: Create scenarios with name, description, expected output -- **Scenario Deletion**: Remove scenarios no longer needed -- **Persona Selection**: Choose from available personas or create custom ones to simulate different caller types -- **Language Support**: Generate scenarios in multiple languages (English, Spanish, etc.) - -#### 🎙️ Voice Configuration (Per Scenario) - -Control every detail of how the simulated caller sounds and behaves. - -- **Voice Selection**: Choose from a library of voices with audio preview (multiple voice providers and accents available) -- **Voice Preview**: Listen to voice samples with animated waveform visualization before selecting -- **Background Sound**: Enable simulated background noise (15 presets: cafe, street, factory, rain, crowd, market, train, radio interference, etc.) -- **Response Timing**: Configure wait time after speech ends (0.5–5.0 seconds) to handle agents that speak in multiple sentences -- **Max Call Duration**: Set maximum call length (60–1800 seconds) to prevent runaway calls -- **First Speaker**: Choose who speaks first, the simulated user or the agent - -#### 👤 Agent Profiles (Inbound-Specific) - -Create reusable caller personas to standardize how test calls are placed across suites. - -- **Agent Profile Creation**: Configure agent personas with name, phone number, voice, and background noise -- **Profile Library**: Organization-level reusable agent profiles -- **Active/Inactive Toggle**: Enable or disable profiles - -#### 🗂️ Test Suites - -Batch your inbound scenarios into suites and run them all with a single action. - -- **Suite Creation**: Group multiple scenarios with per-scenario voice and phone configuration -- **Test Profile Association**: Link test data for data-driven voice testing -- **Agent Profile Assignment**: Associate agent profiles with suites -- **Run Suites**: Execute all scenarios in a suite with a single action - -#### 📡 Call Execution & Monitoring - -Trigger, track, and manage live test calls in real-time. - -- **Initiate Test Calls**: Trigger simulated inbound calls to your voice agent -- **Real-Time Monitoring**: Track call status as calls progress -- **Call Duration Tracking**: Live duration counter during active calls -- **Call Termination**: End calls in progress if needed - -#### 📂 Voice Analytics - -Upload and analyze real production recordings, no new calls needed. - -- **Recording Upload**: Upload production call recordings (MP3, WAV) from real customer interactions -- **Transcript Upload**: Upload transcripts alongside recordings for STT comparison -- **Batch Upload & Analysis**: Upload and analyze multiple recordings in parallel -- **Selective Metric Analysis**: Choose specific metric categories or individual metrics to analyze -- **Bookmarking**: Bookmark important recordings for review -- **Tagging**: Organize recordings with custom tags -- **Search & Filter**: Find recordings by name, status, date, or tags - -#### ▶️ Recording Playback - -Listen to any call and follow along with a full, speaker-identified transcript. - -- **Audio Player**: Play/pause controls with duration tracking -- **Full Transcript**: Speaker-identified transcript (agent vs. user) -- **DTMF Detection**: Phone keypad inputs (0–9, *, #) captured and displayed in transcript -- **Download**: Download recording audio and transcript files - -#### 🚦 Go-Live Assessment - -**Production Readiness Verdicts** -| Verdict | Score Range | Meaning | -|---------|------------|---------| -| 🟢 **GREEN** | ≥ 80 | Ready for Production | -| 🟡 **YELLOW** | 65–79 | Ready with Caveats | -| 🔴 **RED** | < 65 | Not Ready | -| ⚪ **NO_DATA** | - | Insufficient data to assess | - -- **Overall Score & Confidence**: Weighted score with confidence based on call volume (HIGH: 100+, MEDIUM: 50–99, LOW: 20–49, VERY_LOW: < 20) -- **Dimension Scores**: Functional Completeness, Quality Standards, Risk Profile, Operational Readiness -- **Scenario Coverage**: Well-tested vs. untested vs. high-risk scenarios -- **Failure Pattern Analysis**: AI-powered root cause analysis -- **Validation Criteria Compliance**: Aggregated pass/fail rates -- **Prioritized Action Items**: Improvement recommendations with expected impact - -#### 🎚️ Metric Configuration - -Select exactly which metrics to run. Skip what's not relevant to reduce time and cost. - -- **Configurable Metrics**: Select which metric categories to evaluate per project, or choose individual metrics for granular control -- **Category Toggles**: Enable/disable entire metric categories -- **Reduce Complexity**: Skip non-critical metrics to reduce analysis time and cost - -#### 🗓️ Scheduled Runs - -Keep coverage running continuously without manual effort. - -- **Cron-Based Scheduling**: Schedule test executions with preset frequencies (Hourly, Daily, Weekdays, Weekly, Monthly) or custom cron expressions -- **Timezone Support**: Full IANA timezone selection -- **Pause/Resume**: Temporarily pause and resume scheduled runs -- **Run History**: Track all scheduled execution results - -Phone Caller agents are evaluated across **8 metric categories** with **30+ individual metrics**. - -#### A. Conversation Flow & Interaction Dynamics - -| Metric | Unit | What It Measures | -|--------|------|-----------------| -| Average Latency | ms | Time taken for the agent to respond after user stops speaking | -| Words Per Minute (WPM) | wpm | Agent's speaking speed | -| AI Talk Ratio | % | Percentage of call time the agent is speaking | -| User Talk Ratio | % | Percentage of call time the user is speaking | -| AI Interrupting User | % | How often the agent interrupts the user | -| User Interrupting AI | % | How often the user interrupts the agent | - -#### B. Accuracy & Effectiveness - -| Metric | Unit | What It Measures | -|--------|------|-----------------| -| First Call Resolution (FCR) | % | Whether the issue was resolved in a single call | -| Intent Recognition Accuracy | % | How accurately the agent understood user intent | -| Task Completion Success Rate | % | Percentage of assigned tasks completed successfully | -| Instruction Following | % | Adherence to configured instructions | -| Response Consistency | % | Consistency of responses to similar inputs | - -#### C. User Experience & Satisfaction - -| Metric | Unit | What It Measures | -|--------|------|-----------------| -| CSAT (Customer Satisfaction) | % | Overall customer satisfaction score | -| CSAT Reason | Text | Explanation for the satisfaction score | -| User Sentiment | Text | Detected emotional sentiment from user speech | -| Early Termination | % | Percentage of calls NOT terminated prematurely | - -#### D. Business Operational Metrics - -| Metric | Unit | What It Measures | -|--------|------|-----------------| -| Containment Rate | % | Issues resolved without human escalation | -| AI to Human Handoff Rate | % | Frequency of escalation to human agents | - -#### E. Audio Voice Quality - -| Metric | Unit | What It Measures | -|--------|------|-----------------| -| Average Pitch | Hz | Voice pitch measurement (normal: 85–300 Hz) | -| Voice Quality Index | 0–5 scale | Composite voice quality score | -| Signal-to-Noise Ratio | % | Audio clarity vs. background noise | - -#### F. Speech-to-Text (STT) Evaluation - -| Metric | Unit | What It Measures | -|--------|------|-----------------| -| STT Accuracy | % | Transcription accuracy | -| STT Verdict | Pass/Fail | Overall transcription quality judgment | -| STT Summary | Text | Detailed transcription quality notes | -| Mismatch Examples | List | Specific instances where transcription differed from actual speech | - -#### G. Validation Results - -| Metric | Unit | What It Measures | -|--------|------|-----------------| -| Compliance | % | Compliance rate against custom validation criteria | -| Pass/Fail/Unable to Verify | Count | Per-criterion validation breakdown | - -#### H. Detected Issue Tags (Automated) - -The system automatically detects and flags the following issues in every call recording. - -| Issue Tag | What It Detects | -|-----------|----------------| -| Latency Issues | Slow response times | -| Hallucination in Call Flow | Agent generating incorrect information | -| Transcript Issue | Transcription errors | -| Patchy Audio | Audio quality problems | -| Running in Loop | Agent repeating the same response | -| Incorrect STT | Speech recognition errors | -| Interruption Handling | Poor handling of user interruptions | -| Number Issue | Numeric data errors | -| Background Noise | Noise interference | -| No Response | Agent silence when a response was expected | -| BLANK/EMPTY STT | Empty transcription segments | - -#### Threshold Configurations - -| Metric | 🟢 Excellent | 🟡 Good | 🔴 Poor | -|--------|-------------|--------|--------| -| Average Latency | ≤ 1000ms | ≤ 2500ms | > 2500ms | -| Words Per Minute | ≥ 160 (Fast) | 131–160 (Good) | < 110 (Slow) | -| Voice Quality Index | ≥ 2.5 / 5 | - | < 2.5 / 5 | -| Average Pitch | 85–300 Hz (Normal) | - | < 85 or > 300 Hz | - -## Phone Caller Outbound Agent - -Phone Caller Outbound supports the **same two evaluation modes** as Inbound (Pre-evaluation and Post-evaluation) and shares all features, with a few key differences in pre-evaluation mode only. - -#### Outbound-Specific Pre-evaluation Features - -- **Scenario Generation**: Generate up to **7** outbound test scenarios (vs. 20 for inbound) -- **Caller Profile Selection**: Select an outbound caller profile when generating scenarios -- **Outbound Number Pool**: Reserve phone numbers from the outbound pool for test calls -- **Passive Mode**: Listen to outbound calls without interfering (for QA monitoring) -- **First Speaker Default**: Agent speaks first (vs. simulator for inbound) - -#### Outbound Pool Management - -- **View Pool Status**: See available outbound numbers -- **Reservations**: View and manage per-suite number reservations -- **Clear Reservations**: Release reserved numbers when done - -All other features (phone number management, voice configuration, test suites, call execution, post-evaluation, go-live assessment, metrics, and scheduling) are **identical to Phone Caller Inbound**. - -Same as Phone Caller Inbound, all **8 categories** and **30+ metrics**. - -## Image Analyzer Agent - -#### 🖼️ Image Analysis - -Upload single images or batch-process up to 50 at once, via file upload or URL. - -- **Single Image Analysis**: Upload an image (or provide a URL) along with the original prompt used to generate it -- **Batch Analysis**: Analyze up to 50 images at once with a shared prompt -- **Drag & Drop Upload**: Drag and drop images directly into the upload area -- **Supported Formats**: JPG, JPEG, PNG, GIF, WEBP, BMP (max 20 MB per image) - -#### 🎯 Custom Evaluation Criteria - -Define what "good" means for your images. Choose from three criteria types: - -- Allowed colors and prohibited colors -- Required fonts -- Logo requirements (text description) - -- Required dimensions (width × height) -- Aspect ratio requirements -- Allowed file formats -- Maximum file size -- Minimum resolution - -- Freeform rule text -- Checklist items to validate - -All criteria support Active/Inactive toggling, create/edit/delete operations, and search by name, description, or type. - -#### 📋 Analysis History - -- **Search**: Find past analyses by image name or prompt text -- **Status Tracking**: View analysis status (Pending, Completed, Failed) -- **Detailed View**: Click any analysis to view full results -- **Bookmarking**: Bookmark important analyses for quick access - -#### 📊 Analytics Dashboard - -- **Overall Statistics**: Average score, highest score, lowest score, total analyses count -- **Quality Trends**: Daily score breakdown over the last 30 days with bar chart visualization -- **Prompt Performance**: Top 20 prompts ranked by average quality score, showing min/max scores, usage count, and comparison vs. overall average - -| Metric | Scale | What It Measures | -|--------|-------|-----------------| -| **Quality Score** | 0–100 | Overall image quality and prompt adherence | -| **Matches** | List | Elements that correctly match the original prompt | -| **Discrepancies** | List | Missing or incorrect elements vs. the prompt | -| **Overall Assessment** | Text | Summary of how well the image matches the prompt | -| **Detailed Observations** | Text | In-depth analysis of specific image aspects | - -**Quality Score Interpretation:** - -| Score Range | Label | Rating | -|-------------|-------|--------| -| 90–100 | Excellent | 🟢 | -| 80–89 | Good | 🟢 | -| 60–79 | Fair | 🟡 | -| 0–59 | Poor | 🔴 | - -**Custom Criteria Compliance:** - -For each active criterion, results show: -- Compliance status: **PASS**, **FAIL**, or **PARTIAL** (color-coded badges) -- Compliance details with specific observations - -## Shared Features Across Agents - -### 🗂️ Project Management - -- **Create Agents**: Name, description, and agent type selection -- **Agent Listing**: View all agents with type-specific icons and filtering -- **Agent Dashboard**: Overview of workflows, suites, and recent activity - -### 📋 Test Profiles - -**Available for: Chat · Voice · Phone Caller (Inbound/Outbound)** - -- Custom key-value data with typed fields (string, number, boolean, email, URL, textarea, JSON) -- Default profile marking -- Import/Export as JSON - -### 🎭 Personas - -**Available for: Chat · Voice · Phone Caller (Inbound/Outbound)** - -- Pre-built persona library -- Custom persona creation -- Persona assignment to scenarios - -### 🗓️ Scheduling - -**Available for: Chat · Voice · Phone Caller (Inbound/Outbound)** - -- Cron-based scheduling (Hourly, Daily, Weekdays, Weekly, Monthly, Custom) -- Timezone support (full IANA timezone list) -- Pause/Resume/Delete scheduled runs -- View next scheduled run time and run history - -### 🚦 Go-Live Assessment - -**Available for: Chat · Voice · Phone Caller (Inbound/Outbound)** - -| Verdict | Score Range | Meaning | -|---------|------------|---------| -| 🟢 GREEN | ≥ 80 | Ready for Production | -| 🟡 YELLOW | 65–79 | Ready with Caveats | -| 🔴 RED | < 65 | Not Ready | - -- Overall score with confidence level -- Scenario coverage analysis -- AI-powered risk assessment and recommendations - -### 🌐 Environment Management - -**Available for: All agent types** - -- Create and manage test environments -- Variable management (name, value, type, persistence) -- Per-environment variable scoping -- Bulk variable creation - -### ✅ Validation Criteria - -**Available for: Chat · Voice · Phone Caller (Inbound/Outbound)** - -- Custom pass/fail criteria per scenario -- Evidence-based validation with confidence levels (High/Medium/Low) -- Compliance percentage tracking -- Per-criterion results: Pass, Fail, or Unable to Verify - -## Feature Availability Matrix - -| Feature | Chat | Voice | Phone Caller Inbound | Phone Caller Outbound | Image Analyzer | -|---------|:----:|:-----:|:--------------------:|:---------------------:|:--------------:| -| Workflow & Document Upload | ✅ | ✅ | ❌ | ❌ | ❌ | -| AI Scenario Generation | ✅ | ✅ | ✅ (up to 20) | ✅ (up to 7) | ❌ | -| Manual Scenario Creation | ✅ | ✅ | ✅ | ✅ | ❌ | -| Test Suites | ✅ | ✅ | ✅ | ✅ | ❌ | -| Endpoint Profiles | ✅ | ✅ | ❌ | ❌ | ❌ | -| Test Profiles | ✅ | ✅ | ✅ | ✅ | ❌ | -| Personas | ✅ | ✅ | ✅ | ✅ | ❌ | -| Playground | ✅ | ✅ | ❌ | ❌ | ❌ | -| Audio-Based Conversations | ❌ | ✅ | ❌ | ❌ | ❌ | -| Phone Numbers | ❌ | ❌ | ✅ | ✅ | ❌ | -| Voice Selection (per scenario) | ❌ | ❌ | ✅ | ✅ | ❌ | -| Background Noise Simulation | ❌ | ❌ | ✅ | ✅ | ❌ | -| Live Call Execution (Pre-eval) | ❌ | ❌ | ✅ | ✅ | ❌ | -| Production Recording Upload (Post-eval) | ❌ | ❌ | ✅ | ✅ | ❌ | -| Batch Recording Analysis | ❌ | ❌ | ✅ | ✅ | ❌ | -| Call Recording Playback | ❌ | ❌ | ✅ | ✅ | ❌ | -| DTMF Support | ❌ | ❌ | ✅ | ✅ | ❌ | -| Passive Mode | ❌ | ❌ | ❌ | ✅ | ❌ | -| Outbound Pool | ❌ | ❌ | ❌ | ✅ | ❌ | -| Image Upload & Analysis | ❌ | ❌ | ❌ | ❌ | ✅ | -| Custom Evaluation Criteria | ❌ | ❌ | ❌ | ❌ | ✅ | -| Brand Guideline Checks | ❌ | ❌ | ❌ | ❌ | ✅ | -| Image Analytics Dashboard | ❌ | ❌ | ❌ | ❌ | ✅ | -| Metric Thresholds | ✅ | ✅ | ✅ | ✅ | ❌ | -| Go-Live Assessment | ✅ | ✅ | ✅ | ✅ | ❌ | -| Validation Criteria | ✅ | ✅ | ✅ | ✅ | ❌ | -| Scheduled Runs | ✅ | ✅ | ✅ | ✅ | ❌ | -| HyperExecute Integration | ✅ | ✅ | ❌ | ❌ | ❌ | -| Import/Export Profiles | ✅ | ✅ | ❌ | ❌ | ❌ | - -## Metrics Availability Matrix - -| Metric Category | Chat | Voice | Phone Caller Inbound | Phone Caller Outbound | Image Analyzer | -|----------------|:----:|:-----:|:--------------------:|:---------------------:|:--------------:| -| Bias Detection | ✅ | ✅ | ❌ | ❌ | ❌ | -| Hallucination Detection | ✅ | ✅ | ❌ | ❌ | ❌ | -| Completeness | ✅ | ✅ | ❌ | ❌ | ❌ | -| Context Awareness | ✅ | ✅ | ❌ | ❌ | ❌ | -| Response Quality | ✅ | ✅ | ❌ | ❌ | ❌ | -| Conversation Flow | ✅ | ✅ | ❌ | ❌ | ❌ | -| User Satisfaction | ✅ | ✅ | ❌ | ❌ | ❌ | -| File Handling Quality | ✅ | ✅ | ❌ | ❌ | ❌ | -| File Generation Accuracy | ✅ | ✅ | ❌ | ❌ | ❌ | -| Latency & Interaction Dynamics | ❌ | ❌ | ✅ | ✅ | ❌ | -| Accuracy & Effectiveness (FCR, etc.) | ❌ | ❌ | ✅ | ✅ | ❌ | -| CSAT & User Experience | ❌ | ❌ | ✅ | ✅ | ❌ | -| Business Operational Metrics | ❌ | ❌ | ✅ | ✅ | ❌ | -| Audio Voice Quality | ❌ | ❌ | ✅ | ✅ | ❌ | -| STT Evaluation | ❌ | ❌ | ✅ | ✅ | ❌ | -| Issue Detection Tags | ❌ | ❌ | ✅ | ✅ | ❌ | -| Validation Criteria | ✅ | ✅ | ✅ | ✅ | ❌ | -| Image Quality Score | ❌ | ❌ | ❌ | ❌ | ✅ | -| Prompt Compliance | ❌ | ❌ | ❌ | ❌ | ✅ | -| Brand/Technical Spec Compliance | ❌ | ❌ | ❌ | ❌ | ✅ | diff --git a/static/docs/agent-skills.md b/static/docs/agent-skills.md deleted file mode 100644 index d9fcde0e9..000000000 --- a/static/docs/agent-skills.md +++ /dev/null @@ -1,81 +0,0 @@ -# Agent Skills - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -**Agent Skills** are instruction packages that teach an AI coding assistant how to write and run test automation on TestMu AI. Install one, and your assistant works like an SDET who already knows that framework's project layout, dependency versions, cloud execution config, and CI wiring. - -Each skill is open source and framework-specific. A skill covers one framework or platform, so the agent uses that tool's own idioms instead of generic automation code. The full set lives in the [TestMu AI Agent Skills](https://github.com/LambdaTest/agent-skills) repository. - -Skills work with Claude Code, GitHub Copilot, Cursor, Gemini CLI, and any assistant that loads custom skills. - -## How to Install a Skill - -Use the `agentskillsforall` installer to add a skill straight from the repository. No manual cloning is required. - -Add skills by name. Run the installer once per skill you want: - -```bash -npx agentskillsforall add https://github.com/LambdaTest/agent-skills.git --skill selenium-skill -npx agentskillsforall add https://github.com/LambdaTest/agent-skills.git --skill playwright-skill -npx agentskillsforall add https://github.com/LambdaTest/agent-skills.git --skill cypress-skill -``` - -List every skill in the repository, then install the one you want: - -```bash -npx agentskillsforall list https://github.com/LambdaTest/agent-skills.git -``` - -Once installed, ask your assistant to write and run tests. It picks up the matching skill automatically. - -## Core Skills by Framework and Platform - -Pick the skill for the framework you already test in. Each card opens that skill's source on GitHub. Web frameworks run on the TestMu AI browser grid, and app frameworks target real Android and iOS devices. - -Web Automation - - Selenium↗ - WebDriver tests that run on the TestMu AI browser grid. - - Cypress↗ - End-to-end specs that run locally and on the cloud. - - Playwright↗ - Cross-browser tests with fast cloud execution. - - Puppeteer↗ - Headless Chrome automation on the TestMu AI cloud. - -App Automation - - Appium↗ - One suite for real Android and iOS devices. - - Espresso↗ - Native Android UI tests with Espresso. - - XCUITest↗ - Native iOS UI tests with XCUITest. - - Flutter↗ - Widget and integration tests for Flutter apps. - -Cloud, Visual & Platform - - HyperExecute↗ - Shard test suites across the HyperExecute cloud. - - SmartUI↗ - Visual regression with pixel-diff baselines. - - Browser Cloud↗ - A real cloud browser for any AI agent. - - Kane CLI↗ - Natural-language browser automation in agent mode. - -## Explore the Full Skill Catalog - -The 12 skills above are the most common starting points. The repository ships 70+ skills across eight testing categories. - -Beyond web and mobile, it covers BDD (Cucumber, Behave, SpecFlow), unit testing (Jest, Pytest, JUnit 5, TestNG), and API testing (Postman, Newman, OpenAPI). List everything with `npx agentskillsforall list`, or browse the [full skill catalog on GitHub](https://github.com/LambdaTest/agent-skills). diff --git a/static/docs/agent-testing-platform-faqs.md b/static/docs/agent-testing-platform-faqs.md deleted file mode 100644 index ff589615e..000000000 --- a/static/docs/agent-testing-platform-faqs.md +++ /dev/null @@ -1,124 +0,0 @@ -# Agent Testing Platform FAQs: Setup, Security, Pricing - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Common questions about TestMu AI's Agent Testing Platform, grouped by plans, setup, security, and evaluation quality. - -## Plans and Pricing - -### Availability for Small Teams and Startups - -Yes. The Pay-As-You-Go tier starts at $0 with no monthly commitment, and you pay only for the credits you consume at $0.01 each. The Starter and Growth plans suit small and mid-size teams, while Scale and Enterprise add SLAs and compliance documentation for high-volume organizations. - -### Free Trial and the Free Tier - -The Pay-As-You-Go tier is effectively a permanent free tier. There is no time limit and no commitment. You pay only for the credits you use at $0.01 each. - -### How Agent Testing Differs From LambdaTest Browser Testing - -TestMu AI historically focused on browser and device testing. Agent Testing is a distinct product for AI agent quality evaluation, and it does not test browser UIs. The shared infrastructure, such as HyperExecute and the enterprise security stack, is the connection point between them. - -### AI Models Behind the Testing Agents - -The platform uses a hybrid stack that combines third-party large language models with TestMu AI's in-house agentic framework. Multiple LLMs contribute to evaluation reasoning, which reduces single-model blind spots. Specific model names are disclosed under NDA. - -## Connecting and Running Tests - -### Connecting an Agent Without Code Changes - -No changes to your agent are required. The agent needs an accessible endpoint, which is an HTTP API, a voice endpoint, or a phone number. There is no SDK to install, and the platform interacts with the agent through the same interface production users use. - -### Testing an Agent Behind a Firewall - -Yes. Use the HyperExecute secure tunnel, and the agent does not need a public-facing URL. The tunnel establishes an encrypted connection from the testing infrastructure to your private endpoint. - -### Scenarios Generated From a Requirements Document - -The platform generates 60 to 100+ scenarios per workflow by default. You can increase that by adding more persona types, expanding focus areas, or adding scenarios manually. - -### How Long a Full Evaluation Takes - -A typical chat or voice evaluation runs to completion in under 2 hours for most scenario volumes. Exact run time depends on the agent's response latency, the scenario count, and available concurrency. - -### Testing Against a Production or Staging Agent - -Running against a production agent is technically possible, but evaluating against staging is the recommended practice. To assess real production behavior, upload recorded production calls and use post-production recording analysis. - -### How Phone Testing Places Real Calls - -The platform dials real phone numbers. For an inbound agent, it calls the agent's actual number as a real caller would, then drives the conversation with a simulated caller. SIP endpoint simulation is also available for teams that prefer to test through SIP without routing over the PSTN. - -## Security and Data Handling - -### Whether Your Model and Knowledge Base Leave Your Environment - -No. The platform sends test inputs to your agent endpoint, the same inputs any user would send, and receives responses. The agent's internal model, knowledge base, and code do not leave your environment. - -### HIPAA Compliance - -TestMu AI is HIPAA certified. Healthcare teams should use synthetic patient data in test scenarios rather than real patient records. - -### Who Can See Your Evaluation Results - -Evaluation results are scoped to the project and accessible only to users with the appropriate role-based access control permissions within your organization. - -### On-Premises and VPC Deployment - -On-premises and VPC deployment is available for enterprise contracts. Contact the enterprise sales team to evaluate fit for your environment. - -## Evaluation Quality and Accuracy - -### How Verdict Accuracy Is Backed by Evidence - -Every verdict is backed by evidence, the specific conversation transcripts that drove each metric score. You can review the failing transcripts and validate whether the platform's assessment matches your own judgment. Confidence levels flag when the scenario volume is too low for a reliable verdict. - -### Handling a False Positive - -Review the failing scenario transcript. If the assessment does not match expert human judgment, refine the agent prompt or add custom validation criteria that define the expected behavior more precisely. Tuning the criteria to your agent's context improves accuracy over time. - -### Comparing Results Across Model Versions - -Yes. The dashboard shows metric score deltas between test runs and tracks trends over time, so you can confirm whether a model update improved or regressed quality. - -## Platform Capabilities - -### Multilingual Requirement Analysis and Multi-Language Testing - -Yes. The platform supports AI-enabled multilingual requirement analysis, so teams can extract and analyze requirements from documents written in multiple languages. It generates test scenarios that account for multilingual support requirements and supports testing AI agents that handle multiple languages across interactions. - -Key capabilities include: - -- Multilingual requirement extraction and analysis from documents. -- AI-driven test scenario generation aligned with multilingual support requirements. -- Support for analyzing AI agents designed to operate across multiple languages. - -### Model Governance for GenAI Systems - -Yes. Model governance is supported through controlled workflows, version tracking, access controls, audit trails, and policy-aligned oversight across GenAI model lifecycles. - -### NLP, Unstructured Data, and Metadata Support - -Yes. The platform provides NLP, unstructured data processing, and metadata management for GenAI engineering workflows. It enables multi-modal document processing, requirement extraction, natural language understanding, and metadata traceability across formats using multiple LLMs. - -Key capabilities include: - -- Multi-modal document processing for text (PDF, DOCX, XLSX, TXT, MD), images (with OCR), audio (with speech-to-text), and video (with frame extraction and transcription). -- Structured requirement extraction from unstructured data, including entity extraction, business rules, integration points, and security requirements. -- NLP capabilities such as natural language understanding, conversational AI testing, intent recognition, context memory testing, and multi-turn reasoning. -- Metadata management with document and file metadata extraction, requirement relationship mapping, source requirement ID traceability, and cross-document linkage. -- Unstructured data processing including OCR for images, speech-to-text for audio, video frame analysis, and semantic understanding across formats. - -### Out-of-the-Box Data Source Connectors - -The platform provides three native out-of-the-box connectors and supports additional integrations through third-party services and secure tunnel infrastructure. This enables secure access to both public and private data sources. - -The three native connectors are: - -- **GitHub** for repository content extraction, README retrieval, and file access, including private repositories via secure tunnels. -- **JIRA** for ticket retrieval, project information access, and issue tracking, including private JIRA instances via secure tunnels. -- **Confluence** for page content import, documentation retrieval, and knowledge base integration, including private Confluence instances via secure tunnels. - -## Related TestMu AI Guides - -- See how to [run your first evaluation from the dashboard](/support/docs/testing-your-first-ai-agent/). -- See how the platform [runs an evaluation end to end](/support/docs/architecture-and-how-evaluation-works/). diff --git a/static/docs/ai-agent-testing-platform-overview.md b/static/docs/ai-agent-testing-platform-overview.md deleted file mode 100644 index d591059a6..000000000 --- a/static/docs/ai-agent-testing-platform-overview.md +++ /dev/null @@ -1,76 +0,0 @@ -# Quality Dimensions and Go-Live Readiness for AI Agents - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The Agent Testing Platform scores every agent conversation across a set of quality dimensions, applies the pass/fail thresholds you configure, and rolls the results into a Green, Yellow, or Red go-live verdict. This page explains what the platform measures, how thresholds work, and how the production readiness verdict is decided. - -## How the Platform Scores Conversations - -Every completed conversation, voice or chat, is scored by the evaluation engine across a set of quality dimensions. Each dimension is scored on a normalized 0.0 to 1.0 scale, with configurable pass/fail thresholds per project. - -The exact set of metrics depends on the agent type. Chat and voice agents are scored across 9 quality metrics, and phone agents across 30+ call metrics. - -Commonly scored dimensions include the following. - -| Quality Dimensions | | | -|----------------------|-|-| -| Hallucination Detection | Bias Detection | Response Quality | -| Context Awareness | Completeness | Conversation Flow | -| User Satisfaction | Intent Recognition | Multi-Turn Reasoning | -| Error Handling & Recovery | Consistency | Proactive Behavior | -| Compliance & Governance | Data Privacy | Security Posture | -| Escalation Accuracy | Personality & Tone | Audit Logging Quality | - -## Configure Metric Thresholds - -Each project can define custom minimum thresholds per dimension. A regulated deployment might require a Compliance score of at least 0.95 and a Data Privacy score of at least 0.98, while a consumer chatbot might prioritize Conversation Flow and User Satisfaction at a lower floor. - -Threshold profiles are version-controlled and can differ across environments such as development, staging, and production. A conversation passes a dimension when its score meets or exceeds the threshold you set for it. - -## Read the Go-Live Readiness Verdict - -The Go-Live Assessment aggregates results across all evaluated scenarios and dimensions into a three-tier production readiness verdict. - -| Verdict | Meaning | -|---------|-------------| -| 🟢 **Green** | All critical thresholds met. The agent behaves consistently across tested scenarios. Cleared for production. | -| 🟡 **Yellow** | Most thresholds met; one or more secondary metrics fall below target. Conditional approval with identified risk areas to monitor. | -| 🔴 **Red** | One or more critical thresholds not met. Specific failure categories are surfaced for remediation before production. | - -Each verdict carries an overall aggregate score and a confidence level, plus a pass/fail breakdown across the behavioral test categories. - -### Confidence Levels - -The confidence level tells you how far to trust the verdict, based on how many scenarios backed it. A Green verdict at High confidence is a stronger signal than the same verdict at Low confidence. - -| Confidence | Evaluations behind the score | -|---|---| -| **High** | 100 or more | -| **Medium** | 50 to 99 | -| **Low** | 20 to 49 | -| **Very Low** | Fewer than 20 | - -For compliance and hallucination metrics especially, raise the scenario count until confidence is High before making a deployment decision. - -## Behavioral Test Categories - -Generated scenarios are organized across sixteen behavioral categories, so a run covers the full spread of agent capabilities and failure modes rather than only the happy path. - -| Test Categories | | | -|----------------|-|-| -| Conversational Flow | Intent Recognition | Context & Memory | -| Multi-Turn Reasoning | User Experience | Personality & Tone | -| Proactive Behavior | Multimodal Interactions | Third-Party Integration | -| Error Handling | Consistency | Security | -| Compliance & Governance | Data Privacy | Performance | -| Recovery Mechanisms | | | - -The Go-Live verdict reports pass/fail rates per category, so a Yellow or Red result points at the specific category to fix. - -## Track Cost and Usage - -The platform tracks per-test execution costs, such as call minutes consumed and AI evaluation compute, and aggregates usage at the project, suite, and organization level. This gives teams budget visibility and lets them attribute testing costs across products or internal teams. - -## Related TestMu AI Guides - -- See how the platform [runs an evaluation end to end](/support/docs/architecture-and-how-evaluation-works/), from context to verdict. diff --git a/static/docs/ai-powered-test-failure-analysis.md b/static/docs/ai-powered-test-failure-analysis.md deleted file mode 100644 index cd976f957..000000000 --- a/static/docs/ai-powered-test-failure-analysis.md +++ /dev/null @@ -1,19 +0,0 @@ -# AI Native Test Failure Analysis in HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The AI Native Root Cause Analysis (RCA) and Error Detection for HyperExecute testing helps you to quickly identify various error categories and offers a comprehensive overview of their underlying causes. It analyzes your test logs and identify the root cause of failures, so you can take corrective action quickly and efficiently. - -## Why Test Failure Analysis? - -Conventional test failure resolution demands manual analysis of logs and data sources, resulting in substantial delays. These inefficiencies hinder productivity and disrupt software development. HyperExecute's AI Native Test Failure Analysis seamlessly addresses these challenges, offering a proactive approach to identifying and fixing test case failures along with other benefits including: - -- **Reduced time to identify and fix failures:** The AI Native analysis can quickly identify the root cause of failures, so you can take corrective action quickly and efficiently. This can save you a significant amount of time, which can be used to focus on other tasks. -- **Improved accuracy of failure analysis:** The AI Native analysis is more accurate than manual analysis, which can help you to identify and fix failures more effectively. This can lead to a higher quality product. -- **Increased productivity of testing teams:** The AI Native analysis can free up testing teams to focus on other tasks, such as creating new test cases or debugging code. This can lead to a more efficient testing process. - -## Generate the RCA of your failed Jobs - -To use the AI Native Test Failure Analysis feature, simply click on the Failure Analysis" (Beta) tab of your failed Job. Click on the **Generate RCA** Button. This usually takes around 20-30 seconds to generate the RCA. - -After categorizing errors, we offer a structured approach to address each failure type. You can efficiently get to the precise corrective measures or **Remedies** recommended alng with **Additional suggestions** for the particular error. This targeted approach expedites the issue resolution process, eliminating unnecessary steps and optimizing efficiency. diff --git a/static/docs/airbrake-integration.md b/static/docs/airbrake-integration.md deleted file mode 100644 index 5e3e8798a..000000000 --- a/static/docs/airbrake-integration.md +++ /dev/null @@ -1,63 +0,0 @@ -# Airbrake Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Airbrake is a software tool and service designed to assist developers in tracking and monitoring errors within web and mobile applications. Its primary purpose is promptly identifying and diagnosing software errors and exceptions, enabling developers to react swiftly and effectively. - -TestMu AI's integration with Airbrake provides a seamless solution for tracking and managing bugs and issues encountered during cross-browser testing. With this integration, developers can effortlessly log bugs directly from the TestMu AI platform to their Airbrake instance, eliminating the need for manual maintenance of tasks and issues. - -## Integrating Airbrake With TestMu AI - -**Step 1:** Sign in to your TestMu AI account. Don't have an account, register for free. - -**Step 2:** Select **Integrations** from the left sidebar and head to **Airbrake**. - -**Step 3:** It will route you to the installing console window, where you are required to enter your Airbrake **Auth Token, Project ID, and Project Key**. - -## Getting The Airbrake Auth Token, Project ID, and Project Key - -**Step 1:** Login to your **Airbrake** account. - -**Step 2:** From the left sidebar menu, hover on the **profile icon** and click the **Profile and Notifications** tab. - -**Step 3:** Copy the **Auth Token**. - -To access your **Project ID** and **Key**, head to the **Projects** option in the top-left corner. - -**Step 1:** Select your preferred project or create a new project. - -**Step 2:** Head to the **Settings** tab. - -**Step 3:** Copy the **Project ID** and **Project Key**. - -Copy and paste the required credentials from your Airbrake account and click **Install**. - -Now you have successfully integrated your TestMu AI account with Airbrake. - -## One-Click Bug Logging With Airbrake - -**Step 1:** In your TestMu AI Dashboard, head to **Real-Time Testing > Browser Testing** from the left navigation menu. - -**Step 2:** To perform **live-interactive testing** of your website or a web app, enter the web URL, select the testing type (Desktop/Mobile), select the browser version, OS, and resolution, and click **Start**. - -Upon clicking, a cloud-based virtual machine will fire up, running real browsers and operating systems where you can test websites and web apps for filing bugs. - -**Step 3:** Click on the **Bug icon** from the left in-session toolbar to take a screenshot of any bug you encounter. - -**Step 4:** After marking the bugs, click on the **Bug** icon in front of **Airbrake** under the integrations section. - -**Step 5:** In the Airbrak ticket, fill the bug details like **Project, Summary, and Description** and click **Mark As Bug** to log the bugs in Airbrake. - -**Step 6:** To view the logged issues, visit TestMu AI Dashboard and head to **Manage > Issue Tracker** from the left navigation bar. - -**Step 7:** Inside the **Issue Tracker**, you'll notice the logged bugs and their test environment details. Click on the **upper right arrow** in the square button to view your logged bugs in Airbrake instance. - -**Step 8:** Upon clicking, you will be redirected to the Airbrake dashboard to find your **logged bugs**. - -## Uninstalling The Airbrake Integration - -**Step 1:** Sign in to your TestMu AI account and visit **Integrations** from the left sidebar menu. - -**Step 2:** From My Integrations, click **Remove** under the **Airbrake** block. - -> That was all you need to know for TestMu AI + Airbrake Integration. Increase your productivity with our integrations. If you still have any questions for us, please feel free to let us know. Our experts are always available on **chat** to help you out with any roadblock regarding our product. Happy testing! diff --git a/static/docs/airtable-integration.md b/static/docs/airtable-integration.md deleted file mode 100644 index 0bc6a4938..000000000 --- a/static/docs/airtable-integration.md +++ /dev/null @@ -1,74 +0,0 @@ -# Airtable Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Airtable is a cloud-collaboration platform to help you organize your team with intuitive & interactive spreadsheets that works like a database. It offers a variety of views such as Grid view, Kanban view, Calendar view, Form view. You can log a task with priority, notes, deadline and more. You can even tag attachments to a task with a simple drag-and-drop. Be it marketing, project management, product design & UX, or a simple distribution record for offline businesses, Airtable helps you to manage a track of anything. - -Using TestMu AI integration with Airtable, you'll be able to log bugs over your project in Airtable from TestMu AI in just a single click. You can [capture a screenshot](https://www.testmuai.com/full-page-screen-capture) of the bug, highlight the anomaly, annotate extra information, choose an assignee, provide a suitable title and description to help relay further actions. You can do all that directly from TestMu AI and then mark a bug to your Airtable project with just a single click. - -## How To Integrate Airtable With Your TestMu AI Account? - -**Step 1:** Login to your TestMu AI account. You would need to have Admin or User level access to see and install integrations. - -**Step 2:** Select 'Integration' from the left navigation menu bar. This will guide you to a screen where you will find a list of 3rd party applications, available to integrate with your TestMu AI account. - -**Step 3:** Click on the block that says **'Airtable'**. - -**Note:** If you have already installed a TestMu AI integration to any project management tool then you would come across a prompt asking you to disable your existing TestMu AI integration for integrating Airtable with your TestMu AI account. - -**Step 4:** You would now be routed to a screen where you'll need to enter your Airtable **App Key** & Airtable **API Key**. - -**Note:** You need to have **Bug Tracker** as a template in your Airtable account for TestMu AI integration with Airtable to work. After all, we're here to help you collect bugs and post them there. If you don't have a **Bug Tracker** template installed in your Airtable account, refer to [Airtable's official support documentation](https://airtable.com/templates/product-design-and-ux/expOzMycWirMsUOTL/bug-and-issue-tracker) for instantly setting up the template. - -**Step 5:** If this is your first project in Airtable, you'll need to generate an API Key that will help TestMu AI to authenticate your user information from Airtable for integration purpose. Go to your **Account Overview** & click on the button to **Generate API Key**. - ->**Note**: Consider API Key as your passwords, the API Key will help anyone grant a third-party application on the behalf of your Airtable account so be sure to keep it safe and sound. In case you feel your API Key is revealed to someone then you must re-generate your Airtable API Key. -> - -**Step 6:** Now, to find your App ID, you need to visit [https://airtable.com/api](https://airtable.com/api). Then select your base project i.e. Bug Tracker for integrating TestMu AI with Airtable. - -**Step 7:** After you open the Bug tracker, you'll find the App ID for your base project. Copy this App ID. - -**Step 8:** Next, you need to paste you API Key & App ID over TestMu AI integration screen for Airtable. Click the install button. - -Kudos! You've successfully integrated TestMu AI with Airtable. You will find Airtable Integration highlighted under the section "**My Integration**". - -## How To Log Bug Using TestMu AI Integration To Airtable? - -**Step 1:** Go for any of the test from the left navigation menu. For demo, we will be taking **"Real Time Test"** option. - -**Step 2:** Present a URL of the web-app you need to test in the dialog box. After that, select any configuration for browser and operating system of your choice & hit **'Start'**. - -**Step 3:** After the VM is launched and operable. You can perform testing on your web-app for finding bugs. If a bug gets revealed, then you need to click on the **Bug icon** from the left panel to capture a screenshot of the bug. - -**Step 4:** After a screenshot is captured, you can annotate any issue or a task with an built-in image editor. Once you are done highlighting the bug, click on the button that says "**Mark as Bug**". - -**Step 5:** After clicking on "Mark as Bug" button a form specific to **Airtable **would open up. Fill the fields as per your requirement, and create the issue. - ->You can simply field the Name field and create a bug or you can go in-depth around the bug by providing the below details as well: ->* Name -* Priority -* Status -* Associated features -* Created by -* Associated to -* Bug source -* Description - -**Note:** After you click on **"Create Bug"**, you will be able to observe it being successfully marked through a single click effort. You will get prompt messages on top of your Virtual Machine indicating the progress of bug logging. After few seconds you will be notified with a **prompt message** "Bug successfully marked" indicating that the screenshot has been pushed to your **Airtable project**. - -**Step 6:** Now, if you visit your Bug tracker in Airtable, you'll find the bug that you logged from TestMu AI in it with all the provided details along with the screenshot of the bug. The bug with details can also be found under the features tab with the associated feature. - -**Step 7:** Once you click over the Name, you will find a card which has all the details provided from TestMu AI. - -## How To Remove TestMu AI Integration With Airtable? - ->You can work with one integration at a time. So if you would want to integrate to a similar 3rd party application, then you would have to **remove** your current integration. Here is how you can do that. - -**Step 1:** Login to your TestMu AI account. - -**Step 2:** Select **'Integrations'** from the left-navigation menu. This will guide you to a screen where you will find 3rd party applications, available to integrate with your TestMu AI account. - -**Step 3:** Under "My Integrations", click on **'Remove'** button. You can find also find the remove button under the 'Airtable' block. - -That was all you need to know for TestMu AI + Airtable Integration. Increase your productivity with our integrations. If you still have any questions for us, please feel free to let us know. Our experts are always available on **chat** to help you out with any roadblock regarding our product. Happy testing! 🙂 diff --git a/static/docs/algoqa-integration.md b/static/docs/algoqa-integration.md deleted file mode 100644 index fe4c14437..000000000 --- a/static/docs/algoqa-integration.md +++ /dev/null @@ -1,41 +0,0 @@ -# algoQA Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -algoQA is an end-to-end automation testing platform powered by artificial intelligence. It lets you generate test cases, executable automation scripts and datasets in a jiffy, thereby saving your testing and test maintenance cost and enhance your overall test coverage. - -By integrating TestMu AI with algoQA, you can perform automated browser testing on an online browser farm of 3000+ real browsers and operating system combinations, and achieve greater test coverage. - -## Prerequisites - -1. Make sure you have a TestMu AI account. If you don't have an account yet, sign up for free. - -2. To run automated tests, you will need TestMu AI Username and Access Key. You can get them from the TestMu AI Automation Dashboard under **Access Key**. - -## Integrating algoQA With TestMu AI - -1. Login to your TestMu AI account. - -2. Paste your TestMu AI Username and Access Key in the test scripts for generating a secure remote connection with algoQA through TestMu AI Remote URL. - -3. Navigate to TestMu AI [Automation Capabilities Generator](https://www.testmuai.com/capabilities-generator/) for declaring desired capabilities. For instance, if you wish to test on macOS High Sierra using Firefox browser then desired capabilities for Java frameworks would look like below. - -4. Visit algoQA platform to generate Selenium scripts in Java. - -5. After generating the Selenium Java scripts, open algoQA scripts and go to the directory `src/test/java`. - -6. Open ApplicationSettings.xml file to modify TestMu AI environment changes as highlighted below. - -* BrowserType -* TestMu AIBuild -* TestMu AIURL - -Once you have defined the above parameters, your script will be ready for execution. - -7. Navigate back to pom.xml directory and open Command Prompt in the same directory. - -8. Pass the below command in the Command Prompt to trigger the script execution. - -``` -mvn test -``` diff --git a/static/docs/allure-reports.md b/static/docs/allure-reports.md deleted file mode 100644 index 0594f5869..000000000 --- a/static/docs/allure-reports.md +++ /dev/null @@ -1,64 +0,0 @@ -# Allure Reports on HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Allure Framework is a versatile, lightweight, multi-language test reporting tool designed to provide a concise representation of tested functionalities in a visually appealing web report format. It facilitates easy extraction of valuable information for all stakeholders involved in the development process. - -## Integration with HyperExecute - -If your testing framework already generates Allure reports, you can seamlessly integrate it with HyperExecute. This integration enables the generation of a consolidated Allure report on HyperExecute for each job, accessible through the report section of the HyperExecute dashboard. - -## Allure Report Configuration - -**Step 1:** To configure the Allure reporter in the **WDIO framework**, update your `wdio.conf.js` file as follows: - -```javascript title="wdio.conf.js" -export const config = { -reporters: [['allure', { -outputDir: 'reports/allure-results', -disableWebdriverStepsReporting: true, -disableWebdriverScreenshotsReporting: true, -}]], -} -``` -- The `outputDir` parameter specifies the directory where Allure reports will be stored. In this example, it is set to **'reports/allure-results'**. This parent folder is essential for HyperExecute integration, as detailed later. - -- The `disableWebdriverStepsReporting` and `disableWebdriverScreenshotsReporting` parameters allow customization of reporting options. - -Alternatively this can also be done by creating an `allure.properties` file in the `src/test/resources` directory. - -```yaml title="allure.properties" -allure.results.directory=reports/allure-results -``` - -**Step 2:** Add the following report parameters in your HyperExecute YAML file - -```yaml -report: true -partialReports: -location: reports -type: html -frameworkName: allure -``` - -> **NOTE:** The above example is only for the WDIO framework and the baseline concept remains the same across all the frameworks that the allure-results should be saved in a parent folder and that location of the parent folder is to be provided in the location parameters of reports in the HyperExecute YAML file. - -To enrich Allure reports with more specific details, we can use custom plugins. These plugins require configurations to function properly. - -To achieve this, you can use `configPath` flag within HyperExecute YAML. This specifies the location of the configuration file containing the custom plugin settings, allowing Allure to generate reports with the desired level of detail and this will be passed as `allure --config ` - -```bash -report: true -partialReports: -location: reports -# highlight-next-line -configPath: allure-config.yml -type: html -frameworkName: allure -``` - -## Allure Test Insights - -This helps you to get an overview of the test execution results using the Allure reports. This widget provides insights into the test execution results, test status, and test duration. - -> **NOTE :** Refer to the [Allure API Test Insights with HyperExecute documentation](/support/docs/analytics-allure-api-widgets/) for a comprehensive explanation diff --git a/static/docs/alm-octane-integration.md b/static/docs/alm-octane-integration.md deleted file mode 100644 index ece821a98..000000000 --- a/static/docs/alm-octane-integration.md +++ /dev/null @@ -1,110 +0,0 @@ -# ALM Octane Integration with TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -[ALM Octane](https://www.microfocus.com/en-us/products/alm-octane/overview) is an enterprise-grade Application Lifecycle Management (ALM) platform from Micro Focus. It provides comprehensive test management capabilities including test planning, test case management, defect tracking, and detailed analytics for your software development lifecycle. - -TestMu AI supports seamless integration with ALM Octane through your test automation scripts. If you already have ALM Octane configured in your test framework, your tests will work seamlessly when executed on TestMu AI's cloud infrastructure. - -## How ALM Octane Integration Works - -ALM Octane integration is implemented directly within your test automation scripts using ALM Octane's SDK or REST API. When you run your automated tests on TestMu AI's Selenium Grid or HyperExecute, the test results are automatically reported to your ALM Octane instance based on your script configuration. - -This script-based approach offers several advantages: - -- **Flexibility**: Configure exactly what data gets sent to ALM Octane -- **Customization**: Map test results to specific ALM Octane entities (test runs, test suites, etc.) -- **Seamless Execution**: No additional configuration required on TestMu AI platform - -## Prerequisites - -- An active [TestMu AI account](https://testmuai.com/register/) -- An ALM Octane instance with API access -- ALM Octane credentials (Client ID and Client Secret) -- Your test automation framework configured with ALM Octane SDK/API - -## Configuring ALM Octane in Your Test Scripts - -### Step 1: Set Up ALM Octane API Credentials - -In your ALM Octane instance, generate API credentials: - -1. Navigate to **Settings** > **Spaces** > **API Access** -2. Create a new API client with appropriate permissions -3. Note down the **Client ID** and **Client Secret** - -### Step 2: Install ALM Octane SDK - -For Java-based projects using Maven, add the ALM Octane SDK dependency: - -```xml - -com.microfocus.adm.almoctane.sdk -sdk-src -16.1.100 - -``` - -For other languages, refer to the ALM Octane API documentation in the Micro Focus online help. - -### Step 3: Configure Test Result Reporting - -Here's an example of how to configure your test framework to report results to ALM Octane: - -**Java (TestNG) Example:** - -```java -import com.hp.octane.integrations.OctaneSDK; - -public class ALMOctaneReporter { - -private static final String OCTANE_URL = "https://your-octane-instance.com"; -private static final String SHARED_SPACE_ID = "your-shared-space-id"; -private static final String WORKSPACE_ID = "your-workspace-id"; -private static final String CLIENT_ID = "your-client-id"; -private static final String CLIENT_SECRET = "your-client-secret"; - -public void reportTestResult(String testName, String status) { -// Initialize Octane SDK and report results -// Refer to ALM Octane SDK documentation for detailed implementation -} -} -``` - -### Step 4: Run Tests on TestMu AI - -Once your test scripts are configured with ALM Octane reporting, run them on TestMu AI: - -**Using Selenium Grid:** - -```java -DesiredCapabilities capabilities = new DesiredCapabilities(); -capabilities.setCapability("browserName", "Chrome"); -capabilities.setCapability("version", "latest"); -capabilities.setCapability("platform", "Windows 10"); -capabilities.setCapability("build", "ALM Octane Integration Build"); -capabilities.setCapability("name", "ALM Octane Test"); - -WebDriver driver = new RemoteWebDriver( -new URL("https://" + username + ":" + accessKey + "@hub.lambdatest.com/wd/hub"), -capabilities -); - -// Your test code here -// ALM Octane reporting happens automatically based on your script configuration -``` - -**Using HyperExecute:** - -Create your `hyperexecute.yaml` configuration and run tests as usual. The ALM Octane reporting configured in your scripts will work seamlessly. - -## Best Practices - -1. **Environment Variables**: Store ALM Octane credentials as environment variables rather than hardcoding them in scripts -2. **Error Handling**: Implement proper error handling for ALM Octane API calls to prevent test failures due to reporting issues -3. **Batch Reporting**: For large test suites, consider batch reporting to optimize API calls -4. **Test Mapping**: Maintain a clear mapping between your automated tests and ALM Octane test entities - -## Support - -If you encounter any issues with ALM Octane integration while running tests on TestMu AI, feel free to reach out to our **24/7 chat support** or email us at [support@testmuai.com](mailto:support@testmuai.com). diff --git a/static/docs/alttester-unity-game-automation.md b/static/docs/alttester-unity-game-automation.md deleted file mode 100644 index e9e48eda0..000000000 --- a/static/docs/alttester-unity-game-automation.md +++ /dev/null @@ -1,210 +0,0 @@ -# How to Automate Unity Games With AltTester on TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -If you build Unity mobile games and need to test them on real hardware, you can automate them with AltTester on TestMu AI Real Device Cloud. - -AltTester reads the live Unity scene graph, so your tests assert on game objects, components, and `PlayerPrefs` instead of raw screen coordinates. - -You upload an AltTester-instrumented build once, point the suite at TestMu AI, and run it with `pytest` against real Android and iOS devices in the cloud. - -This guide uses the [testmuai-alttester-unity-game-automation](https://github.com/hjsblogger/testmuai-alttester-unity-game-automation) sample, which automates the open-source TrashCat Unity game. - -This guide assumes you already have an AltTester-instrumented `.apk` or `.ipa`. Instrumenting a Unity build with the AltTester SDK happens inside the Unity editor and is out of scope here. See the [AltTester instrumentation documentation](https://alttester.com/docs/sdk/) for that step. - -## Prerequisites - -Before you run the suite, make sure the following are in place. - -- **Python 3.9 or higher**, with a virtual environment recommended. -- A **TestMu AI account.** The free tier works for this sample. If you don't have one, [register for free](https://www.testmuai.com/register/). -- An **AltTester-instrumented TrashCat build** (`.apk` for Android or `.ipa` for iOS) uploaded to TestMu AI App Automation. Note the `lt://` app URL returned after upload. The pre-instrumented TrashCat APK used by this sample is available from the [sample's download link](https://drive.google.com/file/d/1A1cB6KtaeTCY6XRfq148OtP_DPqhHEi5/view?usp=sharing). -- Your TestMu AI **Username** and **Access Key**, found under **Profile** in the dashboard. -- The **`LT` tunnel binary** at `tunnel/LT`. The sample repo ships the macOS binary. Download the binary for your platform from the [TestMu AI tunnel downloads](https://www.testmuai.com/support/docs/testing-locally-hosted-pages/) if you are on Windows or Linux. - -To upload your build and get the `lt://` URL, see [how to upload apps to the Real Device Cloud](/support/docs/upload-apps-on-real-device-cloud/). - -## How to Set Up the AltTester Unity Project - -Clone the sample, create an isolated Python environment, and install the four dependencies it needs. - -### Clone the Repository - -Clone the project and change into it. - -```bash -git clone https://github.com/hjsblogger/testmuai-alttester-unity-game-automation.git -cd testmuai-alttester-unity-game-automation -``` - -### Create a Virtual Environment and Install Dependencies - -Create and activate a virtual environment, then install the requirements. - -```bash -python -m venv venv -source venv/bin/activate # Windows: venv\Scripts\activate -pip install -r requirements.txt -``` - -The suite depends on four packages. AltDriver drives the Unity objects, Appium provisions the device, pytest runs the tests, and python-dotenv loads your credentials. - -| Package | Version | Purpose | -|---|---|---| -| `alttester-driver` | ≥ 2.2.5 | AltTester Python SDK (AltDriver) | -| `Appium-Python-Client` | ≥ 4.0.0 | Appium session management | -| `pytest` | ≥ 8.0.0 | Test runner | -| `python-dotenv` | ≥ 1.0.0 | Load `.env` credentials | - -Verified against alttester-driver 2.2.5, Appium-Python-Client 4.0.0, pytest 8.0.0, Python 3.9+, June 2026. - -### Configure Your TestMu AI Credentials - -The suite reads your credentials and app URL from a `.env` file in the project root. The file is gitignored, so create it yourself. - -```bash -touch .env -``` - -Add your TestMu AI credentials and the `lt://` URL of the app you uploaded. - -```ini -LT_USERNAME=your_testmu_ai_username -LT_ACCESS_KEY=your_testmu_ai_access_key -LT_APP_URL=lt://your_app_url -``` - -Each value comes from a specific place in the dashboard. - -| Variable | Where to find it | -|---|---| -| `LT_USERNAME` | TestMu AI dashboard, **Profile** section | -| `LT_ACCESS_KEY` | TestMu AI dashboard, **Profile** section | -| `LT_APP_URL` | **App Automation**, the uploaded app's `lt://` URL | - -Never commit `.env` or hard-code your Access Key in test files. The Access Key grants full API access to your account. Keep it in `.env` or a CI secret. - -## How to Configure the TestMu AI Capabilities - -The capabilities live in the `lt_options` dictionary in `tests/conftest.py`. This object tells TestMu AI which device to allocate and how to route the session through the tunnel. The sample sets it up for a real Android device and routes AltDriver traffic through a named tunnel. - -```python -lt_options = { -"user": username, -"accessKey": access_key, -"app": app_url, -"deviceName": "Pixel.*", -"platformVersion": "14", -"platformName": "android", -# "deviceName": "iPhone 14", -# "platformVersion": "16", -# "platformName": "ios", -"build": "AltTester TestMu AI demo with TrashCat", -"name": f"tests - {datetime.now().strftime('%B %d - %H:%M')}", -"isRealMobile": True, -"idleTimeout": 300, -"tunnel": True, -"tunnelName": TUNNEL_NAME, -} - -options = AppiumOptions() -options.set_capability("lt:options", lt_options) -options.set_capability("platformName", "android") - -appium_driver = appium_webdriver.Remote( -command_executor=f"https://{username}:{access_key}@mobile-hub.lambdatest.com/wd/hub", -options=options, -) -``` - -Every key in `lt_options` is required for this session to start and connect through the tunnel. The table below explains what each one controls and why it matters for an AltTester run. - -| Key | Required | What it does | -|---|---|---| -| `user` | Yes | Your TestMu AI username, read from `LT_USERNAME`. | -| `accessKey` | Yes | Your TestMu AI Access Key, read from `LT_ACCESS_KEY`. | -| `app` | Yes | The `lt://` URL of the instrumented build to install, read from `LT_APP_URL`. | -| `deviceName` | Yes | Device to allocate. `Pixel.*` is a regex that matches any available Pixel model. | -| `platformVersion` | Yes | OS version of the device, for example `14` for Android 14. | -| `platformName` | Yes | `android` or `ios`. Set it in both `lt_options` and the top-level capability. | -| `isRealMobile` | Yes | Allocates a physical device rather than an emulator. AltTester game tests need a real device. | -| `tunnel` | Yes | Routes the session through the LT tunnel so AltDriver can reach port `13000` on the device. | -| `tunnelName` | Yes | Binds the session to the named tunnel that `conftest.py` started. Must match the tunnel's `--tunnelName`. | -| `idleTimeout` | Optional | Seconds the session waits on an idle command before it is killed. Set to `300` here because AltDriver setup adds a startup delay. | -| `build` | Optional | Groups runs under one build name on the dashboard. | -| `name` | Optional | Per-test session name shown on the dashboard. | - -The `tunnel` and `tunnelName` keys are the non-obvious part. AltDriver does not connect to TestMu AI directly. It connects to the AltTester Server running inside the app on the cloud device, and the only path to that in-app server is through the tunnel. If `tunnel` is false or `tunnelName` does not match the running tunnel, the Appium session still starts but AltDriver cannot connect. - -The tunnel itself is launched by `conftest.py` with the binary at `tunnel/LT`. The session starts it with these flags before any test runs. - -```python -[ -"./tunnel/LT", -"--user", username, -"--key", access_key, -"--tunnelName", TUNNEL_NAME, -"--verbose", -"--infoAPIPort", str(TUNNEL_INFO_PORT), -] -``` - -`--infoAPIPort` exposes the tunnel's local info API. `conftest.py` polls that port until the tunnel reports ready, then starts the Appium session. `--tunnelName` must be the same value used in `tunnelName` inside `lt_options`. - -## How to Run the Unity Game Tests - -With the environment configured and the tunnel binary in place, run the suite with `pytest`. The fixtures start the tunnel and the Appium session automatically, so you do not start them by hand. - -Run the full suite. - -```bash -pytest -``` - -Run a single test file with verbose output. - -```bash -pytest tests/test_start_page.py -v -pytest tests/test_main_menu.py -v -pytest tests/test_game_play.py -v -pytest tests/test_store.py -v -pytest tests/test_user_journey.py -v -``` - -Run one test by name. - -```bash -pytest tests/test_main_menu.py::TestMainMenu::test_main_menu_page_loaded_correctly -v -``` - -The tunnel forwards the WebSocket connection to port `13000` on the cloud device for the entire run, so AltDriver stays connected to the in-app AltTester Server from the first test to the last. - -As the suite runs, each test logs its progress in the terminal while the tunnel stays active. - -Track the session live on the [TestMu AI automation dashboard](https://automation.lambdatest.com/build?pageType=build), where each step annotation pushed through `lambdatest_executor` appears against the test. - -Open a session to see its annotated steps, status, and logs. - -## How to Change the Target Device - -The suite runs on **Pixel 8 (Android 14)** by default. To target a different device or switch to iOS, edit the `lt_options` block in `tests/conftest.py`. The commented iOS lines show the keys to change. - -```python -# Android -"deviceName": "Pixel.*", -"platformVersion": "14", -"platformName": "android", - -# iOS: uncomment and adjust -# "deviceName": "iPhone 14", -# "platformVersion": "16", -# "platformName": "ios", -``` - -When you switch platforms, change `LT_APP_URL` to the matching build. An Android `lt://` URL points to an `.apk` and an iOS one to an `.ipa`. The app must be instrumented with AltTester for the platform you target. To confirm the device name and OS version are available, check the [supported real device list](/support/docs/app-testing-on-real-devices/). - -## Related TestMu AI Guides - -- See [how to upload apps to the Real Device Cloud](/support/docs/upload-apps-on-real-device-cloud/) to get the `lt://` app URL this suite needs. -- See [how to get started with Real Device App Testing](/support/docs/app-testing-on-real-devices/) for manual testing on the same device pool. -- See [how to set up the TestMu AI tunnel](/support/docs/testing-locally-hosted-pages/) for tunnel binaries on Windows and Linux. diff --git a/static/docs/analytics-ai-root-cause-analysis.md b/static/docs/analytics-ai-root-cause-analysis.md deleted file mode 100644 index 99050220c..000000000 --- a/static/docs/analytics-ai-root-cause-analysis.md +++ /dev/null @@ -1,320 +0,0 @@ -# AI Root Cause Analysis (AI RCA) - Test Intelligence - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -AI Root Cause Analysis (AI RCA) in TestMu AI Analytics uses advanced AI-powered LLM technology to automatically analyze failed tests and identify their root causes. Instead of manually sifting through logs and error messages, AI RCA provides: - -- **Intelligent root cause identification** - distinguishes between primary causes and cascading symptoms -- **Actionable fixes** - specific recommendations to resolve issues -- **Error timelines** - chronological sequence of events leading to failures -- **Trend analysis** - identifies recurring patterns across test executions - -## Prerequisites for AI RCA - -1. **Active TestMu AI Account**: You should have an active TestMu AI account with appropriate permissions. -2. **Subscription Plan**: This feature is available for users with HyperExecute or App/Web Automation subscription plans. -3. **Available Credits**: Tests will only be processed through RCA when sufficient credits are available. For more information on credit management, see our [Credits Management](/docs/credits-management/) documentation. -4. **Test Failures**: AI RCA requires at least one test failure to generate analysis. The system learns from your test execution patterns. -5. **Access to Insights**: You should have access to the TestMu AI Insights platform under **Insights** tab. - -## RCA Generation Methods - -TestMu AI offers two methods to generate Root Cause Analysis: - -- **Automatic RCA**: RCA is generated automatically for failed tests based on your configuration settings. Set it up once in Organization Settings, and RCA runs continuously for matching test failures. - -- **Manual RCA**: Generate RCA on-demand for any failed test by clicking the **Generate RCA** button. Useful for investigating specific failures without setting up targeting rules. - -**When to Use Each Method** -- Use **Automatic RCA** for continuous monitoring of critical test suites where you want immediate analysis of every failure. -- Use **Manual RCA** for ad-hoc debugging when you need to investigate a specific test failure without setting up automated rules. - -| Feature | Automatic RCA | Manual RCA | -|---------|---------------|------------| -| **Trigger** | Runs automatically when configured tests fail | Click "Generate RCA" button | -| **Targeting Rules** | Follows Analysis Scope and Intelligent Targeting settings | Any failed test on-demand | -| **Custom RCA Categories** | ✓ Applied | ✓ Applied | -| **Special Instructions** | ✓ Applied | ✓ Applied | -| **Best For** | Continuous monitoring | Ad-hoc debugging | - -**Shared Configuration** -**Custom RCA Categories** and **Special Instructions** configured in Organization Settings apply to both Automatic and Manual RCA generation. This ensures consistent categorization and analysis context across all RCA results. - -## Generate RCA Manually - -You can manually generate RCA for any failed test from the **Test Manager** or **HyperExecute** dashboards. - -### From Test Manager (TMS) - -1. Navigate to **Test Manager** → open a **Test Run** -2. Go to the **Test Instances** tab -3. Locate the failed test and click the **Generate RCA** button -4. View the RCA output with root cause, severity, and recommended fixes - -### From HyperExecute - -1. Navigate to **HyperExecute** → open the failed **Job** -2. Select the **Tasks** tab and expand the failed scenario -3. Click the **Generate RCA** button next to the failed test -4. View the generated RCA with actionable insights - -### From Insights Dashboard - -The Insights dashboard provides aggregated RCA analytics and trend analysis across all your test executions. - -1. Navigate to **Insights** from the TestMu AI dashboard -2. Access the **RCA Category Trends** widget to view aggregated data -3. Click on specific categories to drill down into detailed analysis -4. Track historical trends to monitor improvement over time - -## Understanding RCA Output - -When you generate or view an RCA, a detailed analysis panel opens with the following components: - -### Error Classification - -At the top of the RCA panel, you'll see: - -- **Failure Category**: The type of failure (e.g., `App Bug`, `Product Bug`) -- **Root Cause Type**: The specific error category (e.g., `JavaScriptError`, `Backend Contract Violation`) -- **Error Summary**: A one-line description of the error with file location -- **Error Trends (Test)**: Shows how often this specific test has failed recently -- **Error Trends (Global)**: Shows the error pattern across all tests over time - -### Detailed Analysis - -Expand this section to see AI-generated insights explaining: -- Why the component or test failed -- What the error indicates about the underlying issue -- The specific file and line number where the error occurred - -### Event Timeline - -The Event Timeline shows the chronological sequence of events during test execution: - -- **✓ Passed steps**: Steps that completed successfully (e.g., "Test Execution Started") -- **✗ Failed steps**: Steps where issues occurred, tagged as: - - **Root Cause**: The primary reason for failure - - **Effect**: Secondary failures caused by the root cause - -Click **Show/Hide** on any event to expand details including: -- **Code**: View the relevant source code -- **Stacktrace**: See the full error stack trace -- **Assets**: Access screenshots, logs, or other artifacts - -### How to Fix It - -At the bottom of the RCA panel, you'll find numbered, actionable steps to resolve the issue. These recommendations include: -- Specific actions to take in your application or test -- Configuration changes to consider -- Debugging steps to verify the fix - -### Provide Feedback - -Help improve RCA accuracy using the feedback options at the bottom: -- **Thumbs up/down**: Rate if the analysis was helpful -- **Suggest Improvements**: Click to open a feedback form where you can: - - Adjust the failure category and root cause classification - - Provide a custom root cause if the AI classification was incorrect - - Add additional context or feedback - -## Configure Automatic RCA - -Set up Automatic RCA to continuously analyze test failures based on your targeting rules. - -### Step 1: Access Organization Settings - -1. Navigate to **Organization Settings** in your TestMu AI dashboard -2. In the left sidebar, expand the **Insights** section under **Org Product Preferences** -3. Select **Automatic AI RCA** from the available options - -### Step 2: Enable AI RCA - -**Toggle the Feature**: Use the blue toggle switch to enable "Automatic AI RCA" - -### Step 3: Configure Analysis Scope - -In the **Analysis Scope** section, choose which types of test failures to analyze: -- **All failures**: Analyze every failed test, regardless of previous status -- **New failures**: Analyze only tests that have failed recently after having passed at least 10 consecutive times previously. -- **Consistent Failures**: Analyze only tests that have failed in all of their previous 5 runs to identify persistent issues. - -### Step 4: Configure Intelligent Targeting - -Configure intelligent targeting rules to precisely control which tests, builds, tags, projects, or jobs are included in AI-powered analysis: - -1. **Add Targeting Rules**: Enter regex patterns in the input field -2. **Click Include (+) or Exclude (-)**: Choose whether to include or exclude matching tests -3. **Configure Multiple Criteria**: Set targeting rules for: - - **Test Names**: Target specific test suites or test patterns - - **Build Names**: Include or exclude builds with specific names (e.g., hourly, nightly) - - **Test Tags**: Include or exclude tests with specific tags (e.g., playwright_test, atxHyperexecute_test) - - **Build Tags**: Include or exclude builds with specific tags (e.g., hourly, nightly) - - **Project Names**: Include or exclude tests from specific projects using regex patterns - - **Job Labels**: Include tests with specific job labels or tags - -#### Rule Logic and Application - -The intelligent targeting system applies rules using the following logic: - -**Rule Evaluation Process:** -1. **Include Rules (AND Logic)**: All Include rules within the same category must match for a test to be considered -2. **Exclude Rules (OR Logic)**: Any Exclude rule that matches will immediately exclude the test from analysis -3. **Cross-Category Logic**: Include rules across different categories (Test Names, Build Tags, Project Names, etc.) must ALL match -4. **Exclusion Precedence**: Exclude rules take priority over Include rules - if any exclude rule matches, the test is excluded regardless of include matches - -**Best Practices for Rule Configuration:** -- **Start Broad**: Begin with general include rules, then add specific exclusions -- **Use Specific Patterns**: Avoid overly broad regex patterns that might include unintended tests -- **Test Your Rules**: Verify rule behavior with sample test names and tags before applying -- **Regular Review**: Periodically review and update rules based on changing test patterns - -#### Example Configuration for Production Test Analysis - -**Test Name:** -- **Include**: `.*prod.*` - Only analyze tests with name containing "prod" -- **Exclude**: `.*non-critical.*` - Skip tests with name containing "non-critical" - -**Build Tags:** -- **Include**: `^hourly` - Only analyze builds with tag starting with "hourly" - -**Test Tags:** -- **Include**: `playwright_test|atxHyperexecute_test` - Focus on specific test frameworks -- **Exclude**: `.*smoke.*` - Skip smoke tests - -**Project Names:** -- **Include**: `^ecommerce|^payment` - Only analyze tests from projects starting with "ecommerce" or "payment" -- **Exclude**: `.*staging.*` - Skip tests from staging projects - -**Result**: AI-powered analysis will run only on production tests (excluding non-critical ones) from hourly builds, focusing on Playwright or HyperExecute test tags, while excluding smoke tests. The analysis will target ecommerce and payment projects, excluding staging projects. This configuration helps narrow down analysis to the most critical test scenarios. - -### Step 5: Manage Custom RCA Categories (Optional) - -Custom RCA Categories allow you to define intelligent classification categories that automatically categorize and organize test failure analysis results. This helps you group similar failures together, track trends, and prioritize fixes more effectively. - -#### Managing Categories - -1. In the **Automatic AI RCA** configuration page, locate the **Custom RCA Categories** section -2. Click the **Manage** button to open the category management drawer -3. **Create**: Click **Add Category**, enter a name and description, select **Active** or **Inactive** status, then click **Create RCA Category** -4. **Edit**: Click the edit icon on any category card to modify its details -5. **Delete**: Click the delete icon and confirm to remove a category -6. **Search**: Use the search box to filter categories by name or description - -**Category Status:** -- **Active**: Used by AI for automatic classification and appears in RCA results -- **Inactive**: Saved but not used for classification; can be reactivated later - -**Best Practices:** - -- **Be Specific**: Create distinct categories (e.g., "Database Connection Timeouts" vs "Database Issues") -- **Use Clear Names**: Choose names your team understands immediately -- **Start Small**: Begin with 5-10 active categories for your most common failure types -- **Review Regularly**: Periodically refine categories based on your failure patterns - -**Example Custom RCA Categories:** - -| Category Name | Description | -|--------------|-------------| -| **UI Element Not Found** | Failures where tests cannot locate expected UI elements due to timing issues, selector changes, or DOM modifications | -| **API Timeout Errors** | Failures caused by API requests exceeding timeout thresholds, often related to third-party service reliability | -| **Database Connection Issues** | Failures due to database connection pool exhaustion, connection timeouts, or query performance problems | -| **Authentication Token Expiration** | Failures related to expired or invalid authentication tokens, session timeouts, or OAuth refresh issues | -| **Network Connectivity Issues** | Failures caused by network interruptions, DNS failures, proxy issues, or unstable network connections | - -### Step 6: Set Special Instructions (Optional) - -Provide context or specific guidance for the AI to consider during analysis: - -1. Click on the **Special Instructions** section -2. Enter any special instructions or context that should be considered during AI root cause analysis -3. Use the "Show examples" link for guidance on effective instruction writing - -**Example Instructions:** - -**Environment-Specific Context:** -- Running on Staging environment with test data -- Database may have lag issues during peak hours (9 AM - 5 PM EST) -- Test environment has limited resources compared to production (2GB RAM vs 8GB) -- Network latency is higher in test environment (average 150ms vs 50ms in production) - -**Known Issues & Patterns:** -- Payment gateway timeouts during high traffic periods (especially between 2-4 PM) -- Cache invalidation issues occur immediately after deployments -- Third-party API rate limits: Salesforce (1000 requests/hour), HubSpot (500 requests/hour) -- Database connection pool is limited to 20 connections - look for pool exhaustion patterns -- OAuth token expiration happens every 24 hours - failures around token refresh time are expected - -**Analysis Preferences:** -- Focus on recent failures over recurring issues when prioritizing -- Consider browser compatibility differences (Chrome vs Firefox behavior variations) -- Check for timing-related failures (elements loading asynchronously) -- Distinguish between environment-specific issues vs application bugs -- Prioritize failures affecting core user journeys: Login, Checkout, Dashboard, Profile Management - -**Business Context:** -- Critical user journeys: Login, Checkout, Dashboard, Profile Management -- Performance thresholds: Page load < 3s, API response < 500ms -- Peak usage hours: 10 AM - 2 PM and 6 PM - 9 PM EST -- High-value features: Payment processing, Order management, Customer support portal - -**Technical Constraints:** -- Flaky network connections in mobile tests (use retry logic) -- Third-party service dependencies may be unstable (payment gateway, email service) -- Custom error handling: All errors logged to Sentry, user-friendly messages displayed -- Test data cleanup runs nightly - some data may be stale during day - -**Ignore These Common False Positives:** -- Browser console warnings that don't affect functionality -- Network requests to analytics services (Google Analytics, Hotjar, Mixpanel) -- Minor UI layout shifts that don't break functionality (< 5px) -- Expected 404s for optional resources (favicon, tracking pixels) -- Third-party script loading delays that don't impact core functionality - -### Step 7: Save Configuration - -1. Click **Save Configuration** to apply your settings -2. The settings will be applied to all users in your organization and cannot be modified by individual users or need admin level privileges. - -## Best Practices - -- **Focus on Root Cause events** in the Event Timeline rather than Effect events when debugging -- **Check Error Trends** to see if this is a recurring issue or a new failure -- **Use the Code and Stacktrace tabs** to pinpoint the exact location of the error -- **Follow the "How to Fix It" steps** systematically for faster resolution -- **Provide feedback** using thumbs up/down to help improve RCA accuracy over time -- **Monitor trends in Insights** to identify patterns and prioritize fixes - -## Troubleshooting Common Issues - -RCA Not Generating - -- **Check prerequisites**: Ensure you have the required subscription plan -- **Verify credits availability**: AI RCA processing requires credits to be available in your account. Check your [Credits Management](/docs/credits-management/) page to ensure sufficient credits -- **Verify test failures**: AI RCA requires actual test failures to analyze -- **Review configuration**: Confirm AI RCA is enabled in organization settings -- **Check permissions**: Ensure you have access to Analytics and Test Intelligence features - -Inaccurate RCA Results - -- **Refine special instructions**: Provide more specific context about your application -- **Update intelligent targeting**: Exclude irrelevant tests that might confuse the analysis -- **Review error categorization**: Ensure test failures are properly categorized -- **Refine custom RCA categories**: Update category descriptions to better match your failure patterns -- **Provide feedback**: Use any available feedback mechanisms to improve accuracy - -Missing RCA Data - -- **Check time range**: Ensure you're looking at the correct time period -- **Verify test execution**: Confirm tests actually failed during the specified period -- **Review dashboard filters**: Check if any filters are excluding relevant data -- **Contact support**: Reach out if RCA data appears to be missing - -## Support - -For any queries or issues related to AI Root Cause Analysis, please reach out to our [24/7 customer support](mailto:support@testmuai.com). We're here to help you maximize the value of this powerful debugging tool! - -**Related Documentation:** -- [Smart Tags - Test Intelligence](/docs/analytics-smart-tags-test-intelligence/) -- [Failure Categorization AI](/docs/analytics-test-failure-classification/) -- [Test Insights Overview](/docs/analytics-test-insights/) diff --git a/static/docs/analytics-allure-api-widgets.md b/static/docs/analytics-allure-api-widgets.md deleted file mode 100644 index f5f926ade..000000000 --- a/static/docs/analytics-allure-api-widgets.md +++ /dev/null @@ -1,76 +0,0 @@ -# Allure Test Insights with HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The `Allure Test Insights` module helps you to get an overview of the test execution results using the Allure. This widget provides insights into the test execution results, test status, and test duration. - -## How To Use Allure Test Insights? - -**Please Note** -The `Allure Test Insights` widget is available for the `HyperExecute` product only. You need to have access to the `HyperExecute` product to use these widgets. - -Additionally, you need to have the Allure report generated for your test cases within the `HyperExecute` YAML file changes and add the job label for the test cases to be displayed in the widget. - -Read more about the [HyperExecute Reports](/docs/allure-reports/) product to get started. - -You can check the Allure Test Insights for your test environments by following the below steps: - -- Navigate to the `Insights` module from the left navigation menu. -- Create a new `Custom` dashboard or select an existing dashboard. -- Click on the `HyperExecute` product. -- Search for the `Allure Test` widgets from the list of available widgets. -- Add the required widget to your dashboard. - -## What are the Key Insights Provided by Allure Test Insights Widget? - -The `Allure Test Insights` widget provides the following key insights: - -- **Test Execution Results**: The test execution results for the test cases executed using the Allure report. -- **Test Status**: The status of the test cases executed using the Allure report. -- **Test Duration**: The duration of the test cases executed using the Allure report. -- **Test Suite**: The suite in which the test cases were executed using the Allure report. -- **Test Method**: The method in which the test cases were executed using the Allure report. -- **Job Labels**: The job labels associated with the test cases executed using the Allure report. -- **Test History**: The history of the test cases executed using the Allure report. - -## Allure Test Trends - -The `Allure Test Trends` widget provides insights into the test execution results, test status, and test duration for the test cases executed using the Allure report. This widget helps you to get an overview of the test execution results and identify the bottlenecks in the test execution process. - -#### Use Cases of Allure Test Trends Widget -- Get insights about the time series trends of the test execution count. -- Identify the trends of the tests run over a period of time. -- Monitor for any anomalies in the test execution counts. - -## Allure Suite Health - -The `Allure Suite Health` widget provides insights into the health of the test suites executed using the Allure report. This widget helps you to get an overview of the test suite health and identify the test suites that require attention. - -#### Use Cases of Allure Suite Health Widget -- Get insights into the health of the test suites executed using the Allure report. -- Identify the test suites that require attention based on the test status. -- Monitor the test suite health trends over time. - -## Allure Test Status Ratio - -The `Allure Test Status Ratio` widget provides insights into the test status ratio for the test cases executed using the Allure report. This widget helps you to get an overview of the test status ratio and identify the test cases that require attention. - -#### Use Cases of Allure Test Status Ratio Widget -- Get insights into the test status ratio for the test cases executed using the Allure report. -- Check for any anomalies in the test status ratio and identify the test cases that require attention. -- Monitor the test status ratio trends over time and take corrective actions. - -## Allure Test Duration - -The `API Test Duration` widget provides insights into the average test duration for the test cases executed using the Allure report. This widget helps you to get an overview of the test duration and identify the test cases that require attention. - -#### Use Cases of Allure Test Duration Widget -- Get insights into the average test duration for the test cases executed using the Allure report. -- Identify the test cases with high test duration and optimize the test execution process. -- Monitor the test duration trends over time and reduce the test duration for better efficiency. - -## Support and Assistance - -We understand that adopting new technologies can sometimes be challenging. If you encounter any difficulties or have questions regarding the Allure API Insights with HyperExecute, our dedicated support team is here to assist you. Feel free to reach out to our [24/7 customer support](mailto:support@testmuai.com) for expert guidance and troubleshooting. We are committed to ensuring your success and providing you with the best possible experience while using the Allure API Insights with HyperExecute. - -Embrace the future of data analysis with the Allure API Insights with HyperExecute and unlock the full potential of your data. Start exploring, asking questions, and discovering valuable insights today! diff --git a/static/docs/analytics-bar-line-widgets.md b/static/docs/analytics-bar-line-widgets.md deleted file mode 100644 index 4b259127c..000000000 --- a/static/docs/analytics-bar-line-widgets.md +++ /dev/null @@ -1,350 +0,0 @@ -# Bar and Line Chart Widgets - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Bar and Line chart widgets in LambdaTest Insights enable you to visualize test data trends, comparisons, and patterns. Bar charts are ideal for comparing values across categories, while line charts excel at showing trends and changes over time. - -## What are Bar and Line Chart Widgets? - -### Bar Chart Widgets - -Bar chart widgets display data using rectangular bars where the length of each bar represents the value. They are particularly effective for: - -- **Comparing values** across different categories (e.g., test counts by browser, pass rates by OS) -- **Showing distributions** of data across groups -- **Highlighting differences** between categories at a glance -- **Grouped comparisons** when using the Group By feature - -### Line Chart Widgets - -Line chart widgets display data as a series of points connected by lines. They are particularly effective for: - -- **Tracking trends** over time (e.g., test execution trends, pass rate changes) -- **Identifying patterns** and anomalies in time series data -- **Comparing multiple series** on the same chart -- **Forecasting** based on historical patterns - -## Prerequisites - -Before creating a bar or line chart widget, ensure you have completed the [common prerequisites and setup steps](/docs/dashboards-custom-widgets#prerequisites) for custom widgets. - -## Step-by-Step Guide to Creating Bar and Line Chart Widgets - -**Common Steps** -For prerequisites and common setup steps (Navigate to Insights, Create Dashboard, Choose Product, Select Data Source, Configure Filters), see the [Common Steps for Creating Custom Widgets](/docs/dashboards-custom-widgets#common-steps-for-creating-custom-widgets) section in the main Custom Widgets documentation. - -### Step 6: Choose Visualization Type - -1. In the **Choose Visualisation** section, select either: - - **Bar** for bar chart visualization - - **Line** for line chart visualization - -### Step 7: Configure What to Plot - -This is the most critical step for creating meaningful charts: - -#### X-Axis Configuration - -1. **X-Axis Field*** (Mandatory): Select the field to display on the horizontal axis. This determines what categories or time points will be shown. - - For time-based trends: Use fields like `create_timestamp`, `start_time`, or `end_time` - - For categorical comparisons: Use fields like `browser`, `os`, `status`, `project_name` - -2. **Label for X-Axis** (Optional): Provide a custom label for the X-axis to make your chart more readable (e.g., "Test Execution Date", "Browser Type"). - -#### Y-Axis Configuration - -1. **Y-Axis Field*** (Mandatory): Select the field to aggregate and display on the vertical axis. - - Common fields: `test_id`, `duration`, or any numeric field - -2. **Aggregation Function*** (Mandatory): Choose how to aggregate the Y-axis field: - - **Count**: Number of records (most common for test counts) - - **Sum**: Sum of values - - **Average**: Average value - - **Minimum**: Minimum value - - **Maximum**: Maximum value - - **Unique Count**: Count of unique values - - **Percentiles** (50th, 75th, 90th, 95th, 99th): For duration and other numeric fields - -3. **Label for Y-Axis** (Optional): Provide a custom label for the Y-axis (e.g., "Number of Tests", "Average Duration (seconds)"). - -For a complete list of available keys for each product, see the [Available Keys for Custom Widgets](/docs/dashboards-custom-widgets#available-keys-for-custom-widgets) section in the main Custom Widgets documentation. - -### Step 8: Configure Group By (Optional) - -The **Group By** field allows you to create multiple series on the same chart: - -1. Select a field to group by (e.g., `status`, `browser`, `os`). -2. This will create separate bars or lines for each unique value in the selected field. -3. Each group will be displayed with a different color and appear in the legend. - -**Example**: Grouping by `status` will show separate bars/lines for "passed", "failed", "queued", etc., allowing you to compare test statuses over time or across categories. - -### Step 9: Name Your Widget - -1. Enter a descriptive name in the **Widget Name** field (e.g., "Test Execution Trends by Status", "Browser Comparison - Test Counts"). -2. Use clear, descriptive names that indicate what the chart shows. - -### Step 10: Preview Your Chart - -1. Click the **Preview** button to see how your chart will look with the current configuration. -2. Review the visualization to ensure it displays the data as expected. -3. Use the refresh icon to update the preview if you make changes. - -### Step 11: Create the Dashboard - -1. Enter a **Dashboard Name** in the bottom section. -2. Click **Create Dashboard** to save your chart widget. -3. Your chart will be added to the dashboard and ready for analysis. - -## How Bar and Line Charts Work - -Understanding how charts encode information is crucial for effective analysis: - -### X-Axis: Categories or Time - -The **X-axis** represents the categories or time points you want to analyze: - -- **Time-based X-axis**: Use timestamp fields to show trends over time. The chart will display data points chronologically. -- **Category-based X-axis**: Use categorical fields (browser, OS, status) to compare values across different groups. - -### Y-Axis: Aggregated Values - -The **Y-axis** represents the aggregated metric you want to visualize: - -- The value is calculated using the selected aggregation function (Count, Sum, Average, etc.) -- For each X-axis value, the Y-axis shows the aggregated result -- Multiple Y-axis values can be shown when using Group By - -### Group By: Multiple Series - -When you use **Group By**, the chart displays multiple series: - -- Each unique value in the Group By field becomes a separate series -- Each series is displayed with a different color -- A legend shows what each color represents -- This allows you to compare multiple dimensions simultaneously - -**Example**: -- X-axis: `create_timestamp` (time) -- Y-axis: `test_id` with Count aggregation -- Group By: `status` - -This creates a chart showing test counts over time, with separate lines/bars for each status (passed, failed, queued, etc.). - -## Generating Meaningful Insights from Charts - -### Understanding Bar Charts - -- **Bar Height**: Indicates the magnitude of the value -- **Bar Comparison**: Compare heights across categories to identify patterns -- **Stacked Bars**: When using Group By, bars are displayed as stacked segments, with each segment representing a different group value (e.g., different statuses). Each segment is color-coded according to the legend, allowing you to see both the total value and the breakdown within each bar -- **Segment Analysis**: Each colored segment in a stacked bar represents the contribution of that group to the total value - -### Understanding Line Charts - -Line charts display multiple data series as colored lines, each representing a different group when using Group By. Key elements to understand: - -- **Trend Direction**: Upward trends indicate growth, downward trends indicate decline -- **Steepness**: Steep lines indicate rapid changes -- **Plateaus**: Flat sections indicate stable periods -- **Intersections**: Where lines cross indicates when values become equal -- **Gaps**: Missing data points may indicate issues or gaps in testing -- **Multiple Series**: When using Group By, each unique value becomes a separate line with its own color, allowing you to compare trends across different groups simultaneously -- **Data Points**: Each point on a line represents the aggregated value at that X-axis position -- **Legend**: The legend shows which color corresponds to which group, helping you identify each series - -### Key Insights to Look For - -1. **Trends Over Time**: Identify increasing, decreasing, or stable patterns in your test execution -2. **Peaks and Valleys**: Spot unusual spikes or drops that may indicate issues or changes -3. **Comparisons**: Compare values across categories to identify best/worst performers -4. **Correlations**: When using Group By, observe relationships between different series -5. **Anomalies**: Unexpected patterns that deviate from normal behavior -6. **Seasonality**: Recurring patterns that repeat at regular intervals - -### Interacting with Charts - -Bar and line charts provide interactive features to explore your data in detail: - -- **Hover**: Hover over bars (in bar charts) or data points (in line charts) to see detailed tooltips showing: - - The exact X-axis value (timestamp or category) - - Breakdown of values for each group (when using Group By) - - Specific counts or aggregated values for each series - - Color-coded indicators matching the legend - - **Example for Bar Charts**: When hovering over a bar in a chart grouped by `status`, the tooltip will show the timestamp or category, along with separate counts for "passed", "failed", "queued", "completed", etc., each with its corresponding color indicator. - - **Example for Line Charts**: When hovering over a data point on a line chart grouped by `status`, the tooltip displays the exact timestamp and shows the breakdown of values for all statuses at that point in time (e.g., passed: 4,721, failed: 1,015, error: 24, etc.), allowing you to see the complete picture for that specific moment. - -- **Legend Interaction**: Click on legend items to show/hide specific series, allowing you to focus on particular groups -- **Zoom and Pan**: Some chart configurations may support zooming and panning for detailed exploration -- **Export**: Charts can be exported or shared as part of your dashboard - -## Use Cases for Bar and Line Chart Widgets - -### 1. Test Execution Trends Over Time - -**Use Case**: Track the number of tests executed over time to monitor testing activity. - -**Configuration**: -- X-Axis: `create_timestamp` or `start_time` -- Y-Axis: `test_id` with Count aggregation -- Group By: `status` (optional, to see breakdown by status) -- Visualization: Line Chart - -**Insight**: Identify testing activity patterns, peak testing periods, and trends in test execution volume. When grouped by status, you can see separate trend lines for passed, failed, and other test statuses, making it easy to compare how different statuses change over time. - -### 2. Test Status Distribution by Browser - -**Use Case**: Compare test results across different browsers to identify browser-specific issues. - -**Configuration**: -- X-Axis: `browser` -- Y-Axis: `test_id` with Count aggregation -- Group By: `status` -- Visualization: Bar Chart - -**Insight**: Quickly identify which browsers have the most failures and which perform best. - -### 3. Average Test Duration by OS - -**Use Case**: Analyze test execution times across different operating systems. - -**Configuration**: -- X-Axis: `os` -- Y-Axis: `duration` with Average aggregation -- Visualization: Bar Chart - -**Insight**: Identify OS-specific performance differences and optimize testing strategy. - -### 4. Test Count Trends by Project - -**Use Case**: Monitor testing activity across different projects over time. - -**Configuration**: -- X-Axis: `create_timestamp` -- Y-Axis: `test_id` with Count aggregation -- Group By: `project_name` -- Visualization: Line Chart - -**Insight**: Track project-specific testing trends and ensure balanced testing across projects. - -### 5. Pass Rate Trends Over Time - -**Use Case**: Monitor test pass rates to track quality improvements or regressions. - -**Configuration**: -- X-Axis: `create_timestamp` -- Y-Axis: Calculate pass rate (may require custom aggregation or filtering) -- Visualization: Line Chart - -**Insight**: Identify quality trends and correlate with releases or changes. - -### 6. Browser Version Comparison - -**Use Case**: Compare test execution across different browser versions. - -**Configuration**: -- X-Axis: `browser_version` -- Y-Axis: `test_id` with Count aggregation -- Group By: `status` -- Visualization: Bar Chart - -**Insight**: Identify problematic browser versions and ensure comprehensive version coverage. - -### 7. Daily Test Execution Summary - -**Use Case**: Get a daily overview of test execution activity. - -**Configuration**: -- X-Axis: `create_timestamp` (grouped by day) -- Y-Axis: `test_id` with Count aggregation -- Group By: `status` -- Visualization: Bar Chart or Line Chart - -**Insight**: Understand daily testing patterns and identify days with unusual activity. - -## Available Keys for Bar and Line Chart Widgets - -For a complete list of available keys that can be used for X-axis, Y-axis, and Group By fields in bar and line chart widgets, see the [Available Keys for Custom Widgets](/docs/dashboards-custom-widgets#available-keys-for-custom-widgets) section in the main Custom Widgets documentation. - -Choose keys that represent categorical dimensions (like browser, OS, project name) for X-axis, and numeric or countable fields for Y-axis aggregation. For time-based trends, use timestamp fields for the X-axis. - -## Best Practices for Bar and Line Chart Widgets - -### 1. Choose Appropriate Chart Type - -- **Use Bar Charts** for: - - Comparing values across categories - - Showing distributions - - When categories are discrete and not time-based - -- **Use Line Charts** for: - - Showing trends over time - - Comparing multiple time series - - Identifying patterns in sequential data - -### 2. Select Meaningful Axes - -- **X-Axis**: Choose fields that create logical groupings or time sequences -- **Y-Axis**: Select fields that provide meaningful metrics when aggregated -- **Aggregation**: Choose the right aggregation function (Count for test counts, Average for durations, etc.) - -### 3. Use Group By Strategically - -- Use Group By to compare multiple dimensions simultaneously -- Avoid too many groups (more than 5-7) as it can make charts cluttered -- Choose grouping fields that create meaningful comparisons - -### 4. Apply Filters Effectively - -- Use filters to focus on relevant subsets of data -- Combine multiple filters to create highly targeted charts -- Filter by time ranges, projects, or other relevant dimensions - -### 5. Name Widgets Clearly - -Use descriptive names that indicate what the chart shows, including: -- The metric being visualized -- The time period or categories -- Any grouping applied - -### 6. Regular Review and Updates - -- Review charts regularly to identify new patterns or issues -- Update filters and dimensions as your testing strategy evolves -- Create multiple charts with different configurations to explore various aspects - -## Troubleshooting - -### Chart Shows No Data - -- **Check Filters**: Ensure filters aren't too restrictive -- **Verify Data Source**: Confirm you have test data for the selected product -- **Check Field Selection**: Ensure the selected X-axis and Y-axis fields have data -- **Verify Time Range**: If using time-based X-axis, ensure data exists in the selected time period - -### Chart is Too Cluttered - -- **Reduce Group By Values**: Use filters to limit the number of groups -- **Simplify X-Axis**: Choose X-axis fields with fewer unique values -- **Use Filters**: Narrow down the data using filters - -### Chart Doesn't Show Expected Trends - -- **Check Aggregation**: Verify the aggregation function is appropriate for your Y-axis field -- **Review Data Range**: Ensure you have sufficient data points -- **Adjust Time Granularity**: For time-based charts, the granularity may affect trend visibility - -## Next Steps - -After creating your bar or line chart widget: - -1. **Add to Dashboard**: Combine charts with other widgets for comprehensive analysis -2. **Share Dashboard**: Share your dashboard with team members for collaborative insights -3. **Create Variations**: Create multiple charts with different configurations to explore various aspects -4. **Review Regularly**: Use insights from charts to identify areas needing attention and track improvements - -## Support - -If you have any questions or need assistance with creating bar or line chart widgets, please reach out to our [24/7 customer support](mailto:support@testmuai.com) or visit our [Support Center](https://www.testmuai.com/support/). diff --git a/static/docs/analytics-build-comparison.md b/static/docs/analytics-build-comparison.md deleted file mode 100644 index 4d8044837..000000000 --- a/static/docs/analytics-build-comparison.md +++ /dev/null @@ -1,147 +0,0 @@ -# Build Comparison - Compare Test Builds and Track Regressions - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Overview - -Build Comparison allows you to compare two builds side by side to instantly see what changed - which tests started failing, which got fixed, and which remain stable. Use it to validate releases, debug regressions, and track test stability. - -## Accessing Build Comparison - -1. Navigate to **Insights** → **Build Insights** -2. Click on any build to open the **Build Details** page -3. Select the **Compare** tab - -## Selecting Builds to Compare - -When you first open the Compare tab, you'll see an empty state prompting you to select a build for comparison. - -Click **Select build to compare** to open the build selection dialog. - -### Build Selection Dialog - -The dialog provides options to find builds: - -| Option | Description | -|--------|-------------| -| **Past runs of same build** | Shows previous executions of the current build (default) | -| **All Builds** | Shows all builds across your account for cross-build comparison | -| **Search** | Search bar to find builds by name | - -Each build in the list displays: -- **Build Name** - Full build identifier -- **Duration** - Total execution time (e.g., 52m 53s) -- **Test Count** - Number of tests executed -- **Execution Timestamp** - Execution date and time -- **User** - Associated username who executed the build (e.g., atxSmoke) -- **Results Summary** - Quick pass/fail/other counts (🟢 passed, 🔴 failed, ⚫ other) - -Select a build and click **Compare Builds** to run the comparison. The selected build becomes the **Compare** build, while the current build you navigated from becomes the **Base** build. - -For release validation, select your last stable production build as **Base** and the release candidate as **Compare**. - -## Key Comparison Metrics - -**Understanding Failed Statuses** -The following statuses are considered **failed statuses**: **Failed**, **Error**, **Lambda Error**, **Idle Timeout**, and **Queue Timeout**. Change detection is based on whether a test transitions to or from these statuses. - -| Metric | Description | When to Act | -|--------|-------------|-------------| -| **New Failures** | Tests not failing in Base but failing in Compare (see details below) | 🚨 Investigate immediately before release - these are regressions | -| **Pass Rate** | Percentage of passed tests with delta (↗ or ↘) from Base. | Set release gates (e.g., "Release only if >95%") | -| **Fixed** | Tests that failed in Base but passed in Compare. | Verify fixes are genuine, not flaky behavior | -| **No Change** | Tests with same non-passing status in both builds. | Review for persistent infrastructure issues | -| **Additional Tests** | New tests in Compare not present in Base. | Confirm new features have test coverage | -| **Dropped Tests** | Tests in Base but missing from Compare. | ⚠️ Investigate if not intentionally removed | - -### Understanding New Failures - -The **New Failures** metric includes two scenarios: - -| Scenario | Description | Label in Table | -|----------|-------------|----------------| -| **Regression** | Test existed in Base with a non-failed status but has a failed status in Compare | New Failure | -| **New test failing** | Test did not exist in Base but has a failed status in Compare | New Failure (Additional) | - -Both scenarios are counted together in the **New Failures** metric shown in the summary cards and charts. In the Test Instances table, tests that didn't exist in Base are labeled as **New Failure (Additional)** to help you distinguish between regressions in existing tests versus failures in newly added tests. - -## Results Comparison Chart - -The horizontal bar chart compares test counts by status between builds: -- **Purple bar**: Base build -- **Orange bar**: Compare build - -If the orange bar is longer for Failed/Error statuses, more tests are failing in the newer build. - -## Status Changes Chart - -The donut chart categorizes tests by how their status changed: - -| Category | Description | Action | -|----------|-------------|--------| -| **New Failures** | Non-failed → Failed (includes New Failure Additional) | Prioritize - check recent code changes | -| **Fixed Instances** | Failed → Passed | Verify fix is stable, not flaky | -| **Stable Instances** | Passed → Passed | No action - reliable tests ✓ | -| **Consistent Failures** | Failed in both builds | Triage - document or fix before release | - -## Test Instances Comparison Table - -| Column | Description | Use Case | -|--------|-------------|----------| -| **Test Instances** | Test name, spec file, platform, browser | Click to view detailed logs and recordings | -| **Base** | Status and duration in Base build | Reference point for comparison | -| **Compare** | Status and duration in Compare build | Identify status changes at a glance | -| **Duration Change** | Time difference (+slower, -faster) | Flag tests with >30% increase for performance review | -| **Change Type** | Stable, Status Change, Fixed, New Failure (Additional), etc. | Filter to focus on specific change categories | - -### Filtering Options - -| Filter | Description | -|--------|-------------| -| **All** | Filter by change type | -| **Search** | Find tests by name or spec file | -| **OS** | Filter by operating system | -| **Browser** | Filter by browser type | -| **Test Tags** | Filter by custom tags | - -Use filters to isolate platform-specific issues. If failures only occur on a specific browser or OS, it helps prioritize the fix. - -## Common Use Cases - -### Pre-Release Validation -Compare your last stable build (Base) with the release candidate (Compare). Proceed only if **New Failures = 0** and pass rate meets standards. - -### Debugging a Broken Build -Compare the last passing build (Base) with the failing build (Compare). Review **New Failures** and use filters to isolate platform-specific issues. - -### Measuring Stabilization Progress -Compare the sprint-start build (Base) with the latest build (Compare). Use **Fixed** count and reduced **Consistent Failures** to demonstrate progress. - -### Environment Comparison -Compare production build (Base) with staging build (Compare) to identify environment-specific failures. - -### Cross-Browser Compatibility -Compare Chrome build (Base) with Firefox/Safari builds (Compare) to catch browser-specific issues. - -## Best Practices - -1. **Compare similar test suites** - Comparing different test sets leads to misleading Additional/Dropped counts. -2. **Investigate New Failures immediately** - These are potential regressions. -3. **Verify Fixed tests** - Run them multiple times to confirm stability. -4. **Monitor Duration Changes** - Increases >20-30% may indicate performance issues. -5. **Document Consistent Failures** - Maintain a list of known, accepted failures. -6. **Establish comparison baselines** - Define standard comparison points (last production release, previous nightly, sprint-start). - -## FAQ - -**Can I compare builds from different projects?** -Yes, but for meaningful results, compare builds with similar test suites. - -**Why are tests showing as "Dropped"?** -Tests may be skipped in configuration, failed to execute, or removed from the suite. - -**How is Pass Rate calculated?** -`(Passed Tests / Total Tests) × 100`. The delta shows the change from Base. - -**How far back can I compare?** -Any two builds within your data retention period. diff --git a/static/docs/analytics-build-insights.md b/static/docs/analytics-build-insights.md deleted file mode 100644 index c921423bf..000000000 --- a/static/docs/analytics-build-insights.md +++ /dev/null @@ -1,316 +0,0 @@ -# Build Insights - Analyze your test builds and get build level insights - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Overview - -Build Insights is your build-level health dashboard. It shows how stable each build is, how long it took, and which tests are causing problems so you can decide quickly whether a build is safe to promote or needs more work. - -With Build Insights, you can view all your unique builds in a centralized list, then drill down into individual build details to explore comprehensive metrics and test-level insights. The feature is designed to be intuitive and accessible, whether you're a QA engineer analyzing test results or a team lead tracking overall build health. - -Build Insights also supports **Unique Instances** view, which consolidates retry runs by grouping tests by name and environment (browser + OS + device + resolution), showing only the final run result for cleaner, more accurate reporting. For a comprehensive guide on how unique instances and retry attempts are detected, see [Unique Test Instances and Intelligent Retry Detection](/support/docs/analytics-unique-instances-retry-detection/). - -## Build Insights Flow - -Build Insights organizes your test data into two main views: - -1. **Build Insights Page** – scan all builds and spot risky ones using high-level metrics. -2. **Build Details Page** – open a specific build to understand *why* it looks good or bad, using detailed charts and test-level data. - -## Page 1: Build Insights - List of All Unique Builds - -Use this page to monitor all builds at a glance and decide which ones need attention. - -### Search Functionality - -Use the search bar to quickly find a specific build (for example, by suite name like **Smoke**, **Regression**, or **Nightly**) instead of scrolling through the full list. - -### Build Information Table - -The main table displays your builds with the following columns: - -#### Build Name Column - -Each build entry shows: -- **Build Name**: The full name of the build (e.g., `PROD_Analytics_Playwright_Smoke_2025-12-02`) -- **Duration**: How long the build took to execute, displayed in a readable format (e.g., "27m 59s", "2h 15m") -- **Execution Timestamp**: The date and time when the build was executed (e.g., "02/12/2025, 12:58:46") -- **Project/Tag**: The associated project or tag name (e.g., "atxSmoke") -- **Build Tags**: Visual tags associated with the build (e.g., "atxSmoke_build", "playwright_build") - -#### Last Build Summary Column - -Use this column to quickly judge the latest run of a build: -- **Total**: Total number of tests executed -- **Passed**: Number of tests that passed (displayed in green) -- **Failed**: Number of tests that failed (displayed in red) -- **Others**: Number of tests in other statuses like blocked, skipped, etc. (displayed in grey) - -#### Result History Column - -Use this to understand how reliable a build has been over time (not just in the last run): -- **Donut Chart**: A circular chart showing the overall pass/fail ratio for the build -- **Bar Chart**: A series of 10 vertical bars representing the last 10 build executions, with: - - Green segments indicating successful runs - - Red segments indicating failed runs - - The height of colored segments showing the proportion of pass/fail results - -#### Duration History Column - -A line graph showing how build duration has changed over time for the last 10 builds. Use it to: -- Spot builds that are gradually getting slower. -- Detect sudden spikes that may indicate performance regressions or environment issues. -- Compare duration trends between builds when optimizing your pipeline. - -### Navigation - -Use the pagination controls at the bottom of the table to navigate through multiple pages of builds. Click "Previous" or "Next >" to browse through your build history. - -## Page 2: Build Details - Individual Build Analysis - -Open this page when you want to understand *why* a build looks healthy or unhealthy. The page is split into two tabs: **Insights** (build-level metrics) and **Tests** (test-level details). - -### Navigation and Breadcrumbs - -At the top of the page, you'll see: -- A back arrow to return to the Build Insights list -- The build name as a breadcrumb path (e.g., `PROD_Analytics_Playwright_Smoke_2025-12-02`) - -### Filters and Sharing Options - -Use filters to narrow analysis to exactly the slice you care about (for example, only Chrome failures on macOS). You can also share the build details page with your team. - -**Available Filters:** -- **Browser**: Filter by browser type (Chrome, Firefox, Safari, Edge, etc.) -- **Status**: Filter by test status (Passed, Failed, Error, etc.) -- **OS**: Filter by operating system (Windows, macOS, Linux, etc.) -- **Project**: Filter by project name -- **Build Tags**: Filter by build-specific tags -- **Test Tags**: Filter by test-specific tags -- **Choose Custom Tags**: Select from your custom tag definitions - -**Sharing Options:** -- Click the share icon next to the filters to share the current view with your team -- Generate shareable links to the build details page (note: filter settings are not preserved in shared links) - -### Unique Test Instances - -The **Show Unique Instances** toggle consolidates retry runs to give you a cleaner, more accurate view of your test results. This feature applies to both the **Insights** and **Tests** tabs. - -**How it works:** - -- When **ON**: Within the current build, tests are grouped by **test name + environment** (browser + OS + device + resolution) as a single instance. Only the **final run** of each instance is considered in reporting, eliminating noise from intermediate retry attempts. This affects: - - **Insights tab**: Key metrics, charts, and Smart Tags reflect deduplicated counts based on final run results - - **Tests tab**: The test list shows only the final execution per unique test-environment combination - -- When **OFF**: All individual test executions are shown, including every retry attempt. - -The grouping is scoped to the individual build ID, meaning each build's metrics reflect only the deduplicated results within that specific build run. - -**Your Settings Are Remembered:** -Your Show Unique Instances preference is automatically saved, so you don't need to re-enable it each time you visit. Build Insights and Test Insights remember their settings independently, allowing you to configure each page to suit your workflow. - -**Processing Time** -Retry run consolidation requires a small amount of processing time after test execution completes. If you've just finished a build, wait a moment before toggling on Unique Instances to ensure all data is consolidated. - -**Learn More** -Want to understand exactly how unique instances and retry attempts are detected? See our comprehensive guide on [Unique Test Instances and Intelligent Retry Detection](/support/docs/analytics-unique-instances-retry-detection/) for step-by-step examples and best practices. - -## Tab 1: Insights - -Use the **Insights** tab to understand the overall health and performance of the selected build before you dive into individual tests. - -### Key Metrics Summary - -At the top of the Insights tab, you'll see a summary row displaying: -- **Total Unique Tests**: The number of unique test cases in the build -- **Total Tests**: The total number of test executions (including reruns) -- **Passed**: Count of passed tests (with green indicator) -- **Failed**: Count of failed tests (with red indicator) -- **Error**: Count of tests that errored (with dark red indicator) -- **Others**: Count of tests in other statuses (with grey indicator) - -### Monthly Progress Bar - -Use this bar to track how the build has behaved over the last 2 months: -- Green segments represent periods where most tests passed. -- Red segments highlight time ranges with frequent failures. -- Together they help you see whether the build is stabilizing or becoming riskier over time. - -### Build History Chart - -This stacked bar chart shows how many tests passed, failed, or errored in each execution of the build: -- **Y-axis**: Number of tests -- **X-axis**: Timestamps of build executions -- **Color-coded segments**: - - Green: Passed tests - - Red: Failed tests - - Dark Red: Error status - - Yellow: Idle timeout or other statuses -- **Legend**: Color-coded legend below the chart explains each status type - -Use it to: -- Identify trends in success/failure rates across runs. -- Quickly see when a spike in failures started. -- Compare executions before and after a code or configuration change. - -### Build Summary Chart - -This pie chart summarizes the current execution: -- **Largest segment**: Passed tests (typically shown in green with percentage) -- **Other segments**: Failed, Error, and other statuses with their respective percentages -- **Legend**: Color-coded legend showing what each segment represents - -Use it for a quick “go / no-go” signal on the current run. - -### Smart Tags Summary - -A grid displaying intelligent test categorization: -- **Total Tests Run**: Overall count of test executions -- **Performance Anomaly**: Tests flagged for unusual performance patterns -- **New Failure**: Recently introduced test failures -- **Flaky Test**: Tests with inconsistent pass/fail patterns -- **Always Failing**: Tests that consistently fail - -Each metric points you directly to tests that need attention (for example, focus first on **New Failure** and **Always Failing** before refactoring **Flaky Test**). - -## Tab 2: Tests - -Use the **Tests** tab when you are ready to debug at the individual test level. - -**Build Comparison** -Want to compare two builds side by side? Use the **Compare** tab to identify new failures, fixed tests, and stability changes between any two builds. This is especially useful for release validation and regression detection. Learn more in the [Build Comparison](/support/docs/analytics-build-comparison/) documentation. - -### Search Functionality - -Use the search bar to jump straight to a specific test by name (for example, when a developer shares a failing spec file name). - -### Group By - -The **Group By** dropdown allows you to organize tests by specific environment attributes, making it easier to identify patterns across different configurations. - -**Location**: The Group By dropdown is positioned adjacent to the search bar. - -**Available Grouping Options:** -- **Browser**: Group tests by browser version (e.g., Chrome 120, Firefox 115) -- **OS**: Group tests by operating system version (e.g., Windows 10, macOS Ventura) -- **Resolution**: Group tests by screen resolution (e.g., 1920x1080, 1366x768) -- **Device**: Group tests by device type (e.g., Desktop, Mobile) - -**Default Behavior:** -- By default, all grouping options are selected, which displays a flat list view showing all tests without grouping -- Deselecting one or more options groups the tests by the remaining selected attributes - -**Your Settings Are Remembered:** -Your Group By selections are automatically saved, so when you return to Build Insights, your preferred grouping configuration is already applied. - -**Use Cases:** -- **Browser compatibility analysis**: Deselect all except Browser to quickly see which browsers have the most failures -- **OS-specific debugging**: Group by OS to identify platform-specific issues -- **Resolution testing**: Group by Resolution to spot responsive design problems -- **Multi-dimensional grouping**: Combine multiple grouping options to create hierarchical views (e.g., group by Browser and then by OS within each browser group) - -### Test Results Table - -The main table displays individual test executions with three key columns: - -#### Test Name Column - -For each test, you'll see: - -**Primary Line:** -- **Test Name**: The full name of the test (e.g., `PROD_Verify FTD feature for the build atxRD_flakyBuild - flaky_test_detection.spec.ts`) - -**Secondary Line (Environment Info):** -Each test row displays detailed execution context with visual icons in the following order: -- **Duration**: How long the test took to execute (e.g., "82s", "2m 15s") -- **Timestamp**: When the test was executed in a readable format (e.g., "Jan 20, 2026 5:12 PM") -- **Username**: The user who executed the test -- **Browser**: Browser name and version with browser icon (e.g., Chrome icon with "Chrome 120") -- **OS**: Operating system with OS icon (e.g., Windows icon with "Windows 10") -- **Device**: Device type indicator (e.g., Desktop, Mobile) -- **Resolution**: Screen resolution used (e.g., "1920x1080") -- **Smart Tags**: Intelligent test categorization tags (e.g., "Flaky Test", "New Failure", "Performance Anomaly") - -**Additional Information:** -- **Project/Tag**: Associated project or tag (e.g., "atxSmoke") -- **Test Tags**: Clickable tags associated with the test (e.g., "playwright_test", "atxSmoke_test") - -The environment icons match the visual style used throughout the Test Intelligence drilldown views, providing consistent visual identification of execution contexts. - -#### History Column - -A visual representation of the test's recent execution history: -- **10 colored circles**: Each circle represents one of the last 10 test executions - - **Green circles**: Successful test runs - - **Red circles**: Failed test runs -- This visual history helps you quickly identify: - - Test stability patterns - - Flaky tests (alternating green/red patterns) - - Consistently failing tests (mostly red) - - Stable tests (mostly green) - -#### Failure/Blocked Reason Column - -For each test, this column displays: -- **Error Message**: If the test failed or was blocked, the reason is shown in a colored box -- **No Error**: If the test passed, this is indicated in a light yellow box -- This information helps you quickly understand why tests failed without opening individual test details - -### Filtering and Analysis - -Use the filters at the top to: -- Focus on specific browsers or operating systems (for example, only Safari failures on macOS). -- Filter by test status (Passed, Failed, Error) so you can work through failed tests first. -- Narrow down by project or tags to isolate a particular suite or component. -- Apply custom tag filters to align analysis with your internal categorization. - -### Pagination - -Navigate through multiple pages of test results using the "Previous" and "Next >" controls at the bottom of the table. - -## How Teams Typically Use Build Insights - -- **Release readiness checks**: Use the Build Insights page, Key Metrics Summary, and Build Summary chart to decide if a build is safe to ship. -- **Regression and incident analysis**: Use the Build History chart, Duration History, and test History column to find when a regression started and which tests were affected. -- **Stability improvement work**: Use Smart Tags and the Tests tab filters to prioritize fixing always-failing and flaky tests. - -## Build Naming Best Practices - -### Maintain Common Build Names - -To get the most value from Build Insights, we recommend maintaining common build names instead of adding unique identifiers (UIDs) daily or weekly. Here's why: - -**Benefits of Common Build Names:** - -- **Historical Tracking**: When you use consistent build names (e.g., `PROD_Analytics_Playwright_Smoke`), Build Insights can aggregate all executions of that build over time, giving you: - - Accurate Result History charts showing trends across multiple runs - - Meaningful Duration History graphs that track performance over time - - Better visibility into build health patterns - -- **Easier Analysis**: Common build names make it easier to: - - Compare performance across different time periods - - Identify trends and patterns in test stability - - Track improvements or regressions in your test suite - -- **Better Organization**: Instead of creating new build names with dates or UIDs (e.g., `Build_2025-12-02`, `Build_UID_12345`), reuse the same build name for similar test suites. The system automatically tracks each execution with its timestamp, so you don't need unique names to distinguish runs. - -**Recommended Approach:** - -- Use descriptive, consistent names like: `PROD_Smoke_Tests`, `Regression_Chrome`, `Nightly_Build` -- Avoid adding dates or UIDs to build names unless necessary for specific use cases -- Let the execution timestamps handle the temporal distinction between runs -- Use tags and filters to further categorize and organize your builds - -This approach ensures that Build Insights can provide you with meaningful historical analysis and trend identification for your test suites. - -## Best Practices - -1. **Check builds early and often**: Start your day on the Build Insights page to spot risky builds before they block releases. -2. **Filter with intent**: Use filters to answer specific questions (for example, "Are failures only on Windows?") instead of browsing everything at once. -3. **Trust history, not one run**: Use Result History, Duration History, and the test History column to judge stability over time, not just a single execution. -4. **Share context, not just failures**: When sharing a build, also mention which metrics you looked at (for example, "pass rate dropped from 98% to 90% in the last 3 runs"). -5. **Standardize build names**: Maintain common build names so histories stay meaningful and easy to compare across days and weeks. -6. **Use Unique Instances for accurate reporting**: Toggle on "Show Unique Instances" to consolidate retry runs and see the true pass/fail state of each test-environment combination, especially when your pipeline uses automatic retries. -7. **Leverage Group By for pattern analysis**: Use the Group By dropdown in the Tests tab to organize tests by Browser, OS, Resolution, or Device. This helps quickly identify environment-specific failures and compatibility issues. -8. **Combine grouping with filters**: Apply filters first to narrow down to relevant tests (for example, failed tests only), then use Group By to see how those failures distribute across environments. diff --git a/static/docs/analytics-create-dashboard.md b/static/docs/analytics-create-dashboard.md deleted file mode 100644 index ff87ca099..000000000 --- a/static/docs/analytics-create-dashboard.md +++ /dev/null @@ -1,43 +0,0 @@ -# Creating your first Dashboard using TestMu AI Analytics - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Using Custom Dashboards - -Add widgets of multiple products and various insights to your dashboard. You can create a custom dashboard with the widgets of your choice. You can add widgets of the same type multiple times and also edit the layout of the dashboard according to your configuration. - -### Prerequisites - -The following are requirements: - -- Should be logged in to the TestMu AI application using your credentials. If account is not created, please create an new account here. -- Have executed any tests on the TestMu AI Cloud Platform. _(Optional)_ - -### Step 1: Navigate to TestMu AI Insights - -Once, logged in to the application, please navigate to https://analytics.lambdatest.com/ or you can select **Insights** from the sidebar navigation as well. - -### Step 2: Creating the dashboard - -Now, you can select the available options to **create new**. Once, selected you will be able to select the required **widgets** according to the choice of your product for which you want to create the dashboard for. - -**Other Dashboard Options** - -You can also create a **Demo Dashboard** on choosing this option you can explore the platform with dummy data for exploring the dashboard and widgets supported by the platform. - -Or you try using the **Pre-built templates** for quick dashboard creating for your required reports. To know more click [here](/docs/analytics-dashboard-templates/). - -### Step 3: Adding the required widgets - -Now, you select any widget you want to be added to the dashboard from the available list. You can filter by product and by category. - -Add **Dashboard Name** of your choice, according to the **report** which needs to be created. Then, click on the `Create New Dashboard` button and you're done! - -Your dashboard is ready to start **Analyzing** the data of all your tests which are executed using the TestMu AI Cloud platform. - -You can add more widgets of the same type to your dashboard multiple times and also edit the layout of the dashboard according to your configuration. - -## Explore More Dashboard Features - -* Share the Dashboard with a public or a private link with your team - [Read More](/docs/analytics-dashboard-features/#expiry-settings-for-dashboard-share-link) -* Exporting the Dashboard as a PDF - [Read More](/docs/analytics-dashboard-features/#export-dashboard-as-pdf) diff --git a/static/docs/analytics-dashboard-copilot.md b/static/docs/analytics-dashboard-copilot.md deleted file mode 100644 index 8d0aa3cd6..000000000 --- a/static/docs/analytics-dashboard-copilot.md +++ /dev/null @@ -1,67 +0,0 @@ -# Getting Started with AI CoPilot Dashboard - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The AI CoPilot Dashboard is an innovative feature designed to revolutionize the way you analyze and gain insights from your data. By leveraging the power of artificial intelligence, the dashboard provides intelligent recommendations, insights, and predictions tailored to your specific data, enabling you to make data-driven decisions faster than ever before. - -## Prerequisites - -To access and utilize the full potential of the AI CoPilot Dashboard, ensure that you meet the following requirements: - -1. **TestMu AI Paid Account**: You must have an active TestMu AI paid account to unlock the AI CoPilot Dashboard feature, please contact our support team at [support@testmuai.com](mailto:support@testmuai.com). - -This feature is exclusively available to our valued paid users. - -2. **Data and Dashboard Setup**: To maximize the benefits of the AI CoPilot Dashboard, ensure that you have relevant data and a well-structured dashboard created with the necessary widgets in your account. The AI CoPilot relies on the existing data and dashboard configuration to generate accurate and meaningful insights. - -To know about the widgets and how to create a dashboard, refer to the [Analytics Widgets](/docs/analytics-widgets/) documentation. - -## Accessing the AI CoPilot Dashboard - -Follow these step-by-step instructions to access the AI CoPilot Dashboard: - -1. **Login**: Sign in to your TestMu AI account using your registered credentials. - -2. **Navigate to Insights**: From the left navigation menu, locate and click on the "Insights" section to access the analytics and dashboard features. - -3. **Select or Create Dashboard**: Choose the desired dashboard you want to analyze using the AI CoPilot. If you haven't created a dashboard yet, take a moment to set one up with the required widgets and data. - -4. **Launch AI CoPilot**: Once you have selected the dashboard, click on the "AI CoPilot" button located at the top right corner of the dashboard options. This action will initiate the AI CoPilot feature for the selected dashboard. - -5. **Start Analyzing**: Begin exploring your data and gaining valuable insights by asking questions or entering queries in natural language. The AI CoPilot is designed to understand and interpret your input, providing relevant recommendations and insights based on your data. - -## Key Features of AI CoPilot - -The AI CoPilot Dashboard offers a wide range of powerful features to enhance your data analysis experience: - -1. **Natural Language Query**: Communicate with the AI CoPilot using natural language. Ask questions, seek clarifications, and express your data analysis requirements in a conversational manner, just as you would with a human expert. - -2. **Intelligent Recommendations**: Benefit from data-driven recommendations generated by the AI CoPilot. These recommendations are based on patterns, trends, and insights identified within your data, helping you make informed decisions and uncover hidden opportunities. - -3. **Actionable Insights**: Gain valuable insights derived from your data through the AI CoPilot's advanced analytics capabilities. These insights provide a deeper understanding of your data, enabling you to identify key drivers, anomalies, and areas for improvement. - -4. **Predictive Analytics**: Harness the predictive power of the AI CoPilot to forecast future trends and outcomes based on historical data patterns. By leveraging machine learning algorithms, the AI CoPilot helps you anticipate potential challenges and make proactive decisions. - -5. **Trend Analysis**: Identify and visualize trends within your data using the AI CoPilot's trend analysis feature. Uncover patterns, seasonality, and significant changes over time, empowering you to adapt your strategies accordingly. - -6. **Comparative Analysis**: Conduct comprehensive comparisons between different metrics, segments, or time periods using the AI CoPilot's comparison capabilities. Gain insights into performance variations, identify outliers, and benchmark your data against relevant industry standards. - -7. **Customization and Flexibility**: Tailor the AI CoPilot's analysis to your specific requirements. Customize data ranges, filter criteria, and visualization preferences to align with your unique business needs and objectives. - -8. **Seamless Collaboration**: Share insights, reports, and dashboards generated by the AI CoPilot with your team members effortlessly. Foster a culture of data-driven decision-making by enabling collaboration and knowledge sharing within your organization. - -## AI CoPilot Dashboard Widgets - -All the widgets available within the dashboard's widget library are AI CoPilot enabled, allowing you to interact with them seamlessly. Simply ask questions or enter queries related to a specific widget, and the AI CoPilot will provide relevant insights and recommendations based on the data represented in that widget. - -## Rate Limit and Usage Guidelines - -To ensure optimal performance and availability of the AI CoPilot Dashboard, we have implemented rate limits and usage guidelines. These measures are designed to prevent abuse, maintain system stability, and provide a consistent user experience for all customers. - -We currently allow a maximum of **5 prompts per day** for each organization account. This limit is subject to change based on usage patterns and system load. If you exceed the daily query limit, you may experience temporary restrictions on AI CoPilot functionality until the limit resets. - -## Support and Assistance - -We understand that adopting new technologies can sometimes be challenging. If you encounter any difficulties or have questions regarding the AI CoPilot Dashboard, our dedicated support team is here to assist you. Feel free to reach out to our [24/7 customer support](mailto:support@testmuai.com) for expert guidance and troubleshooting. We are committed to ensuring your success and providing you with the best possible experience while using the AI CoPilot Dashboard. - -Embrace the future of data analysis with the AI CoPilot Dashboard and unlock the full potential of your data. Start exploring, asking questions, and discovering valuable insights today! diff --git a/static/docs/analytics-dashboard-edit-layout.md b/static/docs/analytics-dashboard-edit-layout.md deleted file mode 100644 index 1dcc0f022..000000000 --- a/static/docs/analytics-dashboard-edit-layout.md +++ /dev/null @@ -1,57 +0,0 @@ -# TestMu AI Analytics - Customizing Your Dashboard with the New Widget Management Feature - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Introduction - -TestMu AI insights is a powerful platform designed to provide custom insights for test execution on cloud-based web and mobile platforms. As a software QA tester or manager, you can now further personalize your insights experience with **Edit Layout** feature. This feature allows you to change the layout, resize, and reposition the widgets on your dashboard, providing an optimized view for your specific needs. - -This technical support document will walk you through the steps to utilize this feature effectively. - -## Prerequisites - -To use the new widget management feature in Lambdatest insights, ensure that you have the following: - -- An active Lambdatest account -- Existing dashboards with widgets - -## Step by Step Process to Use the Feature - -Follow these steps to utilize the new widget management feature in Lambdatest insights: - -1. Log in to your Lambdatest account. -2. Go to Insights section in the sidebar menu. -3. Navigate to the dashboard you want to customize. -4. Click the 'Edit' button located at the top-right corner of the dashboard to enter the edit mode. -5. Perform the following actions to customize your widgets: - - Change the layout: Click and hold the widget's header, then drag and drop it to a new location within the dashboard. - - Resize the widgets: Hover over the bottom-right corner of the widget until the resize handle appears, click and drag to adjust the size. - - Change the position: Click and hold the widget's header, then drag and drop it to a new position in the grid layout. -6. After making the desired changes, click the 'Save' button at the top-right corner of the dashboard to save your customizations. - -## Explaining the Features with Sample Use Cases - -### Use Case 1: Organizing widgets based on priority - -As a QA Manager, you may want to organize your dashboard to display high-priority widgets at the top. Using the widget management feature, you can easily drag and drop widgets to rearrange them based on priority, making it easier to access and analyze important information quickly. - -### Use Case 2: Resizing widgets to optimize screen space - -As a QA tester, you may have a variety of widgets displaying different types of data. Some widgets may require more screen space for better visibility. The widget management feature allows you to resize widgets, enabling you to allocate more screen space for critical data and less for less important information. - -### Use Case 3: Grouping similar widgets together - -By grouping widgets with similar data or purpose, you can create a more organized and visually appealing dashboard. The widget management feature makes it easy to rearrange widgets, allowing you to group related widgets together for a more streamlined analysis. - -### Use Case 4: Creating multiple dashboards with different layouts - -You may need to present your data to different stakeholders who have different preferences for data visualization. With the widget management feature, you can create multiple dashboards with unique layouts, catering to the individual preferences of each stakeholder. - -### Use Case 5: Optimizing dashboard for mobile devices - -If you frequently access your dashboard on a mobile device, the widget management feature allows you to optimize the layout and size of widgets, ensuring that the dashboard is easily accessible and readable on a smaller screen. - -## 5. FAQs - -Q: Can I revert my dashboard to its previous state after making changes? -A: Yes, you can revert to the previous state by clicking the 'Cancel' button during the edit mode. However, once you have saved the changes, there is no direct way to revert to the previous state. diff --git a/static/docs/analytics-dashboard-features.md b/static/docs/analytics-dashboard-features.md deleted file mode 100644 index bf268a81f..000000000 --- a/static/docs/analytics-dashboard-features.md +++ /dev/null @@ -1,70 +0,0 @@ -# Analytics Dashboard Features to manage your data - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Export Dashboard as PDF - -#### Summary of the feature -This feature allows users to export their dashboard as a PDF file. This is useful for sharing the analytics data with team members, clients, or stakeholders who may not have access to the analytics platform. - -#### Steps to use -1. Navigate to the dashboard you want to export. -2. Click on the "Export" icon located in the upper-right corner of the dashboard. - -#### Usecases with examples -1. **Team meetings:** Share dashboard insights during team meetings by exporting the dashboard as a PDF and including it in the meeting agenda. -2. **Client presentations:** Export the dashboard as a PDF to include in a presentation or report for clients, showcasing the success of a marketing campaign or product launch. -3. **Internal reporting:** Use the PDF export feature for internal reporting purposes, such as sharing key performance indicators (KPIs) with management. -4. **Documentation:** Archive dashboard snapshots in a project's documentation for historical reference. -5. **Offline analysis:** Export dashboards as PDFs for team members who may not have access to the analytics platform or for instances where internet access is limited. - -## Expiry Settings for Dashboard Share Link - -#### Summary of the feature -This feature allows users to set an expiration date for the share link generated for the dashboard. This ensures that access to the dashboard is restricted after a certain time period, enhancing security and control over the shared data. - -#### Steps to use -1. Navigate to the dashboard you want to share. -2. Click on the "Settings" gear icon located in the upper-right corner of the dashboard. -3. Select the desired expiration date from the available dates in the settings. -4. Click "Generate Link" to create the share link with the specified expiry settings. - -#### Usecases with examples -1. **Limited-time client access:** Provide clients with temporary access to a dashboard during a campaign or project, ensuring that access is revoked once the project is completed. -2. **Time-sensitive data sharing:** Share dashboards containing time-sensitive information with team members or stakeholders, ensuring that the data is only accessible for a limited period. -3. **Promotional offers:** Share a dashboard with promotional data for a limited time, encouraging users to take advantage of the offer before the link expires. -4. **Security compliance:** Set expiration dates for shared dashboard links to meet internal or regulatory security requirements. -5. **Iterative reporting:** Share links to dashboards that need regular updates, ensuring that users always access the most recent version by setting previous versions to expire. - -#### Ability to set the Dashboard Password -For the users who are external to the organization, we have added the ability to set the password for the dashboard. This will ensure that only the intended users can access the dashboard. - -#### Steps to set the password for the dashboard -1. Navigate to the dashboard you want to share. -2. Click on the "Settings" gear icon located in the upper-right corner of the dashboard. -3. Select the checkbox "Setup the custom password" and enter the password. -4. Save the settings. -5. Now, when you share the dashboard with the new share link, the user will be prompted to enter the password to access the dashboard. - -The above password is only for those users who are external to the organization accessing through the share link of the dashboard. For the internal users, the dashboard will be accessible without any password. - -## Dashboard Visibility Settings - -#### Summary of the feature -This feature allows users to change the visibility of a dashboard from being accessible to all organization users to being private and only accessible to admin user and creator of the dashboard. This provides better control over who can view and interact with the dashboard. - -#### Steps to use -1. Navigate to the dashboard you want to change the visibility of. -2. Click on the "Settings" button located in the upper-right corner of the dashboard. -3. Toggle the visibility of the dashboard between "Public (All User)" and "Private (Creator Only)." - -An admin user is the user who has created the dashboard. Only the creator is allowed to access the dashboard settings and change the visibility of the dashboard. - -#### Usecases with examples -1. **Sensitive data protection:** Restrict access to dashboards containing sensitive or confidential information by setting visibility to "Private (Creator Only)." -2. **Role-based access control:** Limit dashboard access - -## Admin Access for Dashboards - -### Summary of the feature -This feature allows the admin user to access all the dashboards created by the organization users. The admin user can view, edit, and manage the dashboards created by other users within the organization. diff --git a/static/docs/analytics-dashboard-settings.md b/static/docs/analytics-dashboard-settings.md deleted file mode 100644 index cb61bedea..000000000 --- a/static/docs/analytics-dashboard-settings.md +++ /dev/null @@ -1,87 +0,0 @@ -# Analytics Dashboard Settings - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Dashboard Settings Overview - -The Analytics Dashboard Settings page provides a comprehensive set of controls to help you manage notifications, sharing, and access for your test analytics dashboards. This guide details each setting and how to use it for optimal collaboration and visibility. - -### Email Notifications - -Keep your team informed about dashboard activity and test results by configuring email notifications. - -- **Enable/Disable Notifications:** Use the toggle to activate or deactivate email notifications for the dashboard. When disabled, no email updates will be sent. -- **Frequency:** Choose how often you want to receive email updates from the dropdown. Options include **Daily**, **Weekly**, or **Monthly**. -- **Time & Period:** Set a specific delivery time for your scheduled reports. Select the hour from the **Time** dropdown (e.g., `9:00`) and choose **AM** or **PM** from the **Period** dropdown. -- **Date Range:** Select **Default** to use the platform's default date range, or select **Custom** and specify a number of days. For example, setting a custom range to `7` includes data from the last 7 days in each email. -- **Emails:** Enter one or more email addresses to receive notifications. Click **+ Add** after each address to include multiple recipients. This is useful for keeping stakeholders, QA leads, or team members in the loop. -- **Update:** After configuring your preferences, click **Update** to save changes. Your notification settings will take effect immediately. - -> **Tip:** Use custom date ranges and recipient lists to tailor notifications for different teams or reporting needs. - -For a detailed guide on scheduling reports with custom delivery times, see [Report Scheduling](/docs/analytics-report-scheduling/). - -### Slack Notification Settings - -Integrate your dashboard with Slack to receive instant updates and foster real-time collaboration. - -- **Enable/Disable Notifications:** Toggle Slack notifications on or off as needed. When enabled, updates will be sent directly to your selected Slack channel. -- **Notification Frequency:** Set how often notifications are sent to Slack (e.g., Daily, Weekly). This helps avoid notification fatigue while ensuring timely updates. -- **Date Range:** Choose the default or set a custom range (number of days) to define the scope of data included in each notification. -- **View Selection:** Select the specific dashboard view or analytics channel for which notifications should be sent. This allows you to target relevant information to the right Slack channels or teams. -- **Update & Save Preferences:** Click **Update** to apply your Slack notification settings. - -> **Best Practice:** Use Slack notifications for critical dashboards or to alert teams about important trends, regressions, or test failures. - -### MS Teams Notification Settings - -Integrate your dashboard with Microsoft Teams to receive timely updates and enhance team collaboration. - -- **Enable/Disable Notifications:** Toggle MS Teams notifications on or off as needed. When enabled, updates will be sent directly to your selected Teams channel. -- **Notification Frequency:** Set how often notifications are sent to MS Teams (e.g., Daily, Weekly). This helps ensure your team stays informed without overwhelming them with messages. -- **Date Range:** Choose the default or set a custom range (number of days) to define the scope of data included in each notification. -- **View Selection:** Select the specific dashboard view or analytics channel for which notifications should be sent. This allows you to target relevant information to the right Teams channels or groups. -- **Update & Save Preferences:** Click **Update** to apply your MS Teams notification settings. - -> **Best Practice:** Use MS Teams notifications for dashboards that require broad visibility or to alert teams about important trends, regressions, or test failures. - -### Share Link Settings - -Easily share your dashboard with colleagues, clients, or external stakeholders while maintaining control over access and privacy. - -- **Link Expiration:** Set the share link to expire after 7, 15, or 30 days, or select **Never** for a permanent link. Expiring links are recommended for temporary collaborations or reviews. -- **Privacy Controls:** - - **Make Dashboard Private:** Restrict access so only you can view the dashboard. Use this for sensitive or in-progress dashboards. - - **Allow Anyone with the Link:** Enable open access for anyone who has the link, ideal for broad sharing within your organization. - - **Set a Custom Password:** Add an extra layer of security by requiring users to enter a password to access the dashboard. This is especially useful when sharing with external users or when the link has no expiry. - -Password setup is mandatory if you select "Never" for link expiry. Exporting the dashboard will be disabled in this case to protect sensitive data. - -- **Export Restrictions:** If you set the link to never expire and require a password, dashboard export functionality will be disabled to enhance security. - -> **Security Tip:** Always use password protection and link expiration when sharing dashboards outside your core team. - -### Edit Dashboard Name - -Personalize your dashboard for clarity and easy identification. - -1. Enter a new name in the **Dashboard Name** field at the top of the settings panel. Use descriptive names that reflect the dashboard's purpose (e.g., "SMOKE Dashboard for Prod Playwright"). -2. Click **Save** to apply the new name. The updated name will be visible to all users with access to the dashboard. - -> **Tip:** Use naming conventions to organize dashboards by project, environment, or team. - -### Delete the Dashboard - -Remove dashboards that are no longer needed to keep your workspace organized and secure. - -1. Scroll to the bottom of the settings panel. -2. Click the **Delete** button next to "Delete this Dashboard". -3. Confirm the deletion when prompted. - -This action is irreversible. Once deleted, the dashboard and all its data cannot be recovered. Ensure you have exported or backed up any important information before proceeding. - -> **Recommendation:** Only delete dashboards when you are certain they are no longer required by any team members. - -### Additional Support - -For more information and advanced configuration options, reach out to our support team at [support@testmuai.com](mailto:support@testmuai.com). diff --git a/static/docs/analytics-dashboard-templates.md b/static/docs/analytics-dashboard-templates.md deleted file mode 100644 index ae7d24ebd..000000000 --- a/static/docs/analytics-dashboard-templates.md +++ /dev/null @@ -1,180 +0,0 @@ -# Analytics Dashboard Pre-built templates - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## About the Dashboard templates: - -Dashboard templates provide a pre-defined set of widgets and visualizations tailored for specific testing requirements. - -These templates help testers save time by not having to create custom dashboards from scratch. They also offer a standardized way to analyze test data and share insights across the team. - -With various templates available for different test analytics purposes, testers can quickly gain insights into test performance, error reports, resource usage, and trends. - -## Prerequisites - -You should have an account subscribed to [TestMu AI Account](https://www.testmuai.com/login/) and have relevant test data available in the system. - -## How to use it? - -To use a dashboard template, follow these steps: - -- Go to [TestMu AI Analytics](https://analytics.lambdatest.com). -- Now, click on `Create Dashboard` Button on the home page. -- Select a `pre-build` dashboards template based on your test analytics requirements. -- Apply relevant `filters` to customize the data displayed on the dashboard. (Optional) -- Review the generated dashboard with pre-associated widgets and visualizations. - -## Dashboard Templates Available on the platform - -### Tests Summary Report - -The Tests Summary Report is designed to provide QA testers with a comprehensive overview of their test suite's performance and distribution. This template allows users to assess the overall health of their test suite quickly and efficiently. - -By using the Tests Summary Report, QA testers can gain valuable insights into their test suite's performance and identify areas of improvement. This information helps them optimize their testing process, ensuring better test coverage and more reliable results, ultimately leading to higher-quality software products. - -The following are the widgets added to this dashboard: - -- **Test Summary**: Displays the total number of tests, their statuses, and distribution. -- **Test Case Health Snapshot**: Shows the health of test cases based on their pass, fail, or error statuses. -- **Test Browser**: Visualizes the distribution of tests across different browsers. -- **Test OS**: Presents the distribution of tests across different operating systems. -- **Test Device Coverage (Only App Automation)**: Illustrates the device coverage for app automation tests. -- **Jobs Summary (Only HyperExecute)**: Summarizes HyperExecute jobs. -- **Stage Summary (Only HyperExecute)**: Provides a summary of HyperExecute stages. -- **Tasks Summary (Only HyperExecute)**: Summarizes HyperExecute tasks. - -The following are the use cases for the template: - -1. Quickly assess overall test performance and distribution to prioritize actions and allocate resources effectively. -2. Monitor key testing metrics, such as test case health, browser, OS, and device coverage, to ensure comprehensive testing. -3. Identify trends in test executions, enabling better planning and decision-making for future testing efforts. -4. Streamline test analytics and reporting, saving time and effort for QA teams. -5. Enhance collaboration between QA and development teams by sharing insights on test performance and distribution. - -### Error Insights Report - -The Error Insights Reports template is specifically designed to help QA testers identify, analyze, and debug tests with errors status. This template enables testers to gain a deeper understanding of their test suite's error landscape, ultimately improving test reliability and robustness. - -This template includes several widgets that focus on errors, such as error statistics, test status ratios, and error distributions across browsers and operating systems. The combination of these widgets allows testers to quickly spot patterns and trends related to errors, facilitating faster debugging and resolution. - -Utilizing the Error Insights Reports, QA testers can pinpoint problematic areas in their test suite, leading to quicker error resolution and improved test stability. This template's insights empower testers to enhance their testing process, ultimately resulting in more reliable and higher-quality software products. - -The following are the widgets added to this dashboard: - -- **Error Stats**: Shows statistics related to errors in test statues. -- **Test Status Ratio**: Displays the ratio of error statuses in tests. -- **Test Summary**: Provides a summary of tests with errors. -- **Test Case Health Snapshot**: Displays the health of test cases with errors. -- **Test Browser**: Visualizes the distribution of errors across different browsers. -- **Test OS**: Illustrates the distribution of errors across different operating systems. - -The following are the use cases for the template: - -1. Rapidly identify and resolve errors within test executions, improving overall test quality and efficiency. -2. Monitor error trends to proactively address recurring issues and minimize their impact on test outcomes. -3. Gain valuable insights into error patterns, leading to more effective debugging and resolution efforts. -4. Improve collaboration between QA and development teams by sharing error insights and promoting a shared understanding of issues. -5. Prioritize error resolution efforts based on the severity and frequency of identified issues. - -### Resource Consumption and Concurrency usage - -The Resource Consumption and Concurrency usage template is designed to help QA testers monitor resource utilization effectively, analyze concurrency trends, and plan resource allocation. This template enables better resource management, ensuring optimal usage of available testing resources procured or to be planned for your upcoming budgets. - -The template includes widgets that visualize concurrency trends, test summaries, test case health snapshots, and test distribution across browsers and operating systems. These widgets provide a comprehensive view of resource consumption and parallel thread usage, helping testers make informed decisions about resource availability trends. - -With the Resource Consumption and Concurrency usage template, QA testers can optimize their testing process by effectively managing resources and ensuring that critical releases have sufficient resources allocated. This leads to a more efficient testing process, cost savings, and improved product quality. - -The following are the widgets added to this dashboard: - -- **Concurrency Trends:** Visualizes trends in resource consumption and parallel thread usage. -- **Test Summary:** Displays the total number of tests and their distribution. -- **Test Case Health Snapshot:** Shows the health of test cases based on their pass, fail, or error statuses. -- **Test Browser:** Presents the distribution of tests across different browsers. -- **Test OS:** Illustrates the distribution of tests across different operating systems. - -The following are the use cases for the template: - -1. Optimize resource allocation by understanding the distribution of tests across different platforms and concurrency usage. -2. Identify potential bottlenecks in resource consumption, allowing for better planning and management of testing efforts. -3. Monitor trends in concurrency usage to ensure optimal utilization of subscribed resources. -4. Improve overall testing efficiency by aligning resource consumption with testing priorities. -5. Enable data-driven decision-making for resource planning and capacity management. - -### All Trends Report - -The All Trends Report template is designed to enable QA Managers and testers to monitor and analyze trends in test performance, error statistics, and resource usage regularly. This template provides valuable insights to drive continuous improvement in the testing process. - -This template features widgets that display test summaries, test case health snapshots, test distribution across browsers and operating systems, test trends, error statistics, and concurrency trends. These widgets work together to provide a holistic view of the testing process, allowing QA managers and testers to identify patterns and trends easily. - -By using the All Trends Report, QA testers can gain a better understanding of their testing process's overall performance and identify areas for improvement. These insights help drive continuous improvement, resulting in more efficient testing processes, quicker error resolutions and save time in preparing these reports. - -The following are the widgets added to this dashboard: - -- **Concurrency Trends:** Visualizes trends in resource consumption and parallel thread usage. -- **Test Summary:** Displays the total number of tests and their distribution. -- **Test Case Health Snapshot:** Shows the health of test cases based on their pass, fail, or error statuses. -- **Test Browser:** Presents the distribution of tests across different browsers. -- **Test OS:** Illustrates the distribution of tests across different operating systems. - -The following are the use cases for the template: - -1. Identify trends in test performance and execution to monitor progress and continuously improve testing efforts. -2. Gain insights into test case health, browser and OS distribution, and error patterns to optimize testing processes. -3. Monitor concurrency trends to ensure optimal utilization of resources and improve overall testing efficiency. -4. Enhance collaboration between QA and development teams by sharing insights on test trends and performance. -5. Support data-driven decision-making for resource planning, capacity management, and testing priorities. - -### OS and Browser Coverage Report - -The OS & Browser Report is designed to provide comprehensive insights into the testing coverage across various browsers, operating systems, and devices. This report is particularly valuable for QA teams looking to ensure their applications are compatible and function seamlessly across multiple platforms, thus providing a consistent user experience. - -With a collection of carefully selected widgets that display key metrics pertaining to test case health, browser and operating system distribution, and device coverage. By presenting these insights in a cohesive and easy-to-understand manner, the template allows QA teams to quickly identify areas of focus, prioritize testing efforts, and optimize resource allocation. - -The following are the widgets added to this dashboard: - -- **Test Case Health Snapshot**: Review the overall health of your test cases, enabling you to identify areas that require further attention or improvement. -- **Test Browser**: Analyze the distribution of test executions across different browsers, ensuring comprehensive coverage and addressing compatibility issues. -- **Test OS**: Gain insights into the testing distribution across various operating systems, helping you prioritize efforts and achieve better compatibility. -- **Test Device Coverage (Only App Automation)**: Monitor device-specific testing coverage for mobile applications, ensuring seamless functionality across a wide range of devices. -- **Concurrency Trends (split by OS & Browser)**: Examine trends in concurrent test executions, broken down by operating system and browser, to optimize resource allocation and testing efficiency. - -The following are the use cases for the template: - -1. Identify gaps in browser and OS testing coverage, enabling targeted testing efforts and improved application compatibility. -2. Monitor device coverage to ensure comprehensive testing across various devices, especially for mobile applications. -3. Gain insights into test case health across browsers and operating systems, helping prioritize resolution efforts. -4. Optimize resource allocation by understanding the distribution of tests across different platforms. -5. Enhance collaboration between QA and development teams by sharing browser and OS-specific insights. - -### HyperExecute Report - -The HyperExecute Report template is meticulously designed to cater to the sophisticated requirements of large corporations and enterprises, providing QA managers with an in-depth analysis of all jobs executed through the HyperExecute platform. This template empowers users to effortlessly assess job performance, tasks, and stages with a comprehensive and efficient view, ensuring streamlined testing processes and improved operational efficiency. - -The HyperExecute Report template comprises a suite of pre-defined, high-impact widgets that deliver crucial insights related to job insights, task trends, stage trends, job summaries, task summaries, stage summaries, task status summaries, and error statistics. These widgets are expertly crafted to work in harmony, offering a crystal-clear and detailed representation of HyperExecute jobs, enabling QA managers to effectively oversee and optimize their testing processes on the platform. - -By leveraging the HyperExecute Report template, QA managers at large enterprises can uncover invaluable insights into the performance of jobs executed on the platform. This knowledge allows them to identify and eliminate bottlenecks, optimize task distribution, and significantly enhance overall test execution efficiency. Consequently, QA managers can drive accelerated test execution times, minimize resource consumption, and deliver industry-leading, high-quality software products that meet the rigorous demands of today's technological era. - -The following are the widgets added to this dashboard: - -- **Job Insights**: Gain a comprehensive understanding of each job's performance, helping you identify areas of improvement and optimize your test execution process. -- **Task Trends**: Analyze the historical trends of tasks within your jobs, allowing you to pinpoint patterns and make data-driven decisions to enhance efficiency. -- **Stage Trends**: Evaluate the progress and performance of various stages within your jobs to ensure seamless execution and identify potential bottlenecks. -- **Job Summary**: Obtain a high-level overview of all jobs executed on the platform, enabling you to quickly assess their status and prioritize actions accordingly. -- **Task Summary**: Review a concise summary of all tasks within your jobs, helping you monitor their progress and allocate resources effectively. -- **Stage Summary**: Get a snapshot of each stage within your jobs, allowing you to track their performance and make informed decisions for continuous improvement. -- **Task Status Summary**: Stay informed about the current status of all tasks, enabling you to proactively address any issues and maintain a smooth testing process. -- **Error Stats**: Identify and analyze errors within your jobs, providing valuable insights for faster debugging and resolution, ultimately resulting in more reliable and higher-quality software products. - -The following are the use cases for the template: - -1. Gain comprehensive insights into job performance, helping identify areas of improvement and optimize test execution processes. -2. Analyze task and stage trends within jobs to pinpoint patterns and make data-driven decisions for enhanced efficiency. -3. Evaluate the progress and performance of various stages within jobs, ensuring seamless execution and identifying potential bottlenecks. -4. Obtain a high-level overview of all jobs executed on the platform to assess their status and prioritize actions accordingly. -5. Assess the efficiency of HyperExecute workflows by analyzing job, task, and stage summaries, enabling continuous optimization and improved test execution times. - -## Explore More Dashboard Features - -* Scheduling a Email notification of the dashboard - Read More -* Share the Dashboard with a public or a private link with your team - [Read More](/docs/analytics-dashboard-features/#expiry-settings-for-dashboard-share-link) -* Exporting the Dashboard as a PDF - [Read More](/docs/analytics-dashboard-features/#export-dashboard-as-pdf) diff --git a/static/docs/analytics-faqs.md b/static/docs/analytics-faqs.md deleted file mode 100644 index f70892a71..000000000 --- a/static/docs/analytics-faqs.md +++ /dev/null @@ -1,56 +0,0 @@ -# Frequently Asked Questions - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -### How do I start using Analytics? - -Getting started with Analytics is simple. - -Step 1: Login into your [TestMu AI](https://www.testmuai.com/) account - -Step 2: Visit the [Analytics](https://analytics.lambdatest.com/) page on the TestMu AI dashboard - -Step 3: Get started with creating a dashboard - -### How to edit the name of a dashboard? - -One can simply edit the name of a dashboard by clicking on the pencil icon on the right and typing the desired name of the dashboard. - -### How to share a dashboard? - -One can easily share the dashboard by clicking on the share icon on the right of the date selection dropdown inside the dashboard page. - -### How to delete a dashboard? - -A dashboard can be deleted in 2 simple ways - -- By clicking on the trash icon on the right of the dashboard - -- By clicking on the red trash icon inside the dashboard - -### How to favourite a dashboard? - -A dashboard can be favourited by clicking on the star icon on the left of the dashboard name. After favouriting the dashboard, it can be seen under the favourites section on the Analytics page. - -### How to edit a widget? - -To edit a widget click the three dots on the right of the widget and then click on the `Edit` button from the dropdown. - -One can edit the name of the widget and even select from the variety of filters available and click on the `Update Widget` button to update the widget. - -### How to add a new widget to the dashboard? - -A new widget can easily be added to a dashboard by clicking on the `Add Widget` button inside the dashboard. - -One can select the widgets they want to add from the variety of widgets available and click on the `Update Dashboard` button to update the dashboard with the new widgets. - -### What TestMu AI products does Analytics support? - -Analytics is currently supported by 3 TestMu AI products - -- Web Automation -- HyperExecute -- App Automation - -**Note** -Other TestMu AI products will be added soon with Analytics support. diff --git a/static/docs/analytics-filter-by-custom-data.md b/static/docs/analytics-filter-by-custom-data.md deleted file mode 100644 index 7123dfef3..000000000 --- a/static/docs/analytics-filter-by-custom-data.md +++ /dev/null @@ -1,47 +0,0 @@ -# Filter by Custom Data - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Ability to sent any custom `key` and `value` pair to TestMu AI platform and then filter the test cases based on the custom data. This feature is available for `Web Automation` and `App Automation` coming soon. - -## Prerequisites For Insights: - -1. You should have an active TestMu AI account. -2. You should han active subscription plan with Web Automation. -3. You should have executed at least one test on the TestMu AI Web Automation platform. - -## Capture Custom Data Filter for Insights in Web Automation   - -### Update your existing test capabilities - -You can use the `customData` capability to send custom data to the TestMu AI platform. The custom data will be displayed in the `Builds Comparison Insights` module. The custom data will be displayed in the `Custom Data` Filter. You can filter the test cases based on the custom data. - -```js -{ -"capabilities": { -"browserName": "chrome", -"version": "latest", -"platform": "Windows 10", -// highlight-start -"customData": { -"key": "value" -} -// highlight-end -} -} - -``` - -## How To Access Custom Filters? - -Currently, the feature is in the phase and will be supported for Build Comparison for Web Automation. - -1. Go to the `Insights` tab on the left navigation bar and click on the `Builds Comparison`. -2. Select the `Build` from the list to drill-down. -3. Now, search for `Test Case` widgets in the `Web Automation, App Automation & HyperExecute` products. -4. Add the widgets to the dashboard by clicking on the `Add Widget` button. - -## Custom Filters Use Cases - -1. You can send the `key` and `value` pair to the TestMu AI platform. -2. You can filter the test cases based on the custom data. diff --git a/static/docs/analytics-heatmap-widgets.md b/static/docs/analytics-heatmap-widgets.md deleted file mode 100644 index c3edd5cca..000000000 --- a/static/docs/analytics-heatmap-widgets.md +++ /dev/null @@ -1,299 +0,0 @@ -# Heatmap Widgets - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Heatmap widgets in LambdaTest Insights provide a powerful way to visualize complex data relationships and identify patterns across multiple dimensions. By representing data density and relationships through color intensity, heatmaps help you quickly spot trends, anomalies, and correlations in your test execution data. - -## What is a Heatmap Widget? - -A heatmap widget displays data as a grid where each cell's **size** and **color** convey different information: - -- **Box Size**: Represents the **total test count** for that combination. Larger boxes indicate more tests executed for that browser-resolution or OS-browser combination. -- **Color/Shade**: Represents the **pass rate** percentage, using a color gradient from red (low pass rates) to green (high pass rates). - -**Current Limitations** -Currently, heatmap widgets use fixed metrics: -- **Box Size**: Always based on **total test count** (not configurable) -- **Box Color**: Always based on **pass rate** (not configurable) - -Custom metric selection for box size and color , allowing you to choose different metrics for visualization. - -This dual-encoding visualization is particularly effective for: - -- **Identifying patterns** across two or more dimensions (e.g., browser × resolution) -- **Spotting outliers** and anomalies in test execution -- **Comparing performance** across different configurations -- **Understanding data distribution** at a glance -- **Quickly identifying** problematic combinations (small boxes with red colors) vs. well-tested successful combinations (large boxes with green colors) - -## Prerequisites - -Before creating a heatmap widget, ensure you have completed the [common prerequisites and setup steps](/docs/dashboards-custom-widgets#prerequisites) for custom widgets. - -## Step-by-Step Guide to Creating a Heatmap Widget - -**Common Steps** -For prerequisites and common setup steps (Navigate to Insights, Create Dashboard, Choose Product, Select Data Source, Configure Filters), see the [Common Steps for Creating Custom Widgets](/docs/dashboards-custom-widgets#common-steps-for-creating-custom-widgets) section in the main Custom Widgets documentation. - -### Step 6: Choose Heatmap Visualization - -1. In the **Choose Visualisation** section, select **Heatmap** from the available options (Line, Bar, Pie, Table, Billboard, Heatmap). - -### Step 7: Configure Grouping - -This is the most critical step for creating meaningful heatmaps: - -1. **Group By*** (Mandatory): Select the primary dimension for grouping your data. Common options include: - - `browser` - Group by browser type (Web Automation, HyperExecute) - - `resolution` - Group by screen resolution (Web Automation, HyperExecute) - - `os` - Group by operating system (all products) - - `os_version` - Group by OS version (all products) - - `device` - Group by device type (App Automation, HyperExecute) - - `build_name` - Group by build name (all products) - - `project_name` - Group by project name (all products) - - Custom data keys you've defined - - For a complete list of available keys for each product, see the [Available Keys for Custom Widgets](/docs/dashboards-custom-widgets#available-keys-for-custom-widgets) section in the main Custom Widgets documentation. - -2. **Then By** (Optional): Select a secondary dimension for nested grouping. This creates a two-dimensional heatmap showing relationships between the two dimensions. - - **Example**: Group by `browser` first, then by `resolution` to see which browser-resolution combinations have the most test activity. - -The combination of Group By and Then By creates the axes of your heatmap. - -### Step 8: Name Your Widget - -1. Enter a descriptive name in the **Widget Name** field (e.g., "Browser × Resolution Test Distribution"). -2. Use clear, descriptive names that indicate what the heatmap shows. - -### Step 9: Preview Your Heatmap - -1. Click the **Preview** button to see how your heatmap will look with the current configuration. -2. Review the visualization to ensure it displays the data as expected. -3. Use the refresh icon to update the preview if you make changes. - -### Step 10: Create the Dashboard - -1. Enter a **Dashboard Name** in the bottom section. -2. Click **Create Dashboard** to save your heatmap widget. -3. Your heatmap will be added to the dashboard and ready for analysis. - -## How Heatmaps Work - -Understanding how heatmaps encode information is crucial for effective analysis: - -### Box Size: Total Test Count - -The **size of each box** in the heatmap represents the **total number of tests** executed for that specific combination (e.g., Chrome × 1920x1080). - -- **Larger boxes** = More tests executed for that combination -- **Smaller boxes** = Fewer tests executed for that combination -- **No box** = No tests executed for that combination - -Currently, box size is always determined by the total test count. This metric is not configurable at the moment, but custom metric selection for box size . - -This helps you quickly identify: -- Which combinations have the most test coverage -- Which combinations may need more testing -- Testing activity distribution across different configurations - -### Color: Pass Rate - -The **color of each box** represents the **pass rate percentage** for that combination, using a color gradient from red (low pass rates) to green (high pass rates). - -| Pass Rate Range | Color | Interpretation | -|----------------|-------|----------------| -| 0-20% | Dark Red | Critical issues - very low pass rate, needs immediate attention | -| 20-40% | Medium Red | Significant problems - low pass rate, requires investigation | -| 40-50% | Light Red | Below average - pass rate needs improvement | -| 50-60% | Light Green | Average performance - acceptable but could be better | -| 60-80% | Medium Green | Good performance - solid pass rate | -| 80-100% | Dark Green | Excellent performance - high pass rate, well-tested | - -Currently, box color is always determined by the pass rate percentage. This metric is not configurable at the moment, but custom metric selection for box color . - -### Combining Size and Color - -The combination of box size and color provides powerful insights: - -- **Large Dark Green Box**: High test count with excellent pass rate - well-tested and stable combination -- **Large Dark Red Box**: High test count with very low pass rate - critical issue affecting many tests -- **Small Dark Red Box**: Low test count with very low pass rate - may need more testing or investigation -- **Small Dark Green Box**: Low test count with high pass rate - limited testing but good results so far - -## Generating Meaningful Insights from Heatmaps - -Heatmaps excel at revealing patterns that might be hidden in tabular data. Here's how to interpret and extract insights: - -### Understanding the Visualization - -Heatmaps use size and color to encode information. Refer to the [How Heatmaps Work](#how-heatmaps-work) section above for detailed explanations of box size and color encoding. - -### Key Insights to Look For - -1. **High-Activity Combinations**: Identify which browser-resolution or OS-browser combinations have the most test activity (large boxes). This helps prioritize testing efforts and understand where most of your testing is focused. - -2. **Failure Patterns**: Red-colored boxes (especially dark red) indicate problematic combinations with low pass rates that need immediate attention. Large red boxes are particularly concerning as they affect many tests. - -3. **Success Patterns**: Green-colored boxes (especially dark green) indicate well-performing combinations. Large green boxes show stable, well-tested configurations. - -4. **Coverage Gaps**: Missing or very small boxes reveal combinations that haven't been tested, helping you identify coverage gaps in your testing strategy. - -5. **Anomalies**: Unexpected combinations (e.g., large red boxes where you expect green, or small boxes in critical configurations) can indicate issues or opportunities for optimization. - -6. **Trends Across Dimensions**: Observe how box sizes and colors change as you move across one dimension while keeping another constant. This reveals patterns like "Chrome performs well across all resolutions" or "1920x1080 has issues across multiple browsers." - -7. **Priority Issues**: Focus on large red boxes first - these represent combinations with high test volume but poor pass rates, indicating critical problems affecting many tests. - -### Interacting with Heatmaps - -- **Hover**: Hover over boxes to see detailed metrics including total test count, pass rate percentage, total passed/failed tests, and other relevant metrics -- **Click**: Some heatmaps allow drilling down into specific combinations for more detailed analysis -- **Filter**: Apply additional filters to focus on specific subsets of data -- **Color Legend**: Refer to the color scale to understand what each shade represents in terms of pass rate ranges - -## Use Cases for Heatmap Widgets - -### 1. Browser and Resolution Analysis - -**Use Case**: Identify which browser-resolution combinations are most commonly used and which have the highest failure rates. - -**Configuration**: -- Group By: `browser` -- Then By: `resolution` -- Source: Tests - -**Insight**: Helps prioritize browser-resolution combinations for testing and identify problematic configurations. Large boxes indicate high test activity, red boxes highlight low pass rates, and green boxes show well-performing combinations. - -### 2. OS and Browser Compatibility Testing - -**Use Case**: Understand test distribution and performance across different OS-browser combinations. - -**Configuration**: -- Group By: `os` -- Then By: `browser` -- Source: Tests - -**Insight**: Ensures comprehensive cross-platform testing coverage and identifies compatibility issues. - -### 3. Device and OS Version Analysis (App Automation) - -**Use Case**: Analyze test execution patterns across different devices and OS versions. - -**Configuration**: -- Group By: `device` -- Then By: `os` or `os_version` -- Source: Tests - -**Insight**: Optimizes device testing strategy and identifies devices/OS versions that need more attention. - -### 4. Custom Data Dimension Analysis - -**Use Case**: Visualize test patterns based on custom attributes like release version, feature flags, or environment. - -**Configuration**: -- Group By: Custom key (e.g., `release_version`) -- Then By: `browser` or `os` -- Source: Tests -- Filters: Apply custom data filters - -**Insight**: Track how different releases or configurations perform across different environments. - -### 5. Auto Heal Effectiveness Analysis - -**Use Case**: Visualize Auto Heal performance across different browser-OS combinations. - -**Configuration**: -- Group By: `browser` -- Then By: `os` -- Source: Auto Heal - -**Insight**: Identify which combinations benefit most from Auto Heal and where improvements are needed. Note: Custom metrics like Auto Heal Success Rate for box color selection. - -### 6. Test Coverage Visualization - -**Use Case**: Ensure comprehensive test coverage across all browser-OS combinations. - -**Configuration**: -- Group By: `browser` -- Then By: `os` -- Source: Tests - -**Insight**: Identify gaps in test coverage and ensure all critical combinations are tested. - -## Available Keys for Heatmap Widgets - -For a complete list of available keys that can be used in the **Group By** and **Then By** fields when configuring your heatmap, see the [Available Keys for Custom Widgets](/docs/dashboards-custom-widgets#available-keys-for-custom-widgets) section in the main Custom Widgets documentation. - -For heatmaps, choose keys that represent categorical dimensions (like browser, OS, project name) rather than numeric or date values for best visualization results. - -### Common Metrics for Heatmaps - -Currently, heatmap widgets use fixed metrics (see [How Heatmaps Work](#how-heatmaps-work) section above). Custom metric selection for box size and color , allowing you to choose different metrics like failure rate, total passed/failed tests, average duration, and Auto Heal success rate. - -## Best Practices for Heatmap Widgets - -### 1. Choose Meaningful Dimensions - -- Select dimensions that have a clear relationship or that you want to compare -- Avoid dimensions with too many unique values (this can make the heatmap cluttered) -- Use filters to focus on relevant subsets of data - -### 2. Combine Dimensions Strategically - -Choose dimensions that create meaningful cross-sections. Common effective combinations include Browser × Resolution, OS × Browser, Device × OS Version, and Custom Key × Browser/OS. - -### 3. Use Filters Effectively - -- Use custom data filters to analyze specific releases, features, or environments -- Apply filters based on available keys (browser, OS, project, etc.) -- Combine multiple filters to create highly targeted heatmaps - -### 4. Name Widgets Clearly - -Use descriptive names that indicate what the heatmap shows, including the dimensions being visualized (e.g., "Browser × Resolution Test Distribution"). - -### 5. Regular Review and Updates - -- Review heatmaps regularly to identify new patterns or issues -- Update filters and dimensions as your testing strategy evolves -- Create multiple heatmaps with different configurations to explore various aspects - -### 6. Share Insights - -- Share heatmap dashboards with team members and stakeholders -- Use heatmaps in reports and presentations to communicate findings -- Create multiple heatmaps for different audiences (QA, Dev, Product) - -## Troubleshooting - -### Heatmap Shows No Data - -- **Check Filters**: Ensure filters aren't too restrictive -- **Verify Data Source**: Confirm you have test data for the selected product -- **Check Dimensions**: Ensure the selected grouping keys have data in your test executions - -### Heatmap is Too Cluttered - -- **Use Filters**: Narrow down the data using filters -- **Choose Fewer Dimensions**: Consider using only one grouping dimension -- **Focus on Specific Values**: Filter to specific browsers, OS, or other dimensions - -### Colors Don't Show Clear Patterns - -- **Check Metric**: Ensure you're visualizing a meaningful metric -- **Review Data Range**: Verify you have sufficient data points -- **Adjust Filters**: Try different filter combinations - -## Next Steps - -After creating your heatmap widget: - -1. **Add to Dashboard**: Combine heatmaps with other widgets for comprehensive analysis -2. **Share Dashboard**: Share your dashboard with team members for collaborative insights -3. **Create Variations**: Create multiple heatmaps with different dimensions to explore various aspects -4. **Review Regularly**: Use insights from heatmaps to identify areas needing attention and track improvements - -## Support - -If you have any questions or need assistance with creating heatmap widgets, please reach out to our [24/7 customer support](mailto:support@testmuai.com) or visit our [Support Center](https://www.testmuai.com/support/). diff --git a/static/docs/analytics-modules-accessibility-widgets.md b/static/docs/analytics-modules-accessibility-widgets.md deleted file mode 100644 index 8ec964952..000000000 --- a/static/docs/analytics-modules-accessibility-widgets.md +++ /dev/null @@ -1,59 +0,0 @@ -# Accessibility Dashboard Widgets - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI Accessibility Dashboard widgets provide comprehensive insights into your accessibility testing efforts. These widgets help you optimize your accessibility testing efforts by providing detailed insights into your accessibility testing scans, -Top 10 URLs Issue Criticality, and more. - -The Accessibility Widgets module is currently in . If you have any feedback or suggestions, please feel free to reach out to us at [support@testmuai.com](mailto:support@testmuai.com). - -## Onboarding: where to find the widgets - -1. Open **Insights** (or the analytics area your account exposes) from the main TestMu AI navigation. -2. Add or enable the **Accessibility** widget pack on your dashboard canvas (exact UI depends on the analytics release). -3. Pick a **date range** that includes at least one completed Accessibility scan; empty widgets usually mean no data in range. -4. Start with **Accessibility Test Overview** for totals, then **Types of Scans** to confirm manual vs automated mix, then **Top 10 URLs Issue Criticality** for remediation focus. -5. Click through any **drill-down** links the widgets expose to jump into the underlying Accessibility report when available. -6. For executive readouts, **export** or screenshot the widgets and pair them with narrative from **[Accessibility Compliance Guide](/support/docs/accessibility-compliance-guide/)**. Widgets summarize engineering signals, not legal conclusions. - -## Accessibility Test Overview - -The Accessibility Test Overview widget provides a comprehensive overview of your accessibility testing efforts. It displays the total number of accessibility WCAG related issues, critical issues, serious issues, moderate issues, and minor issues detected in your accessibility testing scans. - -### How it works? - -* This widget helps you understand the overall accessibility status of your website or web application. -* It provides a detailed breakdown of the accessibility issues detected in your accessibility testing scans. -* You can use this widget to identify the most critical accessibility issues that need immediate attention. - -### Value Proposition - -By using this widget, you can get a quick overview of your accessibility testing efforts and identify the most critical accessibility issues that need immediate attention. This helps you optimize your accessibility testing efforts and improve the overall accessibility of your website or web application. - -## Types of Scans - -The Types of Scans widget provides a detailed breakdown of the types of accessibility testing scans conducted on your website or web application. It displays the total number of manual scans, automated scans, and hybrid scans conducted in your accessibility testing efforts. - -### How it works? - -* This widget helps you understand the types of accessibility testing scans conducted on your website or web application. -* It provides a detailed breakdown of the manual scans, automated scans, and hybrid scans conducted in your accessibility testing efforts. -* You can use this widget to identify the most effective type of accessibility testing scan for your website or web application. - -### Value Proposition - -By using this widget, you can get insights into the types of accessibility testing scans conducted on your website or web application. This helps you optimize your accessibility testing efforts and choose the most effective type of accessibility testing scan for your website or web application. - -## Top 10 URLs Issue Criticality - -The Top 10 URLs Issue Criticality widget provides a detailed breakdown of the top 10 URLs with the highest issue criticality detected in your accessibility testing scans. It displays the URL, issue criticality, and the total number of issues detected for each URL. - -### How it works? - -* This widget helps you identify the top 10 URLs with the highest issue criticality in your accessibility testing scans. -* It provides a detailed breakdown of the URL, issue criticality, and the total number of issues detected for each URL. -* You can use this widget to prioritize the URLs that need immediate attention and address the critical accessibility issues on your website or web application. - -### Value Proposition - -By using this widget, you can identify the top 10 URLs with the highest issue criticality in your accessibility testing scans. This helps you prioritize the URLs that need immediate attention and address the critical accessibility issues on your website or web application. diff --git a/static/docs/analytics-modules-automation-test-overview.md b/static/docs/analytics-modules-automation-test-overview.md deleted file mode 100644 index 4b67a58f8..000000000 --- a/static/docs/analytics-modules-automation-test-overview.md +++ /dev/null @@ -1,116 +0,0 @@ -# Web & App Automation Modules - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The Test Overview module in TestMu AI Analytics is a powerful tool that provides you with a comprehensive view of your test performance across various dimensions. By leveraging this module, you can gain valuable insights into test flakiness, total tests run, and test categorization by status and platform. - -## Test Health Snapshot - -The Test Health Snapshot widget enables you to quickly assess the flakiness or inconsistency of your tests executed on the TestMu AI platform. This widget displays the count of unique test names and their consistency in test status over a specified duration. - -#### How It Works -- The widget calculates the total number of unique test names executed on the platform. -- For each unique test name, it determines the consistency of the test status (passing or failing) across multiple runs. -- Tests that consistently pass or fail are considered stable, while tests with varying statuses are marked as inconsistent. - -#### Value Proposition -By identifying inconsistent tests, you can focus your efforts on stabilizing your test suite, reducing flaky tests, and improving overall test reliability. This widget helps you prioritize your test maintenance tasks and ensures a more robust testing process. - -## Build Health Snapshot - -The Build Health Snapshot widget provides an overview of the performance and consistency of your builds created on the TestMu AI platform. It displays the count of unique build names and their consistency in build status over a specified duration. - -#### How It Works -- The widget calculates the total number of unique build names created on the platform. -- For each unique build name, it determines the consistency of the build status (successful or error) across multiple runs. -- Builds that consistently succeed or fail are considered stable, while builds with varying statuses are marked as inconsistent. - -#### Value Proposition -By monitoring the health of your builds, you can identify and resolve any inconsistencies or errors in your build process. This widget helps you ensure the reliability and stability of your builds, leading to a more efficient and effective testing workflow. - -## Test Summary - -The Test Summary widget provides a high-level overview of the total tests run on the TestMu AI platform, grouped by their status. It also includes information about various test-related errors, such as test case errors, idle timeouts, queue timeouts, and Lambda errors. - -#### How It Works -- The widget calculates the total number of tests executed on the platform. -- It categorizes the tests based on their status (passed, failed, skipped, etc.). -- The widget also tracks and displays the count of different test-related errors encountered during execution. - -#### Value Proposition -The Test Summary widget allows you to quickly gauge the overall health and performance of your tests. By understanding the distribution of test statuses and identifying common errors, you can take targeted actions to improve test stability, reduce errors, and optimize your testing process. - -### Understanding Unique Instances in Test Summary - -The Test Summary widget supports unique instance calculation, which consolidates retry runs for more accurate reporting. When configuring the widget, you can use the **"Show test retries separately"** option to control how retries are displayed: - -- **Checked (Show test retries separately)**: All test executions are displayed individually, including every retry attempt. Use this when you need to see the full execution history. -- **Unchecked (Default)**: Tests are grouped by **test name + environment** (browser + OS + device + resolution), showing only the final status for each unique instance. This provides a cleaner view of your true test health. - -**Example**: If a test fails initially but passes on retry, with the option unchecked, it shows as "Passed" in the summary. With the option checked, both the failed and passed executions appear separately. - -For a comprehensive guide on how unique instances and retry attempts are detected, including step-by-step examples, see [Unique Test Instances and Intelligent Retry Detection](/support/docs/analytics-unique-instances-retry-detection/). - -## Test Browser Distribution - -The Test Browser Distribution widget provides a breakdown of the tests executed across different browsers on the TestMu AI platform. It displays the distribution of tests by browser and allows you to drill down into specific browser versions. - -#### How It Works -- The widget calculates the total number of tests executed on each browser. -- It presents the distribution of tests across different browsers in a visual format. -- You can click on a specific browser to view the top 10 browser versions based on the number of tests executed. - -#### Value Proposition -By understanding the browser distribution of your tests, you can ensure adequate coverage across different browsers and versions. This widget helps you identify any browser-specific issues and optimize your test coverage to deliver a seamless user experience across various browsers. - -## Test OS Distribution - -The Test OS Distribution widget provides an overview of the tests executed across different operating systems on the TestMu AI platform. It displays the distribution of tests by operating system and allows you to drill down into specific OS versions. - -#### How It Works -- The widget calculates the total number of tests executed on each operating system. -- It presents the distribution of tests across different operating systems in a visual format. -- You can click on a specific operating system to view the top 10 OS versions based on the number of tests executed. - -#### Value Proposition -Understanding the distribution of tests across different operating systems helps you ensure compatibility and identify any OS-specific issues. This widget enables you to optimize your test coverage and deliver a consistent user experience across various operating systems. - -## Test Trends - -The Test Trends widget provides a visual representation of the trends in test execution over time. It allows you to analyze the performance of your tests and compare current trends with historical data. - -#### How It Works -- The widget tracks the number of tests executed on the platform over a specified time period. -- It presents the test trends in a graph format, allowing you to visualize the changes in test volume over time. -- You can interact with the graph to view data for specific time intervals and compare trends across different periods. - -#### Value Proposition -By analyzing test trends, you can identify patterns, spikes, or dips in test execution. This information helps you understand the impact of changes in your testing process, identify areas for optimization, and make data-driven decisions to improve your overall testing strategy. - -## Test Device Coverage - -The Test Device Coverage widget, specifically designed for App Automation, provides an overview of the tests executed across different mobile devices on the TestMu AI platform. It displays the distribution of tests by device and allows you to drill down into specific device models. - -#### How It Works -- The widget calculates the total number of tests executed on each mobile device. -- It presents the distribution of tests across different devices in a visual format. -- You can click on a specific device to view the distribution of tests across different models of that device. - -#### Value Proposition -Understanding the device coverage of your tests is crucial for ensuring the compatibility and performance of your mobile applications. This widget helps you identify any device-specific issues, optimize your test coverage, and deliver a seamless user experience across a wide range of mobile devices. - -## Test Status Ratio - -The Test Status Ratio widget provides a breakdown of the test statuses (passed, failed, skipped, etc.) for the tests executed on the TestMu AI platform. It allows you to assess the overall quality and stability of your test cases. - -#### How It Works -- The widget calculates the total number of tests executed on the platform. -- It categorizes the tests based on their status (passed, failed, skipped, etc.). -- The widget presents the distribution of test statuses in a visual format, such as a pie chart or bar graph. - -#### Value Proposition -By analyzing the test status ratio, you can quickly identify the proportion of passing, failing, and skipped tests. This information helps you assess the overall health and reliability of your test suite. You can use this widget to track improvements in test stability over time and identify areas that require further attention and optimization. - -The Test Overview module in TestMu AI Analytics empowers you to gain comprehensive insights into your test performance, identify trends, and make data-driven decisions to optimize your testing process. By leveraging the various widgets available in this module, you can ensure test reliability, improve test coverage, and deliver high-quality applications across different browsers, operating systems, and devices. - -Start exploring the Test Overview module today and unlock the full potential of your testing data to drive continuous improvement and success in your testing efforts. diff --git a/static/docs/analytics-modules-error-insights.md b/static/docs/analytics-modules-error-insights.md deleted file mode 100644 index 37194d3a7..000000000 --- a/static/docs/analytics-modules-error-insights.md +++ /dev/null @@ -1,14 +0,0 @@ -# Error Insights - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The `Error Insights` module enables the users to get all the insights into the test errors on TestMu AI in one place. The user can easily find information about the count, and type of the test errors through the highly customizable widgets. - -## Error Stats - -The `Error Stats` widget allows the user to easily analyze the tests categorized by their Status on the TestMu AI platform including Test Case Errors, Idle Timeout, Queue Timeout, and Lambda Error. The user can even filter the graph by clicking on the relevant legends at the bottom of the graph. - -**Use case** -John is a QA Manager, and his team runs more than 50,000 tests in a month across various TestMu AI products like Web Automation, App Automation, and HyperExecute. - -John wants to understand the error distribution of the executed tests on the platform. With the Error Stats widget he can identify the list of tests that have a particular error and resolve them. diff --git a/static/docs/analytics-modules-groups.md b/static/docs/analytics-modules-groups.md deleted file mode 100644 index cf9d25ca3..000000000 --- a/static/docs/analytics-modules-groups.md +++ /dev/null @@ -1,19 +0,0 @@ -# Usage insights for Groups - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -### Introduction to Groups Insights - -TestMu AI Analytics Modules provide comprehensive insights into your test groups, helping you understand the performance of your test groups and the distribution of your test cases across different groups. This helps you identify the test groups that are performing well and the ones that need improvement. You can also track the execution time of your test groups and the number of test cases executed in each group. - -### Usage by Groups - -The Usage by Groups widget provides insights into the performance of your test groups. You can this widget to your dashboard and apply required filters to view the performance of your test groups. The widget displays the following information: - -* **Group Name**: The name of the test group. -* **Total Test Cases**: The total number of test cases in the group. -* **Total Hours of Usage**: The total number of test hours executed in the group. - -## Support and Assistance - -In case you need any assistance, reach out to our experts at [email](mailto:support@testmuai.com) or you can connect with our 24/7 chat support. We will be happy to help you! diff --git a/static/docs/analytics-modules-hyperexecute.md b/static/docs/analytics-modules-hyperexecute.md deleted file mode 100644 index c5372ef58..000000000 --- a/static/docs/analytics-modules-hyperexecute.md +++ /dev/null @@ -1,139 +0,0 @@ -# HyperExecute Analytics - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The HyperExecute Analytics module empowers QA managers to gain comprehensive insights into their HyperExecute testing pipeline. This module provides a centralized view of all the key metrics related to jobs, tasks, and stages executed on the TestMu AI platform. - -## Job Trends - -The Job Trends widget allows you to analyze the trends of jobs executed on the platform, categorized by their status: Failed, Aborted, Lambda Error, Timeout, and Completed. You can easily filter the data using the legends at the top of the graph. - -- X-Axis: Represents the time intervals at which the job trends are measured. -- Y-Axis: Represents the number of jobs categorized by their status. - -#### How It Works -- The widget tracks the number of jobs and their respective statuses over a specified time period. -- It presents the job trends in a graph format, displaying the number of jobs for each status at each time interval. -- You can hover over specific data points to view the exact number of jobs for each status at that particular time. - -#### Value Proposition -By examining the Job Trends widget, you can identify patterns, fluctuations, or anomalies in your job execution over time. This information helps you assess the stability and reliability of your HyperExecute testing pipeline, allowing you to proactively address any emerging issues and ensure the consistent quality of your jobs. - -#### Use Case -As a QA Manager, John's team runs over 50,000 jobs per month across various TestMu AI products. With the Job Trends widget, John can: -- Understand the status distribution of jobs executed by his team. -- Identify any failing job numbers and troubleshoot them by viewing the logs. - -## Job Queue Time Trends - -The Job Queue Time Trends widget provides a comprehensive view of the total queue time for all jobs in a visual format. It displays a stacked line or bar chart, with the total queue time for each job on the y-axis and the job number in descending order from right on the x-axis. - -- X-Axis: Represents the job numbers in descending order. -- Y-Axis: Represents the total queue time for each job. - -#### How It Works -- The widget calculates the total queue time for each job executed on the platform. -- It presents the queue time trends in a graph format, displaying the total queue time for each job. -- You can hover over specific data points to view the exact queue time for a particular job. - -#### Value Proposition -By analyzing the Job Queue Time Trends widget, you can identify bottlenecks, optimize resource allocation, and minimize waiting times in your HyperExecute testing pipeline. This information helps you streamline your testing process, ensure efficient utilization of resources, and reduce overall execution time. - -#### Use Cases -- Identify trends in queue time and investigate the causes behind fluctuations. -- Optimize resource allocation to reduce overall queue time. -- Monitor the impact of changes made to testing processes, infrastructure, or configurations. -- Compare performance over time by selecting different date ranges. -- Share insights with team members to foster collaboration and drive improvements. - -## Job Summary - -The Job Summary widget enables you to track the total number of jobs run on the platform, grouped by their status: Completed, Partially Completed, Failed, Aborted, etc. - -#### How It Works -- The widget calculates the total number of jobs executed on the platform. -- It categorizes the jobs based on their status and provides a summary of the count for each status. - -#### Value Proposition -The Job Summary widget gives you a high-level overview of the overall health and progress of your HyperExecute testing efforts. By monitoring the total count of jobs and their status distribution, you can assess the efficiency and effectiveness of your testing pipeline and identify areas that require attention or optimization. - -#### Use Case -John can use the Job Summary widget to: -- Get a total count of jobs executed on the platform for a specific environment or status. -- Monitor the overall health and progress of his team's testing efforts. - -## Stage Summary - -The Stage Summary widget allows you to track the total number of stages run on the platform, grouped by their status: Completed, Skipped, Failed, Aborted, etc. - -#### How It Works -- The widget calculates the total number of stages executed on the platform. -- It categorizes the stages based on their status and provides a summary of the count for each status. - -#### Value Proposition -The Stage Summary widget provides a granular view of the execution progress at the stage level. By monitoring the total count of stages and their status distribution, you can identify any bottlenecks, failures, or inefficiencies within specific stages of your HyperExecute testing pipeline. This information helps you focus your optimization efforts on the critical stages that require attention. - -#### Use Case -John can use the Stage Summary widget to: -- Get a total count of stages executed on the platform for a specific environment or status. -- Identify any bottlenecks or issues at the stage level. - -## Stage Trends - -The Stage Trends widget enables you to analyze the trends of stages for each job executed on the platform, categorized into Completed, Failed, Lambda Error, and Cancelled. You can filter the data by a specific date and time. - -- X-Axis: Represents the time intervals at which the stage trends are measured. -- Y-Axis: Represents the number of stages categorized by their status. - -#### How It Works -- The widget tracks the number of stages and their respective statuses over a specified time period. -- It presents the stage trends in a graph format, displaying the number of stages for each status at each time interval. -- You can hover over specific data points to view the exact number of stages for each status at that particular time. - -#### Value Proposition -By examining the Stage Trends widget, you can identify patterns, fluctuations, or anomalies in your stage execution over time. This information helps you assess the stability and reliability of specific stages within your HyperExecute testing pipeline, allowing you to proactively address any emerging issues and ensure the consistent quality of your stages. - -#### Use Case -John can use the Stage Trends widget to: -- Understand the status distribution of stages in the jobs executed by his team. -- Identify any failing stages and gather information for troubleshooting. - -## Task Summary - -The Task Summary widget allows you to track the total number of tasks run on the platform, grouped by their status: Completed, Skipped, Failed, Aborted, etc. - -#### How It Works -- The widget calculates the total number of tasks executed on the platform. -- It categorizes the tasks based on their status and provides a summary of the count for each status. - -#### Value Proposition -The Task Summary widget provides a granular view of the execution progress at the task level. By monitoring the total count of tasks and their status distribution, you can identify any bottlenecks, failures, or inefficiencies within specific tasks of your HyperExecute testing pipeline. This information helps you focus your optimization efforts on the critical tasks that require attention. - -#### Use Case -John can use the Task Summary widget to: -- Get a total count of tasks executed on the platform for a specific environment or status. -- Monitor the granular progress of his team's testing efforts. - -## Task Trends - -The Task Trends widget enables you to analyze the trends of tasks for each job executed on the platform, categorized into Completed, Failed, Aborted, Lambda Error, and Cancelled. You can filter the data by a specific date and time. - -- X-Axis: Represents the time intervals at which the task trends are measured. -- Y-Axis: Represents the number of tasks categorized by their status. - -#### How It Works -- The widget tracks the number of tasks and their respective statuses over a specified time period. -- It presents the task trends in a graph format, displaying the number of tasks for each status at each time interval. -- You can hover over specific data points to view the exact number of tasks for each status at that particular time. - -#### Value Proposition -By examining the Task Trends widget, you can identify patterns, fluctuations, or anomalies in your task execution over time. This information helps you assess the stability and reliability of specific tasks within your HyperExecute testing pipeline, allowing you to proactively address any emerging issues and ensure the consistent quality of your tasks. - -#### Use Case -John can use the Task Trends widget to: -- Understand the status distribution of tasks in the jobs executed by his team. -- Identify any failing tasks and gather information for troubleshooting. - -The HyperExecute Analytics module provides a powerful set of tools to gain deep insights into your HyperExecute testing pipeline. By leveraging these widgets, you can optimize your testing process, identify bottlenecks, and ensure the smooth execution of your jobs, stages, and tasks. - -Unlock the full potential of HyperExecute Analytics today and take your testing game to the next level! diff --git a/static/docs/analytics-modules-manual-test-overview.md b/static/docs/analytics-modules-manual-test-overview.md deleted file mode 100644 index 9476f81c1..000000000 --- a/static/docs/analytics-modules-manual-test-overview.md +++ /dev/null @@ -1,119 +0,0 @@ -# Realtime & Real-Device Modules - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The Test Overview module in TestMu AI Analytics is a powerful tool that provides you with a comprehensive view of your test performance across various dimensions. By leveraging this module, you can gain valuable insights into test flakiness, total tests run, and test categorization by status and platform. - -## Browser Testing Summary - -The Browser Testing Summary widget offers a comprehensive overview of your browser testing efforts on the TestMu AI platform. It provides key metrics such as total tests executed, issues/bugs identified, time consumed, and unique launch URLs tested. - -#### How It Works -- The widget calculates the total number of tests executed on the platform. -- It categorizes the tests based on their status, such as total tests, total issues/bugs marked, total time consumed, and total launch URLs. -- The widget presents the test counts for each status category in a scorecard format. - -#### Value Proposition -This widget enables you to assess the overall health and effectiveness of your browser testing pipeline. By tracking the total number of tests and issues/bugs marked, you can gauge the thoroughness of your testing process and identify areas requiring attention. The time consumed metric helps you optimize test execution efficiency, while the total launch URLs provide insights into the breadth of your test coverage. - -## App Testing Summary - -The App Testing Summary widget provides a holistic view of your app testing efforts on the TestMu AI platform. It showcases essential metrics such as total tests executed, issues/bugs identified, time consumed, and total apps tested. - -#### How It Works -- The widget calculates the total number of tests executed on the platform. -- It categorizes the tests based on their status, such as total tests, total issues/bugs marked, total time consumed, and total apps tested. -- The widget presents the test counts for each status category in a scorecard format. - -#### Value Proposition -This widget allows you to evaluate the effectiveness and efficiency of your app testing process. By monitoring the total number of tests and issues/bugs marked, you can assess the robustness of your app testing strategy and identify areas for improvement. The time consumed metric enables you to optimize test execution speed, while the total apps tested gives you visibility into the breadth of your app coverage. - -## Browser Testing - OS - -The Browser Testing - OS widget provides a detailed breakdown of your browser tests across different operating systems on the TestMu AI platform. It visualizes the distribution of tests executed on each supported OS, giving you valuable insights into your test coverage. - -#### How It Works -- The widget calculates the total number of tests executed on each operating system. -- It presents the distribution of tests across different operating systems in a visual format, such as a list or pie chart. -- You can hover over each operating system to view the exact number of tests executed on that OS. - -#### Value Proposition -By analyzing the OS distribution, you can ensure that your web application is thoroughly tested across a wide range of operating systems. This widget helps you identify any OS-specific issues, allocate testing resources effectively, and make informed decisions to provide a seamless user experience across different platforms. - -## App Testing - OS - -The App Testing - OS widget offers a comprehensive breakdown of your app tests across different operating systems on the TestMu AI platform. It presents a visual representation of the distribution of tests executed on each supported mobile OS, providing valuable insights into your app test coverage. - -#### How It Works -- The widget calculates the total number of tests executed on each operating system. -- It presents the distribution of tests across different operating systems in a visual format, such as a list or pie chart. -- You can hover over each operating system to view the exact number of tests executed on that OS. - -#### Value Proposition -By examining the OS distribution, you can ensure that your mobile app is rigorously tested across a diverse range of operating systems. This widget enables you to identify any OS-specific compatibility issues, optimize testing efforts, and deliver a consistent user experience across different mobile platforms. - -## Browser Testing - Browser - -The Browser Testing - Browser widget provides a granular view of your browser tests across different web browsers on the TestMu AI platform. It showcases the distribution of tests executed on each supported browser, offering insights into your browser compatibility testing efforts. - -#### How It Works -- The widget calculates the total number of tests executed on each browser. -- It presents the distribution of tests across different browsers in a visual format, such as a list or pie chart. -- You can hover over each browser to view the exact number of tests executed on that browser. - -#### Value Proposition -By analyzing the browser distribution, you can ensure that your web application is thoroughly tested across a wide range of browsers. This widget helps you identify any browser-specific issues, allocate testing resources effectively, and make informed decisions to deliver a consistent user experience across different browsers. - -## Browser Testing - Device - -The Browser Testing - Device widget provides a detailed breakdown of your browser tests across different devices on the TestMu AI platform. It visualizes the distribution of tests executed on each supported device, giving you valuable insights into your cross-device browser testing coverage. - -#### How It Works -- The widget calculates the total number of tests executed on each device. -- It presents the distribution of tests across different devices in a visual format, such as a list or pie chart. -- You can hover over each device to view the exact number of tests executed on that device. - -#### Value Proposition -By analyzing the device distribution, you can ensure that your web application is thoroughly tested across a wide range of devices. This widget helps you identify any device-specific issues, optimize testing efforts, and make informed decisions to provide a seamless user experience across different devices. - -## App Testing - Device - -The App Testing - Device widget offers a comprehensive breakdown of your app tests across different devices on the TestMu AI platform. It presents a visual representation of the distribution of tests executed on each supported device, providing valuable insights into your app testing coverage. - -#### How It Works -- The widget calculates the total number of tests executed on each device. -- It presents the distribution of tests across different devices in a visual format, such as a list or pie chart. -- You can hover over each device to view the exact number of tests executed on that device. - -#### Value Proposition -By examining the device distribution, you can ensure that your mobile app is rigorously tested across a diverse range of devices. This widget enables you to identify any device-specific compatibility issues, optimize testing efforts, and deliver a consistent user experience across different mobile devices. - -## Browser Testing Trends - -The Browser Testing Trends widget provides a visual representation of your browser testing activity over time on the TestMu AI platform. It allows you to track the volume of tests executed and analyze trends in your testing efforts. - -- X-Axis: Represents the time intervals at which the browser tests are executed. -- Y-Axis: Represents the number of browser tests executed at each time interval. - -#### How It Works -- The widget tracks the number of browser tests executed on the platform over a specified time period. -- It presents the browser testing trends in a graph format, displaying the number of tests executed at each time interval. -- You can hover over specific data points to view the exact number of tests executed at that particular time. - -#### Value Proposition -By examining the browser testing trends, you can gain valuable insights into your testing patterns, identify peak testing periods, and monitor the overall progress of your browser testing initiatives. This widget helps you make data-driven decisions, optimize resource allocation, and ensure the continuity and effectiveness of your browser testing process. - -## App Testing Trends - -The App Testing Trends widget provides a visual representation of your app testing activity over time on the TestMu AI platform. It allows you to track the volume of tests executed and analyze trends in your app testing efforts. - -- X-Axis: Represents the time intervals at which the app tests are executed. -- Y-Axis: Represents the number of app tests executed at each time interval. - -#### How It Works -- The widget tracks the number of app tests executed on the platform over a specified time period. -- It presents the app testing trends in a graph format, displaying the number of tests executed at each time interval. -- You can hover over specific data points to view the exact number of tests executed at that particular time. - -#### Value Proposition -By examining the app testing trends, you can gain valuable insights into your testing patterns, identify peak testing periods, and monitor the overall progress of your app testing initiatives. This widget helps you make data-driven decisions, optimize resource allocation, and ensure the continuity and effectiveness of your app testing process. diff --git a/static/docs/analytics-modules-resource-utilization.md b/static/docs/analytics-modules-resource-utilization.md deleted file mode 100644 index 05fbad353..000000000 --- a/static/docs/analytics-modules-resource-utilization.md +++ /dev/null @@ -1,138 +0,0 @@ -# Concurrency Usage Insights - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The Concurrency Usage Insights module enables QA Managers to get an overview of TestMu AI resources being utilized by their teams. It provides widgets to monitor concurrency trends at different levels and track peak usage patterns. - -## Available Widgets - -| Widget | Description | -|--------|-------------| -| Concurrency Trends | Org-level view of concurrent sessions over time | -| Group Concurrency Trends | Group-level concurrency utilization | -| SubOrg Concurrency Trends | Sub-organization level concurrency utilization | -| Custom Concurrency Trends | Custom KPI-based concurrency with dimension breakdowns | - -## Concurrency Trends (Org Level) - -The Concurrency Trends widget provides a visual representation of your parallel test execution at the organization level on the TestMu AI platform. It allows you to monitor the number of concurrent sessions running over time, helping you optimize resource utilization and identify peak testing periods. - -### Chart Axes - -- **X-Axis**: Represents the time intervals at which the concurrent sessions are measured -- **Y-Axis**: Represents the number of concurrent sessions, categorized into "Queued" and "In Use" sessions - -### How It Works - -- The widget tracks the number of concurrent sessions running on the platform over a specified time period -- It presents the concurrency trends in a graph format, displaying the number of sessions in use and the number of sessions queued at each time interval -- You can hover over specific data points to view the exact number of sessions in use and queued at that particular time - -## Group Concurrency Trends - -The Group Concurrency Trends widget provides concurrency utilization data segmented by groups within your organization. This helps you understand how different teams or groups are utilizing the parallel test execution capacity. - -### Key Insights - -- View concurrency patterns for each group -- Compare resource utilization across different groups -- Identify which groups have higher queue times - -## SubOrg Concurrency Trends - -The SubOrg Concurrency Trends widget provides concurrency utilization data at the sub-organization level. This is useful for organizations with multiple sub-organizations to track and compare resource usage. - -### Key Insights - -- Monitor concurrency at sub-organization level -- Track queued vs in-use sessions per sub-organization -- Optimize resource allocation across sub-organizations - -## Custom Concurrency Trends - -The Custom Concurrency Trends widget visualizes queued and running test concurrency over time for custom KPIs. It highlights peak usage trends with clear visibility into maximum concurrency levels and their corresponding timestamps for the selected KPI. - -Unlike the standard Concurrency Trends widgets, Custom Concurrency Trends allows you to break down concurrency data by specific dimensions (KPIs), giving you granular insights into resource utilization patterns. - -### Key Features - -- **Stacked Visualization**: View concurrency data as stacked bar charts or stacked area charts -- **KPI-based Segmentation**: Analyze concurrency trends grouped by custom dimensions -- **Queued vs In-Use Metrics**: Track both queued and running sessions separately -- **Peak Usage Identification**: Easily identify periods of maximum resource utilization -- **Flexible Time Range**: Analyze trends across daily, weekly, or custom time periods - -### Supported KPIs - -| KPI | Status | -|-----|--------| -| Project Name | Available | -| Browser | Coming Soon | -| OS | Coming Soon | - -### How to Create a Dashboard with Custom Concurrency Trends - -**Step 1:** Log in to your LambdaTest account and navigate to **Insights** from the left sidebar. - -**Step 2:** Click on the **+ Create New** button and select **Pre-built Widgets**. - -**Step 3:** Select your product (Web Automation, App Automation, HyperExecute, Real Time, Real Device, Smart UI, Accessibility, or Test Manager). - -**Step 4:** From the widget list, select **Custom Concurrency Trends**, enter a Dashboard Name, and click **Create Dashboard**. - -### Configure Widget - -Click on the three-dot menu (⋯) on the widget and select **Configure** to customize the widget. - -#### Supported Filters - -| Filter Category | Filter Options | -|-----------------|----------------| -| General | Project Names, Build Names, Test Names | -| Status | Test execution status | -| Browser & OS | Browser, Operating System | -| Users | Users, Groups, Teams | -| SubOrgs | Sub-organizations | -| Tags | Build Tags, Test Tags | - -#### Display Options - -- **Graph Type**: Stacked Area or Stacked Bar (default) -- **Concurrency KPI**: Project Name -- **Select legends to show**: In Use, Queued (both selected by default) - -### Recommended Usage - -#### Viewing Multiple Dimensions (All Projects) - -When visualizing concurrency trends across all dimensions (e.g., all projects at once), display **either In-Use or Queued**, not both simultaneously. This provides a cleaner view for comparing usage patterns across different projects. - -**How to configure:** -1. Open **Configure Widget** > **Display Options** -2. Under **Select legends to show**, uncheck either "In Use" or "Queued" to show only one metric -3. Click **Apply Filters** - -#### Viewing Single Dimension (Specific Project) - -When focusing on a single dimension (e.g., one specific project), keep **both In-Use and Queued visible**. This gives you a complete picture of concurrency utilization for that specific project. - -**How to configure:** -1. Open **Configure Widget** > **General** -2. Filter by the specific **Project Name** you want to analyze -3. Under **Display Options**, ensure both "In Use" and "Queued" are selected in **Select legends to show** -4. Click **Apply Filters** - -**Coming Soon** -Drilldown functionality for Custom Concurrency Trends widget is coming soon. This will allow you to click through to individual test details from the aggregated concurrency view. - -**Minute-Level Granularity** -When you apply a time filter of **1 day or less**, the concurrency data is displayed at **minute-level granularity**. This is useful for pinpointing exact concurrency spikes, debugging queue buildups during specific test runs, or analyzing resource usage during dedicated time windows. - -## Value Proposition - -By analyzing the concurrency trends, you can make informed decisions about scaling your testing infrastructure, ensuring efficient resource allocation, and minimizing queuing times. These widgets empower you to strike the right balance between test execution speed and cost-effectiveness. - -**Use Case** -John is a QA Manager, and his team runs more than 50,000 Jobs in a month across various TestMu AI products like Web Automation, App Automation, and HyperExecute. - -John wants to know the duration of the tests kept in queue and the duration of tests put in running state. With the Concurrency Trends widgets he can easily track the duration and make a decision to optimize the TestMu AI plan currently subscribed by his team. Using Custom Concurrency Trends, he can further drill down into which projects are consuming the most concurrency and schedule test runs more efficiently. diff --git a/static/docs/analytics-modules-smartui-analytics.md b/static/docs/analytics-modules-smartui-analytics.md deleted file mode 100644 index c909436ec..000000000 --- a/static/docs/analytics-modules-smartui-analytics.md +++ /dev/null @@ -1,101 +0,0 @@ -# SmartUI Analytics - Visual Regression Testing Insights - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The SmartUI Analytics module provides a suite of widgets to help you analyze and optimize your visual regression testing efforts on the TestMu AI platform. By leveraging these widgets, you can gain valuable insights into the scale, coverage, and performance of your visual testing initiatives, enabling you to make data-driven decisions and ensure the highest quality of your application's visual integrity. - -The SmartUI Analytics module is currently in . If you have any feedback or suggestions, please feel free to reach out to us at [support@testmuai.com](mailto:support@testmuai.com). - -## SmartUI Snapshot - -The SmartUI Snapshot widget provides a comprehensive overview of the Visual Regression tests run on the TestMu AI platform. It offers key metrics to help you understand the scale and efficiency of your visual testing efforts. - -#### How It Works -- The widget calculates the total number of projects, builds run, and screenshots compared on the platform. -- It presents these metrics in a clear and concise format, giving you a quick snapshot of your visual testing activities. - -#### Value Proposition -By leveraging the SmartUI Snapshot widget, you can easily track the volume of your visual testing initiatives. This information helps you assess the coverage and completeness of your visual regression testing, ensuring that your application's visual integrity is thoroughly validated across different projects and builds. - -## Builds Summary - -The Builds Summary widget offers a detailed breakdown of the status distribution for the Visual Regression tests run on the TestMu AI platform. It provides valuable insights into the overall health and success rate of your visual testing builds. - -#### How It Works -- The widget categorizes the builds based on their status, such as Approved, Pending-Approval, Rejected, Partially-Rejected, and Error. -- It calculates the percentage distribution of builds across each status category. -- The widget presents the status distribution in a clear and intuitive format, allowing you to quickly assess the quality of your visual testing builds. - -#### Value Proposition -By analyzing the Builds Summary widget, you can gain a deeper understanding of the effectiveness and reliability of your visual testing process. The status distribution helps you identify any potential issues or bottlenecks in your visual testing workflow, enabling you to take corrective actions and optimize your testing efforts. - -## Builds Trends - -The Builds Trends widget provides a visual representation of the status trends for the Visual Regression tests run on the TestMu AI platform over time. It allows you to analyze the performance and consistency of your visual testing builds. - -- X-Axis: Represents the time intervals at which the build trends are measured. -- Y-Axis: Represents the number of builds categorized by their status (Current and Previous). - -#### How It Works -- The widget tracks the number of builds and their respective statuses over a specified time period. -- It presents the build trends in a graph format, displaying the number of builds for the current and previous time intervals. -- You can hover over specific data points to view the exact number of builds for each status at that particular time. - -#### Value Proposition -By examining the Builds Trends widget, you can identify patterns, fluctuations, or anomalies in your visual testing build status over time. This information helps you assess the stability and reliability of your visual testing process, allowing you to proactively address any emerging issues and ensure the consistent quality of your builds. - -## Screenshot Trends - -The Screenshot Trends widget provides a visual representation of the screenshot comparison trends for the Visual Regression tests run on the TestMu AI platform over time. It allows you to track the volume and progress of your visual testing screenshots. - -- X-Axis: Represents the time intervals at which the screenshot trends are measured. -- Y-Axis: Represents the number of screenshots compared (Current and Previous). - -#### How It Works -- The widget tracks the number of screenshots compared over a specified time period. -- It presents the screenshot trends in a graph format, displaying the number of screenshots compared for the current and previous time intervals. -- You can hover over specific data points to view the exact number of screenshots compared at that particular time. - -#### Value Proposition -By analyzing the Screenshot Trends widget, you can gain insights into the volume and velocity of your visual testing efforts. This information helps you understand the scale and coverage of your visual regression testing, allowing you to optimize resource allocation and ensure comprehensive visual validation across your application. - -## Screenshots Browsers/Device Coverage - -The Screenshots Browsers/Device Coverage widget provides a detailed breakdown of the browser and device coverage for the Visual Regression tests run on the TestMu AI platform. It helps you understand the distribution of your visual testing efforts across different browsers and devices. - -#### How It Works -- The widget calculates the total number of screenshots compared for each supported browser or device. -- It presents the coverage distribution in a visual format, such as a bar chart or pie chart, allowing you to easily identify the browsers and devices with the highest screenshot coverage. - -#### Value Proposition -By leveraging the Screenshots Browsers/Device Coverage widget, you can ensure that your visual testing efforts are well-distributed across different browsers and devices. This information helps you identify any gaps in your visual testing coverage and make informed decisions to optimize your testing strategy, ensuring a consistent visual experience for your users across various platforms. - -## Screenshots Viewport Coverage - -The Screenshots Viewport Coverage widget provides a comprehensive overview of the viewport coverage for the Visual Regression tests run on the TestMu AI platform. It helps you understand the distribution of your visual testing efforts across different viewport sizes. - -#### How It Works -- The widget calculates the total number of screenshots compared for each supported viewport size. -- It presents the coverage distribution in a clear and concise format, allowing you to easily identify the viewport sizes with the highest screenshot coverage. - -#### Value Proposition -By analyzing the Screenshots Viewport Coverage widget, you can ensure that your visual testing efforts are well-distributed across different viewport sizes. This information helps you identify any gaps in your visual testing coverage and make informed decisions to optimize your testing strategy, ensuring a consistent visual experience for your users across various screen sizes and resolutions. - -## Build Status Ratio - -The Build Status Ratio widget provides a visual representation of the status ratio trends for the Visual Regression tests run on the TestMu AI platform over time. It allows you to track the proportion of builds with different statuses and monitor the overall health of your visual testing process. - -- X-Axis: Represents the time intervals at which the status ratio trends are measured. -- Y-Axis: Represents the ratio of builds categorized by their status (Approved and Pending-Approval). - -#### How It Works -- The widget tracks the ratio of builds with different statuses over a specified time period. -- It presents the status ratio trends in a graph format, displaying the proportion of approved and pending-approval builds at each time interval. -- You can hover over specific data points to view the exact ratio of builds for each status at that particular time. - -#### Value Proposition -By examining the Build Status Ratio widget, you can gain insights into the overall quality and efficiency of your visual testing process. The status ratio trends help you identify any shifts or imbalances in the approval process, allowing you to take corrective actions and optimize your visual testing workflow to ensure a higher proportion of approved builds. - -The SmartUI Analytics widgets provide a comprehensive suite of tools to analyze and optimize your visual regression testing efforts on the TestMu AI platform. By leveraging these widgets, you can gain valuable insights into the scale, coverage, and performance of your visual testing initiatives, enabling you to make data-driven decisions and ensure the highest quality of your application's visual integrity. - -Unlock the full potential of SmartUI Analytics today and take your visual regression testing to the next level! diff --git a/static/docs/analytics-modules-test-intelligence-command-logs-analytics.md b/static/docs/analytics-modules-test-intelligence-command-logs-analytics.md deleted file mode 100644 index 452d2c8b7..000000000 --- a/static/docs/analytics-modules-test-intelligence-command-logs-analytics.md +++ /dev/null @@ -1,85 +0,0 @@ -# Command Logs Analytics with Test Intelligence - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -### Introduction to AI Native Command Logs Analytics - -TestMu AI Analytics help you intelligently analyze your test execution steps on the platform. It provides you with the insights on the errors that are causing your test runs to fail. - -### Prerequisites for using Command Logs Analytics -The following are the steps to use Command Logs Analytics: -1. You should have a TestMu AI account. If you don't have one, you can sign up here. -2. You should have access to the `Test Intelligence` subscription plan. You can join the waitlist [here](https://www.testmuai.com/test-intelligence). -3. Go to `Analytics` from the left navigation menu and create a new Dashboard by clicking on the `Create New Dashboard` button. -4. Select the `Create Custom Dashboard` option from the `Create New Dashboard` dropdown. -5. Select the `Command Logs` option from the `Analytics Modules` section. -6. Click on `Add` for the widgets required to your dashboard. -7. Input your `Dashboard Name` and click on `Create Dashboard` to create your dashboard. - -### **Command Status Summary Widget** - - **About the Widget** - - The Command Logs Status Summary Widget offers a summary of the response statuses for Selenium commands executed on the platform. This is visualized using a tree map, giving a quick and clear overview of the system's status. - - **Widget Simulation** - - A tree map is shown with each rectangle representing different command response codes. The size of each rectangle corresponds to the number of commands with that response code. This gives you a visual representation of the overall system health at a glance. - - **Use Cases** - 1. Quickly identifying the system's health status - 2. Monitoring status codes over a specific timestamps - 3. Tracking system performance and identifying potential issues - 4. Analyzing command logs status distribution - 5. Filtering and focusing on specific response codes for further investigation - -### **Command Error Status Trends Widget** - - **About the Widget** - - The Command Logs Status Trends Widget provides a time-series representation of command log HTTP response statuses. It groups tests with similar response statuses for a clearer view of the system's health and performance trends. - - **Widget Simulation** - - A line or bar chart is displayed with the x-axis representing the timestamp of the tests run and the y-axis representing the total number of unique errors in tests with command logs of respective response statuses. - - **Use Cases** - 1. Tracking response status trends over time - 2. Identifying periods of increased error response codes - 3. Evaluating system health based on response status trends - 4. Comparing system performance over different periods - 5. Analyzing the frequency of retries or error status codes over time - -### **Command Type Trends Widget** - - **About the Widget** - - The Command Logs Type Trends Widget shows the trends of endpoint usage over time by displaying the count of commands for each command name mapped to the endpoint/request path. It helps to identify potential system bottlenecks and optimization opportunities. - - **Widget Simulation** - - A horizontal bar chart is shown with the y-axis representing the W3C command name and the x-axis showing the count of commands. The command response codes are stacked on each bar for further granularity. - - **Use Cases** - 1. Identifying the most frequently used command types - 2. Discovering potential bottlenecks in specific command types - 3. Comparing usage of different command types - 4. Tracking changes in command type usage over time - 5. Correlating command type usage with system performance - -### **Command Error Messages Categorization Widget** - - **About the Widget** - - The Command Logs Error Messages Categorization Widget categorizes error messages from command logs, allowing users to identify common issues and prioritize bug fixes. - - **Widget Simulation** - - A table displays unique error messages, occurrences dots, the percentage of occurrences, the timestamp of the last error message, and tests with the error message command logs. - - **Use Cases** - 6. Identifying common error messages for prioritizing bug fixes - 7. Tracking the frequency of specific error messages - 8. Analyzing the proportion of error messages in the total commands - 9. Finding correlations between error message occurrences and system performance - 10. Inspecting tests with specific error messages for debugging diff --git a/static/docs/analytics-modules-test-intelligence-flaky-test-analytics.md b/static/docs/analytics-modules-test-intelligence-flaky-test-analytics.md deleted file mode 100644 index a8d06195c..000000000 --- a/static/docs/analytics-modules-test-intelligence-flaky-test-analytics.md +++ /dev/null @@ -1,43 +0,0 @@ -# Flaky Test Analytics with Test Intelligence - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The best way to analyze your flaky tests is to use Test Intelligence. Test Intelligence is a machine learning-based algorithm that helps you identify flaky tests and get insights on the flaky tests that are causing your test runs to fail. By observing the historical test runs, Test Intelligence can identify the flaky tests and provide you with the insights to help you fix them. - -**Get Flaky Alerts on Slack** -Don't want to keep checking the dashboard? If you have the [Slack Integration](/docs/slack-integration/) enabled, turn on the **Flaky Test Messages** preference to receive a Slack alert the **first time** TestMu AI detects a test as flaky, including the test name, build, and flake rate. Alerts are opt-in and off by default. - -## Flakiness Trends -The Flakiness Trends widget allows the QA teams to analyze the trends of the flaky tests executed on the platform categorized into Passed, Failed, and Flaky which can easily be filtered with the legends added at the top of the graph. - -The graph shows the time series of the flaky tests executed on the platform. The QA teams can easily filter the graph with the legends added at the top of the graph. The QA teams can also hover over the graph to get the exact count of the flaky tests executed on the platform. - -The following are the use cases of the Flakiness Trends widget: -- The QA teams can check the decrease or increase in the flaky tests executed on the platform over time. -- Easily navigate the to the test logs and fix the issues identified on the test logs. - -## Flakiness Severity Summary -Analyze the flaky tests severity summary which is broadly categorized to High, Medium and Low depending on the `Flake Rate` of the tests. The following are the distribution of for the severity: - -- `High` - If the flake rate is more than `75%` in compared to the previous test runs. -- `Medium` - If the flake rate is between `50%` to `75%` in compared to the previous test runs. -- `Low` - If the flake rate is less than `50%` in compared to the previous test runs. - -**FLAKY RATE** -The flake rate is calculated on the basis of the total number of flaky tests executed on the platform in compared to the previous test runs. - -For example, if the total number of flaky tests executed on the platform is `100` and the total number of flaky tests executed on the platform in compared to the previous test runs is `50` then the flake rate will be `50%`. - -The following are the use cases of the Flakiness Severity Summary widget: -- The QA teams can easily identify the flaky tests that are causing the test runs to fail on the basis of the severity. -- The QA can prioritize on the basis of the severity and fix the flaky tests that are causing the test runs to fail. - -## Flakiness Tests - Test Group View -The Flakiness Tests - Test Group View widget allows the QA teams to analyze the flaky tests on the basis of the test groups. The QA teams can easily analyze the flaky tests on the basis of the test groups and get insights on the flaky tests that are causing the test runs to fail. - -The following are the use cases of the Flakiness Tests - Test Group View widget: -- The QA teams can easily analyze the flaky tests on the basis of the test groups. -- The QA teams can identify the % of occurrence of the flaky tests on the basis of each test group. - -**Flaky Rate Calculation** -To read more about the Flaky Tests Analytics, you can refer the documentation at [Flaky Tests - Test Intelligence](/docs/test-intelligence-flakiness-test-detection/) diff --git a/static/docs/analytics-modules-test-manager-widgets.md b/static/docs/analytics-modules-test-manager-widgets.md deleted file mode 100644 index bd751fd94..000000000 --- a/static/docs/analytics-modules-test-manager-widgets.md +++ /dev/null @@ -1,174 +0,0 @@ -# Test Manager Dashboard Widgets - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI Test Manager Dashboard widgets provide comprehensive insights into your test automation or manual efforts. These widgets help you optimize your testing efforts by providing detailed insights into your test execution, test case insights, and more. - -The Test Manager Widgets module is currently in . If you have any feedback or suggestions, please feel free to reach out to us at [support@testmuai.com](mailto:support@testmuai.com). - -**Filtering Widgets** -Every widget supports configurable filters. Click the **three-dot menu** on any widget to apply filters and narrow down the displayed results based on your specific criteria. - -**Drill Down into Data** -All widgets support drill-down. Click on any data point in a chart (such as a segment in a pie chart, a bar in a bar chart, or a status value) to view the underlying list of associated records (e.g., test cases, test runs, or issues) that make up that data point. - -## Test Cases Summary - -The Test Cases Summary widget displays the total count of test cases and the split between **Automated** and **Manual** test cases. The classification is determined by the **Automation Status** field on each test case. If set to *Automated*, it is counted under Automated; all others fall into the Manual category. - -### How it works? - -* View the total number of unique test cases created in your project. -* See the breakdown between Automated and Manual test cases based on the Automation Status field. - -### Value Proposition - -This widget gives you a quick snapshot of your test case inventory and the balance between automated and manual coverage, helping you plan your testing strategy. - -## Test Cases Trend - -The Test Cases Trend widget tracks the number of test cases created over a period of time, helping you visualize how your test suite is growing. - -### How it works? - -* Displays a trend line of test cases created over the selected time period. -* Use this to monitor the pace of test case creation across your team. - -### Value Proposition - -This widget helps you track how your test suite evolves over time, identify periods of high or low activity, and ensure consistent test coverage as your product grows. - -## Test Runs Summary - -The Test Runs Summary widget provides an overall summary of your test runs, with the data split by test instance statuses and their counts. - -### How it works? - -* Displays the total number of test runs and the breakdown of test instances by status (Not Started, Passed, Failed, Skipped, and custom statuses). -* Quickly identify test runs with the highest number of failed or pending instances. - -### Value Proposition - -This widget gives you a high-level view of test run health across your project, helping you prioritize runs that need attention and track overall execution progress. - -## Milestone Trends - -The Milestone Trends widget displays a stacked bar chart of milestones, with each bar broken down by test instance statuses from the associated test runs. - -### How it works? - -* Each bar represents a milestone, with stacked segments showing the count of test instances by status. -* Supported statuses include **Not Started**, **Passed**, **Failed**, **Skipped**, as well as any **custom statuses** created for your manual test runs. -* Filter the data by date range to analyze milestone performance over specific time periods. - -### Value Proposition - -This widget helps you quickly assess milestone health, identify milestones with high failure rates, and monitor release readiness. By tracking status trends across milestones, you can spot regressions early and make data-driven decisions about testing effort. - -## Issue Summary - -The Issue Summary widget shows the trend of issues and requirements linked to test cases or raised during test runs in Test Manager. - -### How it works? - -* Tracks issues linked to test cases as well as issues raised directly within test runs. -* Visualize whether issues are increasing or decreasing over time. - -### Value Proposition - -This widget helps you monitor the volume and trend of issues across your testing efforts, enabling you to identify patterns and prioritize resolution. - -## Test Cases Types - -The Test Cases Types widget displays a pie chart of test cases split by their **Type** field (e.g., Regression, Smoke, Sanity, or any custom type defined in your project). - -### How it works? - -* Each segment of the pie chart represents a test case type, based on the **Type** field of each test case. -* Quickly see the distribution of test case types across your project. - -### Value Proposition - -This widget helps you understand the composition of your test suite and ensure the right balance of test types (Regression, Smoke, Sanity, etc.) for effective coverage. - -## Execution Progress - -The Execution Progress widget displays overall test execution progress as a percentage on a visual gauge, alongside a breakdown of total, executed, and remaining test run instances. - -### How it works? - -* The gauge shows the completion percentage, computed as **(Total test run instances − Not Started instances) / Total test run instances**. -* Any test instance in a status other than *Not Started* (such as Passed, Failed, Skipped, Blocked, or custom statuses) is counted under **Executed**. -* A side panel next to the gauge displays three counters: - * **Total**: the total number of test run instances in scope. - * **Executed**: instances that have moved out of the *Not Started* status. - * **Remaining**: instances still in the *Not Started* status. -* Apply filters from the three-dot menu to scope the widget to a specific test run, phase, or stage. - -### Value Proposition - -This widget gives you a single at-a-glance metric for test execution progress, making it easy to track how close a release, milestone, or phase is to completion and to communicate status to stakeholders without drilling into individual runs. - -## Test Execution Burndown - -The Test Execution Burndown widget tracks how remaining test executions trend down over the duration of a test run, and surfaces a rich footer strip of delivery metrics so you can forecast whether execution will complete on time. - -### How it works? - -* The **X-axis** represents dates across the selected execution window, and the **Y-axis** represents the number of remaining test run instances (instances still in *Not Started* status). -* Three trend lines are plotted: - * **Actual** (solid blue): remaining *Not Started* instances per day, reflecting real execution progress up to today. - * **Expected** (dashed grey): an ideal linear burndown from the total instance count on Day 0 down to zero on the final day. - * **Projected** (dashed blue): a forward-looking extrapolation from today to the end date, based on your current velocity, so you can see where execution will land if the current pace holds. -* A **Today** marker highlights the current date on the chart, making it easy to see where the Actual line hands off to the Projected line relative to the Expected ideal. -* Hovering over any point on the chart opens a tooltip showing, for that day: - * **Remaining**: remaining instances on that date. - * **Ideal**: the expected remaining count for that date. - * **Gap**: how far ahead or behind the ideal line you are. - * **Executed today**: the number of instances executed on that day. - * **7-day avg**: the trailing 7-day execution velocity. -* The **footer strip** below the chart summarises execution health at a glance: **Total**, **Executed**, **Remaining**, **Complete** (percentage), **Status** (ahead or behind by N instances), **Velocity** (instances/day), projected **Finish** date, and the **Pace needed** (instances/day required to hit the target). -* Use the **date range picker** in the widget header to change the execution window. Use the **filter icon** (next to the *Test Manager* badge) or the three-dot menu to filter by test run, phase, or stage. - -### Drill down into remaining instances - -Clicking any point on the Actual line opens a drill-down modal titled **Test Execution Burndown**, which lists every test run instance that was still *Remaining* as of that date. From the modal you can: - -* See the exact count of remaining instances for the selected date (e.g. *Remaining as of Mar 14: 1000*). -* Adjust the date range and apply additional filters: **Project**, **Test Run Instance Name**, and **Milestone Name**. -* Browse the paginated list of instances with their current status, project, and last-updated timestamp. -* **Export** the list for sharing or offline analysis. - -### Value Proposition - -By comparing the actual burndown against the expected line and watching the footer metrics, you can quickly see whether execution is ahead, on track, or slipping, enabling early course correction, better capacity planning, and more reliable release predictions. The drill-down makes it easy to go from a high-level slip directly to the specific instances that are holding the run back. - -## Defects by Severity - -The Defects by Severity widget displays a bar chart of Jira defects linked to your test cases and test runs, grouped by their severity level. - -### How it works? - -* Each bar represents a severity level (**Critical**, **High**, **Medium**, and **Low**) with its value showing the count of linked Jira defects at that severity. -* Defects are sourced from Jira issues linked to test cases or raised during test runs in Test Manager. -* Click any bar to drill down into the underlying list of issues for that severity. -* Apply filters from the three-dot menu to scope the widget to a specific test run, phase, or stage. - -### Value Proposition - -This widget helps you assess release risk at a glance by surfacing how many high-impact defects are outstanding. By focusing on Critical and High severity bars first, you can prioritize the issues most likely to block a release. - -## Tester Assignment - -The Tester Assignment widget displays a horizontal stacked bar chart with one bar per assigned tester, showing the workload and execution progress of each tester side by side. - -### How it works? - -* Each bar represents a tester, and its length reflects the total number of test run instances assigned to them. -* Each bar is broken down into stacked segments by test instance status: **Not Started**, **Passed**, **Failed**, **Skipped**, as well as any **custom statuses** created for your manual test runs. -* Click any segment to drill down into the underlying list of test run instances for that tester and status. -* Apply filters from the three-dot menu to scope the widget to a specific test run, phase, or stage. - -### Value Proposition - -This widget lets you compare workload and execution progress across testers at a glance, making it easy to spot uneven assignment, identify testers who are behind, and rebalance effort to keep a release on track. diff --git a/static/docs/analytics-overview.md b/static/docs/analytics-overview.md deleted file mode 100644 index 87b9aa7f3..000000000 --- a/static/docs/analytics-overview.md +++ /dev/null @@ -1,79 +0,0 @@ -# TestMu AI - Test Insights for visualizing your tests runs - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -**Explore AI Dashboard CoPilot** - -We have recently launched an AI Dashboard CoPilot feature that provides intelligent recommendations, insights, and predictions tailored to your specific data. - -To know more about this feature, refer to the [AI Dashboard CoPilot](/docs/analytics-dashboard-copilot/) documentation. - -## What is Insights? - -Insights in TestMu AI allows the QA managers and executives to efficiently track the tests conducted on the platform and get deep-level insights with the numerous widgets that the product provides. - -## Why is Insights needed? - -When managing multiple product lines it gets arduous for the QA managers to track the progress and quality of tests running on the platform. - -Insights effectively allows these managers to track and improve the tests at one sight with the help of the highly customizable dashboards available with Insights. - -## Insights Modules - - Dashboards - Explore various dashboards that provide a comprehensive view of your testing activities, enabling you to monitor performance, identify issues, and make data-driven decisions. - - Concurrency Usage - Learn how to to get an overview of the TestMu AI resources being utilized by your teams. - - Overview - Learn how to get an overview of the tests running on the TestMu AI. - - Error Insights - Learn how to get all the insights into the test errors on the TestMu AI. - - Flaky Tests AI - Monitor all the Flaky tests run on the platform and review the quality of the feature on your SMOKE and Regression Jobs. - - Test Case Insights - Get insights for `describe()` block level for your test frameworks in HyperExecute. Monitor the quality metrics at each test case. - - API Test Insights - Get API test time-series insights using Allure report integration with your test suite and run it on HyperExecute. - - Sub Organization Insights   - Monitor and allocate the right resources required for your organizations with advanced insights. - - Private Real Device Insights   - Gain detailed insights for ensuring optimal performance and resource utilization for your dedicated devices. - - Usage Report   - Get detailed insights into your product usage and optimize your testing efforts. - -## Product Specific Widgets - - Web Automation Widgets - Gain valuable insights into your web automation testing efforts with comprehensive analytics and metrics. - - App Automation Widgets - Unlock the power of analytics to optimize your mobile app testing process and improve app quality. - - HyperExecute Widgets - Leverage advanced analytics to streamline your HyperExecute testing pipeline and boost testing efficiency. - - Realtime Widgets   - Get real-time insights into your testing activities and make data-driven decisions on the fly. - - Real-Device Widgets - Analyze your real device testing performance and identify optimization opportunities. - - SmartUI Widgets   - Gain actionable insights into your SmartUI testing efforts and ensure exceptional user experiences. - - Accessibility Widgets   - Ensure your application or website meets WCAG standards with comprehensive accessibility insights. - - Test Manager Widgets   - Manage your project-level tests comprehensively, track the progress of test suite development with detailed insights. - -The features are available for limited users only. If you wish to try them out, please reach out to our [24/7 customer support](mailto:support@testmuai.com). diff --git a/static/docs/analytics-project-report.md b/static/docs/analytics-project-report.md deleted file mode 100644 index 3a98bfd79..000000000 --- a/static/docs/analytics-project-report.md +++ /dev/null @@ -1,122 +0,0 @@ -# Project - Reports and C-Suite Insights - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The Project Analytics Dashboard is not just a tool for QA teams, it's a strategic asset for C-Suite executives. With real-time, high-level insights, it empowers leadership to make data-driven decisions, monitor release readiness, and communicate quality metrics to stakeholders and boards with confidence. - -## Introduction - -The Project Analytics Dashboard is a comprehensive platform designed to provide deep insights into your test automation projects. It consolidates key metrics, trends, and health indicators into a single, interactive view, empowering QA teams, managers, and especially C-Suite stakeholders to make informed decisions, optimize test coverage, and ensure release readiness. This guide covers all dashboard features, widgets, workflows, and best practices to help you extract maximum value from your analytics data. - -## Prerequisites - -- **Active TestMu AI Account:** Some advanced features may require a paid subscription. -- **Project Data:** At least one project with executed tests and available data. -- **Dashboard Setup:** Dashboards and widgets configured for your projects. -- **Permissions:** Appropriate access to analytics and export features. - -## Accessing the Project Analytics Dashboard - -1. **Login** to your TestMu AI account. -2. Navigate to the **Insights** section. -3. Select or create a project dashboard. -4. Use **Date Range** and **Project Name** filters for targeted analysis. - -1. **Login** with your executive credentials. -2. Go to **Insights** and select the relevant project or portfolio dashboard. -3. Use the **Executive View** (if enabled) for high-level summaries and export-ready reports. -4. Leverage the **Export** and **Share** features to present data to the board or stakeholders. - -## Feature Overview & Widgets - -The dashboard’s visualizations and health indicators are designed for rapid executive consumption, enabling quick identification of risk, ROI, and release readiness without deep technical dives. - -### 1. Testing Velocity -- **Test Volume:** Total tests executed in the selected period. Useful for demonstrating testing scale and resource utilization to leadership. -- **Test Velocity:** Average daily test execution, indicating operational efficiency. -- **Bar Chart Visualization:** Instantly communicates trends and bottlenecks to non-technical audiences. - -### 2. Testing Distribution -- **Browser & Device Coverage:** Showcases breadth of testing, supporting claims of cross-platform reliability in executive reports. -- **Distribution Chart:** Visualizes test spread, helping C-Suite members assess risk exposure across environments. - -### 3. Test Case Quality Trends -- **Quality Trend Charts:** Pass/fail/anomaly rates for web, app, HyperExecute, and Smart UI. These trends are essential for tracking quality improvements and highlighting risk areas in board meetings. - -### 4. Browser Compatibility & Device Performance -- **Compatibility Charts:** Quickly identify problematic browsers/devices, supporting investment decisions for additional coverage or optimization. - -### 5. Projects Health -- **Project Cards:** Summarize health, success rates, and instability. Color coding and trend indicators make it easy for executives to spot at-risk projects and track progress over time. - -## Step-by-Step Usage - -1. Set filters for date and project. -2. Review velocity and distribution widgets. -3. Analyze quality trends and compatibility. -4. Investigate project health cards for instability. -5. Export/share data as needed. - -1. Use the **Executive View** for a high-level summary. -2. Focus on color-coded health cards and trend arrows for quick risk assessment. -3. Download/export charts and summaries for board presentations. -4. Leverage the dashboard’s visualizations to communicate ROI, risk, and release readiness to stakeholders. - -## Use Cases - -- **Executive Reporting:** - - Instantly export high-level summaries and visualizations for board decks and investor updates. - - Use trend charts to demonstrate continuous improvement or highlight areas needing investment. - - Communicate risk and release readiness with color-coded health indicators. -- **Quality Assurance Optimization:** - - Identify flaky or unstable tests, monitor automation stability, and improve test coverage. -- **Release Readiness Assessment:** - - Use project health and quality trends to determine if your application is ready for release. -- **Cross-Browser/Device Testing:** - - Ensure a consistent user experience by analyzing browser and device compatibility. -- **Continuous Improvement:** - - Track trends and anomalies over time to drive ongoing improvements. -- **Stakeholder Communication:** - - Share dashboard links or exports with external teams, clients, or partners. - -## Best Practices - -- Schedule regular reviews of the dashboard before major releases or board meetings. -- Use exported charts and summaries to support strategic decisions and communicate with non-technical stakeholders. -- Focus on trend indicators and color-coded health cards for rapid risk assessment. - -- **Regular Monitoring:** Review the dashboard frequently to catch regressions early. -- **Targeted Analysis:** Use filters for focused insights. -- **Investigate Instability:** Prioritize projects with low success rates or high instability. -- **Optimize Coverage:** Ensure all critical environments are covered. -- **Leverage Exports:** Use exports for deeper analysis or reporting. - -## FAQs - -**Q: Can I customize the dashboard layout?** - -A: No, you can rearrange the widgets but not customize the layout. If you need to add or remove widgets, please create a new dashboard. - -**Q: How do I export analytics data?** - -A: Use the export/download option in the dashboard (feature availability may depend on your plan). - -**Q: Who can I contact for support?** - -A: Reach out to [support@testmuai.com](mailto:support@testmuai.com) for assistance. - -**Q: How can I use this dashboard for board reporting?** - -A: Export high-level summaries and visualizations directly from the dashboard for use in board decks and executive presentations. - -**Q: What are the key metrics C-Suite should focus on?** - -A: Success rates, trend indicators, and color-coded health cards provide the most actionable insights for executive decision-making. - -**Q: Can I share dashboard views with other executives?** - -A: Yes, use the share/export features to distribute insights across your leadership team. - -## Support and Assistance - -For further details, troubleshooting, or personalized guidance, refer to the respective documentation sections or contact TestMu AI support. Our team is available 24/7 to help you maximize the value of your Project Analytics Dashboard and ensure your testing initiatives are successful. diff --git a/static/docs/analytics-report-scheduling.md b/static/docs/analytics-report-scheduling.md deleted file mode 100644 index 1bf16d8f3..000000000 --- a/static/docs/analytics-report-scheduling.md +++ /dev/null @@ -1,93 +0,0 @@ -# Schedule Analytics Widget Reports - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Analytics report scheduling allows you to customize the delivery time for your dashboard widget reports. Instead of receiving reports at a fixed default time, you can choose a specific time that aligns with your team's workflow. - -## Overview - -With the customizable scheduling feature, you can: - -- Set a preferred delivery time for each dashboard's email reports. -- Configure notification frequency (Daily, Weekly, or Monthly). -- Choose the data range included in each report. -- Manage email recipients for scheduled reports. - -## Prerequisites - -- You must be logged in to your TestMu AI account. -- You must have at least one [Insights dashboard](/docs/analytics-create-dashboard/) created. -- You must be the dashboard creator or an admin user to modify scheduling settings. - -## Schedule a Report - -### Step 1: Open Dashboard Settings - -1. Navigate to your Insights dashboard. -2. Click the **Settings** icon in the dashboard toolbar to open the **Dashboard Settings** panel. - -### Step 2: Enable Email Notifications - -In the **Dashboard Settings** panel, expand the **Email Notifications** section and toggle the switch to enable notifications for the dashboard. - -### Step 3: Set the Delivery Schedule - -1. Under **Frequency**, select your preferred schedule from the dropdown: - - **Daily**: Receive reports every day at your chosen time. - - **Weekly**: Receive reports once a week. - - **Monthly**: Receive reports once a month. - -2. Under **Time**, select the hour from the dropdown (e.g., `9:00`). - -3. Under **Period**, select **AM** or **PM** from the dropdown. - -### Step 4: Configure the Date Range - -Select the date range for the data included in each report: - -- **Default**: Uses the platform's default date range. -- **Custom**: Specify a custom number of days. For example, setting this to `7` includes data from the last 7 days in each report. - -### Step 5: Add Recipients - -1. Under **Emails**, enter the email address in the **Enter Email address** field. -2. Click **+ Add** to include the recipient. -3. Repeat to add multiple recipients. - -### Step 6: Save Your Settings - -Click **Update** to save your scheduling preferences. The changes take effect immediately. - -Each dashboard has its own independent scheduling settings. You can configure different delivery times and recipients for different dashboards to match each team's needs. - -## Update an Existing Schedule - -1. Navigate to the dashboard whose schedule you want to modify. -2. Open the **Dashboard Settings** panel. -3. Adjust the frequency, time, period, date range, or recipients as needed. -4. Click **Update** to save your changes. - -When you update the delivery time for a dashboard, the new schedule applies only to that dashboard. Other dashboards retain their existing schedules. - -## Disable Scheduled Reports - -To stop receiving scheduled reports for a dashboard: - -1. Open the **Dashboard Settings** panel. -2. Toggle the **Email Notifications** switch to the off position. -3. Click **Update** to confirm. - -The notification preferences (time, recipients) are preserved so you can re-enable them later without reconfiguring. - -## Common Issues - -| Issue | Solution | -|---|---| -| Email notification toggle appears disabled | Ensure you have saved the dashboard after creation. Try refreshing the page and toggling again. | -| Delivery time reverts after clicking Update | Verify you selected the time and period before clicking Update. If the issue persists, refresh the page and try again. | - -## Related Docs - -- [Create Your Custom Dashboard](/docs/analytics-create-dashboard/) -- [Dashboard Settings](/docs/analytics-dashboard-settings/) -- [Dashboard Features](/docs/analytics-dashboard-features/) diff --git a/static/docs/analytics-smart-tags-test-intelligence.md b/static/docs/analytics-smart-tags-test-intelligence.md deleted file mode 100644 index 38134efa1..000000000 --- a/static/docs/analytics-smart-tags-test-intelligence.md +++ /dev/null @@ -1,43 +0,0 @@ -# Smart Tags - Test Intelligence - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Smart Tags in TestMu AI Analytics help you analyze your test automation execution faster. With Smart Tags, you can categorize your tests based on different parameters and get insights on your test execution. This feature helps you identify patterns, trends, and anomalies in your test results, making it easier to optimize your test automation strategy. - -## Pre-requisites For Smart Tags: - -1. You should have an active TestMu AI account. -2. You should run at least 10 tests on the TestMu AI automation platform. -3. This feature is available for users with the HyperExecute or App / Web Automation subscription plan. -4. You should have access to the TestMu AI Analytics platform. -5. You can see the smart tags on creating a dashboard in the Insights section. -6. Click on the `Create New` button to create a new dashboard. -7. Choose any of the available widgets and add them to the dashboard. -8. Click on the widget drill down to see the smart tags. - -## What are Smart Tags? - -Smart Tags are labels that are assigned by the system depending on the execution patterns of your tests. These tags are currently of the following types: - -* **Flaky:** Tests that have inconsistent results across multiple executions. -* **Always Failing:** Tests that have consistently failed across multiple executions. -* **New Failures:** Tests that have started failing recently. -* **Performance Anomalies:** Tests that have shown a significant change in performance. - -## Flaky Tests: - -Flaky tests are tests that have inconsistent results across multiple executions. These tests may pass or fail randomly, making it difficult to identify the root cause of the failure. By categorizing tests as flaky, you can focus on fixing these tests to improve the reliability of your test suite. - -For more information on Flaky Tests, you can refer to our [Flaky Tests Guide](/docs/test-intelligence-flakiness-test-detection/). - -## Always Failing Tests: - -Always failing tests are tests that have consistently failed across multiple executions. These tests are critical as they indicate a persistent issue in your application or test automation scripts. By identifying and fixing these tests, you can ensure the stability and reliability of your test suite. - -## New Failures: - -New failures are tests that have started failing recently. These tests may indicate a recent change in your application or test environment that has caused the failures. By monitoring new failures, you can quickly identify and address issues before they impact your test suite. - -## Performance Anomalies: - -Performance anomalies are tests that have shown a significant change in performance. These tests may have suddenly become slower or faster than usual, indicating a potential issue in your application or test environment. By tracking performance anomalies, you can investigate the root cause of the performance change and optimize your test suite accordingly. diff --git a/static/docs/analytics-sub-organization-widgets.md b/static/docs/analytics-sub-organization-widgets.md deleted file mode 100644 index 702d6004a..000000000 --- a/static/docs/analytics-sub-organization-widgets.md +++ /dev/null @@ -1,48 +0,0 @@ -# Sub Organizations Insights - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Monitor and Analyze Sub-Organizations with Insights - -Sub Organization Widgets provide a comprehensive overview of the test health and trends of your sub-organizations. The module helps you to monitor and analyze the test concurrency utilization, browser distribution, OS distribution, and device coverage of your sub-organizations. This helps you to identify the bottlenecks in the testing process and optimize the testing strategy for your sub-organizations. - -The features are available for limited users only. If you wish to try them out, please reach out to our [24/7 customer support](mailto:support@testmuai.com). - -## How to use Sub-Organization Widgets? -To use the Sub-Organization Widgets, follow the below steps: -- Navigate to the `Insights` module from the left navigation menu. -- Create a new `Custom` dashboard or select an existing dashboard. -- Click on the product your want to see the Sub Organization Widgets. -- Select the `Sub Organization` widget from the list of available widgets. -- Click on the `Add Widget` button for existing dashboard and Click on the `Create Dashboard` button for new dashboard. - -**Please Note** -The Sub Organization Widgets are available for the feature enabled users only. You need to have access to admin control to use these widgets. Please reach out to our [24/7 customer support](mailto:support@testmuai.com) to enable these features. - -## What are the Key Insights Provided by Sub-Organization Widgets? -The `Sub-Organization` widget provides the following key insights: -- **Sub Organization Concurrency Trends**: The widget provides the concurrency utilization of your sub-organizations. This helps you to identify the concurrency utilization of your sub-organizations and optimize the concurrency utilization for your sub-organizations. -- **Sub Organization Trends**: The widget provides a trend of the tests run of your sub-organizations. This helps you to analyze the test trends and identify the test health of your sub-organizations. -- **Sub Organization Browser/OS Distribution**: The widget provides a distribution of the browsers and OS used by your sub-organizations. This helps you to identify the popular browsers and OS used by your sub-organizations. - -## Sub Organization Concurrency Trends - -The `Sub Organization Concurrency Trends` widget provides the concurrency utilization of your sub-organizations. This helps you to identify the concurrency utilization of your sub-organizations and optimize the concurrency utilization for your sub-organizations. The widget provides the following key insights: -- **Concurrency Trends**: The widget provides the concurrency trends of your sub-organizations. This helps you to analyze the concurrency trends and identify the bottlenecks in the concurrency utilization of your sub-organizations. -- **Concurrency Distribution**: The widget provides the distribution of the concurrency utilization of your sub-organizations. This helps you to identify the popular concurrency utilization of your sub-organizations. -- **Concurrency Utilization Ratio**: The widget provides the concurrency utilization ratio of your sub-organizations. This helps you to identify the concurrency utilization ratio of your sub-organizations and optimize the concurrency utilization for your sub-organizations. - -## Sub Organization Test Trends - -The `Sub Organization Test Trends` widget provides a trend of the tests run of your sub-organizations. This helps you to analyze the test trends and identify the test health of your sub-organizations. The widget provides the following key insights: -- **Test Trends**: The widget provides the test trends of your sub-organizations. The total number of tests run, passed, failed and other statues are displayed in the widget. This helps you to analyze the test trends and identify the testing scope of your sub-organizations. -- **Testing Pattern Analysis**: The widget provides the testing pattern analysis of your sub-organizations. This helps you to analyze the testing pattern and identify the testing scope of your sub-organizations. - -## Sub Organization Browser/OS Distribution -The `Sub Organization Browser/OS Distribution` widget provides a distribution of the browsers and OS used by your sub-organizations. This helps you to identify the popular browsers and OS used by your sub-organizations. The widget provides the following key insights: -- **Browser Distribution**: The widget provides the distribution of the browsers used by your sub-organizations. This helps you to identify the popular browsers used by your sub-organizations. -- **OS Distribution**: The widget provides the distribution of the OS used by your sub-organizations. This helps you to identify the popular OS used by your sub-organizations. -- **Device Coverage**: The widget provides the device coverage of your sub-organizations. This helps you to identify the device coverage of your sub-organizations and optimize the device coverage for your sub-organizations. - -## Support and Assistance -If you have any queries or need assistance with the Sub Organization Widgets, feel free to reach out to our [24/7 customer support](mailto:support@testmuai.com). Our experts will be happy to help you with your queries and provide you with the necessary assistance. diff --git a/static/docs/analytics-table-widgets.md b/static/docs/analytics-table-widgets.md deleted file mode 100644 index 7eaad580d..000000000 --- a/static/docs/analytics-table-widgets.md +++ /dev/null @@ -1,313 +0,0 @@ -# Table Widgets - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Table widgets in LambdaTest Insights enable you to display structured test data in a tabular format, making it easy to view detailed information, compare values across multiple dimensions, and analyze data with multiple columns and aggregations. - -## What is a Table Widget? - -A table widget displays data in rows and columns, providing a structured view of your test data. Tables are particularly effective for: - -- **Displaying detailed data** with multiple attributes in a single view -- **Comparing values** across different dimensions simultaneously -- **Showing aggregated metrics** alongside categorical data -- **Grouping data** by multiple fields to create hierarchical views -- **Creating summary reports** with multiple columns of information - -## Prerequisites - -Before creating a table widget, ensure you have completed the [common prerequisites and setup steps](/docs/dashboards-custom-widgets#prerequisites) for custom widgets. - -## Step-by-Step Guide to Creating a Table Widget - -**Common Steps** -For prerequisites and common setup steps (Navigate to Insights, Create Dashboard, Choose Product, Select Data Source, Configure Filters), see the [Common Steps for Creating Custom Widgets](/docs/dashboards-custom-widgets#common-steps-for-creating-custom-widgets) section in the main Custom Widgets documentation. - -### Step 6: Choose Table Visualization - -1. In the **Choose Visualisation** section, select **Table** from the available options (Line, Bar, Pie, Table, Billboard, Heatmap). - -### Step 7: Configure What to Plot - -This is the most critical step for creating meaningful tables. You can add multiple columns to your table: - -#### Adding Columns to Your Table - -1. **Field Selection*** (Mandatory): Select the field you want to display in this column. - - For row identifiers: Use fields like `test_name`, `project_name`, `build_name` - - For metrics: Use fields like `test_id`, `duration`, or other numeric fields - - For categories: Use fields like `browser`, `os`, `status` - -2. **Choose Aggregations** (Optional): If you selected a numeric field, choose how to aggregate it: - - **Count**: Number of records - - **Sum**: Sum of values - - **Average**: Average value - - **Minimum**: Minimum value - - **Maximum**: Maximum value - - **Unique Count**: Count of unique values - - **Percentiles** (50th, 75th, 90th, 95th, 99th): For duration and other numeric fields - -3. **Group By** (Optional): Select a field to group the data in this column. When you use Group By: - - The column will be split into sub-columns for each unique value in the grouping field - - Each sub-column will show the aggregated value for that group - - This is useful for creating breakdowns (e.g., test counts grouped by status will create sub-columns for "Passed", "Failed", "Error", etc.) - -4. **Label for Columns** (Optional): Provide a custom label for the column header to make your table more readable. - -5. **SubHeader** (Optional): Check this box if you want this column to appear as a sub-header in the table structure. - -6. **Add Another**: Click this button to add additional columns to your table. - -For a complete list of available keys for each product, see the [Available Keys for Custom Widgets](/docs/dashboards-custom-widgets#available-keys-for-custom-widgets) section in the main Custom Widgets documentation. - -**Example Configuration:** -- **Column 1**: `test_name` (no aggregation, no group by) - Creates a "Test Name" column showing individual test names -- **Column 2**: `test_id` with Count aggregation, Group By `status` - Creates a "Test Count" column with sub-columns for each status (Passed, Failed, Error, etc.) - -### Step 8: Name Your Widget - -1. Enter a descriptive name in the **Widget Name** field (e.g., "Test Summary by Status", "Project Test Counts"). -2. Use clear, descriptive names that indicate what the table shows. - -### Step 9: Preview Your Table - -1. Click the **Preview** button to see how your table will look with the current configuration. -2. Review the table structure to ensure it displays the data as expected. -3. Use the refresh icon to update the preview if you make changes. - -### Step 10: Create the Dashboard - -1. Enter a **Dashboard Name** in the bottom section. -2. Click **Create Dashboard** to save your table widget. -3. Your table will be added to the dashboard and ready for analysis. - -## How Table Widgets Work - -Understanding how tables structure and display data is crucial for effective analysis: - -### Column Structure - -Each column in your table can be configured independently: - -- **Simple Column**: A single field without aggregation or grouping - displays the raw value or field name -- **Aggregated Column**: A field with an aggregation function - shows the calculated metric (Count, Sum, Average, etc.) -- **Grouped Column**: A field with Group By - creates sub-columns for each unique value in the grouping field - -### Row Structure - -- **Rows** represent individual records or groups based on your first column configuration -- The first column typically defines what each row represents (e.g., test names, projects, builds) -- Subsequent columns show metrics or breakdowns for each row - -### Group By Behavior - -When you use **Group By** in a column: - -- The column header shows the aggregation and grouping (e.g., "Test Count (Group By Status)") -- Sub-columns are created for each unique value in the grouping field -- Each sub-column displays the aggregated value for that specific group -- This allows you to see breakdowns within each row - -**Example**: If you group `test_id` (Count) by `status`, you'll see: -- Column header: "Test Count (Group By Status)" -- Sub-columns: "Error", "Failed", "Idle Timeout", "Passed", "Queue Timeout" -- Each row shows the count of tests for each status - -### SubHeader Feature - -The **SubHeader** option allows you to create hierarchical table structures: - -- Columns with SubHeader checked appear as section headers -- This helps organize related columns together -- Useful for grouping related metrics or creating multi-level table structures - -## Generating Meaningful Insights from Tables - -### Understanding Table Data - -- **Row Analysis**: Each row represents a specific entity (test, project, build, etc.) -- **Column Comparison**: Compare values across columns to identify patterns -- **Grouped Breakdowns**: When using Group By, analyze the distribution within each row -- **Summary Columns**: Some configurations may show summary information in additional columns - -### Key Insights to Look For - -1. **High-Value Rows**: Identify rows with the highest aggregated values (most tests, longest durations, etc.) -2. **Distribution Patterns**: When using Group By, observe how values are distributed across groups -3. **Anomalies**: Rows with unexpected values or distributions that deviate from normal patterns -4. **Comparisons**: Compare values across rows to identify best/worst performers -5. **Trends**: If your first column is time-based, observe how values change across rows -6. **Coverage**: Identify gaps where certain groups have zero or very low values - -### Interacting with Tables - -- **Sorting**: Click on column headers to sort the table by that column (ascending/descending) -- **Scrolling**: Tables support horizontal and vertical scrolling for large datasets -- **Export**: Tables can be exported to CSV or other formats for further analysis -- **Filtering**: Some table configurations may support in-table filtering -- **Drill-down**: Click on specific cells or rows to drill down into detailed views - -## Use Cases for Table Widgets - -### 1. Test Summary by Test Name - -**Use Case**: Create a comprehensive summary table showing test names with their status breakdowns. - -**Configuration**: -- **Column 1**: `test_name` (no aggregation, no group by) - Label: "Test Name" -- **Column 2**: `test_id` with Count aggregation, Group By `status` - Label: "Test Count" -- Source: Tests - -**Insight**: Quickly see which tests have the most activity and how their results are distributed across different statuses. Identify tests with high failure rates or unusual status distributions. - -### 2. Project Test Counts by Browser - -**Use Case**: Compare test execution across different projects, broken down by browser. - -**Configuration**: -- **Column 1**: `project_name` (no aggregation, no group by) - Label: "Project Name" -- **Column 2**: `test_id` with Count aggregation, Group By `browser` - Label: "Test Count by Browser" -- Source: Tests - -**Insight**: Understand browser usage patterns across projects and identify projects that need better browser coverage. - -### 3. Build Performance Summary - -**Use Case**: Analyze build performance with multiple metrics in a single table. - -**Configuration**: -- **Column 1**: `build_name` (no aggregation, no group by) - Label: "Build Name" -- **Column 2**: `test_id` with Count aggregation - Label: "Total Tests" -- **Column 3**: `test_id` with Count aggregation, Group By `status` - Label: "Test Count by Status" -- **Column 4**: `duration` with Average aggregation - Label: "Average Duration" -- Source: Tests - -**Insight**: Get a comprehensive view of build performance including total tests, status breakdown, and execution time. - -### 4. Test Status Distribution by OS - -**Use Case**: See how test statuses vary across different operating systems. - -**Configuration**: -- **Column 1**: `os` (no aggregation, no group by) - Label: "Operating System" -- **Column 2**: `test_id` with Count aggregation, Group By `status` - Label: "Test Count by Status" -- Source: Tests - -**Insight**: Identify OS-specific issues by comparing status distributions across different operating systems. - -### 5. Custom Data Analysis - -**Use Case**: Analyze test patterns based on custom attributes like release version or feature flags. - -**Configuration**: -- **Column 1**: Custom key (e.g., `release_version`) - Label: "Release Version" -- **Column 2**: `test_id` with Count aggregation, Group By `status` - Label: "Test Count by Status" -- Source: Tests -- Filters: Apply custom data filters - -**Insight**: Track how different releases perform and identify release-specific issues. - -### 6. Duration Analysis by Test Type - -**Use Case**: Compare test execution durations across different test types. - -**Configuration**: -- **Column 1**: `test_type` (no aggregation, no group by) - Label: "Test Type" -- **Column 2**: `duration` with Average aggregation - Label: "Average Duration" -- **Column 3**: `duration` with Maximum aggregation - Label: "Max Duration" -- **Column 4**: `duration` with Minimum aggregation - Label: "Min Duration" -- Source: Tests - -**Insight**: Identify which test types take the longest to execute and optimize accordingly. - -## Available Keys for Table Widgets - -For a complete list of available keys that can be used for configuring table columns, aggregations, and grouping, see the [Available Keys for Custom Widgets](/docs/dashboards-custom-widgets#available-keys-for-custom-widgets) section in the main Custom Widgets documentation. - -Choose keys that represent categorical dimensions (like browser, OS, project name) for row identifiers, and numeric or countable fields for aggregations. For grouped columns, use fields with a reasonable number of unique values to avoid creating too many sub-columns. - -## Best Practices for Table Widgets - -### 1. Choose Meaningful Row Identifiers - -- Use the first column to represent what each row stands for (test names, projects, builds, etc.) -- Select fields that create logical groupings and are meaningful for your analysis -- Avoid fields with too many unique values as this can create very long tables - -### 2. Use Aggregations Appropriately - -- Use **Count** aggregation for `test_id` or other identifier fields to get test counts -- Use **Average**, **Sum**, **Min**, **Max** for numeric fields like `duration` -- Use **Percentiles** for understanding distributions (e.g., 90th percentile duration) - -### 3. Leverage Group By Strategically - -- Use Group By to create breakdowns within each row -- Common grouping fields: `status`, `browser`, `os`, `failure_category` -- Avoid grouping by fields with too many unique values (creates too many sub-columns) -- Group By is most useful when you want to see distributions (e.g., test counts by status) - -### 4. Organize Columns Logically - -- Place the most important identifier column first -- Follow with aggregated metrics -- Use SubHeader to group related columns together -- Limit the number of columns to keep tables readable (typically 3-5 columns work well) - -### 5. Use Filters Effectively - -- Apply filters to focus on relevant subsets of data -- Combine multiple filters to create highly targeted tables -- Filter by time ranges, projects, or other relevant dimensions - -### 6. Name Columns Clearly - -- Use descriptive labels for columns to make tables self-explanatory -- Include aggregation type in labels when using aggregations (e.g., "Average Duration") -- Include grouping information when using Group By (e.g., "Test Count (Group By Status)") - -### 7. Regular Review and Updates - -- Review tables regularly to identify new patterns or issues -- Update filters and column configurations as your testing strategy evolves -- Create multiple tables with different configurations to explore various aspects - -## Troubleshooting - -### Table Shows No Data - -- **Check Filters**: Ensure filters aren't too restrictive -- **Verify Data Source**: Confirm you have test data for the selected product -- **Check Field Selection**: Ensure the selected fields have data in your test executions -- **Review Aggregations**: Verify that aggregation functions are appropriate for the selected fields - -### Table is Too Wide or Has Too Many Columns - -- **Reduce Group By Values**: Use filters to limit the number of groups -- **Remove Unnecessary Columns**: Focus on the most important metrics -- **Simplify Grouping**: Choose grouping fields with fewer unique values - -### Group By Creates Too Many Sub-Columns - -- **Use Filters**: Filter to specific values of the grouping field -- **Choose Different Grouping**: Select a field with fewer unique values -- **Combine Groups**: Consider using multiple tables for different aspects - -### Table Doesn't Show Expected Values - -- **Check Aggregation**: Verify the aggregation function is appropriate -- **Review Data Range**: Ensure you have sufficient data points -- **Verify Field Types**: Ensure numeric fields are used with numeric aggregations - -## Next Steps - -After creating your table widget: - -1. **Add to Dashboard**: Combine tables with other widgets for comprehensive analysis -2. **Share Dashboard**: Share your dashboard with team members for collaborative insights -3. **Create Variations**: Create multiple tables with different configurations to explore various aspects -4. **Review Regularly**: Use insights from tables to identify areas needing attention and track improvements - -## Support - -If you have any questions or need assistance with creating table widgets, please reach out to our [24/7 customer support](mailto:support@testmuai.com) or visit our [Support Center](https://www.testmuai.com/support/). diff --git a/static/docs/analytics-test-case-insights.md b/static/docs/analytics-test-case-insights.md deleted file mode 100644 index 353d2a5dd..000000000 --- a/static/docs/analytics-test-case-insights.md +++ /dev/null @@ -1,165 +0,0 @@ -# Test Case Insights - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Analyzing the test case level insights of your test automation execution on TestMu AI is now easier than ever with the `Test Case Insights` module. The user can easily find information about the count, and type of the test cases through the highly customizable widgets. - -## Prerequisites For Insights: - -1. You should have an active TestMu AI account. -2. You should han active subscription plan with HyperExecute or App Automation. -3. You should have executed at least one test on the TestMu AI HyperExecute / App Automation platform. - -## How To Access Test Case Insights? - -1. Go to the `Insights` tab on the left navigation bar and click on the `Create New`. -2. Select the `Custom Dashboard` option from the menu. -3. Now, search for `Test Case` widgets in the `Web Automation, App Automation & HyperExecute` products. -4. Add the widgets to the dashboard by clicking on the `Add Widget` button. - -## Capture Test Case Insights for Web Automation - -### Capture by WebHook - -You can use LambdaHooks to start and end a test case within a single Selenium session. - -#### Test Case Start - -To start a test case, use the `lambda-testCase-start` hook: - -```java -// To start a test case -((JavascriptExecutor) driver).executeScript("lambda-testCase-start={Your Test Case Name}"); -``` - -#### Test Case End - -To end a test case, use the `lambda-testCase-end` hook: - -```java -// To end a test case -((JavascriptExecutor) driver).executeScript("lambda-testCase-end={Your Test Case Name}"); -``` - -## Capture Test Case Insights for App Automation - -### Capture by WebHook - -You can use LambdaHooks to start and end a test case within a single Appium session. - -#### Test Case Start - -To start a test case, use the `lambda-testCase-start` hook: - -```java -// To start a test case -((JavascriptExecutor) driver).executeScript("lambda-testCase-start={Your Test Case Name}"); -``` - -#### Test Case End - -To end a test case, use the `lambda-testCase-end` hook: - -```java -// To end a test case -((JavascriptExecutor) driver).executeScript("lambda-testCase-end={Your Test Case Name}"); - -``` - -To know more about the hooks and how to use them, you can refer to the [LambdaHooks for Appium](/support/docs/appium-testmu-hooks/#differentiating-test-cases-in-single-session) documentation. - -## Capture Test Case Insights for HyperExecute - -### Capture by WebHook - -You can capture the test case insights by using the WebHook. You need to add the following `WebHook` in your test script. - -```bash -// For Stage of test case `START` -driver.executeScript(`lambda-testCase-start=${Name of the test case}`) - -// For Stage of test case `END` -driver.executeScript(`lambda-testCase-end=${Name of test case}`) - -``` - -### Capture by Playwright reporter - -#### Step 1 - Configure your reporter in YAML - -You can capture the test case insights by using the `reporter` configuration in your `YAML` file for capturing and seeing the test cases in our test case widgets on dashboard. - -Here is a link to documentation for setting up the reporter for `PlaywrightJS`: [Click here](/docs/playwright-html-report/#step-1-update-your-playwright-configuration) - -#### Step 2 - Add the code to lambdatest-setup file - -Once, you have setup the `reporter` then the following code snippet should be added to your `lambdatest-setup.js` config. - -```js title=lambdatest- -try { -const response = JSON.parse(await ltPage.evaluate( -(_, data) => { -return window.eval(data) -}, -`lambdatest_action: ${JSON.stringify({ action: 'getTestDetails' })}` -)) - -if (response?.data?.test_id) { -testInfo.annotations.push({ -type: 'lt_test_id', -description: response.data.test_id, -}) -console.log('LambdaTest Test ID:', response.data.test_id) -} -} catch (err) { -console.warn('Could not fetch LambdaTest test details:', err.message) -} -``` - -#### Step 3 - Execute the tests using HyperExecute - -You can now, execute the tests on our `HyperExecute` platform and you can see the results captured in the `Test Case Widgets` on the dashboards. - -## Capture by NPM Package for WebdriverIO - -You can capture the test case insights by using the NPM Package. You need to add the following `NPM Package` in your test script. Here is the link to the NPM package: [wdio-lambdatest-test-case-analytics-service](https://www.npmjs.com/package/wdio-lambdatest-test-case-analytics-service) - -The package currently only supports for `WebdriverIO` framework. We will be adding support for other frameworks soon. - -```bash -npm i wdio-lambdatest-test-case-analytics-service -``` - -Now, once the package has been installed, you need to add the following code in your `wdio.conf.js` file. - -```javascript -exports.config = { -// ... -services: [ -["lambdatest-test-case-analytics", {}], // Add the service -// ... other services -], -// ... -}; -``` - -Once, you have completed the configuration and start executing your tests on the `HyperExecute` platform, you will be able to see the test case insights on the `Test Case Insights` module. - -## Test Case Insights Widgets - -### Test Case Health Snapshot - -You can analyze the health of your test cases by using the `Test Case Health Snapshot` widget. The widget will display the total number of test cases success, failed. The widget will be displayed in card view and you can also view the details of the test cases by clicking on the `View Details` button. - -### Test Case Group View - -You can analyze the test cases by their group by using the `Test Case Group View` widget. The widget will display the total number of test cases in each group. The widget will be displayed in card view and you can also view the details of the test cases by clicking on the `View Details` button. - -## Test Case Use Cases - -The user can use the `Test Case Insights` module to analyze the test case level insights of their test automation execution on TestMu AI. The user can use the module to analyze the following use cases: - -- Analyze the test case level insights of your test automation execution on TestMu AI. -- Check the health of your test cases. -- Detect the test cases that are failing frequently. -- Drill down to the test cases for faster debugging. diff --git a/static/docs/analytics-test-failure-classification.md b/static/docs/analytics-test-failure-classification.md deleted file mode 100644 index d80177867..000000000 --- a/static/docs/analytics-test-failure-classification.md +++ /dev/null @@ -1,50 +0,0 @@ -# Failure Categorization AI - Test Intelligence - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI Analytics uses Failure Categorization AI to classify test failures into different categories. This helps in identifying the root cause of failures and improving test automation efficiency. The AI model analyzes the test execution data and categorizes the failures based on various parameters such as environment, browser, OS, and failure type. This allows you to focus on the most critical issues and prioritize your testing efforts. - -## Pre-requisites For Failure Categorization AI - -1. You should have an active TestMu AI account. -2. This feature is available for users with the HyperExecute or App / Web Automation subscription plan. -3. You should add the `remark` capability in your test script to enable the Failure Categorization AI. The `remark` capability is used to add comments or notes to the test execution data. This helps in providing additional context to the AI model and improves the accuracy of the failure categorization. - -Here is the link to the documentation for the remark capability: -* [Web Automation](/support/docs/appium-testmu-hooks/#adding-custom-status--remark) -* [App Automation](/support/docs/appium-testmu-hooks/#adding-custom-status--remark) - -4. You should have at least one test failure in your test execution data. The AI model requires a minimum of one test failure to categorize the failures. If there are no test failures, the AI model will not be able to categorize the failures. -5. You should have access to the TestMu AI Insights platform. -6. You can see the failure categorization AI on creating a dashboard in the Insights section. -7. Click on the `Create New` button to create a new dashboard. -8. Choose any of the available widgets and add them to the dashboard. -9. Click on the widget drill down to see the failure categorization AI options. - -## What is Failure Categorization AI? - -The test failure categorization AI is a machine learning model that classifies test failures into different categories. The AI model uses various parameters such as environment, browser, OS, and failure type to categorize the failures. This helps in identifying the root cause of failures and improving test automation efficiency. - -For the first failure as a user you will need to select the type of failure from the list of categories available. The categories are as follows: -- **Product Bug:** This category is used when the failure is due to a bug in the product being tested. This could be a UI issue, a functional issue, or any other type of bug that affects the product's functionality. -- **Test Automation Bug:** This category is used when the failure is due to a bug in the test automation code. This could be a script issue, a framework issue, or any other type of bug that affects the test automation process. -- **Environment Issue:** This category is used when the failure is due to an issue with the test environment. This could be a configuration issue, a network issue, or any other type of issue that affects the test environment. -- **No Action Required:** This category is used when the failure does not require any action. This could be a known issue, a false positive, or any other type of issue that does not require any action from the user. - -Once the user selects the category for the first failure, the AI model will learn from the user's selection and will automatically categorize the subsequent failures based on the user's selection. This helps in reducing the manual effort required to categorize the failures and improves the overall efficiency of the test automation process. - -## How will the Failure Categorization AI help you? - -The failure categorization AI helps you in the following ways: - -- **Automated Classification:** The AI model automatically classifies the test failures into different categories, reducing the manual effort required to categorize the failures. - -- **Improved Efficiency:** By categorizing the failures, the AI model helps in identifying the root cause of failures and improving test automation efficiency. - -- **Prioritization of Testing Efforts:** The AI model helps in prioritizing the testing efforts by identifying the most critical issues that need to be addressed. - -- **Better Insights:** The AI model provides better insights into the test execution data, helping you to make data-driven decisions. - -## Support - -For any queries or issues related to the Failure Categorization AI, please reach out to our [24/7 customer support](mailto:support@testmuai.com). We are here to help you! diff --git a/static/docs/analytics-test-insights.md b/static/docs/analytics-test-insights.md deleted file mode 100644 index 52217bfe0..000000000 --- a/static/docs/analytics-test-insights.md +++ /dev/null @@ -1,58 +0,0 @@ -# Test Insights - Advanced insights for analyzing test runs - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## About Test Insights -TestMu AI Analytics provides Test Insights, a powerful feature designed to help you analyze your test runs and gain valuable insights into your testing processes. With Test Insights, you can visualize your test runs, track progress, and improve the quality of your tests with actionable analytics. - -## Prerequisites for Test Insights -1. **TestMu AI Account**: You must have an active TestMu AI account to access Test Insights. -2. **Subscription Plan**: Test Insights is available for users with the HyperExecute or App/Web Automation subscription plan. -3. **Test Execution Data**: You should have test execution data available in your TestMu AI account. Test Insights analyzes this data to provide insights into your test runs. -4. **Access to Analytics**: Ensure you have access to the TestMu AI Analytics platform to view and analyze your test insights. - -## Getting Started with Test Insights - -To get started with Test Insights, follow these steps: -1. **Log in to your TestMu AI account**: Go to the [TestMu AI website](https://www.testmuai.com) and log in with your credentials. -2. **Navigate to the Analytics section**: Click on the "Insights" tab in the left sidebar of the TestMu AI dashboard. -3. **Select Test Insights**: In the Insights section, click on "Test Insights" to access the insights dashboard. -4. **View Test Runs**: You will see a list of your test runs, along with various metrics and visualizations. -5. **Analyze Test Data**: Use the available filters and options to analyze your test data, including execution time, pass/fail rates, and more. - -## Key Features of Test Insights -- **Test Run Visualization**: Easily visualize your test runs with detailed graphs and charts, allowing you to identify trends and patterns in your testing data. -- **Test Performance Analysis**: Analyze the performance of your tests, including execution time, pass/fail rates, and more, to optimize your testing processes. -- **Test Failure Classification**: Automatically classify test failures to help you quickly identify and address issues in your code. -- **Actionable Analytics**: Gain actionable insights from your test data to make informed decisions and improve the overall quality of your software. -- **Historical Data Comparison**: Compare your current test runs with historical data to identify regressions and improvements over time. - -## Unique Test Instances - -The **Show Unique Instances** toggle consolidates retry runs to give you a cleaner, more accurate view of your test results. - -**How it works:** - -- When **ON**: Within a single build run, tests are grouped by **test name + environment** (browser + OS + device + resolution) as a single instance. Only the **final run** of each instance is considered in reporting, eliminating noise from intermediate retry attempts. - -- When **OFF**: All individual test executions are shown, including every retry attempt. - -The grouping is scoped to individual build IDs, meaning each build's metrics reflect only the deduplicated results within that specific build run. - -This feature is especially useful when your test pipeline uses automatic retries, as it shows the true pass/fail state of each test-environment combination rather than inflating counts with intermediate failures. - -**Your Settings Are Remembered:** -Your Show Unique Instances preference is automatically saved, so you don't need to re-enable it each time you visit. Test Insights and Build Insights remember their settings independently, allowing you to configure each page to suit your workflow. - -**Processing Time** -Retry run consolidation requires a small amount of processing time after test execution completes. If you've just finished running tests, wait a moment before toggling on Unique Instances to ensure all data is consolidated. - -**Learn More** -Want to understand exactly how unique instances and retry attempts are detected? See our comprehensive guide on [Unique Test Instances and Intelligent Retry Detection](/support/docs/analytics-unique-instances-retry-detection/) for step-by-step examples and best practices. - -## Use Cases for Test Insights - -- **Test Optimization**: Use Test Insights to identify slow-running tests and optimize them for better performance. -- **Quality Assurance**: Monitor the quality of your tests by analyzing pass/fail rates and identifying areas for improvement. -- **Regression Detection**: Quickly detect regressions in your code by comparing current test runs with historical data. -- **Team Collaboration**: Share insights with your team to foster collaboration and improve the overall testing process. diff --git a/static/docs/analytics-test-time-trends.md b/static/docs/analytics-test-time-trends.md deleted file mode 100644 index d5478b9d5..000000000 --- a/static/docs/analytics-test-time-trends.md +++ /dev/null @@ -1,29 +0,0 @@ -# Time Insights - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Understanding The Setup Time Insights Widget - -The `Setup Time Insights` module enables the QA Managers to get an overview of the time taken for the setup of test environments. This helps the QA Managers to identify the bottlenecks in the setup process and optimize the setup time for test environments. - -## How To Use Setup Time Insights? -You can check the setup time insights for your test environments by following the below steps: -- Navigate to the `Insights` module from the left navigation menu. -- Create a new `Custom` dashboard or select an existing dashboard. -- Click on the `Add Widget` button. -- Select the `Average Setup Time Trends` widget from the list of available widgets. -- Add the widget to your dashboard. - -## What are the Key Insights Provided by Setup Time Insights Widget? -The `Setup Time Insights` widget provides the following key insights: -- **Average Setup Time**: The average time taken for the setup of test environments. -- **Trend Analysis**: The trend analysis of the setup time for test environments over a period of time. -- **Comparison**: The comparison of the setup time for test environments across different browsers, devices, and operating systems. -- **Optimization Recommendations**: Recommendations to optimize the setup time for test environments using [AI Copilot](/docs/analytics-dashboard-copilot). -- **Real-time Data**: Real-time data of the setup time for test environments. -- **Customization**: Customization options to filter the data based on specific criteria. -- **Export**: Export the setup time insights data in PDF format for further analysis. -- **Collaboration**: Share the setup time insights with team members for collaboration and decision-making using [Dashboard Share](/docs/analytics-dashboard-features/#expiry-settings-for-dashboard-share-link). - -**Upcoming Features** -**Alerts** : Set up alerts for the setup time of test environments to get notified about any anomalies to your `Slack, MS Teams, etc`. diff --git a/static/docs/analytics-unique-instances-retry-detection.md b/static/docs/analytics-unique-instances-retry-detection.md deleted file mode 100644 index e3349ff1d..000000000 --- a/static/docs/analytics-unique-instances-retry-detection.md +++ /dev/null @@ -1,243 +0,0 @@ -# Understanding Unique Test Instances and Retry Detection - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Overview - -When executing automated tests with retry mechanisms, your results often contain multiple executions of the same test under identical conditions. This can distort metrics and obscure the true health of your test suite. TestMu AI Analytics addresses this challenge with **Intelligent Unique Instance Detection**, which automatically consolidates retry attempts and surfaces only the definitive outcome. - -This guide explains how unique instances and retry attempts are detected, where this intelligence applies across TestMu AI Analytics, and how to leverage these insights for precision reporting. - -## Core Concepts - -### What is a Unique Test Instance? - -A **Unique Test Instance** represents a distinct test-environment combination, defined by: - -- **Test Name**: The identifier of your test case -- **Environment Fingerprint**: The complete execution context comprising: - - Browser (e.g., Chrome, Firefox, Safari) - - Operating System (e.g., Windows 10, macOS, Ubuntu) - - Device Type (Desktop, Mobile) - - Resolution (e.g., 1920x1080, 1366x768) - -Two test executions are classified as the **same instance** when they share identical values across all these attributes within a single build. - -**Example**: `Login Test` executing on `Chrome/Windows 10/Desktop/1920x1080` constitutes one unique instance. The same `Login Test` executing on `Firefox/Ubuntu/Desktop/1366x768` represents a separate unique instance. - -### What is a Retry Attempt? - -A **Retry Attempt** is any subsequent execution of an identical unique test instance within a single build. When your test framework or CI pipeline automatically re-executes a failed test, that re-execution is detected and classified as a retry. - -- The **initial execution** of a unique instance is classified as `Execution Type = Initial` -- **Subsequent executions** of the same instance are classified as `Execution Type = Retry` - -### What is Definitive Status? - -The **Definitive Status** represents the conclusive outcome of the final execution for a unique test instance. When reporting with unique instances enabled, this authoritative status determines your metrics. - -- If a test fails initially but passes on retry, the **Definitive Status is Passed** -- If a test fails across all retry attempts, the **Definitive Status is Failed** - -## How Intelligent Instance Detection Works - -### Raw Execution Data - -When tests execute on TestMu AI, every run is captured individually, including all retry attempts. Each execution record includes an **Execution Type** classification indicating whether it's an initial run or a retry. - -**Example: Raw Executions (5 total executions)** - -| # | Test Name | Environment | Status | Execution Type | -|---|-----------|-------------|--------|----------------| -| 1 | Login Test | Chrome/Win10/Desktop/1920x1080 | Failed | Initial | -| 2 | Login Test | Chrome/Win10/Desktop/1920x1080 | Passed | Retry | -| 1 | Login Test | Firefox/Ubuntu/Desktop/1366x768 | Failed | Initial | -| 1 | Checkout Test | Chrome/Win10/Desktop/1920x1080 | Failed | Initial | -| 2 | Checkout Test | Chrome/Win10/Desktop/1920x1080 | Failed | Retry | - -The **#** column indicates the execution sequence within each unique test-environment combination. - -### Step-by-Step Detection Process - -**Step 1: Identify Unique Instances** - -Executions are grouped by `Test Name + Environment Fingerprint`: - -- Instance 1: `Login Test` on `Chrome/Win10/Desktop/1920x1080` (2 executions) -- Instance 2: `Login Test` on `Firefox/Ubuntu/Desktop/1366x768` (1 execution) -- Instance 3: `Checkout Test` on `Chrome/Win10/Desktop/1920x1080` (2 executions) - -**Step 2: Resolve Definitive Status** - -For each instance, the status of the final execution is extracted: - -- Instance 1: Final execution was **Passed** -- Instance 2: Single execution was **Failed** -- Instance 3: Final execution was **Failed** - -**Step 3: Aggregate Retry Attempts** - -Executions classified as `Execution Type = Retry` are tallied for each instance: - -- Instance 1: 1 retry attempt (the second execution) -- Instance 2: 0 retry attempts (single execution) -- Instance 3: 1 retry attempt (the second execution) - -**Result: Consolidated Unique Instances** - -| Test Name | Environment | Definitive Status | Retry Attempts | -|-----------|-------------|-------------------|----------------| -| Login Test | Chrome/Win10/Desktop/1920x1080 | Passed | 1 | -| Login Test | Firefox/Ubuntu/Desktop/1366x768 | Failed | 0 | -| Checkout Test | Chrome/Win10/Desktop/1920x1080 | Failed | 1 | - -### Metrics Comparison - -This table illustrates how metrics transform between viewing all executions versus consolidated unique instances: - -| Metric | Toggle OFF (All Executions) | Toggle ON (Unique Instances) | -|--------|----------------------------|------------------------------| -| Total Tests | 5 | 3 | -| Passed | 1 | 1 | -| Failed | 4 | 2 | -| Tests with Retries | - | 2 | -| Pass Rate | 20% | 33.3% | - -The unique instances view delivers a more precise representation of your test suite health by reflecting tests that ultimately succeed after recovery attempts. - -## Where Unique Instance Detection Applies - -### Build Insights - -In [Build Insights](/support/docs/analytics-build-insights/), the **Show Unique Instances** toggle transforms both the Insights and Tests tabs: - -- **Insights Tab**: Key metrics, visualizations, and Smart Tags reflect deduplicated counts based on definitive results -- **Tests Tab**: The test listing displays only the final execution per unique test-environment combination - -The toggle is positioned at the top of the Build Details page and governs all metrics displayed on that page. - -### Test Insights - -In [Test Insights](/support/docs/analytics-test-insights/), the **Show Unique Instances** toggle consolidates retry attempts across your test data: - -- When **ON**: Tests are grouped by test name + environment, surfacing only definitive results -- When **OFF**: All individual executions are displayed, including every retry attempt - -### Dashboard Widgets - -The [Test Summary widget](/support/docs/analytics-modules-automation-test-overview/) in your Analytics dashboard includes a configuration option for retry handling: - -**"Show test retries separately"** checkbox in widget configuration: - -- **Checked**: All executions displayed individually (equivalent to toggle OFF) -- **Unchecked**: Unique instances shown with definitive status only (equivalent to toggle ON) - -This enables you to customize how each dashboard widget processes retry data independently. - -## Practical Scenarios - -### Scenario 1: CI Pipeline with Automatic Retries - -Your pipeline automatically retries failed tests twice. A test run produces: - -| Execution | Test | Environment | Status | -|-----------|------|-------------|--------| -| 1 | API Test | Chrome/Linux | Failed | -| 2 | API Test | Chrome/Linux | Failed | -| 3 | API Test | Chrome/Linux | Passed | - -**With Unique Instances ON:** -- Total Tests: 1 -- Passed: 1 -- Failed: 0 -- Retry Attempts: 2 - -The test is reported as passed because it ultimately succeeded. - -### Scenario 2: Flaky Test Identification - -A test executes across multiple builds: - -| Build | Status (All Executions) | Status (Unique Instance) | -|-------|------------------------|--------------------------| -| Build 1 | Failed, Passed | Passed | -| Build 2 | Passed | Passed | -| Build 3 | Failed, Failed, Passed | Passed | -| Build 4 | Failed, Failed | Failed | -| Build 5 | Passed | Passed | - -**Analysis with Unique Instances:** -- 4 out of 5 builds show Passed (80% build-level pass rate) -- This test is flagged as potentially flaky due to requiring retry attempts - -### Scenario 3: Cross-Browser Validation - -You execute the same test across 3 browsers: - -| Test | Browser | Executions | Definitive Status | -|------|---------|------------|-------------------| -| Login | Chrome | 2 (Failed → Passed) | Passed | -| Login | Firefox | 1 (Passed) | Passed | -| Login | Safari | 3 (Failed → Failed → Failed) | Failed | - -**Unique Instances Summary:** -- Total Unique Instances: 3 -- Passed: 2 -- Failed: 1 -- Pass Rate: 66.7% - -This clearly reveals Safari has a compatibility issue, while Chrome and Firefox are stable. - -## Best Practices - -1. **Enable Unique Instances for precision reporting**: When your pipeline employs automatic retries, activate the Unique Instances toggle to reveal the true pass/fail state of your tests. - -2. **Monitor retry frequency**: Elevated retry counts signal instability even when tests ultimately pass. Investigate tests with consistently high retry frequency. - -3. **Leverage both views strategically**: - - Use **All Executions** to debug specific failures and analyze retry patterns - - Use **Unique Instances** for release decisions and health assessments - -4. **Allow consolidation time**: Retry detection requires a brief processing period after test execution. Wait a moment before enabling the toggle if you've just completed a build. - -5. **Standardize environment configurations**: Consistent browser/OS/resolution combinations make unique instance grouping more meaningful and comparable across builds. - -6. **Track trends over time**: Use the unique instances view to monitor genuine improvements in test stability, filtering out noise from retry-based recoveries. - -## Frequently Asked Questions - -### Q: Why do my metrics change when I toggle Unique Instances? - -**A:** The toggle transforms what gets measured. With the toggle OFF, every execution counts individually (including retries). With the toggle ON, only unique test-environment combinations count, using the definitive execution's status. - -### Q: How are retry attempts detected? - -**A:** Retry attempts are detected by identifying multiple executions of the same test name and environment fingerprint within a single build. The count equals total executions minus one for each unique instance. - -### Q: Does this affect historical data? - -**A:** Yes, intelligent instance detection applies to all historical data within the selected date range. You can toggle between views at any time. - -### Q: What if the same test runs on different environments? - -**A:** Each test-environment combination constitutes a separate unique instance. `Login Test` on Chrome and `Login Test` on Firefox are two distinct instances, each with their own definitive status and retry count. - -### Q: Why might my pass rate be higher with Unique Instances enabled? - -**A:** Because only the definitive execution counts. Tests that fail initially but pass on retry display as "Passed" in the unique instances view, elevating your overall pass rate. - -### Q: Does this work with parallel test execution? - -**A:** Yes. The grouping is based on test name and environment fingerprint, regardless of execution timing. Parallel executions on different environments create separate unique instances. - -### Q: How long does processing take after a build completes? - -**A:** Typically a few moments. If you toggle on Unique Instances immediately after a build and observe unexpected results, wait briefly and refresh. - -## Related Documentation - -- [Build Insights](/support/docs/analytics-build-insights/) - Build-level health dashboard with unique instance toggle -- [Test Insights](/support/docs/analytics-test-insights/) - Test-level analytics with unique instance support -- [Web & App Automation Modules](/support/docs/analytics-modules-automation-test-overview/) - Dashboard widgets including Test Summary configuration -- [Build Comparison](/support/docs/analytics-build-comparison/) - Compare builds side by side -- [Flaky Test Analytics](/support/docs/analytics-modules-test-intelligence-flaky-test-analytics/) - Identify and track flaky tests diff --git a/static/docs/analytics-widget-drill-down-export.md b/static/docs/analytics-widget-drill-down-export.md deleted file mode 100644 index 47b7b1274..000000000 --- a/static/docs/analytics-widget-drill-down-export.md +++ /dev/null @@ -1,28 +0,0 @@ -# Analytics Drill-Down Widgets - Export Data Options - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -You can export the data from the Analytics Drill-Down widgets in various formats like CSV, Excel, and PDF. This feature allows you to share the data with your team members, stakeholders, or save it for future reference. - -## About Widget Drill-Down - -Widgets in TestMu AI Analytics provide a visual representation of your testing data, enabling you to track and analyze your test execution trends. The Drill-Down feature allows you to view detailed insights by breaking down the data into specific categories, helping you identify patterns and make data-driven decisions. - -## Export to CSV - -The following steps will guide you on how to export data from the Drill-Down widgets to a CSV file: -1. Go to the widget you want to export data from. -2. Click on the `Export As` dropdown located at the top right of the list screen. -3. Select `CSV` from the dropdown list. - -## Export to PDF -The following steps will guide you on how to export data from the Drill-Down widgets to a PDF file: -1. Go to the widget you want to export data from. -2. Click on the `Export As` dropdown located at the top right of the list screen. -3. Select `PDF` from the dropdown list. - -## Value Proposition -By exporting data from the Drill-Down widgets, you can share detailed insights with your team members, stakeholders, or save the data for future reference. This feature enables you to collaborate effectively, make informed decisions, and optimize your testing processes. - -**Export Limitations** -We only allow a limit of maximum **1000** records to be exported at a time. If you have more than 1000 records, you can apply filters to narrow down the data and export it in parts. For further assistance, you can reach out to our [support team](mailto:support@testmuai.com). diff --git a/static/docs/analytics-widget-filter-by-regex.md b/static/docs/analytics-widget-filter-by-regex.md deleted file mode 100644 index 7e82598fa..000000000 --- a/static/docs/analytics-widget-filter-by-regex.md +++ /dev/null @@ -1,83 +0,0 @@ -# Analytics Widgets - Filter by Regex - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## About Regex Mode - -Our analytics platform now supports regular expressions (Regex) for advanced text searching, pattern matching, and data extraction. This feature enhances your ability to filter and analyze data with greater precision and flexibility. - -## Accessing Regex Mode - -1. Navigate to your dashboard. -2. Click on "Configure Widget." -3. In the filter options, you'll see a radio button to switch between "List" and "Regex" modes. -4. Select "Regex" to enable Regex filtering. - -## Using Regex in Filters - -Once in Regex mode, you can enter Regex patterns in the filter input field. As you type, you'll see auto-suggestions for common Regex patterns. - -### Key Features: - -- **Auto-suggest**: Get suggestions for commonly used Regex patterns. -- **Result Preview**: See a list of matching results and the total count of filtered items. -- **Persistent Mode**: Regex mode remains active across different filter types until you switch back to list mode. - -## Supported Regex Patterns - -Our platform supports a wide range of Regex patterns, including: - -- Character classes (e.g., `\d`, `\w`, `\s`) -- Anchors (e.g., `^`, `$`, `\b`) -- Quantifiers (e.g., `*`, `+`, `?`, `{n}`) -- Groups and ranges (e.g., `(abc)`, `[a-z]`) -- Assertions (e.g., `(?=...)`, `(?!...)`) -- And more! - -For a complete list of supported patterns, click the "See Documentation" link in the Regex mode view. - -## Use Cases - -### 1. Filtering Build Names for Automation Products - -**Scenario**: You want to filter all build names that start with "Auto_" followed by a version number. - -**Regex Pattern**: `^Auto_v\d+(\.\d+)*$` - -This pattern will match strings like: -- Auto_v1 -- Auto_v2.0 -- Auto_v3.1.4 - -### 2. Filtering Test Names with Specific Keywords - -**Scenario**: You want to filter all test names that contain the word "Login" or "Signup." - -**Regex Pattern**: `Login|Signup` - -This pattern will match strings like: -- User Login Test -- Signup Page Test -- Login and Signup Flow - -### 3. Filtering by Build with Date & Time Prefix or Suffix - -**Scenario**: You want to filter all builds with a date and time prefix or suffix. - -**Regex Pattern**: `^\d{4}-\d{2}-\d{2}_\d{2}:\d{2}:\d{2}$|\d{2}:\d{2}:\d{2}_\d{4}-\d{2}-\d{2}$` - -This pattern will match strings like: -- 2022-01-01_12:00:00 -- 12:00:00_2022-01-01 -- 2022-01-01_12:00:00_UTC - -## Best Practices - -1. Start with simple patterns and gradually add complexity. -2. Use the auto-suggest feature to learn common Regex constructs. -3. Always test your Regex patterns on a small dataset before applying them to large-scale analyses. -4. Remember that Regex is case-sensitive by default. Use flags or character classes to make it case-insensitive if needed. - -## Need Help? - -If you have any questions or need assistance with Regex filtering, feel free to reach out to our [support team](mailto:support@testmuai.com). We're here to help you make the most of our advanced analytics features! diff --git a/static/docs/analytics-widgets-tests-overview-landing.md b/static/docs/analytics-widgets-tests-overview-landing.md deleted file mode 100644 index 591dbcac5..000000000 --- a/static/docs/analytics-widgets-tests-overview-landing.md +++ /dev/null @@ -1,25 +0,0 @@ -# Test Overview Widgets - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Analytics in TestMu AI allows the QA managers and executives to efficiently track the tests conducted on the platform and get deep-level insights with the numerous widgets that the product provides. - - Web Automation Widgets - Gain valuable insights into your web automation testing efforts with comprehensive analytics and metrics. - - App Automation Widgets - Unlock the power of analytics to optimize your mobile app testing process and improve app quality. - - HyperExecute Widgets - Leverage advanced analytics to streamline your HyperExecute testing pipeline and boost testing efficiency. - - Realtime Widgets   - Get real-time insights into your testing activities and make data-driven decisions on the fly. - - Real-Device Widgets - Analyze your real device testing performance and identify optimization opportunities. - - SmartUI Widgets   - Gain actionable insights into your SmartUI testing efforts and ensure exceptional user experiences. - -The features are available for limited users only. If you wish to try them out, please reach out to our [24/7 customer support](mailto:support@testmuai.com). diff --git a/static/docs/analytics-widgets.md b/static/docs/analytics-widgets.md deleted file mode 100644 index 6d0d01e2e..000000000 --- a/static/docs/analytics-widgets.md +++ /dev/null @@ -1,68 +0,0 @@ -# Widgets - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Analytics in TestMu AI allows the QA managers and executives to efficiently track the tests conducted on the platform and get deep-level insights with the numerous widgets that the product provides. - -## Insights Modules - -**Explore AI Dashboard CoPilot** - -We have recently launched an AI Dashboard CoPilot feature that provides intelligent recommendations, insights, and predictions tailored to your specific data. - -To know more about this feature, refer to the [AI Dashboard CoPilot](/docs/analytics-dashboard-copilot/) documentation. - - Concurrency Usage - Learn how to to get an overview of the TestMu AI resources being utilized by your teams. - - Tests Overview - Learn how to get an overview of the tests running on the TestMu AI. - - Error Insights - Learn how to get all the insights into the test errors on the TestMu AI. - - Flaky Tests AI - Monitor all the Flaky tests run on the platform and review the quality of the feature on your SMOKE and Regression Jobs. - - Test Case Insights - Get insights for `describe()` block level for your test frameworks in HyperExecute. Monitor the quality metrics at each test case. - - API Test Insights - Get API test time-series insights using Allure report integration with your test suite and run it on HyperExecute. - - Sub Organization Insights   - Monitor and allocate the right resources required for your organizations with advanced insights. - - Private Real Device Insights   - Gain detailed insights for ensuring optimal performance and resource utilization for your dedicated devices. - - Usage Report   - Get detailed insights into your product usage and optimize your testing efforts. - -## Product Specific Widgets - - Web Automation Widgets - Gain valuable insights into your web automation testing efforts with comprehensive analytics and metrics. - - App Automation Widgets - Unlock the power of analytics to optimize your mobile app testing process and improve app quality. - - HyperExecute Widgets - Leverage advanced analytics to streamline your HyperExecute testing pipeline and boost testing efficiency. - - Real-Device Widgets - Analyze your real device testing performance and identify optimization opportunities. - - Realtime Widgets   - Get real-time insights into your testing activities and make data-driven decisions on the fly. - - SmartUI Widgets   - Gain actionable insights into your SmartUI testing efforts and ensure exceptional user experiences. - - Accessibility Widgets   - Ensure your application or website meets WCAG standards with comprehensive accessibility insights. - - Test Manager Widgets   - Manage your project-level tests comprehensively, track the progress of test suite development with detailed insights. - -The features are available for limited users only. If you wish to try them out, please reach out to our [24/7 customer support](mailto:support@testmuai.com). diff --git a/static/docs/angularjs-with-karma-running-angular-tests-on-testmu-selenium-grid.md b/static/docs/angularjs-with-karma-running-angular-tests-on-testmu-selenium-grid.md deleted file mode 100644 index b26472c4d..000000000 --- a/static/docs/angularjs-with-karma-running-angular-tests-on-testmu-selenium-grid.md +++ /dev/null @@ -1,118 +0,0 @@ -# Selenium With AngularJS and Karma - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Run AngularJS tests with Karma on the TestMu AI cloud grid. This guide covers setup, running a sample test, configuring capabilities, and testing locally hosted pages. - -**Deprecated Framework** -AngularJS reached end-of-life in December 2021. Consider migrating to Angular with [Karma](/support/docs/karma-integration-with-testmu/) or [Playwright](/support/docs/playwright-testing/). This guide is maintained for existing users. - -**Sample repo** -All the code used in this guide is available in the sample repository. - - View on GitHub - -## Prerequisites - -Complete these steps before running AngularJS tests with Karma on TestMu AI. - -1. Create a [TestMu AI account](https://www.testmuai.com/register/?redirectTo=https://accounts.lambdatest.com/dashboard) and get your username and access key from the dashboard. -2. Install **NodeJS** v6 or newer from [nodejs.org](https://nodejs.org/en/). -3. Install **npm** and upgrade to the latest version: `npm install npm@latest -g` -4. Install [Angular CLI](https://angular.io/cli): `npm install -g @angular/cli` -5. Download [Selenium JavaScript bindings](https://www.selenium.dev/downloads/) from the official website. - -## Step 1: Clone the Sample Project - -Clone the TestMu AI AngularJS Karma sample repository to your local machine. - -```bash -git clone https://github.com/LambdaTest/angular-karma-sample -cd angular-karma-sample -``` - -Install the required dependencies: -```bash -npm install -``` - -## Step 2: Set Your Credentials - -Set your TestMu AI username and access key as environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -## Step 3: Configure Your Test Capabilities - -Update the capabilities in the `karma.conf.js` file to define the browser and platform settings. - -```js -customLaunchers: { -chrome: { -base: 'WebDriver', -config: webdriverConfig, -browserName: 'chrome', -platform: 'windows 10', -version: '71.0', -name: 'Karma With Heartbeat', -tunnel: true, // In case karma is running on local machine -video: true, // capture video for your test -user: process.env.LT_USERNAME, -accessKey: process.env.LT_ACCESS_KEY, -pseudoActivityInterval: 15000 // 15000 ms heartbeat to avoid timeouts -} -} -``` - -Generate capabilities for your test requirements with the [Capabilities Generator](https://www.testmuai.com/capabilities-generator/). - -## Step 4: Run the Test - -Execute the test using the following command. - -```bash -karma start karma.conf.js -``` - -Or run the test using: -```bash -npm test -``` - -**Parallel tests:** -```bash -ng test -``` - -## Step 5: View Your Results - -After running the test, view your results on the [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build). - -The dashboard provides: -- Video recordings of each test session -- Screenshots captured at each step -- Console logs from the browser -- Network logs for debugging -- Detailed command logs - -## Run AngularJS with Karma Tests Using Agent Skills - -Use AI coding assistants to generate and run AngularJS with Karma tests with the TestMu AI Agent Skill. - -The [karma-skill](https://github.com/LambdaTest/agent-skills/tree/main/karma-skill) is part of [TestMu AI Agent Skills](https://github.com/LambdaTest/agent-skills/) - structured packages that teach AI coding assistants how to write production-grade test automation. - -Install the skill: - -```bash -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/karma-skill .claude/skills/ - -# For Cursor / Copilot -cp -r agent-skills/karma-skill .cursor/skills/ -``` - -Install all available framework skills at once by cloning the repository directly into your tool's skills directory (e.g., `.claude/skills/`, `.cursor/skills/`). diff --git a/static/docs/app-auto-network-throttling.md b/static/docs/app-auto-network-throttling.md deleted file mode 100644 index 37f72e395..000000000 --- a/static/docs/app-auto-network-throttling.md +++ /dev/null @@ -1,90 +0,0 @@ -# Network Throttling for Appium tests - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -In Appium testing, assessing your app's performance under diverse network conditions (2G/3G/LTE) and offline scenarios, is crucial. Fluctuating upload and download speeds can significantly impact your app's behavior across different devices. - -TestMu AI simplifies testing by enabling simulation of diverse network conditions. Whether starting with defaults or custom profiles, these features replicate real-world scenarios, proving invaluable for Appium tests. The device maintains uninterrupted internet connectivity throughout, ensuring a reliable testing experience for your mobile applications. - -## Workflow -### Initialization - -- **Capability:** Initiate a test session with predefined network profiles using the `networkProfile` capability. Example: -```python -caps = { -"network": True, -"networkProfile": "2g-gprs-good" # Set the desired network profile -} -``` - -To utilize the **networkProfile** capability, ensure that you include `network: True` in the capabilities. - -### During Test Execution - -- **LambdaHook:** Dynamically alter the network profile within the test session using the following LambdaHook: -```python -driver.execute_script("updateNetworkProfile=3g-umts-good") -``` - -### Supported Network Profiles - -| Profile Name | Download Speed | Upload Speed | Latency | -|--------------------------|-----------------|--------------|---------| -| 2g-gprs-poor | 20 Kbps | 6 Kbps | 1000 ms | -| 2g-gprs-good | 50 Kbps | 16 Kbps | 500 ms | -| 3g-umts-poor | 200 Kbps | 64 Kbps | 400 ms | -| 4g-lte-poor | 1 Mbps | 500 Kbps | 200 ms | -| 3g-umts-good | 5 Mbps | 2 Mbps | 100 ms | -| 4g-lte-good | 15 Mbps | 7 Mbps | 70 ms | -| 4g-lte-advanced-good | 25 Mbps | 12 Mbps | 20 ms | -| default | NA | NA | NA | - -### Custom Profiles - -- **LambdaHook:** Define and implement custom network profiles with LambdaHook by specifying the maximum download speed (kbps), maximum upload speed (kbps), and latency (ms) for the custom condition, as illustrated in the example. -```python -driver.execute_script("customNetworkProfile: { \"downloadSpeed\": 500, \"uploadSpeed\" : 250, \"latency\": 100 }" ) -``` - -### Default/Reset Network Configuration - -- **LambdaHook:** Employ this webhook to seamlessly restore the device's network profile to its default state. Invocation of this LambdaHook removes any predefined or custom network settings, ensuring the device is reset to its original configuration. -```python -driver.execute_script("updateNetworkProfile=default") -``` - -### Offline Mode - -- To initialize tests in **offline mode**, set the `networkProfile` capability to `offline` during session initiation: -```python -caps = { -"network": True, -"networkProfile": "offline" # Set to offline mode -} -``` - -- **LambdaHook:** You can also switch to offline mode during the test execution with the following command: -```python -driver.execute_script("updateNetworkProfile=offline") -``` - -## Toggle Offline/Online Mode via API - -- For Android and iOS devices, you can use the offline/online mode API within the running test session: - -```bash -curl --location 'https://mobile-api.lambdatest.com/mobile-automation/api/v1/sessions//update_network' \ ---header 'Authorization: Basic ' \ ---header 'Content-Type: application/json' \ ---data '{"mode": "offline"}' -``` - -- **Network throttling** results may vary sometimes based on multiple factors including network conditions and device performance. - -- **Wrong Capability Name or Value:** Providing an incorrect capability value results in a bad request error with a descriptive message. - -- **LambdaHook Error:** If you pass any wrong value in LambdaHook, you will receive an error message. - -- In offline mode, **live video** will not be available for the duration of the test session. - -> In case you have any questions, feel free to share them with us.Our experts are available on **24/7 Customer chat support**. You can also drop us a mail at support@testmuai.com. Happy testing! 🙂 diff --git a/static/docs/app-automation-dashboard.md b/static/docs/app-automation-dashboard.md deleted file mode 100644 index 486dae733..000000000 --- a/static/docs/app-automation-dashboard.md +++ /dev/null @@ -1,57 +0,0 @@ -# Inside TestMu AI App Automation Dashboard - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -You can view your App automated test results on TestMu AI App Automation Dashboard. TestMu AI App Automation dashboard enables you to debug failed test sessions utilizing different test logs. The capabilities for a given test session may be specified in the test script and generated using the TestMu AI Capability Generator. - -## Debugging Tests - -TestMu AI allows you to debug and fix failed test cases using extensive test logs such as Appium logs, device logs, network logs, screenshots, and video logs. - -1. **Device Logs** - -Device logs allows you to capture system logs for Android and iOS apps. It allows the user to keep track of the application logs in case the application malfunctions. - -Use the `devicelog` capability to enable the Device logs. - -```java -capabilities.setCapability("devicelog", "true") -``` - -2. **Network Logs** - -Network logs enables you to take screenshots of multiple network requests during testing and to monitor the performance of network requests and responses. - -Use the `network` capability to enable the Network logs. - -```java -capabilities.setCapability("network", "true") -``` - -3. **Appium Logs** - -Appium logs allows you to capture the logs created by the Appium server during test execution, that include a comprehensive session log. - ->**Note**: On TestMu AI platform, Appium logs are enabled by default. - -4. **Visual Logs** - -With Visual logs, you can capture the screenshots of the entire test session. - -Use the `visual` capability to enable the Visual logs. - -```java -capabilities.setCapability("visual", "true") -``` - -5. **Video Logs** - -Visual logs enables you to record the video of the complete test session. - -Use the `video` capability to enable the Video logs. - -```java -capabilities.setCapability("video", "true") -``` - ->In case you have any questions or need any additional information, drop them at our **24X7 Chat Support** or mail us directly at support@testmuai.com. diff --git a/static/docs/app-automation-on-emulators-simulators.md b/static/docs/app-automation-on-emulators-simulators.md deleted file mode 100644 index 1cead91b5..000000000 --- a/static/docs/app-automation-on-emulators-simulators.md +++ /dev/null @@ -1,349 +0,0 @@ -# App Automation Using Emulators and Simulators on TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI enables developers and testers to automate mobile app testing using cloud-based emulators and simulators. This eliminates the need for physical devices, streamlining the process of validating app functionality across a wide range of configurations. - -In this documеntation, learn how to run app automated tests on Emulators and Simulators using TestMu AI virtual testing cloud. - -**Note** - -Currently, App Automation is available for Emulators and Simulators in the Beta phase, with support for the Appium framework. To use this feature, [Contact Sales](https://www.testmuai.com/contact-us). - -## Prerequisites - -Bеforе you gеt startеd with App Automation on LambdaTеst, makе surе you havе thе following prеrеquisitеs in placе: - -1. Sign up for a LambdaTеst account if you havеn't alrеady. - -2. Install [Appium Java Client](https://github.com/appium/java-client). - -3. A TestMu AI Username and Access Key. - -4. Ensure you have an access to an **Android** app (*.apk* or *.aab* file) or an **iOS** *zip* file (containing *.app* file). - -Don't have **.apk** or **.app** filеs? You can still run your samplе tеsts on LambdaTеst using our samplе Android and iOS applications, accеssiblе through thе following links: - -* [Samplе Android App](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk) -* [Samplе iOS App](https://prod-mobile-artefacts.lambdatest.com/assets/docs/firefox.zip) - -## Run Your First Test - -Here are the following steps to run your first app automated test on Emulators and Simulators. - -### Step 1: Upload Your Application - -Upload your **iOS** application (*.app* file) or **Android** application (*.apk* file) to the TestMu AI servers using our **REST API**. You need to provide your **Username** and **AccessKey** in the format `Username:AccessKey` in the **cURL** command for authentication. Make sure to add the path of the **appFile** in the cURL request. Here is an example cURL request to upload your app using our REST API: - -{`curl -u ":" \ ---location --request POST 'https://manual-api.lambdatest.com/app/upload/virtualDevice' \ ---form 'name="Android_App"' \ ---form 'appFile=@"/Users/macuser/Downloads/proverbial_android.apk"' -`} - -{`curl -u ":" \ ---location --request POST 'https://manual-api.lambdatest.com/app/upload/virtualDevice' \ ---form 'name="Android_App"' \ ---form 'appFile=@"/Users/macuser/Downloads/proverbial_android.apk"' -`} - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" \\ ---location --request POST 'https://manual-api.lambdatest.com/app/upload/virtualDevice' \\ ---form 'name="Android_App"' \\ ---form 'url="https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk"'`} - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" --location --request POST "https://manual-api.lambdatest.com/app/upload/virtualDevice" --header "Content-Type: application/x-www-form-urlencoded" --data-urlencode "url=:https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk" --data-urlencode "name=Proverbial_App"`} - -**Note** - -Response of above cURL will be a **JSON** object containing the `App URL` of the format - ``lt://APP123456789123456789`` and will be used in the Step 2. - -### Step 2: Write Your Automation Script - -1. Write your automation script in your preferred language that Appium framework supports. - -In the below test script, ensure to update the `app_url`, `username` and `accesskey`. - -**Set the Capability** -To run the test on Emulator/Simulator, set the `isRealMobile` capability to `false`. - -```python -# Tested on python3, Selenium 4.9.0, Appium-Python-Client 2.2.0 - -from appium import webdriver -from selenium.webdriver.common.by import By -import time - -def startingTest(): -print(f"Starting test") -desiredCap = caps[0].copy() - -print( -f"deviceName {desiredCap['lt:options'].get('deviceName')}, platformVersion {desiredCap['lt:options'].get('platformVersion')}" -) - -try: -driver = webdriver.Remote(command_executor=url, desired_capabilities=desiredCap) -except Exception as e: -print("err", e) -return - -try: -print(f"driver created") -time.sleep(10) - -ctx = driver.current_context -print("ctx", ctx) -sessionId = driver.session_id -print("sessionId", sessionId) -isKeyBoardShown = driver.is_keyboard_shown() -print(f"isKeyboardShown {isKeyBoardShown}") -isLocked = driver.is_locked() -print("isLocked", isLocked) -isInstalled = driver.is_app_installed("com.example.QAapp") -print("isInstalled", isInstalled) - -print(driver.page_source) -elem = driver.find_element(By.ID, "com.example.QAapp:id/webpage").click() -print("element", elem) -time.sleep(2) - -driver.find_element(By.ID, "com.example.QAapp:id/websiteName").send_keys( -"ThisIsDemoText" -) -driver.find_element(By.ID, "websiteName").clear() -elem = driver.find_element(By.ID, "com.example.QAapp:id/findButton") -print("element", elem) - -driver.find_element(By.ID, "com.example.QAapp:id/websiteName").click() -params = {"command": "input-text", "text": "thisIsMyText"} -result = driver.execute_script("lambda-adb", params) -print("result", result) -time.sleep(2) - -driver.find_element(By.ID, "com.example.QAapp:id/websiteName").send_keys( -"https://www.ifconfig.me" -) -driver.find_element(By.ID, "findButton").click() -time.sleep(2) -driver.find_element(By.ID, "com.example.QAapp:id/websiteName").send_keys( -"https://google.com" -) -driver.find_element(By.ID, "findButton").click() -time.sleep(2) - -if desiredCap["lt:options"].get("tunnel"): -driver.find_element(By.ID, "com.example.QAapp:id/websiteName").send_keys( -"http://localhost.lambdatest.com:8001" -) -driver.find_element(By.ID, "findButton").click() -time.sleep(5) - -driver.orientation = "LANDSCAPE" - -print("Quitting test") -driver.quit() -except Exception as e: -print(e) -driver.quit() - -USERNAME = "YOUR_USERNAME" -ACCESS_KEY = "YOUR_ACCESS_KEY" -url = "https://{USERNAME}:{ACCESS_KEY}@mobile-hub.lambdatest.com/wd/hub" - -buildName = "Testing build" -numTests = 1 - -caps = [ -{ -"lt:options": { -"w3c": True, -"platformName": "Android", -"allowInvisibleElements": True, -"deviceName": "Galaxy A33 5G", -"platformVersion": "13", -"app": "YOUR_APP_URL", -"devicelog": True, -"build": buildName, -"visual": True, -"network": True, -"tunnel": False, -"video": True, -"isRealMobile": false, -}, -} -] -startingTest() -``` - -```python -from appium import webdriver -from selenium.webdriver.common.by import By -import time - -def simulatorTest(): -print(f"Starting test") -try: -driver = webdriver.Remote(command_executor=url, desired_capabilities=caps) -except Exception as e: -print(f"Error creating driver", e) -return - -try: -print(f"driver created") -time.sleep(10) -for i in range(0, 1): -ctx = driver.current_context -print("ctx", ctx) - -sshot = driver.get_screenshot_as_base64() -print("sshot", sshot[:100]) - -print("Getting page source") -print("page_src1", driver.page_source) - -sessionId = driver.session_id -print("sessionId", sessionId) -driver.is_keyboard_shown() -isLocked = driver.is_locked() -print("isLocked", isLocked) -element = driver.switch_to.active_element -print(f"element", element) -element.send_keys("https://google.com\n") -time.sleep(5) - -if caps["lt:options"].get("tunnel"): -driver.find_element(By.NAME, "url").send_keys( -"http://localhost.lambdatest.com:8001\n" -) -time.sleep(5) - -driver.find_element(By.NAME, "url").send_keys("https://mylocationnow.io/\n") -time.sleep(5) - -driver.find_element(By.NAME, "url").send_keys("http://www.fast.com\n") -time.sleep(2) - -print(f"Quitting") -driver.quit() -except Exception as e: -print(f', error platformVersion {caps["lt:options"]["platformVersion"]} :: {e}') -driver.execute_script("lambda-status=failed") -driver.quit() - -# prod -url = "https://{USERNAME}:{ACCESS_KEY}@mobile-hub.lambdatest.com/wd/hub" - -buildName = "Testing build" -numTests = 1 - -caps = { -"lt:options": { -"w3c": True, -"platformName": "ios", -"deviceName": "iPhone 12", -"platformVersion": "14.5", -"app": "YOUR_APP_URL", -"devicelog": True, -"build": buildName, -"visual": True, -"network": True, -"video": True, -"isRealMobile": false, -}, -} - -simulatorTest() -``` - -2. Create `.XML` file in order to run your test and define device capabilities. Please find sample code below for the same. - -```xml - - - - - - - - - - - - - - - - - - - - - - -``` - -```xml - - - - - - - - - - - - - - - - - - - - - - -``` - -### Step 3: Execute Your Test Case - -Debug and run your code. Run `iOSApp.java` or `AndroidApp.java` in your editor. - -### Step 4: View Test Execution - -Once you have run your tests, you can view the test execution along with logs. You will be able to see the test cases passing or failing. You can view the same at TestMu AI [App Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://appautomation.lambdatest.com/build). - -## More About Desired Capabilities - -Sample Capabilities for both Android and iOS are mentioned below - - -```java -{ -"deviceName": "Galaxy Tab S4", -"platformName": "android", -"platformVersion": "10", -"app": "App_url", -"visual": true, -"console": true, -"deviceOrientation": "PORTRAIT", -"build": "new-12", -"isRealMobile": false, -} -``` - -```java -{ -"deviceName": "iPhone 12 Mini", -"platformName": "ios", -"platformVersion": "14", -"app": "App_url", -"isRealMobile": false, -"visual": true, -"console": true, -"build": "lt-web-4", -"network": false, -} -``` diff --git a/static/docs/app-automation-slack-notifications.md b/static/docs/app-automation-slack-notifications.md deleted file mode 100644 index 883d7292a..000000000 --- a/static/docs/app-automation-slack-notifications.md +++ /dev/null @@ -1,99 +0,0 @@ -# Slack Notifications for App Automation - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -With Slack Notifications for App Automation, you can receive real-time updates about your test sessions and build summaries directly in your Slack channels. This eliminates the need to manually monitor the dashboard and keeps your entire team informed about test execution results. - -## Prerequisites - -- A TestMu AI account with App Automation access. -- [Slack Integration](/support/docs/slack-integration/) enabled on your TestMu AI account. -- The Slack channel(s) you want to send notifications to must already exist in your Slack workspace. - -## Setting Up Slack Notifications - -To enable Slack notifications for your App Automation tests, add the `slackChannel` capability to your test configuration. Notifications are sent automatically when each test session completes and when the entire build finishes execution. - -### Capability Reference - -| Attribute | Details | -|-----------|---------| -| **Capability** | `slackChannel` | -| **Type** | String | -| **Required** | No (optional) | -| **Example** | `mobile-test-results` | - -### Configuration Examples - -```java -DesiredCapabilities capabilities = new DesiredCapabilities(); -capabilities.setCapability("platformName", "Android"); -capabilities.setCapability("deviceName", "Galaxy S23"); -capabilities.setCapability("app", "lt://APP123456789"); -capabilities.setCapability("build", "Regression Suite v2.0"); -capabilities.setCapability("name", "Login Test"); -capabilities.setCapability("isRealMobile", true); -//highlight-next-line -capabilities.setCapability("slackChannel", "mobile-test-results"); -``` - -```python -desired_caps = { -"platformName": "Android", -"deviceName": "Galaxy S23", -"app": "lt://APP123456789", -"build": "Regression Suite v2.0", -"name": "Login Test", -"isRealMobile": True, -#highlight-next-line -"slackChannel": "mobile-test-results" -} -``` - -```javascript -const capabilities = { -platformName: "Android", -deviceName: "Galaxy S23", -app: "lt://APP123456789", -build: "Regression Suite v2.0", -name: "Login Test", -isRealMobile: true, -//highlight-next-line -slackChannel: "mobile-test-results" -}; -``` - -## What Notifications You Receive - -TestMu AI sends two types of Slack notifications: - -| Notification Type | When It's Sent | What It Contains | -|-------------------|----------------|------------------| -| **Session Update** | When each test session completes | Session name, status (passed/failed), device info | -| **Build Summary** | When all sessions in a build finish | Overall pass/fail count, build duration, build link | - -## How Channel Routing Works - -The `slackChannel` capability is set at the session level. If you update the channel name in a subsequent session under the same build, notifications for that session onward are sent to the updated channel. - -When multiple users or multiple sessions specify different channels concurrently within the same build, all channels are merged and notifications are sent to all of them. - -## Fallback Behavior - -If the `slackChannel` capability is not provided or the specified channel is invalid, TestMu AI follows this fallback logic: - -1. **Channel provided and valid** → Notifications are sent to the specified channel. -2. **Channel provided but invalid** (e.g., channel doesn't exist or fails validation) → Notifications fall back to the default channel configured in your [Slack Integration settings](/support/docs/slack-integration/). -3. **No channel provided** → Notifications are sent to the default channel if one is configured; otherwise, notifications are skipped silently. - -**Important** -Slack notifications never block or impact your test execution. If a notification fails to deliver for any reason, the test continues to run normally. - -## Related Resources - -- [Slack Integration Setup](/support/docs/slack-integration/) -- [Desired Capabilities in App Automation](/support/docs/desired-capabilities-in-appium/) -- [Getting Started with App Automation](/support/docs/getting-started-with-appium-testing/) - -> Got any questions? -> Please reach out at our **24x7 Chat Support** or you could also mail us at [support@testmuai.com](https://support.lambdatest.com/). diff --git a/static/docs/app-automation-using-app-center.md b/static/docs/app-automation-using-app-center.md deleted file mode 100644 index 5ab85dafe..000000000 --- a/static/docs/app-automation-using-app-center.md +++ /dev/null @@ -1,247 +0,0 @@ -# App Test Automation Using App Center - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Visual Studio App Center is an integrated solution for mobile app development on iOS, Android, Windows, and macOS. It combines a number of mobile development services, such as build, test, distribute, monitoring, diagnostics, etc., into a single integrated cloud platform. By automating the upload process, Visual Studio App Center can ease the manual effort of uploading an app. - -In this documentation, learn how to install apps from App Center for automated app testing. - -## Prerequisites - -Getting started with the setup requires access to the following: - -1. Access to TestMu AI Real Devices. -2. Access to App Center and permissions to generate API tokens on App Center. - -## Step 1 - Generating API Token On App Center - -Below are the steps to generate the API Token on App Center - - -1. Launch App Center. Navigate to your Profile avatar and click **Account Settings**. - -2. Click on **User API tokens** to generate a token. - -3. Click on **Add new API token** to add the new token. - ->**Important**: It’s recommended to provide **Read Only** access. - -4. Once you get the API token, click on the Copy icon to copy the generated token. This token will be used in Step 2. - -## Step 2: Listing All The Apps For The API Token - -You can get all the Apps (specific to the API token) by using the API call below with the API token you generated in the previous step. - -```js -curl -sX GET "https://api.appcenter.ms/v0.1/apps" \ --H "Content-Type: application/json" \ --H "X-Api-Token: {your_api_token}" -``` - -Shown below is the response of the above API call. - -```json -[ -{ -"id":"19c6645d-3146-4853-97f4-22b1fc0a3ec4", -"app_secret":"9bdaef87-c3b7-4b8b-8a64-944bafdd870a", -"description":null, -"display_name":"Demo-2", -"name":"Demo-2", -"os":"iOS", -"platform":"Objective-C-Swift", -"origin":"appcenter", -"icon_url":null, -"created_at":"2022-03-31T11:29:39.019Z", -"updated_at":"2022-03-31T11:29:39.019Z", -"release_type":"Alpha", -"owner":{ -"id":"29536631-e7c3-4035-a78e-4040bc96e9ce", -"avatar_url":null, -"display_name":"Gaurav Raj", -"email":"gauravraj@lambdatest.com", -"name":"gauravraj-lambdatest", -"type":"user" -}, -"azure_subscription":null, -"member_permissions":[ -"manager" -] -}, -{ -"id":"2c90096a-ac95-4911-a545-4b40e4f66b7b", -"app_secret":"781a8f31-b593-48ce-8c31-497be0da89b5", -"description":null, -"display_name":"TodoApp", -"name":"TodoApp", -"os":"Android", -"platform":"Java", -"origin":"appcenter", -"icon_url":"https://appcenter-filemanagement-distrib4ede6f06e.azureedge.net/364affef-e00b-4f93-81 d3-5ee4a00755c9/launcher.png?sv=2019-02-02&sr=c&sig=OCEgMuaJlZt2wHqrLp4j8aO9xMT%2BOcy8 X4ty%2B2SKsGo%3D&se=2022-04-11T13%3A28%3A49Z&sp=r", -"created_at":"2022-03-07T11:44:09.627Z", -"updated_at":"2022-03-14T12:23:34.342Z", -"release_type":"Alpha", -"owner":{ -"id":"29536631-e7c3-4035-a78e-4040bc96e9ce", -"avatar_url":null, -"display_name":"Gaurav Raj", -"email":"gauravraj@lambdatest.com", -"name":"gauravraj-lambdatest", -"type":"user" -}, -"azure_subscription":null, -"member_permissions":[ -"manager" -] -}, -{ -"......." -} -] -``` - -## Step 3: Fetch All The Releases For An App - -If you have the {name} and {owner.name} (as received in Step 2), you can use the API call below to retrieve all the Release information for a particular app. - -```js -curl -X GET "https://api.appcenter.ms/v0.1/apps/{owner.name}/{name}/releases" \ -H "Content-Type: application/json" \ --H "X-Api-Token: {your_api_token}" -``` - -Shown below is the response of the above API call. - -```js -[ -{ -"origin": "appcenter", -"id": 1, -"short_version": "2.7.50393-r-2022-02-10", -"version": "50393", -"uploaded_at": "2022-03-07T11:46:06.590Z", -"enabled": true, -"is_external_build": false, -"file_extension": "apk", -"destinations": [ -{ -"id": "00000000-0000-0000-0000-000000000000", -"name": "Collaborators", -"destination_type": "group" -} -], -"distribution_groups": [ -{ -"id": "00000000-0000-0000-0000-000000000000", -"name": "Collaborators" -} -] -}, -{ -"origin": "appcenter", -"id": 4, -"short_version": "2.7.50392-r-2022-01-24", -"version": "50392", -"uploaded_at": "2022-03-17T10:44:11.524Z", -"enabled": true, -"is_external_build": false, -"file_extension": "apk", -"destinations": [ -{ -"id": "00000000-0000-0000-0000-000000000000", -"name": "Collaborators", -"destination_type": "group" -} -], -"distribution_groups": [ -{ -"id": "00000000-0000-0000-0000-000000000000", -"name": "Collaborators" -} -] -} -] -``` - -## Step 4: Fetch The Required App Release Information For An App - -Using the API call below, use the {name} and {owner.name} (received from Step 2), and the id (received from Step 3) to fetch Release information for a specific App: - -```js -curl -X GET "https://api.appcenter.ms/v0.1/apps/{owner.name}/{name}/releases/{id}" \ -H "Content-Type: application/json" \ --H "X-Api-Token: {your_api_token}" -``` - -Shown below is the response of the above API call. - -```js -{ -"app_name": "BCDE-1", -"app_display_name": "BCDE-1", -"app_os": "Android", -"app_icon_url": "https://appcenter-filemanagement-distrib1ede6f06e.azureedge.net/ad7ca44e-0459-459c-9e de-0c72fa78ae03/launcher.png?sv=2019-02-02&sr=c&sig=eFvMnfD%2B9FdYReu4OiG%2BnqvFkTTdXE LlN0dpoAItqig%3D&se=2022-04-14T12%3A03%3A16Z&sp=r", -"is_external_build": false, -"origin": "appcenter", -"id": 1, -"version": "50393", -"short_version": "2.7.50393-r-2022-02-10", -"size": 13833051, -"min_os": "5.0", -"android_min_api_level": "21", -"device_family": null, -"bundle_identifier": "org.wikipedia", -"fingerprint": "6a42317cc91fae15b9847f4bb4745e6f", -"uploaded_at": "2022-03-07T11:46:06.590Z", -"download_url": "https://appcenter-filemanagement-distrib3ede6f06e.azureedge.net/bd52eda2-b239-480b-b2 de-25205f705e63/Wikipedia_v2.7.50393-r-2022-02-10_apkpure.com.apk?sv=2019-02-02&sr=c&s ig=i170%2FkTlpX8146XrKVyxQRLiTUiDoPkfJYGOm%2B1UeZM%3D&se=2022-04-09T09%3A58%3A42Z&sp=r ", -"install_url": "https://appcenter-filemanagement-distrib3ede6f06e.azureedge.net/bd52eda2-b239-480b-b2 de-25205f705e63/Wikipedia_v2.7.50393-r-2022-02-10_apkpure.com.apk?sv=2019-02-02&sr=c&s\nig=i170%2FkTlpX8146XrKVyxQRLiTUiDoPkfJYGOm%2B1UeZM%3D&se=2022-04-09T09%3A58%3A42Z&sp=r ", -"enabled": true, -"fileExtension": "apk", -"release_notes": "", -"package_hashes": [ -"d1c86bcc8784a35a2ee5ada2112cde3056c2b6fb0bc9ca700af5f2826564fc98" -], -"destinations": [ -{ -"destination_type": "group", -"id": "00000000-0000-0000-0000-000000000000", -"name": "Collaborators" -} -], -"destination_type": "group", -"distribution_groups": [ -{ -"id": "00000000-0000-0000-0000-000000000000", -"name": "Collaborators" -} -] -} -``` - -The download_url (For e.g. `https://appcenter-filemanagement-distrib3ede6f06e.azureedge.net/de-25205f705e63/Wikipedia_v2.7.5393-r-2022-02-10_am.apk?sv=2019-&s ig=i170%2FkTlpX%3D&se=202Z&sp=r`) from the above API response will be used to upload your App Release on TestMu AI. - -## Step 5: Upload the Release Version Of An App On TestMu AI - -You can upload the release version of your app on TestMu AI by using the following API call - - -```js -curl -u ":" \ --X POST "https://manual-api.lambdatest.com/app/upload/realDevice" \ -F "url=" \ --F "name=" -``` - -Below is the response of the above cURL request. - -```js -{ -"app_id": "APP100201841649371040473417", -"name": "name", -"type": "ios", -"app_url": "lt://APP100201848749371040473417" -} -``` - -Using the `app_url` obtained in the response, you can execute your Appium tests on TestMu AI. - -## Step - 6 Executing Your Tests - -Once your app is successfully uploaded, you can refer to [Getting Started With Appium Testing](/support/docs/getting-started-with-appium-testing/) for more information on executing Appium tests on TestMu AI. - -> That’s all! In case you have any questions or need any additional information, you could reach out at our 24X7 Chat Support or mail us directly at support@testmuai.com. diff --git a/static/docs/app-center.md b/static/docs/app-center.md deleted file mode 100644 index cdcd07845..000000000 --- a/static/docs/app-center.md +++ /dev/null @@ -1,64 +0,0 @@ -# Configuring App Center on TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This article will guide you on how to configure App Center on TestMu AI & manage your applications. - -## Objective - -By the end of this article, you will be able to : - -- Install apps from App Center by configuring them on TestMu AI. -- Manage synced apps on TestMu AI. - -## What is App Center? - -**Visual Studio App Center** is an integrated mobile development lifecycle solution for iOS, Android, Windows and macOS apps. It brings together multiple services commonly used by mobile developers, including build, test, distribute, monitoring, diagnostics, etc., into one single integrated cloud solution. - -It can be used to decrease the manual effort of uploading the app & **automate** the process. - -## Prerequisites - -Before getting started with the setup, you would need access to the following: - -1. Access to TestMu AI [Real Devices](https://www.testmuai.com/pricing). -2. Access to App Center & permissions to be able to generate API Token on App Center. - -## Generating API Token on App Center - -1. Click on your Profile icon > Account Settings - -2. Click on the "User API Tokens" to generate a token. - -3. Click on "Add new API token" to add the new token. - -4. Now, you’ll get the API token, click on the Copy Button & we’ll use it in Step 2 in TestMu AI Steps. - -## TestMu AI Real-Time App Testing - -1. Click on the "Configure" Button - -2. Paste the API Token we copied 'App Center' & click 'Validate'. - -3. Now Select your Builds on App Center by clicking on "Select Builds" to Select the "Builds" from App Center. - -4. Click on Sync to sync the Builds with TestMu AI. -> **Note:** Only the Builds synced will appear on TestMu AI. In case you missed something, Click on 'Configure' to select other Builds later. - -App Center is now configured to be used with TestMu AI. - -5. Selected Builds from App Center will now appear on TestMu AI. Click on them to toggle them & select the Version from the Build that you’d like to run the test on. - -6. Click on the sync icon & start the test using the selected application. - -## Managing App Center Builds on TestMu AI - -> If an app is not synced, the test will not start. Make sure to sync the app before starting the test. - -1. The app is not Synced. - -2. The app is ready for Testing - -3. Delete the app by hovering over the Ready for Testing Apps. - -> That’s all! In case you have any questions or need any additional information, you could reach out at our **[24X7 Chat Support]** or mail us directly at support@testmuai.com. diff --git a/static/docs/app-distribution.md b/static/docs/app-distribution.md deleted file mode 100644 index a5e8404a1..000000000 --- a/static/docs/app-distribution.md +++ /dev/null @@ -1,113 +0,0 @@ -# App Distribution on Real Devices - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI **App Distribution** lets your team upload Android and iOS app builds from a CI/CD pipeline or the dashboard, organize them by package, and distribute them to internal QA and external beta testers through secure, authenticated download links. Every new build can auto-notify the right testers, and any build can be launched on a real device in one click. - -App Distribution lives in the sidebar under **Real Device → App Management and Distribution** and includes two sections: - -| Section | Purpose | -| --- | --- | -| **App Manager** | Upload, organize, and manage your apps and builds. | -| **Beta Testers** | Manage individual testers and tester teams used for distribution. | - -**Currently in BETA** - -App Distribution is currently in **Beta** and only supports builds signed with an **Enterprise certificate**. We're actively working on expanding the features and gradually rolling it out. If you have any queries, please reach out to our support team. - -## Before You Begin - -- A TestMu AI account with **App Distribution enabled** for your organization. -- An Android (`.apk` / `.aab`) or iOS (`.ipa`) build, no larger than **1 GB**. -- For CI/CD uploads, your TestMu AI **username** and **access key** ([find them on your dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/security)). - -## Manage Beta Testers - -Before distributing builds, set up the people who'll receive them. This is a one-time setup at the organization level, done from the **Beta Testers** section in the sidebar. - -For the full walkthrough of adding individual testers and creating tester teams, see the [Beta Testers Management](/support/docs/beta-testers-management/) guide. - -At a glance: - -- **Beta Testers** are individuals you invite by email. -- **Beta Teams** are groups of testers. Assign a team to an app's sharing list and all current (and future) members of that team get access automatically. - -## Upload Builds - -You can add builds to App Distribution either from the dashboard or directly from your CI/CD pipeline. - -### From the Dashboard - -**Step 1:** Open **App Manager** from the sidebar. - -**Step 2:** Drag and drop your build file into the upload zone, or click **Click to upload** to browse. - -**Supported formats** - -| Platform | Supported format | -| --- | --- | -| Android | `.apk`, `.aab` | -| iOS | `.ipa` | - -**Maximum size per build:** 1 GB - -### From Your CI/CD Pipeline - -Builds produced by Azure DevOps, GitHub Actions, Jenkins, Bitrise, CircleCI, or any other pipeline can be uploaded to App Distribution using the same Upload App API used by App Manager. See [App Testing APIs](/support/docs/app-testing-apis/) for the request format. - -### How Builds Are Organized - -App Distribution auto-detects each build's app from the **package name** (Android) or **bundle ID** (iOS) and groups every version of that app together. You don't need to create apps manually. - -| Upload | Result | -| --- | --- | -| New package name / bundle ID | A new app entry is created. | -| Same package + new version | A new build is added under the existing app. | -| Same package + same version | The previous build is **overwritten**. | - -## Distribution Page and Sharing - -Every app has a **Distribution Page** (a hosted landing page testers visit to download builds) and a **Sharing** configuration (who can access it and who gets notified). Both are configured from the App View tabs. - -### Distribution Page Settings - -Open any app and go to the **Distribution Page** tab. Here you can - -- **Enable or disable** the landing page. When disabled, visitors see a "Landing page closed" message instead of a download. -- **Customize the URL slug**, your landing page lives at a friendly URL such as `https://app-management.lambdatest.com/app-dist/app/android-qa`. -- **Choose the default build** to serve: - - **Latest build** - always serve the most recent upload. - - **Specific version** - pin the page to a particular build (useful for release candidates). -- **Set access control**: - - **Public** - anyone with the link can download. - - **Authenticated** - only testers in the app's sharing list can download from the link received in their email. - -### Sharing Settings - -Open any app and go to the **Sharing** tab. Here you can: - -- **Toggle Auto-Notify** - when ON, every new build upload automatically emails the configured testers. -- **Set the notification scope**: - - **Invited testers only** - notify just the teams and members on this app's sharing list. - - **All testers in the organization** - notify every beta tester in your org. -- **Invite beta testers** - search and add **teams** or **individual members** to the sharing list. Testers added here can download authenticated builds and receive notifications. - -### Notification Emails - -When Auto-Notify is ON and a new build is uploaded, configured testers receive an email with a link to the Distribution Page. - -You can also send notifications manually: - -1. Open the build row in App Manager. -2. Click the **Share** icon. -3. Click **Notify Testers**. - -## Test on Real Devices - -Any build in App Distribution can be launched on a TestMu AI real device in one click. - -**Step 1:** From an app's **Builds** tab, locate the build you want to test. - -**Step 2:** Click the **Run** icon on the build row. The device selection modal opens with the build pre-loaded. - -**Step 3:** Select an Android or iOS real device and start your session. The build is installed automatically and ready to test. diff --git a/static/docs/app-manager.md b/static/docs/app-manager.md deleted file mode 100644 index 37a6b4f07..000000000 --- a/static/docs/app-manager.md +++ /dev/null @@ -1,98 +0,0 @@ -# App Manager on Real Devices - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The TestMu AI App Manager is a tool within the Real Device App Testing section. It lets you easily upload, manage, and test different versions of your mobile apps. This guide explains how to access the App Manager, upload apps, search and filter them, and run any version on Real Devices. - -**Currently in BETA** - -To enable it for your organization, please contact us via **24×7 chat support** or you can also drop a mail to **support@testmuai.com**. - -## Accessing App Manager -To access the App Manager in TestMu AI: - -**Step 1:** Navigate to **Real Device** under **App Testing** in the TestMu AI dashboard. - -**Step 2:** Click on the **App Manager** CTA. - -**Step 3:** This will open the **App Manager** in a new tab. - -## Uploading Apps on App Manager - -In the **App Manager** window, users can **upload mobile app** files Either by click **“Click to upload”** or **drag and drop your file** into the upload box. - -**Supported formats :** -| Platform | Supported Format | -|----------|---------------------------| -| Android | .apk, .aab | -| iOS | .ipa | - -**Maximum App upload limit (Per batch):** 10 - -**Maximum size per app:** 1 GB - -## Searching and Filtering Apps -Once your apps are uploaded, you can easily search and filter them using the tools provided: - -- **Search Bar**: Search by: - - - **App Name** – the name of the app you uploaded. - - - **Package Name** – the identifier for all versions of an Android app. - - - **Bundle ID** – the identifier for all versions of an iOS app. - -- **OS Filter**: - The OS Filter in the App Manager allows you to easily sort and view apps based on their operating system. This is especially useful when managing a mix of Android and iOS builds. - - You can filter the app list to show: - - - **Android** apps only - - - **iOS** apps only - - - Both **Android** and **iOS** apps together - -## Version Control - -**TestMu AI App Manager** offers a convenient version control system for managing multiple versions of the same application: -- When you click on an app name in the file bundle section, a **App Card** opens. -- This **App Card** displays all versions of the selected app that have been uploaded. - -- The **most recent version** appears at the top, with older versions listed below. -- Users can: - - **Search** through the uploaded App versions. - - Run any version by clicking the **Green Run** button or recent version by clicking on **Run App**. - -### Release Notes for Versions -For each app version, you can add multiline release notes to help differentiate between multiple builds. -- To add release notes, open the three dots menu on the desired version in the App Card and click Version Details. - -- Inside Version Details, you’ll see an option to enter release notes. - -- Once added, a notes icon appears on the version. On hover, the release note is shown directly from the App Card. - -- Maximum release notes length is upto 500 characters. -- On hover, only the first 100 characters are displayed, followed by ... if the text is longer. -- Only the owner of the version has permission to add or edit release notes, Other users can only view them. - -## Launching an App on a Real Device -Once a specific version is selected: - - **Step 1:** Click the **Run App** button beside the version you want to test. - - **Step 2** This action redirects you to the **Real Device App Testing** screen - - **Step 3** A list of available Real devices is displayed. Click on the device you want to use for testing. - - **Step 4** Click on **Start button**. - - **Step 5** This will **launch your Session**. - -## Use Case - -- Maintain multiple versions of the same app for **consistent version control** -- Validate app behavior across different OS versions and device models -- Compare versions for **debugging** or **verifying fixes** -- Share builds with team members or stakeholders for **review and collaboration** -- **Launch and test** any uploaded version directly on real devices -- Maintain version history for **compliance** and **QA tracking** diff --git a/static/docs/app-settings.md b/static/docs/app-settings.md deleted file mode 100644 index 994c11a3d..000000000 --- a/static/docs/app-settings.md +++ /dev/null @@ -1,27 +0,0 @@ -# App Settings in Real Device Cloud - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -App settings play a crucial role in testing the environment for mobile applications. These settings, provided by TestMu AI, offer developers a range of capabilities to enhance testing procedures and ensure comprehensive validation of their apps functionality. - -## Image Injection - -Enable the Image Injection feature to capture images, QR codes, and barcode scans seamlessly within your app using TestMu AI devices. Our approach utilizes Sensor Instrumentation to seamlessly integrate your app with various mobile sensors, such as the camera. By enabling Image Injection for a specific session, TestMu AI seamlessly injects camera code modules into your app, effectively mocking or overriding the Android or iOS SDK used in your app. - -Check out our detailed [support documentation](/support/docs/camera-image-injection/) to learn in detail about the image injection feature. - -## Biometric Authentication - -Enable this setting to effortlessly test your biometric authentication-reliant applications on designated remote TestMu AI devices. This functionality enables the emulation of diverse biometric authentication techniques such as fingerprint scanning, facial recognition and others. Leveraging this feature ensures thorough validation of your applications' security and functionality across a spectrum of realistic usage scenarios. - -Check out our detailed [support documentation](/support/docs/biometric-authentication/) to learn in detail about the Biometric supported APIs and much more. - -## Disable Screenshot Block (only Android) - -Enable this setting to conduct uninterrupted testing of your application's performance, even when screenshot capture is restricted within your app. This tool facilitates testing in both app-live and app automation scenarios, ensuring seamless evaluation of your applications. - -Check out our detailed [support documentation](/support/docs/disable-screenshot-block/) to learn in detail about disable screenshot block feature. - -## Default Network Logs - -Enable this feature to seamlessly initiate network log capturing at the beginning of each session within your application. By enabling this setting at the app level, you prioritize the logging of network activities over device logs, ensuring comprehensive monitoring of network interactions right from the start. diff --git a/static/docs/app-testing-apis.md b/static/docs/app-testing-apis.md deleted file mode 100644 index c9b7ceae7..000000000 --- a/static/docs/app-testing-apis.md +++ /dev/null @@ -1,173 +0,0 @@ -# Commonly Used APIs For Mobile App Testing - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -In this documentation, we look at some APIs that will help you optimize your mobile app testing workflow. If you are performing live or automated app testing, you can use these APIs in your scripts to perform various actions. - -## Fetching The Devices Available for Testing - -To fetch the Devices that are available for running Tests. - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X GET "https://mobile-api.lambdatest.com/mobile-automation/api/v1/list?region=us" -`} - -| PARAMETER | EXAMPLE | DESCRIPTION | -|-----------------|-------------|------------| -| `region` | `region=us` OR `region=ap` OR `region=eu` | Filter devices by region. `us` - USA `ap` - Asia Pacific `eu` - Europe | -| `os` | `os=android` OR `os=ios` | Filter devices by platform. Omit to get both Android and iOS devices. | - -## Uploading your Application - -| PARAMETER | EXAMPLE | DESCRIPTION | -|-----------------|-------------|------------| -| `custom_id` | `-F "custom_id="Proverbial_1.0"` | You do not have to remember the `app_URL` and only use the `custom_id` to run your automation on the same app. | -| `storage` | `-F "storage=file"` DEFAULT: `url` | Used to change the way TestMu AI stores the link. Used when we Upload using App URL | -| `visibility` | `-F "visibility=team"` DEFAULT: `individual` | Used to change the visibility of the application being uploaded. Once the app is uploaded using the `team`, everyone in the organisation can use the same URL to run the tests. | - -**Using App File:** - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "appFile=@"/Users/macuser/Downloads/Appname.apk"" -F "name="appname"" -`} - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/virtualDevice" -F "appFile=@"/Users/macuser/Downloads/Appname.apk"" -F "name="appname"" -`} - -**Using App URL:** - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "url=https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk" -F "name=Proverbial_App" -F "custom_id=sampleName" -F "storage=url" -F "visibility=individual"`} - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/virtualDevice" -F "url=https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk" -F "name=Proverbial_App" -F "custom_id=sampleName" -F "storage=url" -F "visibility=individual"`} - -- If you do not have any **.apk** or **.ipa** file, you can run your sample tests on TestMu AI by using our sample :link: [Android app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk) or sample :link: [iOS app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_ios.ipa). -- Response of above cURL will be a **JSON** object containing the `App URL` of the format - ``lt://APP123456789123456789`` - -## Fetching your Applications for Real Devices - - {`curl --location --request GET "https://${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}@manual-api.lambdatest.com/app/data?type=android&level=user"`} - - {`curl --location --request GET "https://${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}@manual-api.lambdatest.com/app/data?type=ios&level=user"`} - -## Fetching your Applications for Virtual Devices - - {`curl --location --request GET "https://${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}@manual-api.lambdatest.com/app/data?type=emulator&level=user"`} - - {`curl --location --request GET "https://${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}@manual-api.lambdatest.com/app/data?type=simulator&level=user"`} - -Shown below is the response to the above cURL request. - -{`{ - "metaData": { - "type": "ios", - "total": 1 - }, - "data": [ - { - "app_id": "APP100245789181570497850", - "name": "proverbial_ios.ipa", - "type": "ios", - "updated_at": "2022-05-10T11:19:30.000Z", - "shared": false, - "source": "web-client" - } - ] -}`} - -## Deleting your Application - -To delete your uploaded apps, run the below cURL command. - -{`curl --location --request DELETE "https://${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}@manual-api.lambdatest.com/app/delete" \ ---header 'Content-Type: application/json' \ ---data-raw '{ - "appIds" : "APPID1,APPID2" -}' -`} - -{`curl --location --request DELETE "https://${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}@manual-api.lambdatest.com/app/delete" \ ---header 'Content-Type: application/json' \ ---data-raw '{ - "appIds" : "APPID1,APPID2" -}' -`} - -Shown below is the response to the above cURL request. - -{`{ - "message": "Deleted successfully." -}`} - -## Processing check for your Application - -To unlock features such as network logs, image injection, and screenshotunblock feature for your application, app needs to undergo a processing phase. This processing takes a few minutes after the application is uploaded. You can verify if the processing is complete before running your automation script using the following API. - -{`curl --location --request POST 'https://mobile-api.lambdatest.com/mobile-automation/api/v1/fetchpatchedapkurl' \ ---header 'Authorization: Basic ' \ ---header 'Content-Type: application/json' \ ---data-raw '{ - "appId": "APP10160161171698993659206876", - "imageInjectionEnabled": true, - "screenshotUnblockEnabled": true -}'`} - -{`curl --location --request POST 'https://mobile-api.lambdatest.com/mobile-automation/api/v1/fetchpatchedapkurl/virtualDevice' \ ---header 'Authorization: Basic ' \ ---header 'Content-Type: application/json' \ ---data-raw '{ - "appId": "APP10160161171698993659206876", - "imageInjectionEnabled": true, - "screenshotUnblockEnabled": true -}'`} - -The payload allows you to check the processing status for specific features. If the **patched_url** is empty, the processing is still in progress. To check if the processing for image injection or screenshot unblock is complete, pass either **imageInjectionEnabled** or **screenshotUnblockEnabled** as `true` based on the feature you are testing. - -{`{ - "data": { - "imageinjection_ready": false, //current processing status - "patched_url": "", - "screenshotunblock_ready": false, //current processing status - "status": "success" - }, - "status": "success" -}`} - -## Fetching the concurrency details - -To get your concurrency details, run the below cURL command. - -{`curl -u "${YOUR_LAMBDATEST_USERNAME()}:${YOUR_LAMBDATEST_ACCESS_KEY()}" -X GET "https://mobile-api.lambdatest.com/mobile-automation/api/v1/org/concurrency"`} - -Shown below is the response to the above cURL request. - -{`{ - "data":{ - "max_concurrency":10, - "max_queue":150, - "queued":0, - "running":0 - }, - "status":"success" -}`} - -## Generate publicly shareable build and test links - -To generate public shareable links, run the below cURL command. - -{`curl 'https://api.lambdatest.com/lshs/api/v1.0/share-item/generate-sharable-link' \ - -H 'Content-Type: application/json' \ - -u "${YOUR_LAMBDATEST_USERNAME()}:${YOUR_LAMBDATEST_ACCESS_KEY()}" \ - --data-raw '{"entityIds":["buildID"],"entityType":"App Automation Build","expiresAt":7}'`} - -{`curl 'https://api.lambdatest.com/lshs/api/v1.0/share-item/generate-sharable-link' \ - -H 'Content-Type: application/json' \ - -u "${YOUR_LAMBDATEST_USERNAME()}:${YOUR_LAMBDATEST_ACCESS_KEY()}" \ - --data-raw '{"entityIds":["testID"],"entityType":"App Automation Test","expiresAt":7}'`} - -| PARAMETER | DESCRIPTION | -|-----------|-------------| -| `entityIds` | Pass the build ID or test ID | -| `entityType` | Use `App Automation Build` for builds or `App Automation Test` for tests | -| `expiresAt` | Number of days after which the shareable link will expire (7, 15, or 30) | - -The shareable links are valid for a period of 7, 15, or 30 days, after which they will expire. - ->That’s all! In case you have any questions or need any additional information, you could reach out at our **24X7 Chat Support** or mail us directly at support@testmuai.com. diff --git a/static/docs/app-testing-on-real-devices.md b/static/docs/app-testing-on-real-devices.md deleted file mode 100644 index e8d44096e..000000000 --- a/static/docs/app-testing-on-real-devices.md +++ /dev/null @@ -1,25 +0,0 @@ -# Getting Started with Real Device App Testing - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Real device app testing is the process of testing a mobile application to ensure that their functionality and usability is not comprised when installed across multiple Android and iOS devices. With TestMu AI, you can test mobile applications manually to ensure they perform seamlessly across 3000 + real mobile devices. - -**Step 1:** Login to your TestMu AI account. Go to **App Testing** under **Real Device** from the left sidebar. - -**Step 2**: Choose the OS version of your choice. ( Android or iOS ) - -**Step 3**: You can upload an application directly from your system or by entering a valid public URL. Once the app gets uploaded you can select the app from the **applist** or use the search bar to find a specific application. The uploading time may vary depending on the size of the application. - -In the case of **Android**, you can install from Play Store or App Center. - -In the case of **iOS**, you can install from the App Store, Test Flight or from App Center. - -**Step 4**: You can select your preferred region from the **Region** dropdown. Choose from the available regions i.e., North America, Europe, Asia, or select All. If you would like to choose devices from specific regions, please reach out to [support@testmuai.com](mailto:support@testmuai.com) to get the option enabled. - -**Step 5**: Select the **Tunnel** option in the top right corner (optional) to test your locally hosted and privately hosted websites. - -Check out our detailed [support documentation](/support/docs/testing-locally-hosted-pages/) to learn how to activate the LT Tunnel. - -**Step 6**: Select the preferred **Brand**, **Device Name** and **OS Version** to easily navigate to your required device. You can also use the search bar to select your devices. - -**Step 7**: Click on **Start** to run your test session. diff --git a/static/docs/appium-androidtv.md b/static/docs/appium-androidtv.md deleted file mode 100644 index fe03782db..000000000 --- a/static/docs/appium-androidtv.md +++ /dev/null @@ -1,89 +0,0 @@ -# Android TV Automation - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Tutorial To Run Your First Test On TestMu AI - -As smart TVs continue to grow in popularity, relying solely on manual testing is no longer sufficient. Automating Android TV testing provides broader device coverage, enhances user experience by validating remote-based navigation and voice commands, and eliminates repetitive manual tasks. It also helps uncover issues earlier in the development cycle, ensuring a smooth and consistent experience across different Android TV models. -In this guide, you will learn how to set up and run your **Android TV** automation testing scripts with **Appium** on the **TestMu AI Real Device Cloud platform**. - -> Android TV support is available only with Private Device Plan. -To enable it for your organization, please contact us via **24×7 chat support** or you can also drop a mail to **support@testmuai.com**. - -## Objective - -By the end of this topic, you will be able to: - -1. Set up an environment for testing your Apps using **Android TV** with **Appium**. -2. Understand and configure the core capabilities required for your Appium test suite. -3. Explore the advanced features of TestMu AI. - -## Prerequisites - -Before you can start performing App automation testing with Appium, you would need to follow these steps: - -- You have access to TestMu AI username and accessKey. If you have not registered yet, you can do the same by visiting our website. You will be able to access the credentials in the [TestMu AI Profile](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile) -- Install the latest Python build from the [official website](https://www.python.org/downloads/). We recommend using the latest version. -- Make sure **pip** is installed in your system. You can install **pip** from [pip documentation](https://pip.pypa.io/en/stable/installation/). - -## Supported Models - -| Device Model | Platform Version | -|----------------------|------------------| -| Nvidia Shield TV | Android 11 | - -## Run Your First Test - -### 1. Set Up Authentication - -Make sure you have your TestMu AI credentials with you to run test automation scripts on TestMu AI. To obtain your access credentials, [purchase a plan](https://billing.lambdatest.com/billing/plans) or access the [Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://appautomation.lambdatest.com/). Then, set TestMu AI `Username` and `Access Key` in environment variables with following commands. - - {`export LT_USERNAME=${ YOUR_LAMBDATEST_USERNAME()} \\ -export LT_ACCESS_KEY=${ YOUR_LAMBDATEST_ACCESS_KEY()}`} - - {`set LT_USERNAME=${ YOUR_LAMBDATEST_USERNAME()} \` -set LT_ACCESS_KEY=${ YOUR_LAMBDATEST_ACCESS_KEY()}`} - -### 2. Upload your application -Upload your **Android TV** application (.ipa file) to the TestMu AI servers using our **REST API**. You need to provide your **Username** and **AccessKey** in the format `Username:AccessKey` in the **cURL** command for authentication. Make sure to add the path of the **appFile** in the cURL request. Here is an example cURL request to upload your app using our REST API: - - **Using App File from System:** - -{`curl -u "$LT_USERNAME:$LT_ACCESS_KEY" \ --X POST "https://manual-api.lambdatest.com/app/upload/realDevice" \ --F "appFile=@smarttube_stable.apk" \ --F "name=androidtv_app" \ --F "type=androidtv" -`} - -### 3. Configure Desired Capabilities - -In your automation script, set up the **platform capability** to specify that you are testing on an Android TV device. - -```java -DesiredCapabilities capabilities = new DesiredCapabilities(); -HashMap ltOptions = new HashMap(); - -ltOptions.put("build", "Android TV Demo"); -ltOptions.put("name", "Android TV Demo"); -ltOptions.put("deviceName", "Nvidia Shield TV"); -ltOptions.put("platformVersion", "11"); -ltOptions.put("platformName", "androidtv"); -ltOptions.put("isRealMobile", true); -ltOptions.put("privateCloud", true); -ltOptions.put("w3c", true); -ltOptions.put("automationName", "UiAutomator2"); -ltOptions.put("app", "lt://APP_ID"); -/* Optional Capability */ -/* ltOptions.put("appPackage", "org.smarttube.stable"); */ - -capabilities.setCapability("LT:Options", ltOptions); -``` - -Shown below is an execution snapshot from TestMu AI Dashboard: - -## Additional Links - -- [Advanced Configuration for Capabilities](/support/docs/desired-capabilities-in-appium/) -- [How to test locally hosted apps](/support/docs/testing-locally-hosted-pages/) -- [How to integrate TestMu AI with CI/CD](/support/docs/integrations-with-ci-cd-tools/) diff --git a/static/docs/appium-app-performance-analytics.md b/static/docs/appium-app-performance-analytics.md deleted file mode 100644 index 6c2d24c3e..000000000 --- a/static/docs/appium-app-performance-analytics.md +++ /dev/null @@ -1,120 +0,0 @@ -# Application Performance Analytics - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -App performance is critical factor for user satisfaction and adoption. Identifying and resolving bottlenecks early is essential for delivering a high-quality experience but it requires constant monitoring and optimization across various device metrics. TestMu AI’s App Profiling feature helps you to detect and optimize performance issues before release by providing real-time insights into key metrics like CPU usage, memory consumption, and network activity on real devices. - -You can enable performance tracking in your existing automation tests with the following capability: - -```javascript -"appProfiling" : true; -``` - -> This feature is available for Appium tests on **iOS and Android (version 9+)**. For full functionality, ensure the `resignApp` capability is set to `true`. - -## Basic Summary -Here is a description of each metric from the App Profiling summary: - -- **Cold Startup Time :** Time taken by the app to launch from a complete shutdown or a non-running state. A lower cold startup time improves user experience, as it reduces wait time when the app is opened for the first time. -- **Hot Startup Time :** Time taken by the app to resume from a background state. A fast hot startup ensures that users can quickly switch between apps without delays. -- **Maximum CPU Utilisation :** It indicates the peak percentage of CPU consumed by the app during its execution. High CPU utilization can affect other processes and contribute to battery drain, so it's important to keep it optimized. -- **Average CPU :** Average CPU utilization gives a more balanced view of the app’s typical CPU usage during a session. This value helps assess the app’s efficiency in resource usage over time. -- **Maximum Memory Usage :** It refers to the highest amount of RAM consumed by the app during the session. Monitoring this metric helps developers ensure that their app doesn't consume excessive memory, which can lead to slower performance or crashes on low-memory devices. -- **Average Memory Usage :** Average memory usage shows the typical amount of memory used by the app. This value is useful for identifying memory leaks and optimizing the overall resource consumption of the application. -- **Maximum Disk Usage :** Highest amount of storage consumed by the app on the device. Keeping disk usage minimal helps improve app performance and ensures better storage management for end users. -- **Average Disk Usage :** It reflects the typical amount of storage space used by the app over time. This metric is important for identifying opportunities to reduce the app’s footprint on the device's storage. -- **Average Frame Rate :** The frame rate measures the smoothness of the app’s graphical rendering. A higher frame rate (measured in frames per second or fps) indicates smoother animations and a better visual experience. A rate close to 60 fps or higher is typically desired for smooth app performance. -- **Network Download/Upload :** This metric tracks the amount of data downloaded and uploaded by the app during the session. Monitoring network usage helps developers optimize how the app handles data transmission to ensure it uses bandwidth efficiently. -- **Application Not Responding :** ANR events occur when the app becomes unresponsive and prompts the user to wait or close the app. A value of zero indicates the app has not faced any ANR issues, which is crucial for maintaining a stable and responsive app. You can also ANR Dumps, and Data logs. -- **App Crashes :** This metric tracks the number of times the app has crashed during the session. Zero crashes indicate that the app is stable, ensuring a reliable user experience. - -## Key Performance Metrics - -### 1. CPU Consumption -The CPU handles all operations within a mobile device. High CPU usage impacts overall system performance and drains the battery faster. By monitoring CPU consumption, you can pinpoint which features are resource-heavy and optimize them for better performance. TestMu AI provides two CPU metrics in a time-series format: - -- **System CPU Usage (%):** Total CPU usage across the system. -- **App CPU Usage (%):** CPU usage specific to your app. - -Using these insights, you can minimize CPU-intensive operations, delivering a smoother user experience and enhancing device efficiency. - -### 2. Memory Consumption -RAM is a critical resource for mobile apps. Excessive memory consumption can lead to memory leaks, degraded performance, and application crashes. - -TestMu AI offers three memory metrics: -- **System Memory Usage (MB):** Overall memory consumption during the session. -- **App Memory Usage (MB):** Memory consumed by your application. -- **Available Memory (MB):** Remaining memory available for use. - -By tracking these, you can optimize memory utilization and prevent crashes caused by memory leaks. - -### 3. Disk Usage -The Disk Usage metric provides insights into how much disk space is being utilized by both the system and the application during a session. Monitoring disk usage is crucial for optimizing storage consumption and ensuring efficient app performance, especially on devices with limited storage capacity. - -- **System Disk (MB):** Indicates the total amount of disk space used by the entire system, including the operating system and other background processes, displayed in megabytes (MB). -- **App Disk (MB):** Shows the disk space specifically consumed by your application (App Under Test) during the session, helping you track how much space your app is utilizing and whether it might cause storage-related issues. - -### 4. Rendering (Frame Rate) -Smooth rendering directly affects how responsive your app feels to users. Rendering performance is a typical performance issue that any app faces. It is measured at different stages in the app’s lifecycle to ensure that users do not have a bad experience with the app. - -TestMu AI measures your app's Frames Per Second (FPS) across its lifecycle, which helps you identify performance bottlenecks in rendering and improve user experience. - -- **Frames Per Second (FPS):** Real-time measurement of rendering performance throughout the session. - -### 5. Network Usage -Understanding how your app utilizes network resources is essential for optimizing data transfer and ensuring efficient performance. - -TestMu AI tracks: -- **Download Size (MB):** Amount of data downloaded during the session. -- **Upload Size (MB):** Amount of data uploaded during the session. - -This helps you to monitor: - -- **Data Efficiency:** Optimize data usage to enhance performance and reduce costs. -- **Performance Optimization:** Identify and mitigate high network usage areas that may affect app responsiveness. - -### 6. Battery Consumption -Apps that consume excessive battery negatively affect user experience. With TestMu AI, you can monitor how your app impacts battery drain, identify features that are energy hogs, and optimize them. This will help extend battery life for devices running your app. - -- **Battery Drain Rate:** Amount of battery consumed while using the app. - -Battery consumption data is only displayed for Android applications. This feature is not available for iOS applications. - -### 7. Temperature Monitoring -High device temperature can degrade performance and harm the user experience. TestMu AI lets you monitor battery temperature during app usage, helping you optimize features causing excessive heat generation and reducing the risk of device overheating. - -- **Temperature:** Temperature of the device battery during app usage. - -### 8. ANR (Application Not Responding) - -Sometimes, your application stops responding, and you receive a pop-up to wait or close the app. Application freezes and crashes are frustrating for users. TestMu AI provides detailed logs on ANR events, that helps you diagnose the root causes and eliminate them. - -> Currently, ANR monitoring is available for **Android devices** only. - -Metrics Provided: - -- **ANR Count:** Number of times the app encountered ANR during the session. -- **Log Dumps:** Detailed logs for each ANR event to aid in troubleshooting. - -### 9. Application Start-up Time -Application start-up time significantly impacts the first impression and overall user experience. Slow startup times create a poor first impression. TestMu AI tracks both cold and hot startup times for Android devices, giving you the data needed to reduce delays and deliver a faster app launch. - -## Fetch your App Profiling details via API -TestMu AI provides an API that allows you to programmatically access detailed app profiling metrics captured during your test sessions. By integrating this API into your CI/CD pipelines or automation frameworks, you can fetch real-time data such as CPU usage, memory consumption, disk usage, and network activity for deeper analysis. This enables continuous monitoring and performance optimization without manual intervention. - -To retrieve profiling details, simply make a request to the TestMu AI API using your **authentication token (username and access key)** and the relevant **session ID**. The response will include comprehensive metrics in JSON format, allowing easy integration with other tools or reporting systems for further insights. - - {`https://${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}@mobile-api.lambdatest.com/mobile-automation/api/v1/sessions/SESSION_ID/log/appmetrics`} - -**Step 1:** Run your app automation test on the Real Device Cloud. - -**Step 2:** Get your **SESSION_ID** from the **Meta Data** section. - -**Step 3:** Replace the `YOUR_SESSION_ID` parameter in the given API with your actual session id. - - {`https://${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}@mobile-api.lambdatest.com/mobile-automation/api/v1/sessions/YOUR_SESSION_ID/log/appmetrics`} - -**Step 4:** Now go to any API testing platform, and pass the above URL (GET Method) to fetch the relevant results. - -> Got any questions? -> Please reach out at our **24x7 Chat Support** or you could also mail us at [support@testmuai.com](https://support.lambdatest.com/). diff --git a/static/docs/appium-appletv.md b/static/docs/appium-appletv.md deleted file mode 100644 index b110f8e04..000000000 --- a/static/docs/appium-appletv.md +++ /dev/null @@ -1,226 +0,0 @@ -# Apple TV With Appium - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Tutorial To Run Your First Test On TestMu AI - -As smart TVs continue to grow in popularity, relying solely on manual testing is no longer sufficient. Automating Apple TV testing provides broader device coverage, enhances user experience by validating remote-based navigation, and eliminates repetitive manual tasks. It also helps uncover issues earlier in the development cycle, ensuring a smooth and consistent experience across different Apple TV models. -In this guide, you will learn how to set up and run your **Apple TV** automation testing scripts with **Appium** on the **TestMu AI Real Device Cloud platform**. - -> To enable it for your organization, please contact us via **24×7 chat support** or you can also drop a mail to **support@testmuai.com**. - -## Objective - -By the end of this topic, you will be able to: - -1. Set up an environment for testing your Apps using **Apple TV** with **Appium**. -2. Understand and configure the core capabilities required for your Appium test suite. -3. Explore the advanced features of TestMu AI. - -**Sample repo** - -All the code samples in this documentation can be found on **TestMu AI's Github Repository**. You can either download or clone the repository to quickly run your tests. View on GitHub - -## Prerequisites - -Before you can start performing App automation testing with Appium, you would need to follow these steps: - -- You have access to TestMu AI username and accessKey. If you have not registered yet, you can do the same by visiting our website. You will be able to access the credentials in the [TestMu AI Profile](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile) -- Install the latest Python build from the [official website](https://www.python.org/downloads/). We recommend using the latest version. -- Make sure **pip** is installed in your system. You can install **pip** from [pip documentation](https://pip.pypa.io/en/stable/installation/). - -## Supported Models ## - -| Device | OS Version | -|-----------------------------|------------------------| -| Apple TV | 18 | -| Apple TV 4K | 18 | - -## Run your first test - -### 1. Upload your application -Upload your **Apple TV** application (.ipa file) to the TestMu AI servers using our **REST API**. You need to provide your **Username** and **AccessKey** in the format `Username:AccessKey` in the **cURL** command for authentication. Make sure to add the path of the **appFile** in the cURL request. Here is an example cURL request to upload your app using our REST API: - - **Using App File from System:** - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "appFile=@"/Users/macuser/Downloads/apple-tv-sample-app.ipa"" -F "name="appletv_app"" -`} - -- If you do not have any **.ipa** file, you can run your sample tests on TestMu AI by using our sample :link: [AppleTV app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/apple-tv-sample-app.ipa). -- Response of above cURL will be a **JSON** object containing the `APP_URL` of the format - ``lt://APP123456789123456789`` and will be used in the next step. - -### 2. Clone the sample project - -1. Clone the TestMu AI’s [LT-appium-appletv](https://github.com/LambdaTest/LT-appium-appletv) and navigate to the code directory as shown below: - -```bash -git clone https://github.com/LambdaTest/LT-appium-appletv -cd LT-appium-appletv -``` - -### 3. Set up your authentication - -Make sure you have your TestMu AI credentials with you to run test automation scripts on TestMu AI. To obtain your access credentials, [purchase a plan](https://billing.lambdatest.com/billing/plans) or access the [Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://appautomation.lambdatest.com/). Then, set TestMu AI `Username` and `Access Key` in environment variables with following commands. - - {`export LT_USERNAME=${ YOUR_LAMBDATEST_USERNAME()} \\ -export LT_ACCESS_KEY=${ YOUR_LAMBDATEST_ACCESS_KEY()}`} - - {`set LT_USERNAME=${ YOUR_LAMBDATEST_USERNAME()} \` -set LT_ACCESS_KEY=${ YOUR_LAMBDATEST_ACCESS_KEY()}`} - -### 4. Write your automation script - -An automation script for the sample application available above has been provided below. Ensure to update the `APP_URL`, `username` and `accessKey` in the code scripts before running the tests. - -```python title="appletv.py" -from appium import webdriver -import os -import time -from appium.webdriver.common.appiumby import AppiumBy - -def getCaps(): -ltOPtions = { -"deviceName" : "Apple TV", -"platformVersion" : "18", -"platform" : "tvos", -"isRealMobile": True, -"build": "Apple TV Testing", -"video": True, -"app":"APP_URL", #Enter app url here -"network": True, -"devicelog": True, -"visual" : True -} - -desired_caps = {"lt:options": ltOPtions} - -return desired_caps - -def runTest(): -if os.environ.get("LT_USERNAME") is None: -# Enter LT username below if environment variables have not been added -username = "username" -else: -username = os.environ.get("LT_USERNAME") -if os.environ.get("LT_ACCESS_KEY") is None: -# Enter LT accesskey below if environment variables have not been added -accesskey = "accesskey" -else: -accesskey = os.environ.get("LT_ACCESS_KEY") - -# grid url -gridUrl = "mobile-hub.lambdatest.com/wd/hub" - -# capabilities -desired_cap = getCaps() -url = "https://"+username+":"+accesskey+"@"+gridUrl - -print("Initiating remote driver on platform: "+desired_cap["lt:options"]["deviceName"]) -driver = webdriver.Remote( -desired_capabilities=desired_cap, -command_executor= url -) - -# run test -print(driver.session_id) - -element = driver.switch_to.active_element -active = element.get_attribute('label') - -if active != "Keyboard": -element = driver.find_element(by=AppiumBy.ACCESSIBILITY_ID, value="EnterWebsiteName") -element.click() -time.sleep(1) - -element = driver.find_element(by=AppiumBy.ACCESSIBILITY_ID, value="EnterWebsiteName") -element.clear() -time.sleep(1) - -element = driver.find_element(by=AppiumBy.CLASS_NAME, value="//XCUIElementTypeApplication[@name=\"chief.qaTvTestApp\"]/XCUIElementTypeWindow/XCUIElementTypeOther[2]/XCUIElementTypeOther/XCUIElementTypeOther[3]/XCUIElementTypeOther/XCUIElementTypeOther[2]/XCUIElementTypeOther/XCUIElementTypeOther[1]/XCUIElementTypeOther/XCUIElementTypeOther/XCUIElementTypeOther[2]/XCUIElementTypeTextView/XCUIElementTypeOther/XCUIElementTypeOther") -element.send_keys("https://ifconfig.me") - -driver.execute_script('mobile: pressButton', { 'name': 'Right' }) -driver.execute_script('mobile: pressButton', { 'name': 'Right' }) -driver.execute_script('mobile: pressButton', { 'name': 'Right' }) -driver.execute_script('mobile: pressButton', { 'name': 'Right' }) -driver.execute_script('mobile: pressButton', { 'name': 'Right' }) -driver.execute_script('mobile: pressButton', { 'name': 'Right' }) -driver.execute_script('mobile: pressButton', { 'name': 'Right' }) -driver.execute_script('mobile: pressButton', { 'name': 'Right' }) -driver.execute_script('mobile: pressButton', { 'name': 'Right' }) -driver.execute_script('mobile: pressButton', { 'name': 'Right' }) -driver.execute_script('mobile: pressButton', { 'name': 'Down' }) -driver.execute_script('mobile: pressButton', { 'name': 'Down' }) -driver.execute_script('mobile: pressButton', { 'name': 'Down' }) - -element = driver.switch_to.active_element -element.click() -time.sleep(1) - -driver.execute_script('mobile: pressButton', { 'name': 'Right' }) -element = driver.switch_to.active_element -element.click() -time.sleep(3) -driver.execute_script("lambda-status=passed") -driver.quit() - -if __name__ == "__main__": -runTest() -``` - -### Configure the test capabilities - -You can update your custom capabilities in test scripts. In this sample project, we are passing platform name, platform version, device name and app url (generated earlier) along with other capabilities like build name and test name via capabilities object. The capabilities object in the sample code are defined as: - -**Supported Capabilities:** -Same as iOS - -**Platform:** -`"tvOS"` - -```python title="appletv.py" -def getCaps(): -ltOPtions = { -"deviceName" : "Apple TV", -"platformVersion" : "18", -"platform" : "tvos", -"isRealMobile": True, -"build": "Apple TV Testing", -"video": True, -"app":"APP_URL", #Enter app url here -"network": True, -"devicelog": True, -"visual" : True -} - -desired_caps = {"lt:options": ltOPtions} -``` - -**Note** - -- You must add the generated **APP_URL** to the `"app"` capability in the config file. -- You can generate capabilities for your test requirements with the help of our inbuilt **Capabilities Generator**.For more details, please refer to our guide on [Desired Capabilities in Appium](/support/docs/desired-capabilities-in-appium/). - -### 5. Execute your test case - -1. Install the required packages from the cloned project directory: - -```bash -pip install -r requirements.txt -``` - -2. Run the following command in the directory where your project has been saved to execute your build. - -```python -python appletv.py -``` - -If you are unable to run the automation script with the above mentioned commands try **'python3'** command except for **'python'**. - -Your test results would be displayed on the test console (or command-line interface if you are using terminal/cmd) and on the [TestMu AI App Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://appautomation.lambdatest.com/build). - -## Additional Links - -- [Advanced Configuration for Capabilities](/support/docs/desired-capabilities-in-appium/) -- [How to test locally hosted apps](/support/docs/testing-locally-hosted-pages/) -- [How to integrate TestMu AI with CI/CD](/support/docs/integrations-with-ci-cd-tools/) diff --git a/static/docs/appium-csharp-nunit.md b/static/docs/appium-csharp-nunit.md deleted file mode 100644 index d83b39a56..000000000 --- a/static/docs/appium-csharp-nunit.md +++ /dev/null @@ -1,351 +0,0 @@ -# Appium with NUnit - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -In this documentation, you will learn how to trigger a automation script of **NUnit** for application testing with **Appium** on TestMu AI, set the [**desired capabilities**](/support/docs/desired-capabilities-in-appium/) for appium testing, and other advanced features of TestMu AI. - -## Prerequisites - -- Your TestMu AI [Username and Access key](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/security). -- Install the MS Visual Studio 2013 or later version for C#. We recommend using the latest version. -- Install the framework [NUnit3.0](https://nunit.org/), and [NuGet](https://www.nuget.org/downloads) plugin for Visual Studio and add the NuGet CLI executable installed in your path. -- Access to an Android app (.apk or .aab file) or an iOS app (.ipa file). - -## Install and Setup the Dependencies - -- Install the NuGet packages for the project: -```bash -nuget.exe install ..\NUnitSelenium\packages.config -``` -- Clean and rebuild the project -```bash -nmake clean build -``` - -## Try our Sample Repository - -### Step 1: Get a Sample Project -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** -All the code samples in this documentation can be found on **TestMu AI's Github Repository**. You can either download or clone the repository to quickly run your tests. View on GitHub - -> Open the Android/iOS project using the file with a .sln extension. - -### Step 2: Setup the Environment Variables - -You need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in your [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/security). Run the below mentioned commands in your terminal to setup the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -### Step 3: Upload your Application -Upload your **_iOS_** application (.ipa file) or **_android_** application (.apk or .aab file) to the TestMu AI servers using our **REST API**. You need to provide your **Username** and **AccessKey** in the format `Username:AccessKey` in the **cURL** command for authentication. - -Make sure to add the path of the **appFile** in the cURL request. Below is an example cURL request to upload your app using our REST API: - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "appFile=@"/Users/macuser/Downloads/proverbial_android.apk"" -F "name="proverbial_app""`} - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "url=:https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk" -F "name=Proverbial_App"`} - -- If you do not have any **.apk** or **.ipa** file, you can run your sample tests on TestMu AI by using our sample apps, :link: [Android app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk) or :link: [iOS app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_ios.ipa). - -- Response of above cURL will be a **JSON** object containing the `APP_URL` of the format - `lt://APP123456789123456789` and will be used in the next step - -### Step 4: Update your Automation Script - -An automation script for the sample application given above has been provided here. Ensure to update the `APP_URL`, `username` and `accessKey` in the code scripts before running the tests. - -```csharp title="NUnitAppiumTests.cs" -using System; -using OpenQA.Selenium; -using OpenQA.Selenium.Remote; - -using NUnit.Framework; -using System.Threading; -using System.Collections.Generic; -using OpenQA.Selenium.Appium; -using OpenQA.Selenium.Appium.Android; -using OpenQA.Selenium.Appium.iOS; -using OpenQA.Selenium.Support.UI; - -namespace NUnitSelenium -{ -//-------------------Running two parallel test cases---------------------------- -//highlight-next-line -[TestFixture("OnePlus 6", "8", "Android", "APP_URL")] // Android Testing -//highlight-next-line -[TestFixture("iPhone 11", "14", "iOS", "APP_URL")] // iOS testing -[Parallelizable(ParallelScope.Fixtures)] -public class NUnitSeleniumSample -{ -//--------------------We can initialize username and access Key with hub url to authenticate our test script------------------------------------- -public static string LT_USERNAME = Environment.GetEnvironmentVariable("LT_USERNAME") ==null ? "your username" : Environment.GetEnvironmentVariable("LT_USERNAME"); -public static string LT_ACCESS_KEY = Environment.GetEnvironmentVariable("LT_ACCESS_KEY") == null ? "your accessKey" : Environment.GetEnvironmentVariable("LT_ACCESS_KEY"); -public static bool tunnel = Boolean.Parse(Environment.GetEnvironmentVariable("LT_TUNNEL")== null ? "false" : Environment.GetEnvironmentVariable("LT_TUNNEL")); -public static string build = Environment.GetEnvironmentVariable("LT_BUILD") == null ? "your build name" : Environment.GetEnvironmentVariable("LT_BUILD"); -public static string seleniumUri = "https://mobile-hub.lambdatest.com:443/wd/hub"; - -//-------------------------Initialization of Driver-------------------------- -AndroidDriver driver; - -// Initialization some parameter -private String deviceName; -private String platformVersion; -private String platformName; -private String app; - -public NUnitSeleniumSample(String deviceName, String platformVersion, String platformName, String app) -{ -this.deviceName = deviceName; -this.platformVersion = platformVersion; -this.platformName = platformName; -this.app = app; -} - -[SetUp] -public void Init() -{ -//-----------------------------------Create instance for passing capabilities----------------------------------------------------------------- -AppiumOptions capabilities = new AppiumOptions(); -//highlight-next-line -capabilities.AddAdditionalCapability("user", "LT_USERNAME"); //Add LambdaTest username here -//highlight-next-line -capabilities.AddAdditionalCapability("accessKey", "LT_ACCESS_KEY"); //Add LambdaTest accessKey here -capabilities.AddAdditionalCapability("app",app); -capabilities.AddAdditionalCapability("deviceName", deviceName); -capabilities.AddAdditionalCapability("platformVersion", platformVersion); -capabilities.AddAdditionalCapability("platformName", platformName); -capabilities.AddAdditionalCapability("build", "Csharp NUnit"); -capabilities.AddAdditionalCapability("name", "NUnit Test"); -capabilities.AddAdditionalCapability("isRealMobile", true); - -driver = new AndroidDriver (new Uri(seleniumUri), capabilities, TimeSpan.FromSeconds(600)); - -// Console.Out.WriteLine(driver); -Console.Out.WriteLine("On Which Device/Platform test is running:"+deviceName+" "+platformVersion+" "+platformName); - -} - -[Test] -public void Todotest() -{ -{ -//----------------------Text Color Changes--------------------------------- -Console.WriteLine("1.Text Color Changes"); -AndroidElement searchElement = (AndroidElement)new WebDriverWait( -driver, TimeSpan.FromSeconds(20)).Until( -SeleniumExtras.WaitHelpers.ExpectedConditions.ElementToBeClickable( -MobileBy.Id("color")) -); -System.Threading.Thread.Sleep(1000); -searchElement.Click(); -System.Threading.Thread.Sleep(1000); -searchElement.Click(); - -System.Threading.Thread.Sleep(1000); - -//----------------------Text Changes by clicking a button--------------------------------- -Console.WriteLine("2.Text Changes by clicking a button"); - -AndroidElement changeelement = (AndroidElement)new WebDriverWait( -driver, TimeSpan.FromSeconds(10)).Until( -SeleniumExtras.WaitHelpers.ExpectedConditions.ElementToBeClickable( -MobileBy.Id("Text")) -); -changeelement.Click(); - -System.Threading.Thread.Sleep(1000); - -//----------------------Toast--------------------------------- -Console.WriteLine("3.Toast"); - -AndroidElement toast = (AndroidElement)new WebDriverWait( -driver, TimeSpan.FromSeconds(10)).Until( -SeleniumExtras.WaitHelpers.ExpectedConditions.ElementToBeClickable( -MobileBy.Id("toast")) -); -toast.Click(); - -System.Threading.Thread.Sleep(1000); - -//----------------------Notification By clicking a button--------------------------------- -Console.WriteLine("4.Notification Button clicked"); - -AndroidElement Notification = (AndroidElement)new WebDriverWait( -driver, TimeSpan.FromSeconds(10)).Until( -SeleniumExtras.WaitHelpers.ExpectedConditions.ElementToBeClickable( -MobileBy.Id("notification")) -); -Notification.Click(); -System.Threading.Thread.Sleep(2000); - -//----------------------Geolocation button--------------------------------- -Console.WriteLine("5.Geolocation"); - -AndroidElement geolocation = (AndroidElement)new WebDriverWait( -driver, TimeSpan.FromSeconds(10)).Until( -SeleniumExtras.WaitHelpers.ExpectedConditions.ElementToBeClickable( -MobileBy.Id("geoLocation")) -); -geolocation.Click(); -System.Threading.Thread.Sleep(4000); -driver.PressKeyCode(AndroidKeyCode.Back); -System.Threading.Thread.Sleep(1000); - -//----------------------Speed Test Button--------------------------------- -Console.WriteLine("6.Speed Test Button Clicked"); - -AndroidElement speed = (AndroidElement)new WebDriverWait( -driver, TimeSpan.FromSeconds(10)).Until( -SeleniumExtras.WaitHelpers.ExpectedConditions.ElementToBeClickable( -MobileBy.Id("speedTest")) -); -speed.Click(); -System.Threading.Thread.Sleep(5000); -driver.PressKeyCode(AndroidKeyCode.Back); -System.Threading.Thread.Sleep(1000); - -//----------------------Browser Button--------------------------------- -// Console.WriteLine("Browser Button Clicked"); - -AndroidElement BROWSER = (AndroidElement)new WebDriverWait( -driver, TimeSpan.FromSeconds(30)).Until( -SeleniumExtras.WaitHelpers.ExpectedConditions.ElementToBeClickable( -MobileBy.XPath("//android.widget.FrameLayout[@content-desc=\"Browser\"]/android.widget.FrameLayout/android.widget.ImageView")) - -); -BROWSER.Click(); - -AndroidElement url = (AndroidElement)new WebDriverWait( -driver, TimeSpan.FromSeconds(10)).Until( -SeleniumExtras.WaitHelpers.ExpectedConditions.ElementToBeClickable( -MobileBy.Id("url")) -); - -url.Click(); - -url.SendKeys("www.lambdatest.com"); - -System.Threading.Thread.Sleep(1000); - -driver.PressKeyCode(AndroidKeyCode.Back); - -System.Threading.Thread.Sleep(3000); -} -} - -[TearDown] -public void Cleanup() - -{ - -bool passed = TestContext.CurrentContext.Result.Outcome.Status == NUnit.Framework.Interfaces.TestStatus.Passed; -try -{ -//-----------------Marking Test status passed or failed ----------------------------------------- -((IJavaScriptExecutor)driver).ExecuteScript("lambda-status=" + (passed ? "passed" : "failed")); - -} -finally -{ -//---------------------Quit the session----------------------- - -driver.Quit(); -} -} -} -} - -``` - -- You must set **isRealMobile** capability to `False` in the config file to run on **Virtual Devices** - -### Step 5: Configure the Test Capabilities - -You can update your custom capabilities in test scripts. In this sample project, we are passing platform name, platform version, device name and app url _(generated earlier)_ along with other capabilities like build name and test name via capabilities object. - -The capabilities object in the sample code are defined as: - -```csharp -AppiumOptions capabilities = new AppiumOptions(); -capabilities.AddAdditionalCapability("user", "LT_USERNAME"); //Add LambdaTest username here -capabilities.AddAdditionalCapability("accessKey", "LT_ACCESS_KEY"); //Add LambdaTest accessKey here -capabilities.AddAdditionalCapability("app",app); -capabilities.AddAdditionalCapability("deviceName", deviceName); -capabilities.AddAdditionalCapability("platformVersion", platformVersion); -capabilities.AddAdditionalCapability("platformName", platformName); -capabilities.AddAdditionalCapability("build", "Csharp NUnit"); -capabilities.AddAdditionalCapability("name", "NUnit Test"); -capabilities.AddAdditionalCapability("isRealMobile", true); -``` - -- You must set **isRealMobile** capability to `False` in the config file to run on **Virtual Devices** - -- You must add the generated **APP_URL** to the `app` capability in the config file. -- You must set **isRealMobile** capability to `False` in the config file to run on **Virtual Devices** -- You can generate capabilities for your test requirements with the help of our inbuilt [**Capabilities Generator tool**](https://www.testmuai.com/capabilities-generator/).For more details, please refer to our guide on [**Desired Capabilities in Appium**](/support/docs/desired-capabilities-in-appium/). - -### Step 6: Execute and Monitor your Tests - -Run the following command in your project directory to execute your build and run the tests parallelly. - -```bash -nmake all -``` - -OR - -Go to **Build menu** in Visual Studio Code menu bar and click on **Build Solution**. After the solution is built navigate built navigate to **Test menu** and click on **Test All** to execute the tests. - -**Warning** -Make sure that **DotNetSeleniumExtras.WaitHelpers** in the NuGet packages which you can check from **Project Menu -> Manage NuGet Packages**. - - > Your test results would be displayed on the test console (or CLI if you are using terminal/cmd) and on the [TestMu AI App Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://appautomation.lambdatest.com/build). - -## Using the NUnit Agent Skill with TestMu AI - -The [nunit-skill](https://github.com/LambdaTest/agent-skills/tree/main/nunit-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The nunit-skill package includes: - -``` -nunit-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing NUnit Agent Skill - -Install a NUnit Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/nunit-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/nunit-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only nunit-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). - -## Reference Guides - -- [Advanced Configuration for Capabilities](/support/docs/desired-capabilities-in-appium/) -- [How to test locally hosted apps](/support/docs/testing-locally-hosted-pages/) -- [How to integrate TestMu AI with CI/CD](/support/docs/integrations-with-ci-cd-tools/) diff --git a/static/docs/appium-csharp.md b/static/docs/appium-csharp.md deleted file mode 100644 index 951dd7f0a..000000000 --- a/static/docs/appium-csharp.md +++ /dev/null @@ -1,321 +0,0 @@ -# Appium with C# - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -In this documentation, you will learn how to trigger a automation script of **CSharp** for application testing with **Appium** on TestMu AI, set the [**desired capabilities**](/support/docs/desired-capabilities-in-appium/) for appium testing, and other advanced features of TestMu AI. - -## Prerequisites - -- Your TestMu AI [Username and Access key](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/security). -- Install the MS Visual Studio 2013 or later version for C#. We recommend using the latest version. - -## Try our Sample Repository - -### Step 1: Get a Sample Project -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** -All the code samples in this documentation can be found on **TestMu AI's Github Repository**. You can either download or clone the repository to quickly run your tests. View on GitHub - -> Open the Android/iOS project using the file with a .sln extension. - -### Step 2: Setup the Environment Variables - -You need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in your [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/security). Run the below mentioned commands in your terminal to setup the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -### Step 3: Upload your Application -Upload your **_iOS_** application (.ipa file) or **_android_** application (.apk or .aab file) to the TestMu AI servers using our **REST API**. You need to provide your **Username** and **AccessKey** in the format `Username:AccessKey` in the **cURL** command for authentication. - -Make sure to add the path of the **appFile** in the cURL request. Below is an example cURL request to upload your app using our REST API: - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "appFile=@"/Users/macuser/Downloads/proverbial_android.apk"" -F "name="proverbial_app""`} - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "url=:https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk" -F "name=Proverbial_App"`} - -- If you do not have any **.apk** or **.ipa** file, you can run your sample tests on TestMu AI by using our sample apps, :link: [Android app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk) or :link: [iOS app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_ios.ipa). - -- Response of above cURL will be a **JSON** object containing the `APP_URL` of the format - `lt://APP123456789123456789` and will be used in the next step - -### Step 4: Update your Automation Script - -An automation script for the sample application given above has been provided here. Ensure to update the `APP_URL`, `username` and `accessKey` in the code scripts before running the tests. - -```csharp title="csharp-appium-first.sln" -using System; -using OpenQA.Selenium.Appium; -using OpenQA.Selenium.Support.UI; -using OpenQA.Selenium.Appium.iOS; -using System.Threading; - -namespace csharp_appium_first - -{ -class Program -{ -static void Main(string[] args) -{ -AppiumOptions caps = new AppiumOptions(); - -// Set your LambdaTest access credentials -//highlight-next-line -caps.AddAdditionalCapability("LT_USERNAME", "username"); //Enter the Username here -//highlight-next-line -caps.AddAdditionalCapability("LT_ACCESSKEY", "accessKey"); //Enter the Access key here - -// Set URL of the application under test -//highlight-next-line -caps.AddAdditionalCapability("app", "APP_URL"); //Enter the App URL here. - -// Specify device and os_version -caps.AddAdditionalCapability("deviceName", "iPhone 12"); //Change the device name here -caps.AddAdditionalCapability("platformVersion", "15"); -caps.AddAdditionalCapability("platformName", "iOS"); -caps.AddAdditionalCapability("isRealMobile", true); -caps.AddAdditionalCapability("network", false); - -// Specify the platformName -//caps.PlatformName = "iOS"; - -// Set other LambdaTest capabilities -caps.AddAdditionalCapability("project", "First CSharp project"); -caps.AddAdditionalCapability("build", "CSharp iOS"); -caps.AddAdditionalCapability("name", "first_test"); - -// Initialize the remote Webdriver using LambdaTest remote URL -// and desired capabilities defined above -IOSDriver driver = new IOSDriver( -new Uri("https://mobile-hub.lambdatest.com/wd/hub"), caps); - -// Test case for the sample iOS app. -// If you have uploaded your app, update the test case here. -IOSElement color = (IOSElement)new WebDriverWait(driver, TimeSpan.FromSeconds(30)).Until( -SeleniumExtras.WaitHelpers.ExpectedConditions.ElementToBeClickable(MobileBy.Id("color")) -); -color.Click(); -color.Click(); - -IOSElement text = (IOSElement)new WebDriverWait(driver, TimeSpan.FromSeconds(30)).Until( -SeleniumExtras.WaitHelpers.ExpectedConditions.ElementToBeClickable(MobileBy.Id("Text")) -); -text.Click(); - -IOSElement toast = (IOSElement)new WebDriverWait(driver, TimeSpan.FromSeconds(30)).Until( -SeleniumExtras.WaitHelpers.ExpectedConditions.ElementToBeClickable(MobileBy.Id("toast")) -); -toast.Click(); - -IOSElement nf = (IOSElement)new WebDriverWait(driver, TimeSpan.FromSeconds(30)).Until( -SeleniumExtras.WaitHelpers.ExpectedConditions.ElementToBeClickable(MobileBy.Id("notification")) -); -nf.Click(); - -IOSElement gl = (IOSElement)new WebDriverWait(driver, TimeSpan.FromSeconds(30)).Until( -SeleniumExtras.WaitHelpers.ExpectedConditions.ElementToBeClickable(MobileBy.Id("geoLocation")) -); -gl.Click(); - -Thread.Sleep(5000); - -driver.Navigate().Back(); - -IOSElement st = (IOSElement)new WebDriverWait(driver, TimeSpan.FromSeconds(30)).Until( -SeleniumExtras.WaitHelpers.ExpectedConditions.ElementToBeClickable(MobileBy.Id("speedTest")) -); -st.Click(); - -Thread.Sleep(5000); - -driver.Navigate().Back(); - -IOSElement browser = (IOSElement)new WebDriverWait(driver, TimeSpan.FromSeconds(30)).Until( -SeleniumExtras.WaitHelpers.ExpectedConditions.ElementToBeClickable(MobileBy.AccessibilityId("Browser")) -); -browser.Click(); - -IOSElement inputBox = (IOSElement)new WebDriverWait(driver, TimeSpan.FromSeconds(30)).Until( -SeleniumExtras.WaitHelpers.ExpectedConditions.ElementToBeClickable(MobileBy.Id("url")) -); -inputBox.Click(); - -driver.Quit(); - -} -} -} -``` - -- You must set **isRealMobile** capability to `False` in the config file to run on **Virtual Devices** - -```csharp title="csharp-appium-first.sln" -using System; -using OpenQA.Selenium.Appium; -using OpenQA.Selenium.Support.UI; -using OpenQA.Selenium.Appium.Android; -using System.Threading; - -namespace csharp_appium { -class Program { -static void Main(string[] args) { -AppiumOptions caps = new AppiumOptions(); - -caps.AddAdditionalCapability("user", "########"); //Add the LT Username -caps.AddAdditionalCapability("accessKey", "########"); //Add the LT Access key - -// Set URL of the application under test -caps.AddAdditionalCapability("app", "###########"); //Add the App ID - -// Specify device and os_version -caps.AddAdditionalCapability("deviceName", "Galaxy S21 Ultra 5G"); //Add the Device Details -caps.AddAdditionalCapability("platformVersion", "11"); -caps.AddAdditionalCapability("platformName", "Android"); -caps.AddAdditionalCapability("isRealMobile", true); -caps.AddAdditionalCapability("network", false); -caps.AddAdditionalCapability("w3c", true); -caps.AddAdditionalCapability("visual", true); -caps.AddAdditionalCapability("video", true); -caps.AddAdditionalCapability("project", "CSharp Sample Android"); -caps.AddAdditionalCapability("build", "CSharp Sample Android"); -caps.AddAdditionalCapability("name", "CSharp Sample Android"); - -// Initialize the remote Webdriver using LambdaTest remote URL -// and desired capabilities defined above -AndroidDriver < AndroidElement > driver = new AndroidDriver < AndroidElement > ( -new Uri("https://mobile-hub.lambdatest.com/wd/hub"), caps); - -// Test case for the sample Android app. -// If you have uploaded your app, update the test case here. -AndroidElement color = (AndroidElement) new WebDriverWait(driver, TimeSpan.FromSeconds(30)).Until( -SeleniumExtras.WaitHelpers.ExpectedConditions.ElementToBeClickable(MobileBy.Id("color")) -); -color.Click(); -color.Click(); - -AndroidElement text = (AndroidElement) new WebDriverWait(driver, TimeSpan.FromSeconds(30)).Until( -SeleniumExtras.WaitHelpers.ExpectedConditions.ElementToBeClickable(MobileBy.Id("Text")) -); -text.Click(); - -AndroidElement toast = (AndroidElement) new WebDriverWait(driver, TimeSpan.FromSeconds(30)).Until( -SeleniumExtras.WaitHelpers.ExpectedConditions.ElementToBeClickable(MobileBy.Id("toast")) -); -toast.Click(); - -AndroidElement nf = (AndroidElement) new WebDriverWait(driver, TimeSpan.FromSeconds(30)).Until( -SeleniumExtras.WaitHelpers.ExpectedConditions.ElementToBeClickable(MobileBy.Id("notification")) -); -nf.Click(); - -AndroidElement gl = (AndroidElement) new WebDriverWait(driver, TimeSpan.FromSeconds(30)).Until( -SeleniumExtras.WaitHelpers.ExpectedConditions.ElementToBeClickable(MobileBy.Id("geoLocation")) -); -gl.Click(); - -Thread.Sleep(5000); - -driver.Navigate().Back(); - -AndroidElement st = (AndroidElement) new WebDriverWait(driver, TimeSpan.FromSeconds(30)).Until( -SeleniumExtras.WaitHelpers.ExpectedConditions.ElementToBeClickable(MobileBy.Id("speedTest")) -); -st.Click(); - -Thread.Sleep(5000); - -driver.Navigate().Back(); -``` - -### Step 5: Configure the Test Capabilities - -You can update your custom capabilities in test scripts. In this sample project, we are passing platform name, platform version, device name and app url _(generated earlier)_ along with other capabilities like build name and test name via capabilities object. - -The capabilities object in the sample code are defined as: - -```csharp title="iOS(.ipa)" -AppiumOptions caps = new AppiumOptions(); -caps.AddAdditionalCapability("LT_USERNAME", "username"); -caps.AddAdditionalCapability("LT_ACCESSKEY", "accessKey"); -caps.setCapability("platformName", "iOS"); -caps.setCapability("deviceName", "iPhone 13 Pro"); -caps.setCapability("isRealMobile", true); -caps.setCapability("platformVersion","15.0"); -caps.setCapability("Visual", true); -caps.setCapability("Console", true); -caps.setCapability("Network", false); - -``` - -```csharp title="Android(.apk)" -AppiumOptions caps = new AppiumOptions(); -caps.AddAdditionalCapability("LT_USERNAME", "username"); -caps.AddAdditionalCapability("LT_ACCESSKEY", "accessKey"); -caps.setCapability("platformName", "Android"); -caps.setCapability("deviceName", "Galaxy S20"); -caps.setCapability("isRealMobile", true); -caps.setCapability("platformVersion","11"); -caps.setCapability("Visual", true); -caps.setCapability("Console", true); -caps.setCapability("Network", false); - -``` - -- You must set **isRealMobile** capability to `False` in the config file to run on **Virtual Devices** - -- You must add the generated **APP_URL** to the `app` capability in the config file. -- You must set **isRealMobile** capability to `False` in the config file to run on **Virtual Devices** -- You can generate capabilities for your test requirements with the help of our inbuilt [**Capabilities Generator tool**](https://www.testmuai.com/capabilities-generator/).For more details, please refer to our guide on [**Desired Capabilities in Appium**](/support/docs/desired-capabilities-in-appium/). - -### Step 6: Execute and Monitor your Tests - -- Click the **Play** icon to run the test. - - > Your test results would be displayed on the test console (or CLI if you are using terminal/cmd) and on the [TestMu AI App Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://appautomation.lambdatest.com/build). - -## Using the Appium Agent Skill with TestMu AI - -The [appium-skill](https://github.com/LambdaTest/agent-skills/tree/main/appium-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The appium-skill package includes: - -``` -appium-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing Appium Agent Skill - -Install a Appium Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/appium-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/appium-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only appium-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). - -## Reference Guides - -- [Advanced Configuration for Capabilities](/support/docs/desired-capabilities-in-appium/) -- [How to test locally hosted apps](/support/docs/testing-locally-hosted-pages/) -- [How to integrate TestMu AI with CI/CD](/support/docs/integrations-with-ci-cd-tools/) diff --git a/static/docs/appium-custom-headers.md b/static/docs/appium-custom-headers.md deleted file mode 100644 index d6c4d3a14..000000000 --- a/static/docs/appium-custom-headers.md +++ /dev/null @@ -1,110 +0,0 @@ -# CustomHeaders support for automation - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Custom headers provide you the ability to keep crucial information about the request or response, such as the method, URL, and body content. You can modify the parameters of the HTTP requests delivered by your tests by manipulating with these headers, thereby around firewall restrictions. - -In this documentation, we will look at LambdaTest CustomHeaders, a `capability` that allows you to add custom headers to your tests and bypass firewall restrictions while performing automated browser testing. - -## How to use CustomHeaders Capability on TestMu AI? - -To add custom headers using TestMu AI's CustomHeader capability, you need to use TestMu AI's Desired Capabilities class. - -1. Create an instance of the Desired Capabilities class. - -2. Use the customHeaders capability to add your custom headers. - -```java -DesiredCapabilities capabilities = new DesiredCapabilities(); -capabilities.setCapability("customHeaders", new HashMap() {{ -put("headerName", "headerValue"); -}}); -``` - -In the above code snippet, you need to replace **headerName** and **headerValue** with the actual name and value of the header. You can add multiple headers based on your requirements. - -## CustomHeaders: Use Cases and Examples - -Custom headers serves different purposes for development and network communications. Here are some of its key use cases: - -1. **User Identification and Session Management**: Used to send tokens, session IDs to authenticate and identify users. This helps in managing user sessions, implement stateless authentication mechanisms, track user behavior. - -**Example**: The Custom header `X-Session-ID`is used to track user sessions. - -```java -X-Session-ID: 1234567890 -``` - -2. **Content Negotiation**: Determines how the client and server decide on the data format to exchange. For instance, the "Accept" header can specify the format (like JSON or XML) that the client prefers for the response data. - -**Example**: `Accept` header specifies the client-preferred format of the response data. - -```java -Accept: application/json -``` - -3. **Rate Limiting**: Custom headers are often used by APIs to provide information about rate limits. They might send headers indicating how many requests a client can make in a given time period, how many requests they have left, or when they can make new requests. - -**Example** - -```java -X-RateLimit-Limit: 60 -X-RateLimit-Remaining: 56 -X-RateLimit-Reset: 1372700873 -``` - -4. **Debugging and Performance Tracking**: Some services include custom headers in their responses to provide additional information that can help in debugging or performance tracking. This can include server version numbers, execution times, or other internal details. - -**Example** - -```java -X-Session-ID: 1234567890 //custom header X-Session-ID to track user sessions. -``` - -5. **CORS (Cross-Origin Resource Sharing)**: The CORS standard uses custom headers to allow browsers and servers to interact securely with resources from different origins. This includes headers like "Access-Control-Allow-Origin" and "Access-Control-Allow-Methods". - -**Example** - -```java -X-Session-ID: 1234567890 //custom header X-Session-ID to track user sessions. -``` - -6. **Custom Application Logic**: Custom headers can also be used to implement specific application-level logic. For instance, a custom header could be used to determine the language of the response, to enable or disable features, or to specify version numbers for API versioning. - -**Example** - -```java -X-Session-ID: 1234567890 //custom header X-Session-ID to track user sessions. -``` - -7. **Bypassing Firewalls or Proxies**: In some cases, custom headers can be used to bypass certain network restrictions, such as firewalls or proxy servers. This should, however, be done responsibly and in accordance with security policies. - -**Example** - -```java -X-Session-ID: 1234567890 //custom header X-Session-ID to track user sessions. -``` - -8. **Server Health and Status Information**: Some applications use custom headers to provide health and status information about the server or application. This can be used for monitoring and maintaining the health of the system. - -**Example** - -```java -X-Session-ID: 1234567890 //custom header X-Session-ID to track user sessions. -``` - -9. **SEO Optimization**: Custom headers like canonical and pagination headers can be used to guide search engines and optimize SEO. - -**Example** - -```java -X-Session-ID: 1234567890 //custom header X-Session-ID to track user sessions. -``` - -10. **A/B Testing**: Custom headers can be used to control or track A/B testing, where different versions of a service are tested against each other to determine which performs better. - -**Example** - -```java -X-Session-ID: 1234567890 //custom header X-Session-ID to track user sessions. -``` diff --git a/static/docs/appium-date-and-time.md b/static/docs/appium-date-and-time.md deleted file mode 100644 index 545e242ee..000000000 --- a/static/docs/appium-date-and-time.md +++ /dev/null @@ -1,59 +0,0 @@ -# Set Device Date & Time - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI now supports configuring custom **date**, **time**, and **hour-format** on Real Devices via Appium during automation testing sessions. This allows developers and testers to simulate specific date/time conditions to validate use cases like scheduled events, alarms, chats, or localization for time formats. - -By the end of this document, you will be able to: -- Set a specific date and time on the device -- Toggle between 12-hour and 24-hour formats -- Enable/disable automatic time syncing with network settings - -## Support for Automation on Real Devices - -To modify the date and time settings programmatically, use the `lambda_executor` Appium hook with the `updateDeviceSettings` action. You can provide multiple arguments including custom date, time, format toggle, and auto-time setting. - -## Supported Arguments - -| Argument | Format | Description | -| ------------------ | ------------- | --------------------------------------------------------------------------- | -| `customDate` | `MMM DD YYYY` | Sets the device date. You can select a date up to **7 days** from today. | -| `customTime` | `HH:MM` | Sets time in 24-hour format `(e.g., 13:30)`. Display adapts based on `twelveHourTime` (shown as `1:30 PM` if enabled).| -| `twelveHourTime` | `On` / `Off` | `On` for 12-hour, `Off` for 24-hour format. | -| `setAutomatically` | `On` / `Off` | Enables or disables syncing with network time. | - -> 💡 **Note:** When `setAutomatically` is turned **On**, all Other Arguments for **Date and Time** are automatically disabled on the device. - -### Appium Hook Example - -{`driver.execute_script('lambda_executor: { - "action": "updateDeviceSettings", - "arguments": { - "customDate": "Jun 20 2025", - "customTime": "15:05", - "twelveHourTime": "On", - "setAutomatically": "On" - } -}')`} - -## Supported Platforms - -| Platform | OS Version | Support Status | -| --------- | ------------------- | ------------------ | -| iOS | 14 and above | Fully Supported | -| Android | 10 and above | Fully Supported | - -**Unsupported Android Devices** -Custom date and time configuration is not supported on certain Android device models, primarily from **Motorola, Xiaomi, Oppo, and other Chinese OEMs**. On these devices, the Appium hook will return the following error: - -`Custom date and time hook is not supported on this device. Please try on another device_id.` - -If you encounter this error, retry your automation run on a different Android device. - -## Example Use Cases - -- Validate how the app behaves across different time zones or locales. -- Test reminder or calendar event triggering logic. -- Test 12-hour vs. 24-hour format layout rendering. - -> For Configuring Device **Date and Time** during **Manual Testing** on Real Devices, please refer to our [Set Date & Time on Real Devices (Manual)](/support/docs/set-date-time-hour-format-real-devices/) guide. diff --git a/static/docs/appium-firebase-app-upload.md b/static/docs/appium-firebase-app-upload.md deleted file mode 100644 index cbdd4bf11..000000000 --- a/static/docs/appium-firebase-app-upload.md +++ /dev/null @@ -1,58 +0,0 @@ -# Uploading app via Firebase - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Tutorial to test an app uploaded on Firebase - -**Firebase** is a platform developed by **Google** for creating mobile and web applications. **TestMu AI cloud platform** provides you various methods to upload your files on a real device like from playstore and appstore. Uploading an app through Firebase is one such method. - -## How to Start with Firebase - -Before using TestMu AI you should have a Firebase account where you can upload your app and use later. We will demonstrate the steps required to set up a Firebase account. - -1. Visit [**official website**](https://firebase.google.com/) of Firebase if you have not used it earlier and get started. -2. You will find an option to **Add a Project**. -3. After this you will be asked to name your project. We named it **Proverbial**. You would require to add the parent resource connected to your google account. In our case it would be `lambdatest.com`. - -4. If you desire, you can connect your **Google Analytics Account** to your project. -5. After you have done the above steps successfully you will find yourself redirected to **Firebase Dashboard**. You have successfully set up a Firebase account and created a Project too. - -## How to Register App on Firebase Account - -After you have created the **Firebase account** and added a project. Now, you need to **Upload/Register your app** in the account so that you can use it further without any link by just logging in to your Firebase account. Let's see how to do so: - -1. In the sidebar of the dashboard, you will see a **Build Option**. -2. Go to **Build** and choose **App Distribution** option from the menu. - -3. Choose the application type (ios or android) now add the relevant details on the next page as shown below. In the step 3, click next or feel free to add the Firebase SDK to your project. - -4. We are done with the initial setup, please click on **Continue to Console** now and then get started. -5. We are now at the **App Distribution page**, now just upload your application here! - -6. After the app is uploaded, add the relevant testers (these persons will be allowed to use the application through google login). Now, we see that the **Distribution is successful**. - -## How to use Firebase App for Real Device App Testing? - -After you have uploaded your application to your Firebase account, you can use the same account on a **TestMu AI Real Device** to retrieve your app and start testing. Below are the steps for the same: - -1. Go to [**Real Device App Testing**](https://applive.lambdatest.com/app) on TestMu AI console and start the test session. -2. After you start the test, go to the **Gmail** or any other mail client on the phone and login to your registered **Google Account**. You would have received an invitation if you added yourself as a tester. - -3. Post accepting the invite received you will be taken to Firebase console on the real device where you can see your app with a **download link**. -4. Download the app and **Allow installation of unknown apps** to install the app on the selected real device. -5. Finally you have your application running on a TestMu AI real device. Continue testing the features of your app. - -## How to use Firebase App for App Automation? - -After you have uploaded your application to your Firebase account, you can use the same account on a **TestMu AI Real Device** to retrieve your app and start testing. Below are the steps for the same: - -1. Get the Firebase **projects.apps.releases.get** API. You can find it from [Firebase official documentation](https://firebase.google.com/docs/reference/app-distribution/rest/v1/projects.apps.releases/get). - -2. It will generate a **Release** response containing a public downloadable URI. - -3. You can use this downloadable URI while uploading apps using App URL on the TestMu AI cloud. - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "url=:https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk" -F "name=Proverbial_App" -`} - -4. Now use the App ID generated in your automation scripts to run App Automation tests on real devices. diff --git a/static/docs/appium-firetv.md b/static/docs/appium-firetv.md deleted file mode 100644 index 832d65974..000000000 --- a/static/docs/appium-firetv.md +++ /dev/null @@ -1,193 +0,0 @@ -# Fire TV With Appium - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Tutorial To Run Your First Test On TestMu AI - -In this topic, you will learn how to configure and run your **Fire TV** automation testing scripts with **Appium** on **TestMu AI Real Device Cloud platform**. - -## Objective - -By the end of this topic, you will be able to: - -1. Set up an environment for testing your Apps using **Fire TV** with **Appium**. -2. Understand and configure the core capabilities required for your Appium test suite. -3. Explore the advanced features of TestMu AI. - -**Sample repo** - -All the code samples in this documentation can be found on **TestMu AI's Github Repository**. You can either download or clone the repository to quickly run your tests. View on GitHub - -## Prerequisites - -Before you can start performing App automation testing with Appium, you would need to follow these steps: - -- You have access to TestMu AI username and accessKey. If you have not registered yet, you can do the same by visiting our website. You will be able to access the credentials in the [TestMu AI Profile](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile) -- Install the latest Python build from the [official website](https://www.python.org/downloads/). We recommend using the latest version. -- Make sure **pip** is installed in your system. You can install **pip** from [pip documentation](https://pip.pypa.io/en/stable/installation/). - -## Run your first test - -### 1. Upload your application -Upload your **Fire TV** application (.apk file) to the TestMu AI servers using our **REST API**. You need to provide your **Username** and **AccessKey** in the format `Username:AccessKey` in the **cURL** command for authentication. Make sure to add the path of the **appFile** in the cURL request. Here is an example cURL request to upload your app using our REST API: - - **Using App File from System:** - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "appFile=@"/Users/macuser/Downloads/fireos-sample-app.apk"" -F "name="fireos_app"" -`} - -- If you do not have any **.apk** file, you can run your sample tests on TestMu AI by using our sample :link: [FireTV app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/fireos-sample-app.apk). -- Response of above cURL will be a **JSON** object containing the `APP_URL` of the format - ``lt://APP123456789123456789`` and will be used in the next step. - -### 2. Clone the sample project - -1. Clone the TestMu AI's [LT-appium-firetv](https://github.com/LambdaTest/LT-appium-firetv) and navigate to the code directory as shown below: - -```bash -git clone https://github.com/LambdaTest/LT-appium-firetv -cd LT-appium-firetv -``` - -### 3. Set up your authentication - -Make sure you have your TestMu AI credentials with you to run test automation scripts on TestMu AI. To obtain your access credentials, [purchase a plan](https://billing.lambdatest.com/billing/plans) or access the [Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://appautomation.lambdatest.com/). Then, set TestMu AI `Username` and `Access Key` in environment variables with following commands. - - {`export LT_USERNAME=${ YOUR_LAMBDATEST_USERNAME()} \\ -export LT_ACCESS_KEY=${ YOUR_LAMBDATEST_ACCESS_KEY()}`} - - {`set LT_USERNAME=${ YOUR_LAMBDATEST_USERNAME()} \` -set LT_ACCESS_KEY=${ YOUR_LAMBDATEST_ACCESS_KEY()}`} - -### 4. Write your automation script - -An automation script for the sample application available above has been provided below. Ensure to update the `APP_URL`, `username` and `accessKey` in the code scripts before running the tests. - -```python title="firetv.py" -from xml.dom.expatbuilder import Rejecter -from appium import webdriver -from selenium.webdriver.common.by import By -import time - -def getCaps(): -desired_cap= { -"deviceName" : "Amazon Fire TV Stick", -"platformVersion" : "7", -"platformName":"fireos", -"isRealMobile":True, -"build": "firetv", -"video": True, -"app":"APP_URL", #Add app url here -"network": False, -"geoLocation": "RU", -"devicelog": True, -"visual":True -} - -return desired_cap - -def runTest(): -username = "YOUR_LAMBDATEST_USERNAME" #Add your username here -accessToken = "YOUR_LAMBDATEST_ACCESSKEY" #Add your accessKey here - -gridUrl = "mobile-hub-internal.lambdatest.com/wd/hub" - -# capabilities -desired_cap = getCaps() -url = "http://"+username+":"+accessToken+"@"+gridUrl - -print("Initiating remote driver on platform: " + -desired_cap["deviceName"]+" browser: "+" version: "+desired_cap["platformVersion"]) - -start = time.time() -driver = webdriver.Remote( -desired_capabilities=desired_cap, -command_executor=url -) - -# run test -print(driver.session_id) -time.sleep(10) - -inputfield = driver.find_element(by = By.ID, value ="enterText") -inputfield.send_keys("https://ifconfig.me") - -time.sleep(2) - -inputfield = driver.find_element(by = By.ID, value ="JustAButton") -inputfield.click() - -time.sleep(10) -list2 = driver.find_element(by= By.XPATH, value="//*[@resource-id='ip_address_cell']") -print(list2.text) - -time.sleep(50) - -driver.execute_script("lambda-status=passed") - -driver.quit() -end = time.time() - -print("time taken: ", end - start) - -if __name__ == "__main__": -runTest() -``` - -### Configure the test capabilities - -You can update your custom capabilities in test scripts. In this sample project, we are passing platform name, platform version, device name and app url (generated earlier) along with other capabilities like build name and test name via capabilities object. The capabilities object in the sample code are defined as: - -**Supported Model:** -- Device: `"Amazon Fire TV Stick"` ; OS Version:`"7"` - -**Supported Capabilities:** -Same as Android. - -**Platform:** -`"fireos"` - -```python title="firetv.py" -def getCaps(): -desired_cap= { -"deviceName" : "Amazon Fire TV Stick", -"platformVersion" : "7", -"platformName":"fireos", -"isRealMobile":True, -"build": "firetv", -"video": True, -"app":"APP_URL", #Add app url here -"network": False, -"geoLocation": "RU", -"devicelog": True, -"visual":True -} -``` - -**Note** - -- You must add the generated **APP_URL** to the `"app"` capability in the config file. -- You can generate capabilities for your test requirements with the help of our inbuilt **Capabilities Generator**.For more details, please refer to our guide on [Desired Capabilities in Appium](/support/docs/desired-capabilities-in-appium/). - -### 5. Execute your test case - -1. Install the required packages from the cloned project directory: - -```bash -pip install -r requirements.txt -``` - -2. Run the following command in the directory where your project has been saved to execute your build. - -```python -python firetv.py -``` - -If you are unable to run the automation script with the above mentioned commands try **'python3'** command except for **'python'**. - -Your test results would be displayed on the test console (or command-line interface if you are using terminal/cmd) and on the [TestMu AI App Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://appautomation.lambdatest.com/build). - -## Additional Links - -- [Advanced Configuration for Capabilities](/support/docs/desired-capabilities-in-appium/) -- [How to test locally hosted apps](/support/docs/testing-locally-hosted-pages/) -- [How to integrate TestMu AI with CI/CD](/support/docs/integrations-with-ci-cd-tools/) diff --git a/static/docs/appium-flutter-integration.md b/static/docs/appium-flutter-integration.md deleted file mode 100644 index fb3704b4c..000000000 --- a/static/docs/appium-flutter-integration.md +++ /dev/null @@ -1,113 +0,0 @@ -# Appium Flutter Integration Driver Testing - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI supports testing Flutter applications on real Android and iOS devices using the [**Appium Flutter Integration Driver**](https://github.com/AppiumTestDistribution/appium-flutter-integration-driver/). This driver enables you to automate Flutter app testing in multiple programming languages, providing a powerful and flexible way to validate your Flutter apps on real devices in the cloud. - -Testing Flutter apps requires embedding the **Flutter Integration Server** inside your app to enable communication between Appium and Flutter UI elements. - -After preparing your Flutter app with this integration (refer to the setup documentation below), you can build, upload, and execute your tests on TestMu AI real devices using the Appium Flutter Integration Driver. - -## Native Flutter Integration Driver vs Appium Flutter Integration Driver - -| Feature / Use Case | Native Flutter Driver | Appium Flutter Integration Driver | -|----------------------------------------------------|----------------------|----------------------------------| -| Writing tests in languages other than Dart | No | Yes | -| Support for Flutter apps with embedded webviews or native views | No | Yes | -| Running tests on multiple devices simultaneously | No | Yes | -| Compatible with device farms supporting Appium | No | Yes | -| Interaction beyond Flutter context (e.g., OTP apps) | No | Yes | - -## Why Use Appium Flutter Integration Driver? - -- **Multi-language support:** Write tests in Java, Python, Ruby, JavaScript, and more. -- **Automatic UI synchronization:** Waits intelligently for UI elements and animations to complete. -- **Seamless context switching:** No manual switching between Flutter and native app contexts. -- **Powerful gesture support:** Supports long press, drag and drop, scroll to element, double-click, and more. -- **Element chaining:** Easily find child elements under specific parent widgets. -- **Real device cloud ready:** Supported on TestMu AI real Android and iOS devices. - -## Getting Your Flutter App Ready for Testing - -### 1. Add the Flutter Integration Server dependency - -In your Flutter app’s `pubspec.yaml`, add: - -```yaml -dev_dependencies: -appium_flutter_server: 0.0.28 -``` - -### 2. Create integration test files - -Create an `integration_test` folder at the root of your Flutter project. Inside it, create `appium_test.dart` with: - -```dart -import 'package:appium_flutter_server/appium_flutter_server.dart'; -import 'package:your_app_package/main.dart'; // Replace with your app's main.dart import - -void main() { -initializeTest(app: const MyApp()); -} -``` - -Or with setup callback: - -```dart -import 'package:your_app_package/main.dart' as app; - -void main() { -initializeTest( -callback: (WidgetTester tester) async { -// Add prerequisite setup here -await tester.pumpWidget(const app.MyApp()); -}, -); -} -``` - -### 3. Build your Flutter app with integration test - -{`./gradlew app:assembleDebug -Ptarget=$(pwd)/../integration_test/appium_test.dart`} - -{`flutter build ios integration_test/appium_test.dart --simulator`} - -{`flutter build ipa --release integration_test/appium_test.dart`} - -## Uploading Your Flutter App to TestMu AI - -- For detailed instructions on how to upload your application to TestMu AI, please refer to our comprehensive guide. -- Visit the [**Upload your Application**](/support/docs/application-setup-via-api/#upload-your-application) documentation to learn more. -- Use the returned **App ID** from above step in your automation scripts. - -## Sample Desired Capabilities - -{`desired_caps = { - "deviceName":"Galaxy S20", - "platformName":"Android", - "platformVersion":"15", - "isRealMobile":True, // Use False for Virtual Devices - "app":"YOUR_APP_URL", - "build":"Sample Build", - "name":"Sample Test", - "automationName": "FlutterIntegration" #Using Flutter Integration Driver -} - -`} - -{`desired_caps = { - "deviceName":"iPhone 16", - "platformName":"ios", - "platformVersion":"18", - "isRealMobile":True, // Use False for Virtual Devices - "app":"YOUR_APP_URL", - "build":"Sample Build", - "name":"Sample Test", - "automationName": "FlutterIntegration" #Using Flutter Integration Driver -} - -`} - -- You must add the generated **APP_URL** to the `app` capability in the config file. -- You must set **isRealMobile** capability to `False` in the config file to run on **Virtual Devices** -- You can generate capabilities for your test requirements with the help of our inbuilt [**Capabilities Generator tool**](https://www.testmuai.com/capabilities-generator/).For more details, please refer to our guide on [**Desired Capabilities in Appium**](/support/docs/desired-capabilities-in-appium/). diff --git a/static/docs/appium-inspector-integration.md b/static/docs/appium-inspector-integration.md deleted file mode 100644 index b90df543c..000000000 --- a/static/docs/appium-inspector-integration.md +++ /dev/null @@ -1,77 +0,0 @@ -# Appium Inspector Integration on TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This post will help you in integrating **Appium Inspector** to interact with **TestMu AI's Real Devices**. - -## Introduction - -A **GUI inspector** for mobile apps and more, powered by a (separately installed) **Appium server**. Appium Inspector is an Appium client (like WebdriverIO, Appium's Java client, Appium's Python client, etc) with a User Interface. We can use the interface for specifying Appium Server Version, Setting Capabilities. Once the Appium Server is up & running with the App, we can interact with elements and run other Appium Commands. - -By the end of this topic, you will be able to: - -1. Upload your Application to TestMu AI Server & Receive the Unique App URL. -2. Run the Test using the Unique App URL. - -Please download the latest version of Appium Inspector - :link: [Official Appium Inspector](https://github.com/appium/appium-inspector/releases). - -## Starting an Appium Session - -### Step 1: Upload your Application. - -Upload your **_iOS_** application (.ipa file) or **_android_** application (.apk file) to the TestMu AI servers using our **REST API**. You need to provide your **Username** and **AccessKey** in the format `Username:AccessKey` in the **cURL** command for authentication. Make sure to add the path of the **appFile** in the cURL request. Here is an example cURL request to upload your app using our REST API: - -**Using App File:** - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" \\ ---location --request POST 'https://manual-api.lambdatest.com/app/upload/realDevice' \\ ---form 'name="Android_App"' \\ ---form 'appFile=@"/Users/macuser/Downloads/proverbial_android.apk"' -`} - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "appFile=@"/Users/macuser/Downloads/proverbial_android.apk""`} - -**Using App URL:** - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" --location --request POST "https://manual-api.lambdatest.com/app/upload/realDevice" --header "Content-Type: application/x-www-form-urlencoded" --data-urlencode "url=:https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk" --data-urlencode "name=Proverbial_App"`} - -{`curl --location --request POST "https://manual-api.lambdatest.com/app/upload/realDevice" -u "${YOUR_LAMBDATEST_USERNAME()}:${YOUR_LAMBDATEST_ACCESS_KEY()}" --header "Content-Type: application/x-www-form-urlencoded" --data-urlencode "url=https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk" --data-urlencode "name=test" --data-urlencode "custom_id=my_app"`} - -- If you do not have any **.apk** or **.ipa** file, you can run your sample tests on TestMu AI by using our sample :link: [Android app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk) or sample :link: [iOS app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_ios.ipa). -- Response of above cURL will be a **JSON** object containing the `App URL` of the format - ``lt://APP123456789123456789`` and will be used in the next step. - -### Step 2: Start a Session on Appium Inspector - -Start the Appium Inspector & Select TestMu AI from the list of Cloud Test Providers. - -### Step 3: Configure your Credentials - -Enter in your UserName & Access Key. You may find the credentials available on [TestMu AI Dashboard](https://www.testmuai.com/login/?redirectTo=https://appautomation.lambdatest.com/) - -### Step 4: Configure Desired Capabilities & Start the Session - -Configure TestMu AI capabilities in the desired capabilities tab on Appium inspector. Use the unique app URL obtained in Step 1 to set the app capability Value.For more details, please refer to our guide on [Desired Capabilities in Appium](/support/docs/desired-capabilities-in-appium/). - -Alternatively, we can also go to our [capabilities generator](https://www.testmuai.com/capabilities-generator/) and generate the Capability Representation using GUI. - -Once this is complete, you can now run the test by clicking on start session. Once you start the session, a video recording along with detailed information and logs of the test run will be available on the [TestMu AI Dashboard](https://www.testmuai.com/login/?redirectTo=https://appautomation.lambdatest.com/build). - -## Attaching to a running session in Appium Inspector - -### Step 1: Configure your Credentials -After you have added your credentials from [Step 2 of Starting an Appium Session](/support/docs/appium-inspector-integration/#attaching-to-a-running-session-in-appium-inspector), click on Attach to Session. - -### Step 2: Run a session -Start an App Automation that you'd like to attach to. - -### Step 3: Choose the session to attach -Click on the refresh button and choose a session to attach to. - -### Step 4: Attach to Session -Once you have selected a session, click on Attach to Session. - -### Step 5: Viewing the session -We have now successfully attached to the session and can see & perform operations as well. - -> Got any questions? -> Please reach out at our **24x7 Chat Support** or you could also mail us at [support@testmuai.com](https://support.lambdatest.com/). diff --git a/static/docs/appium-install-uninstall-testmu-hooks.md b/static/docs/appium-install-uninstall-testmu-hooks.md deleted file mode 100644 index c6fd3e0c5..000000000 --- a/static/docs/appium-install-uninstall-testmu-hooks.md +++ /dev/null @@ -1,108 +0,0 @@ -# Ability to Install and Uninstall Apps in App Automation Scripts - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Introduction - -TestMu AI now allows you to install and uninstall apps in the middle of your automated tests. This means you don’t need to restart the session every time you want to switch apps, upgrade them, or clean up the device. - -## Installing and Uninstalling Apps - -TestMu AI enables you to install and uninstall applications during automation test execution by using commands that can be run through WebDriver's script execution functionality. - -### Install App - -You can install apps uploaded to the TestMu AI platform directly within your automation scripts using the `lambda-install-app` command followed by the app’s ID. This ID can be either the TestMu AI-generated ID or a custom ID assigned during upload. - -**Python example:** -```python -driver.execute_script("lambda-install-app=myApp") -driver.execute_script("lambda-install-app=lt://APP100000000123456789123456789") -``` - -**JavaScript example:** -```javascript -await browser.execute("lambda-install-app", { -/* Change the App URL */ -appUrl: "lt://APP1234567890ABCDEF1234567890", -/* Optional */ -retainData: true -}); -``` - -### Uninstall App - -You can uninstall apps during the execution of an automation test using the command `lambda-uninstall-app` followed by the app's package name (for Android apps) or bundle ID (for iOS apps). - -**Python example:** -```python -driver.execute_script("lambda-uninstall-app=com.myApp.beta") -driver.execute_script("lambda-uninstall-app=com.apple.myApp") -``` - -**JavaScript example:** -```javascript -/* Change the App ID */ -const APP_ID = "com.lambdatest.proverbial"; -await browser.execute(`lambda-uninstall-app=${APP_ID}`); -``` - -### Test App Upgrades with Hooks - -Testing app upgrades is important because users often update to the latest version instead of reinstalling the app. To ensure existing user data is preserved and the app continues to function properly after an upgrade, you can use the following hooks: - -**Python example:** - -```python -# Payload required to retain app data while uninstalling it -data = { -"appPackage": "sampleapp.android.app", -"retainData": True -} - -driver.execute_script("lambda-uninstall-app", data) - -# Payload required to reuse data of the old app while installing the new one -data = { -"appUrl": "lt://APPID", -"retainData": True -} -driver.execute_script("lambda-install-app", data) -``` - -```python -# Send the current app in background -driver.background_app(-1) - -# Payload required while installing the new upgrade -data = { -"appUrl": "lt://APPID", -"retainData": True -} -driver.execute_script("lambda-install-app", data) - -# Note : In case of enterprise app, user have to pass the below payload -data = { -"appUrl": "lt://APPID", -"resignApp": False, -"retainData": True -} - -``` - -## Validation Errors - -You might encounter some errors while using these features. The following are some common validation errors and their meaning: - -| Error Message | Meaning | -| --------------------------------------------------- | -------------------------------------------------------------- | -| `No app_url has been provided for lambda-install-app. Please check and try again.`| The install command is missing the app URL or ID. | -| `The app provided for lambda-install-app is not accessible. Please check and try again.`| The app ID does not belong to your account or is inaccessible. | -| `Failed to fetch app details.` | The app ID is invalid or app is not found. | -| `Failed to install the app using lambda-install-app.` | Installation failed due to compatibility or other issues. | -| `No app package or app bundle id has been provided for lambda-uninstall-app. Please check and try again.` | The uninstall command is missing the app package or bundle ID. | -| `Failed to uninstall the app using lambda-uninstall-app.`| Uninstallation failed; app may not be installed or wrong ID.| - -Multiple errors may occur based on the scenario. Understanding these errors will help you identify and resolve issues more efficiently, ensuring smooth test execution. Once the code snippets are added, the tests can be executed and will appear on the[TestMu AI App Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://appautomation.lambdatest.com/build). - -> **Note:** If the user had enabled any of the following capabilities in the previously installed app, they will remain available and functional in the upgraded app as well : `EnableScreenshotUnblock`,`EnableImageInjection`,`EnableVideoInjection`,`Network`,`AppProfiling`,`EnableWebContentsDebugging`,`EnableBiometricInjection`.This ensures that key testing features continue to work seamlessly after the app is updated, without requiring additional configuration. diff --git a/static/docs/appium-ios-app-settings.md b/static/docs/appium-ios-app-settings.md deleted file mode 100644 index bb5964d88..000000000 --- a/static/docs/appium-ios-app-settings.md +++ /dev/null @@ -1,155 +0,0 @@ -# iOS App Settings - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -**iOS App Settings** are the permissions or preferences that can be enabled/disabled for an app through iOS settings. Accessing the device settings is restricted on the iOS public cloud devices of TestMu AI due to security constraints. However, in multiple cases, the native app must be tested for various permissions which can only be enabled and disabled with the settings app. - -This document will guide you on how to access and use iOS settings for **App Automation session.** - -This feature is supported only on iOS 14 and above devices. - -## How to use iOS app settings in App Automation - -To make it easier for the user to access the iOS app settings, we have created a **lambda-hook**. This web-hook supports the opening and manipulation of the settings. - -The hook `"lambda-ios-settings"` takes a JSON parameter which tells which settings we need to manipulate. Below shown is an example of the lambda hook written in python3. - -```bash -params = {"Permission Settings":{"Camera":"Off", "Location":"While using the app"},"Reset App":"On","Next Page":{"Child Toggle":"On"},"Others":"White","Slider-1":"0.1", "TextField-2":"ABCD", "TextField-1":"XYZ"} -driver.execute_script("lambda-ios-settings", params) -``` - -If you look at the code snippet above you can see `params` which contains the **JSON object** for manipulating the iOS app settings. This JSON object contains **Permission settings**. Let’s learn more about that. - -## iOS app settings Supported by Lambda Hook - -### App-specific permission settings -The permission settings are available in the OS and changed through permissions provided by the user. - -**Supported permission settings** - -Currently, App Automate supports the following permission settings: - -- Camera - -- Contacts - -- Photos - -- Notifications - -- Location - -- Precise Location - -One important thing to remember while handling Permission Settings is that, we have to first take action on the **Permission Pop ups** from the app when they appear and then only those particular Permission Settings will be reflected in the app settings page. For example: - -Let's say we pass Camera: "On" in the lambda hook’s preferences even before handling Camera’s permission popup from the app, the Camera key would be missing on the app settings page which would result in failure. So we have to be sure that we have handled the permission pop ups before calling lambda hook. - -Note that we have separate key for Precise Location. We do not pass it inside Location object as shown: - -```bash -params = {"Permission Settings":{"Location":"While using the app", "Precise Location": "On"}} -``` - -### Custom App Settings - -These are the settings added by the app developer using the [iOS Settings Bundle](https://developer.apple.com/library/archive/documentation/Cocoa/Conceptual/UserDefaults/Preferences/Preferences.html#//apple_ref/doc/uid/10000059i-CH6-SW11). - -For a step-by-step guide on adding a Settings Bundle to your app, refer to Apple's official documentation: [Building a settings bundle for your app](https://developer.apple.com/documentation/foundation/building-a-settings-bundle-for-your-app). - -**Supported custom settings** - -Currently, App Automate supports the following custom settings: - -- Toggles - -- Multivalued Settings - -- Nested Settings - -- Sliders - -- Textfields - -Apart from Textfields and Sliders, for every other settings, we have to pass the Name of that particular setting as the **key-value pair**. For example: - -- "Reset App" : "On". - -For **textfields and sliders**, we pass the index of that particular textfield or slider starting from 1 when we traverse from top to bottom. For example: - -1. "Slider-1":"0.1", - -2. "TextField-2":"ABCD" - -The values in Slider are entered on a scale from 0-1 which means the values are not integers but decimals. - -If any key is not present, we should get an error saying one of the keys is missing in the passed preferences. - -### Remember - -The title of the keys displayed on your iOS app settings page must be unique. - -## Apply iOS app settings with the `updateAppSettings` capability - -Besides the in-session `lambda-ios-settings` hook, which applies settings **on demand** during a running test, you can now apply the same iOS Settings Bundle values **automatically at session start** by passing the `updateAppSettings` capability. TestMu AI applies the settings right after the app is installed and **before it launches**, so the app reads the desired values on its very first launch, with no in-test hook call required. - -**When to use which** - -| | `updateAppSettings` capability | `lambda-ios-settings` hook | -|---|---|---| -| Applied | Once, pre-launch (at session start) | On demand, any time during the test | -| Best for | Baseline settings the app should have before first launch | Changing settings mid-run | - -You can use both in the same session. The hook applies over the capability values if you change them later. - -### Requirements - -- iOS **real device**, **App Automation** (Appium) session. -- The app under test must include a **Settings Bundle** (`Settings.bundle`). -- The payload uses the **same structure** as the `lambda-ios-settings` hook (see [iOS app settings Supported by Lambda Hook](#ios-app-settings-supported-by-lambda-hook) above). - -### Usage - -Pass `updateAppSettings` inside `lt:options` (W3C). Below shown is an example written in python3. - -```python -options = { -"platformName": "iOS", -"lt:options": { -"deviceName": "iPhone 15", -"platformVersion": "17", -"isRealMobile": True, -"app": "lt://APP_ID", -"updateAppSettings": { -"Permission Settings": { -"Location": "While using the app", -"Precise Location": "On" -}, -"Allow Cross-Website Tracking": "On", -"Environment": "QA_1", -"Slider-1": "0.5", -"TextField-1": "sample text" -} -} -} -``` - -The keys and values follow the same rules as the hook: setting **titles must match the app's iOS Settings page exactly** and be unique, sliders use a 0–1 decimal scale, and textfields and sliders are indexed (e.g. `Slider-1`, `TextField-2`). - -### Errors and validation - -`updateAppSettings` is validated when the session is created and **fails fast** (no device is allocated) if the target is not applicable: - -| Scenario | Error message | -|---|---| -| App has no Settings Bundle | `Invalid test parameters: updateAppSettings capability is not supported for the app_id specified in the 'app' params - the app does not have a Settings Bundle. Remove the updateAppSettings capability or upload a build that includes Settings.bundle.` | -| Web / no-app session | `updateAppSettings capability is only supported for iOS real device app automation tests` | -| Android session | `updateAppSettings capability is not supported with android platform` | - -## Additional Links - -- [Advanced Configuration for Capabilities](/support/docs/desired-capabilities-in-appium/) -- [How to test locally hosted apps](/support/docs/testing-locally-hosted-pages/) -- [How to integrate TestMu AI with CI/CD](/support/docs/integrations-with-ci-cd-tools/) -- [Building a settings bundle for your app (Apple Developer)](https://developer.apple.com/documentation/foundation/building-a-settings-bundle-for-your-app) diff --git a/static/docs/appium-ip-geolocation.md b/static/docs/appium-ip-geolocation.md deleted file mode 100644 index 58a54bab2..000000000 --- a/static/docs/appium-ip-geolocation.md +++ /dev/null @@ -1,206 +0,0 @@ -# IP Geolocation - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI's IP Geolocation feature empowers users to test their websites and mobile applications from various geographical locations worldwide. This feature enables users to simulate traffic originating from different countries and regions, allowing for comprehensive testing of platform functionalities across diverse global markets. - -By the end of this document, you'll learn how to use this feature in both manual and automation tests. Additionally, this document will provide a list of all supported IP geolocations on TestMu AI. - -## Support for Automation on Real Devices - -To harness the **IP Geolocation** feature in your automated tests, employ the `geoLocation` capability along with a two-letter country code from the list of supported countries provided below. This capability enables you to modify the device's geolocation to the specified location. If this capability is not added, the device will automatically update its geolocation based on its physical location. - -**Java Example:** -```java -// Java code for configuring IP Geolcation in tests on TestMu AI -DesiredCapabilities capabilities = new DesiredCapabilities(); -capabilities.setCapability("geoLocation", "FR"); -// Add other capabilities as needed -``` - -## Supported IP Geolocations - -Below is the list of supported countries along with their respective country codes: - -| Name | Code | -| --------------------------------- | -------- | -| Adelaide (Australia) | AU/AL | -| Albania | AL | -| Alberta (Canada) | CA/AB | -| Albuquerque, New Mexico (US) | US/ABQ | -| Argentina | AR | -| Armenia | AM | -| Ashburn, Virginia (US) | US/AB | -| Atlanta, Georgia (US) | US/AT | -| Australia | AU | -| Austria | AT | -| Azerbaijan | AZ | -| Bahrain | BH | -| Bangladesh | BD | -| Belarus | BY | -| Belgium | BE | -| Berlin | V2 | -| Berkeley Springs, WV (US) | US/BS | -| Billings, Montana (US) | US/BL | -| Boca Raton, Florida (US) | US/BR | -| Bolivia (La Paz) | BO/LPZ | -| Bosnia and Herzegovina | BA | -| Boston, Massachusetts (US) | US/BO | -| Brazil | BR | -| British Colombia (Canada) | CA/BC | -| Brunswick, Maine (US) | US/BWK | -| Buffalo, New York (US) | US/BU | -| Bulgaria | BG | -| Cambodia | KH | -| Canada | CA | -| Canberra (Australia) | AU/CN | -| Charlotte, NC (US) | US/CH | -| Cheyenne, Wyoming (US) | US/CY | -| Chile | CL | -| China | CN | -| Chicago, Illinois (US) | US/CHI | -| Colombia | CO | -| Columbus, Ohio (US) | US/CO | -| Connecticut (US) | US/CN | -| Costa Rica | CR | -| Cromwell, Connecticut (US) | US/CM | -| Curacao | CW | -| Cyprus | CY | -| Czech Republic | CZ | -| Dallas, Texas (US) | US/DL | -| Dallas-Ft. Worth, Texas (US) | US/DFW | -| Denmark | DK | -| Denver, Colorado (US) | Y6 | -| Des Moines, Iowa (US) | US/DM | -| Detroit, Michigan (US) | US/DET | -| Dominican Republic | DO | -| Ecuador | EC | -| Egypt | EG | -| El Salvador | SV | -| Estonia | EE | -| Finland | FI | -| France | FR | -| Georgia | GE | -| Germany | DE | -| Greece | GR | -| Guatemala | GT | -| Harrisburg, Pennsylvania (US) | US/HB | -| Hollywood, Florida (US) | US/HW | -| Honduras | HN | -| Hong Kong | HK | -| Honolulu, HI | U3 | -| Houston, Texas (US) | US/HO | -| Hungary | HU | -| Iceland | IS | -| India | IN | -| Indonesia | ID | -| Indianapolis, Indiana (US) | US/IN | -| Iowa (US) | US/IW | -| Ireland | IE | -| Isle of Man | IM | -| Israel | IL | -| Italy | IT | -| Jamaica | JM | -| Japan | JP | -| Jersey City, New Jersey (US) | US/JC | -| Jordan | JO | -| Kansas City, Missouri (US) | US/KC | -| Kazakhstan | KZ | -| Kenya | KE | -| Korea | KR | -| Kuwait | KW | -| Kyrgyzstan | KG | -| La Paz (Bolivia) | BO/LPZ | -| Las Vegas, Nevada (US) | US/LV | -| Latvia | LV | -| Lebanon | LB | -| Lincoln, Nebraska (US) | US/LC | -| Lithuania | LT | -| London | W8 | -| Los Angeles, California (US) | US/LA | -| Luxembourg | LU | -| Manhattan, New York (US) | US/MHT | -| Maryland (US) | US/ML | -| Malta | MT | -| Melbourne (Australia) | AU/ML | -| Memphis, Tennessee (US) | W4 | -| Mexico | MX | -| Miami, Florida (US) | US/MI | -| Moldova | MD | -| Mongolia | MN | -| Montana (US) | US/MT | -| Monticello, Illinois (US) | US/MO | -| Morocco | MA | -| Nebraska (US) | US/NB | -| New Hamisphere (US) | US/NH | -| New Jersey, New Jersey | X4 | -| New Orleans, Louisiana (US) | US/NO | -| New York City, New York (US) | US/NYC | -| New York City, New York (US) | Z0 | -| New Zealand | NZ | -| Nicaragua | NI | -| Nigeria | NG | -| Norway | NO | -| Oklahoma City, Oklahoma (US) | US/OKC | -| Oman | OM | -| Ontario (Canada) | CA/ON | -| Orlando, Florida (US) | US/ORL | -| Pakistan | PK | -| Panama | PA | -| Paraguay | PY | -| Perth (Australia) | AU/PT | -| Peru | PE | -| Philippines | PH | -| Phoenix, Arizona (US) | US/PHX | -| Pittsburgh, Pennsylvania (US) | US/PIT | -| Poland | PL | -| Portugal | PT | -| Puerto Rico | PR | -| Qatar | QA | -| Raleigh, North Carolina (US) | US/RA | -| Reston, Virginia (US) | US/RT | -| Richmond, Virginia (US) | US/RM | -| Romania | RO | -| Russian Federation | RU | -| Sacramento, California (US) | US/SAC | -| Salem, New Hampshire (US) | US/SLE | -| Salem, NH | U5 | -| Salt Lake City, Utah (US) | US/SLC | -| San Diego, California (US) | US/SD | -| San Francisco, California (US) | US/SF | -| San Jose, California (US) | US/SJ | -| Santa Clara, California (US) | US/SC | -| Saudi Arabia | SA | -| Serbia | RS | -| Seattle, Washington (US) | US/SEA | -| Singapore | SG | -| Sioux Falls, South Dakota (US) | US/SFL | -| Slovakia | SK | -| Slovenia | SI | -| South Africa | ZA | -| South Bend, Indiana (US) | US/SB | -| Spain | ES | -| Sterling, Virginia (US) | US/ST | -| St Louis, Missouri (US) | US/STL | -| Sweden | SE | -| Switzerland | CH | -| Sydney (Australia) | AU/SY | -| Taiwan | TW | -| Tampa, Florida (US) | US/TPA | -| Tanzania | TZ | -| Thailand | TH | -| Tunisia | TN | -| Turkey | TR | -| Turkmenistan | TM | -| Ukraine | UA | -| United Arab Emirates | AE | -| United Kingdom | GB | -| United States | US | -| Uruguay | UY | -| Uzbekistan | UZ | -| Venezuela | VE | -| Vietnam | VN | -| Washington | Z6 | -| Washington (Herndon), Virginia | US/WAV | -| Wilmington, DE | U4 | -| Wilmington, Delaware (US) | US/WILM | diff --git a/static/docs/appium-java-cucumber.md b/static/docs/appium-java-cucumber.md deleted file mode 100644 index 6f1ae43fd..000000000 --- a/static/docs/appium-java-cucumber.md +++ /dev/null @@ -1,244 +0,0 @@ -# Appium With Cucumber for Java - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -In this documentation, you will learn how to trigger a automation script of **Cucumber** for application testing with **Appium** on TestMu AI, set the [**desired capabilities**](/support/docs/desired-capabilities-in-appium/) for appium testing, and other advanced features of TestMu AI. - -## Prerequisites - -- Your TestMu AI [Username and Access key](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/security). -- You should have [Java client library](https://github.com/appium/java-client) installed for Selenium and Appium. -- Install the latest Java development environment i.e. JDK 8 or higher. We recommend using the < JDK 11 version. -- Download and install **Maven** from [the official website](https://maven.apache.org/). For **Linux/macOS** you can use [Homebrew](https://brew.sh/) package manager. - -## Try our Sample Repository - -### Step 1: Get a Sample Project -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** -All the code samples in this documentation can be found on **TestMu AI's Github Repository**. You can either download or clone the repository to quickly run your tests. View on GitHub - -### Step 2: Setup the Environment Variables - -You need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in your [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/security). Run the below mentioned commands in your terminal to setup the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -### Step 3: Upload your Application -Upload your **_iOS_** application (.ipa file) or **_android_** application (.apk or .aab file) to the TestMu AI servers using our **REST API**. You need to provide your **Username** and **AccessKey** in the format `Username:AccessKey` in the **cURL** command for authentication. - -Make sure to add the path of the **appFile** in the cURL request. Below is an example cURL request to upload your app using our REST API: - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "appFile=@"/Users/macuser/Downloads/proverbial_android.apk"" -F "name="proverbial_app""`} - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "url=:https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk" -F "name=Proverbial_App"`} - -- If you do not have any **.apk** or **.ipa** file, you can run your sample tests on TestMu AI by using our sample apps, :link: [Android app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk) or :link: [iOS app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_ios.ipa). - -- Response of above cURL will be a **JSON** object containing the `APP_URL` of the format - `lt://APP123456789123456789` and will be used in the next step - -### Step 4: Update your Automation Script - -An automation script for the sample application given above has been provided here. Ensure to update the `APP_URL`, `username` and `accessKey` in the code scripts before running the tests. - -```java package MyRunner; -import java.net.URL; - -import io.cucumber.testng.AbstractTestNGCucumberTests; -import io.cucumber.testng.CucumberOptions; -import io.cucumber.testng.TestNGCucumberRunner; -import org.openqa.selenium.remote.CapabilityType; -import org.openqa.selenium.remote.DesiredCapabilities; -import org.openqa.selenium.remote.AppiumDriver; -import org.testng.annotations.AfterClass; -import org.testng.annotations.BeforeClass; -import org.testng.annotations.BeforeMethod; -import org.testng.annotations.DataProvider; -import org.testng.annotations.Parameters; - -import org.openqa.selenium.JavascriptExecutor; -import java.net.MalformedURLException; - -@CucumberOptions( -features = "src/main/java/Features/todo.feature", -glue = {"stepDefinitions"}, -plugin = "json:target/cucumber-reports/CucumberTestReport.json") - -public class TestRunner extends AbstractTestNGCucumberTests { - -private TestNGCucumberRunner testNGCucumberRunner; - -public static RemoteWebDriver connection; - -@BeforeClass(alwaysRun = true) -public void setUpCucumber() { -testNGCucumberRunner = new TestNGCucumberRunner(this.getClass()); -} - -@BeforeMethod(alwaysRun = true) -@Parameters({ "deviceName", "platformVersion", "platformName" }) -public void setUpClass(String deviceName, String platformVersion, String platformName) throws Exception { - -String username = System.getenv("LT_USERNAME") == null ? "YOUR_LT_USERNAME" : System.getenv("LT_USERNAME"); //Enter your LambdaTest username at the place of YOUR_LT_USERNAME -String accesskey = System.getenv("LT_ACCESS_KEY") == null ? "YOUR_LT_ACCESSKEY" : System.getenv("LT_ACCESS_KEY"); //Enter your LambdaTest accessKey at the place of YOUR_LT_ACCESSKEY - -DesiredCapabilities capability = new DesiredCapabilities(); - -capability.setCapability("platformName", platformName); -capability.setCapability("deviceName", deviceName); -capability.setCapability("platformVersion",platformVersion); - -capability.setCapability("build", "Native App automate Demo"); -capability.setCapability("test", "Test Parallel"); -capability.setCapability("isRealMobile", true); -// highlight-next-line -capability.setCapability("app","lt://proverbial-android"); //Enter the app url here -capability.setCapability("network", false); -capability.setCapability("video", true); -capability.setCapability("console", true); -capability.setCapability("visual", true); - -String gridURL = "https://" + username + ":" + accesskey + "@mobile-hub.lambdatest.com/wd/hub"; -System.out.println(gridURL); -connection = new RemoteWebDriver(new URL(gridURL), capability); -System.out.println(capability); -System.out.println(connection.getSessionId()); -} - -@DataProvider -public Object[][] features() { -return testNGCucumberRunner.provideScenarios(); -} - -@AfterClass(alwaysRun = true) -public void tearDownClass() { -testNGCucumberRunner.finish(); -} -} -``` - -- You must set **isRealMobile** capability to `False` in the config file to run on **Virtual Devices** - -### Step 5: Configure the Test Capabilities - -You can update your custom capabilities in test scripts. In this sample project, we are passing platform name, platform version, device name and app url _(generated earlier)_ along with other capabilities like build name and test name via capabilities object. - -The capabilities object in the sample code are defined as: - -```java - - - - - - - - - - - - -``` - -```java - - - - - - - - - - - - - - - - - - - - - -``` - -- You must set **isRealMobile** capability to `False` in the config file to run on **Virtual Devices** - -- You must add the generated **APP_URL** to the `app` capability in the config file. -- You can generate capabilities for your test requirements with the help of our inbuilt [**Capabilities Generator tool**](https://www.testmuai.com/capabilities-generator/).For more details, please refer to our guide on [**Desired Capabilities in Appium**](/support/docs/desired-capabilities-in-appium/). - -### Step 6: Execute and Monitor your Tests - -- Run the following commands to install the required dependencies: - -```bash -mvn clean install -``` - -- The tests can be executed in the terminal using the following command: - -```bash -mvn test -D suite=single.xml -``` - -```bash -mvn test -D suite=parallel.xml -``` - - > Your test results would be displayed on the test console (or CLI if you are using terminal/cmd) and on the [TestMu AI App Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://appautomation.lambdatest.com/build). - -## Using the Cucumber Agent Skill with TestMu AI - -The [cucumber-skill](https://github.com/LambdaTest/agent-skills/tree/main/cucumber-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The cucumber-skill package includes: - -``` -cucumber-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing Cucumber Agent Skill - -Install a Cucumber Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/cucumber-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/cucumber-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only cucumber-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). - -## Reference Guides - -- [Advanced Configuration for Capabilities](/support/docs/desired-capabilities-in-appium/) -- [How to test locally hosted apps](/support/docs/testing-locally-hosted-pages/) -- [How to integrate TestMu AI with CI/CD](/support/docs/integrations-with-ci-cd-tools/) diff --git a/static/docs/appium-java-jbehave.md b/static/docs/appium-java-jbehave.md deleted file mode 100644 index 7f653334b..000000000 --- a/static/docs/appium-java-jbehave.md +++ /dev/null @@ -1,278 +0,0 @@ -# Appium With JBehave - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -In this documentation, you will learn how to trigger a automation script of **JBehave** for application testing with **Appium** on TestMu AI, set the [**desired capabilities**](/support/docs/desired-capabilities-in-appium/) for appium testing, and other advanced features of TestMu AI. - -## Prerequisites - -- Your TestMu AI [Username and Access key](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/security). -- You should have [Java client library](https://github.com/appium/java-client) installed for Selenium and Appium. -- Download and install **Maven** from [the official website](https://maven.apache.org/). For **Linux/macOS** you can use [Homebrew](https://brew.sh/) package manager. - -## Try our Sample Repository - -### Step 1: Get a Sample Project -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** -All the code samples in this documentation can be found on **TestMu AI's Github Repository**. You can either download or clone the repository to quickly run your tests. View on GitHub - -### Step 2: Setup the Environment Variables - -You need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in your [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/security). Run the below mentioned commands in your terminal to setup the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -### Step 3: Upload your Application -Upload your **_iOS_** application (.ipa file) or **_android_** application (.apk or .aab file) to the TestMu AI servers using our **REST API**. You need to provide your **Username** and **AccessKey** in the format `Username:AccessKey` in the **cURL** command for authentication. - -Make sure to add the path of the **appFile** in the cURL request. Below is an example cURL request to upload your app using our REST API: - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "appFile=@"/Users/macuser/Downloads/proverbial_android.apk"" -F "name="proverbial_app""`} - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "url=:https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk" -F "name=Proverbial_App"`} - -- If you do not have any **.apk** or **.ipa** file, you can run your sample tests on TestMu AI by using our sample apps, :link: [Android app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk) or :link: [iOS app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_ios.ipa). - -- Response of above cURL will be a **JSON** object containing the `APP_URL` of the format - `lt://APP123456789123456789` and will be used in the next step - -### Step 4: Update your Automation Script - -An automation script for the sample application given above has been provided here. Ensure to update the `APP_URL`, `username` and `accessKey` in the code scripts before running the tests. - -```java - -package com.lambdatest; -import java.io.FileReader; -import java.net.URL; -import java.util.Map; -import java.util.ArrayList; -import java.util.Arrays; -import java.util.List; -import java.util.Iterator; -import java.util.Collection; -import java.lang.reflect.Constructor; -import org.json.simple.JSONObject; -import org.json.simple.JSONArray; -import org.json.simple.parser.JSONParser; - -import org.jbehave.core.embedder.Embedder; -import org.junit.Test; -import org.junit.After; -import org.junit.Before; -import org.junit.runner.RunWith; -import org.junit.runners.Parameterized; -import org.junit.runners.Parameterized.Parameters; -import org.junit.runners.Parameterized.Parameter; - -import org.openqa.selenium.WebDriver; -import org.openqa.selenium.remote.RemoteWebDriver; -import org.openqa.selenium.remote.DesiredCapabilities; - -@RunWith(Parameterized.class) -public class LambdaTestJBehaveRunner { - -public WebDriver driver; -// private Local l; - -private static JSONObject config; - -@Parameter(value = 0) -public int taskID; - -@Parameters -public static Collection data() throws Exception { -List taskIDs = new ArrayList(); -if(System.getProperty("config") != null) { -JSONParser parser = new JSONParser(); -config = (JSONObject) parser.parse(new FileReader("src/test/resources/conf/" + System.getProperty("config"))); -int envs = ((JSONArray)config.get("environments")).size(); - -for(int i=0; i envCapabilities = (Map) envs.get(taskID); -Iterator it = envCapabilities.entrySet().iterator(); -while (it.hasNext()) { -Map.Entry pair = (Map.Entry)it.next(); -capabilities.setCapability(pair.getKey().toString(), pair.getValue().toString()); -} - -Map commonCapabilities = (Map) config.get("capabilities"); -it = commonCapabilities.entrySet().iterator(); -while (it.hasNext()) { -Map.Entry pair = (Map.Entry)it.next(); -if(capabilities.getCapability(pair.getKey().toString()) == null){ -capabilities.setCapability(pair.getKey().toString(), pair.getValue().toString()); -} -} - -String username = System.getenv("LT_USERNAME") == null ? "YOUR_LT_USERNAME" : System.getenv("LT_USERNAME"); //Replace YOUR_LT_USERNAME with your LambdaTest username - -String accessKey = System.getenv("LT_ACCESS_KEY") == null ? "YOUR_LT_ACCESS_KEY" : System.getenv("LT_ACCESS_KEY"); //Replace YOUR_LT_ACCESS_KEY with your LambdaTest accessKey - -driver = new RemoteWebDriver(new URL("http://"+username+":"+accessKey+"@"+config.get("server")+"/wd/hub"), capabilities); -} - -@After -public void tearDown() throws Exception { -driver.quit(); -} - -@Test -public void runStories() throws Exception { -Class c = Class.forName(System.getProperty("embedder")); -Constructor cons = c.getConstructor(WebDriver.class); -Embedder storyEmbedder = (Embedder) cons.newInstance(driver); - -List storyPaths = Arrays.asList(System.getProperty("stories")); -storyEmbedder.runStoriesAsPaths(storyPaths); -} -} -``` - -- You must set **isRealMobile** capability to `False` in the config file to run on **Virtual Devices** - -### Step 5: Configure the Test Capabilities - -You can update your custom capabilities in test scripts. In this sample project, we are passing platform name, platform version, device name and app url _(generated earlier)_ along with other capabilities like build name and test name via capabilities object. - -The capabilities object in the sample code are defined as: - -```java -//Single Tests -{ -"server": "mobile-hub.lambdatest.com", - -"capabilities": { -"build": "jbehave-LambdaTest-AppAutomate-Single", -"name": "Single Test 1" -}, - -"environments": [ -{ -"platformName": "android", -"deviceName": "Galaxy S21 5G", -"platformVersion": "11" -} -] -} -``` - -```java -//Parallel Tests -{ -"server": "mobile-hub.lambdatest.com", - -"capabilities": { -"build": "jbehave-LambdaTest-AppAutomate-Parallel", -"name": "Parallel Test 1" -}, - -"environments": [ -{ -"platformName": "android", -"deviceName": "Galaxy S21 5G", -"platformVersion": "11" -}, -{ -"platformName": "android", -"deviceName": "OnePlus 10 Pro", -"platformVersion": "12" -}, -{ -"platformName": "android", -"deviceName": "Pixel 3a", -"platformVersion": "10" -} -] -} -``` - -- You must set **isRealMobile** capability to `False` in the config file to run on **Virtual Devices** - -- You must add the generated **APP_URL** to the `app` capability in the config file. -- You can generate capabilities for your test requirements with the help of our inbuilt [**Capabilities Generator tool**](https://www.testmuai.com/capabilities-generator/).For more details, please refer to our guide on [**Desired Capabilities in Appium**](/support/docs/desired-capabilities-in-appium/). - -### Step 6: Execute and Monitor your Tests - -- Run the following commands to install the required dependencies: - -```bash -mvn clean install -``` - -- The tests can be executed in the terminal using the following command: - -```bash -mvn test -P single -``` - -```bash -mvn test -P parallel -``` - - > Your test results would be displayed on the test console (or CLI if you are using terminal/cmd) and on the [TestMu AI App Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://appautomation.lambdatest.com/build). - -## Using the Appium Agent Skill with TestMu AI - -The [appium-skill](https://github.com/LambdaTest/agent-skills/tree/main/appium-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The appium-skill package includes: - -``` -appium-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing Appium Agent Skill - -Install a Appium Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/appium-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/appium-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only appium-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). - -## Reference Guides - -- [Advanced Configuration for Capabilities](/support/docs/desired-capabilities-in-appium/) -- [How to test locally hosted apps](/support/docs/testing-locally-hosted-pages/) -- [How to integrate TestMu AI with CI/CD](/support/docs/integrations-with-ci-cd-tools/) diff --git a/static/docs/appium-java-junit.md b/static/docs/appium-java-junit.md deleted file mode 100644 index 5ff6a2dfe..000000000 --- a/static/docs/appium-java-junit.md +++ /dev/null @@ -1,361 +0,0 @@ -# Appium With JUnit - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -In this documentation, you will learn how to trigger a automation script of **JUnit** for application testing with **Appium** on TestMu AI, set the [**desired capabilities**](/support/docs/desired-capabilities-in-appium/) for appium testing, and other advanced features of TestMu AI. - -## Prerequisites - -- Your TestMu AI [Username and Access key](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/security). -- You should have [Java client library](https://github.com/appium/java-client) installed for Appium. -- Download and install **Maven** from [the official website](https://maven.apache.org/). For **Linux/macOS** you can use [Homebrew](https://brew.sh/) package manager. - -## Try our Sample Repository - -### Step 1: Get a Sample Project -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** -All the code samples in this documentation can be found on **TestMu AI's Github Repository**. You can either download or clone the repository to quickly run your tests. View on GitHub - -### Step 2: Setup the Environment Variables - -You need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in your [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/security). Run the below mentioned commands in your terminal to setup the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -### Step 3: Upload your Application -Upload your **_iOS_** application (.ipa file) or **_android_** application (.apk or .aab file) to the TestMu AI servers using our **REST API**. You need to provide your **Username** and **AccessKey** in the format `Username:AccessKey` in the **cURL** command for authentication. - -Make sure to add the path of the **appFile** in the cURL request. Below is an example cURL request to upload your app using our REST API: - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "appFile=@"/Users/macuser/Downloads/proverbial_android.apk"" -F "name="proverbial_app""`} - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "url=:https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk" -F "name=Proverbial_App"`} - -- If you do not have any **.apk** or **.ipa** file, you can run your sample tests on TestMu AI by using our sample apps, :link: [Android app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk) or :link: [iOS app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_ios.ipa). - -- Response of above cURL will be a **JSON** object containing the `APP_URL` of the format - `lt://APP123456789123456789` and will be used in the next step - -### Step 4: Update your Automation Script - -An automation script for the sample application given above has been provided here. Ensure to update the `APP_URL`, `username` and `accessKey` in the code scripts before running the tests. - -```java -package com.lambdatest; - -import io.appium.java_client.MobileBy; -import org.junit.After; -import org.junit.Before; -import org.junit.Test; -import org.openqa.selenium.remote.DesiredCapabilities; -import org.openqa.selenium.remote.RemoteWebDriver; -import org.openqa.selenium.By; -import org.openqa.selenium.support.ui.ExpectedConditions; -import org.openqa.selenium.support.ui.WebDriverWait; -import java.net.MalformedURLException; -import java.net.URL; - -public class android { -String username = System.getenv("LT_USERNAME") == null ? "LT_USERNAME" //Enter the Username here -: System.getenv("LT_USERNAME"); -String accessKey = System.getenv("LT_ACCESS_KEY") == null ? "LT_ACCESS_KEY" //Enter the accessKey here -: System.getenv("LT_ACCESS_KEY"); -public static RemoteWebDriver driver = null; -public String gridURL = "@mobile-hub.lambdatest.com/wd/hub"; -public String status = "passed"; -@Before -public void setUp() throws Exception { -DesiredCapabilities capabilities = new DesiredCapabilities(); - -capabilities.setCapability("build", "JUNIT Native App automation"); -capabilities.setCapability("name", "Java JUnit Android Pixel 6"); -capabilities.setCapability("platformName", "android"); -capabilities.setCapability("deviceName", "Pixel 6"); -capabilities.setCapability("isRealMobile", true); -capabilities.setCapability("platformVersion","12"); -// highlight-next-line -capabilities.setCapability("app","APP_URL"); //Enter your app (.apk) url -capabilities.setCapability("deviceOrientation", "PORTRAIT"); -capabilities.setCapability("console",true); -capabilities.setCapability("network",false); -capabilities.setCapability("visual",true); -try -{ -driver = new RemoteWebDriver(new URL("https://" + username + ":" + accessKey + gridURL), capabilities); -} -catch (MalformedURLException e) -{ -System.out.println("Invalid grid URL"); -} catch (Exception e) -{ -System.out.println(e.getMessage()); -} -} - -@Test -public void testSimple() throws Exception -{ -try -{ -WebDriverWait wait = new WebDriverWait(driver, 30); -wait.until(ExpectedConditions.elementToBeClickable(MobileBy.id("color"))).click(); - -wait.until(ExpectedConditions.elementToBeClickable(MobileBy.id("geoLocation"))).click();; -Thread.sleep(5000); -driver.navigate().back(); - -wait.until(ExpectedConditions.elementToBeClickable(MobileBy.id("Text"))).click(); - -wait.until(ExpectedConditions.elementToBeClickable(MobileBy.id("notification"))).click();; - -wait.until(ExpectedConditions.elementToBeClickable(MobileBy.id("toast"))).click(); - -wait.until(ExpectedConditions.elementToBeClickable(By.id("Browser"))).click();; -Thread.sleep(10000); - -wait.until(ExpectedConditions.elementToBeClickable(MobileBy.id("url"))).sendKeys("https://www.testmuai.com/"); - -wait.until(ExpectedConditions.elementToBeClickable(MobileBy.id("find"))).click(); -Thread.sleep(5000); -driver.navigate().back(); - -status="passed"; -} -catch (Exception e) -{ -System.out.println(e.getMessage()); -status="failed"; -} -} -@After -public void tearDown() throws Exception -{ -if (driver != null) -{ -driver.executeScript("lambda-status=" + status); -driver.quit(); -} -} -} -``` - -- You must set **isRealMobile** capability to `False` in the config file to run on **Virtual Devices** - -```java -package com.lambdatest; - -import io.appium.java_client.MobileBy; -import org.junit.After; -import org.junit.Before; -import org.junit.Test; -import org.openqa.selenium.remote.DesiredCapabilities; -import org.openqa.selenium.remote.RemoteWebDriver; -import org.openqa.selenium.By; -import org.openqa.selenium.support.ui.ExpectedConditions; -import org.openqa.selenium.support.ui.WebDriverWait; -import java.net.MalformedURLException; -import java.net.URL; - -public class ios { -String username = System.getenv("LT_USERNAME") == null ? "LT_USERNAME" //Add username here -: System.getenv("LT_USERNAME"); -String accessKey = System.getenv("LT_ACCESS_KEY") == null ? "LT_ACCESS_KEY" //Add accessKey here -: System.getenv("LT_ACCESS_KEY"); -public static RemoteWebDriver driver = null; -public String gridURL = "@mobile-hub.lambdatest.com/wd/hub"; -public String status = "passed"; -@Before -public void setUp() throws Exception { -DesiredCapabilities capabilities = new DesiredCapabilities(); - -capabilities.setCapability("build", "JUNIT Native App automation"); -capabilities.setCapability("name", "Java JUnit iOS iPhone 12"); -capabilities.setCapability("platformName", "ios"); -capabilities.setCapability("deviceName", "iPhone 12"); -capabilities.setCapability("isRealMobile", true); -capabilities.setCapability("platformVersion","15"); -// highlight-next-line -capabilities.setCapability("app","APP_URL"); //Enter your app (.ipa) url -capabilities.setCapability("deviceOrientation", "PORTRAIT"); -capabilities.setCapability("console",true); -capabilities.setCapability("network",false); -capabilities.setCapability("visual",true); -try -{ -driver = new RemoteWebDriver(new URL("https://" + username + ":" + accessKey + gridURL), capabilities); -} -catch (MalformedURLException e) -{ -System.out.println("Invalid grid URL"); -} catch (Exception e) -{ -System.out.println(e.getMessage()); -} -} - -@Test -public void testSimple() throws Exception -{ -try -{ -WebDriverWait wait = new WebDriverWait(driver, 30); -wait.until(ExpectedConditions.elementToBeClickable(MobileBy.id("color"))).click(); - -wait.until(ExpectedConditions.elementToBeClickable(MobileBy.id("geoLocation"))).click(); -Thread.sleep(5000); -driver.navigate().back(); - -wait.until(ExpectedConditions.elementToBeClickable(MobileBy.id("Text"))).click(); - -wait.until(ExpectedConditions.elementToBeClickable(MobileBy.id("notification"))).click(); - -wait.until(ExpectedConditions.elementToBeClickable(MobileBy.id("toast"))).click(); - -wait.until(ExpectedConditions.elementToBeClickable(By.id("Browser"))).click(); -Thread.sleep(10000); - -wait.until(ExpectedConditions.elementToBeClickable(MobileBy.id("url"))).sendKeys("https://www.testmuai.com/");; - -wait.until(ExpectedConditions.elementToBeClickable(MobileBy.id("find"))).click(); -Thread.sleep(5000); -driver.navigate().back(); - -status="passed"; -} -catch (Exception e) -{ -System.out.println(e.getMessage()); -status="failed"; -} -} -@After -public void tearDown() throws Exception -{ -if (driver != null) -{ -driver.executeScript("lambda-status=" + status); -driver.quit(); -} -} -} -``` - -- You must set **isRealMobile** capability to `False` in the config file to run on **Virtual Devices** - -### Step 5: Configure the Test Capabilities - -You can update your custom capabilities in test scripts. In this sample project, we are passing platform name, platform version, device name and app url _(generated earlier)_ along with other capabilities like build name and test name via capabilities object. - -The capabilities object in the sample code are defined as: - -```java -DesiredCapabilities capabilities = new DesiredCapabilities(); -capabilities.setCapability("build", "JUNIT Native App automation"); -capabilities.setCapability("name", "Java JUnit Android Pixel 6"); -capabilities.setCapability("platformName", "android"); -capabilities.setCapability("deviceName", "Pixel 6"); -capabilities.setCapability("isRealMobile", true); -capabilities.setCapability("platformVersion","12"); -// highlight-next-line -capabilities.setCapability("app","YOUR_APP_URL"); //Enter your app (.apk) url -capabilities.setCapability("deviceOrientation", "PORTRAIT"); -capabilities.setCapability("console",true); -capabilities.setCapability("network",false); -capabilities.setCapability("visual",true); -``` - -```java -DesiredCapabilities capabilities = new DesiredCapabilities(); -capabilities.setCapability("build", "JUNIT Native App automation"); -capabilities.setCapability("name", "Java JUnit iOS iPhone 12"); -capabilities.setCapability("platformName", "ios"); -capabilities.setCapability("deviceName", "iPhone 12"); -capabilities.setCapability("isRealMobile", true); -capabilities.setCapability("platformVersion","15"); -// highlight-next-line -capabilities.setCapability("app","YOUR_APP_URL"); //Enter your app (.ipa) url -capabilities.setCapability("deviceOrientation", "PORTRAIT"); -capabilities.setCapability("console",true); -capabilities.setCapability("network",false); -capabilities.setCapability("visual",true); -``` - -- You must set **isRealMobile** capability to `False` in the config file to run on **Virtual Devices** - -- You must add the generated **APP_URL** to the `app` capability in the config file. -- You can generate capabilities for your test requirements with the help of our inbuilt [**Capabilities Generator tool**](https://www.testmuai.com/capabilities-generator/).For more details, please refer to our guide on [**Desired Capabilities in Appium**](/support/docs/desired-capabilities-in-appium/). - -### Step 6: Execute and Monitor your Tests - -- Run the following commands to install the required dependencies: - -```bash -mvn clean -``` - -- The tests can be executed in the terminal using the following command: - -```bash -mvn test -P android -``` - -```bash -mvn test -P ios -``` - - > Your test results would be displayed on the test console (or CLI if you are using terminal/cmd) and on the [TestMu AI App Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://appautomation.lambdatest.com/build). - -## Using the JUnit 5 Agent Skill with TestMu AI - -The [junit-5-skill](https://github.com/LambdaTest/agent-skills/tree/main/junit-5-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The junit-5-skill package includes: - -``` -junit-5-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing JUnit 5 Agent Skill - -Install a JUnit 5 Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/junit-5-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/junit-5-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only junit-5-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). - -## Additional Links - -- [Advanced Configuration for Capabilities](/support/docs/desired-capabilities-in-appium/) -- [How to test locally hosted apps](/support/docs/testing-locally-hosted-pages/) -- [How to integrate TestMu AI with CI/CD](/support/docs/integrations-with-ci-cd-tools/) - -## Reference Guides - -- [Advanced Configuration for Capabilities](/support/docs/desired-capabilities-in-appium/) -- [How to test locally hosted apps](/support/docs/testing-locally-hosted-pages/) -- [How to integrate TestMu AI with CI/CD](/support/docs/integrations-with-ci-cd-tools/) diff --git a/static/docs/appium-java-testng.md b/static/docs/appium-java-testng.md deleted file mode 100644 index da785f405..000000000 --- a/static/docs/appium-java-testng.md +++ /dev/null @@ -1,370 +0,0 @@ -# Appium With TestNG - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -In this documentation, you will learn how to trigger a automation script of **TestNG** for application testing with **Appium** on TestMu AI, set the [**desired capabilities**](/support/docs/desired-capabilities-in-appium/) for appium testing, and other advanced features of TestMu AI. - -## Prerequisites - -- Your TestMu AI [Username and Access key](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/security). -- You should have [Java client library](https://github.com/appium/java-client) installed for Selenium and Appium. -- Download and install **Maven** from [the official website](https://maven.apache.org/). For **Linux/macOS** you can use [Homebrew](https://brew.sh/) package manager. - -## Try our Sample Repository - -### Step 1: Get a Sample Project -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** -All the code samples in this documentation can be found on **TestMu AI's Github Repository**. You can either download or clone the repository to quickly run your tests. View on GitHub - -### Step 2: Setup the Environment Variables - -You need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in your [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/security). Run the below mentioned commands in your terminal to setup the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -### Step 3: Upload your Application -Upload your **_iOS_** application (.ipa file) or **_android_** application (.apk or .aab file) to the TestMu AI servers using our **REST API**. You need to provide your **Username** and **AccessKey** in the format `Username:AccessKey` in the **cURL** command for authentication. - -Make sure to add the path of the **appFile** in the cURL request. Below is an example cURL request to upload your app using our REST API: - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "appFile=@"/Users/macuser/Downloads/proverbial_android.apk"" -F "name="proverbial_app""`} - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "url=:https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk" -F "name=Proverbial_App"`} - -- If you do not have any **.apk** or **.ipa** file, you can run your sample tests on TestMu AI by using our sample apps, :link: [Android app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk) or :link: [iOS app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_ios.ipa). - -- Response of above cURL will be a **JSON** object containing the `APP_URL` of the format - `lt://APP123456789123456789` and will be used in the next step - -### Step 4: Update your Automation Script - -An automation script for the sample application given above has been provided here. Ensure to update the `APP_URL`, `username` and `accessKey` in the code scripts before running the tests. - -```java title="AndroidApp.java" -import io.appium.java_client.AppiumDriver; -import io.appium.java_client.MobileBy; -import io.appium.java_client.MobileElement; -import io.appium.java_client.android.AndroidElement; -import org.openqa.selenium.remote.DesiredCapabilities; -import org.openqa.selenium.support.ui.ExpectedConditions; -import org.openqa.selenium.support.ui.WebDriverWait; -import org.testng.annotations.Test; - -import java.net.URL; -import java.util.List; - -public class AndroidApp { - -String userName = System.getenv("LT_USERNAME") == null ? -"username" : System.getenv("LT_USERNAME"); //Add username here -String accessKey = System.getenv("LT_ACCESS_KEY") == null ? -"accessKey" : System.getenv("LT_ACCESS_KEY"); //Add accessKey here - -public String gridURL = "@mobile-hub.lambdatest.com/wd/hub"; - -AppiumDriver driver; - -@Test -@org.testng.annotations.Parameters(value = {"device", "version", "platform"}) -public void AndroidApp1(String device, String version, String platform) { -try { -DesiredCapabilities capabilities = new DesiredCapabilities(); -capabilities.setCapability("build","Java TestNG Android"); -capabilities.setCapability("name",platform+" "+device+" "+version); -capabilities.setCapability("deviceName", device); -capabilities.setCapability("platformVersion",version); -capabilities.setCapability("platformName", platform); -capabilities.setCapability("isRealMobile", true); -// highlight-next-line -capabilities.setCapability("app", "APP_URL"); //Enter your app (.apk) url -capabilities.setCapability("deviceOrientation", "PORTRAIT"); -capabilities.setCapability("console", true); -capabilities.setCapability("network", false); -capabilities.setCapability("visual", true); -capabilities.setCapability("devicelog", true); - -String hub = "https://" + userName + ":" + accessKey + gridURL; -driver = new AppiumDriver(new URL(hub), capabilities); - -MobileElement color = (MobileElement) driver.findElementById("com.lambdatest.proverbial:id/color"); -//Changes color to pink -color.click(); -Thread.sleep(1000); -//Back to original color -color.click(); - -MobileElement text = (MobileElement) driver.findElementById("com.lambdatest.proverbial:id/Text"); -//Changes the text to "Proverbial" -text.click(); - -//toast will be visible -MobileElement toast = (MobileElement) driver.findElementById("com.lambdatest.proverbial:id/toast"); -toast.click(); - -//notification will be visible -MobileElement notification = (MobileElement) driver.findElementById("com.lambdatest.proverbial:id/notification"); -notification.click(); -Thread.sleep(2000); - -//Opens the geolocation page -MobileElement geo = (MobileElement) driver.findElementById("com.lambdatest.proverbial:id/geoLocation"); -geo.click(); -Thread.sleep(5000); - -//takes back to home page -MobileElement home = (MobileElement) driver.findElementByAccessibilityId("Home"); -home.click(); - -//Takes to speed test page -MobileElement speedtest = (MobileElement) driver.findElementById("com.lambdatest.proverbial:id/speedTest"); -speedtest.click(); -Thread.sleep(5000); - -MobileElement Home = (MobileElement) driver.findElementByAccessibilityId("Home"); -Home.click(); - -//Opens the browser -MobileElement browser = (MobileElement) driver.findElementByAccessibilityId("Browser"); -browser.click(); - -MobileElement url = (MobileElement) driver.findElementById("com.lambdatest.proverbial:id/url"); -url.sendKeys("https://www.testmuai.com"); - -MobileElement find = (MobileElement) driver.findElementById("com.lambdatest.proverbial:id/find"); -find.click(); - -driver.quit(); - -} catch (Exception e) { -e.printStackTrace(); -try{ -driver.quit(); -}catch(Exception e1){ -e.printStackTrace(); -} -} -} -} -``` - -```java title="iOSApp.java" -import io.appium.java_client.AppiumDriver; -import io.appium.java_client.MobileBy; -import io.appium.java_client.MobileElement; -import io.appium.java_client.android.AndroidElement; -import org.openqa.selenium.remote.DesiredCapabilities; -import org.openqa.selenium.support.ui.ExpectedConditions; -import org.openqa.selenium.support.ui.WebDriverWait; -import org.openqa.selenium.JavascriptExecutor; -import org.openqa.selenium.remote.DesiredCapabilities; -import java.net.URL; -import org.testng.annotations.Test; - -public class iOSApp { - -String userName = System.getenv("LT_USERNAME") == null ? -"username" : System.getenv("LT_USERNAME"); //Add username here -String accessKey = System.getenv("LT_ACCESS_KEY") == null ? -"accessKey" : System.getenv("LT_ACCESS_KEY"); //Add accessKey here - -public String gridURL = "@mobile-hub.lambdatest.com/wd/hub"; - -AppiumDriver driver; - -@Test -@org.testng.annotations.Parameters(value = {"device", "version", "platform"}) -public void iOSApp1(String device, String version, String platform) { - -try { -DesiredCapabilities capabilities = new DesiredCapabilities(); -capabilities.setCapability("build","Java TestNG iOS"); -capabilities.setCapability("name",platform+" "+device+" "+version); -capabilities.setCapability("deviceName", device); -capabilities.setCapability("platformVersion",version); -capabilities.setCapability("platformName", platform); -capabilities.setCapability("isRealMobile", true); -// highlight-next-line -capabilities.setCapability("app", "APP_URL"); //Enter your app (.ipa) url -capabilities.setCapability("deviceOrientation", "PORTRAIT"); -capabilities.setCapability("console", true); -capabilities.setCapability("network", false); -capabilities.setCapability("visual", true); -capabilities.setCapability("devicelog", true); -//capabilities.setCapability("geoLocation", "HK"); - -String hub = "https://" + userName + ":" + accessKey + gridURL; -driver = new AppiumDriver(new URL(hub), capabilities); - -WebDriverWait Wait = new WebDriverWait(driver,30); - -//Changes the color of the text -Wait.until(ExpectedConditions.presenceOfElementLocated(MobileBy.AccessibilityId("color"))).click(); -Thread.sleep(1000); - -//Changes the text to "Proverbial" -Wait.until(ExpectedConditions.presenceOfElementLocated(MobileBy.AccessibilityId("Text"))).click(); -Thread.sleep(1000); - -//Toast will be visible -Wait.until(ExpectedConditions.presenceOfElementLocated(MobileBy.AccessibilityId("toast"))).click(); -Thread.sleep(1000); - -//Notification will be visible -Wait.until(ExpectedConditions.presenceOfElementLocated(MobileBy.AccessibilityId("notification"))).click(); -Thread.sleep(4000); - -//Opens the geolocation page -Wait.until(ExpectedConditions.presenceOfElementLocated(MobileBy.AccessibilityId("geoLocation"))).click(); -Thread.sleep(4000); - -//Takes back -driver.navigate().back(); - -//Takes to speedtest page -Wait.until(ExpectedConditions.presenceOfElementLocated(MobileBy.AccessibilityId("speedTest"))).click(); -Thread.sleep(4000); - -driver.navigate().back(); - -//Opens the browser -Wait.until(ExpectedConditions.presenceOfElementLocated(MobileBy.AccessibilityId("Browser"))).click(); -Thread.sleep(1000); - -MobileElement url = (MobileElement) driver.findElementByAccessibilityId("url"); -url.click(); -url.sendKeys("https://www.testmuai.com"); - -Wait.until(ExpectedConditions.presenceOfElementLocated(MobileBy.AccessibilityId("find"))).click(); -Thread.sleep(1000); - -driver.quit(); - -} catch (Exception e) { -e.printStackTrace(); -try{ -driver.quit(); -}catch(Exception e1){ -e.printStackTrace(); -} -} - -} -} -``` - -- You must set **isRealMobile** capability to `False` in the config file to run on **Virtual Devices** - -### Step 5: Configure the Test Capabilities - -You can update your custom capabilities in test scripts. In this sample project, we are passing platform name, platform version, device name and app url _(generated earlier)_ along with other capabilities like build name and test name via capabilities object. - -The capabilities object in the sample code are defined as: - -```java -DesiredCapabilities capabilities = new DesiredCapabilities(); -capabilities.setCapability("build","Java TestNG Android"); -capabilities.setCapability("name",platform+" "+device+" "+version); -capabilities.setCapability("deviceName", device); -capabilities.setCapability("platformVersion",version); -capabilities.setCapability("platformName", platform); -capabilities.setCapability("isRealMobile", true); -// highlight-next-line -capabilities.setCapability("app", "APP_URL"); //Enter your app (.apk) url -capabilities.setCapability("deviceOrientation", "PORTRAIT"); -capabilities.setCapability("console", true); -capabilities.setCapability("network", false); -capabilities.setCapability("visual", true); -capabilities.setCapability("devicelog", true); -``` - -```java -DesiredCapabilities capabilities = new DesiredCapabilities(); -capabilities.setCapability("build","Java TestNG iOS"); -capabilities.setCapability("name",platform+" "+device+" "+version); -capabilities.setCapability("deviceName", device); -capabilities.setCapability("platformVersion",version); -capabilities.setCapability("platformName", platform); -capabilities.setCapability("isRealMobile", true); -// highlight-next-line -capabilities.setCapability("app", "APP_URL"); //Enter your app (.ipa) url -capabilities.setCapability("deviceOrientation", "PORTRAIT"); -capabilities.setCapability("console", true); -capabilities.setCapability("network", false); -capabilities.setCapability("visual", true); -capabilities.setCapability("devicelog", true); -``` - -- You must set **isRealMobile** capability to `False` in the config file to run on **Virtual Devices** - -- You must add the generated **APP_URL** to the `app` capability in the config file. -- You must set **isRealMobile** capability to `False` in the config file to run on **Virtual Devices** -- You can generate capabilities for your test requirements with the help of our inbuilt [**Capabilities Generator tool**](https://www.testmuai.com/capabilities-generator/).For more details, please refer to our guide on [**Desired Capabilities in Appium**](/support/docs/desired-capabilities-in-appium/). - -### Step 6: Execute and Monitor your Tests - -- Run the following commands to install the required dependencies: - -```bash -mvn clean install -``` - -- The tests can be executed in the terminal using the following command: - -```bash -mvn test -P android-single -``` - -```bash -mvn test -P ios-single -``` - - > Your test results would be displayed on the test console (or CLI if you are using terminal/cmd) and on the [TestMu AI App Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://appautomation.lambdatest.com/build). - -## Using the TestNG Agent Skill with TestMu AI - -The [testng-skill](https://github.com/LambdaTest/agent-skills/tree/main/testng-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The testng-skill package includes: - -``` -testng-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing TestNG Agent Skill - -Install a TestNG Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/testng-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/testng-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only testng-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). - -## Reference Guides - -- [Advanced Configuration for Capabilities](/support/docs/desired-capabilities-in-appium/) -- [How to test locally hosted apps](/support/docs/testing-locally-hosted-pages/) -- [How to integrate TestMu AI with CI/CD](/support/docs/integrations-with-ci-cd-tools/) diff --git a/static/docs/appium-java.md b/static/docs/appium-java.md deleted file mode 100644 index f79925105..000000000 --- a/static/docs/appium-java.md +++ /dev/null @@ -1,352 +0,0 @@ -# Appium with Java - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -In this documentation, you will learn how to trigger a automation script of **Java** for application testing with **Appium** on TestMu AI, set the [**desired capabilities**](/support/docs/desired-capabilities-in-appium/) for appium testing, and other advanced features of TestMu AI. - -## Prerequisites - -- Your TestMu AI [Username and Access key](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/security). -- You should have [Java client library](https://github.com/appium/java-client) installed for Selenium and Appium. -- Download and install **Maven** from [the official website](https://maven.apache.org/). For **Linux/macOS** you can use [Homebrew](https://brew.sh/) package manager. - -## Try our Sample Repository - -### Step 1: Get a Sample Project -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** -All the code samples in this documentation can be found on **TestMu AI's Github Repository**. You can either download or clone the repository to quickly run your tests. View on GitHub - -### Step 2: Setup the Environment Variables - -You need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in your [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/security). Run the below mentioned commands in your terminal to setup the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -### Step 3: Upload your Application -Upload your **_iOS_** application (.ipa file) or **_android_** application (.apk or .aab file) to the TestMu AI servers using our **REST API**. You need to provide your **Username** and **AccessKey** in the format `Username:AccessKey` in the **cURL** command for authentication. - -Make sure to add the path of the **appFile** in the cURL request. Below is an example cURL request to upload your app using our REST API: - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "appFile=@"/Users/macuser/Downloads/proverbial_android.apk"" -F "name="proverbial_app""`} - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "url=:https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk" -F "name=Proverbial_App"`} - -- If you do not have any **.apk** or **.ipa** file, you can run your sample tests on TestMu AI by using our sample apps, :link: [Android app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk) or :link: [iOS app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_ios.ipa). - -- Response of above cURL will be a **JSON** object containing the `APP_URL` of the format - `lt://APP123456789123456789` and will be used in the next step - -### Step 4: Update your Automation Script - -An automation script for the sample application given above has been provided here. Ensure to update the `APP_URL`, `username` and `accessKey` in the code scripts before running the tests. - -```java title="vanilla_android.java" - -import io.appium.java_client.AppiumDriver; -import io.appium.java_client.MobileBy; -import io.appium.java_client.MobileElement; -import org.openqa.selenium.JavascriptExecutor; -import org.openqa.selenium.remote.DesiredCapabilities; -import org.openqa.selenium.remote.RemoteWebDriver; -import java.net.MalformedURLException; -import java.net.URL; - -public class vanilla_android { -//highlight-next-line -public static String userName = System.getenv("LT_USERNAME") == null ? "LT_USERNAME" //Add LambdaTest username here -: System.getenv("LT_USERNAME"); -//highlight-next-line -public static String accessKey = System.getenv("LT_ACCESS_KEY") == null ? "LT_ACCESS_KEY" //Add LambdaTest accessKey here -: System.getenv("LT_ACCESS_KEY"); - -private static AppiumDriver driver; - -public static void main(String args[]) throws MalformedURLException, InterruptedException { - -try { -DesiredCapabilities capabilities = new DesiredCapabilities(); -capabilities.setCapability("deviceName", "Galaxy S20"); -capabilities.setCapability("platformVersion", "11"); -capabilities.setCapability("platformName", "Android"); -capabilities.setCapability("isRealMobile", true); -//highlight-next-line -capabilities.setCapability("app", "APP_URL"); //Enter your app url -capabilities.setCapability("deviceOrientation", "PORTRAIT"); -capabilities.setCapability("build", "Java Vanilla - Android"); -capabilities.setCapability("name", "Sample Test Java"); -capabilities.setCapability("console", true); -capabilities.setCapability("network", false); -capabilities.setCapability("visual", true); -capabilities.setCapability("devicelog", true); - -driver = new AppiumDriver(new URL("https://" +userName + ":" + accessKey + "@mobile-hub.lambdatest.com/wd/hub"), capabilities); - -MobileElement color = (MobileElement) driver.findElement(MobileBy.id("com.lambdatest.proverbial:id/color")); -color.click(); - -MobileElement text = (MobileElement) driver.findElement(MobileBy.id("com.lambdatest.proverbial:id/Text")); -//Changes the text to proverbial -text.click(); - -//toast is visible -MobileElement toast = (MobileElement) driver.findElement(MobileBy.id("com.lambdatest.proverbial:id/toast")); -toast.click(); - -//notification is visible -MobileElement notification = (MobileElement) driver.findElement(MobileBy.id("com.lambdatest.proverbial:id/notification")); -notification.click(); - -//Open the geolocation page -MobileElement geo = (MobileElement) driver.findElement(MobileBy.id("com.lambdatest.proverbial:id/geoLocation")); -geo.click(); -Thread.sleep(5000); - -//takes back to home page -MobileElement el3 = (MobileElement) driver.findElementByAccessibilityId("Home"); - -driver.navigate().back(); -Thread.sleep(2000); - -//Takes to speed test page -MobileElement speedtest = (MobileElement) driver.findElement(MobileBy.id("com.lambdatest.proverbial:id/speedTest")); -speedtest.click(); -Thread.sleep(5000); - -driver.navigate().back(); - -//Opens the browser -MobileElement browser = (MobileElement) driver.findElement(MobileBy.AccessibilityId("Browser")); -browser.click(); - -MobileElement url = (MobileElement) driver.findElement(MobileBy.id("com.lambdatest.proverbial:id/url")); -url.sendKeys("https://www.testmuai.com"); -MobileElement find = (MobileElement) driver.findElement(MobileBy.id("com.lambdatest.proverbial:id/find")); -find.click(); - -} catch (AssertionError a) { -((JavascriptExecutor) driver).executeScript("lambda-status=failed"); -a.printStackTrace(); -} -// The driver.quit statement is required, otherwise the test continues to execute, leading to a timeout. -driver.quit(); -} -} -``` - -```java title="vanilla_ios.java" -import io.appium.java_client.AppiumDriver; -import io.appium.java_client.MobileBy; -import io.appium.java_client.MobileElement; -import io.appium.java_client.ios.IOSDriver; - -import org.openqa.selenium.remote.DesiredCapabilities; -import org.openqa.selenium.support.ui.ExpectedConditions; -import org.openqa.selenium.support.ui.WebDriverWait; - -import java.net.URL; - -public class vanilla_ios { -//highlight-next-line -public static String userName = System.getenv("LT_USERNAME") == null ? "LT_USERNAME" //Add LambdaTest username here -: System.getenv("LT_USERNAME"); -//highlight-next-line -public static String accessKey = System.getenv("LT_ACCESS_KEY") == null ? "LT_ACCESS_KEY" //Add LambdaTest accessKey here -: System.getenv("LT_ACCESS_KEY"); - -public static final String URL = "https://" + userName + ":" + accessKey + "@mobile-hub.lambdatest.com/wd/hub"; -public static IOSDriver driver = null; - -public static void main(String[] args) throws Exception { - -try { -DesiredCapabilities caps = new DesiredCapabilities(); -caps.setCapability("platformVersion", "15"); -caps.setCapability("deviceName", "iPhone 12"); -caps.setCapability("isRealMobile", true); -//highlight-next-line -caps.setCapability("app", "APP_URL"); //Enter your app url -caps.setCapability("platformName", "iOS"); -caps.setCapability("build", "Java Vanilla - iOS"); -caps.setCapability("name", "Sample Test Java"); -caps.setCapability("devicelog", true); -caps.setCapability("network", false); - -driver = new IOSDriver(new URL("https://" + userName + ":" + accessKey + "@beta-hub.lambdatest.com/wd/hub"), caps); - -Thread.sleep(2000); - -//Changes color - -driver.findElement(MobileBy.id("color")).click(); -Thread.sleep(1000); - -//Back to black color -driver.navigate().back(); - -Thread.sleep(1000); - -//Changes the text to proverbial -driver.findElement(MobileBy.id("Text")).click(); -Thread.sleep(1000); - -//toast is visible -driver.findElement(MobileBy.id("toast")).click(); -Thread.sleep(1000); - -//notification is visible -driver.findElement(MobileBy.id("notification")).click(); -Thread.sleep(2000); - -//Open the geolocation page -driver.findElement(MobileBy.id("geoLocation")).click(); -Thread.sleep(4000); -driver.navigate().back(); -Thread.sleep(1000); - -//Takes to speed test page -driver.findElement(MobileBy.id("speedTest")).click(); -Thread.sleep(5000); -driver.navigate().back(); -Thread.sleep(1000); - -//Opens the browser -MobileElement browser = (MobileElement) driver.findElementByAccessibilityId("Browser"); -browser.click(); -Thread.sleep(3000); - -WebDriverWait el7 = new WebDriverWait(driver, 30); -el7.until(ExpectedConditions.elementToBeClickable(MobileBy.id("url"))); -driver.findElementById("url").sendKeys("https://www.testmuai.com/"); - -//Clicks on the text box -WebDriverWait el = new WebDriverWait(driver,90); -MobileElement el4 = (MobileElement) driver.findElementByAccessibilityId("find"); -el.until(ExpectedConditions.elementToBeClickable(el4)); -el4.click(); -el4.sendKeys("Lambdatest"); - -//((JavascriptExecutor) driver).executeScript("lambda-status=passed"); -driver.quit(); - -} catch (Exception t) { -System.out.println(t); -driver.quit(); - -} -} -} -``` - -- You must set **isRealMobile** capability to `False` in the config file to run on **Virtual Devices** - -### Step 5: Configure the Test Capabilities - -You can update your custom capabilities in test scripts. In this sample project, we are passing platform name, platform version, device name and app url _(generated earlier)_ along with other capabilities like build name and test name via capabilities object. - -The capabilities object in the sample code are defined as: - -```java -DesiredCapabilities capabilities = new DesiredCapabilities(); -capabilities.setCapability("deviceName", "Galaxy S20"); -capabilities.setCapability("platformVersion", "11"); -capabilities.setCapability("platformName", "Android"); -capabilities.setCapability("isRealMobile", true); -//highlight-next-line -capabilities.setCapability("app", "YOUR_APP_URL"); //Enter your app url -capabilities.setCapability("deviceOrientation", "PORTRAIT"); -capabilities.setCapability("build", "Java Vanilla - Android"); -capabilities.setCapability("name", "Sample Test Java"); -capabilities.setCapability("console", true); -capabilities.setCapability("network", false); -capabilities.setCapability("visual", true); -capabilities.setCapability("devicelog", true); -``` - -- You must set **isRealMobile** capability to `False` in the config file to run on **Virtual Devices** - -```java -DesiredCapabilities caps = new DesiredCapabilities(); -caps.setCapability("platformVersion", "15"); -caps.setCapability("deviceName", "iPhone 12"); -caps.setCapability("isRealMobile", true); -//highlight-next-line -caps.setCapability("app", "YOUR_APP_URL"); //Enter your app url -caps.setCapability("platformName", "iOS"); -caps.setCapability("build", "Java Vanilla - iOS"); -caps.setCapability("name", "Sample Test Java"); -caps.setCapability("devicelog", true); -caps.setCapability("network", false); -``` - -- You must add the generated **APP_URL** to the `app` capability in the config file. -- You must set **isRealMobile** capability to `false` in the config file to run on **Virtual Devices** -- You can generate capabilities for your test requirements with the help of our inbuilt [**Capabilities Generator tool**](https://www.testmuai.com/capabilities-generator/).For more details, please refer to our guide on [**Desired Capabilities in Appium**](/support/docs/desired-capabilities-in-appium/). - -### Step 6: Execute and Monitor your Tests - -- Run the following commands to install the required dependencies: - -```bash -mvn clean install -``` - -- The tests can be executed in the terminal using the following command: - -```bash -mvn test -P android -``` - -```bash -mvn test -P ios -``` - - > Your test results would be displayed on the test console (or CLI if you are using terminal/cmd) and on the [TestMu AI App Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://appautomation.lambdatest.com/build). - -## Using the Appium Agent Skill with TestMu AI - -The [appium-skill](https://github.com/LambdaTest/agent-skills/tree/main/appium-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The appium-skill package includes: - -``` -appium-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing Appium Agent Skill - -Install a Appium Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/appium-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/appium-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only appium-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). - -## Reference Guides - -- [Advanced Configuration for Capabilities](/support/docs/desired-capabilities-in-appium/) -- [How to test locally hosted apps](/support/docs/testing-locally-hosted-pages/) -- [How to integrate TestMu AI with CI/CD](/support/docs/integrations-with-ci-cd-tools/) diff --git a/static/docs/appium-kotlin.md b/static/docs/appium-kotlin.md deleted file mode 100644 index f6074f45f..000000000 --- a/static/docs/appium-kotlin.md +++ /dev/null @@ -1,314 +0,0 @@ -# Appium With Kotlin - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -In this documentation, you will learn how to configure and run your **Kotlin** automation testing scripts with **Appium** on TestMu AI, set the desired capabilities for appium testing, and other advanced features of TestMu AI. - -## Prerequisites - -- Your TestMu AI [Username and Access key](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/security). -- Install the latest version of [**IntelliJ IDEA**](https://www.jetbrains.com/idea/). You can use any other IDE also. -- Download and install [**Maven**](https://maven.apache.org/). You can use [**Homebrew**](https://brew.sh/) package manager for Linux/macOS. -- Make sure that Kotlin plugins are enabled/installed in the IntelliJ IDEA if you are using a previous version. - -## Try our Sample Repository - -### Step 1: Get a Sample Project -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** -All the code samples in this documentation can be found on **TestMu AI's Github Repository**. You can either download or clone the repository to quickly run your tests. View on GitHub - -### Step 2: Setup the Environment Variables - -You need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in your [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/security). Run the below mentioned commands in your terminal to setup the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -### Step 3: Upload your Application -Upload your **_iOS_** application (.ipa file) or **_android_** application (.apk or .aab file) to the TestMu AI servers using our **REST API**. You need to provide your **Username** and **AccessKey** in the format `Username:AccessKey` in the **cURL** command for authentication. - -Make sure to add the path of the **appFile** in the cURL request. Below is an example cURL request to upload your app using our REST API: - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "appFile=@"/Users/macuser/Downloads/proverbial_android.apk"" -F "name="proverbial_app""`} - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "url=:https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk" -F "name=Proverbial_App"`} - -- If you do not have any **.apk** or **.ipa** file, you can run your sample tests on TestMu AI by using our sample apps, :link: [Android app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk) or :link: [iOS app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_ios.ipa). - -- Response of above cURL will be a **JSON** object containing the `APP_URL` of the format - `lt://APP123456789123456789` and will be used in the next step. - -### Step 4: Update your Automation Script - -An automation script for the sample application available above has been provided here. Ensure to update the `APP_URL`, `username` and `accesKey` in the code scripts before running the tests. - -```java title="android.kt" -import io.appium.java_client.MobileBy -import io.appium.java_client.MobileElement -import io.appium.java_client.android.AndroidDriver -import io.appium.java_client.android.AndroidElement -import org.junit.jupiter.api.Test -import org.openqa.selenium.remote.DesiredCapabilities -import java.net.URL - -class android { - -@Test -fun `sample test`() { -//highlight-next-line -val username = "LT_USERNAME" //Add your LambdaTest username here -//highlight-next-line -val accessKey = "LT_ACCESS_KEY" //Add your LambdaTest accessKey here - -var driver: AndroidDriver? = null - -val hub = URL("https://${username}:${accessKey}@mobile-hub.lambdatest.com/wd/hub") - -val caps = DesiredCapabilities() -caps.setCapability("autoAcceptAlerts", true) -caps.setCapability("platformName", "Android") -caps.setCapability("deviceName", "Galaxy S20") -caps.setCapability("platformVersion", "11") -caps.setCapability("platformName", "Android") -caps.setCapability("isRealMobile", true) -//highlight-next-line -caps.setCapability("app", "APP_URL") //Add the app (.apk) url here -caps.setCapability("deviceOrientation", "PORTRAIT") -caps.setCapability("build", "Kotlin Vanilla - Android") -caps.setCapability("name", "Sample Test Kotlin") -caps.setCapability("console", true) -caps.setCapability("network", false) -caps.setCapability("visual", true) -caps.setCapability("device log", true) - -val app = AndroidDriver(hub, caps) - -Thread.sleep(4000) - -val color: MobileElement = -app.findElement(MobileBy.id("com.lambdatest.proverbial:id/color")) as MobileElement -color.click() - -val text: MobileElement = -app.findElement(MobileBy.id("com.lambdatest.proverbial:id/Text")) as MobileElement -text.click() - -val toast: MobileElement = -app.findElement(MobileBy.id("com.lambdatest.proverbial:id/toast")) as MobileElement -toast.click() - -val notification: MobileElement = -app.findElement(MobileBy.id("com.lambdatest.proverbial:id/notification")) as MobileElement -notification.click() - -val geo: MobileElement = -app.findElement(MobileBy.id("com.lambdatest.proverbial:id/geoLocation")) as MobileElement -geo.click() -Thread.sleep(5000) - -val el3: MobileElement = -app.findElementByAccessibilityId("Home") as MobileElement - -app.navigate().back() -Thread.sleep(2000) - -val speedtest: MobileElement = -app.findElement(MobileBy.id("com.lambdatest.proverbial:id/speedTest")) as MobileElement -speedtest.click() -Thread.sleep(5000) - -app.navigate().back() - -if (app != null) { -app.quit() -}; - -} -} -``` - -```java title="iOS.kt" -import io.appium.java_client.MobileBy -import io.appium.java_client.MobileElement -import io.appium.java_client.remote.AppiumCommandExecutor -import io.appium.java_client.android.AndroidDriver -import io.appium.java_client.android.AndroidElement -import io.appium.java_client.ios.IOSDriver -import io.appium.java_client.ios.IOSElement -import org.junit.jupiter.api.Test -import org.openqa.selenium.remote.DesiredCapabilities -import java.net.URL - -class iOS { - -@Test -fun `sample test`() { -//highlight-next-line -val username = "LT_USERNAME" //Add your LambdaTest username here -//highlight-next-line -val accessKey = "LT_ACCESS_KEY" //Add your LambdaTest accessKey here - -var driver: IOSDriver? = null - -val hub = URL("https://${username}:${accessKey}@mobile-hub.lambdatest.com/wd/hub") - -val caps = DesiredCapabilities() -caps.setCapability("platformName", "ios") -caps.setCapability("deviceName", "iPhone 12") -caps.setCapability("platformVersion", "14") -caps.setCapability("isRealMobile", true) -//highlight-next-line -caps.setCapability("app", "APP_URL") //Add the app (.ipa) url here -caps.setCapability("deviceOrientation", "PORTRAIT") -caps.setCapability("build", "Kotlin Vanilla - iOS") -caps.setCapability("name", "Sample Test Kotlin") -caps.setCapability("console", true) -caps.setCapability("network", false) -caps.setCapability("visual", true) -caps.setCapability("device log", true) - -val app = IOSDriver(hub, caps) - -Thread.sleep(4000) - -val color: MobileElement = -app.findElementById("color") -color.click() - -val text: MobileElement = -app.findElementById("Text") -text.click() - -val toast: MobileElement = -app.findElementById("toast") -toast.click() - -val notification: MobileElement = -app.findElementById("notification") -notification.click() - -val geo: MobileElement = -app.findElementById("geoLocation") -geo.click() - -Thread.sleep(5000) - -app.navigate().back() - -if (app != null) { -app.quit() -}; - -} -} -``` - -- You must set **isRealMobile** capability to `False` in the config file to run on **Virtual Devices** - -- You must set **isRealMobile** capability to `False` in the config file to run on **Virtual Devices** - -### Step 5: Configure the Test Capabilities -You can update your custom capabilities in test scripts. In this sample project, we are passing platform name, platform version, device name and app url _(generated earlier)_ along with other capabilities like build name and test name via capabilities object. - -The capabilities object in the sample code are defined as: - -```java -val caps = DesiredCapabilities() -caps.setCapability("platformName", "ios") -caps.setCapability("deviceName", "iPhone 12") -caps.setCapability("platformVersion", "14") -caps.setCapability("isRealMobile", true) -//highlight-next-line -caps.setCapability("app", "APP_URL") //Add the app (.ipa) url here -caps.setCapability("deviceOrientation", "PORTRAIT") -caps.setCapability("build", "Kotlin Vanilla - iOS") -caps.setCapability("name", "Sample Test Kotlin") -caps.setCapability("console", true) -caps.setCapability("network", false) -caps.setCapability("visual", true) -caps.setCapability("device log", true) -``` - -```java -val caps = DesiredCapabilities() -caps.setCapability("autoAcceptAlerts", true) -caps.setCapability("platformName", "Android") -caps.setCapability("deviceName", "Galaxy S20") -caps.setCapability("platformVersion", "11") -caps.setCapability("platformName", "Android") -caps.setCapability("isRealMobile", true) -//highlight-next-line -caps.setCapability("app", "APP_URL") //Add the app (.apk) url here -caps.setCapability("deviceOrientation", "PORTRAIT") -caps.setCapability("build", "Kotlin Vanilla - Android") -caps.setCapability("name", "Sample Test Kotlin") -caps.setCapability("console", true) -caps.setCapability("network", false) -caps.setCapability("visual", true) -caps.setCapability("device log", true) -``` - -- You must set **isRealMobile** capability to `False` in the config file to run on **Virtual Devices** - -- You must add the generated **APP_URL** to the `app` capability in the config file. -- You can generate capabilities for your test requirements with the help of our inbuilt [**Capabilities Generator tool**](https://www.testmuai.com/capabilities-generator/).For more details, please refer to our guide on [**Desired Capabilities in Appium**](/support/docs/desired-capabilities-in-appium/). - -### Step 6: Execute and Monitor your Tests - -- Install the required dependencies: - -```bash -mvn clean install -``` -2. Now, you have to successfully **build** the test in your IDE. -3. Then run the current file by using **Run** button in the IDE. - - > Your test results would be displayed on the test console (or CLI if you are using terminal/cmd) and on the [TestMu AI App Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://appautomation.lambdatest.com/build). - -## Using the Appium Agent Skill with TestMu AI - -The [appium-skill](https://github.com/LambdaTest/agent-skills/tree/main/appium-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The appium-skill package includes: - -``` -appium-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing Appium Agent Skill - -Install a Appium Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/appium-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/appium-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only appium-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). - -## Reference Guides - -- [Advanced Configuration for Capabilities](/support/docs/desired-capabilities-in-appium/) -- [How to test locally hosted apps](/support/docs/testing-locally-hosted-pages/) -- [How to integrate TestMu AI with CI/CD](/support/docs/integrations-with-ci-cd-tools/) diff --git a/static/docs/appium-languages-and-frameworks.md b/static/docs/appium-languages-and-frameworks.md deleted file mode 100644 index cf238c8cb..000000000 --- a/static/docs/appium-languages-and-frameworks.md +++ /dev/null @@ -1,89 +0,0 @@ -# Appium Supported Languages and Frameworks - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Here is a list of languages and frameworks that are supported by the TestMu AI to run Appium automation tests on [TestMu AI Real Device Cloud Platform](https://www.testmuai.com/real-device-cloud). - - Java - - JUnit - - JBehave - - Cucumber - - TestNG - - JavaScript - - WebDriverIO - - Mocha - - Python - - Behave - - Robot - - Gauge - - PyTest - - Ruby - - Cucumber - - RSpec - - PHP - - Behat - - C# - - NUnit - - We support all languages and frameworks that are compatible with Appium, so in case your favorite isn't in the table.Don't worry, you can still run the test. Contact Us for any help. - -**note** - -We are preparing documentation for more frameworks. If you want us to prioritize documentation of your preferred framework then feel free to give us a **shout**. - -## Using the Appium Agent Skill with TestMu AI - -The [appium-skill](https://github.com/LambdaTest/agent-skills/tree/main/appium-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The appium-skill package includes: - -``` -appium-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing Appium Agent Skill - -Install a Appium Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/appium-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/appium-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only appium-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). diff --git a/static/docs/appium-nodejs-mocha.md b/static/docs/appium-nodejs-mocha.md deleted file mode 100644 index 108a354c3..000000000 --- a/static/docs/appium-nodejs-mocha.md +++ /dev/null @@ -1,270 +0,0 @@ -# Appium With Mocha - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -In this documentation, you will learn how to configure and run your **Mocha** automation testing scripts with **Appium** on TestMu AI, set the desired capabilities for appium testing, and other advanced features of TestMu AI. - -## Prerequisites - -- Your TestMu AI [Username and Access key](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/security). -- Install **npm** from the official [npm website](https://www.npmjs.com/). -- Download and install **NodeJS** from official [NodeJS website](https://nodejs.org/en/). You should be having **NodeJS v6** or newer. -- Make sure you are using the latest version of **JavaScript**. - -## Try our Sample Repository - -### Step 1: Get a Sample Project -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** -All the code samples in this documentation can be found on **TestMu AI's Github Repository**. You can either download or clone the repository to quickly run your tests. View on GitHub - -### Step 2: Setup the Environment Variables - -You need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in your [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/security). Run the below mentioned commands in your terminal to setup the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -### Step 3: Upload your Application -Upload your **_iOS_** application (.ipa file) or **_android_** application (.apk or .aab file) to the TestMu AI servers using our **REST API**. You need to provide your **Username** and **AccessKey** in the format `Username:AccessKey` in the **cURL** command for authentication. - -Make sure to add the path of the **appFile** in the cURL request. Below is an example cURL request to upload your app using our REST API: - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "appFile=@"/Users/macuser/Downloads/proverbial_android.apk"" -F "name="proverbial_app""`} - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "url=:https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk" -F "name=Proverbial_App"`} - -- If you do not have any **.apk** or **.ipa** file, you can run your sample tests on TestMu AI by using our sample apps, :link: [Android app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk) or :link: [iOS app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_ios.ipa). - -- Response of above cURL will be a **JSON** object containing the `APP_URL` of the format - `lt://APP123456789123456789` and will be used in the next step. - -### Step 4: Update your Automation Script - -An automation script for the sample application available above has been provided here. - -```javascript title="ios_test.js" -const driver= require("appium-base-driver") - -//const { default: driver } = require("appium-android-driver/build/lib/driver"); -const { By } = require("selenium-webdriver"); -const { element } = require("wd/lib/element-commands"); - -require("appium-base-driver") -var assert= require("assert"), -webdriver = require("selenium-webdriver"), -conf_file= process.argv[3] || "conf/ios.conf.js"; - -var caps = require("../" + conf_file).capabilities; - -var buildDriver = function(caps) { -return new webdriver.Builder() -.usingServer( -"http://" + -LT_USERNAME + -":" + -LT_ACCESS_KEY + -"@mobile-hub.lambdatest.com/wd/hub" -) -.withCapabilities(caps) -.build(); -}; - -describe("Mocha Appium iOS Test " + caps.browserName, function() { -var driver; -this.timeout(0); -it ('Application is launched', function name(done) { -driver=buildDriver(caps); -driver.findElement(By.xpath('//XCUIElementTypeButton[@name="color"]')).click().then(function(){ -console.log("Successfully clicked Color"); - -}); -driver.findElement(By.xpath('//XCUIElementTypeStaticText[@name="Notification"]')).click().then(function(){ -console.log("Successfully clicked Notification"); -}); -driver.findElement(By.xpath('//XCUIElementTypeStaticText[@name="Toast"]')).click().then(function(){ -console.log("Successfully clicked Toast"); -}); -driver.findElement(By.xpath('//XCUIElementTypeButton[@name="Text"]')).click().then(function(){ -console.log("Successfully clicked Text"); -driver.quit() -}); -}); -}); - -``` - -```javascript title="android_test.js" -const { default: driver } = require("appium-android-driver/build/lib/driver"); -const { By } = require("selenium-webdriver"); -const { capabilities } = require("../conf/android.conf"); - -require("appium-android-driver") -var assert= require("assert"), -webdriver = require("selenium-webdriver"), -conf_file= process.argv[3] || "conf/Appp.conf.js"; - -var caps = require("../" + conf_file).capabilities; - -var buildDriver = function(caps) { -return new webdriver.Builder() -.usingServer( -"http://" + -LT_USERNAME + -":" + -LT_ACCESS_KEY + -"@mobile-hub.lambdatest.com/wd/hub" -) -.withCapabilities(caps) -.build(); -}; - -describe("Mocha Appium Test " + caps.browserName, function() { -var driver; -this.timeout(0); - -it ('Application is launched', function name(done) { -driver=buildDriver(caps); - -driver.findElement(By.id('com.lambdatest.proverbial:id/color')).click().then(function(){ -console.log("Successfully clicked Color"); -}); -driver.findElement(By.id('com.lambdatest.proverbial:id/Text')).click().then(function(){ -console.log("Successfully clicked Text"); -}); -driver.findElement(By.id('com.lambdatest.proverbial:id/notification')).click().then(function(){ -console.log("Successfully clicked Notification"); -}) -driver.findElement(By.id('com.lambdatest.proverbial:id/toast')).click().then(function(){ -console.log("Successfully clicked Toast"); -driver.quit() -}) -}); - -}); -``` - -- You must set **isRealMobile** capability to `False` in the config file to run on **Virtual Devices** - -### Step 5: Configure the Test Capabilities - -You can update your custom capabilities in test scripts. In this sample project, we are passing platform name, platform version, device name and app url _(generated earlier)_ along with other capabilities like build name and test name via capabilities object. - -Ensure to update the `APP_URL`, `username` and `accessKey` in the code scripts before running the tests. The capabilities object in the sample code are defined as: - -```javascript title="android.conf.js" -LT_USERNAME = process.env.LT_USERNAME || ""; //Enter your LambdaTest username here -LT_ACCESS_KEY = process.env.LT_ACCESS_KEY || ""; //Enter your LambdaTest accessKey here - -exports.capabilities = { -'build': 'Mocha-Appium-Sample', //Build name -'name': 'Mocha-Android', // Test name -'platformName':'android', // OS name -'deviceName': 'Galaxy S10', // Device name -'platformVersion': '11', // OS version -'app' : 'lt://proverbial-android', // Add app (.apk) url here -'isRealMobile' : true, -'visual': false, // To take step by step screenshot -'network':false, // To capture network Logs -'console':false, // To capture console logs. -'tunnel': false // If you want to run the localhost than change it to true -}; -``` - -- You must set **isRealMobile** capability to `False` in the config file to run on **Virtual Devices** - -```javascript title="ios.conf.js" -LT_USERNAME = process.env.LT_USERNAME || ""; //Enter your LambdaTest username here -LT_ACCESS_KEY = process.env.LT_ACCESS_KEY || ""; //Enter your LambdaTest accessKey here - -exports.capabilities = { -'build': 'Mocha-iOS-Sample', //Build name -'name': 'Mocha-iOS', // Test name -'platformName':'ios', // OS name -'deviceName': 'iPhone 12 Pro', // Device name -'platformVersion': '15', // OS version -'app' : 'lt://proverbial-ios', // Add app (.ipa) url here -'isRealMobile' : true, -'visual': false, // To take step by step screenshot -'network':false, // To capture network Logs -'console':false, // To capture console logs. -'tunnel': false // If you want to run the localhost than change it to true -}; -``` - -- You must add the generated **APP_URL** to the `app` capability in the config file. -- You must set **isRealMobile** capability to `False` in the config file to run on **Virtual Devices** -- You can generate capabilities for your test requirements with the help of our inbuilt [**Capabilities Generator tool**](https://www.testmuai.com/capabilities-generator/).For more details, please refer to our guide on [**Desired Capabilities in Appium**](/support/docs/desired-capabilities-in-appium/). - -### Step 6: Execute and Monitor your Tests - -- Execute the following commands to install the required dependencies: - -```bash -npm i -npm install -npm install selenium-webdriver -npm i appium-android-driver -npm i appium-base-driver -npm install --save -``` -- The tests can be executed in the terminal using the following command: - -```js -npm run android //to run single test -npm run parallel_android //to run parallel tests -``` - -```js -npm run ios //to run single test -npm run parallel_ios //to run parallel tests -``` - - > Your test results would be displayed on the test console (or CLI if you are using terminal/cmd) and on the [TestMu AI App Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://appautomation.lambdatest.com/build). - -## Using the Mocha Agent Skill with TestMu AI - -The [mocha-skill](https://github.com/LambdaTest/agent-skills/tree/main/mocha-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The mocha-skill package includes: - -``` -mocha-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing Mocha Agent Skill - -Install a Mocha Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/mocha-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/mocha-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only mocha-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). - -## Reference Guides - -- [Advanced Configuration for Capabilities](/support/docs/desired-capabilities-in-appium/) -- [How to test locally hosted apps](/support/docs/testing-locally-hosted-pages/) -- [How to integrate TestMu AI with CI/CD](/support/docs/integrations-with-ci-cd-tools/) diff --git a/static/docs/appium-nodejs-webdriverio.md b/static/docs/appium-nodejs-webdriverio.md deleted file mode 100644 index e167f94c1..000000000 --- a/static/docs/appium-nodejs-webdriverio.md +++ /dev/null @@ -1,325 +0,0 @@ -# Appium With WebDriverIO - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -In this documentation, you will learn how to configure and run your **WebdriverIO** automation testing scripts with **Appium** on TestMu AI, set the desired capabilities for appium testing, and other advanced features of TestMu AI. - -## Prerequisites - -- Your TestMu AI [Username and Access key](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/security). -- Install **npm** from the official [npm website](https://www.npmjs.com/). -- Download and install **NodeJS** from official [NodeJS website](https://nodejs.org/en/). You should be having **NodeJS v6** or newer. -- Make sure you are using the latest version of **JavaScript**. - -## Try our Sample Repository - -### Step 1: Get a Sample Project -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** -All the code samples in this documentation can be found on **TestMu AI's Github Repository**. You can either download or clone the repository to quickly run your tests. View on GitHub - -### Step 2: Setup the Environment Variables - -You need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in your [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/security). Run the below mentioned commands in your terminal to setup the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -### Step 3: Upload your Application -Upload your **_iOS_** application (.ipa file) or **_android_** application (.apk or .aab file) to the TestMu AI servers using our **REST API**. You need to provide your **Username** and **AccessKey** in the format `Username:AccessKey` in the **cURL** command for authentication. - -Make sure to add the path of the **appFile** in the cURL request. Below is an example cURL request to upload your app using our REST API: - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "appFile=@"/Users/macuser/Downloads/proverbial_android.apk"" -F "name="proverbial_app""`} - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "url=:https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk" -F "name=Proverbial_App"`} - -- If you do not have any **.apk** or **.ipa** file, you can run your sample tests on TestMu AI by using our sample apps, :link: [Android app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk) or :link: [iOS app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_ios.ipa). - -- Response of above cURL will be a **JSON** object containing the `APP_URL` of the format - `lt://APP123456789123456789` and will be used in the next step. - -### Step 4: Update your Automation Script - -An automation script for the sample application available above has been provided here. Ensure to update the `APP_URL`, `username` and `accesKey` in your scripts inside `specs` directory the before running the tests. - -```javascript title="specs/ios-test.js" -describe("Proverbial APK", () => { -it("Changes color", async () => { -var color = await $("id=color"); -await color.waitForDisplayed({ timeout: 30000 }); -await color.click(); -await color.click(); -}); - -it("Changes text", async () => { -var text = await $("id=Text"); -await text.waitForDisplayed({ timeout: 30000 }); -await text.click(); -}); - -it("Toast", async () => { -var toast = await $("id=toast"); -await toast.waitForDisplayed({ timeout: 30000 }); -await toast.click(); -}); - -it("Notification", async () => { -var nf = await $("id=notification"); -await nf.waitForDisplayed({ timeout: 30000 }); -await nf.click(); -}); - -it("Geolocation", async () => { -var geo = await $("id=geoLocation"); -await geo.waitForDisplayed({ timeout: 30000 }); -await geo.click(); - -driver.back(); -}); - -it("SpeedTest", async () => { -var st = await $("id=speedTest"); -await st.waitForDisplayed({ timeout: 30000 }); -await st.click(); - -await browser.pause(10000); -driver.back(); -}); - -it("Browser", async () => { -var browser = await $("id=Browser"); -await browser.waitForDisplayed({ timeout: 30000 }); -await browser.click(); - -let el7 = await $("id=url"); -await el7.click(); -await el7.setValue("https://www.testmuai.com/"); -driver.back(); -}); -}); -``` - -```javascript title="specs/android-test.js" -describe("Proverbial APK", () => { -it("Changes color", async () => { -var color = await $("id=color"); -await color.waitForDisplayed({ timeout: 30000 }); -await color.click(); -await color.click(); -}); - -it("Changes text", async () => { -var text = await $("id=Text"); -await text.waitForDisplayed({ timeout: 30000 }); -await text.click(); -}); - -it("Toast", async () => { -var toast = await $("id=toast"); -await toast.waitForDisplayed({ timeout: 30000 }); -await toast.click(); -}); - -it("Notification", async () => { -var nf = await $("id=notification"); -await nf.waitForDisplayed({ timeout: 30000 }); -await nf.click(); -}); - -it("Geolocation", async () => { -var geo = await $("id=geoLocation"); -await geo.waitForDisplayed({ timeout: 30000 }); -await geo.click(); - -driver.back(); -}); - -it("SpeedTest", async () => { -var st = await $("id=speedTest"); -await st.waitForDisplayed({ timeout: 30000 }); -await st.click(); - -await browser.pause(10000); -driver.back(); -}); - -it("Browser", async () => { -var browser = await $("id=Browser"); -await browser.waitForDisplayed({ timeout: 30000 }); -await browser.click(); - -let el7 = await $("id=url"); -await el7.click(); -await el7.setValue("https://www.testmuai.com/"); -driver.back(); -}); -}); -``` - -### Step 5: Configure the Test Capabilities - -You need to update your capabilities in `*.conf.js` files. In this sample project, we have provided the examples for running tests on both **Android** and **iOS** apps. You can find the configs for both iOS and Android in the `ios-sample` and `android-sample` directories correspondingly. - -We are passing platform name, platform version, device name and app url (generated earlier) along with other capabilities like build name and test name via capabilities object. You need to pass the path of your test script in `specs` object to run your own automation script. The capabilities object in the sample code for a single test are defined as: - -```javascript title="ios-sample/ios-single.conf.js" -exports.config = { -//highlight-next-line -user: process.env.LT_USERNAME || "YOUR_USERNAME", -//highlight-next-line -key: process.env.LT_ACCESS_KEY || "YOUR_ACCESS_KEY", - -updateJob: false, -//highlight-next-line -specs: ["./../specs/ios-test.js"], //path of your test script -exclude: [], - -capabilities: [ -{ -build: "NodeJS WebDriverIO iOS", -name: "Sample Test - WebDriverIO", -isRealMobile: true, -deviceName: "iPhone 13 Pro", -platformVersion: "15", -platformName: "iOS", -//highlight-next-line -app: "YOUR_APP_URL", //Enter your app (.ipa) url -}, -], - -logLevel: "info", -coloredLogs: true, -screenshotPath: "./errorShots/", -baseUrl: "", -waitforTimeout: 10000, -connectionRetryTimeout: 90000, -connectionRetryCount: 3, -path: "/wd/hub", -hostname: "mobile-hub.lambdatest.com", -port: 80, - -framework: "mocha", -mochaOpts: { -ui: "bdd", -timeout: 20000, -}, -}; -``` - -- You must set **isRealMobile** capability to `False` in the config file to run on **Virtual Devices** - -```javascript title="android-sample/android-single.conf.js" -exports.config = { -//highlight-next-line -user: process.env.LT_USERNAME || "YOUR_USERNAME", -//highlight-next-line -key: process.env.LT_ACCESS_KEY || "YOUR_ACCESS_KEY", - -updateJob: false, -//highlight-next-line -specs: ["./../specs/android-test.js"], //path of your test script -exclude: [], - -capabilities: [ -{ -build: "NodeJS WebDriverIO Android", -name: "Sample Test - WebDriverIO", -isRealMobile: true, -platformName: "Android", -deviceName: "Galaxy S9", -platformVersion: "10", -app: "YOUR_APP_URL", //Enter your app (.apk) url -}, -], - -logLevel: "info", -coloredLogs: true, -screenshotPath: "./errorShots/", -baseUrl: "", -waitforTimeout: 10000, -connectionRetryTimeout: 90000, -connectionRetryCount: 3, -path: "/wd/hub", -hostname: "mobile-hub.lambdatest.com", -port: 80, - -framework: "mocha", -mochaOpts: { -ui: "bdd", -timeout: 20000, -}, -}; -``` - -- You must set **isRealMobile** capability to `False` in the config file to run on **Virtual Devices** - -- You must add the generated **APP_URL** to the `app` capability in the config file. -- You can generate capabilities for your test requirements with the help of our inbuilt [**Capabilities Generator tool**](https://www.testmuai.com/capabilities-generator/).For more details, please refer to our guide on [**Desired Capabilities in Appium**](/support/docs/desired-capabilities-in-appium/). - -### Step 6: Execute and Monitor your Tests - -- Navigate to the corresponding directory based on your app. - -```bash -cd ios -``` -- Install the required dependencies using the following command: - -```bash -npm i -``` -- Execute the following command to run your test on TestMu AI platform: - -```bash -npm run single -``` - - > Your test results would be displayed on the test console (or CLI if you are using terminal/cmd) and on the [TestMu AI App Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://appautomation.lambdatest.com/build). - -## Using the WebdriverIO Agent Skill with TestMu AI - -The [webdriverio-skill](https://github.com/LambdaTest/agent-skills/tree/main/webdriverio-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The webdriverio-skill package includes: - -``` -webdriverio-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing WebdriverIO Agent Skill - -Install a WebdriverIO Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/webdriverio-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/webdriverio-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only webdriverio-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). - -## Reference Guides - -- [Advanced Configuration for Capabilities](/support/docs/desired-capabilities-in-appium/) -- [How to test locally hosted apps](/support/docs/testing-locally-hosted-pages/) -- [How to integrate TestMu AI with CI/CD](/support/docs/integrations-with-ci-cd-tools/) diff --git a/static/docs/appium-nodejs.md b/static/docs/appium-nodejs.md deleted file mode 100644 index 25c0bee91..000000000 --- a/static/docs/appium-nodejs.md +++ /dev/null @@ -1,334 +0,0 @@ -# Appium with Nodejs - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -In this documentation, you will learn how to configure and run your **NodeJS** automation testing scripts with **Appium** on TestMu AI, set the desired capabilities for appium testing, and other advanced features of TestMu AI. - -## Prerequisites - -- Your TestMu AI [Username and Access key](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/security). -- Install **npm** from the official [npm website](https://www.npmjs.com/). -- Download and install **NodeJS** from official [NodeJS website](https://nodejs.org/en/). You should be having **NodeJS v6** or newer. -- Make sure you are using the latest version of **JavaScript**. - -## Try our Sample Repository - -### Step 1: Get a Sample Project -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** -All the code samples in this documentation can be found on **TestMu AI's Github Repository**. You can either download or clone the repository to quickly run your tests. View on GitHub - -### Step 2: Setup the Environment Variables - -You need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in your [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/security). Run the below mentioned commands in your terminal to setup the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -### Step 3: Upload your Application -Upload your **_iOS_** application (.ipa file) or **_android_** application (.apk or .aab file) to the TestMu AI servers using our **REST API**. You need to provide your **Username** and **AccessKey** in the format `Username:AccessKey` in the **cURL** command for authentication. - -Make sure to add the path of the **appFile** in the cURL request. Below is an example cURL request to upload your app using our REST API: - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "appFile=@"/Users/macuser/Downloads/proverbial_android.apk"" -F "name="proverbial_app""`} - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "url=:https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk" -F "name=Proverbial_App"`} - - - You must set **isRealMobile** capability to `False` in the config file to run on **Virtual Devices** - - You can generate capabilities for your test requirements with the help of our inbuilt [**Capabilities Generator tool**](https://www.testmuai.com/capabilities-generator/).For more details, please refer to our guide on [**Desired Capabilities in Appium**](/support/docs/desired-capabilities-in-appium/). - -- If you do not have any **.apk** or **.ipa** file, you can run your sample tests on TestMu AI by using our sample apps, :link: [Android app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk) or :link: [iOS app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_ios.ipa). - -- Response of above cURL will be a **JSON** object containing the `APP_URL` of the format - `lt://APP123456789123456789` and will be used in the next step. - -### Step 4: Update your Automation Script - -An automation script for the sample application available above has been provided here. Ensure to update the `APP_URL`, `username` and `accesKey` in the code scripts before running the tests. - -```javascript title="iOS.js" -var wd = require("wd"); -var assert = require("assert"); -var asserter = wd.asserters; -username = -process.env.LT_USERNAME == undefined -//highlight-next-line -? "username" //Enter the username here -: process.env.LT_USERNAME; -accesskey = -process.env.LT_ACCESS_KEY == undefined -//highlight-next-line -? "access_key" //Enter the access_key here -: process.env.LT_ACCESS_KEY; - -desired_capabilities = { -deviceName: "iPhone 12", -platformVersion: "14", -platformName: "iOS", -isRealMobile: true, -//highlight-next-line -app: "APP_URL", //Enter the app (.ipa) url -visual: true, -video: true, -build: "NodeJS Vanilla - iOS", -name: "Sample Test - NodeJS", -}; - -driver = wd.promiseRemote( -`https://${username}:${accesskey}@mobile-hub.lambdatest.com/wd/hub` -); - -async function iOStest() { -try { -driver -.init(desired_capabilities) -.then(function () { -return driver.waitForElementById("color", 10000); -}) -.then(function (color) { -return color.click(); -}) -.then(function () { -return driver.waitForElementById("Text", 10000); -}) -.then(function (text) { -text.click(); -return driver.waitForElementById("toast", 10000); -}) -.then(function (toast) { -toast.click(); -return driver.waitForElementById("notification", 10000); -}) -.then(function (notification) { -notification.click(); -return driver.waitForElementById("geoLocation", 10000); -}) -.then(function (geoLocation) { -return geoLocation.click(); -}) -.then(async function () { -return driver.waitForElementById("Back", 10000); -}) -.then(function (Back) { -Back.click(); -return driver.waitForElementById("speedTest", 10000); -}) -.then(async function (speedTest) { -speedTest.click(); -return driver.waitForElementById("Back", 10000); -}) -.then(function (back) { -back.click(); -return driver.waitForElementById("Browser", 10000); -}) -.then(function (Browser) { -Browser.click(); -return driver.waitForElementById("url", 10000); -}) -.then(function (url) { -url.type("https://www.lambdatest.com"); -return driver.waitForElementById("find", 10000); -}) -.then(function (find) { -find.click(); -driver.quit(); -}); -} catch (e) { -driver.quit(); -} -} - -iOStest(); -``` - -```javascript title="Android.js" -const wd = require("wd"); -//highlight-next-line -const username = process.env.LT_USERNAME || "username"; //Add username here - -const accessKey = process.env.LT_ACCESS_KEY || "accessKey"; //Add accessKey here - -const desiredCapabilities = { -app: "APP_URL", // Enter the app (.apk) url -build: "NodeJS - Android", -name: "Sample Test NodeJS", -deviceName: "Galaxy S20", -isRealMobile: true, -platformName: "android", -platformVersion: "11", -video: true, -visual: true, -}; - -const driver = wd.promiseRemote( -`https://${username}:${accessKey}@mobile-hub.lambdatest.com/wd/hub` -); - -const DEFAULT_TIMEOUT = 10000; - -async function runAndroidTest() { -try { -driver -.init(desiredCapabilities) -.then(function () { -return driver.waitForElementById("color", DEFAULT_TIMEOUT); -}) -.then(function (colorButton) { -return colorButton.click(); -}) -.then(function () { -return driver.waitForElementById("Text", DEFAULT_TIMEOUT); -}) -.then(function (text) { -text.click(); -return driver.waitForElementById("toast", DEFAULT_TIMEOUT); -}) -.then(function (toast) { -toast.click(); -return driver.waitForElementById("notification", DEFAULT_TIMEOUT); -}) -.then(function (notification) { -notification.click(); -return driver.waitForElementById("geoLocation", DEFAULT_TIMEOUT); -}) -.then(function (geoLocation) { -geoLocation.click(); -return driver.waitForElementById("buttonPage", DEFAULT_TIMEOUT); -}) -.then(function (Home) { -Home.click(); -return driver.waitForElementById("speedTest", DEFAULT_TIMEOUT); -}) -.then(function (speedTest) { -speedTest.click(); -return driver.waitForElementById("webview", DEFAULT_TIMEOUT); -}) -.then(function (Browser) { -Browser.click(); -return driver.waitForElementById("url", DEFAULT_TIMEOUT); -}) -.then(function (url) { -url.type("https://www.lambdatest.com"); -return driver.waitForElementById("find", DEFAULT_TIMEOUT); -}) -.then(function (find) { -find.click(); -driver.quit(); -}); -} catch (e) { -driver.quit(); -} -} - -runAndroidTest(); -``` - -- You must set **isRealMobile** capability to `False` in the config file to run on **Virtual Devices** - -### Step 5: Configure the Test Capabilities - -You can update your custom capabilities in test scripts. In this sample project, we are passing platform name, platform version, device name and app url _(generated earlier)_ along with other capabilities like build name and test name via capabilities object. - -The capabilities object in the sample code are defined as: - -```javascript -desired_capabilities = { -deviceName: "iPhone 12", -platformVersion: "14", -platformName: "iOS", -isRealMobile: true, -//highlight-next-line -app: "APP_URL", //Enter the app (.ipa) url -visual: true, -video: true, -build: "NodeJS Vanilla - iOS", -name: "Sample Test - NodeJS", -}; -``` - -```javascript -desired_capabilities = { -deviceName: "Galaxy S20", -platformVersion: "11", -platformName: "android", -isRealMobile: true, -//highlight-next-line -app: "APP_URL", //Enter the app (.apk) url -visual: true, -video: true, -build: "NodeJS Vanilla - Android", -name: "Sample Test - NodeJS", -}; -``` - -- You must set **isRealMobile** capability to `False` in the config file to run on **Virtual Devices** - -- You must add the generated **APP_URL** to the `app` capability in the config file. -- You can generate capabilities for your test requirements with the help of our inbuilt [**Capabilities Generator tool**](https://www.testmuai.com/capabilities-generator/).For more details, please refer to our guide on [**Desired Capabilities in Appium**](/support/docs/desired-capabilities-in-appium/). - -### Step 6: Execute and Monitor your Tests - -- Run the following commands to install the required dependencies: - -```bash -npm i wd -``` - -- The tests can be executed in the terminal using the following command: - -```bash -node IOS.js -``` - -```bash -node Android.js -``` - - > Your test results would be displayed on the test console (or CLI if you are using terminal/cmd) and on the [TestMu AI App Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://appautomation.lambdatest.com/build). - -## Using the Appium Agent Skill with TestMu AI - -The [appium-skill](https://github.com/LambdaTest/agent-skills/tree/main/appium-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The appium-skill package includes: - -``` -appium-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing Appium Agent Skill - -Install a Appium Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/appium-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/appium-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only appium-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). - -## Reference Guides - -- [Advanced Configuration for Capabilities](/support/docs/desired-capabilities-in-appium/) -- [How to test locally hosted apps](/support/docs/testing-locally-hosted-pages/) -- [How to integrate TestMu AI with CI/CD](/support/docs/integrations-with-ci-cd-tools/) diff --git a/static/docs/appium-php-behat.md b/static/docs/appium-php-behat.md deleted file mode 100644 index 513b10f56..000000000 --- a/static/docs/appium-php-behat.md +++ /dev/null @@ -1,234 +0,0 @@ -# Appium with Behat - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -In this documentation, you will learn how to trigger a automation script of **Behat** for application testing with **Appium** on TestMu AI, set the [**desired capabilities**](/support/docs/desired-capabilities-in-appium/) for appium testing, and other advanced features of TestMu AI. - -## Prerequisites - -- Your TestMu AI [Username and Access key](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/security). -- Download and install the latest version of PHP in your system. - -Previous versions of **MacOS** have **PHP** installed by default. But for the latest **MacOS** versions starting with **Monterey**, **PHP** has to be downloaded and installed manually by using below commands: - -```bash -/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" -brew install php -``` - -```bash -sudo apt-get install curl libcurl3 libcurl3-dev php -``` - -For **Windows**, you can download **PHP** from [here](http://windows.php.net/download/). Also, refer to this [documentation](http://php.net/manual/en/install.windows.php) for ensuring the accessibility of PHP through Command Prompt(cmd). - -Please add PHP to Windows System Variables Path. - -- Download **composer** in the project directory from here ([Linux/MacOS](https://getcomposer.org/download/), [Windows](https://getcomposer.org/doc/00-intro.md#installation-windows)). - -**note** -To use the **composer** command directly, it should have been downloaded in the project directory. In windows the composer gets installed in different directory. So, copy the 'composer.phar' to the project directory using copy command: - -```bash -copy C:\ProgramData\ComposerSetup\bin\composer.phar -``` - -- Install the composer dependencies in the current project directory using the command below: - -```php -composer update -php composer.phar require phpwhois/phpwhois -php composer.phar install -php composer.phar require php-webdriver/webdriver -php composer.phar require behat/behat -``` - -In case of any error, please try restarting. - -## Try our Sample Repository - -### Step 1: Get a Sample Project -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** -All the code samples in this documentation can be found on **TestMu AI's Github Repository**. You can either download or clone the repository to quickly run your tests. View on GitHub - -### Step 2: Setup the Environment Variables - -You need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in your [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/security). Run the below mentioned commands in your terminal to setup the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -### Step 3: Upload your Application -Upload your **_iOS_** application (.ipa file) or **_android_** application (.apk or .aab file) to the TestMu AI servers using our **REST API**. You need to provide your **Username** and **AccessKey** in the format `Username:AccessKey` in the **cURL** command for authentication. - -Make sure to add the path of the **appFile** in the cURL request. Below is an example cURL request to upload your app using our REST API: - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "appFile=@"/Users/macuser/Downloads/proverbial_android.apk"" -F "name="proverbial_app""`} - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "url=:https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk" -F "name=Proverbial_App"`} - -- If you do not have any **.apk** or **.ipa** file, you can run your sample tests on TestMu AI by using our sample apps, :link: [Android app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk) or :link: [iOS app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_ios.ipa). - -- Response of above cURL will be a **JSON** object containing the `APP_URL` of the format - `lt://APP123456789123456789` and will be used in the next step - -### Step 4: Update your Automation Script - -An automation script for the sample application given above has been provided here. - -```php title="FeatureContext.php" -findElement(WebDriverBy::id("color")); -$element->click(); -} - -/** -* @When I click on text element -*/ -public function iClickOnTextElement() -{ -$element = self::$driver->findElement(WebDriverBy::id("Text")); -$element->click(); -} - -/** -* @When I click on notification element -*/ -public function iClickOnNotificationElement() -{ -$element = self::$driver->findElement(WebDriverBy::id("notification")); -$element->click(); -} - -/** -* @Then I click on toast element -*/ -public function iClickOnToastElement() -{ -$element = self::$driver->findElement(WebDriverBy::id("toast")); -$element->click(); -} -} -``` - -### Step 5: Configure the Test Capabilities - -You can update your custom capabilities in test scripts. In this sample project, we are passing platform name, platform version, device name and app url _(generated earlier)_ along with other capabilities like build name and test name via capabilities object. - -Ensure to update the `APP_URL`, `username` and `accessKey` in the code scripts before running the tests. The capabilities object in the sample code are defined as: - -```php title="iossingle.conf.yml" -user: "YOUR USERNAME HERE" #Add LambdaTest username here -key: "YOUR ACCESS KEY HERE" #Add LambdaTest accessKey here -capabilities: -build: "behat-appium-ios" -name: "single-behat-test" -isRealMobile: true -app: "lt://proverbial-ios" #Add app url here -environments: -- -deviceName: iPhone 11 -platform: ios -platformVersion: 14 -``` - -```php title="androidsingle.conf.yml" -user: "YOUR USERNAME HERE" #Add LambdaTest username here -key: "YOUR ACCESS KEY HERE" #Add LambdaTest accessKey here -capabilities: -build: "behat-appium-android" -name: "single-behat-test" -isRealMobile: true -app: "lt://proverbial-android" #Add your app url here -environments: -- -deviceName: Galaxy S21 Ultra 5G -platform: Android -platformVersion: 11 -``` - -- You must add the generated **APP_URL** to the `app` capability in the config file. -- You must set **isRealMobile** capability to `false` in the config file to run on **Virtual Devices** -- You can generate capabilities for your test requirements with the help of our inbuilt [**Capabilities Generator tool**](https://www.testmuai.com/capabilities-generator/).For more details, please refer to our guide on [**Desired Capabilities in Appium**](/support/docs/desired-capabilities-in-appium/). - -### Step 6: Execute and Monitor your Tests - -- Execute the following command to run your test on TestMu AI platform: - -```php -composer iossingle #for single tests -composer iosparallel #for parallel tests -``` - -```php -composer androidsingle #for single tests -composer androidparallel #for parallel tests -``` - -> Your test results would be displayed on the test console (or CLI if you are using terminal/cmd) and on the [TestMu AI App Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://appautomation.lambdatest.com/build). - -## Using the Behat Agent Skill with TestMu AI - -The [behat-skill](https://github.com/LambdaTest/agent-skills/tree/main/behat-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The behat-skill package includes: - -``` -behat-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing Behat Agent Skill - -Install a Behat Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/behat-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/behat-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only behat-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). - -## Reference Guides - -- [Advanced Configuration for Capabilities](/support/docs/desired-capabilities-in-appium/) -- [How to test locally hosted apps](/support/docs/testing-locally-hosted-pages/) -- [How to integrate TestMu AI with CI/CD](/support/docs/integrations-with-ci-cd-tools/) diff --git a/static/docs/appium-php.md b/static/docs/appium-php.md deleted file mode 100644 index f0d25148f..000000000 --- a/static/docs/appium-php.md +++ /dev/null @@ -1,305 +0,0 @@ -# Appium with PHP - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -In this documentation, you will learn how to trigger a automation script of **PHP** for application testing with **Appium** on TestMu AI, set the [**desired capabilities**](/support/docs/desired-capabilities-in-appium/) for appium testing, and other advanced features of TestMu AI. - -## Prerequisites - -- Your TestMu AI [Username and Access key](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/security). -- Download and install the latest version of PHP in your system. - -Previous versions of **MacOS** have **PHP** installed by default. But for the latest **MacOS** versions starting with **Monterey**, **PHP** has to be downloaded and installed manually by using below commands: - -```bash -/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" -brew install php -``` - -```bash -sudo apt-get install curl libcurl3 libcurl3-dev php -``` - -For **Windows**, download [**PHP**](http://windows.php.net/download/) and refer to the [documentation](http://php.net/manual/en/install.windows.php) for ensuring the accessibility of PHP through Command Prompt(cmd). - -Please add PHP to Windows System Variables Path. - -- Download **composer** in the project directory of ([Linux/MacOS](https://getcomposer.org/download/), [Windows](https://getcomposer.org/doc/00-intro.md#installation-windows)). - -**note** - -To use the **composer** command directly, it should have been downloaded in the project directory. In windows the composer gets installed in different directory. So, copy the 'composer.phar' to the project directory using copy command: - -```bash -copy C:\ProgramData\ComposerSetup\bin\composer.phar -``` - -- Install the composer dependencies in the current project directory using the command below: - -```php -composer update -php composer.phar require phpwhois/phpwhois -php composer.phar install -php composer.phar require php-webdriver/webdriver -``` - -In case of any error, please try restarting. - -## Try our Sample Repository - -### Step 1: Get a Sample Project -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** -All the code samples in this documentation can be found on **TestMu AI's Github Repository**. You can either download or clone the repository to quickly run your tests. View on GitHub - -### Step 2: Setup the Environment Variables - -You need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in your [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/security). Run the below mentioned commands in your terminal to setup the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -### Step 3: Upload your Application -Upload your **_iOS_** application (.ipa file) or **_android_** application (.apk or .aab file) to the TestMu AI servers using our **REST API**. You need to provide your **Username** and **AccessKey** in the format `Username:AccessKey` in the **cURL** command for authentication. - -Make sure to add the path of the **appFile** in the cURL request. Below is an example cURL request to upload your app using our REST API: - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "appFile=@"/Users/macuser/Downloads/proverbial_android.apk"" -F "name="proverbial_app""`} - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "url=:https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk" -F "name=Proverbial_App"`} - -- If you do not have any **.apk** or **.ipa** file, you can run your sample tests on TestMu AI by using our sample apps, :link: [Android app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk) or :link: [iOS app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_ios.ipa). - -- Response of above cURL will be a **JSON** object containing the `APP_URL` of the format - `lt://APP123456789123456789` and will be used in the next step - -### Step 4: Update your Automation Script - -An automation script for the sample application given above has been provided here. Ensure to update the `APP_URL`, `username` and `accessKey` in the code scripts before running the tests. - -```php title="iOSApp.php" - "APP_URL", //Enter app_url here -"deviceName" => "iPhone 11", -"platformName" => "ios", -"platformVersion" => "14", -"isRealMobile" => TRUE, -"visual" => TRUE, -"video" => TRUE, -"name" => "Php - iOS test", -"build" => "Php Vanilla - iOS" -); - -//highlight-next-line -$username = getenv("LT_USERNAME") ? getenv("LT_USERNAME") : "USERNAME"; //Enter username here -//highlight-next-line -$accesskey = getenv("LT_ACCESS_KEY") ? getenv("LT_ACCESS_KEY") : "ACCESS_KEY"; //Enter accesskey here -$driver = RemoteWebDriver::create("http://$username:$accesskey@mobile-hub.lambdatest.com/wd/hub",$caps); -try{ -$wait = new WebDriverWait($driver, 30); -$wait->until(WebDriverExpectedCondition::visibilityOfElementLocated(WebDriverBy::id('color'))); -$color_element = $driver->findElement(WebDriverBy::id('color')); -$color_element->click(); -$text_element = $driver->findElement(WebDriverBy::id('Text')); -$text_element->click(); -$toast_element = $driver->findElement(WebDriverBy::id('toast')); -$toast_element->click(); -$notification_element = $driver->findElement(WebDriverBy::id('notification')); -$notification_element->click(); -sleep(2); -$wait->until(WebDriverExpectedCondition::visibilityOfElementLocated(WebDriverBy::id('geoLocation'))); -$geolocation_element = $driver->findElement(WebDriverBy::id('geoLocation')); -$geolocation_element->click(); -sleep(5); -$home_element = $driver->findElement(WebDriverBy::id('Back')); -$home_element->click(); -$speedtest_element = $driver->findElement(WebDriverBy::id('speedTest')); -$speedtest_element->click(); -sleep(5); -$home_element = $driver->findElement(WebDriverBy::id('Back')); -$home_element->click(); -$browser_element = $driver->findElement(WebDriverBy::id('Browser')); -$browser_element->click(); -$url_element = $driver->findElement(WebDriverBy::id('url')); -$url_element->sendKeys("https://www.testmuai.com"); -$find_element = $driver->findElement(WebDriverBy::id('find')); -$find_element->click(); -sleep(2); -$driver->quit(); -} finally { -$driver->quit(); -} -?> -``` - -```php title="AndroidApp.php" - "APP_URL", //Enter app_url here -"deviceName" => "Galaxy S20", -"platformName" => "Android", -"platformVersion" => "10", -"isRealMobile" => TRUE, -"visual" => TRUE, -"video" => TRUE, -"name" => "Php - Android test", -"build" => "Php Vanilla - Android" -); - -//highlight-next-line -$username = getenv("LT_USERNAME") ? getenv("LT_USERNAME") : "USERNAME"; //Enter username here -//highlight-next-line -$accesskey = getenv("LT_ACCESS_KEY") ? getenv("LT_ACCESS_KEY") : "ACCESS_KEY"; //Enter accesskey here - -$driver = RemoteWebDriver::create("http://$username:$accesskey@mobile-hub.lambdatest.com/wd/hub",$caps); - -try{ -$color_element = $driver->findElement(WebDriverBy::id('color')); -$color_element->click(); - -$text_element = $driver->findElement(WebDriverBy::id('Text')); -$text_element->click(); - -$toast_element = $driver->findElement(WebDriverBy::id('toast')); -$toast_element->click(); - -$notification_element = $driver->findElement(WebDriverBy::id('notification')); -$notification_element->click(); - -$geoLocation_element = $driver->findElement(WebDriverBy::id('geoLocation')); -$geoLocation_element->click(); -sleep(5); - -$url_element = $driver->findElement(WebDriverBy::id('url')); -$url_element->sendkeys("https://www.testmuai.com"); - -$find_element = $driver->findElement(WebDriverBy::id('find')); -$find_element->click(); -sleep(2); - -$driver->quit(); -} finally { -$driver->quit(); -} - -?> -``` - -### Step 5: Configure the Test Capabilities - -You can update your custom capabilities in test scripts. In this sample project, we are passing platform name, platform version, device name and app url _(generated earlier)_ along with other capabilities like build name and test name via capabilities object. - -The capabilities object in the sample code are defined as: - -```csharp title="iOS(.ipa)" -$caps = array( -//highlight-next-line -"app"=> "APP_URL", //Enter app_url here -"deviceName" => "iPhone 11", -"platformName" => "ios", -"platformVersion" => "14", -"isRealMobile" => TRUE, -"visual" => TRUE, -"video" => TRUE, -"name" => "Php - iOS test", -"build" => "Php Vanilla - iOS" -); -``` - -```php title="Android(.apk)" -$caps = array( -//highlight-next-line -"app"=> "APP_URL", //Enter app_url here -"deviceName" => "Galaxy S20", -"platformName" => "Android", -"platformVersion" => "10", -"isRealMobile" => TRUE, -"visual" => TRUE, -"video" => TRUE, -"name" => "Php - Android test", -"build" => "Php Vanilla - Android" -); -``` - -- You must add the generated **APP_URL** to the `app` capability in the config file. -- You must set **isRealMobile** capability to `False` in the config file to run on **Virtual Devices** -- You can generate capabilities for your test requirements with the help of our inbuilt [**Capabilities Generator tool**](https://www.testmuai.com/capabilities-generator/).For more details, please refer to our guide on [**Desired Capabilities in Appium**](/support/docs/desired-capabilities-in-appium/). - -### Step 6: Execute and Monitor your Tests - -- Execute the following command to run your test on TestMu AI platform: - -```bash -php IOSApp.php -``` - -```bash -php AndroidApp.php -``` - -> Your test results would be displayed on the test console (or CLI if you are using terminal/cmd) and on the [TestMu AI App Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://appautomation.lambdatest.com/build). - -## Using the Appium Agent Skill with TestMu AI - -The [appium-skill](https://github.com/LambdaTest/agent-skills/tree/main/appium-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The appium-skill package includes: - -``` -appium-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing Appium Agent Skill - -Install a Appium Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/appium-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/appium-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only appium-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). - -## Reference Guides - -- [Advanced Configuration for Capabilities](/support/docs/desired-capabilities-in-appium/) -- [How to test locally hosted apps](/support/docs/testing-locally-hosted-pages/) -- [How to integrate TestMu AI with CI/CD](/support/docs/integrations-with-ci-cd-tools/) diff --git a/static/docs/appium-python-behave.md b/static/docs/appium-python-behave.md deleted file mode 100644 index 2c271cfa0..000000000 --- a/static/docs/appium-python-behave.md +++ /dev/null @@ -1,303 +0,0 @@ -# Appium with Behave - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -In this documentation, you will learn how to trigger a automation script of **Behave** for application testing with **Appium** on TestMu AI, set the [**desired capabilities**](/support/docs/desired-capabilities-in-appium/) for appium testing, and other advanced features of TestMu AI. - -## Prerequisites - -- Your TestMu AI [Username and Access key](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/security). -- You should have [Python](https://www.python.org/downloads/) installed. -- Download and install [**pip**](https://pip.pypa.io/en/stable/installation/). - -## Try our Sample Repository - -### Step 1: Get a Sample Project -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** -All the code samples in this documentation can be found on **TestMu AI's Github Repository**. You can either download or clone the repository to quickly run your tests. View on GitHub - -### Step 2: Setup the Environment Variables - -You need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in your [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/security). Run the below mentioned commands in your terminal to setup the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -### Step 3: Upload your Application -Upload your **_iOS_** application (.ipa file) or **_android_** application (.apk or .aab file) to the TestMu AI servers using our **REST API**. You need to provide your **Username** and **AccessKey** in the format `Username:AccessKey` in the **cURL** command for authentication. - -Make sure to add the path of the **appFile** in the cURL request. Below is an example cURL request to upload your app using our REST API: - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "appFile=@"/Users/macuser/Downloads/proverbial_android.apk"" -F "name="proverbial_app""`} - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "url=:https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk" -F "name=Proverbial_App"`} - -- If you do not have any **.apk** or **.ipa** file, you can run your sample tests on TestMu AI by using our sample apps, :link: [Android app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk) or :link: [iOS app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_ios.ipa). - -- Response of above cURL will be a **JSON** object containing the `APP_URL` of the format - `lt://APP123456789123456789` and will be used in the next step - -### Step 4: Update your Automation Script - -An automation script for the sample application given above has been provided here. You can write or add your own Appium automation scripts in `*StepDef.py` directory to run different tests on your app. - -```python title="AndroidStepDef.py" -import sys -import os -path = os.getcwd() -sys.path.append(os.path.abspath(os.path.join(path, os.pardir))) -from time import time -from behave import given -from appium import webdriver -import appConfig as appConf -from appium.webdriver.common.mobileby import MobileBy -from selenium.webdriver.support.ui import WebDriverWait -from selenium.webdriver.support import expected_conditions as EC - -@given("Start the android app automation test") -def startAndroidAppAutomationTest(self): -if os.environ.get("LT_USERNAME") is None: -#highlight-next-line -username = "username" #Enter username here -else: -username = os.environ.get("LT_USERNAME") -if os.environ.get("LT_ACCESS_KEY") is None: -#highlight-next-line -accesskey = "accesskey" #Enter accessKey here -else: -accesskey = os.environ.get("LT_ACCESS_KEY") - -driver = webdriver.Remote( -command_executor="https://"+username+":"+accesskey+"@mobile-hub.lambdatest.com/wd/hub", -desired_capabilities=appConf.app_android_desired_caps -) -try: -colorElement = WebDriverWait(driver,20).until(EC.element_to_be_clickable((MobileBy.ID,"com.lambdatest.proverbial:id/color"))) -colorElement.click() - -textElement = WebDriverWait(driver,20).until(EC.element_to_be_clickable((MobileBy.ID,"com.lambdatest.proverbial:id/Text"))) -textElement.click() - -toastElement = WebDriverWait(driver,20).until(EC.element_to_be_clickable((MobileBy.ID,"com.lambdatest.proverbial:id/toast"))) -toastElement.click() - -notification = WebDriverWait(driver,20).until(EC.element_to_be_clickable((MobileBy.ID,"com.lambdatest.proverbial:id/notification"))) -notification.click() - -geolocation = WebDriverWait(driver,20).until(EC.element_to_be_clickable((MobileBy.ID,"com.lambdatest.proverbial:id/geoLocation"))) -geolocation.click() - -home = WebDriverWait(driver,20).until(EC.element_to_be_clickable((MobileBy.ID,"com.lambdatest.proverbial:id/Home"))) -home.click() - -speedTest = WebDriverWait(driver,20).until(EC.element_to_be_clickable((MobileBy.ID,"com.lambdatest.proverbial:id/speedTest"))) -speedTest.click() - -home = WebDriverWait(driver,20).until(EC.element_to_be_clickable((MobileBy.ID,"com.lambdatest.proverbial:id/Home"))) -home.click() - -browser = WebDriverWait(driver,20).until(EC.element_to_be_clickable((MobileBy.ID,"com.lambdatest.proverbial:id/Browser"))) -browser.click() - -url = WebDriverWait(driver,20).until(EC.element_to_be_clickable((MobileBy.ID,"com.lambdatest.proverbial:id/url"))) -url.send_keys("https://www.testmuai.com") - -find = WebDriverWait(driver,20).until(EC.element_to_be_clickable((MobileBy.ID,"com.lambdatest.proverbial:id/find"))) -find.click() - -driver.quit() -except: -driver.quit() -``` - -```python title="iOSStepDef.py" -import sys -import os -path = os.getcwd() -sys.path.append(os.path.abspath(os.path.join(path, os.pardir))) -import appConfig as appConf -from behave import given -from appium import webdriver -import time -from appium.webdriver.common.mobileby import MobileBy -from selenium.webdriver.support.ui import WebDriverWait -from selenium.webdriver.support import expected_conditions as EC - -@given("Start the ios app automation test") -def startIOSAppAutomationTest(self): -if os.environ.get("LT_USERNAME") is None: -#highlight-next-line -username = "username" #Enter username here -else: -username = os.environ.get("LT_USERNAME") -if os.environ.get("LT_ACCESS_KEY") is None: -#highlight-next-line -accesskey = "accesskey" #Enter accesskey herE -else: -accesskey = os.environ.get("LT_ACCESS_KEY") - -driver = webdriver.Remote( -command_executor="https://"+username+":"+accesskey+"@mobile-hub.lambdatest.com/wd/hub", -desired_capabilities=appConf.app_ios_desired_caps -) -try: -colorElement = WebDriverWait(driver,20).until(EC.element_to_be_clickable((MobileBy.ACCESSIBILITY_ID,"color"))) -colorElement.click() - -textElement = WebDriverWait(driver,20).until(EC.element_to_be_clickable((MobileBy.ACCESSIBILITY_ID,"Text"))) -textElement.click() - -toastElement = WebDriverWait(driver,20).until(EC.element_to_be_clickable((MobileBy.ACCESSIBILITY_ID,"toast"))) -toastElement.click() - -notification = WebDriverWait(driver,20).until(EC.element_to_be_clickable((MobileBy.ACCESSIBILITY_ID,"notification"))) -notification.click() -time.sleep(3) - -geolocation = WebDriverWait(driver,20).until(EC.element_to_be_clickable((MobileBy.ACCESSIBILITY_ID,"geoLocation"))) -geolocation.click() -time.sleep(3) - -home = WebDriverWait(driver,20).until(EC.element_to_be_clickable((MobileBy.ACCESSIBILITY_ID,"Back"))) -home.click() - -speedTest = WebDriverWait(driver,20).until(EC.element_to_be_clickable((MobileBy.ACCESSIBILITY_ID,"speedTest"))) -speedTest.click() -time.sleep(3) - -home = WebDriverWait(driver,20).until(EC.element_to_be_clickable((MobileBy.ACCESSIBILITY_ID,"Back"))) -home.click() - -browser = WebDriverWait(driver,20).until(EC.element_to_be_clickable((MobileBy.ACCESSIBILITY_ID,"Browser"))) -browser.click() - -url = WebDriverWait(driver,20).until(EC.element_to_be_clickable((MobileBy.ACCESSIBILITY_ID,"url"))) -url.send_keys("https://www.testmuai.com") - -find = WebDriverWait(driver,20).until(EC.element_to_be_clickable((MobileBy.ACCESSIBILITY_ID,"find"))) -find.click() - -driver.quit() -except: -driver.quit() - -``` - -### Step 5: Configure the Test Capabilities - -You need to update your capabilities in `appConfig.py` files. In this sample project, we are passing platform name, platform version, device name and app url (generated earlier) along with other capabilities like build name and test name via capabilities object. The capabilities object in the sample code for a single test are defined as: - -The capabilities for running tests on both **Android** and **iOS** apps are: - -```python title="appConfig.py" -app_ios_desired_caps = { -"lt:options": { -"deviceName":"iPhone 12", -"platformName":"ios", -"platformVersion":"14", -"build":"Python Behave - iOS", -"name":"Sample Test iOS", -#highlight-next-line -"app":"APP_URL" ,#Enter app (.ipa) url here -"isRealMobile":True, -"network":False, -"visual":True, -"video":True, -"w3c":True -} -} -``` - -- You must set **isRealMobile** capability to `False` in the config file to run on **Virtual Devices** - -```python title="appConfig.py" -app_android_desired_caps = { -"lt:options": { -"platformName": "android", -"deviceName": "OnePlus 6", -"platformVersion": "8", -"build": "Python Behave - Android", -"name": "Sample Test Android", -#highlight-next-line -"app": "APP_URL", #Enter app (.apk) url here -"visual": True, -"video": True, -"w3c": True, -"isRealMobile": True -} -} -``` - -- You must set **isRealMobile** capability to `False` in the config file to run on **Virtual Devices** - -- You must add the generated **APP_URL** to the `app` capability in the config file. -- You must set **isRealMobile** capability to `False` in the config file to run on **Virtual Devices** -- You can generate capabilities for your test requirements with the help of our inbuilt [**Capabilities Generator tool**](https://www.testmuai.com/capabilities-generator/).For more details, please refer to our guide on [**Desired Capabilities in Appium**](/support/docs/desired-capabilities-in-appium/). - -### Step 6: Execute and Monitor your Tests - -- Install the required packages from the cloned project directory: - -```bash -pip install -r requirements.txt -``` - -- Execute the following command to run your test on TestMu AI platform: - -```bash -behave --tags @iosApp -``` - -```bash -behave --tags @androidApp -``` - - > Your test results would be displayed on the test console (or CLI if you are using terminal/cmd) and on the [TestMu AI App Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://appautomation.lambdatest.com/build). - -## Using the Behave Agent Skill with TestMu AI - -The [behave-skill](https://github.com/LambdaTest/agent-skills/tree/main/behave-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The behave-skill package includes: - -``` -behave-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing Behave Agent Skill - -Install a Behave Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/behave-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/behave-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only behave-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). - -## Reference Guides - -- [Advanced Configuration for Capabilities](/support/docs/desired-capabilities-in-appium/) -- [How to test locally hosted apps](/support/docs/testing-locally-hosted-pages/) -- [How to integrate TestMu AI with CI/CD](/support/docs/integrations-with-ci-cd-tools/) diff --git a/static/docs/appium-python-gauge.md b/static/docs/appium-python-gauge.md deleted file mode 100644 index 53892861f..000000000 --- a/static/docs/appium-python-gauge.md +++ /dev/null @@ -1,177 +0,0 @@ -# Appium with Gauge - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -In this documentation, you will learn how to trigger a automation script of **Gauge** for application testing with **Appium** on TestMu AI, set the [**desired capabilities**](/support/docs/desired-capabilities-in-appium/) for appium testing, and other advanced features of TestMu AI. - -## Prerequisites - -- Your TestMu AI [Username and Access key](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/security). -- You should have [Python](https://www.python.org/downloads/) installed. -- Download and install [**pip**](https://pip.pypa.io/en/stable/installation/). - -## Try our Sample Repository - -### Step 1: Get a Sample Project -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** -All the code samples in this documentation can be found on **TestMu AI's Github Repository**. You can either download or clone the repository to quickly run your tests. View on GitHub - -### Step 2: Setup the Environment Variables - -You need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in your [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/security). Run the below mentioned commands in your terminal to setup the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -### Step 3: Upload your Application -Upload your **_iOS_** application (.ipa file) or **_android_** application (.apk or .aab file) to the TestMu AI servers using our **REST API**. You need to provide your **Username** and **AccessKey** in the format `Username:AccessKey` in the **cURL** command for authentication. - -Make sure to add the path of the **appFile** in the cURL request. Below is an example cURL request to upload your app using our REST API: - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "appFile=@"/Users/macuser/Downloads/proverbial_android.apk"" -F "name="proverbial_app""`} - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "url=:https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk" -F "name=Proverbial_App"`} - -- If you do not have any **.apk** or **.ipa** file, you can run your sample tests on TestMu AI by using our sample apps, :link: [Android app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk) or :link: [iOS app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_ios.ipa). - -- Response of above cURL will be a **JSON** object containing the `APP_URL` of the format - `lt://APP123456789123456789` and will be used in the next step - -### Step 4: Update your Automation Script - -An automation script for the sample application given above has been provided here. - -```python title="get_started.py" -from sqlite3 import Time -from appium.webdriver.common.mobileby import MobileBy -from selenium.webdriver.support.ui import WebDriverWait -from selenium.webdriver.support import expected_conditions as EC -from getgauge.python import step -from step_impl.utils.driver import Driver - -@step("changes color to pink") -def change_color(): -colorElement = WebDriverWait(Driver.driver, 20).until(EC.element_to_be_clickable( -(MobileBy.ID, "com.lambdatest.proverbial:id/color"))) -colorElement.click() - -@step("changes the text to Proverbial") -def chnage_text(): -textElement = WebDriverWait(Driver.driver, 20).until( -EC.element_to_be_clickable((MobileBy.ID, "com.lambdatest.proverbial:id/Text"))) -textElement.click() - -@step("toast will be visible") -def toast_visible(): -toastElement = WebDriverWait(Driver.driver, 20).until(EC.element_to_be_clickable( -(MobileBy.ID, "com.lambdatest.proverbial:id/toast"))) -toastElement.click() - -@step("notification will be visible") -def notification(): -notification = WebDriverWait(Driver.driver, 20).until(EC.element_to_be_clickable( -(MobileBy.ID, "com.lambdatest.proverbial:id/notification"))) -notification.click() - -@step("perform the speed test") -def speedTest(): -speedTest = WebDriverWait(Driver.driver, 20).until(EC.element_to_be_clickable( -(MobileBy.ID, "com.lambdatest.proverbial:id/speedTest"))) -speedTest.click() - -@step("back to home") -def home(): -home = WebDriverWait(Driver.driver, 20).until(EC.element_to_be_clickable( -(MobileBy.ID, "com.lambdatest.proverbial:id/buttonPage"))) -home.click() -``` - -### Step 5: Configure the Test Capabilities - -You can update your custom capabilities in test scripts. In this sample project, we are passing platform name, platform version, device name and app url _(generated earlier)_ along with other capabilities like build name and test name via capabilities object. - -Ensure to update the `APP_URL`, `username` and `accessKey` in the code scripts before running the tests. The capabilities object in the sample code are defined as: - -```python title="driver.py" -caps['name'] = 'Gauge Sample Test' -caps['build'] = 'Python_Gauge_LambdaTest' -caps['isRealMobile'] = 'true' -caps['platformVersion'] = '11' -caps['platform'] = 'Android' -caps['deviceName'] = 'Galaxy S21 Ultra 5G' -#highlight-next-line -caps['app'] = 'APP_URL' #add app url here -} - -``` - -- You must add the generated **APP_URL** to the `app` capability in the config file. -- You must set **isRealMobile** capability to `False` in the config file to run on **Virtual Devices** -- You can generate capabilities for your test requirements with the help of our inbuilt [**Capabilities Generator tool**](https://www.testmuai.com/capabilities-generator/).For more details, please refer to our guide on [**Desired Capabilities in Appium**](/support/docs/desired-capabilities-in-appium/). - -### Step 6: Execute and Monitor your Tests - -- Install the required packages from the cloned project directory: - -```bash -pip install -r requirements.txt -npm install -g @getgauge/cli -set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python #for Windows -export PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python #for MacOS/Linux -``` - -- Run the following command in the directory where your project has been saved to execute your build. - -```bash -gauge run specs -``` - -> Your test results would be displayed on the test console (or CLI if you are using terminal/cmd) and on the [TestMu AI App Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://appautomation.lambdatest.com/build). - -## Using the Gauge Agent Skill with TestMu AI - -The [gauge-skill](https://github.com/LambdaTest/agent-skills/tree/main/gauge-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The gauge-skill package includes: - -``` -gauge-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing Gauge Agent Skill - -Install a Gauge Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/gauge-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/gauge-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only gauge-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). - -## Reference Guides - -- [Advanced Configuration for Capabilities](/support/docs/desired-capabilities-in-appium/) -- [How to test locally hosted apps](/support/docs/testing-locally-hosted-pages/) -- [How to integrate TestMu AI with CI/CD](/support/docs/integrations-with-ci-cd-tools/) diff --git a/static/docs/appium-python-pytest.md b/static/docs/appium-python-pytest.md deleted file mode 100644 index 3e58cce3d..000000000 --- a/static/docs/appium-python-pytest.md +++ /dev/null @@ -1,241 +0,0 @@ -# Appium with Pytest - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -In this documentation, you will learn how to trigger a automation script of **Pytest** for application testing with **Appium** on TestMu AI, set the [**desired capabilities**](/support/docs/desired-capabilities-in-appium/) for appium testing, and other advanced features of TestMu AI. - -## Prerequisites - -- Your TestMu AI [Username and Access key](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/security). -- You should have [Python](https://www.python.org/downloads/) installed. -- Download and install [**pip**](https://pip.pypa.io/en/stable/installation/). -- Install pytest on your system with following pip command: - -```bash -pip install pytest -``` - -## Try our Sample Repository - -### Step 1: Get a Sample Project -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** -All the code samples in this documentation can be found on **TestMu AI's Github Repository**. You can either download or clone the repository to quickly run your tests. View on GitHub - -### Step 2: Setup the Environment Variables - -You need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in your [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/security). Run the below mentioned commands in your terminal to setup the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -### Step 3: Upload your Application -Upload your **_iOS_** application (.ipa file) or **_android_** application (.apk or .aab file) to the TestMu AI servers using our **REST API**. You need to provide your **Username** and **AccessKey** in the format `Username:AccessKey` in the **cURL** command for authentication. - -Make sure to add the path of the **appFile** in the cURL request. Below is an example cURL request to upload your app using our REST API: - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "appFile=@"/Users/macuser/Downloads/proverbial_android.apk"" -F "name="proverbial_app""`} - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "url=:https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk" -F "name=Proverbial_App"`} - -- If you do not have any **.apk** or **.ipa** file, you can run your sample tests on TestMu AI by using our sample apps, :link: [Android app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk) or :link: [iOS app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_ios.ipa). - -- Response of above cURL will be a **JSON** object containing the `APP_URL` of the format - `lt://APP123456789123456789` and will be used in the next step - -### Step 4: Update your Automation Script - -An automation script for the sample application given above has been provided here. Ensure to update the `APP_URL`, `username` and `accessKey` in the code scripts before running the tests. - -```python title="ios.py" -from os import environ -import pytest -from appium import webdriver - -@pytest.fixture(scope='function') -def test_setup_ios(request): -test_name = request.node.name -build = environ.get('BUILD', "Pytest IOS Sample") -caps = {} -caps["deviceName"] = "iPhone 11" -caps["platformName"] = "iOS" -caps["platformVersion"] = "14" -caps["app"] = "lt://proverbial-ios" #Enter the app (.ipa) url here -caps["isRealMobile"] = True -caps['build'] = build -caps['name'] = test_name -caps['project'] = project_name -driver = webdriver.Remote("https://:@mobile-hub.lambdatest.com/wd/hub", caps) #Add LambdaTest username and accessKey here -request.cls.driver = driver - -yield driver - -def fin(): -#browser.execute_script("lambda-status=".format(str(not request.node.rep_call.failed if "passed" else "failed").lower())) -if request.node.rep_call.failed: -driver.execute_script('lambda-status=failed') -else: -driver.execute_script('lambda-status=passed') -driver.quit() -request.addfinalizer(fin) - -@pytest.hookimpl(tryfirst=True, hookwrapper=True) -def pytest_runtest_makereport(item, call): -# this sets the result as a test attribute for LambdaTest reporting. -# execute all other hooks to obtain the report object -outcome = yield -rep = outcome.get_result() - -# set an report attribute for each phase of a call, which can -# be "setup", "call", "teardown" -setattr(item, "rep_" + rep.when, rep) -``` - -```python title="conftest.py" -from os import environ -import pytest -from appium import webdriver - -@pytest.fixture(scope='function') -def test_setup_android(request): -test_name = request.node.name -build = environ.get('BUILD', "Pytest Android Sample") -caps = {} -caps["deviceName"] = "Galaxy S21 5G" -caps["platformName"] = "Android" -caps["platformVersion"] = "11" -caps["app"] = "lt://proverbial-android" #Enter the app (.apk) url here -caps["isRealMobile"] = True -caps['build'] = build -caps['name'] = test_name -caps['project'] = project_name -driver = webdriver.Remote("https://:@mobile-hub.lambdatest.com/wd/hub", caps) #Add LambdaTest username and accessKey here -request.cls.driver = driver - -yield driver - -def fin(): -#browser.execute_script("lambda-status=".format(str(not request.node.rep_call.failed if "passed" else "failed").lower())) -if request.node.rep_call.failed: -driver.execute_script('lambda-status=failed') -else: -driver.execute_script('lambda-status=passed') -driver.quit() -request.addfinalizer(fin) - -@pytest.hookimpl(tryfirst=True, hookwrapper=True) -def pytest_runtest_makereport(item, call): -# this sets the result as a test attribute for LambdaTest reporting. -# execute all other hooks to obtain the report object -outcome = yield -rep = outcome.get_result() - -# set an report attribute for each phase of a call, which can -# be "setup", "call", "teardown" -setattr(item, "rep_" + rep.when, rep) -``` - -### Step 5: Configure the Test Capabilities - -You can update your custom capabilities in test scripts. In this sample project, we are passing platform name, platform version, device name and app url _(generated earlier)_ along with other capabilities like build name and test name via capabilities object. - -The capabilities object in the sample code are defined as: - -```python title="iOS(.ipa)" -caps = { -"deviceName": "iPhone 11", -"platformName": "iOS", -"platformVersion": "14", -"app": "lt://proverbial-ios" # Enter the app (.ipa) URL here, -"isRealMobile": True, -"build": build, -"name": test_name, -"project": project_name -} -``` - -```python title="Android(.apk)" -caps = { -"deviceName": "Galaxy S21 5G", -"platformName": "Android", -"platformVersion": "11", -"app": "lt://proverbial-android" # Enter the app (.apk) URL here, -"isRealMobile": True, -"build": build, -"name": test_name, -"project": project_name -} -``` - -- You must add the generated **APP_URL** to the `app` capability in the config file. -- You must set **isRealMobile** capability to `False` in the config file to run on **Virtual Devices** -- You can generate capabilities for your test requirements with the help of our inbuilt [**Capabilities Generator tool**](https://www.testmuai.com/capabilities-generator/).For more details, please refer to our guide on [**Desired Capabilities in Appium**](/support/docs/desired-capabilities-in-appium/). - -### Step 6: Execute and Monitor your Tests - -- Install the required packages from the cloned project directory: - -```bash -pip install -r requirements.txt -``` - -- Run the following command in the directory where your project has been saved to execute your build. - -```bash -pytest test_ios.py -``` - -```bash -pytest test.py -``` - -If you are unable to run the automation script with the above mentioned commands try using `python -m` before the given commands. - -> Your test results would be displayed on the test console (or CLI if you are using terminal/cmd) and on the [TestMu AI App Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://appautomation.lambdatest.com/build). - -## Using the Pytest Agent Skill with TestMu AI - -The [pytest-skill](https://github.com/LambdaTest/agent-skills/tree/main/pytest-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The pytest-skill package includes: - -``` -pytest-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing Pytest Agent Skill - -Install a Pytest Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/pytest-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/pytest-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only pytest-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). - -## Reference Guides - -- [Advanced Configuration for Capabilities](/support/docs/desired-capabilities-in-appium/) -- [How to test locally hosted apps](/support/docs/testing-locally-hosted-pages/) -- [How to integrate TestMu AI with CI/CD](/support/docs/integrations-with-ci-cd-tools/) diff --git a/static/docs/appium-python-robot.md b/static/docs/appium-python-robot.md deleted file mode 100644 index fa7fd35c7..000000000 --- a/static/docs/appium-python-robot.md +++ /dev/null @@ -1,251 +0,0 @@ -# Appium with Robot - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -In this documentation, you will learn how to trigger a automation script of **Robot** for application testing with **Appium** on TestMu AI, set the [**desired capabilities**](/support/docs/desired-capabilities-in-appium/) for appium testing, and other advanced features of TestMu AI. - -## Prerequisites - -- Your TestMu AI [Username and Access key](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/security). -- You should have [Python](https://www.python.org/downloads/) installed. -- Download and install [**pip**](https://pip.pypa.io/en/stable/installation/). - -## Set-up Your Virtual Environment for Linux/macOS - -### Create your Virtual Environment - -```python -python3 -m virtualenv venv -``` - -### Activate your Virtual Environment - -```python -source venv/bin/activate -``` -These commands will create a new virtual environment name `venv` and activate it. - -## Setup Your Authentication - -Replace TestMu AI `username` and `accesskey` in the `common.robot` file as mentioned below: - -```js title="common.robot" -*** Settings *** -Library AppiumLibrary - -*** Variables *** - -${platformName} ios -#${platformVersion} 15 # Set your default version -${deviceName} iPhone.* -${visual} True -${network} True -${isRealMobile} True -${LT_APP_ID} '' -${LT_GRID_URL} '' -${TIMEOUT} 3000 - -*** Keywords *** - -Open test app -[Timeout] ${TIMEOUT} -${CAPABILITIES}= Create Dictionary -... platformName=${platformName} -... platformVersion=${version} -... deviceName=${deviceName} -... visual=${visual} -... network=${network} -... devicelog=${devicelog} -... isRealMobile=${isRealMobile} -... name=LT_Appium_Robot_App_iOS -... build=LT_Appium_Robot_App_Automation -... app=${LT_APP_ID} -TRY -${REMOTE_URL}= Set Variable If '%{LT_GRID_URL}' == '' mobile-hub.lambdatest.com %{LT_GRID_URL} -EXCEPT -${REMOTE_URL}= Set Variable mobile-hub.lambdatest.com -END -TRY -${APP_ID}= Set Variable If '%{LT_APP_ID}' == '' lt://proverbial-ios %{LT_APP_ID} -EXCEPT -${APP_ID}= Set Variable lt://proverbial-ios -END -${REMOTE_URL}= Set Variable https://%{LT_USERNAME}:%{LT_ACCESS_KEY}@${REMOTE_URL}/wd/hub - -Open Application ${REMOTE_URL} platformName=ios platformVersion=${version} deviceName=${deviceName} visual=${visual} network=${network} devicelog=${devicelog} isRealMobile=${isRealMobile} app=${APP_ID} name=LT_Appium_Robot_App_iOS build=LT_Appium_Robot_App_Automation - -Close test app -Close All Applications -``` - -- You must set **isRealMobile** capability to `False` in the config file to run on **Virtual Devices** -- You can generate capabilities for your test requirements with the help of our inbuilt [**Capabilities Generator tool**](https://www.testmuai.com/capabilities-generator/). For more details, please refer to our guide on [**Desired Capabilities in Appium**](/support/docs/desired-capabilities-in-appium/). - -## Try our Sample Repository - -### Step 1: Get a Sample Project -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** -All the code samples in this documentation can be found on **TestMu AI's Github Repository**. You can either download or clone the repository to quickly run your tests. View on GitHub - -### Step 2: Setup the Environment Variables - -You need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in your [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/security). Run the below mentioned commands in your terminal to setup the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -### Step 3: Upload your Application -Upload your **_iOS_** application (.ipa file) or **_android_** application (.apk or .aab file) to the TestMu AI servers using our **REST API**. You need to provide your **Username** and **AccessKey** in the format `Username:AccessKey` in the **cURL** command for authentication. - -Make sure to add the path of the **appFile** in the cURL request. Below is an example cURL request to upload your app using our REST API: - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "appFile=@"/Users/macuser/Downloads/proverbial_android.apk"" -F "name="proverbial_app""`} - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "url=:https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk" -F "name=Proverbial_App"`} - -- If you do not have any **.apk** or **.ipa** file, you can run your sample tests on TestMu AI by using our sample apps, :link: [Android app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk) or :link: [iOS app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_ios.ipa). - -- Response of above cURL will be a **JSON** object containing the `APP_URL` of the format - `lt://APP123456789123456789` and will be used in the next step - -### Step 4: Update your Automation Script - -An automation script file `*StepDef.py` for the sample application given above has been provided here. - -```python title="IOS.robot" -*** Settings *** - -Resource ../Resources/Common.robot - -Test Setup Common.Open test app -Test Teardown Common.Close test app - -*** Variables *** -${TIMEOUT} 3000 - -*** Test Cases *** - -Example of connecting to Lambdatest via Robot Framework -[Timeout] ${TIMEOUT} -Click element id=color -Click element id=Text -Click element id=toast -Click element id=notification -Click element id=geoLocation -``` - -```python title="Android.robot" -*** Settings *** - -Resource ../Resources/Common.robot - -Test Setup Common.Open test app -Test Teardown Common.Close test app - -*** Variables *** -${TIMEOUT} 3000 - -*** Test Cases *** - -Example of connecting to Lambdatest via Robot Framework -[Timeout] ${TIMEOUT} -Click element id=color -Click element id=Text -Click element id=toast - -Click element id=notification -Click element id=geoLocation -Sleep 2 seconds -Click element id=Home -Sleep 2 seconds -Click element id=speedTest -Sleep 2 seconds - -Click element id=Browser -Input Text id=url https://lambdatest.com -Click element id=find - -``` - -### Step 5: Configure the Test Capabilities - -You can update your custom capabilities in test scripts `Makefile` file. In this sample project, we are passing platform name, platform version, device name and app url _(generated earlier)_ along with other capabilities like build name and test name via capabilities object. - -```python title="Makefile" -test_Web_ios: -robot --variable version:15 --variable platformName:ios --variable deviceName:"iPhone.*" --variable isRealMobile:true --variable visual:true --variable network:true --variable console:true --variable devicelog:true Tests/AndroidIosWeb.robot - -test_Web_Android: -robot --variable version:11 --variable platformName:android --variable deviceName:"Galaxy.*" --variable isRealMobile:true --variable visual:true --variable network:true --variable console:true --variable devicelog:true Tests/AndroidIosWeb.robot -``` - -- You must add the generated **APP_URL** to the `app` capability in the config file. -- You can generate capabilities for your test requirements with the help of our inbuilt [**Capabilities Generator tool**](https://www.testmuai.com/capabilities-generator/).For more details, please refer to our guide on [**Desired Capabilities in Appium**](/support/docs/desired-capabilities-in-appium/). - -### Step 6: Execute and Monitor your Tests - -- Install the required packages from the cloned project directory: - -```bash -pip install -r requirements.txt -``` - -- Execute the following command to run your test on TestMu AI platform: - -```bash -make test_iOS1 -``` - -```bash -make test_Android1 -``` - - > Your test results would be displayed on the test console (or CLI if you are using terminal/cmd) and on the [TestMu AI App Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://appautomation.lambdatest.com/build). - -## Using the Robot Framework Agent Skill with TestMu AI - -The [robot-framework-skill](https://github.com/LambdaTest/agent-skills/tree/main/robot-framework-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The robot-framework-skill package includes: - -``` -robot-framework-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing Robot Framework Agent Skill - -Install a Robot Framework Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/robot-framework-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/robot-framework-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only robot-framework-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). - -## Reference Guides - -- [Advanced Configuration for Capabilities](/support/docs/desired-capabilities-in-appium/) -- [How to test locally hosted apps](/support/docs/testing-locally-hosted-pages/) -- [How to integrate TestMu AI with CI/CD](/support/docs/integrations-with-ci-cd-tools/) diff --git a/static/docs/appium-python.md b/static/docs/appium-python.md deleted file mode 100644 index e859c1b01..000000000 --- a/static/docs/appium-python.md +++ /dev/null @@ -1,321 +0,0 @@ -# Appium with Python - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -In this documentation, you will learn how to trigger a automation script of **Python** for application testing with **Appium** on TestMu AI, set the [**desired capabilities**](/support/docs/desired-capabilities-in-appium/) for appium testing, and other advanced features of TestMu AI. - -## Prerequisites - -- Your TestMu AI [Username and Access key](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/security). -- You should have [Python](https://www.python.org/downloads/) installed. -- Download and install [**pip**](https://pip.pypa.io/en/stable/installation/). - -## Try our Sample Repository - -### Step 1: Get a Sample Project -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** -All the code samples in this documentation can be found on **TestMu AI's Github Repository**. You can either download or clone the repository to quickly run your tests. View on GitHub - -### Step 2: Setup the Environment Variables - -You need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in your [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/security). Run the below mentioned commands in your terminal to setup the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -### Step 3: Upload your Application -Upload your **_iOS_** application (.ipa file) or **_android_** application (.apk or .aab file) to the TestMu AI servers using our **REST API**. You need to provide your **Username** and **AccessKey** in the format `Username:AccessKey` in the **cURL** command for authentication. - -Make sure to add the path of the **appFile** in the cURL request. Below is an example cURL request to upload your app using our REST API: - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "appFile=@"/Users/macuser/Downloads/proverbial_android.apk"" -F "name="proverbial_app""`} - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "url=:https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk" -F "name=Proverbial_App"`} - -- If you do not have any **.apk** or **.ipa** file, you can run your sample tests on TestMu AI by using our sample apps, :link: [Android app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk) or :link: [iOS app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_ios.ipa). - -- Response of above cURL will be a **JSON** object containing the `APP_URL` of the format - `lt://APP123456789123456789` and will be used in the next step - -### Step 4: Update your Automation Script - -An automation script for the sample application given above has been provided here. Ensure to update the `APP_URL`, `username` and `accessKey` in the code scripts before running the tests. - -```python title="ios.py" -from appium import webdriver -from appium.webdriver.common.mobileby import MobileBy -from selenium.webdriver.support.ui import WebDriverWait -from selenium.webdriver.support import expected_conditions as EC -import time -import os - -desired_caps = { -"deviceName": "iPhone 12", -"platformName": "ios", -"platformVersion": "14", -"isRealMobile": True, -#highlight-next-line -"app": "APP_URL", # Enter app (.ipa) url -"build": "Python Vanilla iOS", -"name": "Sample Test - Python", -"network": False, -"visual": True, -"video": True -} - -def startingTest(): -if os.environ.get("LT_USERNAME") is None: -username = "username" #Add username here -else: -username = os.environ.get("LT_USERNAME") -if os.environ.get("LT_ACCESS_KEY") is None: -accesskey = "accesskey" #Add accessKey here -else: -accesskey = os.environ.get("LT_ACCESS_KEY") - -try: -driver = webdriver.Remote(desired_capabilities=desired_caps, command_executor="https://" + -username+":"+accesskey+"@mobile-hub.lambdatest.com/wd/hub") -time.sleep(3) -colorElement = WebDriverWait(driver, 20).until( -EC.element_to_be_clickable((MobileBy.ACCESSIBILITY_ID, "color"))) -colorElement.click() -textElement = WebDriverWait(driver, 20).until( -EC.element_to_be_clickable((MobileBy.ACCESSIBILITY_ID, "Text"))) -textElement.click() -toastElement = WebDriverWait(driver, 20).until( -EC.element_to_be_clickable((MobileBy.ACCESSIBILITY_ID, "toast"))) -toastElement.click() -notification = WebDriverWait(driver, 20).until( -EC.element_to_be_clickable((MobileBy.ACCESSIBILITY_ID, "notification"))) -notification.click() -time.sleep(3) -geolocation = WebDriverWait(driver, 20).until( -EC.element_to_be_clickable((MobileBy.ACCESSIBILITY_ID, "geoLocation"))) -geolocation.click() -time.sleep(5) -driver.back() -home = WebDriverWait(driver, 20).until( -EC.element_to_be_clickable((MobileBy.ACCESSIBILITY_ID, "Home"))) -home.click() -speedTest = WebDriverWait(driver, 20).until( -EC.element_to_be_clickable((MobileBy.ACCESSIBILITY_ID, "speedTest"))) -speedTest.click() -time.sleep(5) -driver.back() -browser = WebDriverWait(driver, 20).until( -EC.element_to_be_clickable((MobileBy.ACCESSIBILITY_ID, "Browser"))) -browser.click() -url = WebDriverWait(driver, 20).until( -EC.element_to_be_clickable((MobileBy.ACCESSIBILITY_ID, "url"))) -url.send_keys("https://www.testmuai.com") -find = WebDriverWait(driver, 20).until( -EC.element_to_be_clickable((MobileBsy.ACCESSIBILITY_ID, "find"))) -find.click() -driver.quit() -except: -driver.quit() - -startingTest() -``` - -```python title="android.py" -from appium import webdriver -from appium.webdriver.common.mobileby import MobileBy -from selenium.webdriver.support.ui import WebDriverWait -from selenium.webdriver.support import expected_conditions as EC -import time -import os - -desired_caps = { -"deviceName": "Galaxy S20", -"platformName": "Android", -"platformVersion": "10", -#highlight-next-line -"app": "APP_URL", # Enter app (.apk) url -"isRealMobile": True, -"build": "Python Vanilla Android", -"name": "Sample Test - Python", -"network": False, -"visual": True, -"video": True -} - -def startingTest(): -if os.environ.get("LT_USERNAME") is None: -username = "username" #Add username here -else: -username = os.environ.get("LT_USERNAME") -if os.environ.get("LT_ACCESS_KEY") is None: -accesskey = "accesskey" #Add accessKey here -else: -accesskey = os.environ.get("LT_ACCESS_KEY") - -try: -driver = webdriver.Remote(desired_capabilities=desired_caps, command_executor="https://" + -username+":"+accesskey+"@mobile-hub.lambdatest.com/wd/hub") -colorElement = WebDriverWait(driver, 20).until(EC.element_to_be_clickable( -(MobileBy.ID, "com.lambdatest.proverbial:id/color"))) -colorElement.click() - -textElement = WebDriverWait(driver, 20).until( -EC.element_to_be_clickable((MobileBy.ID, "com.lambdatest.proverbial:id/Text"))) -textElement.click() - -toastElement = WebDriverWait(driver, 20).until(EC.element_to_be_clickable( -(MobileBy.ID, "com.lambdatest.proverbial:id/toast"))) -toastElement.click() - -notification = WebDriverWait(driver, 20).until(EC.element_to_be_clickable( -(MobileBy.ID, "com.lambdatest.proverbial:id/notification"))) -notification.click() - -geolocation = WebDriverWait(driver, 20).until(EC.element_to_be_clickable( -(MobileBy.ID, "com.lambdatest.proverbial:id/geoLocation"))) -geolocation.click() -time.sleep(5) - -driver.back() - -home = WebDriverWait(driver, 20).until(EC.element_to_be_clickable( -(MobileBy.ID, "com.lambdatest.proverbial:id/buttonPage"))) -home.click() - -speedTest = WebDriverWait(driver, 20).until(EC.element_to_be_clickable( -(MobileBy.ID, "com.lambdatest.proverbial:id/speedTest"))) -speedTest.click() -time.sleep(5) - -driver.back() - -browser = WebDriverWait(driver, 20).until(EC.element_to_be_clickable( -(MobileBy.ID, "com.lambdatest.proverbial:id/webview"))) -browser.click() - -url = WebDriverWait(driver, 20).until(EC.element_to_be_clickable( -(MobileBy.ID, "com.lambdatest.proverbial:id/url"))) -url.send_keys("https://www.testmuai.com") - -find = WebDriverWait(driver, 20).until(EC.element_to_be_clickable( -(MobileBy.ID, "com.lambdatest.proverbial:id/find"))) -find.click() -driver.quit() -except: -driver.quit() - -startingTest() -``` - -- You must set **isRealMobile** capability to `False` in the config file to run on **Virtual Devices** - -### Step 5: Configure the Test Capabilities - -You can update your custom capabilities in test scripts. In this sample project, we are passing platform name, platform version, device name and app url _(generated earlier)_ along with other capabilities like build name and test name via capabilities object. - -The capabilities object in the sample code are defined as: - -```python title="iOS(.ipa)" -desired_caps = { -"deviceName":"iPhone 12", -"platformName":"ios", -"platformVersion":"14", -"isRealMobile":True, -#highlight-next-line -"app":"YOUR_APP_URL", -"build":"Python Vanilla iOS", -"name":"Sample Test - Python", -"network":False, -"visual":True, -"video":True -} -``` - -```python title="Android(.apk)" -desired_caps = { -"deviceName":"Galaxy S20", -"platformName":"Android", -"platformVersion":"10", -"isRealMobile":True, -#highlight-next-line -"app":"YOUR_APP_URL", -"build":"Python Vanilla Android", -"name":"Sample Test - Python", -"network":False, -"visual":True, -"video":True -} -``` - -- You must add the generated **APP_URL** to the `app` capability in the config file. -- You must set **isRealMobile** capability to `False` in the config file to run on **Virtual Devices** -- You can generate capabilities for your test requirements with the help of our inbuilt [**Capabilities Generator tool**](https://www.testmuai.com/capabilities-generator/).For more details, please refer to our guide on [**Desired Capabilities in Appium**](/support/docs/desired-capabilities-in-appium/). - -### Step 6: Execute and Monitor your Tests - -- Install the required packages from the cloned project directory: - -```bash -pip install -r requirements.txt -``` - -- Run the following command in the directory where your project has been saved to execute your build. - -```bash -python3 ios.py -``` - -```bash -python3 android.py -``` - -If you are unable to run the automation script with the above mentioned commands try **'python'** command except for **'python3'**. - - > Your test results would be displayed on the test console (or CLI if you are using terminal/cmd) and on the [TestMu AI App Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://appautomation.lambdatest.com/build). - -## Using the Appium Agent Skill with TestMu AI - -The [appium-skill](https://github.com/LambdaTest/agent-skills/tree/main/appium-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The appium-skill package includes: - -``` -appium-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing Appium Agent Skill - -Install a Appium Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/appium-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/appium-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only appium-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). - -## Reference Guides - -- [Advanced Configuration for Capabilities](/support/docs/desired-capabilities-in-appium/) -- [How to test locally hosted apps](/support/docs/testing-locally-hosted-pages/) -- [How to integrate TestMu AI with CI/CD](/support/docs/integrations-with-ci-cd-tools/) diff --git a/static/docs/appium-rokutv.md b/static/docs/appium-rokutv.md deleted file mode 100644 index 4fb85518e..000000000 --- a/static/docs/appium-rokutv.md +++ /dev/null @@ -1,228 +0,0 @@ -# Roku TV With Appium - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Tutorial To Run Your First Test On TestMu AI - -As smart TVs continue to grow in popularity, relying solely on manual testing is no longer sufficient. Automating Roku TV testing provides broader device coverage, enhances user experience by validating remote-based navigation, and eliminates repetitive manual tasks. It also helps uncover issues earlier in the development cycle, ensuring a smooth and consistent experience across different Roku TV models. -In this guide, you will learn how to configure and run your **Roku TV** automation testing scripts with **Appium Roku WebDriver** on **TestMu AI Real Device Cloud platform**. -> To enable it for your organization, please contact us via **24×7 chat support** or you can also drop a mail to **support@testmuai.com**. - -## Objective - -By the end of this topic, you will be able to: - -1. Set up an environment for testing your Apps using **Roku TV** with [**Appium Roku WebDriver**](https://github.com/headspinio/appium-roku-driver). -2. Understand and configure the core capabilities required for your Roku WebDriver test suite. -3. Explore the advanced features of TestMu AI. - -**Sample repo** - -All the code samples in this documentation can be found on **TestMu AI's Github Repository**. You can either download or clone the repository to quickly run your tests. View on GitHub - -## Prerequisites - -Before you can start performing App automation testing with Appium, you would need to follow these steps: - -- You have access to TestMu AI username and accessKey. If you have not registered yet, you can do the same by visiting our website. You will be able to access the credentials in the [TestMu AI Profile](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile) -- Install the latest Python build from the [official website](https://www.python.org/downloads/). We recommend using the latest version. -- Make sure **pip** is installed in your system. You can install **pip** from [pip documentation](https://pip.pypa.io/en/stable/installation/). - -## Supported Models ## - -| Device | OS Version | -|-----------------------------|------------------------| -| Roku Ultra | 11 | -| Roku Express | 11 | - -## Run your first test - -### 1. Upload your application -Upload your **Roku TV** application (.zip file) to the TestMu AI servers using our **REST API**. You need to provide your **Username** and **AccessKey** in the format `Username:AccessKey` in the **cURL** command for authentication. Make sure to add the path of the **appFile** in the cURL request. Here is an example cURL request to upload your app using our REST API: - - **Using App File from System:** - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "appFile=@"/Users/macuser/Downloads/roku-sample-app.zip"" -F "name="roku_app"" -`} - -- If you do not have any **.zip** file, you can run your sample tests on TestMu AI by using our sample :link: [RokuTV app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/roku-sample-app.zip). -- Response of above cURL will be a **JSON** object containing the `APP_URL` of the format - ``lt://APP123456789123456789`` and will be used in the next step. - -### 2. Clone the sample project - -1. Clone the TestMu AI’s [LT-appium-rokutv](https://github.com/LambdaTest/LT-appium-rokutv) and navigate to the code directory as shown below: - -```bash -git clone https://github.com/LambdaTest/LT-appium-rokutv -cd LT-appium-rokutv -``` - -### 3. Set up your authentication - -Make sure you have your TestMu AI credentials with you to run test automation scripts on TestMu AI. To obtain your access credentials, [purchase a plan](https://billing.lambdatest.com/billing/plans) or access the [Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://appautomation.lambdatest.com/). Then, set TestMu AI `Username` and `Access Key` in environment variables with following commands. - - {`export LT_USERNAME=${ YOUR_LAMBDATEST_USERNAME()} \\ -export LT_ACCESS_KEY=${ YOUR_LAMBDATEST_ACCESS_KEY()}`} - - {`set LT_USERNAME=${ YOUR_LAMBDATEST_USERNAME()} \` -set LT_ACCESS_KEY=${ YOUR_LAMBDATEST_ACCESS_KEY()}`} - -### 4. Write your automation script - -An automation script for the sample application available above has been provided below. Ensure to update the `APP_URL`, `username` and `accessKey` in the code scripts before running the tests. - -**Info:** -Before running the script, please make sure that the file webDriver.py from Step 2, is in the same directory as this file. - -```python title="main.py" -from appium import webdriver -import os -import time -from appium.options.android import UiAutomator2Options - -def getCaps(): -desired_caps = { -"automationName": "Roku", -"deviceName": "Roku Ultra", -"platformVersion": "11", -"platformName": "roku", -"isRealMobile": True, -"build": "Roku Testing", -"app": "APP_URL", # Enter app url here -"network": False, -"devicelog": True, -"privateCloud": True, -"visual": True, -"idleTimeout": 1800, -} -return desired_caps - -def runTest(): -if os.environ.get("LT_USERNAME") is None: -# Enter LT username below if environment variables have not been added -username = "" -else: -username = os.environ.get("LT_USERNAME") - -if os.environ.get("LT_ACCESS_KEY") is None: -# Enter LT accesskey below if environment variables have not been added -accesskey = "" -else: -accesskey = os.environ.get("LT_ACCESS_KEY") - -# grid url -gridUrl = "mobile-hub.lambdatest.com/wd/hub" - -# capabilities -desired_cap = getCaps() -url = "https://" + username + ":" + accesskey + "@" + gridUrl - -print("Initiating remote driver:") -driver = webdriver.Remote( -options=UiAutomator2Options().load_capabilities(desired_cap), -command_executor=url -) - -# run test -print(driver.session_id) - -# Simulate remote control actions -driver.execute_script("roku: pressKey", {"key": "Down"}) -driver.execute_script("roku: pressKey", {"key": "Down"}) -time.sleep(1) -driver.execute_script("roku: pressKey", {"key": "Right"}) -driver.execute_script("roku: pressKey", {"key": "Up"}) -driver.execute_script("roku: deviceInfo") -time.sleep(1) -driver.execute_script("roku: getApps") -driver.execute_script("roku: pressKey", {"key": "Right"}) - -driver.quit() - -if __name__ == "__main__": -runTest() -``` - -### Configure the test capabilities - -You can update your custom capabilities in test scripts. In this sample project, we are passing platform name, platform version, device name and app url (generated earlier) along with other capabilities like build name and test name via capabilities object. The capabilities object in the sample code are defined as: - -**Supported Capabilities:** -Please check at the end of the doc. - -**Platform:** -`"roku"` - -```python title="main.py" -caps = { -"automationName": "Roku", -"deviceName": "Roku Express", #We also support "Roku Ultra" -"platformVersion": "11", -"isRealMobile": True, -"platformName": "roku", -"build": "Roku Sample Test", -"app": "APP_URL" #Add app id here -"visual": True, -"devicelog": True -} - -``` - -**Note** - -- You must add the generated **APP_URL** to the `"app"` capability in the config file. -- You can generate capabilities for your test requirements with the help of our inbuilt **Capabilities Generator**.For more details, please refer to our guide on [Desired Capabilities in Appium](/support/docs/desired-capabilities-in-appium/). - -### 5. Execute your test case - -1. Install the required packages from the cloned project directory: - -```bash -pip install -r requirements.txt -``` - -2. Run the following command in the directory where your project has been saved to execute your build. - -```python -python main.py -``` - -If you are unable to run the automation script with the above mentioned commands try **'python3'** command except for **'python'**. - -### List of Capabilities supported by Roku: - -| KEY | VALUES | CAPABILITY DESCRIPTION | -| ---------------- | ---------------------------------------------------------------------- | ------------------------------------------------------------------------------------- | -| automationName | TYPE: STRING `roku` | Tells Appium to use the Roku Appium Driver. | -| *deviceName | TYPE: STRING `Roku Ultra` | Name of the device. | -| isRealDevice | TYPE: BOOLEAN DEFAULT: TRUE `video=TRUE` OR `video=FALSE` | It makes sure that the device being allocated is a Real device. | -| *platformName | TYPE: STRING `roku` | Name of the OS. | -| *platformVersion | TYPE: STRING `11` | OS version. | -| build | TYPE: STRING DEFAULT: Untitled `build=iOS Small Run` | You can group your tests like a job containing multiple tests. | -| *app | TYPE: STRING `app=lt://APP100201061631704657918380` | Accepts App URL returned after uploading an app on the TestMu AI servers. | -| visual | TYPE: BOOLEAN DEFAULT: FALSE `visual=TRUE` OR `visual=FALSE` | Command by command screenshots will be recorded at each test step. By default off. | -| video | TYPE: BOOLEAN DEFAULT: TRUE `video=TRUE` OR `video=FALSE` | Video recording of the complete screen. | -| devicelog | TYPE: BOOLEAN DEFAULT: FALSE `devicelog=TRUE` OR `devicelog=FALSE` | Enable Device logs. | - -Your test results would be displayed on the test console (or command-line interface if you are using terminal/cmd) and on the [TestMu AI App Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://appautomation.lambdatest.com/build). - -## Supported Commands -We utilise the Appium Roku Driver to run tests on Roku via Appium, here's a list of all the commands the driver supports from the project's [README](https://github.com/headspinio/appium-roku-driver?tab=readme-ov-file#roku-commands): - -|Command|Parameters|Description| -|-------|----------|-----------| -|`roku: pressKey`|`key`|Press the remote key whose value matches `key` (must be one of the [supported key values](https://developer.roku.com/en-ca/docs/developer-program/debugging/external-control-api.md#keypress-key-values) from the Roku documentation). As addressed in the documentation, Roku TVs also support additional keys such as `PowerOff` and `PowerOn`. | -|`roku: deviceInfo`||Get information about the Roku device| -|`roku: getApps`||Get a list of apps installed on the device. The response will be a list of objects with the following keys: `id`, `type`, `subtype`, `version`, and `name`.| -|`roku: activeApp`||Get information about the active app, in the same format as `roku: getApps`.| -|`roku: activateApp`|`appId` (required), `contentId`, `mediaType`|Launch an app with the corresponding `appId`. Optionally include `contentId` and `mediaType` information (with the same properties as described above for the `activateApp` command)| -|`roku: selectElement`|`elementId` (required) |Moves the focus on an element having locator XPath as `elementId`. If it is unable to focus on the element, the driver will respond with an error.| -|`roku: playerState`||Get the state of the media player. The data will be returned as a JSON object, corresponding to the information included in the [query/media-player ECP result](https://developer.roku.com/en-ca/docs/developer-program/dev-tools/external-control-api.md#querymedia-player-example) -|`roku: deepLink`|`contentId`, `mediaType`|As described in the [Roku dev docs](https://developer.roku.com/en-ca/docs/developer-program/discovery/implementing-deep-linking.md#using-ecp-commands-for-testing-deep-linking), you can deep link into content in the running application using a content ID and media type. For this command, `contentId` is required, and `mediaType` defaults to `movie` and must be one of the [valid media types](https://developer.roku.com/en-ca/docs/developer-program/discovery/implementing-deep-linking.md#mediatype-behavior). Note that this command acts on the currently-running app. If you want to test deep-linking into an app that is not launched, use `activateApp` instead.| -|`roku: ecpInput`|`params`|This command allows calling the `/input` ECP command directly. An arbitrary set of key/value pairs can be sent in as a JSON object. No url-encoding of the values needs to be done. For example, to represent the parameters in the ECP command `POST /input?acceleration.x=0.0&acceleration.y=0.0&acceleration.z=9.8` from the ECP docs, you would construct a `params` of `{"acceleration.x": "0.0", "acceleration.y": "0.0", "acceleration.z": "9.8"}`| - -## Additional Links - -- [Advanced Configuration for Capabilities](/support/docs/desired-capabilities-in-appium/) -- [How to test locally hosted apps](/support/docs/testing-locally-hosted-pages/) -- [How to integrate TestMu AI with CI/CD](/support/docs/integrations-with-ci-cd-tools/) diff --git a/static/docs/appium-ruby-cucumber.md b/static/docs/appium-ruby-cucumber.md deleted file mode 100644 index 4d20491cc..000000000 --- a/static/docs/appium-ruby-cucumber.md +++ /dev/null @@ -1,233 +0,0 @@ -# Appium with Cucumber for Ruby - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -In this documentation, you will learn how to trigger a automation script of **Cucumber** for application testing with **Appium** on TestMu AI, set the [**desired capabilities**](/support/docs/desired-capabilities-in-appium/) for appium testing, and other advanced features of TestMu AI. - -## Prerequisites - -- Your TestMu AI [Username and Access key](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/security). -- Install **Ruby** on your local system: - -For **macOS**, you can run a [Homebrew](https://brew.sh/) command like this: - -```bash -brew install ruby -``` - -For **Linux** or **Ubuntu**, you can run a simple apt command like below: - -```bash -sudo apt-get install ruby-full -``` - -For **Windows**, you can download from the [official website](https://rubyinstaller.org/downloads/). - -- Ensure you have Dependency manager bundler installed. If not installed, you can install with the following command: - -```bash -gem install bundler -``` - -## Try our Sample Repository - -### Step 1: Get a Sample Project -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** -All the code samples in this documentation can be found on **TestMu AI's Github Repository**. You can either download or clone the repository to quickly run your tests. View on GitHub - -### Step 2: Setup the Environment Variables - -You need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in your [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/security). Run the below mentioned commands in your terminal to setup the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -### Step 3: Upload your Application -Upload your **_iOS_** application (.ipa file) or **_android_** application (.apk or .aab file) to the TestMu AI servers using our **REST API**. You need to provide your **Username** and **AccessKey** in the format `Username:AccessKey` in the **cURL** command for authentication. - -Make sure to add the path of the **appFile** in the cURL request. Below is an example cURL request to upload your app using our REST API: - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "appFile=@"/Users/macuser/Downloads/proverbial_android.apk"" -F "name="proverbial_app""`} - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "url=:https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk" -F "name=Proverbial_App"`} - -- If you do not have any **.apk** or **.ipa** file, you can run your sample tests on TestMu AI by using our sample apps, :link: [Android app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk) or :link: [iOS app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_ios.ipa). - -- Response of above cURL will be a **JSON** object containing the `APP_URL` of the format - `lt://APP123456789123456789` and will be used in the next step - -### Step 4: Update your Automation Script - -An automation script file `first_steps.rb` for the sample application given above has been provided here. - -```ruby title="first_steps.rb" -When /^I click on buttons app$/ do - -$driver.find_element(:id, "color").click -#Click on Color Button -$driver.find_element(:id, "color").click -#Click on Geolocation Button -$driver.find_element(:id, "geoLocation").click -#wait of 5 seconds -sleep(5) -#Click on back Button -$driver.back -#Click on text Button to enter value -$driver.find_element(:id, "Text").click -#Click on Notification Button -$driver.find_element(:id, "notification").click -#Click on toast Button -$driver.find_element(:id, "toast").click -#Click on speedtest Button -$driver.find_element(:id, "speedTest").click -#Wait for 10 seconds -sleep(10) -$driver.back -$driver.find_element(:id, "Browser").click -sleep(5) -$driver.find_element(:id, "url").click -$driver.find_element(:id, "url").send_keys "https://www.testmuai.com/" -$driver.find_element(:id, "find").click -sleep(5) -$driver.back -$driver.execute_script("lambda-status=passed") - -end - -``` - -```ruby title="first_steps.rb" -When /^I click on buttons app$/ do - -$driver.find_element(:id, "com.lambdatest.proverbial:id/color").click -#Click on Color Button -$driver.find_element(:id, "com.lambdatest.proverbial:id/color").click -#Click on Geolocation Button -$driver.find_element(:id, "com.lambdatest.proverbial:id/geoLocation").click -#wait of 5 seconds -sleep(5) -#Click on back Button -$driver.back -#Click on text Button to enter value -$driver.find_element(:id, "com.lambdatest.proverbial:id/Text").click -#Click on Notification Button -$driver.find_element(:id, "com.lambdatest.proverbial:id/notification").click -#Click on toast Button -$driver.find_element(:id, "com.lambdatest.proverbial:id/toast").click -#Click on speedtest Button -$driver.find_element(:id, "com.lambdatest.proverbial:id/speedTest").click -#Wait for 10 seconds -sleep(10) -$driver.back -$driver.execute_script("lambda-status=passed") - -end -``` - -### Step 5: Configure the Test Capabilities - -You can update your custom capabilities in test scripts. In this sample project, we are passing platform name, platform version, device name and app url _(generated earlier)_ along with other capabilities like build name and test name via capabilities object. - -Ensure to update the `APP_URL`, `username` and `accesKey` in the code scripts before running the tests. The capabilities object in the sample code are defined as: - -```ruby title="first.config.yml iOS" -server: "mobile-hub.lambdatest.com" -user: "LT_USERNAME" # Add Lambdatest username here -key: "LT_ACCESS_KEY" # Add Lambdtest accessKey here - -common_caps: - -"build": "br-2" - -browser_caps: -- -"isRealMobile": "true" -"platform": "ios" -"deviceName": "iPad (2017)" -"platformVersion": "13" -"app": "APP_URL" # Add your app url here -``` - -```ruby title="first.config.yml Android" -server: "mobile-hub.lambdatest.com" -user: "LT_USERNAME" # Add Lambdatest username here -key: "LT_ACCESS_KEY" # Add Lambdtest accessKey here - -common_caps: - -"build": "br-1" - -browser_caps: -- -"isRealMobile": "true" -"platform": "android" -"deviceName": "Galaxy S21 5G" -"platformVersion": "11" -"app": "APP_URL" # Add your app url here -``` - -- You must add the generated **APP_URL** to the `app` capability in the config file. -- You must set **isRealMobile** capability to `False` in the config file to run on **Virtual Devices** -- You can generate capabilities for your test requirements with the help of our inbuilt [**Capabilities Generator tool**](https://www.testmuai.com/capabilities-generator/).For more details, please refer to our guide on [**Desired Capabilities in Appium**](/support/docs/desired-capabilities-in-appium/). - -### Step 6: Execute and Monitor your Tests - -- Navigate to the `run_first_test` directory under corresponding os based on your app. - -- Go to `config/first.config.yml` and edit the `username`, `accessKey` and add your `app url`. - -- Run the following commands to run your tests - -```ruby -bundle install -bundle exec rake first -``` - > Your test results would be displayed on the test console (or CLI if you are using terminal/cmd) and on the [TestMu AI App Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://appautomation.lambdatest.com/build). - -## Using the Cucumber Agent Skill with TestMu AI - -The [cucumber-skill](https://github.com/LambdaTest/agent-skills/tree/main/cucumber-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The cucumber-skill package includes: - -``` -cucumber-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing Cucumber Agent Skill - -Install a Cucumber Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/cucumber-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/cucumber-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only cucumber-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). - -## Reference Guides - -- [Advanced Configuration for Capabilities](/support/docs/desired-capabilities-in-appium/) -- [How to test locally hosted apps](/support/docs/testing-locally-hosted-pages/) -- [How to integrate TestMu AI with CI/CD](/support/docs/integrations-with-ci-cd-tools/) diff --git a/static/docs/appium-ruby-rspec.md b/static/docs/appium-ruby-rspec.md deleted file mode 100644 index 8ad18b971..000000000 --- a/static/docs/appium-ruby-rspec.md +++ /dev/null @@ -1,199 +0,0 @@ -# Appium with Rspec - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -In this documentation, you will learn how to trigger a automation script of **Rspec** for application testing with **Appium** on TestMu AI, set the [**desired capabilities**](/support/docs/desired-capabilities-in-appium/) for appium testing, and other advanced features of TestMu AI. - -## Prerequisites - -- Your TestMu AI [Username and Access key](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/security). -- Install **Ruby** on your local system. - -For **macOS**, you can run a [Homebrew](https://brew.sh/) command like this: - -```bash -brew install ruby -``` - -For **Linux** or **Ubuntu**, you can run a simple apt command like below: - -```bash -sudo apt-get install ruby-full -``` - -For **Windows**, you can download from the [official website](https://rubyinstaller.org/downloads/). - -- Ensure you have Dependency manager bundler installed. If not installed, you can install with the following command: - -```bash -gem install bundler -``` - -## Try our Sample Repository - -### Step 1: Get a Sample Project -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** -All the code samples in this documentation can be found on **TestMu AI's Github Repository**. You can either download or clone the repository to quickly run your tests. View on GitHub - -### Step 2: Setup the Environment Variables - -You need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in your [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/security). Run the below mentioned commands in your terminal to setup the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -### Step 3: Upload your Application -Upload your **_iOS_** application (.ipa file) or **_android_** application (.apk or .aab file) to the TestMu AI servers using our **REST API**. You need to provide your **Username** and **AccessKey** in the format `Username:AccessKey` in the **cURL** command for authentication. - -Make sure to add the path of the **appFile** in the cURL request. Below is an example cURL request to upload your app using our REST API: - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "appFile=@"/Users/macuser/Downloads/proverbial_android.apk"" -F "name="proverbial_app""`} - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "url=:https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk" -F "name=Proverbial_App"`} - -- If you do not have any **.apk** or **.ipa** file, you can run your sample tests on TestMu AI by using our sample apps, :link: [Android app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk) or :link: [iOS app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_ios.ipa). - -- Response of above cURL will be a **JSON** object containing the `APP_URL` of the format - `lt://APP123456789123456789` and will be used in the next step - -### Step 4: Update your Automation Script - -An automation script for the sample application given above has been provided here. Ensure to update the `APP_URL`, `username` and `accessKey` in the code scripts before running the tests. - -```ruby title="single.config.yml" -server: "mobile-hub.lambdatest.com" -user: "LT_USERNAME" #Add your LambdaTest username here -key: "LT_ACCESS_KEY" #Add your LambdaTest accessKey here - -common_caps: -"build": "RSpec Selenium Demo" - -browser_caps: -- -"isRealMobile": "true" -"platform": "Android" -"deviceName": "Galaxy A31" -"platformVersion": "10" -"app": "APP_URL" #Add the app url here -``` - -```ruby title="parallel.config.yml" -server: "mobile-hub.lambdatest.com" -user: "LT_USERNAME" #Add your LambdaTest username here -key: "LT_ACCESS_KEY" #Add your LambdaTest accessKey here - -common_caps: -"build": "RSpec Parallel" -"visual": false -"network": false -"console": false - -app_caps: -- -"isRealMobile": "true" -"platform": "Android" -"deviceName": "Galaxy A31" -"platformVersion": "10" -"app": "APP_URL" #Add the app url here -- -"isRealMobile": "true" -"platform": "Android" -"deviceName": "Galaxy A32" -"platformVersion": "11" -"app": "APP_URL" #Add the app url here -- -"isRealMobile": "true" -"platform": "iOS" -"deviceName": "iPhone 13 Pro" -"platformVersion": "15.0" -"app": "APP_URL" #Add the app url here -``` - -- You must set **isRealMobile** capability to `False` in the config file to run on **Virtual Devices** -- You can generate capabilities for your test requirements with the help of our inbuilt [**Capabilities Generator tool**](https://www.testmuai.com/capabilities-generator/).For more details, please refer to our guide on [**Desired Capabilities in Appium**](/support/docs/desired-capabilities-in-appium/). - -### Step 5: Configure the Test Capabilities - -You can update your custom capabilities in test scripts `lambdatest.rb`. In this sample project, we are passing platform name, platform version, device name and app url _(generated earlier)_ along with other capabilities like build name and test name via capabilities object. - -The capabilities object in the sample code are defined as: - -```ruby -caps={ -"LT:Options" => { -"build" => "Ruby RSpec", -"name" => "Sample Test", -"platformName" => platform, -"isRealMobile" => isRealMobile, -"deviceName" => deviceName, -"platformVersion" => platformVersion, -"app" => app, -"w3c" => true -}}, -``` - -- You must add the generated **APP_URL** to the `app` capability in the config file. -- You must set **isRealMobile** capability to `False` in the config file to run on **Virtual Devices** -- You can generate capabilities for your test requirements with the help of our inbuilt [**Capabilities Generator tool**](https://www.testmuai.com/capabilities-generator/).For more details, please refer to our guide on [**Desired Capabilities in Appium**](/support/docs/desired-capabilities-in-appium/). - -### Step 6: Execute and Monitor your Tests - -- Run the following command to make sure that all the dependencies required for the test are installed. -```bash -bundle install -``` - -2. Execute the following command to run single test on TestMu AI platform: -```bash -bundle exec rake single -``` - -> In order to run parallel tests, run `bundle exec rake parallel`. Your test results would be displayed on the test console (or CLI if you are using terminal/cmd) and on the [TestMu AI App Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://appautomation.lambdatest.com/build). - -## Using the RSpec Agent Skill with TestMu AI - -The [rspec-skill](https://github.com/LambdaTest/agent-skills/tree/main/rspec-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The rspec-skill package includes: - -``` -rspec-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing RSpec Agent Skill - -Install a RSpec Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/rspec-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/rspec-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only rspec-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). - -## Reference Guides - -- [Advanced Configuration for Capabilities](/support/docs/desired-capabilities-in-appium/) -- [How to test locally hosted apps](/support/docs/testing-locally-hosted-pages/) -- [How to integrate TestMu AI with CI/CD](/support/docs/integrations-with-ci-cd-tools/) diff --git a/static/docs/appium-ruby.md b/static/docs/appium-ruby.md deleted file mode 100644 index bd7fcd4f9..000000000 --- a/static/docs/appium-ruby.md +++ /dev/null @@ -1,313 +0,0 @@ -# Appium with Ruby - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -In this documentation, you will learn how to trigger a automation script of **Ruby** for application testing with **Appium** on TestMu AI, set the [**desired capabilities**](/support/docs/desired-capabilities-in-appium/) for appium testing, and other advanced features of TestMu AI. - -## Prerequisites - -- Your TestMu AI [Username and Access key](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/security). -- Install **Ruby** on your local system. Follow these instructions to install on different operating systems. - -For **macOS**, you can run a [Homebrew](https://brew.sh/) command like this: - -```bash -brew install ruby -``` - -For **Linux** or **Ubuntu**, you can run a simple apt command like below: - -```bash -sudo apt-get install ruby-full -``` - -For **Windows**, you can download from the [official website](https://rubyinstaller.org/downloads/). - -## Try our Sample Repository - -### Step 1: Get a Sample Project -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** -All the code samples in this documentation can be found on **TestMu AI's Github Repository**. You can either download or clone the repository to quickly run your tests. View on GitHub - -### Step 2: Setup the Environment Variables - -You need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in your [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/security). Run the below mentioned commands in your terminal to setup the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -### Step 3: Upload your Application -Upload your **_iOS_** application (.ipa file) or **_android_** application (.apk or .aab file) to the TestMu AI servers using our **REST API**. You need to provide your **Username** and **AccessKey** in the format `Username:AccessKey` in the **cURL** command for authentication. - -Make sure to add the path of the **appFile** in the cURL request. Below is an example cURL request to upload your app using our REST API: - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "appFile=@"/Users/macuser/Downloads/proverbial_android.apk"" -F "name="proverbial_app""`} - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "url=:https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk" -F "name=Proverbial_App"`} - -- If you do not have any **.apk** or **.ipa** file, you can run your sample tests on TestMu AI by using our sample apps, :link: [Android app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk) or :link: [iOS app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_ios.ipa). - -- Response of above cURL will be a **JSON** object containing the `APP_URL` of the format - `lt://APP123456789123456789` and will be used in the next step - -### Step 4: Update your Automation Script - -An automation script for the sample application given above has been provided here. Ensure to update the `APP_URL`, `username` and `accessKey` in the code scripts before running the tests. - -```ruby title="ios.py" -require 'rubygems' -require 'appium_lib' - -#highlight-next-line -username= "username" #Add username here -#highlight-next-line -accessToken= "accesKey" #Add accessKey here - -caps = { -"LT:Options" => { -:deviceName => "iPhone 13 Pro", -:platformName => "iOS", -:platformVersion => "15", -:build => "Ruby Vanilla - iOS", -:name => "Ruby iOS Test", -:isRealMobile => true, -#highlight-next-line -:app => "YOUR_APP_URL", #Enter the App URL here -:w3c => true, -} } - -appium_driver = Appium::Driver.new({ -'caps' => caps, -'appium_lib' => { -:server_url => "http://"+username+":"+accessToken+"@mobile-hub.lambdatest.com/wd/hub" -}}, true) - -driver = appium_driver.start_driver - -wait = Selenium::WebDriver::Wait.new(:timeout => 30) - -el1 = driver.find_element(:id, "color") -el1.click -el2 = driver.find_element(:id, "geoLocation") -el2.click -sleep(5) -driver.back -el3 = driver.find_element(:id, "Text") -el3.click -el4 = driver.find_element(:id, "notification") -el4.click -el5 = driver.find_element(:id, "Toast") -el5.click -el6 = driver.find_element(:id, "speedTest") -el6.click -sleep(10) -driver.back - -puts "Found results - Test Passed" - -driver.quit -``` - -- You must set **isRealMobile** capability to `False` in the config file to run on **Virtual Devices** - -```ruby title="android.py" -require 'rubygems' -require 'appium_lib' - -#highlight-next-line -username= "username" #Add username here -#highlight-next-line -accessToken= "accesKey" #Add accessKey here - -caps = { -"LT:Options" => { -:deviceName => "OnePlus 7", -:platformName => "Android", -:platformVersion => "9", -:build => "Ruby Vanilla - Android", -:name => "Ruby Android Test", -:isRealMobile => true, -#highlight-next-line -:app => "YOUR_APP_URL", #Enter the App URL here -:w3c => true, -} } - -appium_driver = Appium::Driver.new({ -'caps' => caps, -'appium_lib' => { -:server_url => "http://"+username+":"+accessToken+"@mobile-hub.lambdatest.com/wd/hub" -}}, true) - -driver = appium_driver.start_driver - -wait = Selenium::WebDriver::Wait.new(:timeout => 30) -el1 = driver.find_element(:id, "com.lambdatest.proverbial:id/color") -el1.click -el2 = driver.find_element(:id, "com.lambdatest.proverbial:id/geoLocation") -el2.click -sleep(5) -driver.back -el3 = driver.find_element(:id, "com.lambdatest.proverbial:id/Text") -el3.click -el4 = driver.find_element(:id, "com.lambdatest.proverbial:id/notification") -el4.click -el5 = driver.find_element(:id, "com.lambdatest.proverbial:id/toast") -el5.click -el6 = driver.find_element(:id, "com.lambdatest.proverbial:id/speedTest") -el6.click -sleep(10) -driver.back -puts "Found results - Test Passed" - -driver.quit -``` - -### Step 5: Configure the Test Capabilities - -You can update your custom capabilities in test scripts. In this sample project, we are passing platform name, platform version, device name and app url _(generated earlier)_ along with other capabilities like build name and test name via capabilities object. - -The capabilities object in the sample code are defined as: - -```ruby title="iOS(.ipa)" -caps = { -"LT:Options" => { -:deviceName => "iPhone 13 Pro", -:platformName => "iOS", -:platformVersion => "15", -:build => "Ruby Vanilla - iOS", -:name => "Ruby iOS Test", -:isRealMobile => true, -#highlight-next-line -:app => "YOUR_APP_URL", #Enter the App (.ipa) URL here -:w3c => true, -} } -``` - -```ruby title="Android(.apk)" -caps = { -"LT:Options" => { -:deviceName => "OnePlus 7", -:platformName => "Android", -:platformVersion => "9", -:build => "Ruby Vanilla - Android", -:name => "Ruby Android Test", -:isRealMobile => true, -#highlight-next-line -:app => "YOUR_APP_URL", #Enter the App (.apk) URL here -:w3c => true, -} } -``` - -> In case you're using capabilities generated using the JSON Wire Protocol, then it's recommended to use ```appium_lib``` version ```10.6.0```. Run ```gem install appium_lib -v 10.6.0``` command to install it. - -- You must add the generated **APP_URL** to the `app` capability in the config file. -- You must set **isRealMobile** capability to `False` in the config file to run on **Virtual Devices** -- You can generate capabilities for your test requirements with the help of our inbuilt [**Capabilities Generator tool**](https://www.testmuai.com/capabilities-generator/).For more details, please refer to our guide on [**Desired Capabilities in Appium**](/support/docs/desired-capabilities-in-appium/). - -### Step 6: Execute and Monitor your Tests - -If you are using an **iOS** app, the cURL command will generate an app URL for the corresponding iOS app and install the same for running the tests. You can either use our sample :link: [iOS app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_ios.ipa) or upload your own app as discussed earlier. - -- Navigate to the corresponding directory based on your app. - -```bash -cd ios -``` - -- Execute the following command to run your test on TestMu AI platform: - -```bash -ruby ios-sample.rb -``` - -If you are using an **android** app, the cURL command will generate an app URL for the corresponding Android app and install the same for running the tests. You can either use our sample :link: [Android app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk) or upload your own app as discussed earlier. - -- Navigate to the corresponding directory based on your app. - -```bash -cd android -``` - -- Execute the following command to run your test on TestMu AI platform: - -```bash -ruby android-sample.rb -``` - - For Windows Users - -In case of Windows, if you get any error message. Please try this method: - -- Navigate to the corresponding directory based on your app. - -```bash -cd android -``` - -- Refresh the gem bundles through given command -```bash -gem uninstall -aIx -``` - -- Re-install the gems required - -```bash -gem install appium_lib -v 10.6.0 -gem install ffi - -``` -- Now try running the corresponding automation script for your app. - -```bash -ruby android-sample.rb -``` - -> Your test results would be displayed on the test console (or CLI if you are using terminal/cmd) and on the [TestMu AI App Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://appautomation.lambdatest.com/build). - -## Using the Appium Agent Skill with TestMu AI - -The [appium-skill](https://github.com/LambdaTest/agent-skills/tree/main/appium-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The appium-skill package includes: - -``` -appium-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing Appium Agent Skill - -Install a Appium Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/appium-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/appium-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only appium-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). - -## Reference Guides - -- [Advanced Configuration for Capabilities](/support/docs/desired-capabilities-in-appium/) -- [How to test locally hosted apps](/support/docs/testing-locally-hosted-pages/) -- [How to integrate TestMu AI with CI/CD](/support/docs/integrations-with-ci-cd-tools/) diff --git a/static/docs/appium-terminal-logs.md b/static/docs/appium-terminal-logs.md deleted file mode 100644 index 1d3db8d21..000000000 --- a/static/docs/appium-terminal-logs.md +++ /dev/null @@ -1,75 +0,0 @@ -# Terminal Logs for Appium Tests - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI lets you attach **your own terminal logs** (Appium server logs, test-runner output, or CI logs) to any app automation session through a single REST API. Once uploaded, you can view or download them directly from the TestMu AI Automation dashboard under **Logs › Terminal**. - -This keeps your own diagnostic logs next to the session they belong to, so you can correlate them with the session's native logs (Appium, device, network, crash) in one place, without jumping between your CI system, local files, and the dashboard. - -## How it works - -- **One log per session** A session holds at most one terminal log at a time. -- **Latest upload** Every upload replaces the existing log, regardless of the filename. -- **No special capability is required.** Any valid session can receive a terminal log. There is no opt-in flag to set in your test capabilities. -- **Retention** follows the session's normal artifact retention. The log is kept and expires on the same schedule as the session's other logs. - -## Upload a terminal log - -Use the **upload** endpoint to attach a log file to a session. The file is sent as a multipart form field named `file`. Uploading again replaces the previous file. - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://mobile-api.lambdatest.com/mobile-automation/api/v1/sessions//log/terminallog" --form "file=@/path/to/appium.log" -`} - -Replace `` with your actual session ID and `/path/to/appium.log` with the path to your log file. - -On success, the API returns: - -```json -{ "status": "success", "message": "File has been uploaded successfully!" } -``` - -## View terminal logs in the dashboard - -Open the session on the TestMu AI Automation dashboard. The **Terminal** sub-tab is always available under **Logs** (next to **Crash**). Once you upload a log file, it starts showing up in this tab, where you can view the file and read its contents in the browser. - -The **Terminal** tab is always present. It stays empty until a terminal log is uploaded for the session. - -{/* */} - -## Download a terminal log - -To download the uploaded log, add the `download=true` query parameter to the view endpoint: - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X GET "https://mobile-api.lambdatest.com/mobile-automation/api/v1/sessions//log/terminallog?download=true" --output terminal-logs.log -`} - -The downloaded file is always named `terminal-logs.`, where `` is your original file extension. The original base filename is not retained. - -## Delete a terminal log - -To remove the attached log from a session, use the **delete** endpoint: - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X DELETE "https://mobile-api.lambdatest.com/mobile-automation/api/v1/sessions//log/terminallog" -`} - -## Supported formats and limits - -| | Value | -| ------------------- | ---------------------------------------------------------------------------------- | -| Maximum file size | 5 MB | -| Allowed file types | `.txt`, `.json`, `.xml`, `.log`, `.csv`, `.html`, `.yaml`, `.yml`, `.har`, `.gz` | -| Files per session | 1 (each upload replaces the previous file) | -| Empty (0-byte) file | Not allowed | - -## FAQ - -**Can I upload more than one file to a session?** -No. A session holds one terminal log at a time, and each new upload replaces the previous one. - -**How long are terminal logs kept?** -They follow the session's normal artifact retention, on the same schedule as the session's other logs. There is no separate retention setting. - -**Do I need a special capability to use this?** -No. Any valid app automation session can receive a terminal log; there is no opt-in flag to set in your test capabilities. - -If you still have any questions for us, please feel free to let us know via our **24X7 Chat Portal** or mail us to support@testmuai.com diff --git a/static/docs/appium-testmu-hooks.md b/static/docs/appium-testmu-hooks.md deleted file mode 100644 index bfba3f516..000000000 --- a/static/docs/appium-testmu-hooks.md +++ /dev/null @@ -1,134 +0,0 @@ -# Lambda Hooks For Appium Automation - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This document will help you provide lambdahooks which can be used to improve your test scripts to easily debug your test cases for **App Automation on Real Devices** on **TestMu AI Cloud**. - -## Adding custom status & remark - -To add custom status & remark, just add the code snippet using the ```JavascriptExecutor```. - -```JavascriptExecutor``` accepts two arguments as shown below: - -Arguments | Example -------------- | ----------------------------------------------------------------------------------------- -```status``` | ```failed``` OR ```passed``` -```remark``` | Any remark can be added here limited to 255 characters. - -```java -((JavascriptExecutor) driver).executeScript("lambda-hook: {\"action\": \"setTestStatus\",\"arguments\": {\"status\":\"failed\", \"remark\":\"This is a sample remark for failed test \"}} "); -``` - -Here is a sample automation script in Java for the sample status & remark. Ensure to update the ```app_url```, ```username``` & ```accesskey``` in the below code. - -```java title="android.java" -import io.appium.java_client.AppiumDriver; -import io.appium.java_client.MobileBy; -import io.appium.java_client.MobileElement; -import org.openqa.selenium.JavascriptExecutor; -import org.openqa.selenium.remote.DesiredCapabilities; -import java.net.MalformedURLException; -import java.net.URL; -public class vanilla_android { -private static AppiumDriver driver; -public static void main(String args[]) throws MalformedURLException, InterruptedException { - -try { -DesiredCapabilities capabilities = new DesiredCapabilities(); - -capabilities.setCapability("deviceName", "Galaxy S20"); -capabilities.setCapability("platformVersion", "11"); -capabilities.setCapability("platformName", "Android"); -capabilities.setCapability("isRealMobile", true); -capabilities.setCapability("app", "APP_ID"); //Enter your app url -capabilities.setCapability("build", "Java Vanilla - Android"); -capabilities.setCapability("name", "Sample Test Java"); - -//Enter your Username & Accesskey here: -driver = new AppiumDriver(new URL("https://" +userName + ":" + accessKey + "@mobile-hub.lambdatest.com/wd/hub"), capabilities); - -MobileElement color = (MobileElement) driver.findElement(MobileBy.id("com.lambdatest.proverbial:id/color")); -color.click(); - -//Javascript Executor for marking the status and custom remark. -//highlight-next-line -((JavascriptExecutor) driver).executeScript("lambda-hook: {\"action\": \"setTestStatus\",\"arguments\": {\"status\":\"failed\", \"remark\":\"This is a sample remark for failed test \"}} "); -} -// The driver.quit statement is required, otherwise the test continues to execute, leading to a timeout. -driver.quit(); -} -} -``` - -Once you have added the code snippet, the status and remark will be visible on the [TestMu AI App Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://appautomation.lambdatest.com/build) as shown below: - -## Update Test Name - -To update the name of the test, just add the code snippet using the ```JavascriptExecutor```. - -Arguments | Example -------------- | ----------------------------------------------------------------------------------------- -```lambda-name``` | For changing the Test Name. - -```bash -((JavascriptExecutor) driver).executeScript("lambda-name=TestName"); -``` - -## Differentiating Test Cases in Single Session - -TestMu AI has introduced a LambdaHook to help automation engineers differentiate between multiple test cases within a single Appium session. This feature aims to improve debugging capabilities, making it easier to identify which test cases may be causing errors or taking longer to execute. - -### Adding LambdaHooks - -You can use LambdaHooks to start and end a test case within a single Appium session. - -#### Test Case Start - -To start a test case, use the `lambda-testCase-start` hook: - -```java -// To start a test case -((JavascriptExecutor) driver).executeScript("lambda-testCase-start=find Name"); -``` - -#### Test Case End - -To end a test case, use the `lambda-testCase-end` hook: - -```java -// To end a test case -((JavascriptExecutor) driver).executeScript("lambda-testCase-end=find Name"); -``` - -### Labeling and Filtering Command Logs - -Any command logs executed between `lambda-testCase-start` and `lambda-testCase-end` will be labeled with the name you provide. The name should not exceed `255` characters. - -You can also filter the command logs based on these labels.A screenshot has been provided below, demonstrating the use of labels and how to filter command logs based on them. - -## Command Annotations - -Command Annotations allow you to add metadata and debugging information to your test scripts. They work similarly to LambdaHooks by providing structured logs on the **TestMu AI Automation Dashboard**, making it easier to track test execution, debug failures, and navigate specific test sections. - -### Using Command Annotations - -#### Annotation Start -```java -driver.execute_script("lambdatest_executor: {\"action\": \"stepcontext\", \"arguments\": {\"data\": \"youtube\", \"level\": \"debug\"}}"); -``` - -**Arguments:** - -| Argument | Description | -|----------|-------------| -| `data` | String value representing the information you want to log. | -| `level` | Log severity level. Acceptable values: `info` (default), `debug`, `warn`, `error`. | - -#### Annotation End -To close the executor at the end of your script, pass `data` as an empty string: - -```java -driver.execute_script("lambdatest_executor: {\"action\": \"stepcontext\", \"arguments\": {\"data\": \"\"}}"); -``` - -**Mutual exclusivity:** Test case and Command Annotations cannot be used together in the same script. You can only use **one** of them per session. diff --git a/static/docs/appium-uploading-retrieving-files.md b/static/docs/appium-uploading-retrieving-files.md deleted file mode 100644 index f1a3ed166..000000000 --- a/static/docs/appium-uploading-retrieving-files.md +++ /dev/null @@ -1,95 +0,0 @@ -# Uploading/Retrieving files on Real Devices - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Whenever you are testing your application on **TestMu AI Real Devices**, you might have felt the need for some set of sample data that can help you achieve required output. Thus, to fulfill this need TestMu AI provides you with sample data like **pre-loaded contacts, media files etc.** - -You can also upload files to the real device for testing and can also save the files produced for future reference. Let's see how. - -## Transferring the files - -### How to copy a file or folder to the real device? - -We recognise that you need some files or folders on the device to test various things, such as uploading something to your app etc. Therefore, TestMu AI provides you with a set of pre-loaded media files on the real device so that you can test your apps. But, in case you need to **upload** something specific, you can upload and use it wherever required. For this functionality, we generally use Appium’s [push file](https://appium.readthedocs.io/en/latest/en/commands/device/files/push-file/) command to copy files into any folder. - -### How to retrieve a file or directory from the real device? - -If your app has a feature of **saving a file** to the real device and you want to **retrieve** that file for later, you can do it by running the Appium's [pull command](https://appium.readthedocs.io/en/stable/en/commands/device/files/pull-file/) - -Refer to the code snippets given below for push and pull commands in different languages. - -While using pull and push features of Appium, please copy the files and retrieve them from the given folders, `/sdcard/Downloads/`, `/sdcard/Pictures` and `/sdcard/Android/data/`, on the TestMu AI Real Device. - - {` # Push a file -dest_path = '/sdcard/Downloads/file.txt' -driver.push_file(dest_path, 'Hello World'.encode("utf-8")) -# Pull a file -file_base64 = driver.pull_file(dest_path) -`} - - {`// Push a file -let data = new Buffer("Hello LambdaTest").toString('base64'); -driver.pushFile('/sdcard/Downloads/sample.txt', data); -// Pull file -let data = driver.pullFile('/sdcard/Downloads/sample.txt');`} - - {`// Push a file -$driver->pushFile('/sdcard/Downloads/sample.txt', 'QXJlIHlvdXIgYmVlcnMgb2theT8='); -// Pull file -$data = $driver->pullFile('/sdcard/Downloads/sample.txt');`} - - {`// Push a file -driver.pushFile("/sdcard/Downloads/sample.jpg", new File("/Users/macuser/Desktop/sample.jpg")); -// Pull file -byte[] fileBase64 = driver.pullFile("/sdcard/Downloads/sample.jpg");`} - - {`# Push a file -driver.push_file('/sdcard/Downloads/sample.jpg', File.read('/Users/macuser/Desktop/sample.jpg')) -# Pull file -pull_file('/sdcard/Downloads/sample.jpg')`} - - {`// Push a file -driver.PushFile("/sdcard/Downloads/sample.jpg", new FileInfo("/Users/macuser/Desktop/sample.jpg")) -// Pull file -byte[] fileBase64 = driver.PullFile("/sdcard/Downloads/sample.jpg");`} - -While using pull and push feature of Appium in iOS, please adhere to the following points: - -- Copy and save the files in the app's **Documents folder** on the real device. -- Make sure that the app has `UIFileSharingEnabled` key set to true in the **Info.plist** -- Follow this specific format for the destination path while using push feature: `@:Documents/.png` - - {` # Push file -driver.push_file('@com.lambdatest.proverbial:Documents/sample-image1.png', source_path='/Users/macuser/Desktop/sample-image1.png') -# Push a text file -dest_path = '@com.lambdatest.proverbial:Documents/file.txt' -driver.push_file(dest_path, 'Hello LambdaTest'.encode("utf-8")) -# Pull file -file_base64 = driver.pull_file(dest_path) -`} - - {`// Push file -let data = new Buffer("Hello LambdaTest").toString('base64'); -driver.pushFile('@com.lambdatest.proverbial:Documents/sample.txt', data); -// Pull file -let data = driver.pullFile('@com.lambdatest.proverbial:Documents/sample.txt');`} - - {` // Push file - $driver->pushFile('@com.lambdatest.proverbial:Documents/foo.bar', 'QXJlIHlvdXIgYmVlcnMgb2theT8='); -// Pull file -$data = $driver->pullFile('@com.lambdatest.proverbial:Documents/foo.bar');`} - - {`// Push file -driver.pushFile("@com.lambdatest.proverbial:Documents/sample-image.jpg", new File("/Users/macuser/Desktop/sample-image.jpg")); -// Pull file -byte[] fileBase64 = driver.pullFile("@com.lambdatest.proverbial:Documents/sample-image.jpg");`} - - {`# Push file -driver.push_file('@com.lambdatest.proverbial:Documents/sample-image.jpg', File.read('/Users/macuser/Desktop/sample-image.jpg')) -# Pull file -pull_file('@com.lambdatest.proverbial:Documents/sample-image.jpg')`} - - {`// Push file -driver.PushFile("@com.lambdatest.proverbial:Documents/sample-image.jpg", new FileInfo("/Users/macuser/Desktop/sample-image.jpg")) -// Pull file -byte[] fileBase64 = driver.PullFile("@com.lambdatest.proverbial:Documents/sample-image.jpg");`} diff --git a/static/docs/apple-pay-auto.md b/static/docs/apple-pay-auto.md deleted file mode 100644 index 7efd64aae..000000000 --- a/static/docs/apple-pay-auto.md +++ /dev/null @@ -1,186 +0,0 @@ -# Apple Pay Automation on Real Devices - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -**Apple Pay** is Apple’s secure digital wallet and payment system that enables purchases, one-click checkouts, and adds an extra layer of protection for financial transactions. In real-world apps and websites, Apple Pay is widely used for simplifying checkout experiences, reducing friction, and improving user trust. - -For **testing payment flows**, validating Apple Pay becomes critical. Automation of Apple Pay ensures you can reliably test end-to-end purchase scenarios, confirm that payment sheets open correctly, details are pre-filled, and transactions are processed securely, without relying on production cards. - -With TestMu AI, you can automate Apple Pay flows on **real iOS devices**. From provisioning Wallet with sandboxed test cards, to injecting payment details, confirming payments, and entering the passcode, TestMu AI provides a seamless way to run and validate Apple Pay scenarios as part of your automation suite. - -> To enable it for your organization, please contact us via **24×7 chat support** or you can also drop a mail to **support@testmuai.com**. - -**Automating with Playwright?** -This guide covers the **Appium** flow. Apple Pay is also supported for **Playwright** tests on real iOS devices (Safari, Node.js) over the CDP endpoint. See [Apple Pay Automation in the Playwright iOS guide](/support/docs/playwright-ios-device/#apple-pay-automation). - -## Use Cases - -- **Checkout Validation**: Ensure your in-app or web checkout flow works seamlessly with Apple Pay as a payment method. -- **End-to-End Payment Testing**: Validate that Apple Pay sheets open correctly, test cards are provisioned, and transactions are processed end-to-end. -- **CI/CD Integration**: Run Apple Pay scenarios as part of continuous integration pipelines to catch payment flow issues early. -- **Cross-Network Stability**: Test across Visa, MasterCard, and other networks to ensure consistent user experiences. -- **Security & Compliance**: Confirm that sensitive flows like passcode entry and secure card handling behave as expected during automation. - -## Supported Devices -Currently, the Device Passcode feature in App Automation is enabled on the following iOS real devices: - -| iOS Device | iOS Version | -|--------------|-------------| -| iPhone 16 | 18 | -| iPhone 15 | 17 | -| iPhone 14 | 16 | -| iPhone 13 | 15 | - -> We are actively working on expanding coverage to **all supported iOS devices** on TestMu AI Real Device Cloud. - -## Capabilities - -| Capability | Type | Default | Required / Optional | Description | -|----------------------------|------------|---------|---------------------|---------------------------------------------------------------------------------------------| -| **applePay** | Boolean | true | Mandatory | Enables Apple Pay provisioning including Wallet, Sandbox card, AssistiveTouch, and Passcode on supported real iOS devices. | -| **applePayCardType** | Array | None | Optional | Specify preferred payment networks. Currently, you can choose from four supported cards: `["amex", "visa", "master", "discover"]`.| - -The `applePayCardType` array follows a **priority order**. The order you provide determines which card type is used first. For example, `["visa", "master"]` will prioritize Visa over MasterCard. - -## Apple Pay Workflow in Automation Test - -### Step 1: Upload Your App to TestMu AI - -1. **Uploading Your App** – Follow the detailed steps in our [Upload Your Application](/support/docs/application-setup-via-api/) guide. -2. Once uploaded, **note the App ID** returned by the API or dashboard. -3. Use this **App ID** in the `"app"` capability in your automation script. - -### Step 2: Add Desired Capabilities - -To enable Apple Pay automation, include the following capability in your automation session as highlighted in the Capabilities: - -{`desired_caps = { - "deviceName": "iPhone 16", - "platformName": "iOS", - "platformVersion": "18", - "isRealMobile": True, - "app": "YOUR_APP_ID", - "build": "Sample Build", - "name": "Sample Test", - // highlight-next-line - "applePay": true, - // highlight-next-line - "applePayCardType": ["visa", "master"], -}`} - -- You must add the generated **APP_URL** to the `app` capability in the config file. -- You can generate capabilities for your test requirements with the help of our inbuilt [**Capabilities Generator tool**](https://www.testmuai.com/capabilities-generator/). For more details, please refer to our guide on [**Desired Capabilities in Appium**](/support/docs/desired-capabilities-in-appium/). - -### Step 3: Update Shipping, Billing, and Contact Details (Optional) - -Before confirming the Apple Pay payment, you can optionally update the shipping details, billing details, and contact information using the `lambda-applepay-details` hook. This allows you to customize the payment information dynamically during your automation test. - -{`driver.execute_script("lambda-applepay-details", { - "shippingDetails": { - "firstName": "John", - "lastName": "Doe", - "street": "221B Baker Street", - "city": "London", - "postalCode": "NW1 6XE", - "state": "London", - "country": "UK" - }, - "billingDetails": { - "firstName": "John", - "lastName": "Doe", - "street": "221B Baker Street", - "city": "London", - "postalCode": "NW1 6XE", - "state": "London", - "country": "UK" - }, - "contact": { - "email": "john.doe@example.com", - "phone": "+441234567890" - } -})`} - -{`Map shippingDetails = new HashMap<>(); -shippingDetails.put("firstName", "John"); -shippingDetails.put("lastName", "Doe"); -shippingDetails.put("street", "221B Baker Street"); -shippingDetails.put("city", "London"); -shippingDetails.put("postalCode", "NW1 6XE"); -shippingDetails.put("state", "London"); -shippingDetails.put("country", "UK"); - -Map billingDetails = new HashMap<>(); -billingDetails.put("firstName", "John"); -billingDetails.put("lastName", "Doe"); -billingDetails.put("street", "221B Baker Street"); -billingDetails.put("city", "London"); -billingDetails.put("postalCode", "NW1 6XE"); -billingDetails.put("state", "London"); -billingDetails.put("country", "UK"); - -Map contact = new HashMap<>(); -contact.put("email", "john.doe@example.com"); -contact.put("phone", "+441234567890"); - -Map applePayDetails = new HashMap<>(); -applePayDetails.put("shippingDetails", shippingDetails); -applePayDetails.put("billingDetails", billingDetails); -applePayDetails.put("contact", contact); - -driver.executeScript("lambda-applepay-details", applePayDetails);`} - -{`await driver.executeScript("lambda-applepay-details", { - shippingDetails: { - firstName: "John", - lastName: "Doe", - street: "221B Baker Street", - city: "London", - postalCode: "NW1 6XE", - state: "London", - country: "UK" - }, - billingDetails: { - firstName: "John", - lastName: "Doe", - street: "221B Baker Street", - city: "London", - postalCode: "NW1 6XE", - state: "London", - country: "UK" - }, - contact: { - email: "john.doe@example.com", - phone: "+441234567890" - } -});`} - -#### Hook Parameters - -| Parameter | Type | Description | -|-----------|------|-------------| -| **shippingDetails** | Object | Shipping address information including firstName, lastName, street, city, postalCode, state, and country. | -| **billingDetails** | Object | Billing address information with the same fields as shippingDetails. | -| **contact** | Object | Contact information including email and phone number. | - -All parameters are optional. You can provide only the details you need to update. For example, you can update only the billing details without providing shipping or contact information. - -### Step 4: Confirm Apple Pay Payment - -- To confirm Apple Pay payment at the payment step, add a hook to confirm and complete the payment. -- This can be done using TestMu AI hooks. A sample script is provided below to trigger confirmation of the Apple Pay payment step: - -```python -driver.execute_script( -'lambda_executor: {"action": "applePay", "arguments": {"confirmPayment": "true"}}' -) -``` - -### Step 5: Enter Passcode to Complete Payment - -- After confirming the Apple Pay payment, the device will prompt for the passcode to securely authorize the transaction. Your automation script must handle this prompt by entering the passcode using Appium's keyboard input methods to simulate the user securely confirming the payment. - -```Python -driver.find_element(By.XPATH,"//*[@name=\"Passcode field\"]").send_keys("123456") -``` - - The default passcode is `123456` unless configured differently on the device. This step is essential to simulate secure confirmation and complete the Apple Pay transaction during automation. diff --git a/static/docs/apple-pay-manual.md b/static/docs/apple-pay-manual.md deleted file mode 100644 index 9849c0e69..000000000 --- a/static/docs/apple-pay-manual.md +++ /dev/null @@ -1,59 +0,0 @@ -# Apple Pay Testing on Real Devices - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -**Apple Pay** is Apple’s secure digital wallet and payment system that enables in-app and web purchases with an extra layer of protection. Testing of Apple Pay is essential to validate the user experience, including wallet selection, passcode entry, and transaction confirmation. - -With TestMu AI App Live, you can test Apple Pay flows on **Real iOS Devices**. This guide walks you through enabling Apple Pay, configuring wallets, running the application, and completing transactions manually. - -> To enable it for your organization, please contact us via **24×7 chat support** or you can also drop a mail to **support@testmuai.com**. - -## Use Cases - -- **Manual Checkout Testing**: Ensure Apple Pay works correctly across different devices. -- **Wallet Verification**: Confirm users can select the correct card type (Visa, MasterCard, etc.). -- **Passcode Flow**: Validate that passcode entry and Assistive Touch work as expected. -- **End-to-End Transaction Confirmation**: Simulate real user behavior to confirm Apple Pay transactions. - -## Supported Devices - -Currently, Apple Pay manual testing in App Live is supported on these iOS real devices: - -| iOS Device | iOS Version | -|--------------|-------------| -| iPhone 16 | 18 | -| iPhone 15 | 17 | -| iPhone 14 | 16 | -| iPhone 13 | 15 | - -> We are actively working on expanding coverage to **all supported iOS devices** on TestMu AI Real Device Cloud. - -## Manual Apple Pay Workflow in App Live - -### Step 1: Enable Apple Pay in Advanced Settings - -1. Open **App Live** and select your desired iOS device. -2. Navigate to **Advanced Settings**. -3. Locate the **Apple Pay** toggle and enable it, and confirm the device selection accordingly. - -### Step 2: Configure Wallet - -1. Within the app, select the option to configure **Wallet** inside iOS settings. - -2. User can choose any of the four options: **American Express**, **Visa**, **MasterCard**, **Discover Global Network**. - -4. Wait 1–3 minutes for Apple Pay to configure. - -### Step 4: Use Apple Pay in App Live - -Within the App Live session, you can use Apple Pay via the **Assistive Touch** option. Simply click on Apple Pay, and you will be prompted to enter the device passcode (default is `123456`). Once the passcode is entered, you can complete a payment through Apple Pay, simulating a real user transaction. - -- If the **Wallet** is not set up on the device, the **Apple Pay** option will not appear through **Assistive Touch**. Please ensure Wallet is configured before attempting to access Apple Pay. - -## Additional Links - -- [Apple Pay Demo Website](https://developer.apple.com/apple-pay/sandbox-testing/) -- [TestMu AI App Live Overview](https://www.testmuai.com/app-live) -- [iOS Assistive Touch Guide](https://support.apple.com/guide/iphone/assistive-touch-iph3e2e4157/ios) - -Within a single session, only one card can be selected. To switch cards, you’ll need to start a new session with the desired card. diff --git a/static/docs/application-setup-via-api.md b/static/docs/application-setup-via-api.md deleted file mode 100644 index ea684e468..000000000 --- a/static/docs/application-setup-via-api.md +++ /dev/null @@ -1,127 +0,0 @@ -# Perform Operations on your Application via API - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -To test your **iOS** (.ipa file) or **Android** (.apk or .aab file) application on TestMu AI, you can use our public REST APIs. In this documentation, we have listed all the operations you can perform with your application via APIs or cURL commands for both Virtual and Real Devices. - -The maximum size for application should not exceed 1GB. - -- If you do not have any **.apk** or **.ipa** file, you can run your sample tests on TestMu AI by using our sample :link: [Android app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk) or sample :link: [iOS app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_ios.ipa). - -## Upload your Application - -| PARAMETER | EXAMPLE | DESCRIPTION | -|-----------------|-------------|------------| -| `custom_id` | `-F "custom_id="Proverbial_1.0"` | You do not have to remember the `app_URL` and only use the `custom_id` to run your automation on the same app. | -| `storage` | `-F "storage=file"` DEFAULT: `url` | Used to change the way TestMu AI stores the link. Used when we Upload using App URL | -| `visibility` | `-F "visibility=team"` DEFAULT: `individual` | Used to change the visibility of the application being uploaded. Once the app is uploaded using the `team`, everyone in the organisation can use the same URL to run the tests. | - -### Using App File - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "appFile=@"/Users/macuser/Downloads/Appname.apk"" -F "name="appname""`} - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/virtualDevice" -F "appFile=@"/Users/macuser/Downloads/Appname.apk"" -F "name="appname""`} - -### Using App URL - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "url=https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk" -F "name=Proverbial_App" -F "custom_id=sampleName" -F "storage=url" -F "visibility=individual"`} - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/virtualDevice" -F "url=https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk" -F "name=Proverbial_App" -F "custom_id=sampleName" -F "storage=url" -F "visibility=individual"`} - -- Response of above cURL will be a **JSON** object containing the `App URL` of the format - ``lt://APP123456789123456789`` - -**note** -The upload time of your application can range from a few seconds to a minute, depending on the size of your application. Therefore, do not interrupt the cURL command request until you receive the response. - -## Fetch your Applications for Real Devices - - {`curl --location --request GET "https://${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}@manual-api.lambdatest.com/app/data?type=android&level=user"`} - - {`curl --location --request GET "https://${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}@manual-api.lambdatest.com/app/data?type=ios&level=user"`} - -Shown below is the response to the above cURL request. - -```javascript -{ -"metaData": { -"type": "ios", -"total": 1 -}, -"data": [ -{ -"app_id": "APP100245789181570497850", -"name": "proverbial_ios.ipa", -"type": "ios", -"updated_at": "2022-05-10T11:19:30.000Z", -"shared": false, -"source": "web-client" -} -] -} -``` - -## Fetch your Applications for Virtual Devices - - {`curl --location --request GET "https://${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}@manual-api.lambdatest.com/app/data?type=emulator&level=user"`} - - {`curl --location --request GET "https://${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}@manual-api.lambdatest.com/app/data?type=simulator&level=user"`} - -## Deleting your Application - -{`curl --location --request DELETE "https://${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}@manual-api.lambdatest.com/app/delete" \ ---header 'Content-Type: application/json' \ ---data-raw '{ - "appIds" : "APPID1,APPID2" -}' -`} - -{`curl --location --request DELETE "https://${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}@manual-api.lambdatest.com/app/delete" \ ---header 'Content-Type: application/json' \ ---data-raw '{ - "appIds" : "APPID1,APPID2" -}' -`} - -Shown below is the response to the above cURL request. - -```javascript -{ -"message": "Deleted successfully." -} -``` - -## Processing check for your Application - -{`curl --location --request POST 'https://mobile-api.lambdatest.com/mobile-automation/api/v1/fetchpatchedapkurl' \ ---header 'Authorization: Basic ' \ ---header 'Content-Type: application/json' \ ---data-raw '{ - "appId": "APP10160161171698993659206876", - "networkLogsEnabled": true, - "imageInjectionEnabled": true, - "screenshotUnblockEnabled": true -}'`} - -{`curl --location --request POST 'https://mobile-api.lambdatest.com/mobile-automation/api/v1/fetchpatchedapkurl/virtualDevice' \ ---header 'Authorization: Basic ' \ ---header 'Content-Type: application/json' \ ---data-raw '{ - "appId": "APP10160161171698993659206876", - "networkLogsEnabled": true, - "imageInjectionEnabled": true, - "screenshotUnblockEnabled": true -}'`} - -The payload allows you to check the processing status for specific features. If the **patched_url** is empty, the processing is still in progress. To check if the processing for image injection or screenshot unblock is complete, pass either **imageInjectionEnabled** or **screenshotUnblockEnabled** as `true` based on the feature you are testing. - -```javascript -{ -"data": { -"imageinjection_ready": false, //current processing status -"patched_url": "", -"screenshotunblock_ready": false, //current processing status -"status": "success" -}, -"status": "success" -} -``` diff --git a/static/docs/application-setup-via-gui.md b/static/docs/application-setup-via-gui.md deleted file mode 100644 index 524997b5e..000000000 --- a/static/docs/application-setup-via-gui.md +++ /dev/null @@ -1,30 +0,0 @@ -# Perform Operations on your Application via GUI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -To test your iOS (.ipa file) or Android (.apk or .aab file) application on TestMu AI, you can upload them directly from the dashboard. In this documentation, we have listed the the ways via which you can upload your application for both Virtual and Real Devices. - -## Real Device Dashboard -- **Access the Dashboard :** Log in to your TestMu AI account and navigate to the [Real Device dashboard](https://applive.lambdatest.com/app). -- **Initiate the Upload :** Click the Upload button in the dashboard and select your application file from your local system: - - **For Android :** Ensure the file is in `.apk` or `.aab` format. - - **For iOS :** Use `.ipa` format. -Wait for the file to upload and process. A success message will indicate when your application is ready. -- **Open App Settings :** Once uploaded, find your application in the list and click on the Settings button. -- **Copy the App ID :** In the settings, locate the App ID field. Copy this ID, as it will be required to reference your app in automated tests or other configurations. - -> Learn how to [Upload Apps on TestMu AI’s Real Device Cloud](/support/docs/upload-apps-on-real-device-cloud/) in detail. - -- You can access the [**Virtual Device Dashboard**](https://www.testmuai.com/login/?redirectTo=https://app.lambdatest.com/console/realtime/app). -- For **Virtual Devices**, the steps remain identical to those for Real Devices. -- The only difference is that for **iOS applications**, you must upload a **.zip** file instead of an `.ipa` file, as simulators accept `.zip` format. - -## Automation Dashboard -- **Access the App Automation Dashboard :** Log in to your TestMu AI account and navigate to the [App Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://appautomation.lambdatest.com/build). -- **Upload the Application :** Click the Browse File button in the App tab under the Real Device or Virtual Device section. Select your application file from your local system: - - **Android :** Upload a `.aab` or `.apk` file. - - **iOS :** Upload an `.ipa` file. -- **Copy the App URL :** After uploading, an `app_url` will be generated (format: `"lt://"`). Copy this app_url and use it in your test scripts to identify your application. - -- The **App Automation Dashboard** supports both **Real** and **Virtual Devices**. -- To perform testing on Virtual Devices, simply select the **Virtual Device** option during upload. The rest of the flow remains identical. diff --git a/static/docs/applitools-integration-cypress.md b/static/docs/applitools-integration-cypress.md deleted file mode 100644 index 3c780edd7..000000000 --- a/static/docs/applitools-integration-cypress.md +++ /dev/null @@ -1,69 +0,0 @@ -# Applitools Integration With TestMu AI For Cypress - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -**Applitools** provides an end-to-end software testing platform powered by **Visual AI**. This article will guide you on how to integrate your **Cypress** automation tests with **Applitools** and **TestMu AI**. - -> Before getting started, make sure you have made accounts in **TestMu AI** and **Applitools** platforms. - -## Setting Up Applitools Dependencies And Packages - -You can download or clone the sample GitHub project from the following repo - [Cypress-LambdaTest-Applitools-Integration](https://github.com/gaurav8760/Cypress-LambdaTest-Applitools-Integration). Navigate to your project directory and follow these steps: - -- Install the **NPM dependencies** using the below command: - -```bash -npm install -``` - -- Set up **Applitools Eyes** using the below command: - -```bash -npx eyes-setup -``` - -- Add the access token of your Applitools account in the `applitools.config.js` file as highlighted in the below screenshot. You can find your **Applitools API** key by logging into your Applitools account and going to your **Profile > My API Key**. - - > Set the **testConcurrency** and **browser configuration** as per the requirement. - -- Install Applitools Eyes dependencies using the below command: - -```bash -npm install @applitools/eyes-cypress --save-dev -``` - - > It will create Applitools Eyes to an Existing Node.js Cypress Project. - -## Setting up TestMu AI CLI and Dependencies - -- Install the TestMu AI CLI using below command: - -```bash -npm install -g lambdatest-cypress-cli -``` - -- Setup configurations on which you want to run your test using the below command: - -```bash -lambdatest-cypress init -``` - - > It will create a **lambdatest-config.json** file in the root directory of your project. - -- Add spec file path in the `run_setting` and your **TestMu AI credentials** in the auth section of the `lambdatest-config.json` file as shown in the image below. - -## Running The Sample Project - -Execute your first test case using the below command: - -```bash -lambdatest-cypress run -``` - -> The first test case that you execute will be considered as the base image by default by Applitools. - -Edit the URL in the `applitoolsIntegration.spec.js` file and run your second test by using the same run command. - -Applitools will compare the base image with this test and highlight the difference. - -> That's all! you have successfully integrated **Applitools** and **TestMu AI** for running your **Cypress** tests. In case you have any questions or need any additional information, you could reach out at our **24X7 Chat Support** or mail us directly at support@testmuai.com. diff --git a/static/docs/applitools-integration-with-testmu.md b/static/docs/applitools-integration-with-testmu.md deleted file mode 100644 index 96768c946..000000000 --- a/static/docs/applitools-integration-with-testmu.md +++ /dev/null @@ -1,191 +0,0 @@ -# Applitools Integration With TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Applitools offers a cloud-based platform to help you perform AI Native, end-to-end visual testing and monitoring. The platform is designed to make life easier for automation testers by delivering a flexible way to directly integrate their test scripts without worrying about learning a new language or creating a new automation script. - -TestMu AI integration with Applitools will help you perform automated cross browser testing on more than 2000 real browsers running on real operating systems hosted on a cloud-based Selenium Grid. You can execute your automation test script on TestMu AI Selenium Grid directly from your Applitools instance. - -In this topic, you will learn about the: -* Prerequisites for integrating TestMu AI with Applitools. -* Running your first test on TestMu AI Selenium Grid from Applitools. -* Test your locally hosted applications on TestMu AI from Applitools. - -## Prerequisites To Integrate TestMu AI With Applitools - ->**Important:** Visit TestMu AI [GitHub repository for TestMu AI Integration with Applitools.](https://github.com/LambdaTest/applitools-lambdatest-integration) - -* **TestMu AI Authentication Credentials** - - Be aware of your TestMu AI authentication credentials i.e., your TestMu AI username, access key, and HubURL. You need to set them up as your environment variables. You can retrieve them from your [TestMu AI automation dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com) by clicking on the key icon near the help button. -* For Linux/Mac: - -{`$ export LT_USERNAME= ${ YOUR_LAMBDATEST_USERNAME()} -$ export LT_ACCESS_KEY= ${ YOUR_LAMBDATEST_ACCESS_KEY()}`} - -* For Windows: - -{`$ set LT_USERNAME= ${ YOUR_LAMBDATEST_USERNAME()} -$ set LT_ACCESS_KEY= ${ YOUR_LAMBDATEST_ACCESS_KEY()}`} - -* **Applitools API Key** - - You can find your Applitools API key by logging into your Applitools account and going to your **Profile > My API Key.** In your terminal or CMD, you need to export the Applitools API key along with TestMu AI Credentials. - * For Linux/Mac: -``` -$ export APPLITOOLS_API_KEY= -``` - - * For Windows: -``` -$ set APPLITOOLS_API_KEY= -``` - -* **Global Dependencies** - * Make sure you have [Java](https://www.java.com/en/download/help/download_options.xml) installed in your operating system. - * Download and install [Maven Apache.](https://maven.apache.org/download.cgi) - -* Install **Maven dependencies** using the below command. -``` -$ mvn install -``` -## Run Your First Test On TestMu AI Selenium Grid From Applitools - -Below is a code to run a demo test on TestMu AI Selenium Grid from Applitools. - ->**Important:** You can clone the below code from our [GitHub repository for TestMu AI Integration with Applitools](https://github.com/LambdaTest/applitools-lambdatest-integration/blob/master/src/test/java/com/applitools/quickstarts/BasicDemo.java). - -{`package com.applitools.quickstarts; -import static com.google.common.base.Strings.isNullOrEmpty; -import java.net.MalformedURLException; -import java.net.URL; -import com.applitools.eyes.*; -import com.applitools.eyes.selenium.ClassicRunner; -import com.applitools.eyes.selenium.Eyes; -import org.junit.After; -import org.junit.Before; -import org.junit.BeforeClass; -import org.junit.Test; -import org.junit.runner.RunWith; -import org.junit.runners.JUnit4; -import org.openqa.selenium.By; -import org.openqa.selenium.WebDriver; -import org.openqa.selenium.chrome.ChromeDriver; -import org.openqa.selenium.remote.DesiredCapabilities; -import org.openqa.selenium.remote.RemoteWebDriver; -/** - * Runs Applitools test for the demo app https://demo.applitools.com - */ -@RunWith(JUnit4.class) -public class BasicDemo { - private EyesRunner runner; - private Eyes eyes; - private static BatchInfo batch; - private RemoteWebDriver driver; - private static String username; - private static String accesskey; - private static String gridURL; - private static String applitoolsApiKey; - @BeforeClass - public static void setBatch() { - // Must be before ALL tests (at Class-level) - batch = new BatchInfo("Demo batch"); - username = System.getProperty("LT_USERNAME", "${ YOUR_LAMBDATEST_USERNAME()}"); - accesskey = System.getProperty("LT_ACCESS_KEY", "${ YOUR_LAMBDATEST_ACCESS_KEY()}"); - applitoolsApiKey = isNullOrEmpty(System.getenv("APPLITOOLS_API_KEY")) - ? "Your Applitools Api Key" - : System.getenv("APPLITOOLS_API_KEY"); - gridURL = "http://" + username + ":" + accesskey + "@hub.lambdatest.com/wd/hub"; - } - @Before - public void beforeEach() throws MalformedURLException { - // Initialize the Runner for your test. - runner = new ClassicRunner(); - // Initialize the eyes SDK - eyes = new Eyes(runner); - // Raise an error if no API Key has been found. - if (isNullOrEmpty(applitoolsApiKey)) { - throw new RuntimeException("No API Key found; Please set environment variable 'APPLITOOLS_API_KEY'."); - } - // Set your personal Applitols API Key from your environment variables. - eyes.setApiKey(applitoolsApiKey); - // set batch name - eyes.setBatch(batch); - // Use Chrome browser -// driver = new ChromeDriver(); - DesiredCapabilities capabilities = new DesiredCapabilities(); - capabilities.setCapability("network", true); - capabilities.setCapability("visual", true); - capabilities.setCapability("video", true); - capabilities.setCapability("console", true); - capabilities.setCapability("browserName", "Chrome"); - capabilities.setCapability("version", "76"); - capabilities.setCapability("platform", "Windows 10"); - capabilities.setCapability("name", "Applitools Sample Test"); - capabilities.setCapability("build", "Applitools Demo"); - driver = new RemoteWebDriver(new URL(gridURL), capabilities); - } - @Test - public void basicTest() { - // Set AUT's name, test name and viewport size (width X height) - // We have set it to 800 x 600 to accommodate various screens. Feel free to - // change it. - eyes.open(driver, "Demo App", "Smoke Test", new RectangleSize(800, 600)); - // Navigate the browser to the "ACME" demo app. - driver.get("https://demo.applitools.com"); - // To see visual bugs after the first run, use the commented line below instead. - // driver.get("https://demo.applitools.com/index_v2.html"); - // Visual checkpoint #1 - Check the login page. - eyes.checkWindow("Login Window"); - // This will create a test with two test steps. - driver.findElement(By.id("log-in")).click(); - // Visual checkpoint #2 - Check the app page. - eyes.checkWindow("App Window"); - // End the test. - eyes.closeAsync(); - } - @After - public void afterEach() { - // Close the browser. - driver.quit(); - // If the test was aborted before eyes.close was called, ends the test as - // aborted. - eyes.abortIfNotClosed(); - // Wait and collect all test results - TestResultsSummary allTestResults = runner.getAllTestResults(); - // Print results - System.out.println(allTestResults); - } -}`} - -Once you have cloned the above code, you need to execute the below command to trigger the test on TestMu AI Selenium Grid. - -``` -$ mvn -Dtest=BasicDemo test -``` ->Watch the execution of the demo test on the [TestMu AI Automation Dashboard.](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/) - -## Test Your Locally Hosted Applications On TestMu AI From Applitools - -You can test your locally hosted or privately hosted projects with the TestMu AI Selenium Grid cloud using the Lambda Tunnel app. All you would have to do is set up an SSH tunnel using Lambda Tunnel app and pass toggle tunnel = True via desired capabilities. Lambda Tunnel establishes a secure SSH protocol based tunnel that allows you in testing your locally hosted or privately hosted pages, even before they are made live. - ->Refer to our [TestMu AI Tunnel documentation](/docs/testing-locally-hosted-pages/) for more information. - -Download the latest TestMu AI Tunnel binary file and extract it. - ->Download: -* [Lambda Tunnel for Windows](https://downloads.lambdatest.com/tunnel/v3/windows/64bit/LT_Windows.zip) -* [Lambda Tunnel for Mac](https://downloads.lambdatest.com/tunnel/v3/mac/64bit/LT_Mac.zip) -* [Lambda Tunnel for Linux](https://downloads.lambdatest.com/tunnel/v3/linux/64bit/LT_Linux.zip) - -To run your Applitools automation script for locally hosted web applications on TestMu AI Selenium Grid, you would need to set the **Tunnel Capability** flag to ```true```. - -``` -DesiredCapabilities capabilities = new DesiredCapabilities(); -capability.setCapability("tunnel", true); -``` ->**Important Note:** -Some Safari & IE browsers don't support automatic resolution of the URL string *localhost*. Therefore if you test on URLs like `http://localhost/` or `http://localhost:8080` etc., you would get an error in these browsers. A possible solution is to use `localhost.lambdatest.com` or replace the string *localhost* with a machine IP address. For example, if you wanted to test `http://localhost/dashboard` or, and your machine, IP is 192.168.2.6, you can instead try on `http://192.168.2.6/dashboard` or `https://localhost.lambdatest.com/dashboard`. - -**Got Questions?** -Feel free to give us a shout on our 24/7 in-app customer chat support. You can even mail to support@testmuai.com. Happy Testing! 🙂 diff --git a/static/docs/appveyor-ci-integration.md b/static/docs/appveyor-ci-integration.md deleted file mode 100644 index d5e2a448b..000000000 --- a/static/docs/appveyor-ci-integration.md +++ /dev/null @@ -1,50 +0,0 @@ -# AppVeyor CI Integration with TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -**AppVeyor CI** is a very popular continuous integration and deployment platform based on .NET applications. It can also be easily integrated with GitHub and BitBucket projects for CI/CD. It makes testing, deployment, and monitoring builds very easy for developers. It is free for open-source projects. - -**TestMu AI** now integrates with AppVeyor CI to boost your go-to-market delivery. You can perform automated cross-browser testing with TestMu AI to ensure seamless code render through an online Selenium grid. TestMu AI provides 3000+ real browsers running through machines, on the cloud. You can perform automation testing in parallel with TestMu AI’s Selenium grid, which can help you trim down your test cycles drastically. - -## Prerequisites For Integrating AppVeyor CI With Our Online Selenium Grid - -To perform your tests with TestMu AI, you would need the below things to be already set up: - -1. A GitHub or BitBucket repository on which you want to activate the CI pipeline. - -> For demo purposes, we have used [TestMu AI Appveyor CI Github repository](https://github.com/lambdatest/AppveyorCI). - -2. You also need your TestMu AI authentication credentials, which include your TestMu AI username, and access key. - -> To get your Username and Access Key, go to your [TestMu AI automation dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/) and click on the "key" icon left to the help button, on the top right corner. - -Update your username and access key in the "Config.properties" file. - -## Activating A Github Repository For CI/CD - -Here are the steps to activate and integrate your Github repository with AppVeyor CI on TestMu AI: - -Login to your [AppVeyor CI](https://ci.appveyor.com/) platform and click on **New Project** to enable CI/CD on your selected repository. For demo purposes, we have used the GitHub repository mentioned in the Prerequisites above. ] - -Once your repository is added, go to the **Settings -> Build**, and select the trigger choice for build purpose. For this demo, we have used **Script** trigger and passed the below code in command line (**Cmd**). Save the build settings when done. - -``` -mvn test -``` - -Congratulations!! Now you have successfully activated your repository for CI/CD on the AppVeyor platform. - -## Execution Of The AppVeyor CI Pipeline On TestMu AI - -Below is the step-by-step execution of the AppVeyor CI pipeline. - -1. Make sure to add your TestMu AI's hub URL and [Desired Capabilities](/docs/selenium-automation-capabilities/) in your test setup, as done in the _AppveyorCI/src/main/java/stepDefinitions/ToDoStepDefinition.java_ file of the mentioned GitHub repository. - -2. Push the changes to your integrated GitHub repository. For example, we have changed the test name in the forked repository. -3. Once the edit is complete, click on "Commit Changes" to save and commit the changes made. AppVeyor CI is activated as soon as this step is done since we have used "push event" for triggering the AppVeyor CI. - -4. As soon as the AppVeyor CI is triggered (through push event in this case), a new build can be seen in the "Timeline" on your [TestMu AI's automation dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/timeline/). The current status of this build will be "running" or "executing". - -5. Now based on the tests, this build can pass or fail. If passed, a green tick along with the term **Passed** will appear in the summary of the test. Else if failed, a red cross will appear instead, along with the term **Failed**, as shown below. - -> Deploy your code in a reliable manner at scale using AppVeyor CI integration with TestMu AI, and ensure it looks robust across every browser to provide a seamless user experience to all your visitors. If you have any doubt, please feel free to reach out to us via **24/7 chat support** or by mailing to us on [support@testmuai.com](mailto:support@testmuai.com). Happy testing! diff --git a/static/docs/architecture-and-how-evaluation-works.md b/static/docs/architecture-and-how-evaluation-works.md deleted file mode 100644 index f5ec58e47..000000000 --- a/static/docs/architecture-and-how-evaluation-works.md +++ /dev/null @@ -1,195 +0,0 @@ -# How the TestMu AI Agent Testing Platform Evaluates Agents - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The Agent Testing Platform runs every evaluation through five phases: configure context, generate scenarios, run the testing agents, score each interaction, and return a production readiness verdict. 15+ specialized testing agents drive it, each probing one quality dimension, all in parallel against your agent's real endpoint. - -You run it from one hosted web application, the REST API, the CLI, or a CI/CD pipeline. There is no SDK to install and no local infrastructure to operate. - -## The Five-Phase Evaluation Loop - -Every run follows the same path, from the context you upload to a go or no-go verdict. The phases always execute in order. - -1. **Configure agent context.** Upload the documents that describe the agent (PDF, DOCX, images, audio, video, JIRA tickets, Confluence pages), write the agent prompt that defines correct behavior, and add instructions, constraints, or focus areas for the run. - -2. **Generate scenarios.** The platform generates 60 to 100+ test scenarios from that context, distributed across happy-path flows, edge cases, adversarial inputs, persona-specific scenarios, and compliance checks. - -3. **Run autonomous evaluation.** 15+ specialized testing agents interact with your agent through its real channel, a chat API, a voice endpoint, or a phone number, in parallel. - -4. **Score each interaction.** Every scenario is scored per metric with a pass or fail result, a confidence level of High, Medium, or Low, and an evidence excerpt from the conversation. Scores aggregate per metric across the run. - -5. **Return a readiness verdict.** All scores roll up into a single Green, Yellow, or Red production readiness verdict. - -The agent prompt in phase one is the evaluation baseline. It tells the platform what a correct response looks like, so the quality of the verdict depends on how precisely you define it. - -## The Specialized Testing Agents - -The platform does not rely on a single evaluator. 15+ testing agents each own one quality dimension, the way a review panel produces a more reliable assessment than one reviewer. They run against the same conversation in parallel. - -| Testing agent | What it evaluates | -|---|---| -| Hallucination Hunter | Detects invented facts, policies, or data | -| Bias Detector | Identifies differential treatment across demographics | -| Completeness Validator | Confirms responses fully address the user's need | -| Context Specialist | Tracks whether the agent maintains session history | -| Tone and Personality Agent | Evaluates communication style and empathy | -| Edge Case Generator | Probes with unexpected, off-script inputs | -| Escalation Handler | Validates correct handoff to human agents | -| Compliance Validator | Checks adherence to regulatory and policy requirements | -| Reasoning Validator | Assesses logical consistency across turns | -| Toxicity Screener | Flags harmful, offensive, or inappropriate outputs | -| Security Researcher | Tests for data exfiltration and prompt injection | -| Data Privacy Guardian | Checks for PII exposure and mishandling | -| Accessibility Specialist | Evaluates responses for accessibility | -| Performance Monitor | Tracks latency and response completeness under load | -| User Experience Evaluator | Assesses overall user journey quality | - -## Security & red-teaming - -Agent Testing red-teams AI agents with adversarial scenario suites aligned to the OWASP Top 10 for LLM applications: prompt injection (LLM01), insecure output handling (LLM02), sensitive-information disclosure and PII leakage (LLM06), data exfiltration, jailbreak, and policy violations. - -Toxicity, bias and harmful-content generation are evaluated as first-class metrics across the tested persona set. Every finding is evidence-backed - the specific conversation turns that triggered a flag are preserved with per-turn attribution and confidence intervals, exportable for regulatory documentation - and the A2A CLI runs the same scenarios in CI so security testing gates every release. - -## Platform Architecture Layers - -The platform is a single hosted web application. A request flows through six layers, from the interface you configure in to the external providers that place calls and run models. - - Platform Architecture - End-to-end flow from test configuration to production-readiness assessment - - Web Application - - REST API / CLI - - CI/CD Pipelines - -Projects, scenarios, suites, and schedules are configured here - -▼ -Authenticated requests (Bearer / API Key) - - Authentication & Authorization Layer - -Per-organization isolation · Encrypted credential storage · Tunnel proxy for private agents - -▼ -Validated request routed to testing modules - - TESTING ENGINE - - Orchestrates scenario execution across all agent modalities - - 📞 - Phone Testing - Inbound & Outbound - Real calls via telephony providers - - 💬 - Chat Testing - Webhook & API - Multi-turn HTTP conversations - - 🎙️ - Voice Testing - Real-time Multi-transport - REST, WebSocket & WebRTC - - 🖼️ - Image Evaluation - Generation & Scoring - Prompt → image → quality check - - CORE CAPABILITIES - - AI Scenario Generation - 60-100+ from your docs - - Test Orchestration - Parallel suite execution - - Scheduling Engine - Cron-based regression runs - -Executes tests against your agent · Captures recordings & conversation logs - -▼ -Completed conversations sent for AI-powered scoring - - AI EVALUATION LAYER - - Scores every conversation across standardized quality dimensions - - 18+ Quality Dimensions - - Recording Analysis - - Go-Live Assessment - - Threshold Scoring - -Produces per-scenario scores · Aggregates into Go-Live verdict (🟢 Green / 🟡 Yellow / 🔴 Red) - -▼ -Results, recordings & assets persisted - - Managed Database - Scores & metadata - - Cache & Pub/Sub - Live SSE streaming - - Encrypted Object Store - Recordings & uploads - -All data encrypted at rest · TLS in transit · Per-org isolation - -▼ -Platform connects to your existing providers - no vendor lock-in - - EXTERNAL INTEGRATIONS - - Telephony Providers - Call placement & routing - - Voice Providers - Voice session management - - Speech Services - TTS & STT engines - - Issue Trackers - Jira, GitHub, Confluence - - AI Models - Scenario gen & evaluation - -| Layer | What it does | -|---|---| -| Interface & access | Web app, REST API, CLI, and CI/CD pipelines configure projects, scenarios, and schedules. Requests authenticate with a Bearer token or API key. | -| Authentication & isolation | Isolates data per organization, stores third-party credentials encrypted, and reaches private agents through a secure tunnel proxy. | -| Testing engine | Runs the chat, voice, phone, and image modules, generates scenarios, orchestrates parallel execution, and schedules recurring runs. | -| Evaluation layer | Scores conversations with the testing agents, analyzes uploaded recordings, and produces the go-live assessment against configurable thresholds. | -| Data persistence | Stores scores and metadata in a managed database, streams live run status over server-sent events, and keeps recordings and uploads in encrypted object storage. | -| External integrations | Connects telephony and voice providers, speech-to-text and text-to-speech engines, issue trackers (Jira, GitHub, Confluence), and the AI models used to generate and score. | - -## How the Platform Reaches Your Agent - -The testing agents talk to your agent through the same interface real users do. The transport depends on the agent type. - -- **Chat:** HTTP requests to your API endpoint. -- **Voice:** a real-time voice session over REST, WebSocket, or WebRTC. -- **Phone inbound:** a real phone call placed to the agent's number. -- **Phone outbound:** a provisioned number the agent calls, which the platform answers and scores. -- **Image:** prompts or images submitted to the generation endpoint. - -Agents on a private network are reached through the secure tunnel, so the endpoint never needs a public URL. - -## What the Metrics and Verdict Produce - -The testing agents score chat and voice agents across 9 quality metrics and phone agents across 30+ call metrics. Each score carries a confidence level based on how many scenarios backed it. - -The verdict is a deployment decision, not just a number. Green clears the agent for production, Yellow flags specific issues to fix first, and Red blocks deployment. Every verdict links back to the conversation turns that drove it, so a failing result points straight at the fix. - -## Related Agent Testing Guides - -- See how to [run your first evaluation from the dashboard](/support/docs/testing-your-first-ai-agent/). diff --git a/static/docs/asana-integration.md b/static/docs/asana-integration.md deleted file mode 100644 index 789a3aaeb..000000000 --- a/static/docs/asana-integration.md +++ /dev/null @@ -1,72 +0,0 @@ -# Asana Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -> Asana is a popular tool for teams following Agile methodology as a part of their Software Development Life Cycle. With Asana, you can coordinate all your team’s work by eliminating roadblocks and pinpointing risks. It offers a free flowing dashboard for sprint planning. Allowing everyone to acknowledge who is doing what and when! You can manipulate tasks freely from one state to another. You can also set out ambitious goals and create a path towards achieving it by different tasks displayed on the dashboard. - -The TestMu AI Asana Integration allows you to create a task directly to your specified Asana dashboard from TestMu AI platform. Share your UI observations and input with your teammates anytime, by capturing a screenshot, even in the middle of your test session. You can annotate the screenshot & highlight your issue or input. The fields populated by you when marking as bug through TestMu AI are displayed as information on Asana for that testing instance. - -**Asana Integration with TestMu AI, like all of the integrations to 3rd party applications, is available for freemium as well as premium plan.** - -## How To Integrate Asana With Your TestMu AI Account? - -**Step 1:** Login to your TestMu AI account. You would need to have Admin or User level access to see and install integrations. - -**Step 2:** Select **'Integration'** from the left navigation menu bar. This will guide you to a screen where you will find a list of 3rd party applications, available to integrate with your TestMu AI account. - -**Step 3:** Click on **'ADD'** under the block that says 'Asana'. - -**Step 4:** If you are already logged into **Asana**, you'll be asked to grant permission to TestMu AI for accessing your user-owned resources. - -**Step 5:** If you are not already logged in, then you will be redirected to a page where you can do so. Provide your login credentials. You can even login using your **Google account**. - -* **Why we ask for your login credentials?** - -Asana APIs uses OAuth 2.0's [authorization code grant flow](https://tools.ietf.org/html/rfc6749#section-4.1) for generating access tokens on user's behalf. It can also communicate with API from the command line with the help of Personal Access Token. - -* **What is a Personal Access Token?** - -In scenarios where OAuth is excessively utilized, Personal Access Tokens comes to aid in accessing the API. While generating a token, make sure you provide a description that is easy to remember, as your Personal Access Token will be derived from it. Treat this token just as you would treat your own password! Access tokens are strings with authorization key required to access an API. They are issued to the client server and are usually opaque. They are used for requesting access to protected, user-specific resources. Access tokens are vital from a security point of view & can be generated in different formats, depending upon security requirements specified on the resource server. - -**Step 7:** If you are logging into **'Asana'** for the first time, then you will be asked to choose teams concerned to your organizations. This is optional, you can skip and add teams later as well. - -**Step 8:** That's it! Go to Integrations again and you will be able to notice a green tick indicating that Asana is successfully added. - -## How To Log Your First Bug Through Asana Integration? - -> **Note:** If you are using Asana for the first time, then make sure to create a project for yourself. It is a pre-requisite in order to push screenshots from your TestMu AI account. - -**Step 1:** Go for any of the test from the left navigation menu. For demo, we will be taking "**Real Time Testing**" option. - -**Step 2:** Present a URL of the web-app you need to test in the dialog box. After that, select any configuration for browser and operating system of your choice & hit '**Start**'. - -**Step 3:** After the VM is launched and operable. You can perform testing on your web-app for finding bugs. If a bug gets revealed, then you need to click on the **Bug icon** from the left panel for capturing a screenshot of the same. We have highlighted that option with yellow in the below image. - -**Step 4:** After a screenshot is captured, you can annotate any issue or a task with an **in-built image editor**. Once you are done highlighting the bug, click on the button that says **"Mark as Bug"**. - -**Step 5:** After clicking on **"Mark as Bug"** button a form would open up. Fill the fields as per your requirement. - -* You can choose from your **Workspaces** to which you would like to share. -* You can choose from your **Projects** as well. -* You can assign the bug to a colleague by populating the field **'Assignee'**. -* You also get to post a **description** to help relate the cause of the issue or the task. - -At the bottom of the form you will find a button **"Create Issue"**. - -**Step 6:** After you click on **"Create Issue"**, you will be able to observe it being successfully marked through a single click effort. You will get **prompt messages** on top of your Virtual Machine indicating the progress of bug logging. After few seconds you will be notified with a prompt message "**Bug successfully marked**" indicating that the screenshot has been pushed to your workspace. - -Login to your Asana workspace and you will be able to notice the logged issue right away! All the data you provided through TestMu AI would already be presented in it. TestMu AI automatically includes test environment details and related screenshots in the screenshot as attachments. - -You can set a due date to the task in Asana to help you organize your tasks easily. - -## How To Remove Asana Integration? - -> You can work with one integration at a time. So if you would want to integrate to a similar 3rd party application, then you would have to **Remove** your current integration. Here is how you can do that. - -**Step 1:** Login to your TestMu AI account. - -**Step 2:** Select 'Settings' from the left navigation menu bar & click on 'Integrations'. This will guide you to a screen where you will find 3rd party applications, available to integrate with your TestMu AI account. - -**Step 3:** Click on **'Remove'**. You can find the Revoke button right under the **'Asana'** block which would be highlighted with a green tick. - -That was all you need to know for TestMu AI + Asana Integration. Increase your productivity with our integrations. If you still have any questions for us, please feel free to let us know. Our experts are always **available on chat** to help you out with any roadblock regarding our product. Happy testing! diff --git a/static/docs/assembla-integration.md b/static/docs/assembla-integration.md deleted file mode 100644 index e67daf10f..000000000 --- a/static/docs/assembla-integration.md +++ /dev/null @@ -1,46 +0,0 @@ -# Assembla Integration with TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Assembla is a cloud-based platform that provides version control and project management tools specifically designed for software development teams. It supports multiple version control systems, such as Git, Subversion (SVN), and Perforce, allowing teams to manage their source code, track changes, and collaborate more efficiently. - -TestMu AI integrates with Assembla, allowing users to file issues directly from the platform, assign them to team members, and attach screenshots during a test session, with populated fields displayed on the Assembla platform for each testing instance. - -> Assembla Integration with TestMu AI is available for freemium as well as premium plan. - -## Integrate Assembla from your TestMu AI Account - -**Step 1:** Login to your TestMu AI account. You should have Admin or User level access to see and install integrations. - -**Step 2:** Click on Settings -> Integrations -> Bug Tracker. - -**Step 3:** Click on **Connect** button of Assembla block. - -**Step 4:** Now, provide your Assembla API Key, Secret, Hosted URL and select your region to establish integration with TestMu AI and click on **install** button. - -**Fetch your Assembla API Token** - -- Visit your Assembla account -> click on your profile icon and then **Security** tab. -- You will find the "**API Token**" under the section **API Applications and Sessions**. Copy your API token. - -## Lodge your First Bug - -**Step 1:** Create a new space on Assembla if you haven't already. - -**Step 2:** Now, while testing your webapp or application at TestMu AI, click on the **mark as bug** button if you detect any bug. - -**Step 3:** Now update the comments of the bug and other details and click on Create Task button. - -**Step 4:** Go to your dashboard and check a ticket will be created for the same. - -**Step 6:** Go to your dashboard and check a ticket will be created for the same. - -## Uninstall Assembla Integration - -**Step 1:** Login to your TestMu AI account. You should have Admin or User level access to see and install integrations. - -**Step 2:** Click on Settings -> Integrations -> Bug Tracker. - -**Step 3:** Click on the **Remove** button. - -> That was all you need to know for TestMu AI + Assembla Integration. Increase your productivity with our integrations. If you still have any questions for us, please feel free to let us know. Our experts are always available on **chat** to help you out with any roadblock. diff --git a/static/docs/assistive-touch-on-real-ios-browser.md b/static/docs/assistive-touch-on-real-ios-browser.md deleted file mode 100644 index 1e6188a76..000000000 --- a/static/docs/assistive-touch-on-real-ios-browser.md +++ /dev/null @@ -1,41 +0,0 @@ -# Assistive Touch on Real iOS Devices (Browser) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -> To enable it for your organization, please contact us via **24×7 chat support** or you can also drop a mail to **support@testmuai.com**. - -TestMu AI's **Assistive Touch** feature enables you to perform key system-level actions directly from the testing toolbar during manual sessions on real iPhones and iPads. -This allows you to replicate gestures such as going to the Home Screen, opening the Notification Center, taking screenshots, and more without relying on physical device buttons or gestures. - -## Use Cases - -- Navigate to the Home Screen after completing a flow (e.g., logout, relaunch). -- Open the Notification Center to test notification-related workflows. -- Switch between apps to validate background/foreground behavior. -- Capture screenshots for UI validation or bug reporting. -- Simulate a shake gesture to test motion-based features. - -## Supported Devices - -| Device Type | Supported OS Version | -|-------------|----------------------| -| iPhone | iOS 14 and above | -| iPad | iPadOS 14 and above | - -## Supported Actions - -| Action | Description | -|--------|-------------| -| **Home Button** | Returns to the Home Screen. | -| **Notification Center** | Pulls down the system notification tray. | -| **App Switcher** | Opens multitasking to switch or close apps. | -| **Screenshot** | Captures the current screen. | -| **Shake Gesture** | Simulates device shake for undo or motion triggers. | - -## Using Assistive Touch in Manual Testing - -**Step 1:** Start a manual testing session on a real iOS device in **App Live** or **Browser Live**. - -**Step 2:** From the session toolbar, open the **iOS Settings** panel and toggle **Assistive Touch on** . - -**Step 3:** Tap the Assistive Actions icon to open the gesture menu, then select the desired action to simulate it on the device diff --git a/static/docs/assistive-touch-on-real-ios-devices.md b/static/docs/assistive-touch-on-real-ios-devices.md deleted file mode 100644 index 7632237ff..000000000 --- a/static/docs/assistive-touch-on-real-ios-devices.md +++ /dev/null @@ -1,41 +0,0 @@ -# Assistive Touch on Real iOS Devices - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI's **Assistive Touch** feature enables you to perform key system-level actions directly from the testing toolbar during manual sessions on real iPhones and iPads. -This allows you to replicate gestures such as going to the Home Screen, opening the Notification Center, taking screenshots, and more without relying on physical device buttons or gestures. - -> To enable it for your organization, please contact us via **24×7 chat support** or you can also drop a mail to **support@testmuai.com**. - -## Use Cases - -- Navigate to the Home Screen after completing a flow (e.g., logout, relaunch). -- Open the Notification Center to test notification-related workflows. -- Switch between apps to validate background/foreground behavior. -- Capture screenshots for UI validation or bug reporting. -- Simulate a shake gesture to test motion-based features. - -## Supported Devices - -| Device Type | Supported OS Version | -|-------------|----------------------| -| iPhone | iOS 14 and above | -| iPad | iPadOS 14 and above | - -## Supported Actions - -| Action | Description | -|--------|-------------| -| **Home Button** | Returns to the Home Screen. | -| **Notification Center** | Pulls down the system notification tray. | -| **App Switcher** | Opens multitasking to switch or close apps. | -| **Screenshot** | Captures the current screen. | -| **Shake Gesture** | Simulates device shake for undo or motion triggers. | - -## Using Assistive Touch in Manual Testing - -**Step 1:** Start a manual testing session on a real iOS device in **App Live** or **Browser Live**. - -**Step 2:** From the session toolbar, open the **iOS Settings** panel and toggle **Assistive Touch on** . - -**Step 3:** Tap the Assistive Actions icon to open the gesture menu, then select the desired action to simulate it on the device diff --git a/static/docs/audio-injection-manual-browser.md b/static/docs/audio-injection-manual-browser.md deleted file mode 100644 index 9f4a444cd..000000000 --- a/static/docs/audio-injection-manual-browser.md +++ /dev/null @@ -1,111 +0,0 @@ -# Audio Injection Manual Testing on Real Device Browsers - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -**Audio Injection** lets you simulate microphone input on real Android and iOS device browsers during a manual **Real Time** browser session. Use it to test speech-to-text web apps, voice search on the mobile web, browser-based voice assistants, web-based KYC voice verification, and any other microphone-dependent web feature, without speaking into a physical mic. - -With TestMu AI Real Time browser testing, you can either inject a **pre-uploaded audio file** or stream **Live Input** directly from your system microphone into the device browser. - -> To enable it for your organization, please contact us via **24×7 chat support** or you can also drop a mail to **support@testmuai.com**. - -## Use Cases - -- **Web voice search**: Validate browser-based voice search and dictation flows. -- **Speech-to-text web apps**: Confirm transcription accuracy on the mobile web. -- **Web-based KYC**: Test identity flows that record a voice sample in the browser. -- **WebRTC and conferencing**: Verify mic capture in browser conferencing tools. -- **Live exploratory testing**: Use Live Input to drive ad-hoc voice interactions on web pages in real time. - -## Supported Platforms - -| Platform | Minimum OS Version | -|---|---| -| **Android** | Android 13 (SDK 33) and above -| **iOS** | iOS 16 and above - -Audio Injection is supported on **selected real devices only** - -## Supported File Formats - -| Format | Max Size | -|---|---| -| MP3 | 200 MB | -| WAV | 200 MB | - -Files are uploaded one at a time and injected one at a time per session. - -## Manual Audio Injection Workflow in Browser Testing - -### Step 1: Enable Audio Injection in Advanced Settings - -1. Go to the TestMu AI **Dashboard** and open **Real Device Browser Testing**. -2. Click **Advanced Settings** on top. -3. Toggle **Enable Audio Injection** on. - -Once enabled, the device list is filtered to show **only the devices that support Audio Injection** for the respective OS. - -### Step 2: Select a Device and Start Your Session - -1. From the filtered device list, pick a supported Android or iOS real device along with the browser you want to test on. -2. Click **Start** to launch the session. - -### Step 3: Open Media Injection - -1. Inside the live browser session, locate **Media Injection** in the in-session toolbar. -2. Click to open the Media Injection panel. -3. Switch to the **Audio** tab. - -You will see two options: - -- **Select Files**: inject a pre-recorded audio file -- **Inject Live Input**: stream audio from your system microphone in real time - -### Step 4a: Inject an Audio File - -Use this mode when you want a deterministic, repeatable input, for example, the same voice query run across many device-browser combinations. - -1. In the **Audio** tab, select **Select Files**. -2. Click **Upload** and choose an `.mp3` or `.wav` file (up to **200 MB**, one file at a time). -3. Once uploaded, the file appears in your audio library (latest 5 uploaded). -4. In the browser, navigate to the page that captures microphone input (e.g., tap a voice search icon or **Start Recording**). -5. Grant the page mic permission when prompted. -6. Select the uploaded file and click **Inject**. -7. Once a file is injected, the controls are limited to **Play** and **Pause**. Clicking **Play** streams the audio into the device's microphone pipeline as live mic input, and the browser captures it as if the user were speaking. -8. To switch audio, select a different file and click **Inject** on it. Only **one file can be injected at a time**, and the new file replaces the previously injected one. - -Inject and start playback **after** the page has opened the mic. Some recognizers need 1–2 seconds of silence to initialize before they accept speech. - -### Step 4b: Use Live Input - -Use this mode when you want to drive the device microphone interactively, for example, having an unscripted conversation with a web voice assistant or testing custom prompts on the fly. - -1. In the **Audio** tab, select **Inject Live Input**. -2. Grant microphone access to your browser when prompted. -3. In the device browser, open the page that captures microphone input. -4. Click **Start**. Your system microphone is now streamed directly into the device's mic. -5. Speak into your mic. The web page receives your voice in real time. -6. Click **Stop** to end the live stream. - -Live Input streams from the same browser tab running the Real Time session. Avoid muting your system mic or switching tabs mid-session. The stream will be interrupted. - -## Execution Rules - -- The web page must be granted microphone permission. Audio Injection does **not** bypass permission prompts. -- Only one audio source is active at a time. Switching from **Files** to **Live Input** (or vice versa) replaces the previous source. -- For files, only one file can be injected and played at a time. -- The last injected audio is the active source until you stop it or inject another. - -## Tips and Best Practices - -- Keep audio files short (under 5 minutes) for predictable timing. -- Use 16 kHz mono MP3 or WAV for the most consistent results across Android and iOS browsers. -- For voice-recognition pages, allow the device 1–2 seconds of silence before injecting speech. -- Use **Files** for repeatable regression runs across device-browser pairs; use **Live Input** for exploratory and conversational testing. - -## Related Resources - -- [Audio Injection Manual Testing (App)](/docs/audio-injection-manual/): Same feature on App Live -- [Audio Injection on Real Devices (Automation)](/docs/audio-injection/): Inject audio via Appium / Selenium tests -- [Browser Testing on Real Devices](/docs/browser-testing-on-real-devices/) - -> **Need help?** Reach out via **24×7 chat support** or email **support@testmuai.com**. diff --git a/static/docs/audio-injection-manual.md b/static/docs/audio-injection-manual.md deleted file mode 100644 index 3debb0011..000000000 --- a/static/docs/audio-injection-manual.md +++ /dev/null @@ -1,110 +0,0 @@ -# Audio Injection Manual Testing on Real Devices - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -**Audio Injection** lets you simulate microphone input on real Android and iOS devices during a manual **App Live** session. Use it to test speech-to-text, voice commands, voice assistants, in-app recording, KYC voice verification, and any other microphone-dependent feature, without speaking into a physical mic. - -With TestMu AI App Live, you can either inject a **pre-uploaded audio file** or stream **Live Input** directly from your system microphone into the device under test. - -> To enable it for your organization, please contact us via **24×7 chat support** or you can also drop a mail to **support@testmuai.com**. - -## Use Cases - -- **Voice command testing**: Validate voice assistants and in-app voice search. -- **Speech-to-text validation**: Confirm transcription accuracy across devices. -- **In-app audio recording**: Verify that the app captures and stores microphone input correctly. -- **KYC and voice verification**: Test identity flows that require a recorded voice sample. -- **Live conversational testing**: Use Live Input to drive ad-hoc, interactive voice flows in real time. - -## Supported Platforms - -| Platform | Minimum OS Version | -|---|---| -| **Android** | Android 13 (SDK 33) and above -| **iOS** | iOS 16 and above - -Audio Injection is supported on **selected real devices only** - -## Supported File Formats - -| Format | Max Size | -|---|---| -| MP3 | 200 MB | -| WAV | 200 MB | - -Files are uploaded one at a time and injected one at a time per session. - -## Manual Audio Injection Workflow in App Live - -### Step 1: Enable Audio Injection in Advanced Settings - -1. Go to the TestMu AI **Dashboard** and open **Real Device App Testing**. -2. Click **Advanced Settings** on top. -3. Toggle **Enable Audio Injection** on. - -Once enabled, the device list is filtered to show **only the devices that support Audio Injection** for the respective OS. - -### Step 2: Select a Device and Start Your Session - -1. From the filtered device list, choose your app and pick a supported Android or iOS real device. -2. Click **Start** to launch the session. - -### Step 3: Open Media Injection - -1. Inside the live session, locate the **Media Injection** option in the in-session toolbar. -2. Click to open the Media Injection panel. -3. Switch to the **Audio** tab. - -You will see two options: - -- **Select Files**: inject a pre-recorded audio file -- **Inject Live Input**: stream audio from your system microphone in real time - -### Step 4a: Inject an Audio File - -Use this mode when you want a deterministic, repeatable input, for example, the same voice command run across many devices. - -1. In the **Audio** tab, select **Select Files**. -2. Click **Upload** and choose an `.mp3` or `.wav` file (up to **200 MB**, one file at a time). -3. Once uploaded, the file appears in your audio library (latest 5 uploaded) -4. In the app under test, navigate to the screen that captures microphone input (e.g., tap **Record** or **Start Voice Search**). -5. Select the uploaded file and click **Inject**. -6. Once a file is injected, the controls are limited to **Play** and **Pause**. Clicking **Play** streams the audio into the device's microphone pipeline as live mic input, and the app captures it as if a user were speaking. -7. To switch audio, select a different file and click **Inject** on it. Only **one file can be injected at a time**, and the new file replaces the previously injected one. - -Inject and start playback **after** the app has opened the mic. Some recognizers need 1–2 seconds of silence to initialize before they accept speech. - -### Step 4b: Use Live Input - -Use this mode when you want to drive the device microphone interactively, for example, holding an unscripted conversation with a voice assistant or testing custom prompts on the fly. - -1. In the **Audio** tab, select **Inject Live Input**. -2. Grant microphone access to your browser when prompted. -3. In the app under test, open the screen that captures microphone input. -4. Click **Start**. Your system microphone is now streamed directly into the device's mic. -5. Speak into your mic. The app receives your voice in real time. -6. Click **Stop** to end the live stream. - -Live Input streams from the same browser tab running App Live. Avoid muting your system mic or switching tabs mid-session. The stream will be interrupted. - -## Execution Rules - -- The app must be granted microphone permission. Audio Injection does **not** bypass permission prompts. -- Only one audio source is active at a time. Switching from **Files** to **Live Input** (or vice versa) replaces the previous source. -- For files, only one file can be injected and played at a time. -- The last injected audio is the active source until you stop it or inject another. - -## Tips and Best Practices - -- Keep audio files short (under 5 minutes) for predictable timing. -- Use 16 kHz mono MP3 or WAV for the most consistent results across Android and iOS. -- For voice-recognition tests, allow the device 1–2 seconds of silence before injecting speech. -- Use **Files** for repeatable regression tests; use **Live Input** for exploratory and conversational testing. - -## Related Resources - -- [Audio Injection on Real Devices (Automation)](/docs/audio-injection/): Inject audio via Appium / Selenium tests -- [Camera Image Injection on Real Devices](/docs/camera-image-injection-on-real-devices/) -- [Biometric Authentication on Real Devices](/docs/biometric-authentication-on-real-devices/) - -> **Need help?** Reach out via **24×7 chat support** or email **support@testmuai.com**. diff --git a/static/docs/audio-injection.md b/static/docs/audio-injection.md deleted file mode 100644 index 5ce8d9f9b..000000000 --- a/static/docs/audio-injection.md +++ /dev/null @@ -1,169 +0,0 @@ -# Audio Injection on Real Devices - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Test audio-driven and microphone-dependent features on real Android and iOS devices through Appium / Selenium automation. Inject pre-recorded audio files directly into the device microphone, no physical mic input required. - -> To enable it for your organization, please contact us via **24×7 chat support** or you can also drop a mail to **support@testmuai.com**. - -## Overview - -Audio Injection enables you to simulate microphone input on real devices by injecting pre-recorded audio files (MP3, WAV) into the device's mic pipeline during automated tests. - -Use it to test speech-to-text, voice commands, voice assistants, in-app recording, KYC voice verification, and any other microphone-dependent feature. - -## Supported Platforms - -| Platform | Minimum OS Version | -|---|---| -| **Android** | Android 13 (SDK 33) and above | -| **iOS** | iOS 16 and above | - -Audio Injection is supported on **real devices only**. It is **not** available on emulators or simulators. - -## Supported File Formats - -| Format | Max Size | -|---|---| -| MP3 | 200 MB | -| WAV | 200 MB | - -The audio file must be uploaded to TestMu AI before it can be injected. - -## Step 1 - Upload Audio File - -Use the **Upload Media API** to upload an audio file to your TestMu AI account. - -```bash -curl -u "LT_USERNAME:LT_ACCESS_KEY" -X POST "https://api.lambdatest.com/mfs/v1.0/media/upload" -F "media_file=@/path/to/test.wav" -F "type=audio" -F "custom_id=SampleAudio" -``` - -**Response** - -```json -{ -"media_url": "lt://MEDIAID", -"status": "success" -} -``` - -Save the returned `media_url`. You will use it in subsequent steps. - -## Step 2 - Enable Audio Injection on Your Session - -Set the `enableAudioInjection` capability when creating your driver session. - -```java -DesiredCapabilities caps = new DesiredCapabilities(); -caps.setCapability("enableAudioInjection", true); -caps.setCapability("media", "lt://MEDIA1234567890abcdef"); // optional: pre-set audio -``` - -```python -desired_caps = { -"enableAudioInjection": True, -"media": "lt://MEDIA1234567890abcdef", # optional -# ... other capabilities -} -``` - -```javascript -const capabilities = { -enableAudioInjection: true, -media: "lt://MEDIA1234567890abcdef", // optional -// ... other capabilities -}; -``` - -Setting the `media` capability pre-injects the audio when the session starts. You can still use the `lambda-audio-injection` hook later to switch to a different file mid-session. - -## Step 3 - Inject and Control Audio with Lambda Hooks - -Use the following hooks via `driver.executeScript`: - -| Hook | Description | -|---|---| -| `lambda-audio-injection=` | Set the audio file to be injected | -| `lambda-audio-start` | Start playing the injected audio into the device microphone | -| `lambda-audio-stop` | Stop audio playback | - -```java -// 1. Set the audio file -driver.executeScript("lambda-audio-injection=lt://MEDIA1234567890abcdef"); - -// 2. Trigger the app's mic input (e.g., tap Record / Start Voice Search) -driver.findElement(AppiumBy.id("recordButton")).click(); - -// 3. Start audio playback, the app receives the file as live mic input -driver.executeScript("lambda-audio-start"); - -// ... wait for the app to finish capturing ... - -// 4. Stop audio playback -driver.executeScript("lambda-audio-stop"); -``` - -```python -driver.execute_script("lambda-audio-injection=lt://MEDIA1234567890abcdef") -driver.find_element(AppiumBy.ID, "recordButton").click() -driver.execute_script("lambda-audio-start") -# ... -driver.execute_script("lambda-audio-stop") -``` - -```javascript -await driver.executeScript("lambda-audio-injection=lt://MEDIA1234567890abcdef"); -await driver.findElement(By.id("recordButton")).click(); -await driver.executeScript("lambda-audio-start"); -// ... -await driver.executeScript("lambda-audio-stop"); -``` - -## Execution Rules - -- Audio must be **injected before** triggering the microphone in the app. The last injected audio is the active input. -- The session must be created with `enableAudioInjection: true`. Hooks called without the capability return **HTTP 403 Forbidden**. -- Calling `lambda-audio-start` before any file has been injected returns **HTTP 400** with `AUDIO_INJECTION_MEDIA_FILE_NOT_PROVIDED_ERROR`. -- Multiple injections in the same session: the last injected audio is used on the next `lambda-audio-start`. -- The app must be granted microphone permission. Audio Injection does **not** bypass permission prompts. - -## Best Practices - -- Keep audio files short (under 5 minutes) for predictable timing. -- Use 16 kHz mono PCM-encoded MP3 or WAV for best compatibility across devices. -- Inject the audio file once at session start, then call `lambda-audio-start` / `lambda-audio-stop` around your test steps for precise control. -- For voice-recognition tests, allow the device 1-2 seconds of silence before injecting speech to let the recognizer initialize. - -## FAQs - -**Q. Can I use Audio Injection with my own automation framework?** - -Yes. Audio Injection is a server-side feature controlled entirely by capabilities and `executeScript` hooks. It works with any Appium- or Selenium-compatible framework. - -**Q. Does Audio Injection work in parallel test runs?** - -Yes. Each session has its own isolated injection pipeline. Multiple parallel sessions can inject different audio files simultaneously without interference. - -**Q. What audio formats give the best results?** - -16 kHz mono MP3 or WAV files give the most consistent results across both Android and iOS. iOS internally upscales to 48 kHz; Android operates natively at 16 kHz. - -**Q. My speech-to-text app isn't picking up the injected audio. What's happening?** - -Confirm: -1. `enableAudioInjection: true` is set on the session. -2. `lambda-audio-injection=` was called **before** `lambda-audio-start`. -3. The app has been granted microphone permission. -4. Some apps (e.g., Google Recorder on Pixel) use privileged hardware audio paths that bypass standard injection. Use a different recorder app to verify. - -**Q. Can I switch audio files mid-session?** - -Yes. Call `lambda-audio-injection=` followed by `lambda-audio-start`. The new file replaces the previous one immediately. - -## Related Features - -- [Camera Image Injection](/docs/camera-image-injection/): Inject images into the device camera -- [Video Injection](/docs/video-injection/): Inject videos into the device camera -- [Biometric Authentication](/docs/biometric-authentication/): Simulate fingerprint/face authentication - -**Need help?** Contact Support or chat with us at the bottom-right of any page. diff --git a/static/docs/audit-logs.md b/static/docs/audit-logs.md deleted file mode 100644 index 0e81065b6..000000000 --- a/static/docs/audit-logs.md +++ /dev/null @@ -1,115 +0,0 @@ -# Audit Logs - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Audit logs serve as a record of every **User** and **Organization** Level activities in a TestMu AI account. - -> **Note:** Only organization administrators are able to view and manage the audit logs. - -## Why are Audit Logs important? - -Audit logs can be used to: - -- Monitor account activity and detect unauthorized access. -- Investigate security incidents. -- Track user-level and organization-level activity histories. -- Audit compliance with security policies. - -## How to Access Audit Logs? - -**Step 1:** Go to the [LambdaTest](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard) Dashboard. - -**Step 2:** Click on Settings -> **Organization Settings** and navigate to the **Audit Logs** tab. - -### Audit Logs details - -Each log in the audit logs section has detailed information regarding who performed the action, when the action was performed, the scope of the action, the entities involved and affected by the action etc. The following sections provide a general overview of how to read and interpret the data. - -The following table contains the list of activities and their definition: - -|Sl no.|Event |Event Name |Event Description | -|------|------------------------------------------|---------------------------------------------------|----------------------------------------------| -|1 |user.password.changed |User - Password Changed |User changed their password | -|2 |user.access-key.regenerated |User - Access Key Regenerated |User regenerated their access key | -|3 |org.user.role.updated |Organization - User Role Updated |Admin updated user's role | -|4 |org.user.password.changed |Organization - User Password Changed |Admin changed user's password | -|5 |org.user.access-key.regenerated |Organization - User Access Key Generated |Admin regenerated user's access key | -|6 |org.service-account.access-key.regenerated|Organization - Service Account Access Key Generated|Admin regenerated service account's access key| -|7 |user.email-preference.updated |User - Email Preference Updated |User updated their email preferenece | -|8 |user.profile.deletion-request |User - Account Deletion Request Raised |User requested their account deletion | -|9 |org.user.deleted |Organization - User Removed from Organization |Admin removed a user from their organization | -|10 |user.password.forgot |User - Password Reset |User password reset | -|11 |org.support-access.granted |Organization - Support Access Granted |Admin granted TestMu AI Support Access | -|12 |org.support-access.revoked |Organization - Support Access Revoked |Admin revoked TestMu AI Support Access | -|13 |org.service-account.created |Organization - Service Account Created |Admin created a new Service Account | -|14 |org.service-account.deleted |Organization - Service Account Deleted |service-account deleted | -|15 |user.profile.updated |User - Profile Updated |User updated their profile | -|16 |user.login |User - Logged In |User - Logged In | -|17 |org.invitation.sent |Organization - Invitation Sent |Admin invited users to their organization | -|18 |user.email.verified |User - Email Address Verified |User verified their email address | -|19 |org.invitation.accepted |Organization - Invitation Accepted |Invitation was accepted | -|20 |org.invitation.declined |Organization - Invitation Declined |Invitation was declined | -|21 |user.logout |User - Logged Out |User Logged Out | -|22 |org.details.updated |Organization - Details Updated |Organization details was updated | -|23 |org.user.profile.updated |Organization - User Profile Updated |Admin updated user's profile | -|24 |org.service-account.profile.updated |Organization - Service Account Profile Updated |Admin updated service account's profile | -|25 |org.invitation.resent |Organization - Invitation Resent |Admin invited users to their organization | -|26 |org.invitation.withdrawn |Organization - Invitation Withdrawn |Invitation was withdrawn | -|27 |org.auth.mfa |Organization - MFA Enforced |MFA enforced by admin | -|28 |user.auth.mfa |User - MFA Settings Updated |MFA settings updated by user | -|29 |org.user.auth.mfa |Organization - MFA Settings Updated |Admin update user's MFA settings | -|30 |org.subscription.created |Organization - Subscription Created |Organization subscription created | -|31 |org.subscription.updated |Organization - Subscription Updated |Organization subscription updated | -|32 |org.subscription.cancelled |Organization - Subscription Cancelled |Organization subscription cancelled | -|33 |org.subscription.revoked |Organization - Subscription Revoked |Organization subscription revoked | -|34 |org.payment-method.added |Organization - Payment Method Added |Payment method added | -|35 |org.payment-method.deleted |Organization - Payment Method Deleted |Payment method deleted | -|36 |org.payment-method.default-updated |Organization - Default Payment Method Updated |Default payment method updated | - -As highlighted in the above screenshot, this section contains details that include the event ID, the email address of the **user who performed the action**, the time of the change/action, the IP address of the user and the user agent used. - -As highlighted in the above screenshot, details of the user affected by the change/action are shown. This includes an identifier - email address, and additional details such as the original value and the changed value if present. - -### Filter your Audit Logs - -You can filter the logs based on Action Performed either on User or Organization level Events, User Name and Date. - -#### Filter By Events - -You can select the type of event for which you want to check the logs. You can also filter it by what level you want to check the audit logs at, i.e., User level or Organization level. - -- **User Level ->** Logged In, Access Key Regenerated, Password Reset, Profile Update, Password Changed. - -- **Organization Level ->**Invitation Sent, Invitation Accepted, User Role Updated, User Profile Updated, User Removed from the Organization. - -#### Filter By Date - -You can view the logs of the users for a particular date range as well. Select the start date and end dates from the date filter. - -> You can view Audit Logs for a maximum of 60 days. Please contact support for higher retention period (Supported in Enterprise Plan). - -#### Filter By User - -Based on the given date range, you can review the logs for each individual user in your company as well as for all users at once. - -### Download your Audit Logs - -Click on the **Export Button** displayed in the screenshot below, and your logs will be downloaded in **.csv** format. - -## FAQs - -**Q: Is comprehensive audit logging for all system and user activities supported?** - - Yes, it provides comprehensive audit logging of all activities, ensuring full traceability across conversations, workflows, operations, test executions, and database actions. It maintains detailed audit trails with timestamps, metadata, and historical records, supporting compliance, security, and validation of AI agent capabilities. - -**Key capabilities include:** - -- Conversation history tracking, including timestamped conversations, message metadata, file upload and download activity, session tracking, and complete conversation audit trails. - -- Workflow state tracking with full state persistence, state change history, metadata preservation, and workflow execution audit trails. - -- Operation logging for all API operations, including error tracking, performance metrics logging, and operation-level audit records. - -- Test execution logging covering test scenario execution, test result tracking, evaluation result logging, and audit logging as a supported test category for agent validation. - -- Database audit trail with timestamped database operations, organization-level data isolation, user-level activity tracking, and long-term historical data preservation. diff --git a/static/docs/authentication.md b/static/docs/authentication.md deleted file mode 100644 index e313edc1a..000000000 --- a/static/docs/authentication.md +++ /dev/null @@ -1,66 +0,0 @@ -# Authentication - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Authenticate your Cypress test runs in the following ways - - -1. Set up the **environment variables**. (or) -2. Utilizing the **CLI params**. (or) -3. Mention your **username** and **access key** in the **lambdatest-config.json**. - -**Note** -We use the following order of precedence to determine which auth credentials to use if you use more than one option to pass your auth credentials: -CLI arguments > Options set in lambdatest-config.json > Environment variables - -## Utilizing CLI Params: - -The following args can be used while running tests using the run command. - -| Arg | Shorthand | Accepted values | -| ---------- | --------- | -------------------------- | -| --username | --user | Your TestMu AI username | -| --access_key | --ak | Your TestMu AI access key | - -For example - - - {`lambdatest-cypress run --username=${ YOUR_LAMBDATEST_USERNAME()} --ak=${ YOUR_LAMBDATEST_ACCESS_KEY()}`} - -## Using lambdatest-config.json: - -The auth option will help you in specifying your username and access key. You can find your username and access key in the [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build). Both, the auth credentials set in environment variables and the ones mentioned in the lambdatest-config.json file will get overridden. - -The options supported in the auth are as follows: - -| Arg | Accepted values | -| -------------- | -------------------------- | -| **username** | Your TestMu AI username | -| **access_key** | Your TestMu AI access key | - -For example - - -```json title="lambdatest-config.json" -"lambdatest_auth": { -"username": "", -"access_key": "" -}, -``` - -## Setup the Environment Variables: - -While utilizing the CLI params, you can set up the following environment variables. - -| Env variable | Accepted values | -| ----------------- | -------------------------- | -| **LT_USERNAME** | Your TestMu AI username | -| **LT_ACCESS_KEY** | Your TestMu AI access key | - -Or you can also set environment variables using following commands: - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" \\ -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" \` -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -**Note** -By doing so, the auth credentials you use in your **lambdatest-config.json** file will get overridden only if these options are not provided in **lambdatest-config.json** file. diff --git a/static/docs/author-your-first-desktop-browser-test.md b/static/docs/author-your-first-desktop-browser-test.md deleted file mode 100644 index 12dcd547f..000000000 --- a/static/docs/author-your-first-desktop-browser-test.md +++ /dev/null @@ -1,28 +0,0 @@ -# Author Your First Desktop Browser Test - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -In this guide, we'll walk you through how you can author your desktop browser tests using KaneAI - -## Steps to Author Your Desktop Browser Test - -**Step 1:** From the KaneAI dashboard, click on **Author Browser Test**. - -**Step 2:** A modal would appear from the side. Select **Desktop** from the sidebar. - -**Step 3:** You can also do additional customization with some advanced features such as: - -- **Chrome Options:** Command-line switches that alter the default behavior of the Chrome browser. -- **Custom Headers:** Kееp crucial information about thе rеquеst or rеsponsе, such as thе mеthod, URL, and body contеnt. - -Once done, click **Author Test.** - -**Step 4:** Now tell KaneAI what you want to author using natural language without the need for coding or scripting knowledge required. KaneAI translates your instructions into executable test steps automatically. - -**Step 5:** You can also leverage the **Manual Interaction Mode** to perform any particular step or action directly in the instance window and KaneAI would record it and create an automation step for the same. - -As you add each instruction, KaneAI queues them in real-time without waiting for completion. This allows you to build comprehensive test flows quickly and efficiently. Once you've outlined all your test steps, click Finish Test to finalize your automated test case. - -**Step 6:** Now, select your **Project** and the folder where you want to save your test cases. The **Test Name** and **Description** will be generated automatically, but you can also edit them. Select your test's **Type** and **Status**, as well as the necessary tags that will allow you to search for it later. - -**Step 7:** To execute your test, click on the **Execute test case** button and you will be redirected to the HyperExecute dashboard where your tests will be executed. diff --git a/static/docs/author-your-first-mobile-app-test.md b/static/docs/author-your-first-mobile-app-test.md deleted file mode 100644 index 27068c342..000000000 --- a/static/docs/author-your-first-mobile-app-test.md +++ /dev/null @@ -1,27 +0,0 @@ -# Author Your First Mobile App Test - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -In this guide, we'll walk you through how you can author your mobile app tests using KaneAI - -## Steps to Author Your Mobile App Test - -**Step 1:** From the KaneAI dashboard, click on **Author App Test**. - -**Step 2:** A modal would appear from the side. Upload the app you want to test. - -**Step 3:** Choose your preferred device and the OS version. - -**Step 4:** You can also select advanced configurations from the Advanced Settings option. - -**Step 5:** Once done, click on **Start Testing** to get started. - -**Step 6:** Now tell KaneAI what you want to author using natural language without the need for coding or scripting knowledge required. KaneAI translates your instructions into executable test steps automatically. - -**Step 7:** You can also leverage the **Manual Interaction Mode** to perform any particular step or action directly in the instance window and KaneAI would record it and create an automation step for the same. - -As you add each instruction, KaneAI queues them in real-time without waiting for completion. This allows you to build comprehensive test flows quickly and efficiently. Once you've outlined all your test steps, click Finish Test to finalize your automated test case. - -**Step 8:** Now, select your **Project** and the folder where you want to save your test cases. The **Test Name** and **Description** will be generated automatically, but you can also edit them. Select your test's **Type** and **Status**, as well as the necessary tags that will allow you to search for it later. - -**Step 9:** To execute your test, click on the **Execute test case** button and you will be redirected to the HyperExecute dashboard where your tests will be executed. diff --git a/static/docs/author-your-first-mobile-browser-test.md b/static/docs/author-your-first-mobile-browser-test.md deleted file mode 100644 index b8d23393a..000000000 --- a/static/docs/author-your-first-mobile-browser-test.md +++ /dev/null @@ -1,40 +0,0 @@ -# Author Your First Mobile Browser Test - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -In this guide, we'll walk you through how you can author your mobile browser tests using KaneAI - -## Steps to Author Your Mobile Browser Test - -**Step 1:** From the KaneAI dashboard, click on **Author Browser Test**. - -**Step 2:** A modal would appear from the side. Select **Mobile** from the sidebar. - -**Step 3:** Now, choose your preferred OS (Android or iOS), browser, device and the OS version. - -**Step 4:** You can also toggle on advanced settings such as network throttling or auto-login through Google from the Advanced Settings option. - -**Step 5:** Once done, click on **Author Test**. - -**Step 6:** Now tell KaneAI what you want to author using natural language without the need for coding or scripting knowledge required. KaneAI translates your instructions into executable test steps automatically. - -As you add each instruction, KaneAI queues them in real-time without waiting for completion. This allows you to build comprehensive test flows quickly and efficiently. Once you've outlined all your test steps, click Finish Test to finalize your automated test case. - -**Step 7:** Now, select your **Project** and the folder where you want to save your test cases. The **Test Name** and **Description** will be generated automatically, but you can also edit them. Select your test's **Type** and **Status**, as well as the necessary tags that will allow you to search for it later. - -**Step 8:** To execute your test, click on the **Execute test case** button and you will be redirected to the HyperExecute dashboard where your tests will be executed. - -## Feature Support Matrix -| Feature / Action | Support Status | Notes | -|-----------------------------------------------------------------------|-----------------------|-----------------------------------------------------------------------------------------| -| Element Scroll | Supported | | -| Key Events (e.g. Enter, Space, Tab) | Not Supported | | -| Clear Input Fields | Supported | | -| App Instructions (e.g. background, hide keyboard, show notifications) | Not Supported | | -| Textual & Visual Queries | Supported | Includes driver-based queries. | -| Sites with Permission Popups (notifications, cookies, location) | Supported (Android only) | Popups are auto-accepted on Android. iOS not supported currently. | -| Cookies & Local Storage | Supported | | -| Network Assertions | Not Supported | Not yet available for mobile web; only supported on web. | -| Negative Scenarios from App UX (e.g. switch app, go to home, go back) | Not Supported | These are app-specific and not currently available in browser testing mode. | | - -> _Have any feedback or request? Reach out to us via support@testmuai.com and we would be happy to hear from you._ diff --git a/static/docs/auto-healing.md b/static/docs/auto-healing.md deleted file mode 100644 index ef8fb4a3c..000000000 --- a/static/docs/auto-healing.md +++ /dev/null @@ -1,283 +0,0 @@ -# Auto Healing for Selenium Web Automation - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The TestMu AI Auto Healing feature for Selenium testing automatically recovers from certain types of failures during test execution. When enabled, it reduces test flakiness and improves test reliability by handling unexpected situations and errors in your test suites. - -## Enabling Auto Healing - -Pass the `autoHeal: true` capability in your WebDriver configuration to enable this feature. - -```js -const capability = { -"browserName": "Chrome", -"browserVersion": "114.0", -"LT:Options": { -"platformName": "Windows 10", -"project": "Untitled", -"w3c": true, -"plugin": "node_js-node_js", -"autoHeal": true -} -} -``` -> **Warning:** The `autoHeal` capability only works when `smartWait` is **disabled**. Both features cannot be enabled together in the same test session. - -No prerequisites are required. Enable auto-healing directly via desired capabilities. - -### Language Preferences - -For **Java**, use the following code: - -```java -ChromeOptions browserOptions = new ChromeOptions(); -browserOptions.setBrowserVersion("118.0"); -HashMap ltOptions = new HashMap(); -ltOptions.put("username", "YOUR_LAMBDATEST_USERNAME"); -ltOptions.put("accessKey", "YOUR_LAMBDATEST_ACCESS_KEY"); -ltOptions.put("project", "Untitled"); -ltOptions.put("selenium_version", "4.0.0"); -ltOptions.put("w3c", true); -// highlight-next-line -ltOptions.put("autoHeal", true); -browserOptions.setCapability("LT:Options", ltOptions); -``` - -For **NodeJS**, use the following code: - -```javascript -const capability = { -"browserName": "Chrome", -"browserVersion": "118.0", -"LT:Options": { -"username": "YOUR_LAMBDATEST_USERNAME", -"accessKey": "YOUR_LAMBDATEST_ACCESS_KEY", -"project": "Untitled", -"w3c": true, -"plugin": "node_js-node_js", -// highlight-next-line -"autoHeal": true -} -} -``` - -For **C#**, use the following code: - -```csharp -ChromeOptions capabilities = new ChromeOptions(); -capabilities.BrowserVersion = "118.0"; -Dictionary ltOptions = new Dictionary(); -ltOptions.Add("username", "YOUR_LAMBDATEST_USERNAME"); -ltOptions.Add("accessKey", "YOUR_LAMBDATEST_ACCESS_KEY"); -ltOptions.Add("project", "Untitled"); -ltOptions.Add("w3c", true); -ltOptions.Add("plugin", "c#-c#"); -// highlight-next-line -ltOptions.Add("autoHeal", true); -capabilities.AddAdditionalOption("LT:Options", ltOptions); -``` - -For **PHP**, use the following code: - -```php -$capability = array( -"browserName" => "Chrome", -"browserVersion" => "118.0", -"LT:Options" => array( -"username" => "YOUR_LAMBDATEST_USERNAME", -"accessKey" => "YOUR_LAMBDATEST_ACCESS_KEY", -"project" => "Untitled", -"w3c" => true, -"plugin" => "php-php", -// highlight-next-line -"autoHeal" => true -) -); -``` - -For **Python**, use the following code: - -```python -options = ChromeOptions() -options.browser_version = "118.0" -lt_options = {} -lt_options["username"] = "YOUR_LAMBDATEST_USERNAME" -lt_options["accessKey"] = "YOUR_LAMBDATEST_ACCESS_KEY" -lt_options["project"] = "Untitled" -lt_options["w3c"] = True -lt_options["plugin"] = "python-python" -# highlight-next-line -lt_options["autoHeal"] = True -options.set_capability('LT:Options', lt_options) -``` - -For **Ruby**, use the following code: - -```ruby -options = Selenium::WebDriver::Options.chrome -options.browser_version = "118.0" -lt_options = {} -lt_options[:username] = "YOUR_LAMBDATEST_USERNAME" -lt_options[:accessKey] = "YOUR_LAMBDATEST_ACCESS_KEY" -lt_options[:project] = "Untitled" -lt_options[:w3c] = true -lt_options[:plugin] = "ruby-ruby" -# highlight-next-line -lt_options[:autoHeal] = true -options.set_capability('LT:Options', lt_options) -``` - -## How Auto Healing Works - -Selenium Locator auto-healing adjusts locators by merging attributes and context to handle inconsistent tests. During runtime, it monitors the web page to identify DOM (Document Object Model) changes. - -When an element is successfully located on the page, its DOM path is recorded for later use. If that same element is later referenced on the same page and is missing, the system evaluates the current page and generates new locators for altered elements based on previous benchmarks. - -## Auto Detection of New Locator - -Web elements might change their locators due to updates in the web application. The Auto Healing feature automatically detects the new locator and continues the test execution. - -Here is an example test case demonstrating this: - -```js -import assert from 'assert'; -import { Builder, By, until, Capabilities } from 'selenium-webdriver'; - -describe('Amazon Search Box Test', function () { -this.timeout(30000); -let driver; -let vars; - -const capability = { -"browserName": "Chrome", -"browserVersion": "114.0", -"LT:Options": { -"platformName": "Windows 10", -"project": "Untitled", -"w3c": true, -"plugin": "node_js-node_js", -"autoHeal": true -} -} - -beforeEach(async function () { -driver = await new Builder() -.usingServer('https://YOUR_LAMBDATEST_USERNAME:YOUR_LAMBDATEST_ACCESS_KEY@hub.lambdatest.com/wd/hub') -.withCapabilities(capability) -.build(); -vars = {}; -}); - -afterEach(async function () { -await driver.quit(); -}); - -it('should change id of search box and find element', async function () { -await driver.get('https://www.amazon.com'); -const searchBoxActual = await driver.findElement(By.id('nav-search-submit-button')); -await driver.executeScript("document.getElementById('nav-search-submit-button').id='amazonsearchbox'"); -// const searchBox = await driver.findElement(By.id('amazonsearchbox')); -const searchBoxHeal = await driver.findElement(By.id('nav-search-submit-button')); -assert(searchBoxHeal, 'Element not found'); -}); -}); - -``` - -In the above test case, the script changes the *id* of the search box on Amazon's homepage and then tries to find the element using the old *id*. The Auto Healing feature automatically detects the new *id* and finds the element. - -To run the test, execute the below command: - -```bash -./node_modules/.bin/mocha autohealingTest.js -``` - -## Using Auto Heal with Hooks - -You can start or stop Auto Heal at any point in your test script using hooks. This gives you fine-grained control over when element healing should be applied. - -### Enable Auto Heal - -Use the following hook to **start** Auto Heal at any point in your test script. - -```javascript -driver.execute_script('lambdatest_executor:{"action":"lambda-heal-start"}') -``` - -**Usage:** Place this hook right before interacting with elements that may dynamically change during the test. - -### Disable Auto Heal - -Use the following hook to **stop** Auto Heal at any point in your test script. - -```javascript -driver.execute_script('lambdatest_executor:{"action":"lambda-heal-stop"}') -``` - -**Usage:** Place this hook immediately after the actions requiring Auto Heal are completed. This ensures subsequent test steps execute with normal Selenium behavior. - -### Sample Script - -```python title="Test.py" -import os -import time -from selenium import webdriver -from selenium.webdriver.common.by import By -from selenium.webdriver.chrome.options import Options - -options = Options() -options.browser_version = "latest" -lt_options = { -"build": "Autoheal Hooks Build", -"name": "Autoheal Test via Hooks", -"platformName": "Windows 10", -"w3c": True, -"autoHeal": True -} -options.set_capability("LT:Options", lt_options) - -hub = f"https://{os.getenv('LT_USERNAME')}:{os.getenv('LT_ACCESS_KEY')}@hub.lambdatest.com/wd/hub" -driver = webdriver.Remote(command_executor=hub, options=options) - -driver.get("https://www.selenium.dev") -time.sleep(2) - -# Start auto-heal before interacting with dynamic elements -driver.execute_script('lambdatest_executor:{"action":"lambda-heal-start"}') - -driver.find_element(By.ID, 'td-block-1') -driver.execute_script("document.getElementById('td-block-1').id='updatedtd-block-1';") -driver.find_element(By.ID, 'td-block-1') # Auto-heal detects the changed ID - -# Stop auto-heal after dynamic section -driver.execute_script('lambdatest_executor:{"action":"lambda-heal-stop"}') - -driver.quit() -``` - -## Benefits of Auto Healing - -- **Increased Test Stability:** Tests remain consistent even when the web application's UI undergoes minor changes, reducing flakiness. -- **Reduced Test Maintenance:** The system automatically adapts to evolving interfaces, reducing the manual effort required to update test scripts. -- **Reliable CI Pipeline:** Stable tests feeding into CI pipelines reduce unexpected failures and ensure smoother deployments. - -## Limitations of Auto Healing - -While the Auto Healing feature handles a wide range of issues, there are certain limitations to be aware of. - -* **Non-recoverable errors**: Auto Healing cannot recover from certain types of errors, such as WebDriver initialization errors or system-level failures. - -* **Test accuracy**: While Auto Healing reduces test flakiness, it may also mask real issues in your web application or test scripts. Review the logs and understand why a test needed healing. - -* **Performance impact**: While typically minimal, enabling Auto Healing can have a slight impact on test execution time due to additional checks and recovery mechanisms. - -The Auto Healing feature enhances your test suite, but it does not replace good test design and error handling practices. Always ensure your tests are well-designed, have proper error handling in place, and are reviewed regularly for issues that may be masked by the Auto Healing feature. - -## Related Docs - -- [Playwright Auto Healing](/support/docs/playwright-auto-healing/):Auto-heal for Playwright web automation tests -- [Smart Heal for Appium](/support/docs/smart-heal-appium/):AI-powered self-healing for mobile app automation on real devices -- [Auto Healing in HyperExecute](/support/docs/hyperexecute-auto-healing/):Auto-heal for tests executed via HyperExecute -- [Auto-Heal in KaneAI](/support/docs/kaneai-auto-heal/):AI-native auto-heal with natural language understanding for KaneAI-authored tests - -> If you have any questions, please feel free to let us know. Our experts are always available on **chat** to help you out with any roadblock regarding our product. Happy testing! diff --git a/static/docs/autoheal-with-hooks.md b/static/docs/autoheal-with-hooks.md deleted file mode 100644 index 508ae3900..000000000 --- a/static/docs/autoheal-with-hooks.md +++ /dev/null @@ -1,95 +0,0 @@ -# How to Use AutoHeal Using Hooks - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Autoheal automatically handles dynamic element changes during Selenium test execution. This is especially useful for tests where elements' IDs, XPaths, or attributes may change between runs. Autoheal ensures your tests are more resilient and reduces flakiness. - -Autoheal can be started or stopped at any point in your test script using hooks. This allows for precise control over when your tests should attempt element healing. This guide explains how to enable and disable Autoheal in your Selenium scripts using the TestMu AI hooks. - -## Using Autoheal Hooks in Tests - -Insert Autoheal hooks anywhere in the script depending on where you need dynamic element handling. - -- Navigate to your test page. -- [Start Autoheal](/support/docs/autoheal-with-hooks/#enable-autoheal) before interacting with elements that may change. -- Perform element actions (find, click, send keys, etc.). -- [Stop Autoheal](/support/docs/autoheal-with-hooks/#disable-autoheal) after these actions. -- Continue the rest of your test normally. - -This gives you fine-grained control over Autoheal, improving test stability without affecting unrelated test steps. - -## Enable AutoHeal - -Use the following hook to **start** Autoheal at any point in your Selenium test script. - -```javascript -driver.execute_script('lambdatest_executor:{"action":"lambda-heal-start"}') -``` - -**Usage:** Place this hook right before interacting with elements that may dynamically change during the test. - -## Disable AutoHeal - -Use the following hook to **stop** Autoheal at any point in your Selenium test script. - -```javascript -driver.execute_script('lambdatest_executor:{"action":"lambda-heal-stop"}') -``` - -**Usage:** Place this hook immediately after the actions requiring Autoheal are completed. This ensures subsequent test steps execute with normal Selenium behavior. - -## Sample Script - -Review the following sample script to see how Autoheal hooks work in practice. - -```python title="Test.py" -import os -import time -from selenium import webdriver - -platform = os.getenv('HYPEREXECUTE_PLATFORM') - -desired_cap_chrome = { -"build" : "Lambdatest Build 2", -"name" : "Autoheal Test via Hooks", -"platform" : "Windows 10", -"browserName" : "Chrome", -"version" : "latest", -"visual" : False, -"video" : True, -"network": True, -"console": True, -'goog:chromeOptions': {'args': ['--window-size=400x300']}, -} - -hub = f"https://{os.getenv('LT_USERNAME')}:{os.getenv('LT_ACCESS_KEY')}@hub.lambdatest.com/wd/hub" - -def OneTest(caps, sleepTime): -begin = time.time() -driver = webdriver.Remote(command_executor=hub, desired_capabilities=caps) -end = time.time() -print(f"Startup time for {caps['browserName']} {end - begin} , {begin} , {end}") -driver.get("https://www.selenium.dev") -time.sleep(sleepTime) - -driver.execute_script('lambdatest_executor:{"action":"lambda-heal-start"}') #start hook for autoheal - -driver.find_element_by_id('td-block-1') -driver.execute_script("document.getElementById(\"td-block-1\").id=\"updatedtd-block-1\";") -driver.find_element_by_id('updatedtd-block-1') -driver.find_element_by_id("td-block-1") - -driver.execute_script('lambdatest_executor:{"action":"lambda-heal-stop"}') #stop hook for autoheal - -driver.execute_script("console.log('Test started');") -driver.execute_script("console.info('Navigated to selenium.dev');") -driver.execute_script("console.warn('This is a warning message');") -driver.execute_script("console.error('This is an error message');") -driver.execute_script("console.debug('Debugging test flow');") -driver.execute_script("console.log('Test completed successfully');") -driver.quit() - -OneTest(desired_cap_chrome, 2) -``` - -With these steps, you can integrate TestMu AI HyperExecute Autoheal into your Selenium tests, reducing flaky tests and improving stability. diff --git a/static/docs/automated-screenshot-api-for-cross-browser-testing.md b/static/docs/automated-screenshot-api-for-cross-browser-testing.md deleted file mode 100644 index d96de7e18..000000000 --- a/static/docs/automated-screenshot-api-for-cross-browser-testing.md +++ /dev/null @@ -1,329 +0,0 @@ -# Automated Screenshot API For Cross Browser Testing - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Automated Screenshot API will help you [capture full-page screenshots](https://www.testmuai.com/full-page-screen-capture) of any URL by invoking an API call to TestMu AI cloud servers. You can ensure cross browser compatibility of your website stays in check by performing bulk screenshot testing regularly. You can do that without logging into TestMu AI, by executing a JSON file in your preferred API development environment such as Postman. - -This document will help you leverage TestMu AI [Automated Screenshot](https://www.testmuai.com/automated-screenshot). You would learn how to: - -* Fetch all OS + Browser combinations. -* Fetch all Screen Resolutions. -* Fetch Screenshots of an already performed test session. -* Start a Screenshot Test Session - 1. Test your locally hosted web pages. - 2. Test web pages with basic authentication. - 3. Test web pages with a Callback URL. -- Stop a Screenshot test. - -## Prerequisites - -- **TestMu AI Authentication Credentials** -Ensure that you have your TestMu AI authentication credentials, i.e., your TestMu AI username, access key, and HubURL. You need to set them up as your environment variables. You can retrieve them from your [TestMu AI automation dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com) by clicking on the key icon near the help button. - -- For Linux/Mac: - -$ export LT_USERNAME= `{YOUR_LAMBDATEST_USERNAME}` - -$ export LT_ACCESS_KEY= `{YOUR_LAMBDATEST_ACCESS_KEY}` - -- For Windows: - -$ set LT_USERNAME= `{YOUR_LAMBDATEST_USERNAME}` - -$ set LT_ACCESS_KEY= `{YOUR_LAMBDATEST_ACCESS_KEY}` - -Open API Document For Automated Screenshot Testing - -## Fetch All OS + Browser Combinations - -The below **GET** request will help you extract a list of all the combinations for (OS + Browser) offered by TestMu AI. You can download these combinations as a JSON file. - -**Requested URL:** - -```javascript -https://api.lambdatest.com/screenshots/v1/os-browsers -``` - -Client URL would be as followed: - -```javascript -curl -X GET "https://api.lambdatest.com/screenshots/v1/os-browsers" --H "accept: application/json" -H "Authorization: Basic 123456789abcdefghijklmno=" -``` - -As you execute, you will find the Server response presented below. - -```javascript -{ -"windows 10": { -"chrome": [ -"74", -"73" -], -"firefox": [ -"66", -"65" -], -"opera": [ -"55", -"54" -] -}, -"macos mojave": { -"chrome": [ -"74", -"73" -], -"firefox": [ -"66", -"65" -], -"opera": [ -"55", -"54" -] -} -} -``` - -## Fetch Every Screen Resolutions - -To get a list of all the available **screen resolutions** provided by TestMu AI for you to perform cross browser testing, you would need to execute the below **GET** request. The GET request will help you extract the respective response from our cloud servers in a JSON format. - -**Requested URL:** - -```javascript -https://api.lambdatest.com/screenshots/v1/resolutions -``` -Client URL would be as followed: - -```javascript -curl -X GET "https://api.lambdatest.com/screenshots/v1/resolutions" --H "accept: application/json" -H "Authorization: Basic 123456789abcdefghijklmno=" -``` -As you execute, you will find the Server response presented below. - -```javascript -{ -"mac_res": [ -"1024x768", -"1280x1024" -], -"win_res": [ -"1024x768", -"1280x1024" -] -} -``` -## Fetch Details Of An Executed Test Session - -In case you wish to fetch all the details from an already executed screenshot test session. These details will help you analyze the results of your [screenshot test session](/support/docs/automated-screenshot-testing/) at TestMu AI. You would get the below details: - -* **Test Details**: - * Test ID - * Test status - * Defer time - * Tested URL - * Callback URL(if any) -* **Screenshot Details**: - * Operating System - * Browser - * Browser Version - * Status - * Screenshot URL - * Thumbnail URL - * Activity ID - * Resolution - -**Requested URL:** - -```javascript -https://api.lambdatest.com/screenshots/v1/Test ID -``` - -> **Note:** Test ID is a mandatory field for making use of this GET request. This would fetch details in JSON format for all bulk screenshots captured in a test session specified by you. - -Client URL would be: - -```javascript -curl -X GET "https://api.lambdatest.com/screenshots/v1/TES11111111111111111111111111" --H "accept: application/json" -H "Authorization: Basic 123456789abcdefghijklmno=" -``` - -Example Value: - -```javascript -{ -"test_id": "TES11111111111111111111111111", -"defer_time": 5, -"test_status": "completed", -"url": "https://www.testmuai.com", -"callback_url": "https://www.example.com", -"screenshots": [ -{ -"os": "windows 10", -"browser": "chrome", -"browser_version": "74", -"status": "completed", -"screenshot_url": "https://s3.amazonaws.com/s3-screenshots/prod/ACT100968331562237283340194/screenshot/win-cr-74.0.png", -"thumbnail_url": "https://s3.amazonaws.com/s3-screenshots/prod/ACT100968331562237283340194/screenshot/win-cr-74.0.png", -"activity_id": "ACT100968331562237283340194", -"resolution": "1024x768" -} -] -} -``` - -## Run A Screenshot Test - -To start a screenshot test using the API, you would need to trigger the below **POST** request. - -```javascript -https://api.lambdatest.com/screenshots/v1 -``` - -Client URL would be as followed: - -```javascript -curl -X POST "https://api.lambdatest.com/screenshots/v1" -H "accept: application/json" --H "Authorization: Basic 123456789abcdefghijklmno=" -H "Content-Type: application/json" -d " -{\"url\":\"https://www.testmuai.com\",\"defer_time\":5,\"email\":true,\"mac_res\":\"1024x768\", -\"win_res\":\"1366X768\",\"configs\":{\"windows 10\":{\"chrome\":[\"74\"],\"firefox\":[\"66\"], -\"opera\":[\"58\"],\"ie\":[\"11\"]},\"macos mojave\":{\"chrome\":[\"74\"]}}}" -``` -Example value: - -```javascript -{ -"url": "https://www.testmuai.com", -"defer_time": 5, -"email": true, -"mac_res": "1024x768", -"win_res": "1366X768", -"configs": { -"windows 10": { -"chrome": [ -"74" -], -"firefox": [ -"66" -], -"opera": [ -"58" -], -"ie": [ -"11" -] -}, -"macos mojave": { -"chrome": [ -"74" -] -} -} -} -``` - -### Screenshot Testing For Locally Hosted Web Applications - -To perform a screenshot test on your locally hosted web applications, you would need to declare the below parameter in your request before specifying the configurations: - -`"tunnel": true` - -Example Value: - -```javascript -{ -"url": "https://www.testmuai.com", -"defer_time": 5, -"email": true, -"tunnel": true, -"mac_res": "1024x768", -"win_res": "1366X768", -"configs": { -"windows 10": { -"chrome": [ -"74", -"73" -] -} -} -} -``` -### Screenshot Test With Basic Authentication - -Similarly, to perform a screenshot test on a web application with basic authentication, you would need to declare the below parameter in your request before specifying the configurations. - -```javascript -"url": "https://www.basic-auth.com", -"username": "username", -"password": "password", -``` - -Example: - -```javascript -{ -"url": "https://www.basic-auth.com", -"defer_time": 5, -"email": true, -"mac_res": "1024x768", -"win_res": "1366X768", -"username": "username", -"password": "password", -"configs": { -"windows 10": { -"chrome": [ -"74", -"73" -] - -} -} -} -``` - -### Route Test Details To Your Colleagues Through A Callback URL - -By specifying a Callback URL, you can route your test details to a specific IP or a specific URL. This would help you to share your test results with your colleagues or to keep them on a specific URL, so you could come back and assess these results when you have time. To invoke a Callback URL, you would need to declare the below parameter in your request before specifying the configurations. - -```javascript -"callbackurl": "https://www.example.com", -``` - -Example Value: - -```javascript -{ -"url": "https://www.testmuai.com", -"defer_time": 5, -"email": true, -"mac_res": "1024x768", -"win_res": "1366X768", -"callbackurl": "https://www.example.com", -"configs": { -"windows 10": { -"chrome": [ -"74", -"73" -] -} -} -} -``` -## Stop A Screenshot Test - -In case you wish to stop a screenshot test, you can do so with the help of a PUT request. Test ID is again mandatory over here as you need to convey which particular screenshot test you would want to be aborted. - -**Requested URL:** - -```javascript -https://api.lambdatest.com/screenshots/v1/stop/TES11111111111111111111 -``` - -```javascript -Client URL: curl -X PUT "https://api.lambdatest.com/screenshots/v1/stop/TES100962431563281759448763" --H "accept: application/json" -H "Authorization: Basic 123456789abcdefghijklmno=" -``` - -> **Note:** This will terminate only a running test session. Providing a TestID of an already completed test session will provide you with a 404 error as the server response. diff --git a/static/docs/automated-screenshot-testing.md b/static/docs/automated-screenshot-testing.md deleted file mode 100644 index bc853f967..000000000 --- a/static/docs/automated-screenshot-testing.md +++ /dev/null @@ -1,77 +0,0 @@ -# Automated Screenshot Testing - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -> This feature is **deprecated** and as a result, no further support will be provided for this feature by the TestMu AI Organization. - -Automated Screenshot testing will help you capture screenshots in bulk through different desktops and mobile devices running on various OS in a single go! It is facilitated by instrumented UI test cases to help you realize how seamlessly your web app renders through different screen sizes using different browsers. At TestMu AI, you can capture up to 25 screenshots of different browsers and OS configurations in a single session. You can choose from a wide range of latest to oldest browser versions of Chrome, Firefox, Safari, Edge, Internet Explorer & Opera. Also, you can test on different operating systems Windows, macOS, Android, and iOS. - -## How To Do Screenshot Testing? - -**Step 1:** Click Screenshot in the Visual UI Testing menu. - -**Step 2:** Enter your URL & select the browser-version and OS combinations on which you want to generate a screenshot. Click on the capture, and you will be taken to the result page, where you can view your screenshots. You can also select mobile devices by clicking on the mobile icon on the top left. - -## How To Download Screenshots? - -**Step 1:** If you want to download particular screenshots, you will find a download icon near every screenshot generated. Just click on it to download. - -**Step 2:** If you want to download all the screenshots, you will find the Download all (zip) option at the top center. Click on it to download all the screenshots in zip format. - -## How To View Recent Screenshot Testing Sessions? - -**Step 1:** Click the Screenshot submenu under the Visual UI Testing menu in the menu box. On the screenshot page, you will find the recent view button on the page. Click that button. - -**Step 2:** On the recent session page, you can select the desired recent session from the sessions listed. - -## How To Change Preference For Screenshot Testing Sessions? - -**Step 1:** Select the Screenshot from the Visual UI Testing from the menu bar. - -**Step 2:** Click on the gear icon on the right side of the screenshot page. After clicking on it, a pop-up will appear on the screen. Click the given options to change the desired screenshot settings and then finally click Apply. - -- OSX resolution: Choose the various resolution for your OS X machine. - -- Windows resolution: Choose various resolutions for the Windows machine. - -- Quality: You can either choose the high-quality version or a compressed image as per your requirements. - -- Mobile Layout: If you are taking screenshots on mobile devices, you can choose mobile layout as portrait or in landscape mode. - -- Defer Time: If your page takes time to load completely, you can set a defer time so that there's ample time for each element to load - -- Smart Scroll: You can ignore all the iframes on your page by choosing the smart scroll - -- Email me on completion: You can choose to get an email after the tests are done. You can download all the screenshots from there. - -## How To Schedule Your Screenshot Tests? - -**Step 1:** Select the Screenshot from the Visual UI Testing from the menu bar. Select the browsers you want to perform the tests on and enter your website name. Click on the schedule button on the right side of the screenshot page. - -**Step 2:** Once you click on the schedule button, a pop-up will appear on the screen. - -**Step 3:** Enter the desired time, frequency, and the day you want to schedule your screenshot for. After this, click on the schedule button, and your screenshots would be scheduled! - -## How To Share The Screenshots With Your Team? - -**Step 1:** Select the Screenshot from the Visual UI Testing from the menu bar. After that, click on the recent sessions button. - -**Step 2:** On the recent session page - select the recent session you want to share. - -**Step 3:** On the next screen, you can see the screenshots on all the devices. Here click on the share screenshots button. - -**Step 4:** A pop-up will appear, where you can enter the email-ids you want to share the screenshots with, along with any note you might want to add. You can even set the expiry time for the screenshots. - -## How To Use Basic Authentication For Password Protected Websites - -If the website you want to take screenshots on requires basic authentication, you can click on the basic authentication button and enter your login details. After that, click on Save, the basic authentication would be enabled, and you can take screenshots easily. - -## How To Save Your Browser Combinations - -You can save your favorite mobile and desktop browsers as a list to use them again. You can create up to three different lists for different browser combinations. Just chose your favorite browsers and click on the star icon. - -In case you want to take a screenshot of a page behind the login, you can refer to this support doc on [Taking Screenshots of Pages Behind Login](/docs/taking-screenshots-of-pages-behind-login/). - ->Similarly, you can perform Mobile Screenshot testing on Android emulators and iOS simulators by navigating to the mobile tab . Check [test on mobile devices](https://www.testmuai.com/test-on-mobile-devices). - -That was all you need to know for performing automated screenshot testing on TestMu AI. If you still have any questions for us, please feel free to reach us. Our experts are always available on chat to help you out with any roadblock regarding our product. Happy testing! diff --git a/static/docs/automated-test-cases-linked-using-capability.md b/static/docs/automated-test-cases-linked-using-capability.md deleted file mode 100644 index e60df8540..000000000 --- a/static/docs/automated-test-cases-linked-using-capability.md +++ /dev/null @@ -1,52 +0,0 @@ -# Linking Automated Test Cases to Test Manager - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Test Manager supports linking automated test runs directly to specific test cases using Test Case IDs. This feature enables better traceability and management of your automated testing workflow. - -> Currently, this feature is supported on [HyperExecute](/support/docs/getting-started-with-hyperexecute/) and Web Automation tests. - -## How to Link a Test Case ID - -To link an automated test run with a specific test case, add the `tms.tc_id` key in the `lt:Options` section of your test capabilities. Here is a sample configuration demonstrating how to include the `tms.tc_id` key in your script: - -```javascript -const capabilities = { -"lt:Options": { -"project": "Your Project Name", // Specify the project where the test run should be created -"tms.tc_id": "TC-1470" // Link the test execution to the Test Case ID 'TC-1470' -} -}; -``` - -- `lt:Options` : A JSON object containing additional options for TestMu AI configurations. -- `project` : The name of the project in Test Manager where the test run should be created. If not specified, the test run will be created under **LambdaTest Default Project**. -- `tms.tc_id` : The key used to link a test case in Test Manager. Replace "TC-1470" with your desired Test Case ID. - -## Specifying Target Project - -When you link a test case using `tms.tc_id`, a test run is automatically created with your build name. By default, this test run is created under **LambdaTest Default Project**. - -To ensure the test run is created in the correct project, use the `project` capability along with `tms.tc_id`: - -```javascript -const capabilities = { -"browserName": "Chrome", -"browserVersion": "latest", -"lt:Options": { -"platform": "Windows 10", -"build": "Playwright Build", -"name": "Sample Test", -"project": "Demo-Project", // Target project name -"tms.tc_id": "TC-95668" // Test case ID from the target project -} -}; -``` - -The `project` name must match exactly as it appears in Test Manager. If the project name is incorrect or doesn't exist, the test run will be created under **LambdaTest Default Project**. - -**NOTE** -- Ensure the Test Case ID exists in Test Manager before linking. -- The Test Case ID format should match exactly as shown in Test Manager -- Each automated test run can be linked to one test case at a time -- The test case specified in `tms.tc_id` should belong to the project specified in `project` capability diff --git a/static/docs/automated-test-cases-linked-using-dashboard.md b/static/docs/automated-test-cases-linked-using-dashboard.md deleted file mode 100644 index 008dfc98c..000000000 --- a/static/docs/automated-test-cases-linked-using-dashboard.md +++ /dev/null @@ -1,13 +0,0 @@ -# Manual Linking of Automation Runs with Test Manager - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -1. On the right side of the automation dashboard, there are two toggle buttons: one to open the **Test Case** and another to **Unlink Test**. - -2. To link a test case, click on **Link Test Case**. You can browse the test cases or add new test cases by clicking on **Create new Test Case** for your automation script. - -3. Initiate bug reporting directly from the automation dashboard by selecting **Raise a bug**. - -4. It will open an instance window, enter the details required to automatically link the bug to your Jira account, streamlining the issue creation process. - -5. Once logged, the issue is immediately reflected in the **Issues** section of the corresponding test case. diff --git a/static/docs/automated-test-cases-with-ai.md b/static/docs/automated-test-cases-with-ai.md deleted file mode 100644 index e919c0cff..000000000 --- a/static/docs/automated-test-cases-with-ai.md +++ /dev/null @@ -1,17 +0,0 @@ -# Managing Automated Tests with Test Manager - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Test Manager enables the creation and management of automated test cases directly within its platform. This feature not only streamlines test case development but also integrates AI to auto-generate BDD Scenarios from automated test runs, further optimizing the management of testing activities. - -## Automated Linking of Automation Runs with Test Manager - -1. Initiate the automation script and once it completes, proceed to the automation dashboard. The automation test run will be visible. - -2. Open the Test Manager dashboard, and in the **Projects** section, you will find the **TestMu AI Automation** project automatically created. - -3. Upon opening the newly created project, the automatically generated test case title from the automation process will be visible. - -4. In the Test Case section, **BDD Scenarios** will be automatically visible in the Test Case that is created from the command logs of the linked automated test run. - -> Currently, this feature is supported for Selenium Web frameworks. diff --git a/static/docs/automation-mcp-server.md b/static/docs/automation-mcp-server.md deleted file mode 100644 index 41829e45d..000000000 --- a/static/docs/automation-mcp-server.md +++ /dev/null @@ -1,36 +0,0 @@ -# Getting Started with Automation MCP Tool - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Automation MCP Tool is part of the [TestMu AI MCP Server](/support/docs/testmu-mcp-server/). It connects your AI client to test execution data on TestMu AI, letting you triage and debug automation failures directly from your IDE. - -## Key Benefits - -Use the Automation tool to accelerate debugging and test analysis: - -- Access your test execution data on TestMu AI directly from your IDE or editor. -- Triage and troubleshoot test failures to identify the root cause (RCA). -- Fix your test code based on the RCA and accelerate your development. -- Generate new test cases using execution data from TestMu AI. -- Use network logs, Selenium logs, and console logs to pinpoint bottlenecks and failures. - -## Capabilities - -The Automation MCP Tool provides four capabilities: - -1. **Automation Test Details:** Fetches detailed information about a specific TestID from the TestMu AI cloud. -2. **Automation Command Logs:** Retrieves execution logs for Selenium commands run on TestMu AI. -3. **Automation Network Logs:** Accesses test network logs, offering insights into browser traffic and behavior. -4. **Automation Console Logs:** Fetches browser console logs, surfacing error-level messages and warnings. - -## How to Invoke - -Once your MCP client is connected, use natural language to interact with the Automation tool. Example prompts: - -- "Triage the failure for TestID ``." -- "Show me the network logs for TestID ``." -- "What console errors occurred during the last failed test?" -- "Pull the Selenium command logs for `` and tell me where it failed." -- "Generate a new test case based on the execution flow of TestID ``." - -Your AI client will route the request to the appropriate capability, pull the relevant data from TestMu AI, and return the results inline. diff --git a/static/docs/automation-testing-selenium-with-lettuce.md b/static/docs/automation-testing-selenium-with-lettuce.md deleted file mode 100644 index 8ec36dfd7..000000000 --- a/static/docs/automation-testing-selenium-with-lettuce.md +++ /dev/null @@ -1,113 +0,0 @@ -# Selenium With Lettuce - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Run Lettuce tests on the TestMu AI cloud grid. This guide covers setup, running a sample test, configuring capabilities, and testing locally hosted pages. - -**Deprecated Framework** -Lettuce has not been updated since 2016 and only supports Python 2. Consider migrating to [Behave](/support/docs/behave-with-selenium-running-behave-automation-scripts-on-testmu-selenium-grid/) or [PyTest](/support/docs/pytest-with-selenium-running-pytest-automation-script-on-testmu-selenium-grid/). This guide is maintained for existing users. - -**Sample repo** -All the code used in this guide is available in the sample repository. - - View on GitHub - -## Prerequisites - -Complete these steps before running your first Lettuce Selenium test. - -1. Install the latest Python build from the [official website](https://www.python.org/downloads/). -2. Verify that **pip** is installed in your system. Install **pip** from [pip documentation](https://pip.pypa.io/en/stable/installation/). -3. Download the latest **Selenium Client** and its **WebDriver bindings** from the [official website](https://www.selenium.dev/downloads/). -4. Install **virtualenv** to isolate the build from other setups and ensure tests run with the specified module versions. -```bash -pip install virtualenv -``` - -## Step 1: Clone the Sample Project - -Clone the repository and set up the environment. - -```bash -git clone https://github.com/LambdaTest/sample-lettuce -cd sample-lettuce -``` - -Create a virtual environment and activate it: -```bash -virtualenv venv -source venv/bin/activate -``` - -Install the required packages: -```bash -pip install -r requirements.txt -``` - -## Step 2: Set Your Credentials - -Configure your credentials to connect to the TestMu AI Selenium Grid. - -Set TestMu AI `Username` and `Access Key` in environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -## Step 3: Configure Your Test Capabilities - -Define browser, version, and OS settings for your test run. - -In the `config.json` file, update your test capabilities. This code passes browser, browser version, and operating system information, along with TestMu AI Selenium grid capabilities via the capabilities object. - -```python -capabilities = { -"build": "Sample lettuce Build", -"platformName": "Windows 11", -"browserName": "Chrome", -"browserVersion": "latest", -} -``` - -**Capabilities Generator** -Use the TestMu AI [Capabilities Generator](https://www.testmuai.com/capabilities-generator/) to auto-generate the capabilities class for your test requirements. - -## Step 4: Run the Test - -Execute the Lettuce test from the command line. - -Navigate to the `tests` directory and run: -```bash -cd tests //navigate to tests directory -python lt_sample_todo.py -``` - -## Step 5: View Your Results - -Check the test output on the console and the TestMu AI dashboard. - -Visit the [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) to view your test results. The dashboard provides: - -- Text logs for each test step -- Screenshots captured during execution -- Video recordings of the full test session - -## Run Lettuce Tests Using Agent Skills - -Use AI coding assistants to generate and run Lettuce tests with the TestMu AI Agent Skill. - -The [lettuce-skill](https://github.com/LambdaTest/agent-skills/tree/main/lettuce-skill) is part of [TestMu AI Agent Skills](https://github.com/LambdaTest/agent-skills/) - structured packages that teach AI coding assistants how to write production-grade test automation. - -Install the skill: - -```bash -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/lettuce-skill .claude/skills/ - -# For Cursor / Copilot -cp -r agent-skills/lettuce-skill .cursor/skills/ -``` - -Install all available framework skills at once by cloning the repository directly into your tool's skills directory (e.g., `.claude/skills/`, `.cursor/skills/`). diff --git a/static/docs/automation-testing-with-mocha-and-selenium.md b/static/docs/automation-testing-with-mocha-and-selenium.md deleted file mode 100644 index 6b91457c9..000000000 --- a/static/docs/automation-testing-with-mocha-and-selenium.md +++ /dev/null @@ -1,111 +0,0 @@ -# Selenium With Mocha - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Run Mocha tests on the TestMu AI cloud grid. This guide covers setup, running a sample test, configuring capabilities, and testing locally hosted pages. - -**Sample repo** -All the code used in this guide is available in the sample repository. - - View on GitHub - -## Prerequisites - -Complete these steps before running Mocha Selenium tests on TestMu AI. - -1. Create a [TestMu AI account](https://www.testmuai.com/register/?redirectTo=https://accounts.lambdatest.com/dashboard) and get your username and access key from the dashboard. -2. Install **NodeJS** v6 or newer from [nodejs.org](https://nodejs.org/en/). -3. Install **npm** from the [official npm website](https://www.npmjs.com/). -4. Download [Selenium JavaScript bindings](https://www.selenium.dev/downloads/) from the official website. - -## Step 1: Clone the Sample Project - -Clone the TestMu AI Mocha Selenium sample repository to your local machine. - -```bash -git clone https://github.com/LambdaTest/mocha-selenium-sample -cd mocha-selenium-sample -``` - -Install the required dependencies: -```bash -npm i -npm install selenium-webdriver -``` - -## Step 2: Set Your Credentials - -Set your TestMu AI username and access key as environment variables. - -* For **macOS/Linux**: - -{`export LT_USERNAME= "${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY= "${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -* For **Windows**: - -{`set LT_USERNAME= "${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY= "${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -## Step 3: Configure Your Test Capabilities - -Update the capabilities in `conf/single.conf.js` to define the browser and platform settings. - -```js -exports.capabilities = { -'build': 'Mocha-Selenium-Sample', //Build name -'name': 'Your Test Name', // Test name -'platformName':'Windows 10', // OS name -'browserName': 'chrome', // Browser name -'browserVersion': 'latest', // Browser version -'visual': false, // To take step by step screenshot -'network':false, // To capture network Logs -'console':false, // To capture console logs. -'tunnel': false // If you want to run the localhost than change it to true -}; -``` - -Generate capabilities for your test requirements with the [Capabilities Generator](https://www.testmuai.com/capabilities-generator/). - -## Step 4: Run the Test - -Execute the test using the following command. - -**Single test:** -```bash -npm run single -``` - -**Parallel tests:** -```bash -npm run parallel -``` - -## Step 5: View Your Results - -After running the test, view your results on the [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build). - -The dashboard provides: -- Video recordings of each test session -- Screenshots captured at each step -- Console logs from the browser -- Network logs for debugging -- Detailed command logs - -## Run Mocha Tests Using Agent Skills - -Use AI coding assistants to generate and run Mocha tests with the TestMu AI Agent Skill. - -The [mocha-skill](https://github.com/LambdaTest/agent-skills/tree/main/mocha-skill) is part of [TestMu AI Agent Skills](https://github.com/LambdaTest/agent-skills/) - structured packages that teach AI coding assistants how to write production-grade test automation. - -Install the skill: - -```bash -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/mocha-skill .claude/skills/ - -# For Cursor / Copilot -cp -r agent-skills/mocha-skill .cursor/skills/ -``` - -Install all available framework skills at once by cloning the repository directly into your tool's skills directory (e.g., `.claude/skills/`, `.cursor/skills/`). diff --git a/static/docs/automation-testing-with-selenium-and-jest.md b/static/docs/automation-testing-with-selenium-and-jest.md deleted file mode 100644 index 4c3d6d798..000000000 --- a/static/docs/automation-testing-with-selenium-and-jest.md +++ /dev/null @@ -1,95 +0,0 @@ -# Selenium With Jest - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Run Jest tests on the TestMu AI cloud grid. This guide covers setup, running a sample test, configuring capabilities, and testing locally hosted pages. - -**Sample repo** -All the code used in this guide is available in the sample repository. - - View on GitHub - -## Prerequisites - -Complete these steps before running Jest Selenium tests on TestMu AI. - -1. Create a [TestMu AI account](https://www.testmuai.com/register/?redirectTo=https://accounts.lambdatest.com/dashboard) and get your username and access key from the dashboard. -2. Install **NodeJS** v6 or newer from [nodejs.org](https://nodejs.org/en/). -3. Install **npm** from the [official npm website](https://www.npmjs.com/). -4. Download [Selenium JavaScript bindings](https://www.selenium.dev/downloads/) from the official website. - -## Step 1: Clone the Sample Project - -Clone the TestMu AI Jest Selenium sample repository to your local machine. - -```bash -git clone https://github.com/LambdaTest/jest-selenium-webdriver-sample -cd jest-selenium-webdriver-sample -``` - -Install the required dependencies: -```bash -npm install --save-dev jest --force -``` - -## Step 2: Set Your Credentials - -Set your TestMu AI username and access key as environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -## Step 3: Configure Your Test Capabilities - -Update the capabilities object to define the browser and platform settings. - -```js -const capabilities = { -build: 'jest-LambdaTest-Single', -browserName: 'chrome', -browserVersion: 'latest', -platformName: 'Windows 10', -}; -``` - -Generate capabilities for your test requirements with the [Capabilities Generator](https://www.testmuai.com/capabilities-generator/). - -## Step 4: Run the Test - -Execute the test using the following command. - -```bash -npm test single.test.js -``` - -## Step 5: View Your Results - -After running the test, view your results on the [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build). - -The dashboard provides: -- Video recordings of each test session -- Screenshots captured at each step -- Console logs from the browser -- Network logs for debugging -- Detailed command logs - -## Run Jest Tests Using Agent Skills - -Use AI coding assistants to generate and run Jest tests with the TestMu AI Agent Skill. - -The [jest-skill](https://github.com/LambdaTest/agent-skills/tree/main/jest-skill) is part of [TestMu AI Agent Skills](https://github.com/LambdaTest/agent-skills/) - structured packages that teach AI coding assistants how to write production-grade test automation. - -Install the skill: - -```bash -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/jest-skill .claude/skills/ - -# For Cursor / Copilot -cp -r agent-skills/jest-skill .cursor/skills/ -``` - -Install all available framework skills at once by cloning the repository directly into your tool's skills directory (e.g., `.claude/skills/`, `.cursor/skills/`). diff --git a/static/docs/aws-codepipeline-integration-with-testmu.md b/static/docs/aws-codepipeline-integration-with-testmu.md deleted file mode 100644 index ae24a2da2..000000000 --- a/static/docs/aws-codepipeline-integration-with-testmu.md +++ /dev/null @@ -1,90 +0,0 @@ -# AWS CodePipeline Integration With TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -AWS CodePipeline is a cloud-based continuous delivery platform for fully automating your release pipelines. Provides an intuitive interface for configuring & modeling different stages of release cycles, so you can envision how your code changes would look when pushed across these different changes. You can also track your pipeline in real-time, and look into the pipeline history to understand the timestamps involved in the execution of various pipelines. - -TestMu AI integration with AWS CodePipeline will help you perform automated cross browser testing through a Selenium Grid offering 3000+ real browsers, browser versions. Now, not only can you commit changes to your code using AWS CodePipeline, but you can also validate any UI/UX anomalies of your web application through a wide variety of browsers using TestMu AI, a cloud-based cross browser testing tool. - -## Prerequisite - -* Login details in [AWS CodePipeline CI](https://aws.amazon.com/codepipeline/). -* A GitHub repository. Here is our sample GitHub repository for [AWS CodePipeline](https://github.com/LambdaTest/protractor-selenium-awscodepipeline-sample.git). -* **TestMu AI Authentication Credentials** - Be aware of your TestMu AI authentication credentials, i.e., your TestMu AI username, access key,a and HubURL. You need to set them up as your environment variables. You can retrieve them from your [TestMu AI automation dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/) by clicking on the key icon near the help button. - - - For Linux/Mac: -``` -$ export LT_USERNAME= {YOUR_LAMBDATEST_USERNAME} -$ export LT_ACCESS_KEY= {YOUR_LAMBDATEST_ACCESS_KEY} -``` - - For Windows: - -``` -$ set LT_USERNAME= {YOUR_LAMBDATEST_USERNAME} -$ set LT_ACCESS_KEY= {YOUR_LAMBDATEST_ACCESS_KEY} -``` - -## Integrating Your TestMu AI Account With AWS CodePipeline - -**Step 1:** Login to your AWS CodePipeline instance. Visit the Pipelines page and create a new pipeline. - -**Step 2:** Set your pipeline settings for a **new service role**. - -**Step 3:** Now, provide your git details in the next step to **Add Source Stage**. - -**Step 4:** Follow the steps in AWS CodePipeline & fill the form page on the next step to **Add build stage**. - -**Step 5:** Create a new build project while adding a **Build Stage**. - -**Step 6:** Choose a **Deploy Provider** on **deploy stage** screen and click on **Next**. - ->**Note:** Find the sample code from our [GitHub repository for AWS CodePipeline](https://github.com/LambdaTest/protractor-selenium-awscodepipeline-sample). - -**Step 7:** Start a build and pull the codes from Git. Read the below `buildspec.yml` from your project’s root directory. - -``` -version: 0.2 -env: -variables: -LT_USERNAME: "{your_lambda_username}" -LT_ACCESS_KEY: "{your_lambda_access_key}" -phases: -install: -runtime-versions: -nodejs: 10 -pre_build: -commands: -- echo Installing source NPM dependencies... -- npm install -- npm install -g protractor -build: -commands: -- echo Build started on `date` -- cd conf -- protractor single.conf.js -artifacts: -files: -- '**/*' -base-directory: 'dist*' -discard-paths: yes -``` - -**Step 8:** Monitor your test being triggered on the TestMu AI platform. You will observe the below output generated. - -``` -[Container] 2019/07/09 05:43:39 Running command echo Build started on `date` -Build started on Tue Jul 9 05:43:39 UTC 2019 -[Container] 2019/07/09 05:43:39 Running command cd conf -[Container] 2019/07/09 05:43:39 Running command protractor single.conf.js -[05:43:39] I/launcher - Running 1 instances of WebDriver -[05:43:39] I/hosted - Using the selenium server at https://user:bX3VZpiN4Tp7cLD7MTfoXqqOZK3q7Xov7UcSCwGQkzf1MEB6bI@hub.lambdatest.com/wd/hub -Started -•[32m.•[0m -1 spec, 0 failures -Finished in 15.406 seconds -[05:44:04] I/launcher - 0 instance(s) of WebDriver still running -[05:44:04] I/launcher - chrome67.0 #01 passed -``` - -Kudos! Now, you can rapidly commit changes and validate them for UI/UX anomalies by executing automated cross browser testing using TestMu AI Selenium Grid. Happy Testing! 🙂 diff --git a/static/docs/aws-codepipeline-with-hyperexecute.md b/static/docs/aws-codepipeline-with-hyperexecute.md deleted file mode 100644 index efdcf42cf..000000000 --- a/static/docs/aws-codepipeline-with-hyperexecute.md +++ /dev/null @@ -1,73 +0,0 @@ -# AWS CodePipeline Integration With HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -AWS CodePipeline is a cloud-based continuous delivery platform for fully automating your release pipelines. Provides an intuitive interface for configuring & modeling different stages of release cycles, so you can envision how your code changes would look when pushed across these different changes. You can also track your pipeline in real-time, and look into the pipeline history to understand the timestamps involved in the execution of various pipelines. - -This document will show you how to integrate AWS CodePipeline with HyperExecute to greatly shorten your test cycles. - -## How to Integrate AWS CodePipeline with HyperExecute - -To integrate AWS CodePipeline with HyperExecute, follow the below mentioned steps: - -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** -Download or Clone the code sample from the TestMu AI GitHub repository to run the tests on the HyperExecute. - - View on GitHub - -### Step 1: Create a new AWS CodePipeline pipeline. - -- Enter the Pipeline Name. -- Under Service role, select **New service role.** Click Next. - -### Step 2: Add a Source Provider - -- In the **Source Provider**, select the **GitHub (Version 2)** option. -- Connect your GitHub account. -- Select the desired Repository. -- Select the specified branch name, for which you want to establish the pipeline. - -### Step 3: Select the Build Provider - -- Add a Build action to the pipeline. Select the **CodeBuild** as the build provider. -- Click on **Create a Project** button. -- For the demo we are selecting OS as Amazon Linux 2, Runtime as standard, Image - amazonlinux2-x86_64_standard:4.0, and Runtime Environment as Linux. - -### Step 4: Create a `buildspec.yml` file - -- Create a `buildspec.yml` file in the root directory of the repository where the tests are written. - -- ### Sample `buildspec.yml` file - -```bash -version: 0.2 - -phases: -install: -commands: -# Download Hyperexecute CLI for macOS (adjust for other OS) -- name: Download Hyperexecute CLI -command: wget https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute -# Grant execute permission to the downloaded binary -- name: Make HyperExecute executable -command: chmod u+x hyperexecute -build: -commands: -# Run Hyperexecute with user credentials and configuration file -- name: Trigger HyperExecute tests -command: ./hyperexecute --user --key --config -``` - -- In the **buildspec** specification, choose the first option - **"Use a buildspec file"**. - -- Now Click the **"continue to code pipeline"** button and then click on **Next**. - -- You can "**Skip deploy stage"** and finally click **"Create Pipeline"** button. - -### Save the pipeline. - -Once the pipeline is created, it will start automatically. The first time the pipeline runs, it will fetch the latest code from your GitHub repository and build your project. The built project will then be deployed to HyperExecute. - -**Output** diff --git a/static/docs/axosoft-integration.md b/static/docs/axosoft-integration.md deleted file mode 100644 index 9af058609..000000000 --- a/static/docs/axosoft-integration.md +++ /dev/null @@ -1,63 +0,0 @@ -# Axosoft Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Axosoft is a versatile project management tool designed for Agile teams and organizations seeking to streamline their project planning, tracking, and collaboration. With features for Agile methodologies, collaborative workspaces, roadblock identification, risk management, sprint planning, and goal setting, it empowers teams to work efficiently, eliminate obstacles, and achieve project success across various industries and project management styles. - -Axosoft Integration with TestMu AI streamlines the software development team's project management and testing processes. With a simple one-click integration, you can seamlessly create and manage issues during [cross browser testing](https://www.testmuai.com/online-browser-testing). This integration enables you to push annotated issues to specific projects, assign them to team members, set priorities, and attach relevant screenshots during test sessions. The created issues are automatically reflected in your Axosoft workspace, complete with test session details and annotated screenshots, facilitating efficient issue tracking and resolution within your project management environment. - -Axosoft Integration with TestMu AI, like all of the integrations to 3rd party applications, is available for freemium and premium plans. - -## Integrate Axosoft With Your TestMu AI Account - -**Step 1:** Log in to your TestMu AI account. You need to have Admin or User level access to see and install 3rd party integrations. - -**Step 2:** Select the ‘Integrations’ tab from the left navigation menu bar under "Settings". This will guide you to a screen where you will find a list of 3rd party applications, that have already been integrated as well as that are available to integrate with your TestMu AI account. - -**Step 3:** Hover to the 'Axosoft' block under the "Project Management" category, and click on the "plus" button that appears. - -**Step 4:** You need to [create an account on Axosoft](https://www.axosoft.com/) to integrate your workspace with TestMu AI. If you don't have an account, you can create one. If you are already logged in, this step won't appear in the integration process. You will be required to enter the URL and OAuth token to enable integration. The Axosoft Base URL can be copied from the Axosoft website. - -**Step 5:** To generate an OAuth token for integration, go to your Axosoft Account, click on Tools, then select System Settings. - -**Step 6:** Within Axosoft API Settings, click on Manage API Keys. - -**Step 7:** Once you click on Manage API Keys, you will get an option to create a new key, for which you will need to enter a name (e.g., "TestMu AI") and save it. - -**Step 8:** Click on the Manage Tokens section, and create a non-expiring token for the app you've just created. Select "TestMu AI" under app, "read and write" under scope, and enter create. - -**Step 9:** Copy the generated token, click Create, and paste it into your TestMu AI integration settings for Axosoft. - -**Step 10:** Once you enter "install" Axosoft will be integrated with TestMu AI, and an alert box will appear on the top right corner of the screen stating that the integration is in progress, you will be notified on your mail post successful integration. - -**Step 11:** Congrats! Axosoft has been successfully integrated. Go to the Integrations tab again and you will see Axosoft under the "My Integrations" tab. - -## How To Log Your First Bug Through Axosoft Integration? - -**Step 1:** Select any of the tests from the left navigation menu of TestMu AI. For this instance, we will be taking the "Real Time Testing" option. - -**Step 2:** Enter the URL of the web app to be tested in the URL box, and select "mobile" device for testing. Press 'Start' after selecting the configuration of the browser and operating system as per your choice. - -**Step 3:** After the VM is launched and operable, you can perform testing on your mobile app as per your choice, such as to find issues. Suppose you come across any bug, then you need to click on the Bug icon from the left panel, to capture a screenshot of the current VM instance. - -**Step 4:** A screenshot will be captured and appear on your screen in an in-built image editor. Annotate any issue as per your choice in this editor, and then press the "Mark as Bug" button under "Axosoft" integration. - -**Step 5:** Complete the Bug/Task/Issue Form that appears upon clicking the "Mark as Bug" button, as per your requirement, and click on the "Mark As Bug" option given below. - -**Step 6:** You will now observe it being successfully marked through a single click effort. You will get prompt messages on top of your Virtual Machine indicating the progress of bug logging. After a few seconds, you will be notified with a prompt message "Bug successfully marked" indicating that the screenshot has been pushed to your workspace. - -**Step 7:** Log in to your workspace on Axosoft and you will be able to notice the marked bug in the set channel assigned to the selected person. All the data you provided through the TestMu AI form would already be presented in it. TestMu AI automatically includes test environment details and related screenshots in the screenshot as attachments. - -## How To Remove Axosoft Integration? - -You can work with one integration at a time. So if you want to integrate into a similar 3rd party application, then you would have to remove your current integration. Here is how you can do that. - -**Step 1:** Log in to your TestMu AI account. - -**Step 2:** Select ‘Integrations’ from the left navigation menu bar. This will guide you to a screen where you can remove the Axosoft integration under the tab that says "My Integrations". - -**Step 3:** On the Axosoft block, a "REMOVE" button will appear. Click on ‘REMOVE’. - -**Step 4:** Upon pressing 'REMOVE', a message will prompt on the top right corner of the app stating the same. - -> That was all you need to know for TestMu AI + axosoft Integration. Increase your productivity with our integrations. If you still have any questions for us, please feel free to let us know. Our experts are always available on **chat** to help you out with any roadblock regarding our product. Happy testing! diff --git a/static/docs/azure-with-hyperexecute.md b/static/docs/azure-with-hyperexecute.md deleted file mode 100644 index c31581344..000000000 --- a/static/docs/azure-with-hyperexecute.md +++ /dev/null @@ -1,78 +0,0 @@ -# Azure DevOps Pipeline Integration with Hyperexecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Azure DevOps offers developer services that enable teams to plan their work, collaborate on code development, and build and deploy applications. Azure DevOps fosters a collaborative culture and set of processes that bring together software developers, project managers, and contributors. It enables organizations to create and improve products at a much faster rate than traditional software development approaches allow. - -This document will show you how to integrate Azure Devops Pipeline with HyperExecute to greatly shorten your test cycles. - -## How To Integrate Azure DevOps Pipeline with Hyperexecute - -To integrate Azure DevOps Pipeline with HyperExecute, follow the below steps: - -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** -Download or Clone the code sample from the TestMu AI GitHub repository to run the tests on the HyperExecute. - - View on GitHub - -### 1. Log into your Azure DevOps account - -### 2. Create a New Project - Go to the top-right corner of your Azure Devops profile page and click the **+ New Project** button. - -Follow the prompts to successfully create a new project. - -### 3. Create a New Pipeline - -Next, you need to create a new pipeline. To create a new pipeline, click the **Pipelines** button from the left panel or from the main page. - -### 4. Configure the Azure Pipeline YAML to Execute Hyperexecute CLI Binary: - -To configure the Azure DevOps pipeline YAML to execute the HyperExecute CLI Binary, run the following command: - -```bash -trigger: -- main - -pool: -vmImage: 'ubuntu-latest' # Adjust for macOS if needed - -steps: -- task: Bash@3 -inputs: -targetType: 'inline' -script: | -# Download Hyperexecute CLI for macOS (adjust for other OS) -name: Download HyperExecute CLI -script: wget https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute - -# Grant execute permission to the downloaded binary -name: Make HyperExecute executable -script: chmod +x hyperexecute - -# Run Hyperexecute with user credentials and configuration file -name: Run Hyperexecute Tests -script: ./hyperexecute --user --key --config -``` - -- **Trigger:** In the **trigger** section, state the branch of your your test repository that you would like to trigger. -Here, the **main** branch is selected as the test repository to commit the Azure Pipeline Yaml. -- **Pool:** In the **pool** section, state the OS flavor of the Virtual Machine Image that you would like to use. -- **Steps:** In the **steps** section, declare the flow of the pipeline execution. -- **Script:** In the **script** section, state the command that you would like to execute in the following steps: - - Download a fresh version of the HyperExecute CLI binary - - Execute the Hyperexecute CLI binary. (The Hyperexecute CLI binary contains TestMu AI username, access key and path of the Yaml created for Hyperexecute. You can find more information on this [here.](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid)) - -> **Note**: Ensure that your username and access key for your TestMu AI account is visible in the script. - -### 5. Run the Pipeline -To run the new pipeline that you just created, click the **Run** or **Save and Run** button at the top-right corner of the page. - -**An example of how a test that is run on the Azure DevOps pipeline gets executed:** - -**An example of a Hyperexecute job that was triggered through the above pipeline:** - -> -**Run your tests at speeds never seen before. Happy testing! :)** diff --git a/static/docs/azure.md b/static/docs/azure.md deleted file mode 100644 index 595ee20d8..000000000 --- a/static/docs/azure.md +++ /dev/null @@ -1,104 +0,0 @@ -# Azure AD Scim User Provisioning - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Prerequisites -Integrate SCIM With TestMu AI: - -* You will need an Enterprise plan with TestMu AI. -* SSO must be already integrated. Please complete [TestMu AI SSO & Azure AD Integration](/support/docs/sso-azure-integration/) - -## Integrating SCIM with Azure AD -**Step 1:** Sign in to your TestMu AI account. Don't have an account, register for free. - -**Step 2:** Head to **Settings** and select **Organization Settings** from the dropdown. - -**Step 3:** Head to the **Authentication and SSO** tab and click and copy the **SCIM Base URL and Bearer Token** option. - -**Step 4:** Sign in to the [Azure portal](https://portal.azure.com). Select Enterprise Applications, then select All applications. - -**Step 5:** In the applications list, select application used for TestMu AI SSO setup - -**Step 6:** Select the Provisioning tab. - -**Step 7:** Under the Admin Credentials section, input your TestMu AI SCIM Base URL `https://auth.lambdatest.com/api/scim?aadOptscim062020` and Bearer Token retrieved earlier in Step 3. Click Test Connection to ensure Azure AD can connect to TestMu AI. If the connection fails, ensure your TestMu AI account has Admin permissions and try again. - -**Step 8:** Under the Mappings section, select Synchronize Azure Active Directory Users. - -**Step 9:** Creating Custom Attributes - -- Show advanced options > Edit attribute list for customappsso > Add attributes -- userName, Active, name.givenName, name.familyName are required attributes - -- **`urn:ietf:params:scim:schemas:extension:LambdaTest:2.0:User:OrganizationRole`**: Custom attribute used to set TestMu AI Organization Role for Users, If this attribute is not mapped User role would be set by default. Allowed values are (Admin/Guest/User) - -- **`urn:ietf:params:scim:schemas:extension:LambdaTest:2.0:User:LambdatestGroup`**: Used to assign an existing group in TestMu AI to a new user created in TestMu AI through SCIM. (Applicable only if organisation has group support active) - -For filtering only **userName** attribute is supported and must be selected for filtering, click edit on userPrincipalName and make sure **Apply this mapping** is set to **Always** - -- **Dynamic/Static assignment of custom attributes**: After custom attribute creation, we have to map them using “Add new mapping” - -Now there are three types Mapping type in AzureAD, “Direct”, “Constant” and “Expression”. - -For example we can set Constant association “Guest” for `urn:ietf:params:scim:schemas:extension:LambdaTest:2.0:User:OrganizationRole` - -Or, can create association using the Expression like this , - -```javascript -IIF(SingleAppRoleAssignment([appRoleAssignments])="Admin", "Admin", -IIF(SingleAppRoleAssignment([appRoleAssignments])="Guest", "Guest", -IIF(SingleAppRoleAssignment([appRoleAssignments])="User", "User", "User"))) -``` - -In the above example we are using the appRoleAssignments attribute of microsoft user to set string value. - -After custom attribute creation, we have to map them using “Add new mapping” - -## Provisioning Groups from Azure AD - -Once user provisioning is configured, you can also push Azure AD groups to TestMu AI. - -**Prerequisites** -Group Provisioning must be enabled for your org. Contact **24/7 chat support** to activate it. - -**Step 1:** In Azure portal, go to your TestMu AI Enterprise Application > **Provisioning** > **Mappings**. - -**Step 2:** Click **Provision Azure Active Directory Groups** and ensure it is **Enabled**. - -**Step 3:** Review the attribute mappings. The required mappings are: -- `displayName` → `displayName` -- `members` → `members` - -**Step 4:** Under **Users and groups**, assign the groups you want to provision. - -**Step 5:** Start a provisioning cycle (or wait for the 40-minute auto sync). - -**Step 6:** In TestMu AI, go to **Settings** > **Organization Settings** > **SCIM Group Provisioning** to view the synced groups and configure mappings. - -### Setting Roles on Azure AD Groups - -Azure AD sends roles via the SCIM group extension `urn:ietf:params:scim:schemas:extension:LambdaTest:2.0:Group`. To assign roles: - -1. Create a custom attribute `LambdatestRoles` under the group schema in your Azure AD attribute mappings -2. Map it to an Azure AD attribute or set it as a constant (e.g., `User`, `Admin`, or `Guest`) -3. The role applies to **all** members of the group. Highest role wins across multiple groups (Admin > User > Guest) - -### What Happens After Provisioning - -| Azure AD Action | TestMu AI Effect | -|---|---| -| Group provisioned | Group created, mapping rules evaluated, members synced | -| Member added to group | Member added to all mapped TestMu AI entities | -| Member removed from group | Member removed (if no other group maps them there), role recomputed | -| Group renamed | Group renamed, mapped entity renamed to match, rules re-evaluated | -| Group deprovisioned | Group soft-deleted, members safely unassigned, roles recomputed | - -> For details on mapping, conflicts, and rules, see the [SCIM Provisioning guide](/support/docs/scim/#group-provisioning). - -**Step 10:** To enable the Azure AD provisioning service for TestMu AI, change the Provisioning Status to On in the Settings section. - -**Step 11:** When you are ready to provision, click Save. - -This operation starts the initial synchronization cycle of all users in Scope in the Settings section. The initial cycle takes longer to perform than subsequent cycles, which occur approximately every 40 minutes as long as the Azure AD provisioning service is running. - -> That's all you need to know about TestMu AI SCIM Auto User Provisioning with Azure AD.In case you have any questions please feel free to reach out to us via the **24/7 chat support** or email us over [support@testmuai.com](mailto:support@testmuai.com). diff --git a/static/docs/backlog-integration-with-testmu.md b/static/docs/backlog-integration-with-testmu.md deleted file mode 100644 index 0f9877fab..000000000 --- a/static/docs/backlog-integration-with-testmu.md +++ /dev/null @@ -1,55 +0,0 @@ -# Backlog Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Backlog provides an online platform for project & code management. Using Backlog, you can collaborate with your teams working in different parts of SDLC(Software Development Life Cycle). It lets you organize, discuss, and track all the work in one place. It offers out-of-the-box features to help you orchestrate your release cycles better. Some of the important features include Git hosting, Git version control, and issue tracking. - -TestMu AI integration with Backlog will help you file any UI bug on your Backlog project instance, directly from TestMu AI. If you ever come across a [browser compatibility](https://www.testmuai.com/feature) bug while cross browser testing on TestMu AI, all you need to do is hit a single click and the bug will be logged into your Backlog project. - -## How To Integrate Backlog With Your TestMu AI Account? - -**Step 1:** Login to your TestMu AI account. You would need to have Admin or User level access to see and install integrations. - -**Step 2:** Select ‘Integration’ from the left navigation menu bar. This will guide you to a screen where you will find a list of 3rd party applications, available to integrate with your TestMu AI account. - -**Step 3:** Under the category of Project Management, click on the block that says **‘Backlog’**. - -**Step 4:** You will be routed to the below screen where you are required to fill your Backlog URL and API key. - -**Step 5:** Login to your Backlog account and you will find the Backlog URL on top of your browser’s address bar. - -**Step 6:** On top-right corner of your Backlog instance, you will find a drop-down menu with your profile icon. Click on the drop-down and go to "Personal Settings". - -**Step 7:** In your personal settings, go to API. Enter a random string of characters in the Memo field and hit the Submit button. You will find an API key generated based on the Memo. Copy this API key as it would be used to authenticate yourself while integrating TestMu AI with Backlog. - -**Step 8:** Paste your Backlog URL and API key on the TestMu AI, and then hit the Install button. - -**Step 9:** You will now be routed back to the integrations where you will notice a green tick over the block for Backlog indicating that the integration has been successful. The refresh button will help you synchronize your Backlog account with TestMu AI in just a single click. So if in near future you add more projects on you backlog dashboard then all you need to do is hit that refresh icon and TestMu AI will be able to fetch your latest projects instantly. - -## Log Your First Bug From TestMu AI Integration With Backlog - -**Step 1:** Go for any of the test from the left navigation menu. For demo, we will be taking "Real Time Test" option. - -**Step 2:** Present a URL of the web-app you need to test in the dialog box. After that, select any configuration for browser and operating system of your choice & hit ‘Start‘. - -**Step 3:** After the VM is launched and operable. You can perform testing on your web-app for finding bugs. If a bug gets revealed, then you need to click on the Bug icon from the left panel for [capturing a screenshot](https://www.testmuai.com/full-page-screen-capture) of the same. We have highlighted that option with yellow in the below image. - -**Step 4:** After a screenshot is captured, you can annotate any issue or a task with an in-built image editor. Once you are done highlighting the bug, click on the button that says "Mark as Bug". - -**Step 5:** After clicking on "Mark as Bug" button a form specific to Backlog would open up. Fill the fields as per your requirement, and create the issue. - -> Note: After you click on "Create Bug", you will be able to observe it being successfully marked through a single click effort. You will get prompt messages on top of your Virtual Machine indicating the progress of bug logging. After few seconds you will be notified with a prompt message "Bug successfully marked" indicating that the screenshot has been pushed to your Backlog project. - -**Step 6:** Now, go to your Backlog project and you will notice your logged task under the specific project that you assigned it through TestMu AI. All the test environment details will already be present in the created task. - -## Remove TestMu AI Integration With Backlog - -> You can work with one integration at a time. So if you would want to integrate to a similar 3rd party application, then you would have to remove your current integration. Here is how you can do that. - -**Step 1:** Login to your TestMu AI account. - -**Step 2:** Select ‘Integrations’ from the left-navigation menu. This will guide you to a screen where you will find 3rd party applications, available to integrate with your TestMu AI account. - -**Step 3:** Click on ‘REMOVE’. You can find the remove button under the ‘Backlog’ block which would be highlighted with a green tick. - -> That was all you need to know for TestMu AI + Backlog Integration. Increase your productivity with our integrations. If you still have any questions for us, please feel free to let us know. Our experts are always available on **chat** to help you out with any roadblock regarding our product. Happy testing! 🙂 diff --git a/static/docs/bamboo-integration-with-hyperexecute.md b/static/docs/bamboo-integration-with-hyperexecute.md deleted file mode 100644 index a5a1d0591..000000000 --- a/static/docs/bamboo-integration-with-hyperexecute.md +++ /dev/null @@ -1,119 +0,0 @@ -# Bamboo CI Integration with HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Bamboo is a continuous integration (CI) and continuous delivery (CD) server from Atlassian. It helps teams automate the software development and delivery process, from code commit to production deployment. Bamboo can be used to build, test, and deploy software on a variety of platforms, including cloud, on-premises, and hybrid environments. - -This document will show you how to integrate Bamboo CI Pipeline with HyperExecute to greatly shorten your test cycles. - -## How To Integrate Bamboo CI Pipeline with Hyperexecute - -To integrate CircleCI Pipeline with HyperExecute, follow the below steps: - -### Pre-requisite: - -You can refer to different supported versions from [here](https://confluence.atlassian.com/bamboo/supported-platforms-289276764.html) as per your requirements. - -Below are the mentioned softwares used for this documentation: - -- Java 11 -- **DataBase** - PostgreSQL 14.9 -- **Operating System** - Linux Ubuntu 22.04 LTS - -### Step 1: Setup Bamboo CI in your local device - -1. [Download](https://www.atlassian.com/software/bamboo/download) the Bamboo CI in your device. - -2. Extract the file in your desired location. Let's say it as **ROOT_FOLDER** - -3. Before you run Bamboo for the first time, specify your Bamboo Home Directory, where your Bamboo data is stored. - - Create your Bamboo Home Directory (without spaces in the name). - - > **NOTE:** You should not create your Bamboo Home Directory inside the **ROOT_FOLDER**. They should be entirely separate locations. If you do put the Home Directory in the **ROOT_FOLDER**, it will be overwritten and lost when Bamboo is upgraded. - - - Go to the following file location - -```bash -ROOT_FOLDER/atlassian-bamboo/WEB-INF/classes/bamboo-init.properties -``` - - You have to uncomment the **bamboo.home** line and provide the absolute path to your Home Directory. - -### Step 2: Start the Bamboo Server - -- Change to the **ROOT_FOLDER** from the command-line and start the server - -```bash -cd ROOT_FOLDER - -./bin/start-bamboo.sh -``` - -- Now type ``http://localhost:8085/`` in your browser to open the Bamboo GUI. - -> **NOTE:** If you are running Bamboo for the very first time, then you need to configure the Setup Wizard as well. Follow the instructions [here](https://confluence.atlassian.com/bamboo/running-the-setup-wizard-289276851.html) to configure it. - -### Step 3: Create a Project and a Plan - -- Click on **Create Project** button. - -- Enter the required details for your project: - - - Enter your **Project Name** - - Provide a **Project Key** - - Give a brief description of your project. - - You can choose whether to provide access to this project to everyone or not. - - Click on **Save** button. - -- Now click on **Create Plan** button. - -- Select your required **Project**. -- Enter a **Plan Name** -- Enter your **Project Key** (it should be same as the one you entered in the above step). -- Click on **Create** button. - -### Step 4: Configure the Source Code - -- Select the Repository Host. In this case, we are using Git to manage the source code. -- Enter the **Display Name**. -- Provide the **Repository URL**. -- Select the **Authentication Type** of your choice. We are selecting "Username and Password" -- Enter the **Username** and **Password**. - -- Enter the desired **Branch** name. -- Click on **Test Connection** button to check whether the connection is successfully established or not. -- Define who can access to this repository. -- Click on **Save and Continue** button. - -### Step 5: Configure Job - -- Select the **environment** where do you want to execute this job. For the demo purpose, we are using **Agent Environment**. -- Click on **Add Task** button - -- Select the **Script** option from the provided Task Types. - -- Enter your Task Description. -- Select the **Interpreter** as **Shell**. -- Choose the **Script Location** as **Inline**. -- Add the following code in the **Script Body** - -```bash -curl -O https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute -chmod +x hyperexecute -./hyperexecute --user --key --config -``` - -- Add the Task and click on **Create** button. - -- Configure the pipeline as per your need in the **Trigger** tab or leave it to *Default* - -- Click on **Actions** -> **Enable Plan** button. - -- Click on **Run** -> **Run plan** button. - -- Test will get triggered and start executing. You can check the activity status of your Job here. - -### Step 6: Check the Output - -- Logs of the Job Build - -- Result on HyperExecute Dashboard diff --git a/static/docs/bamboo-integration-with-testmu.md b/static/docs/bamboo-integration-with-testmu.md deleted file mode 100644 index 35a1b3949..000000000 --- a/static/docs/bamboo-integration-with-testmu.md +++ /dev/null @@ -1,78 +0,0 @@ -# Bamboo Integration With TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Launched on 20 February 2007, Bamboo by Atlassian has become a worldwide adopted CI/CD tool. With Bamboo Server, you can generate multi-level build plans with the ease of defining triggers to specific builds post commits. You could also assign crucial builds or deployments to a colleague. Bamboo is free for open-source projects. - -TestMu AI provides an integration with Bamboo, using which you can perform cross browser testing on a cloud-based Selenium Grid consisting 3000+ real browsers, for both desktop & mobile. You can perform tests directly from your respective Bamboo server. It helps you to perform browser compatibility testing as soon as you commit changes to your build. If you are running short on the release window, you can scale your tests according to your need. TestMu AI offers parallel testing to help you run multiple test cases simultaneously. You can now chop down your release cycles and ship faster by integrating your TestMu AI account with your Bamboo CI/CD server. - -## Prerequisites - -- A Bamboo Server installed. -- A GIT repository. -- wget & unzip utility. - -``` -$ sudo apt-get install unzip -$ sudo apt-get install wget -``` - -- Be aware of your TestMu AI authentication credentials, i.e., your TestMu AI username, access key, and HubURL. You need to set them up as your environment variables. You can retrieve them from your [TestMu AI automation dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/) by clicking on the key icon near the help button. - - - For Linux/Mac: -``` -$ export LT_USERNAME= {YOUR_LAMBDATEST_USERNAME} -$ export LT_ACCESS_KEY= {YOUR_LAMBDATEST_ACCESS_KEY} -``` - - For Windows: - -``` -$ set LT_USERNAME= {YOUR_LAMBDATEST_USERNAME} -$ set LT_ACCESS_KEY= {YOUR_LAMBDATEST_ACCESS_KEY} -``` - -## Integrating TestMu AI With Bamboo Server - -**Step 1:** Deploy your Bamboo server & login as an administrator. - -**Step 2:** Create a new plan with Default stage & job. - -**Step 3:** After creating a new job, we would need to add three tasks. - -**Step 4:** To create your first task, "Source Code Checkout," you need to fetch the relevant Git repository. - -**Step 5:** Task number two would require to you install the required package for npm. - -**Step 6:** To create your third task, you would need to configure the below script for running a Protractor test. - -``` -echo "start testing" -//Add access credentials in plan variable -export LT_USERNAME=${bamboo.LT_USERNAME} -export LT_ACCESS_KEY=${bamboo.LT_ACCESS_KEY} - -wget https://downloads.lambdatest.com/tunnel/v3/linux/64bit/LT_Linux.zip -unzip -o LT_Linux.zip - -//To start Tunnel -./LT -user ${bamboo.LT_USERNAME} -key ${bamboo.LT_ACCESS_KEY} & - -node ./node_modules/protractor/bin/protractor conf/single.conf.js -``` - -**Step 7:** Now, you need to save the plan & run it. As you run the test you will find the below logs in Bamboo highlighting that the test was successfully executed on TestMu AI Selenium Grid. - -## Running Multiple Tests In Parallel - -Parallel Testing is one of the most demanding features of TestMu AI Selenium Grid. By parallel testing, you can run more than one test case, simultaneously. So you can execute a single test scenario across different browsers or could run different test scenarios across the same browser but with different browser versions. Wondering how many parallel test cases can you run? That depends entirely on the number of concurrent session under your opted plan. - -To run parallel tests using Bamboo integration with TestMu AI, you need to change the script in Step 6 as below. - -``` -echo "start testing" -export LT_USERNAME= -export LT_ACCESS_KEY= -node ./node_modules/protractor/bin/protractor conf/parallel.conf.js -``` -Monitor and analyze your test result on the TestMu AI Automation Dashboard. -Reliably deploy your code at scale using Bamboo integration with TestMu AI, and ensure it looks robust across every browser to provide a seamless user experience to all your visitors. Happy Testing! 🙂 diff --git a/static/docs/basic-authentication-for-web-automation.md b/static/docs/basic-authentication-for-web-automation.md deleted file mode 100644 index e251d6395..000000000 --- a/static/docs/basic-authentication-for-web-automation.md +++ /dev/null @@ -1,31 +0,0 @@ -# Basic Authentication for Safari Web Automation - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Introduction - -Basic Authentication is a method for an HTTP user agent to provide a user name and password when making a request. In basic HTTP authentication, a request contains a header field in the form of `Authorization: Basic `, where credentials is the **Base64** encoding of ID and password joined by a single colon `:`. - -To provide support for Basic Authentication in Safari during Web Automation, we have introduced a new lambda hook in our `iOS` Real Device (RD) web sessions. - -1. This hook is not compatible with any app-based sessions. -2. The capability `autoAcceptAlerts` must be set to `false`. - -## How to Use - -Here is an example of how to use the Basic Authentication hook in Python: - -```python -data = { -"username": "admin", -"password": "admin", -"url": "https://the-internet.herokuapp.com/basic_auth", -} -driver.execute_script("lambda-ios-set-basic-auth", data) -``` - -It is important to note that all three parameters (username, password, URL) are mandatory and must be passed to the script. - -## Limitations - -Please note, this hook is designed to be used exclusively with Safari on iOS Real Device (RD) web sessions and is not compatible with Android sessions or any app-based sessions. diff --git a/static/docs/behat-with-selenium-running-behat-automation-scripts-on-testmu-selenium-grid.md b/static/docs/behat-with-selenium-running-behat-automation-scripts-on-testmu-selenium-grid.md deleted file mode 100644 index f8ec86ad1..000000000 --- a/static/docs/behat-with-selenium-running-behat-automation-scripts-on-testmu-selenium-grid.md +++ /dev/null @@ -1,138 +0,0 @@ -# Selenium With Behat - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Run Behat tests on the TestMu AI cloud grid. This guide covers setup, running a sample test, configuring capabilities, and testing locally hosted pages. - -**Sample repo** -All the code used in this guide is available in the sample repository. - - View on GitHub - -## Prerequisites - -Complete the following steps before you begin automation testing with Selenium and Behat. - -1. Install the latest **PHP** on your system. Use the following commands in the terminal: - - * **MacOS:** Previous versions of **MacOS** have **PHP** installed by default. For the latest **MacOS** versions starting with **Monterey**, download and install **PHP** manually: -```bash -/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" -brew install php -``` - * **Windows:** Download **PHP** from [PHP for Windows](http://windows.php.net/download/) and add it to your system PATH. Refer to the [PHP Windows installation guide](http://php.net/manual/en/install.windows.php) to ensure PHP is accessible through Command Prompt (cmd). - -2. Download **composer** in the project directory ([Linux/MacOS](https://getcomposer.org/download/), [Windows](https://getcomposer.org/doc/00-intro.md#installation-windows)). - - **Note:** To use the **composer** command directly, it either should have been downloaded in the project directory or should be accessible globally which can be done by the command below: -```bash -mv composer.phar /usr/local/bin/composer -``` - -## Step 1: Clone the Sample Project - -Clone the repository and install dependencies. - -```bash -git clone https://github.com/LambdaTest/behat-selenium-sample -cd behat-selenium-sample -``` - -Install the composer dependencies: -```bash -composer install -``` - -Set up **Behat** automation for your OS using composer: -```bash -php composer.phar require behat/behat -``` - -## Step 2: Set Your Credentials - -Configure your credentials to connect to the TestMu AI Selenium Grid. - -Set TestMu AI `Username` and `Access Key` in environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -## Step 3: Configure Your Test Capabilities - -Define browser, version, and OS settings for your test run. - -In the test script, update your test capabilities to validate your TestMu AI credentials and select OS, browser, browser version, and other settings. - -```php -default: -paths: -features: '../features/single' -bootstrap: '../features/bootstrap' -context: -parameters: -lambdatest: -server: "hub.lambdatest.com" -user: "YOUR_LAMBDATEST_USERNAME" -key: "YOUR_LAMBDATEST_ACCESS_KEY" - -capabilities: -build: "behat-selenium-sample" -name: "single-behat-test" -video: true -visual: true -network: true -console: true - -environments: -- -browserName: chrome -version: 71.0 -platform: Win10 -``` - -**Capabilities Generator** -Use the TestMu AI [Capabilities Generator](https://www.testmuai.com/capabilities-generator/) to auto-generate the capabilities class for your test requirements. - -## Step 4: Run the Test - -Execute the Behat test from the command line. - -```bash -composer single -``` - -To run parallel tests: -```bash -composer parallel -``` - -## Step 5: View Your Results - -Check the test output on the console and the TestMu AI dashboard. - -Visit the [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) to view your test results. The dashboard provides: - -- Text logs for each test step -- Screenshots captured during execution -- Video recordings of the full test session - -## Run Behat Tests Using Agent Skills - -Use AI coding assistants to generate and run Behat tests with the TestMu AI Agent Skill. - -The [behat-skill](https://github.com/LambdaTest/agent-skills/tree/main/behat-skill) is part of [TestMu AI Agent Skills](https://github.com/LambdaTest/agent-skills/) - structured packages that teach AI coding assistants how to write production-grade test automation. - -Install the skill: - -```bash -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/behat-skill .claude/skills/ - -# For Cursor / Copilot -cp -r agent-skills/behat-skill .cursor/skills/ -``` - -Install all available framework skills at once by cloning the repository directly into your tool's skills directory (e.g., `.claude/skills/`, `.cursor/skills/`). diff --git a/static/docs/behave-on-hyperexecute-grid.md b/static/docs/behave-on-hyperexecute-grid.md deleted file mode 100644 index f8ee37483..000000000 --- a/static/docs/behave-on-hyperexecute-grid.md +++ /dev/null @@ -1,238 +0,0 @@ -# Running Behave Framework Tests on HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Behave is a behavior-driven development (BDD) framework for Python that is commonly used with Selenium for automated testing. It allows teams to write test scenarios in a natural language format using Gherkin syntax. - -HyperExecute is an AI Native Test Orchestration Cloud Platform that empowers you to run **end-to-end** tests **quickly** and **efficiently**. It provides Just-in-Time (JIT) testing infrastructure with fast execution **speeds**, **smart orchestration**, and **detailed logs**. - -This guide details how to execute your **Behave** framework tests on **HyperExecute** via two different methods: - -- [**Using Local System**](/support/docs/behave-on-hyperexecute-grid/#1-testing-using-local-system) - You can use your own local machine to execute tests. -- [**Using Gitpod Platform**](/support/docs/behave-on-hyperexecute-grid/#2-testing-using-gitpod) - Execute tests using GitPod. (Requires a [Gitpod](https://gitpod.io/login/) account) - -## 1. Testing Using Local System - -Follow the step-by-step guide to execute your test on HyperExecute. - -### Prerequisites - -To run the Tests on HyperExecute from your Local System, you are required: - -- Your TestMu AI [Username and Access key](/support/docs/hyperexecute-how-to-get-my-username-and-access-key/) -- [HyperExecute YAML](/support/docs/hyperexecute-yaml-version0.2/) file which contains all the necessary instructions. -- [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) in order to initiate a test execution Job. -- Setup the [Environmental Variable](/support/docs/hyperexecute-environment-variable-setup/) - -### Step 1: Configure Your Test Suite - -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** - -Download or Clone the code sample for the JUnit from the TestMu AI GitHub repository to run the tests on the HyperExecute. - - View on GitHub - -If you are using your own project, make sure you update the **Hub endpoint** in your tests file. - -By setting up the Hub endpoint, you establish the communication channel between your tests and the browser nodes, enabling effective test distribution and execution. - -Configure the desired capabilities based on your test requirements. For example: - -```bash -[ -{ -"platform": "Windows 10", -"browserName": "chrome", -"version": "latest", -"build": "Behave Selenium Sample", -"name": "Behave Sample Test" -} -] -``` - -> You can also use **dev** and **beta** browser versions. To generate capabilities for your test requirements, you can use our inbuilt 🔗 Capabilities Generator. - -### Step 2: Setup the CLI in your Test Suite - -After cloning / downloading the sample repo, you need to setup the CLI and the environment variables. - -#### Download the HyperExecute CLI - -The CLI is used for triggering the tests on HyperExecute. It is recommend to download the CLI binary on the host system and keep it in the root directory of the suite to perform the tests on HyperExecute. - -You can download the CLI for your desired platform from the below mentioned links: - -| Platform | HyperExecute CLI | -| ---------| ---------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| MacOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -#### Setup Environment Variable - -Now, you need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in the [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). - -Run the below mentioned commands in your terminal to setup the CLI and the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -### Step 3: Configure YAML in your Test Suite - -Configure your YAML file as per your use cases using **key value** pairs. - -In this sample YAML file, we have mentioned: - -- **version** of the YAML file -- **Timeouts** for executing your project -- **Mode of execution** is [Autosplit](/support/docs/hyperexecute-auto-split-strategy/). You can also opt for [Matrix](/support/docs/hyperexecute-matrix-multiplexing-strategy/) or [Hybrid](/support/docs/hyperexecute-hybrid-strategy/) mode. -- **Pre and Post** commands -- **Reports and Artefacts** that will be generated after the completion of tests -- and other necessary YAML Parameters - -```yaml ---- -version: 0.1 -globalTimeout: 90 -testSuiteTimeout: 90 -testSuiteStep: 90 - -runson: linux - -autosplit: true -retryOnFailure: true - -maxRetries: 1 -concurrency: 2 - -env: -# PAT: ${{ .secrets.testKey }} -TARGET_OS: LINUX - -cacheKey: '{{ checksum "requirements.txt" }}' -cacheDirectories: -- pip_cache -pre: -- pip3 install -r requirements.txt --cache-dir pip_cache -post: -- cat yaml/linux/behave_hyperexecute_autosplit_sample.yaml -upload: -- reports/test_report.json - -mergeArtifacts: true - -uploadArtefacts: -- name: TestReports -path: -- reports/** - -report: true -partialReports: -type: json -location: / -frameworkName: extent - -# Details about HTML Formatter at https://pypi.org/project/behave-html-formatter/ -testDiscovery: -type: raw -mode: remote -command: grep -nri 'Feature' features -ir --include=\*.feature | sed 's/:.*//' - -# Reports are generated in the pretty JSON format -testRunnerCommand: behave -f json.pretty -o reports/test_report.json $test - -jobLabel: [selenium-behave, linux, autosplit] -``` - -### Step 4: Execute your Test Suite - -> **NOTE :** In case of MacOS, if you get a permission denied warning while executing CLI, simply run **`chmod u+x ./hyperexecute`** to allow permission. In case you get a security popup, allow it from your **System Preferences** → **Security & Privacy** → **General tab**. - -Run the below command in your terminal at the root folder of the project: - -```bash -./hyperexecute --config RELATIVE_PATH_OF_YOUR_YAML_FILE -``` - -OR use this command if you have not exported your username and access key in the step 2. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE `} - -### Step 5: Monitor the Test Execution - -Visit the [HyperExecute Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) and check your Job status. - -### Step 6: Download Artifacts and Reports - -HyperExecute also facilitates the provision to download the [Artifacts](/support/docs/hyperexecute-artifacts/) and [Reports](/support/docs/hyperexecute-reports/) on your local machine. Click on the corresponding button to download your generated artifacts and reports. - -## 2. Testing Using Gitpod - -You can also use the Gitpod platform to execute our sample repository. It will fetch all the sample codebases and trigger the CLI to execute the tests. - -Follow the below steps to run Test using Gitpod: - -**Step 1:** Click '**Open in Gitpod**' button. You will be redirected to Login/Signup page. This button is configured to redirect you to the Gitpod platform where you will be able to execute our sample repository. - -**Step 2:** Login with TestMu AI credentials. Once logged in, a pop-up confirmation will appear, asking you to **'Proceed'** to the Gitpod editor in a new tab. The current tab will display the HyperExecute Dashboard. - -**Step 3:** Choose your preferred editor (we recommend VS Code Editor) - -**Step 4:** As you are running a sample project, Fetching of the Test Scripts, [HyperExecute YAML](/support/docs/deep-dive-into-hyperexecute-yaml/), [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) and Triggering your tests using the `Execution Command` will be automated. - -**Step 5:** Once you see the `Job Link` in the logs, you can visit the [HyperExecute dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) to see the tests getting executed. - -You can also implement [Secret Keys](/support/docs/hyperexecute-how-to-save-and-manage-secrets/) in your YAML file. - -## Navigation in Automation Dashboard - -Every test run on the HyperExecute has a unique *jobId* associated with it. Each *jobId* can in turn constitute single (or multiple) *groupId*(s). You can visit [HyperExecute Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) for checking the status of the test execution. - -You can seamlessly navigate between JobId's and taskId's. You need to click on the *testID* to navigate from the HyperExecute logs to the Automation Dashboard. - -The snapshot below shows the videos, logs and other meta data for that specific *test_ID* - -> For any query or doubt, please feel free to contact us via **24×7 chat support** or you can also drop a mail to **support@testmuai.com**. - -## Using the Behave Agent Skill with TestMu AI - -The [behave-skill](https://github.com/LambdaTest/agent-skills/tree/main/behave-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The behave-skill package includes: - -``` -behave-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing Behave Agent Skill - -Install a Behave Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/behave-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/behave-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only behave-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). diff --git a/static/docs/behave-with-selenium-running-behave-automation-scripts-on-testmu-selenium-grid.md b/static/docs/behave-with-selenium-running-behave-automation-scripts-on-testmu-selenium-grid.md deleted file mode 100644 index 7dcb7062c..000000000 --- a/static/docs/behave-with-selenium-running-behave-automation-scripts-on-testmu-selenium-grid.md +++ /dev/null @@ -1,111 +0,0 @@ -# Selenium With Behave - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Run Behave tests on the TestMu AI cloud grid. This guide covers setup, running a sample test, configuring capabilities, and testing locally hosted pages. - -**Sample repo** -All the code used in this guide is available in the sample repository. - - View on GitHub - -## Prerequisites - -Complete these steps before running your first Behave Selenium test. - -1. Install the latest Python build from the [official website](https://www.python.org/downloads/). -2. Verify that **pip** is installed in your system. Install **pip** from [official pip documentation](https://pip.pypa.io/en/stable/installation/). -3. Download the latest **Selenium Client** and its **WebDriver bindings** from the [official website](https://www.selenium.dev/downloads/). -4. Install **virtualenv** to isolate the build from other setups and ensure tests run with the specified module versions. -```bash -pip install virtualenv -``` - -## Step 1: Clone the Sample Project - -Clone the repository and set up the environment. - -```bash -git clone https://github.com/LambdaTest/Python-Behave-Selenium -cd Python-Behave-Selenium -``` - -Create a virtual environment and activate it: -```bash -virtualenv venv -source venv/bin/activate -``` - -Install the required packages: -```bash -pip install -r requirements.txt -``` - -## Step 2: Set Your Credentials - -Configure your credentials to connect to the TestMu AI Selenium Grid. - -Set TestMu AI `Username` and `Access Key` in environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -## Step 3: Configure Your Test Capabilities - -Define browser, version, and OS settings for your test run. - -In `config/config.json`, update your test capabilities. This code passes browser, browser version, and operating system information, along with TestMu AI Selenium grid capabilities via the capabilities object. - -```python -[ -{ -"platformName": "Windows 10", -"browserName": "chrome", -"browserVersion": "latest", -"build": "Behave Selenium Sample", -"name": "Behave Sample Test" -} -] -``` - -**Capabilities Generator** -Use the TestMu AI [Capabilities Generator](https://www.testmuai.com/capabilities-generator/) to auto-generate the capabilities class for your test requirements. - -## Step 4: Run the Test - -Execute the Behave test from the command line. - -```bash -behave features/test.feature -``` - -## Step 5: View Your Results - -Check the test output on the console and the TestMu AI dashboard. - -Visit the [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) to view your test results. The dashboard provides: - -- Text logs for each test step -- Screenshots captured during execution -- Video recordings of the full test session - -## Run Behave Tests Using Agent Skills - -Use AI coding assistants to generate and run Behave tests with the TestMu AI Agent Skill. - -The [behave-skill](https://github.com/LambdaTest/agent-skills/tree/main/behave-skill) is part of [TestMu AI Agent Skills](https://github.com/LambdaTest/agent-skills/) - structured packages that teach AI coding assistants how to write production-grade test automation. - -Install the skill: - -```bash -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/behave-skill .claude/skills/ - -# For Cursor / Copilot -cp -r agent-skills/behave-skill .cursor/skills/ -``` - -Install all available framework skills at once by cloning the repository directly into your tool's skills directory (e.g., `.claude/skills/`, `.cursor/skills/`). diff --git a/static/docs/beta-testers-management.md b/static/docs/beta-testers-management.md deleted file mode 100644 index 732969791..000000000 --- a/static/docs/beta-testers-management.md +++ /dev/null @@ -1,52 +0,0 @@ -# Beta Testers and Teams Management - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -**Beta Testers** in TestMu AI [App Distribution](/support/docs/app-distribution/) are the people you invite to download builds and receive new-build notifications. You can add testers individually, group them into **Beta Teams**, and then assign teams or individual testers to any app's sharing list. - -This is a one-time, **organization-level** setup. Once a tester is added, you can reuse them across every app in your organization. - -You can manage testers from the sidebar under **Real Device → App Management and Distribution → Beta Testers**. - -## Before You Begin - -- A TestMu AI account with **App Distribution enabled** for your organization. -- The **email addresses** of the testers you want to invite. Testers can be inside or outside your organization, new builds can be accessed via the link received in their mail inbox. - -## Add Beta Testers - -Beta Testers are the individual people you invite to access authenticated builds shared from an app's distribution settings. - -**Step 1:** Navigate to **Beta Testers** in the sidebar. - -**Step 2:** Click **+ Add Tester**. - -**Step 3:** Enter the tester's **email address**. New-build notifications will be sent to this email. - -**Step 4 (Optional):** Add the tester directly to one or more existing teams from the same dialog. - -**Step 5:** Click **Add Tester**. This sends the tester a welcome email so they can join your organization as a Beta Tester. - -**Pending invites** - -Until the tester accepts the invite, a **Pending** tag appears next to their email in the Beta Testers list. You can't add a tester to a team until they accept the invite. - -## Create Beta Teams - -Teams are groups of testers that make it easier to manage access and notifications at scale. Instead of adding individual testers to each app, you assign a team, and all current (and future) members of that team get access automatically. - -**Step 1:** Navigate to **Beta Teams**. - -**Step 2:** Click **+ Create Team**. - -**Step 3:** Enter a **team name** (for example, `QA Team`, `Release Candidates`, `External Testers`) and an optional description. - -**Step 4:** Select testers from the list to add as members. Only testers who have accepted their invite are selectable. - -**Step 5:** Click **Create Team**. - -When a team is assigned to an app's sharing list, all members of that team can download authenticated builds and receive notifications. - -## Related Resources - -- [App Distribution on Real Devices](/support/docs/app-distribution/) - upload builds, configure sharing, and notify testers. diff --git a/static/docs/biometric-authentication-on-real-devices.md b/static/docs/biometric-authentication-on-real-devices.md deleted file mode 100644 index 06a2b3cf4..000000000 --- a/static/docs/biometric-authentication-on-real-devices.md +++ /dev/null @@ -1,41 +0,0 @@ -# Biometric Authentication on Real Devices - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Biometric Authentication is a security mechanism that utilizes unique biological traits to verify user identities. This feature is commonly implemented in native applications to enhance security while providing a frictionless user experience. - -TestMu AI has extended its testing capabilities to include Biometric Authentication support for both iOS and Android real devices. This enhancement enables developers and QA teams to thoroughly evaluate their application's biometric authentication functionality across diverse device ecosystems. - -**OS Version:** Biometric authentication is supported only on Android devices with OS version `11` or `above`, and on iOS devices with OS version `13` or `above`. - -## Supported Biometric Authentication APIs - -Below given is the list of Biometric Authentication APIs which are supported. Please ensure that your app uses these APIs only to enable Biometric Authentication. - - | Supported APIs | - |----------------| - | `LAContext`: The LAContext class is used to interact with the local authentication framework.| - | `evaluatePolicy`: The evaluatePolicy(_:localizedReason:reply:) method of LAContext used to actually perform the biometric authentication, such as verifying a user's face or fingerprint. | - |`canEvaluatePolicy`: The canEvaluatePolicy(_:error:) method of LAContext that checks whether a specific biometric authentication policy can be evaluated on the device.| - - | Supported APIs | - |----------------| - | `BiometricPrompt` class's `authenticate` method from the **AndroidX Biometric library** is used for Biometric authentication on Android devices. 📕For details, please check [AndroidX Biometric Documentation](https://developer.android.com/jetpack/androidx/releases/biometric). | - | **Android's Native** `BiometricPrompt` API is part of the Android platform and is used for Biometric authentication on Android devices.📕 For details, please check [Android Developers Documentation](https://developer.android.com/reference/android/hardware/biometrics/BiometricPrompt).| - -**note** -The OS versions and APIs mentioned above are subject to change as per the latest OS updates and best practices recommended by Android and iOS. - -## Biometric Authentication in Real Device App Testing - -This feature allows you to simulate successful or failed biometric authentication during real-time app testing. Follow the steps below to enable biometric authentication in your testing session: - -**Step 1:** Click on the **Real Devices** > **App Testing** - -**Step 2:** Select your required device and upload your application which supports the camera APIs listed above. - -**Step 3:** Now, hover over your app and click on the **Settings** icon. Toggle the **Biometric Authentication** option and click on **Save Changes** button. Click on **Start** to run your app on the selected device. - -After uploading your application, please wait atleast for 1-2 minutes for the application file patching process to complete. - -**Step 4:** Once your session starts and biometric authentication is enabled for the app, use Lambdatest's **Biometric Authentication** pop-up shown below to interact with any instance where biometric authentication is invoked. diff --git a/static/docs/biometric-authentication.md b/static/docs/biometric-authentication.md deleted file mode 100644 index fb14e2e31..000000000 --- a/static/docs/biometric-authentication.md +++ /dev/null @@ -1,61 +0,0 @@ -# Biometric Authentication - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Biometric Authentication is a security process that leverages the unique biological characteristics of individuals to verify their identities. In native applications, it is often used to provide a seamless, secure, and efficient user experience. - -TestMu AI now supports Biometric Authentication for real devices (iOS and android). This new feature allows you to test your application's biometric authentication functionality more accurately and ensures your app provides an optimal user experience across a range of devices and platforms. - -**OS Version:** Biometric authentication is supported only on Android devices with OS version `11` or `above`, and on iOS devices with OS version `13` or `above`. - -## Supported Biometric Authentication APIs - -Below given is the list of Biometric Authentication APIs which are supported. Please ensure that your app uses these APIs only to enable Biometric Authentication. - - | Supported APIs | - |----------------| - | `LAContext`: The LAContext class is used to interact with the local authentication framework.| - | `evaluatePolicy`: The evaluatePolicy(_:localizedReason:reply:) method of LAContext used to actually perform the biometric authentication, such as verifying a user's face or fingerprint. | - |`canEvaluatePolicy`: The canEvaluatePolicy(_:error:) method of LAContext that checks whether a specific biometric authentication policy can be evaluated on the device.| - -**note** - We don’t support apps using IOS Keychain APIs with Biometric access control. For more details [IOS Keychain Documentation](https://developer.apple.com/documentation/localauthentication/accessing-keychain-items-with-face-id-or-touch-id) - - | Supported APIs | - |----------------| - | `BiometricPrompt` class's `authenticate` method from the **AndroidX Biometric library** is used for Biometric authentication on Android devices. 📕For details, please check [AndroidX Biometric Documentation](https://developer.android.com/jetpack/androidx/releases/biometric). | - | **Android's Native** `BiometricPrompt` API is part of the Android platform and is used for Biometric authentication on Android devices.📕 For details, please check [Android Developers Documentation](https://developer.android.com/reference/android/hardware/biometrics/BiometricPrompt).| - | `BiometricManager` API 📕 For more details [BiometricManager Documentation](https://developer.android.com/reference/android/hardware/biometrics/BiometricManager) | - | `KeyguardManager` API 📕 For more details [KeyguardManager Documentation](https://developer.android.com/reference/android/app/KeyguardManager) | - -The OS versions and APIs mentioned above are subject to change as per the latest OS updates and best practices recommended by Android and iOS. - -## Biometric Authentication in App Automation - -### Using desired Capabilities - -To enable biometric authentication feature in your automation script, set the capability `enableBiometricsAuthentication` to `true`. - -```java -DesiredCapabilities capabilities = new DesiredCapabilities(); -capabilities.setcapability("enableBiometricsAuthentication",true) -``` - -### Using Lambda-Hook - -To implement `pass` or `fail` conditions for biometric authentication in your automation script, you have to use our Lambda-hook mentioned below: - -```python -driver.execute_script("lambda-biometric-injection=fail") -``` - -| TYPE | KEY | DATA TYPE| VALUES | -|------|-----|----------|--------| -| capability (for both iOS & android)|enableBiometricsAuthentication|boolean| `true` OR `false` -| lambda-hook | lambda-biometric-injection || `lambda-biometric-injection=fail` OR `lambda-biometric-injection=pass` | - -- To use the web hook, passing `enableBiometricsAuthentication` capability is mandatory. -- If you plan to use the **Uninstall Lambda Hook**, and then the **Install Lambda Hook** for the same application, biometric authentication can be used afterwards only via the lambda-hook. In this case, the initial capability won't be applicable anymore. -- When biometric capability is `true`, set `autoAcceptAlerts` and `autoDismissAlerts` to `false`. For help, contact [support](https://www.testmuai.com/support/). - -> 📕 Check the [documentation for Biometrics Authentication](/support/docs/biometric-authentication-on-real-devices/) feature in Manual App Testing on Real devices diff --git a/static/docs/bitbucket-integration.md b/static/docs/bitbucket-integration.md deleted file mode 100644 index aeba4bce2..000000000 --- a/static/docs/bitbucket-integration.md +++ /dev/null @@ -1,77 +0,0 @@ -# Bitbucket Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Bitbucket – A web-based repository hosting service owned by Atlassian for facilitating version control in development projects. It supports the project with a revision control system based on either Git or Mercurial. It is free to use, though they also offer a commercial plan. With Bitbucket, you can perform **access control** to your source-code, **workflow control** to impose a project workflow. Easy code revaluation through pull requests offering in-line annotation & Integration with Jira for end-to-end development tracing. - -The TestMu AI Bitbucket Integration allows you to create an issue directly in your repository from TestMu AI platform. Push an issue to your respective repository anytime, even in the middle of your test session. The fields populated by you when marking as a bug through TestMu AI are displayed as information on the issue in the repository for a testing instance. - -**Bitbucket Integration with TestMu AI, like all of our other integrations to 3rd party applications, is available for freemium as well as premium plan.** - -## How To Integrate Bitbucket With Your TestMu AI Account? - -**Step 1:** Login to your TestMu AI account. You would need to have Admin or User level access to see and install integrations. - -**Step 2:** Select **‘Integrations’** from the left navigation menu bar. This will guide you to a screen where you will find a list of 3rd party applications, available to integrate with your TestMu AI account. - -**Step 3:** Click on **‘Add’** under the block that says ‘Bitbucket’. - -**Note:** Make sure you have an account signed up with Bitbucket before you begin the integration. If you don’t have then you will be redirected to a screen where you can create one. If you already have an account setup for any of the Atlassian tools then you may use the same credentials as that one. - -**Step 4.** Once you click on the sign up button, you will be asked to provide a unique username. After providing the username, you need to click **‘Continue’.** - -**Step 5.** As you press "ADD" button, you will be directed to an authentication page. Here you will need to grant TestMu AI the access to your user-owned resources on Bitbucket. Hit on the blue button that says **"Grant access".** - -**Why we ask for your login credentials?** - -Bitbucket APIs uses OAuth 2.0’s [authorization code grant flow](https://tools.ietf.org/html/rfc6749#section-4.1) for generating access tokens on user’s behalf. - -**What is an Access token?** - -Access tokens are strings with authorization key required to access an API. They are issued to the client-server and are usually opaque. They are used for requesting access to protected, user-specific resources. Access tokens are vital from a security point of view & can be generated in different formats, depending upon security requirements specified on the resource server. - -That is it, you will be notified with a prompt message on top mentioning that Bitbucket is successfully installed. If you look at your Integrations again and you will be able to notice a **green tick** indicating the same. You can now experience bug logging in a fly from any of your running test session in TestMu AI to your respective repository directly by a single click. - -## Logging Your First Bug Through Bitbucket Integration? - -**Note:** Enable the **Issue tracker** in your respective Bitbucket repository. For enabling the Issue tracker: - - - Visit your repository. - - Go to **Settings.** - - After settings, click on **Issue tracker.** You will find it under **FEATURES.** - - Select the button for **Public issue tracker** & click **Save.** - -**Step 1:** Go for any of the test from the left navigation menu. For demo, we will be taking **"Real Time Test"** option. - -**Step 2:** Present a URL of the web-app you need to test in the dialog box. After that, select any configuration for browser and operating system of your choice & click **‘Start‘.** - -**Step 3:** After the VM is launched and operable. You can perform testing on your web-app for finding bugs. If a bug gets revealed, then you need to click on the **Bug icon** from the left panel for [capturing a screenshot](https://www.testmuai.com/full-page-screen-capture) of the same. We have highlighted that option with yellow in the below image. - -**Step 4:** After a screenshot is captured, you can annotate any issue or a task with an **in-built image editor.** Once you are done highlighting the bug, click on the button that says **"Mark as Bug".** - -When marking as bug, make sure that you have a repository already created into your Bitbucket account. If you don’t have one then you will be encountered with an error message as displayed in the below image. - -**Step 5:** If you are up and ready with a repository, then once you click on **"Mark as Bug"** button a Bitbucket specific form would open up. Fill the fields as per your requirement. - - - You can choose out of your **‘Repositories’,** for logging your UI observation. - - You can set the appropriate **‘Label’** for the same. - - You can assign it to a colleague by populating the field **‘Assignee’.** - - You also get to post a **‘Description’** to help relate the cause of the issue or the task. - -**Step 6:** Click on **"Create Bug"** & observe it being successfully marked through a single click effort. Once you hit the button, you will get **prompt messages** on top of your Virtual Machine indicating the progress of bug logging. You will get a prompt message in few seconds mentioning **"Bug successfully marked"** indicating that the UI observation has been pushed to your Bitbucket repository. - -**Step 7:** Visit your repository. You will be able to notice the logged issue right away under the Issues tab on the left. - -**Step 8:** Click on the **"Issue title."** All the data you provided through TestMu AI would already be presented in it. TestMu AI automatically includes test environment details and related screenshots under your work in Bitbucket. You will also find a clickable link that will open the screenshot attachment. - -## How To Remove Bitbucket Integration? - -You can work with one integration at a time. So if you would want to integrate to a similar 3rd party application, then you would have to **remove** your current integration. Here is how you can do that. - -**Step 1:** Login to your TestMu AI account. - -**Step 2:** Select ‘Integration’ from the left navigation menu bar. This will guide you to a screen where you will find 3rd party applications, available to integrate with your TestMu AI account. - -**Step 3:** Click on ‘REMOVE’. You can find the remove button right under the **‘Bitbucket’** block which would be highlighted with a green tick. - -That was all you need to know for TestMu AI + Bitbucket Integration. Increase your productivity with our integrations. If you still have any questions for us, please feel free to let us know. Our experts are always available on **chat** to help you out with any roadblock regarding our product. Happy testing! diff --git a/static/docs/bitbucket-pipeline-with-hyperexecute.md b/static/docs/bitbucket-pipeline-with-hyperexecute.md deleted file mode 100644 index f3a50f1c4..000000000 --- a/static/docs/bitbucket-pipeline-with-hyperexecute.md +++ /dev/null @@ -1,77 +0,0 @@ -# Bitbucket Pipeline Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Bitbucket Pipelines is a CI/CD service that is built into Bitbucket. It enables you to build, test, and even deploy your code automatically based on a configuration file in your repository. - -This document will show you how to integrate Bitbucket Pipelines with HyperExecute to greatly shorten your test cycles - -### Prerequisites: -- You need to have a Bitbucket Cloud account. -- Your workspace must have at least one repository. - -## How To Integrate Bitbucket Pipeline with Hyperexecute - -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** -Download or Clone the code sample from the TestMu AI GitHub repository to run the tests on the HyperExecute. - - View on GitHub - -### 1. Log into your [Bitbucket](https://www.atlassian.com/software/bitbucket/bundle) cloud account. - -### 2. Select a Repository -- If you do not have a repository set up yet, select **Repositories** then select **Create your own repository** as shown below. - -- Fill in the required information, then select **Create repository**. - -### 3. Create Your Pipeline - -- In your repository, go to the sidebar on the left and select **Pipelines** - -- Click **Create your first pipeline** to scroll down to the template section. - -- Select **Starter pipeline** - -### 4. Define Your Environment Variables -- In the sidebar, go into your repository settings. -- In the **Pipelines** section, select **Deployments** - -- Click on any environment to: - - Change its name - - Set environment-specific deployment variables. - -- You can define custom variables that you can use in the YAML file. To add a variable, fill in the name, the value, check whether you want to encrypt it by clicking the secured box, and click **Add**. - -### 5. Run Your Job -After configuring your environment variables, select **Commit file** at the bottom of your screen to commit and run your test job. - -- You will be redirected to the screen below - -**Below is an example of a Hyperexecute job that was triggered through the above pipeline:** - -## Sample Bitbucket Workflow File - -```bash -image: ubuntu:latest # Adjust for macOS if needed - -pipelines: -default: -branches: -- master # Adjust as needed -steps: -## Download Hyperexecute CLI (descriptive name) -- name: Download Hyperexecute CLI -script: | -wget https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute -chmod u+x hyperexecute - -## Run Hyperexecute tests (descriptive name) -- name: Run Hyperexecute Tests -script: | -./hyperexecute --user --key --config -``` - -> -**Run your tests at speeds never seen before. Happy testing! :)** diff --git a/static/docs/bitbucket-pipelines-integration-with-testmu.md b/static/docs/bitbucket-pipelines-integration-with-testmu.md deleted file mode 100644 index 8c04e2c56..000000000 --- a/static/docs/bitbucket-pipelines-integration-with-testmu.md +++ /dev/null @@ -1,92 +0,0 @@ -# Bitbucket Pipelines Integration With TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Bitbucket Pipelines offers a free, automated continuous integration service through Bitbucket cloud servers. With Bitbucket Pipelines, teams can ship their product faster as they could rapidly test their code as soon as they commit. Once the code is tested, Bitbucket Pipeline deploys it into the Production environment automatically. It does so with the help of containers that can be fully customized according to your project requirements. - -TestMu AI integration with Bitbucket Pipelines will help you perform cross browser testing on 3000+ real browsers & browser versions through a Selenium Grid hosted on TestMu AI cloud servers. All you need is to set up your Bitbucket Pipelines with your TestMu AI account & you can trigger tests onto TestMu AI, directly from your Bitbucket Pipelines. - -In this topic, you will learn about the: - -- Prerequisites -- Setup to integrate TestMu AI with Bitbucket Pipelines -- Parallel execution on TestMu AI Selenium Grid - -## Prerequisites - -**Important:** Visit TestMu AI [GitHub repository for Bitbucket Pipelines](https://github.com/qa-repo/protractor-selenium-bitbucket-sample). - -- Make sure you have your [Bitbucket Credentials](https://bitbucket.org/dashboard/overview). - -- **TestMu AI Authentication Credentials** -Be aware of your TestMu AI authentication credentials i.e. your TestMu AI username, access key, and HubURL. You need to set them up as your environment variables. You can retrieve them from your TestMu AI automation dashboard by clicking on the key icon near the help button. - - - For Linux/Mac: - -``` -$ export LT_USERNAME= {YOUR_LAMBDATEST_USERNAME} -$ export LT_ACCESS_KEY= {YOUR_LAMBDATEST_ACCESS_KEY} -``` - - - For Windows: - -``` -$ set LT_USERNAME= {YOUR_LAMBDATEST_USERNAME} -$ set LT_ACCESS_KEY= {YOUR_LAMBDATEST_ACCESS_KEY} -``` - -## Integrating TestMu AI With Bitbucket Pipelines - -**Step 1:** Log in to your Bitbucket account. Go to repositories & import the TestMu AI Git repository for Bitbucket Pipelines. You will be facing the below screen after import is successful. - -**Step 2:** After importing the Git repository, fetch the [yml file](https://github.com/qa-repo/protractor-selenium-bitbucket-sample/blob/master/bitbucket-pipelines.yml) to run your automation tests using the Protractor framework on TestMu AI Selenium Grid. - -``` -# This is a sample build configuration for JavaScript. -# Check our guides at https://confluence.atlassian.com/x/14UWN for more examples. -# Only use spaces to indent your .yml configuration. -# ----- -# You can specify a custom docker image from Docker Hub as your build environment. -image: node:10.15.3 - -pipelines: -default: -- step: -caches: -- node -script: # Modify the commands below to build your repository. -- npm install -- npm run single -``` -**Step 3:** Now, you need to add TestMu AI environment variables into your Bitbucket pipelines. These environment variables will help to authenticate your test before they are executed on TestMu AI Selenium Grid. Go to **Settings** --> **General** --> **Username aliases** --> **Repository variables**. - -**Step 4:** After adding the environment variables, you need to visit your pipeline & click on **Rerun**. - -**Step 5:** Run the below command to execute a test on TestMu AI Selenium Grid. - -`npm run single` - -You will find the below log generated: - -``` -> protractor-lambdatest@0.1.0 single /opt/atlassian/pipelines/agent/build -> protractor conf/single.conf.js -[10:23:53] I/launcher - Running 1 instances of WebDriver -[10:23:53] I/hosted - Using the selenium server at https://LT_USERNAME:LT_ACCESS_KEY@hub.lambdatest.com/wd/hub -Started -. -1 spec, 0 failures -Finished in 7.713 seconds -[10:24:10] I/launcher - 0 instance(s) of WebDriver still running -[10:24:10] I/launcher - chrome67.0 #01 passed -``` - -## Running Tests In Parallel On TestMu AI Selenium Grid - -TestMu AI Selenium Grid will help you execute multiple test cases in parallel. You can run a single test scenario over numerous browsers + OS combinations, simultaneously. You can also run different test cases in a similar browser + OS combinations. This would significantly trim down the time taken on your cross browser testing activities. - -For running test in parallel, replace the run command from **single** to **parallel** as you update the pipeline’s **yml file**. - -**Replace:** npm run single - -**With:** npm run parallel diff --git a/static/docs/bitrise-integration-xcuitest.md b/static/docs/bitrise-integration-xcuitest.md deleted file mode 100644 index b6c025206..000000000 --- a/static/docs/bitrise-integration-xcuitest.md +++ /dev/null @@ -1,58 +0,0 @@ -# Running XCUITests In Bitrise CI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Bitrise offers a mobile CI/CD platform as a service (PaaS) to accelerate the automation of your software development projects. It provides different mobile-first features like exhaustive mobile stack coverage, store code anywhere in cloud or on-premise, and zero hardware setup that make building, testing, and deploying faster and easier. - -Using TestMu AI with Bitrise, you can effortlessly perform [XCUI testing](https://www.testmuai.com/xcuitest-app-testing) of your mobile apps on an [online device farm](https://www.testmuai.com/online-device-farm) of real Android devices. - -## Prerequisites - -1. Ensure you have access to TestMu AI [real device cloud](https://www.testmuai.com/real-device-cloud). If not, please [contact sales](https://www.testmuai.com/contact-us). - -2. A Bitrise account. - -3. A GitHub / BitBucket / GitLab or other repository to connect with Bitrise. - -**Note** -If you are getting started with Bitrise CI, please refer to our documentation on [running Appium tests in Bitrise CI](/support/docs/bitrise-integration/). - -## Configuring The TestMu AI App Automate - XCUI Step In Bitrise - -1. Use the search bar to find **TestMu AI App Automate - XCUITest**. - -2. Click **TestMu AI App Automate- XCUITest** card. - -3. In **Input Variables**, enter **App ipa path** and **Test suite ipa path**. - -> You can add a public link if your files are accessible through public link. In case, you don't an apk link handy, you can use TestMu AI’s sample application Proverbial for running your first test. -* [Proverbial iOS](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_ios.ipa) -* [XCUITest Suite](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_ios_xcuitest.ipa) - -* In **TestMu AI username**, click Select secret variable. - -4. Provide the list of device and operating system version on which you wish to run your tests. - -Other options: - -| Key | Value | Capability Description -| -------- | -----| ------------ | -| Device List | TYPE: STRING DEFAULT: Untitled `"iPhone 12-14", "iPhone 6S-13"`| Add the devices here on which the test will be executed. Devices are comma separated. | -|Build Name | TYPE: STRING DEFAULT: Untitled `iOS Small Run` | You can group your tests like a job containing multiple tests. | -| Queue Timeout | TYPE: STRING DEFAULT: 600 `300` | Enter the time in seconds after which you want your build to timeout from queue. | -| Idle Timeout | TYPE: STRING DEFAULT: 120 `120` | Enter the time in seconds for maximum running time on a test in the build. | -| Tunnel | TYPE: BOOLEAN `True`OR `False` | To test local applications with TestMu AI. | -| Tunnel name | TYPE: STRING `RabbitHole` | Name of the tunnel. | -| Device logs | TYPE: BOOLEAN DEFAULT: FALSE`True`OR `False`| Enable Device Logs that will show information on the actions performed by the device. | -| Network logs | TYPE: BOOLEAN DEFAULT: FALSE`True`OR `False` | Enable Network Logs that will show the information on the data transmitted & received by the device. | -| Video | TYPE: BOOLEAN DEFAULT: TRUE`True`OR `False` | Video recording of the screen. | - -[XCUI Testing On TestMu AI](/support/docs/getting-started-with-xcuitest/#capabilities-supported) - -[Automation Capabilities Generator For Selenium and Appium](https://www.testmuai.com/capabilities-generator/) - -5. Click **⌘ + S** to save the step. - -6. Let’s go back to the Build and click on `Rebuild` to Build the Step. - -7. Once the build process is complete, visit **TestMu AI App Automation Dashboard** to view your test results. diff --git a/static/docs/bitrise-integration.md b/static/docs/bitrise-integration.md deleted file mode 100644 index 0e08c420a..000000000 --- a/static/docs/bitrise-integration.md +++ /dev/null @@ -1,91 +0,0 @@ -# Running Appium Tests In Bitrise CI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Bitrise offers a mobile CI/CD platform as a service (PaaS) to accelerate the automation of your software development projects. It provides different mobile-first features like exhaustive mobile stack coverage, store code anywhere in cloud or on-premise, and zero hardware setup that make building, testing, and deploying faster and easier. - -Using TestMu AI with Bitrise, you can effortlessly perform [Appium testing](https://www.testmuai.com/appium-mobile-testing) of your mobile apps on an [online device farm](https://www.testmuai.com/online-device-farm) of 3000+ real devices and OS versions. - -## Prerequisites - -1. Ensure you have access to TestMu AI [real device cloud](https://www.testmuai.com/real-device-cloud). If not, please [contact sales](https://www.testmuai.com/contact-us). - -2. A Bitrise account. - -3. A GitHub / BitBucket / GitLab or other repository to connect with Bitrise. - -## Configuring The App Build In Bitrise CI - -1. Login to your Bitrise account. - -2. From Dashboard, click **Add New App on web UI**. - -3. In the **Choose Account** section, select the privacy of your app and click **Next**. - -4. Choose a source code provider (GitHub, BitBucket, GitLab or others). - -5. You will be now asked to authenticate your source code provider with Bitrise. - -On successful authentication, you'll receive a confirmation message. - -6. Now again click **Next**. - -7. Select your repository from the version control system. - -8. From the Setup repository access section, click **No, auto-add SSH key** to add your SSH keys. - -9. Enter your branch name and click **Next**. Your app will be configured and this will take few minutes. - -10. In **Project build configuration**: - -* Specify module and click **Next**. - -* Specify variant and click **Next**. - -11. A build information will show up. Click **Confirm**. - -12. Choose and upload your App icon. Then click **Choose Image and Proceed**. - -13. From the **Webhook setup**, register or skip Webhook registration. In this case, let's select **Register a Webhook for me!** - -14. Now click on the message that says - *We've kicked off your first test build for you!* - -Your app build will now trigger and start running. - -## Generating The App URL - -1. Once your build process is completed, click **Edit workflow**. - -2. Click + icon under **Android Unit Test**. - -3. A search steps box will appear. Enter *TestMu AI Upload* and click TestMu AI Upload card. - -4. In **Input variables** category: - -* Provide your App path. - -* In **TestMu AI username**, click Select secret variable. - -* Enter the key (it can be anything), your TestMu AI username and click **Add new**. - -* In **TestMu AI access key**, click Select secret variable. - -* Enter the key (it can be anything), your TestMu AI access key and click **Add new**. - -* Provide an App name. - -* Enter your Custom Id (optional). - -5. After that, click **Ctrl+S**. - -6. From the breadcrumb navigation, click on the Repo name. - -7. A build will be shown that you just run. Click on the build. - -You'll be taken to the build interface as shown below. - -8. Scroll down and click on three dots adjacent to the TestMu AI icon. - -9. Scroll a bit more and you'll see an App URL. - -You're done! With this App URL, you can now perform Mobile App Automation. This app will also be available under **Real Time Testing > Real Time > App Testing** in the uploaded apps section. diff --git a/static/docs/breeze-integration-with-testmu.md b/static/docs/breeze-integration-with-testmu.md deleted file mode 100644 index beb60a2d7..000000000 --- a/static/docs/breeze-integration-with-testmu.md +++ /dev/null @@ -1,65 +0,0 @@ -# Breeze Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Breeze facilitates project management on-cloud through an intuitive platform, offering Gantt charts, kanban tool, task management, and time tracking to help you finish your release cycles on time. With Breeze, you can keep track of deadlines, an individual effort of your colleagues, and acknowledge who is doing what in a project. - -TestMu AI integration with Breeze will help you log UI bugs or observations in your Breeze project as you perform [cross browser testing](https://www.testmuai.com) on TestMu AI. You can capture a screenshot of your website over a browser discrepancy, highlight the UI bug, report that bug to your colleague, provide a summary of the issue, all from the middle of your test-session at TestMu AI. All of the details that you may mention around a bug would be auto-populated in your respective Breeze project. - -## How To Integrate Breeze With Your TestMu AI Account? - -**Step 1:** Login to your TestMu AI account. You would need to have Admin or User level access to see and install integrations. - -**Step 2:** Select ‘Integration’ from the left navigation menu bar. This will guide you to a screen where you will find a list of 3rd party applications, available to integrate with your TestMu AI account. - -**Step 3:** Click on ‘Add’ under the block that says ‘Breeze’. - -**Step 4:** You need to specify your API token for authenticating your TestMu AI account to your Breeze instance. - -**Step 5:** Go to your Breeze project and hit the profile image holder on top-right. Select the option for Personal settings & password. - -**Step 6:** Select the tab for Integrations and copy the API token. - -> **What is an API token?** -When two different applications interact with each other, a server-to-server communication is established. API token is needed for authenticating user identity from one server to another, by fetching user-owned resources available on the server from where the communication is initiated. API token has been a necessity for maintaining security and data integrity of any application. They are just as vital to an end user as they are to an application. API tokens are relevant if an end user logs in an application through a 2-step verification process. -**Important Note:** It is always a best practice to never share your API tokens, as someone else could authenticate a third-party application on your behalf and misuse it. In case you feel your API token for Breeze project is misplaced then don’t worry! Breeze allows your the option to regenerate the API token. - -**Step 7:** Paste this API token while integrating Breeze from your TestMu AI account. Now, hit the Install button. - -**Step 8:** Kudos! You have successfully integrated your TestMu AI account with Breeze your project. - -## Log Your First Bug Through TestMu AI Integration With Breeze - -**Step 1:** Go for any of the test from the left navigation menu. For demo, we will be taking "Real Time Test" option. - -**Step 2:** Present a URL of the web-app you need to test in the dialog box. After that, select any configuration for browser and operating system of your choice & hit ‘Start‘. - -**Step 3:** After the VM is launched and operable. You can perform testing on your web-app for finding bugs. If a bug gets revealed, then you need to click on the Bug icon from the left panel for capturing a screenshot of the same. We have highlighted that option with yellow in the below image. - -**Step 4:** After a screenshot is captured, you can annotate any issue or a task with an in-built image editor. Once you are done highlighting the bug, click on the button that says "Mark as Bug". - -**Step 5:** After clicking on "Mark as Bug" button a form would open up. Fill the fields as per your requirement, and create the issue. - -* You can select your Breeze project. -* Select the particular list in which you want to put your task. -* Select an assignee. -* Provide a name to your task. -* Mention a description to relay important message around the UI observation. - -> Note: After you click on "Create Bug", you will be able to observe it being successfully marked through a single click effort. You will get prompt messages on top of your Virtual Machine indicating the progress of bug logging. After few seconds you will be notified with a prompt message "Bug successfully marked" indicating that the screenshot has been pushed to your Breeze project. - -**Step 6:** Now, go to your Breeze project and you will notice your logged task under the specific list your assigned it through TestMu AI. - -**Step 7:** As you click on the task. You will be able to find all the details you provided while marking the bug through TestMu AI. You would also find a URL routing to the screenshot you captured while cross browser testing using TestMu AI. - -## Remove TestMu AI Integration With Breeze - -> You can work with one integration at a time. So if you would want to integrate to a similar 3rd party application, then you would have to remove your current integration. Here is how you can do that. - -**Step 1:** Login to your TestMu AI account. - -**Step 2:** Select ‘Integrations’ from the left-navigation menu. This will guide you to a screen where you will find 3rd party applications, available to integrate with your TestMu AI account. - -**Step 3:** Click on ‘REMOVE’. You can find the remove button right under the ‘Breeze’ block which would be highlighted with a green tick. - -> That was all you need to know for TestMu AI + Breeze Integration. Increase your productivity with our integrations. If you still have any questions for us, please feel free to let us know. Our experts are always available on **chat** to help you out with any roadblock regarding our product. Happy testing! 🙂 diff --git a/static/docs/browser-cloud-api-reference.md b/static/docs/browser-cloud-api-reference.md deleted file mode 100644 index e24f82b67..000000000 --- a/static/docs/browser-cloud-api-reference.md +++ /dev/null @@ -1,58 +0,0 @@ -# API Reference (Coming Soon) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The REST API will expose all TestMu AI Browser SDK capabilities over HTTP, making -TestMu AI Browser Cloud accessible from any programming language. In the meantime, the [TestMu AI Browser SDK quickstart](/support/docs/launch-first-session/) provides full access to all these capabilities from Node.js. - -Here's a preview of the planned endpoint structure: - -## Planned Endpoints - -``` -Sessions -POST /v1/sessions Create a session -GET /v1/sessions List sessions -GET /v1/sessions/:id Get session details -GET /v1/sessions/:id/live Get live session details -DELETE /v1/sessions/:id Release a session -DELETE /v1/sessions Release all sessions - -Context -GET /v1/sessions/:id/context Get full context -PUT /v1/sessions/:id/context Set full context -GET /v1/sessions/:id/context/cookies Get cookies -PUT /v1/sessions/:id/context/cookies Set cookies -DELETE /v1/sessions/:id/context Clear context - -Profiles -GET /v1/profiles List profiles -POST /v1/profiles Create/save profile -GET /v1/profiles/:id Get profile -PUT /v1/profiles/:id Update profile -DELETE /v1/profiles/:id Delete profile - -Files -POST /v1/sessions/:id/files Upload file -GET /v1/sessions/:id/files List files -GET /v1/sessions/:id/files/:path Download file -GET /v1/sessions/:id/files/archive Download all (zip) -DELETE /v1/sessions/:id/files/:path Delete file -DELETE /v1/sessions/:id/files Delete all files - -Extensions -POST /v1/extensions Register extension -GET /v1/extensions List extensions -GET /v1/extensions/:id Get extension -DELETE /v1/extensions/:id Delete extension - -Quick Actions -POST /v1/scrape Scrape a URL -POST /v1/screenshot Screenshot a URL -POST /v1/pdf Generate PDF from URL - -Tunnel -POST /v1/tunnel/start Start tunnel -POST /v1/tunnel/stop Stop tunnel -GET /v1/tunnel/status Get tunnel status -``` diff --git a/static/docs/browser-cloud-context.md b/static/docs/browser-cloud-context.md deleted file mode 100644 index 3b66b71c6..000000000 --- a/static/docs/browser-cloud-context.md +++ /dev/null @@ -1,132 +0,0 @@ -# Transfer Cookies and Storage Between Sessions - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Extract and inject browser state - cookies, localStorage, and sessionStorage - across sessions to preserve login and user data without re-authenticating. - -## Why You Need This - -Your agent needs to stay logged in across sessions. Without a way to preserve -authentication state, every new session starts from scratch - your agent has to -repeat the login flow each time. That wastes time, risks triggering security -alerts from frequent logins, breaks on MFA prompts, and burns LLM tokens if -your agent uses AI to navigate login pages. - -The Context Service solves this. It captures everything the browser remembers -about a user's session - login cookies, user preferences in localStorage, -shopping cart data in sessionStorage - and lets you inject that state into a -new session. Your agent logs in once, saves the context, and skips login -entirely in every future session. - -```typescript -interface SessionContext { -cookies?: Cookie[]; -localStorage?: Record>; -sessionStorage?: Record>; -} -``` - -## Before You Begin - -You need an active session with a connected page before you can extract or -inject context. If you have not set that up yet, see -[Connect to a session](/support/docs/connect-to-session/). - -## Framework Agnostic - -The Context Service auto-detects whether you pass a Puppeteer `Page` or a -Playwright `Page`/`BrowserContext`. The same API works with both - you never -need to specify which adapter you're using. - -## Extracting Context - -Get all browser state from a page: - -```typescript -const context = await client.context.getContext(page); - -context.cookies; // Array of cookies -context.localStorage; // { "origin": { "key": "value" } } -context.sessionStorage; // { "origin": { "key": "value" } } -``` - -Or extract individual parts: - -```typescript -const cookies = await client.context.getCookies(page); -const localStorage = await client.context.getLocalStorage(page); -const sessionStorage = await client.context.getSessionStorage(page); -``` - -## Injecting Context - -Set browser state on a new page: - -```typescript -await client.context.setContext(page, { -cookies: [ -{ name: 'session_id', value: 'abc123', domain: '.example.com', path: '/' } -], -localStorage: { -'https://example.com': { theme: 'dark', lang: 'en' } -}, -}); -``` - -Or set individual parts: - -```typescript -await client.context.setCookies(page, cookies); -await client.context.setLocalStorage(page, localStorageData); -await client.context.setSessionStorage(page, sessionStorageData); -``` - -## Clearing Context - -```typescript -await client.context.clearContext(page); // Clear everything -await client.context.clearCookies(page); // Just cookies -await client.context.clearStorage(page); // localStorage + sessionStorage -``` - -## Example: Transfer Login Between Sessions - -The most common use case - log in once, reuse the auth state: - -```typescript -// Session 1: Log in and capture -const session1 = await client.sessions.create({ adapter: 'puppeteer', ... }); -const browser1 = await client.puppeteer.connect(session1); -const page1 = (await browser1.pages())[0]; - -await page1.goto('https://app.example.com/login'); -await page1.type('#email', 'user@example.com'); -await page1.type('#password', 'password'); -await page1.click('#login-button'); -await page1.waitForNavigation(); - -const savedContext = await client.context.getContext(page1); -await browser1.close(); -await client.sessions.release(session1.id); - -// Session 2: Skip login entirely -const session2 = await client.sessions.create({ adapter: 'puppeteer', ... }); -const browser2 = await client.puppeteer.connect(session2); -const page2 = (await browser2.pages())[0]; - -await client.context.setContext(page2, savedContext); -await page2.goto('https://app.example.com/dashboard'); -// Already logged in! -``` - -## How It Works - -- **Puppeteer:** Uses CDP `Network.getAllCookies` / `Network.setCookie` for cookies, and `page.evaluate()` for localStorage/sessionStorage -- **Playwright:** Uses `context.cookies()` / `context.addCookies()` for cookies, and `page.evaluate()` for storage -- Framework detection is automatic based on the page object's available methods - -## Context vs Profiles - -The Context Service transfers state **within a single script run** (in memory). -If you need state to persist **across separate script runs** (on disk), use the -[Profiles for persistent state](/support/docs/browser-cloud-profiles/) instead. diff --git a/static/docs/browser-cloud-debugging.md b/static/docs/browser-cloud-debugging.md deleted file mode 100644 index 753a8db9b..000000000 --- a/static/docs/browser-cloud-debugging.md +++ /dev/null @@ -1,112 +0,0 @@ -# Replay and Debug Sessions - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Watch, replay, and troubleshoot your agent's browser sessions. Every session automatically records video, console logs, and network requests - accessible from the TestMu AI dashboard. - -## Built-In Observability - -When your agent runs at 3am and does something unexpected, you need to know -exactly what happened. Every session runs on TestMu AI's infrastructure, -which means every session automatically comes with video recordings, console -logs, and network request capture - no extra setup required. - -## TestMu AI Web Automation Dashboard - -Every session appears on the TestMu AI Web Automation Dashboard: - -``` -https://automation.lambdatest.com/logs/ -``` - -From the dashboard you can: - -- Watch **live video** of the browser as your agent interacts with it -- **Replay** recorded sessions after they complete -- View captured **console logs** -- Inspect **network requests** -- Review **screenshots** -- Check **test results** (pass/fail) - -## Organizing Sessions - -Use `build` and `name` in `lambdatestOptions` to organize sessions on the -dashboard. This is especially helpful when running multiple agents or scenarios: - -```typescript -const session = await client.sessions.create({ -lambdatestOptions: { -build: 'Price Monitor Agent - v2.1', -name: 'Extract Competitor Pricing - Amazon', -'LT:Options': { -username: process.env.LT_USERNAME, -accessKey: process.env.LT_ACCESS_KEY, -video: true, // Record video -console: true, // Capture console logs, -} -} -}); -``` - -## Debug URLs - -Every session provides URLs for viewing: - -```typescript -console.log(session.debugUrl); // Dashboard URL for this session -console.log(session.sessionViewerUrl); // Live stream of the browser -``` - -## SDK Console Output - -The SDK logs connection steps and actions to stdout: - -``` -Adapter: Connecting to session session_123_abc via Puppeteer... -Adapter: Set stealth user-agent: Mozilla/5.0 (Windows NT 10.0... -Adapter: Set stealth viewport: 1907x1063 -Adapter: Humanized interactions enabled -Adapter: Loading profile my-app-login -``` - -## Common Issues - -Here are the most common issues and how to fix them. - -**Connection Timeout** - -``` -Error: Timed out after 30000ms while waiting for the WebSocket -``` - -Causes: Invalid `LT_USERNAME` or `LT_ACCESS_KEY`, TestMu AI service outage, -or network firewall blocking WebSocket connections. - -Fix: Verify your credentials, check the TestMu AI status page, try a different -network. - -**Session Not Found** - -``` -Session session_xyz not found -``` - -Cause: Session was already released or timed out. The default timeout is 5 -minutes. - -Fix: Increase with `timeout: 600000` (10 minutes) in your session config. - -**Playwright Requires Node 18+** - -``` -Playwright requires Node.js 18 or higher. -``` - -Fix: Upgrade Node.js with `nvm install 18 && nvm use 18`. - -**Profile Not Loading** - -Cause: On the first run, there's no saved profile yet. - -Fix: This is normal. The profile is created when `browser.close()` is called. -Subsequent runs will load it. diff --git a/static/docs/browser-cloud-extensions.md b/static/docs/browser-cloud-extensions.md deleted file mode 100644 index 741c43c94..000000000 --- a/static/docs/browser-cloud-extensions.md +++ /dev/null @@ -1,73 +0,0 @@ -# Load Chrome Extensions - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Install Chrome extensions into cloud browser sessions so they behave identically to local browsers with your required extensions. - -## Overview - -Your agent workflow depends on a Chrome extension - an ad blocker, a cookie manager, or a custom tool your team built. Without it, the browser behaves differently and your automation may not work as expected. - -The Extension Service solves this by letting you register Chrome extensions and inject them into TestMu AI Browser Cloud sessions. You provide a cloud-hosted URL (S3) for each extension, and TestMu AI downloads and installs it into the browser instance when the session starts. Your cloud sessions behave the same way as your local browser. - -**Before You Begin** -Your extension must be hosted at a cloud URL (such as an S3 bucket) before you can register it. See [Current Limitations](#current-limitations) for details on what is supported today. - -## Loading Extensions into a Session - -First, register an extension. Then pass its ID when creating a session: - -```typescript -// Step 1: Register (one time) -const ext = await client.extensions.register({ -name: 'My Extension', -version: '1.0.0', -cloudUrl: 'https://s3.amazonaws.com/bucket/extension.zip', -}); - -// Step 2: Load into sessions -const session = await client.sessions.create({ -adapter: 'puppeteer', -extensionIds: [ext.id], // Pass registered extension IDs -lambdatestOptions: { ... } -}); -``` - -When `extensionIds` are provided, the session manager fetches the cloud URLs -and adds them to TestMu AI capabilities as `lambda:loadExtension`. - -## Managing Extensions - -```typescript -const extensions = await client.extensions.list(); - -const ext = await client.extensions.get('ext_abc123'); - -await client.extensions.delete('ext_abc123'); -``` - -## Extension Object - -```typescript -interface Extension { -id: string; -name: string; -version: string; -description?: string; -enabled: boolean; -createdAt: string; -cloudUrl?: string; // S3 URL -localPath?: string; // Local file path -} -``` - -## Supported Formats - -- `.zip` archives containing Chrome extension files -- `.crx` Chrome extension packages - -## Current Limitations - -- Extension upload to S3 must be done manually (via curl, AWS CLI, or your upload pipeline) -- The automated upload API through TestMu AI is not yet integrated -- Extensions only work with cloud sessions diff --git a/static/docs/browser-cloud-files.md b/static/docs/browser-cloud-files.md deleted file mode 100644 index 01b67d3dd..000000000 --- a/static/docs/browser-cloud-files.md +++ /dev/null @@ -1,77 +0,0 @@ -# Upload and Download Files - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Transfer files between your local machine and cloud browser sessions. Handle document uploads, file exports, and downloads without external storage services. - -## Why You Need This - -Your agent is filling out a government form that requires a document upload. The -form is running in a cloud browser, but the document is on your local machine. -Or your agent triggers a CSV download inside the cloud browser - a report, an -export, a receipt - and you need that file locally. - -The File Service bridges the gap. It transfers files between your local machine -and a remote cloud browser on TestMu AI Browser Cloud - without needing -external storage services like S3 or GCS. Files are sent directly through the -browser's page context using Base64 encoding. - -## Upload a File to the Cloud Browser - -```typescript -const fileBuffer = fs.readFileSync('document.pdf'); -await client.files.uploadToSession(session.id, fileBuffer, 'document.pdf'); -``` - -**How it works:** The buffer is Base64-encoded, sent to the cloud browser via -`page.evaluate()`, decoded in the browser, and set on the file input element. - -## Download a File from the Cloud Browser - -**By URL:** - -```typescript -const result = await client.files.downloadFromSession( -session.id, -'https://example.com/report.csv' -); -fs.writeFileSync('report.csv', result); -``` - -**How it works:** The cloud browser fetches the URL, reads the response with -`FileReader`, converts to Base64, and returns the data back to Node.js. - -For files triggered by button clicks (not direct URLs), the service uses CDP's -`Fetch.enable` to intercept the download. - -## Session-Scoped File API - -All file operations are also available under `client.sessions.files`: - -```typescript -await client.sessions.files.upload(session.id, buffer, 'file.txt'); - -const files = await client.sessions.files.list(session.id); - -const data = await client.sessions.files.download(session.id, '/path/to/file'); - -const archive = await client.sessions.files.downloadArchive(session.id); - -await client.sessions.files.delete(session.id, '/path/to/file'); - -await client.sessions.files.deleteAll(session.id); -``` - -## File Info - -```typescript -interface FileInfo { -path: string; -name: string; -size: number; -createdAt: string; -mimeType?: string; -} -``` - -Files are stored locally in a `.files/` directory organized by session ID. diff --git a/static/docs/browser-cloud-profiles.md b/static/docs/browser-cloud-profiles.md deleted file mode 100644 index 6924b6332..000000000 --- a/static/docs/browser-cloud-profiles.md +++ /dev/null @@ -1,138 +0,0 @@ -# Stay Logged In Across Runs with Profiles - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Persist and reuse browser state - cookies, auth, and settings - across separate script runs. Profiles auto-save when the browser closes and auto-load when you use the same profile ID. - -## Why You Need This - -The [Context service for cookies](/support/docs/browser-cloud-context/) transfers state between sessions -within a single script run. But what if your agent runs on a cron schedule, or -as a serverless function, or is manually triggered days apart? The in-memory -context is lost between runs. - -Profiles solve this by saving your browser's state - cookies, localStorage, -sessionStorage - to disk. Think of a profile as a saved browser identity: a set -of credentials, preferences, and session data that your agent can load each -time it runs. Your agent logs in once, the profile saves the auth state to -disk, and every future run - hours, days, or weeks later - loads the saved -state and skips login entirely. - -You might have a "salesforce-login" profile, a "github-login" profile, or a -"competitor-research" profile - each maintaining its own authentication state -and preferences. - -## How It Works - -1. You set `profileId` in your session config -2. On connect, the adapter checks for a saved profile at `.profiles/{profileId}.json` -3. If found, the saved cookies are loaded into the browser -4. On `browser.close()`, the current cookies are automatically saved back to the file -5. Next time you create a session with the same `profileId`, the saved state is restored - -## Getting Started - -```typescript -// Run 1: Your agent logs in. Profile is saved automatically on close. -const session = await client.sessions.create({ -adapter: 'puppeteer', -profileId: 'my-app-login', // This ID enables auto-save -lambdatestOptions: { ... } -}); - -const browser = await client.puppeteer.connect(session); -const page = (await browser.pages())[0]; - -await page.goto('https://app.example.com/login'); -// ... agent logs in ... - -await browser.close(); // ← Profile auto-saved here -await client.sessions.release(session.id); -``` - -```typescript -// Run 2 (days later): Agent loads saved state. No login needed. -const session2 = await client.sessions.create({ -adapter: 'puppeteer', -profileId: 'my-app-login', // Same ID = loads saved cookies -lambdatestOptions: { ... } -}); - -const browser2 = await client.puppeteer.connect(session2); -const page2 = (await browser2.pages())[0]; - -await page2.goto('https://app.example.com/dashboard'); -// Already logged in - cookies were restored from the profile -``` - -> **First run note:** On the very first run, there's no saved profile yet. -> This is normal. The profile file is created when `browser.close()` is called. -> Subsequent runs will find and load it. - -## Profile File Format - -Profiles are stored as JSON files at `.profiles/{profileId}.json`: - -```json -{ -"id": "my-app-login", -"cookies": [ -{ -"name": "session_token", -"value": "abc123...", -"domain": ".example.com", -"path": "/", -"expires": 1735689600, -"httpOnly": true, -"secure": true -} -], -"updatedAt": "2024-01-15T10:30:00.000Z" -} -``` - -## Manual Profile Management - -Beyond the automatic `profileId` flow, you can manage profiles directly: - -```typescript -// Save a profile manually -await client.profiles.saveProfile('my-profile', page, { note: 'after login' }); - -// Load a profile into a page -await client.profiles.loadProfile('my-profile', page); - -// List all saved profiles -const profiles = await client.profiles.listProfiles(); - -// Delete a profile -await client.profiles.deleteProfile('my-profile'); -``` - -## Profiles vs Context Service - -| | Context Service | Profile Service | -|---|---|---| -| **Where state lives** | In memory (JS object) | On disk (`.profiles/` directory) | -| **Lifetime** | Single script run | Across runs (days/weeks) | -| **Use case** | Transfer state between sessions in the same script | Maintain login state between separate agent invocations | -| **How to use** | Manual `getContext()` / `setContext()` | Automatic via `profileId` in session config | - -**Use Context** when your agent creates multiple sessions in one run. -**Use Profiles** when your agent runs on a schedule and needs to stay logged in -between invocations. - -## Works With All CDP Adapters - -- **Puppeteer:** Saves/loads cookies via CDP page methods -- **Playwright:** Saves/loads cookies via `context.addCookies()` / `context.cookies()` -- **Selenium:** Saves/loads cookies via `driver.manage().addCookie()`, grouped by domain - -## Security Note - -Profile files contain session cookies and tokens in plain text. Add -`.profiles/` to your `.gitignore`: - -``` -.profiles/ -``` diff --git a/static/docs/browser-cloud-quick-actions.md b/static/docs/browser-cloud-quick-actions.md deleted file mode 100644 index aeada2e71..000000000 --- a/static/docs/browser-cloud-quick-actions.md +++ /dev/null @@ -1,126 +0,0 @@ -# One-Liner Scrape, Screenshot, and PDF - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Scrape content, take screenshots, or generate PDFs from any URL - without managing sessions, browsers, or cleanup yourself. You provide -a URL, you get back the result. The TestMu AI Browser SDK handles everything behind -the scenes. - -This is useful when your agent needs data from a page but doesn't need to -interact with it beyond extraction. - -## Scrape - -Extract content from any webpage in your choice of format. - -**Simplest form - just pass a URL:** - -```typescript -const result = await client.scrape('https://example.com'); -console.log(result.content); -``` - -**With options - control format, timing, and selectors:** - -```typescript -const result = await client.scrape({ -url: 'https://example.com', -format: 'markdown', // 'html' | 'markdown' | 'text' | 'readability' -delay: 3000, // Wait 3 seconds for JS-heavy pages -waitFor: '#content', // Wait for this CSS selector before extracting -}); -``` - -**Response:** - -```typescript -interface ScrapeResponse { -title: string; // Page title -content: string; // Extracted content in requested format -url: string; // Final URL (after redirects) -markdown?: string; // Markdown version -html?: string; // Raw HTML -metadata?: Record; // Meta tags -} -``` - -**Choosing a format:** - -| Format | What You Get | When To Use It | -|--------|-------------|----------------| -| `html` | Raw HTML of the page | When your agent needs to parse the DOM | -| `text` | Plain text, tags stripped | When you want minimal tokens for LLM input | -| `readability` | Cleaned article content (like Reader Mode) | When the page has an article you want to extract | -| `markdown` | HTML converted to markdown | When you want structure + low token count for LLMs | - -## Screenshot - -Capture a visual snapshot of any webpage. - -```typescript -// Simple -const result = await client.screenshot('https://example.com'); -fs.writeFileSync('screenshot.png', result.data); - -// With options -const result = await client.screenshot({ -url: 'https://example.com', -fullPage: true, // Capture entire scrollable page -format: 'jpeg', // 'png' | 'jpeg' | 'webp' -quality: 80, // JPEG/WebP quality (1-100) -delay: 2000, // Wait before capturing -}); -``` - -**Response:** `{ data: Buffer, format: string, width: number, height: number }` - -## PDF - -Generate a PDF document from any webpage. - -```typescript -// Simple -const result = await client.pdf('https://example.com'); -fs.writeFileSync('page.pdf', result.data); - -// With options -const result = await client.pdf({ -url: 'https://example.com', -format: 'A4', // 'A4' | 'Letter' | 'Legal' -landscape: false, -printBackground: true, -margin: { top: '1cm', right: '1cm', bottom: '1cm', left: '1cm' }, -}); -``` - -**Response:** `{ data: Buffer, pageCount: number }` - -## Standalone vs Session Mode - -Quick Actions operate in two modes: - -**Standalone mode (default).** The SDK creates a temporary headless browser -with stealth enabled, navigates to the URL, performs the operation, and closes -the browser. Fully automatic. No session management needed. - -**Session mode.** If your agent already has a session running and you want the -Quick Action to use that session's browser (with its cookies, tunnel, or -extensions), register the page first: - -```typescript -const session = await client.sessions.create({ ... }); -const browser = await client.puppeteer.connect(session); -const page = (await browser.pages())[0]; - -// Register the page for Quick Actions -client.quick.registerSessionPage(session.id, page); - -// Now Quick Actions use the existing session -const result = await client.scrape({ -url: 'https://example.com', -sessionId: session.id, -}); -``` - -This is useful when the page requires authentication, a tunnel connection, or -specific extensions to access. diff --git a/static/docs/browser-cloud-sdk-setup.md b/static/docs/browser-cloud-sdk-setup.md deleted file mode 100644 index a8688f39c..000000000 --- a/static/docs/browser-cloud-sdk-setup.md +++ /dev/null @@ -1,31 +0,0 @@ -# Set Up the Browser SDK - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The TestMu AI Browser SDK is the primary way to integrate your agents with TestMu AI Browser Cloud. It handles session creation, browser connections, stealth, profiles, and more from a single Node.js client. - -## TestMu AI Browser SDK - -The TestMu AI Browser SDK is the fastest way to get your agents into the Browser -Cloud. It gives you full programmatic control over sessions, adapters, stealth, -profiles, files, extensions, and tunnels directly from your Node.js application. - -### What you need to get started - -- **Node.js 16+** installed on your machine -- Install the SDK: - -```bash -npm install @testmuai/testmu-cloud -``` - -### Initialize the Client - -From here, you can create sessions, configure stealth mode, manage profiles, -upload files, load extensions, and open tunnels - all through a single client -instance. - -```typescript -import { Browser } from '@testmuai/testmu-cloud'; -const client = new Browser(); -``` diff --git a/static/docs/browser-cloud-session-configuration.md b/static/docs/browser-cloud-session-configuration.md deleted file mode 100644 index f329c83c0..000000000 --- a/static/docs/browser-cloud-session-configuration.md +++ /dev/null @@ -1,106 +0,0 @@ -# Configure Session Options - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Complete reference for all options available when creating a session with the TestMu AI Browser SDK. Use these to control stealth, persistence, browser settings, network, and extensions. - -When you call `client.sessions.create()`, you can pass these options to -configure the session's behavior, browser settings, and capabilities. - -## Quick Example - -Here is a typical `create` call that sets the most common options: - -```javascript -const session = await client.sessions.create({ -adapter: 'playwright', -dimensions: { width: 1920, height: 1080 }, -blockAds: true, -stealthConfig: { -humanizeInteractions: true, -randomizeUserAgent: true, -}, -proxy: 'http://my-proxy:8080', -lambdatestOptions: { -build: 'my-build', -name: 'my-test', -user: process.env.LT_USERNAME, -accessKey: process.env.LT_ACCESS_KEY, -}, -}); -``` - -The sections below describe every available option in detail. - -## Required Options - -| Option | Type | Description | -|--------|------|----------------| -| `adapter` | `'puppeteer' \| 'playwright' \| 'selenium'` | Which automation library to use | -| `lambdatestOptions` | `object` | TestMu AI capabilities (build name, test name, credentials) | - -## Stealth Options - -| Option | Type | Default | Description | -|--------|------|---------|----------------| -| `stealthConfig.humanizeInteractions` | `boolean` | `false` | Add random delays to clicks and typing | -| `stealthConfig.randomizeUserAgent` | `boolean` | `true` | Pick random user-agent from pool | -| `stealthConfig.randomizeViewport` | `boolean` | `true` | Add ±20px jitter to viewport | -| `stealthConfig.skipFingerprintInjection` | `boolean` | `false` | Disable all stealth | - -→ Learn more: [Avoid bot detection with Stealth Mode](/support/docs/browser-cloud-stealth/) - -## Persistence Options - -| Option | Type | Description | -|--------|------|----------------| -| `profileId` | `string` | Load/save persistent browser profile | -| `sessionContext` | `SessionContext` | Pre-load cookies, localStorage, sessionStorage | - -## Browser Options - -| Option | Type | Default | Description | -|--------|------|---------|----------------| -| `dimensions` | `{ width, height }` | - | Browser viewport size | -| `userAgent` | `string` | - | Custom user-agent string | -| `headless` | `boolean` | - | Run in headless mode | -| `timeout` | `number` | `300000` | Session timeout in ms (5 min) | -| `blockAds` | `boolean` | - | Block advertisements | -| `solveCaptcha` | `boolean` | - | Enable CAPTCHA solving | - -## Network Options - -| Option | Type | Description | -|--------|------|----------------| -| `proxy` | `string` | Proxy URL | -| `geoLocation` | `string` | Geolocation code (e.g. `'US'`, `'IN'`) | -| `tunnel` | `boolean` | Enable TestMu AI tunnel | -| `tunnelName` | `string` | Named tunnel identifier | -| `region` | `string` | TestMu AI data center region | -| `optimizeBandwidth` | `boolean \| config` | Block images/media/styles | - -## Extension Options - -| Option | Type | Description | -|--------|------|----------------| -| `extensionIds` | `string[]` | Chrome extension IDs to load | - -## Session Object - -After creation, you receive a Session object with these fields: - -```typescript -interface Session { -id: string; // Unique session ID -websocketUrl: string; // WebSocket URL for adapter connection -debugUrl: string; // TestMu AI dashboard URL -config: SessionConfig; // Original configuration -status: 'live' | 'released' | 'failed'; -createdAt: string; // ISO timestamp -timeout: number; // Session timeout in ms -dimensions: Dimensions; // Viewport dimensions -sessionViewerUrl?: string; // Live session viewer URL -userAgent?: string; // Resolved user-agent -stealthConfig?: StealthConfig; // Active stealth settings -} -``` diff --git a/static/docs/browser-cloud-session-lifecycle.md b/static/docs/browser-cloud-session-lifecycle.md deleted file mode 100644 index 4f398d98c..000000000 --- a/static/docs/browser-cloud-session-lifecycle.md +++ /dev/null @@ -1,115 +0,0 @@ -# Handle Session Lifecycle - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Learn how sessions move from creation to release, and how to manage timeouts and cleanup. -Understanding the session lifecycle helps you avoid leaked sessions, reduce wasted resources, and ensure clean recordings on your dashboard. - -## Session States - -Every session passes through a simple set of states during its lifetime: - -``` -create() ──→ live ──→ released -└──→ failed -``` - -**Live.** The session has been created and is ready for connections. Your agent -can connect, navigate, and interact with the browser. The session stays in this -state until you release it or it times out. - -**Released.** The session has been explicitly released by your code, or it -reached its timeout. The browser is closed, resources are freed, and the -recording is saved to the TestMu AI dashboard. - -**Failed.** Something went wrong - a crash, connection loss, or infrastructure -error. Failed sessions are automatically cleaned up. - -## Session Timeout - -Every session has a timeout that defines how long it stays alive. The default is -**5 minutes** (300,000 ms). After this time, the session is automatically -released whether your agent is still using it or not. - -You can adjust the timeout when creating a session: - -```typescript -const session = await client.sessions.create({ -adapter: 'puppeteer', -timeout: 600000, // 10 minutes -lambdatestOptions: { ... } -}); -``` - -For quick scrapes, the default 5 minutes is usually enough. For multi-step -workflows where your agent navigates through several pages, you may want -10–30 minutes. - -## Releasing Sessions - -When your agent is done, release the session explicitly. This frees resources -immediately and ensures a clean recording on the dashboard: - -```typescript -// Release a single session -await client.sessions.release(session.id); -``` - -If your agent manages multiple sessions in parallel, you can release all of -them at once: - -```typescript -// Release all active sessions -await client.sessions.releaseAll(); -``` - -## Listing and Retrieving Sessions - -You can check which sessions are currently active and retrieve details about any -specific session: - -```typescript -// List all active sessions -const sessions = client.sessions.list(); - -// Get details of a specific session -const session = client.sessions.retrieve('session_12345_abc'); -``` - -## Live Session Details - -While a session is running, you can get real-time information about it: - -```typescript -const details = await client.sessions.liveDetails(session.id); - -console.log(details.pages); // Currently open pages/tabs -console.log(details.wsUrl); // WebSocket URL -console.log(details.sessionViewerUrl); // Live viewer URL -``` - -## Best Practices - -**Always release sessions when done.** Don't rely on timeouts alone - they -exist as a safety net, not as your primary cleanup mechanism: - -```typescript -const session = await client.sessions.create({ ... }); -try { -const browser = await client.puppeteer.connect(session); -// ... your agent's work ... -await browser.close(); -} finally { -await client.sessions.release(session.id); -} -``` - -**Use `releaseAll()` in your shutdown handler.** If your agent process crashes, -you want to make sure no sessions are left running: - -```typescript -process.on('SIGINT', async () => { -await client.sessions.releaseAll(); -process.exit(0); -}); -``` diff --git a/static/docs/browser-cloud-sessions-overview.md b/static/docs/browser-cloud-sessions-overview.md deleted file mode 100644 index 3eeefda8e..000000000 --- a/static/docs/browser-cloud-sessions-overview.md +++ /dev/null @@ -1,50 +0,0 @@ -# Understanding the Sessions API - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The Sessions API lets you create and control cloud-based browser sessions -through simple SDK calls. Each session is like giving your AI agent its own -dedicated browser window, but running on TestMu AI Browser Cloud and controlled -through your code. - -## Sessions at a Glance - -When you create a session, you get a fully isolated Chrome browser running in -TestMu AI Browser Cloud that your code controls. Each browser has its own -state, cookies, and storage, so your AI agent can navigate the web, interact -with sites, and maintain context across multiple steps without interference -from other sessions. - -Specifically, each session gives you: - -1. A **session ID** to track and manage it -2. A **WebSocket URL** to connect your automation tool (Puppeteer, Playwright, or Selenium) -3. A **debug URL** to watch the session on the TestMu AI dashboard -4. A **session viewer URL** for real-time streaming of the browser - -## Before You Begin - -If you have not set up your account and installed the SDK yet, [launch your first session](/support/docs/launch-first-session/) before working with sessions. - -## How It Works - -``` -Your Agent TestMu AI Cloud -│ │ -├── client.sessions.create() ────────→ │ Spins up a real Chrome browser -│ │ Returns session ID + WebSocket URL -│ │ -├── client.puppeteer.connect() ──────→ │ Your agent drives the browser -│ page.goto(...) │ via WebSocket (CDP) -│ page.click(...) │ -│ page.type(...) │ -│ │ -├── browser.close() ─────────────────→ │ Browser disconnects -│ │ -└── client.sessions.release() ───────→ │ Session cleaned up -│ Resources freed -``` - -Your agent creates a session, connects to it using its preferred automation -library, does its work, and releases the session when done. That's the entire -lifecycle. diff --git a/static/docs/browser-cloud-skills.md b/static/docs/browser-cloud-skills.md deleted file mode 100644 index 1cd3b40d0..000000000 --- a/static/docs/browser-cloud-skills.md +++ /dev/null @@ -1,188 +0,0 @@ -# AI Agent Skills - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Give any AI agent a real browser. The `browser-cloud` skill is a structured instruction file that tells an LLM assistant exactly how to generate production-grade browser automation code using TestMu AI Browser Cloud. Drop it into Claude, Cursor, or any LLM tool that supports custom skills, and your agent can spin up cloud browsers on demand. - -No local Chrome. No infrastructure. Just agents that can actually use the web. - -## What the Skill Does - -When you load the skill into your AI tool, the agent gains the ability to: - -- **Spin up isolated cloud browser sessions** via `@testmuai/browser-cloud` -- **Connect via Puppeteer, Playwright, or Selenium** -- **Handle stealth, auth persistence, tunnels, file transfer, and observability** -- **Wrap browser actions** into LangChain, CrewAI, OpenAI function calling, or any custom agent loop - -The skill is **agent-agnostic** - it works with any LLM or agent framework, not just Claude. - -## Before You Begin - -- **Node.js 16+** installed (18+ for Playwright adapter) -- **TestMu AI account** - [Sign up at testmuai.com](https://www.testmuai.com) (free tier available) -- **SDK installed:** - -```bash -npm i @testmuai/browser-cloud -``` - -- **Credentials configured:** - -```bash -# .env -LT_USERNAME=your_testmuai_username -LT_ACCESS_KEY=your_testmuai_access_key -``` - -Get credentials at **testmuai.com → Settings → Account Settings**. - -## Install the Skill - -Install `browser-cloud` from the LambdaTest browser-cloud-skills repo with the Skills CLI: - -```bash -npx skills add https://github.com/LambdaTest/browser-cloud-skills --skill browser-cloud -``` - -Requires Node.js and network access. After installation, your environment picks up the skill automatically where supported. You can also open `SKILL.md` manually if your tool does not integrate with the CLI. - -## Load the Skill into Your AI Tool - -Copy the contents of `browser-cloud/SKILL.md` into a **Project instruction** or **system prompt**. - -Add to `.cursor/rules/browser-cloud.mdc` or paste into **Cursor Settings → Rules for AI**. - -Paste `browser-cloud/SKILL.md` as a **system prompt** or **instruction file**. The skill is plain markdown - it works anywhere. - -## Ask Your Agent to Build Something - -Once the skill is loaded, ask naturally: - -- *"Write me an agent that scrapes competitor pricing from three URLs in parallel"* -- *"Build a LangChain tool that gives my agent access to a real browser"* -- *"Create a browser agent that logs into our staging app and extracts the dashboard data"* -- *"Set up a cloud browser session with stealth mode and Playwright"* - -The agent reads the skill, picks the right pattern, and writes complete, runnable TypeScript. - -## Examples - -### Standalone Scraping Agent - -Scrape a single URL or batch-scrape multiple URLs with concurrency control: - -```typescript -import { scrapeWithAgent, batchScrape } from './browser-cloud/examples/scrape-agent'; - -// Single URL -const result = await scrapeWithAgent('https://example.com'); - -// Multiple URLs, 3 concurrent sessions -const results = await batchScrape( -['https://a.com', 'https://b.com', 'https://c.com'], -3 -); -``` - -### Auth Persistence Across Cron Runs - -Log in once, save the session, and reuse it on every future run. Zero re-logins for scheduled agents: - -```typescript -import { fetchDashboard } from './browser-cloud/examples/auth-persist-agent'; - -// First run: logs in, saves session to .profiles/myapp-login.json -// All future runs: loads saved session, skips login entirely -const data = await fetchDashboard('https://app.example.com/dashboard'); -``` - -### LangChain Agent with Browser Access - -Build a LangChain agent that can browse the web as one of its tools: - -```typescript -import { buildBrowserAgent } from './browser-cloud/examples/langchain-browser-tool'; - -const agent = await buildBrowserAgent(); -const result = await agent.invoke({ -input: 'Research the latest pricing for our top three competitors.', -}); -``` - -### Parallel Research Across URLs - -Run multiple isolated browser sessions concurrently with progress tracking: - -```typescript -import { parallelResearch } from './browser-cloud/examples/parallel-research'; - -const results = await parallelResearch([ -'https://competitor-a.com/pricing', -'https://competitor-b.com/pricing', -'https://competitor-c.com/pricing', -], 3); -``` - -## Supported Frameworks - -| Framework | Language | Status | Reference | -|-----------|----------|--------|-----------| -| **LangChain** | TypeScript | Ready | `browser-cloud/examples/langchain-browser-tool.ts` | -| **OpenAI function calling** | TypeScript | Ready | `browser-cloud/references/integrations/openai-functions.md` | -| **Anthropic tool use** | TypeScript | Ready | `browser-cloud/references/integrations/openai-functions.md` | -| **CrewAI** | Python | Ready (subprocess bridge) | `browser-cloud/references/integrations/crewai.md` | -| **AutoGen** | Python | Use OpenAI pattern | `browser-cloud/references/integrations/openai-functions.md` | -| **Custom agent loop** | Any | Drop-in function | Any file in `browser-cloud/examples/` | - -## Supported Capabilities - -| Capability | What It Does | -|------------|-------------| -| **Stealth mode** | 15+ fingerprint patches (Puppeteer) / 6 init scripts (Playwright). Humanized click/type timing. Randomized UA and viewport. | -| **Auth persistence** | Save login state to disk. Reload on every run. Zero re-logins for scheduled agents. | -| **Context transfer** | Pass cookies and storage between sessions in the same script run. | -| **Tunnel** | Access localhost, staging, or VPN-gated URLs from cloud browsers. | -| **Parallel sessions** | Multiple isolated browsers running concurrently with batching support. | -| **File transfer** | Upload local files to cloud browser. Download exports, receipts, reports back to disk. | -| **Extensions** | Load Chrome extensions into sessions from S3 URLs. | -| **Quick Actions** | One-liner scrape, screenshot, and PDF. No session management required. | -| **Observability** | Every session gets video recording, console logs, and network capture in your TestMu AI automation session history. | - -## Skill Repo Structure - -``` -├── package.json ← npm scripts and SDK dependency -├── README.md -│ -└── browser-cloud/ ← Skill bundle -├── SKILL.md ← Load this into your AI tool -├── examples/ -│ ├── scrape-agent.ts ← Batch scraping with concurrency control -│ ├── form-fill-agent.ts ← Form interaction with Playwright + stealth -│ ├── auth-persist-agent.ts ← Login once, reuse session across runs -│ ├── parallel-research.ts ← Parallel sessions with progress tracking -│ └── langchain-browser-tool.ts ← Four LangChain tool variants -└── references/ -├── patterns/ -│ ├── quick-actions.md ← Scrape / screenshot / PDF one-liners -│ ├── session-navigate.md ← Multi-step navigation and interaction -│ ├── auth-profile.md ← Auth persistence patterns -│ ├── parallel-sessions.md ← Concurrent agent sessions -│ ├── tunnel.md ← Localhost and internal network access -│ └── files.md ← Upload/download between agent and browser -└── integrations/ -├── openai-functions.md ← OpenAI + Anthropic function calling -└── crewai.md ← CrewAI BaseTool (Python + Node bridge) -``` - -## Contributing - -Contributions welcome at the [browser-cloud-skills repo](https://github.com/LambdaTest/browser-cloud-skills). The most useful additions are: - -- New **pattern files** in `browser-cloud/references/patterns/` for use cases not currently covered -- New **integration files** in `browser-cloud/references/integrations/` for frameworks not yet supported -- New **runnable examples** in `browser-cloud/examples/` -- Improvements to `browser-cloud/SKILL.md` that make the LLM output more correct or more concise - -Keep `SKILL.md` under 500 lines. Every addition should remove or replace something weaker. The skill is a decision engine, not a tutorial. diff --git a/static/docs/browser-cloud-stealth.md b/static/docs/browser-cloud-stealth.md deleted file mode 100644 index 6864b10cb..000000000 --- a/static/docs/browser-cloud-stealth.md +++ /dev/null @@ -1,167 +0,0 @@ -# Avoid Bot Detection with Stealth Mode - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Make your agent's browser look like a real human user to bot-detection systems. Stealth mode masks fingerprints, randomizes user-agents, and humanizes interactions automatically. - -## Why Stealth Matters - -If your automated browser is getting blocked, served CAPTCHAs, or shown -different content than a real user would see, it is likely because the target -site's bot detection is flagging your session. Automated browsers leave -detectable fingerprints that these systems check for: - -- `navigator.webdriver` is set to `true` -- Missing Chrome plugins and runtime objects -- Exact, round viewport dimensions like `1920x1080` -- Identical user-agent strings across all sessions -- Inhuman interaction speeds - instant clicks, perfectly uniform typing - -Stealth mode in the TestMu AI Browser SDK removes or fakes these fingerprints, so -the cloud browser is indistinguishable from a regular human user's Chrome -session. - -## Enabling Stealth - -Add a `stealthConfig` to your session. That's it - the adapter handles -everything else automatically: - -```typescript -const session = await client.sessions.create({ -adapter: 'puppeteer', -stealthConfig: { -humanizeInteractions: true, -randomizeUserAgent: true, -randomizeViewport: true, -}, -lambdatestOptions: { ... } -}); -``` - -## Configuration Reference - -| Option | Type | Default | What It Does | -|--------|------|---------|----------------| -| `humanizeInteractions` | `boolean` | `false` | Adds random delays to clicks (50–150ms) and typing (30–130ms per character) | -| `randomizeUserAgent` | `boolean` | `true` | Picks a random user-agent from a pool of 7 realistic Chrome/Firefox strings | -| `randomizeViewport` | `boolean` | `true` | Adds ±20px random jitter to viewport dimensions | -| `skipFingerprintInjection` | `boolean` | `false` | Disables all stealth entirely (Puppeteer only, useful for comparison testing) | - -## What Gets Patched - -The specific evasions depend on which adapter you're using. - -**Puppeteer** uses `puppeteer-extra` with the stealth plugin. This is the most -comprehensive stealth solution available, patching: - -- `navigator.webdriver` (set to `undefined`) -- Chrome runtime objects -- WebGL vendor/renderer -- Permission queries -- Language and platform strings -- iframe contentWindow access -- Console.debug behavior -- And approximately 15 more evasions - -**Playwright** uses custom scripts injected via `page.addInitScript()`. These -scripts run **before** any page JavaScript executes, so detection scripts -cannot observe the original values: - -| Evasion | What It Does | -|---------|----------------| -| `navigator.webdriver = false` | Hides the automation flag | -| Fake `chrome.runtime` | Makes it look like Chrome extensions are present | -| Fake `navigator.plugins` | Injects 3 standard Chrome plugins (PDF Plugin, PDF Viewer, Native Client) | -| `navigator.languages` | Set to `['en-US', 'en']` | -| `permissions.query` | Returns `'denied'` for notifications (matches real Chrome) | -| WebGL spoofing | Reports "Intel Inc." / "Intel Iris OpenGL Engine" as GPU | - -> **Selenium** does not support stealth mode. Use Puppeteer or Playwright if -> your agent needs anti-detection. - -## User-Agent Randomization - -When `randomizeUserAgent` is enabled (it is by default when any `stealthConfig` -is set), the SDK picks from a pool of 7 realistic user-agent strings covering -Chrome 119–120 and Firefox 121 on Windows, macOS, and Linux. - -The user-agent is selected **once** at session creation and stays consistent for -the entire session. This is important - changing user-agent mid-session is a -detection signal. - -If you provide an explicit `userAgent` in the session config, it takes priority -over randomization. - -## Viewport Randomization - -When `randomizeViewport` is enabled, the SDK adds ±20 pixels of random jitter -to your base viewport dimensions. Instead of the perfectly round `1920x1080` -that bots typically fingerprint as, your session might run at `1907x1063` - -subtly different each time. - -## Humanized Interactions - -When `humanizeInteractions` is enabled, the SDK monkey-patches interaction -methods to add random delays: - -| Method | Adapter | Delay | -|--------|---------|--------| -| `page.click(selector)` | Puppeteer, Playwright | Random 50–150ms before clicking | -| `page.type(selector, text)` | Puppeteer, Playwright | Random 30–130ms between each character | -| `page.fill(selector, value)` | Playwright only | Random 50–150ms before filling | - -New pages created during the session automatically inherit the same humanized -behavior. - -## Disabling Stealth - -To explicitly disable stealth (useful for comparison testing or sites that -don't use bot detection): - -```typescript -const session = await client.sessions.create({ -adapter: 'puppeteer', -stealthConfig: { -skipFingerprintInjection: true, -randomizeUserAgent: false, -randomizeViewport: false, -}, -lambdatestOptions: { ... } -}); -``` - -## Testing Your Stealth Setup - -The TestMu AI Browser SDK includes a built-in test that visits the bot detection -site `bot.sannysoft.com` with stealth ON and OFF, saving screenshots for visual -comparison: - -Screenshots are saved to `test-output/`. Stealth ON shows green checks; stealth -OFF shows red failures. - -## How Stealth Works - -``` -Session Creation -│ -├─ stealthConfig present? -│ ├─ Yes → pick random UA, store on session.userAgent -│ └─ No → skip -│ -▼ -Adapter.connect() -│ -├─ Puppeteer: -│ ├─ skipFingerprintInjection? → raw puppeteer.connect() -│ └─ else → puppeteerExtra.connect() with stealth plugin -│ ├─ Set random UA via page.setUserAgent() -│ ├─ Set random viewport via page.setViewport() -│ └─ Humanize: monkey-patch click/type -│ -└─ Playwright: -├─ Inject stealth scripts via page.addInitScript() -├─ Auto-apply to new pages via context.on('page') -├─ Set random UA via page.evaluate() -├─ Set random viewport via page.setViewportSize() -└─ Humanize: monkey-patch click/type/fill -``` diff --git a/static/docs/browser-cloud-tunnel.md b/static/docs/browser-cloud-tunnel.md deleted file mode 100644 index 450dc5a6c..000000000 --- a/static/docs/browser-cloud-tunnel.md +++ /dev/null @@ -1,103 +0,0 @@ -# Access Localhost and Internal Networks Using Tunnel - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Create encrypted tunnels between your local machine and Browser Cloud. Let cloud browsers reach localhost dev servers, staging environments, and private network resources. - -## Why You Need This - -Your cloud browser can only reach public URLs by default. But your agent may need to access localhost, staging servers, or internal tools. For example: - -- Test a local development server before deploying -- Access a staging environment behind a corporate VPN -- Interact with internal tools and dashboards -- Work with a local API backend - -The Tunnel Service solves this by creating an encrypted connection between your local machine and TestMu AI's cloud infrastructure. Once the tunnel is running, your TestMu AI Browser Cloud sessions can reach any URL that your machine can reach - including `localhost`, private IPs, and internal hostnames. - -``` -Cloud Browser --(encrypted tunnel)--> Your Machine --> localhost:3000 ---> staging.internal.company.com ---> 192.168.1.50:8080 -``` - -## Automatic Tunnel (Recommended) - -The easiest approach. Set `tunnel: true` in your session config and the -TestMu AI Browser SDK handles starting and routing the tunnel automatically: - -```typescript -const session = await client.sessions.create({ -adapter: 'puppeteer', -tunnel: true, -tunnelName: 'my-tunnel', // Optional: name for identification -lambdatestOptions: { ... } -}); - -const browser = await client.puppeteer.connect(session); -const page = (await browser.pages())[0]; - -await page.goto('http://localhost:3000'); // This works! -``` - -If you set `tunnel: true` without a `tunnelName`, the SDK auto-generates a name -and starts the tunnel for you. - -## Manual Tunnel - -For more control - for example, starting the tunnel once and reusing it across -multiple sessions: - -```typescript -// Start the tunnel -await client.tunnel.start({ -user: process.env.LT_USERNAME!, -key: process.env.LT_ACCESS_KEY!, -tunnelName: 'my-tunnel', -}); - -console.log('Tunnel running:', client.tunnel.getStatus()); // true - -// Create sessions that use it -const session = await client.sessions.create({ -adapter: 'puppeteer', -tunnel: true, -tunnelName: 'my-tunnel', -lambdatestOptions: { ... } -}); - -// ... agent work ... - -// Stop when done -await client.tunnel.stop(); -``` - -## Tunnel Config - -```typescript -interface TunnelConfig { -user: string; // TestMu AI username -key: string; // TestMu AI access key -tunnelName?: string; // Named tunnel for identification -proxyHost?: string; // Corporate proxy host -proxyPort?: string; // Corporate proxy port -proxyUser?: string; // Proxy auth user -proxyPass?: string; // Proxy auth password -logFile?: string; // Log file path -} -``` - -## API - -```typescript -await client.tunnel.start(config); // Start tunnel -await client.tunnel.stop(); // Stop tunnel -client.tunnel.getStatus(); // Returns true/false -``` - -## How It Works - -The Tunnel Service uses the `@lambdatest/node-tunnel` package to create a -binary tunnel connection to TestMu AI infrastructure. The tunnel name is passed -as a TestMu AI capability so cloud browsers know to route their traffic through -your local machine. diff --git a/static/docs/browser-network-configurations.md b/static/docs/browser-network-configurations.md deleted file mode 100644 index ed215835e..000000000 --- a/static/docs/browser-network-configurations.md +++ /dev/null @@ -1,61 +0,0 @@ -# Network Logs and Configurations in real devices - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Network logs record every network interaction during your test session, from API calls and page requests to server responses and load times. These logs are stored in HAR format, giving you a complete snapshot of **network performance** for each run. By reviewing this data, you can identify slow endpoints, troubleshoot failed requests, and validate that your app communicates with the right services, all without leaving your testing workflow. - -On TestMu AI, you can enable **network configurations** to capture and analyze this traffic in real time on real devices. With flexible options such as content capture and domain-based filtering, you can focus on the most relevant network interactions while reducing noise from unrelated requests. This helps ensure faster debugging, clearer insights, and more efficient test runs. - -> To enable it for your organization, please contact us via **24×7 chat support** or you can also drop a mail to **support@testmuai.com**. - -### Use cases - -- **Debug API calls** by viewing HTTP/S request and response data directly from real device sessions. -- **Verify backend integration** by checking if calls are made to the correct endpoints. -- **Filter noise** by including/excluding specific hosts. -- **Reproduce production issues** that depend on specific network conditions or API behaviors. - -## Enabling Network Configuration - -### Step 1: Access App Testing - -Log into your TestMu AI dashboard and navigate to Real Devices > App Testing. - -### Step 2: Select your device or browser - -Choose a real Android or iOS device, upload your app (APK/IPA), or open a browser URL. - -### Step 3: Configure network log options - -Run your app. In the network logs for your device, open the **Configuration Options** slider and select the options you want to apply for your current session. - -### Configuration Options - -| Setting | Type | Default | Description | -|-----------------------------|------------------------------|----------|--------------------------------------------------------------------------------------------------------------| -| Capture Content | Toggle | Enabled | Captures the full response body of network requests in the DevTools Response tab. Disable if only headers or metadata are needed. | -| Include specific domains | Text input (comma-separated domains) | Empty | Capture network requests only for the specified domains. | -| Exclude specific domains | Text input (comma-separated domains) | Empty | Exclude network requests to the specified domains. | - -## Available configuration options - -### **Include specific domains**: - Use this to capture logs only for network requests to specified domains. Enable the toggle and add the domains separated by commas, as shown below. - -### **Capture content**: -Use this option to capture or exclude response content. This is off by default but can be enabled if needed, as shown below. - -### **Exclude specific domains**: -Use this to exclude certain domains from network logs. This works similarly to **Include specific domains**. If both options are enabled, **Exclude specific domains** takes precedence. - -## Best practices & tips - -- Exclude noisy domains to reduce clutter in your logs. -- Use the include list to focus on relevant domains. -- Avoid broad patterns that may capture excessive traffic. -- Save your settings for reuse in future sessions. - -- Available only on **Pro Plans** for manual testing. -- Apps from the App Store or Play Store may not support network capture. -- Certificate-pinned apps may fail unless pinned hosts are excluded. -:: diff --git a/static/docs/browser-testing-on-real-devices.md b/static/docs/browser-testing-on-real-devices.md deleted file mode 100644 index 1c35146ee..000000000 --- a/static/docs/browser-testing-on-real-devices.md +++ /dev/null @@ -1,23 +0,0 @@ -# Getting Started with Real Device Browser Testing - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Real device browser testing refers to the process of testing web applications and websites on real devices such as smartphones, tablets, and computers to ensure that the user interface and user experience are optimal across various device types, operating systems, and browser combinations. With TestMu AI’s real device cloud, you can test websites and web applications to ensure they perform seamlessly across 3000 + real mobile devices. - -**Step 1:** Login to your TestMu AI account. From the left sidebar, head to Real Device and select **Browser Testing**. - -**Step 2**: Select the **OS** and **browser** of your choice. - ->**Note:** Certain browsers might not be supported on certain devices. - -**Step 3**: Select the **real device** of your choice. You can also search the preferred real **Android** or **iOS** device in the search bar. - -**Step 4**: In the case of **Android** devices, simply enter the website URL in the address bar. - -In the case of **iOS** devices, you can enter the website URL once the session has started. - -**Step 5**: Select the **Tunnel** option in the top right corner to test your locally hosted and privately hosted websites. - -Check out our detailed [support documentation](/support/docs/testing-locally-hosted-pages/) to learn how to activate the LT Tunnel. - -**Step 6**: Click on Start to run your test session. diff --git a/static/docs/browserstack-to-testmu-migration-guide.md b/static/docs/browserstack-to-testmu-migration-guide.md deleted file mode 100644 index 91724ee51..000000000 --- a/static/docs/browserstack-to-testmu-migration-guide.md +++ /dev/null @@ -1,250 +0,0 @@ -# Migration from BrowserStack to TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI and BrowserStack both offer the cloud-based Selenium Grid. Hence, you can easily migrate your existing Selenium test automation scripts (or suites) from BrowserStack to TestMu AI - -[Note: This guide covers the migration of tests running on the cloud grid that uses Selenium 4 and Selenium 3] - -## Authentication -Firstly, you need to change the authentication in your configuration settings of your test suite. For running tests on TestMu AI Selenium Grid, you need to have a valid user_name and access_key to perform tests on our Grid. In case you do not have an account on TestMu AI, visit the TestMu AI signup page and create a new account. - -When migrating your Selenium 4 tests from BrowserStack to TestMu AI, the following updates are required in your existing code: - -1. Get TestMu AI Credentials: You can find these credentials under Account Settings > [Password & Security](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/security/username-accesskey) and copy your Username and Access Key, then add them to the .env file to keep them safe from public exposure. - -2. Create .env file: Securely store your TestMu AI credentials, create a .env file in the root of your project and add the following values: - -``` -LT_USERNAME="" -LT_ACCESS_KEY="" -``` - -Once the .env file is set up, ensure your test framework correctly reads these variables at runtime. This helps keep your authentication secure and avoids hard-coding credentials within your scripts. With the credentials in place, you’re now ready to update your Hub URL for TestMu AI execution. - -## Changes in Hub URL -You need to now change the hub URL in the configuration settings of your test suite. Hub URL is of type String and it defines the Hub location to which the Selenium tests would be submitted for execution. - -```js -@hub-cloud.browserstack.com/wd/hub -``` - -```js -@hub.lambdatest.com/wd/hub -``` - -## Configuring Selenium 4 Tests on TestMu AI - -Migrating your Selenium 4 tests from BrowserStack to TestMu AI requires only a few small updates. In this guide, we’ll walk through how to apply TestMu AI’s W3C-compliant capabilities, configure authentication, and execute your Selenium 4 test sessions on real browsers hosted in the TestMu AI cloud. - -### TestMu AI Automation Capabilities - -Capabilities generator let you configure the desired capabilities (or capabilities) which are configuration options that let you set the following: -* Desired browser -* Desired browser version -* Desired platform (or operating system) - -Optionally, you can also choose the Selenium version and other advanced options present in the Selenium Capabilities Generator. For this migration guide, we have only restricted to the three capabilities listed above. - -To generate capabilities use [TestMu AI Capabilities Generator](https://www.testmuai.com/capabilities-generator/) to define key automation testing parameters, such as browser, version, operating system, and additional test settings. - -For the migration, we have considered Java-based Selenium automation tests. Shown below are the screenshots of capabilities generator of BrowserStack and TestMu AI: - -```js -MutableCapabilities capabilities = new MutableCapabilities(); -HashMap bstackOptions = new HashMap(); -capabilities.setCapability("browserName", "Safari"); -bstackOptions.put("os", "OS X"); -bstackOptions.put("osVersion", "Tahoe"); -bstackOptions.put("browserVersion", "26.0"); -bstackOptions.put("userName", "YOUR_USERNAME"); -bstackOptions.put("accessKey", "YOUR_ACCESS_KEY"); -capabilities.setCapability("bstack:options", bstackOptions); -``` - -```js -SafariOptions browserOptions = new SafariOptions(); -browserOptions.setPlatformName("MacOS Tahoe"); -browserOptions.setBrowserVersion("26"); -HashMap ltOptions = new HashMap(); -ltOptions.put("username", ""); -ltOptions.put("accessKey", ""); -ltOptions.put("w3c", true); -browserOptions.setCapability("LT:Options", ltOptions); -``` - -## Configuring Selenium 3 Tests on TestMu AI - -Migrating your Selenium 3 tests from BrowserStack to TestMu AI requires only a few simple configuration changes. In the sections below, we’ll guide you through updating your test setup, applying Selenium 3–compatible capabilities, and running your test sessions on real browsers hosted on the TestMu AI cloud. - -### TestMu AI Automation Capabilities - -The Capabilities Generator allows you to configure the required desired capabilities for Selenium 3 test execution. These capabilities define the browser environment in which your tests will run. - -With Selenium 3, the core capabilities remain: - -* Desired browser -* Desired browser version -* Desired platform (or operating system) - -Optionally, you may also configure Selenium versions or other advanced settings based on your test requirements. However, for this migration guide, we will primarily focus on the three fundamental capabilities listed above. - -To generate capabilities for Selenium 3, you can use the [TestMu AI Capabilities Generator](https://www.testmuai.com/capabilities-generator/) , which allows you to define essential automation parameters such as browser name, browser version, operating system, along with any additional test configurations required for execution. - -Below are example capability representations from BrowserStack and TestMu AI to help you understand how to adapt your existing configuration during migration. - -```js -DesiredCapabilities caps = new DesiredCapabilities(); -caps.setCapability("browser", "Safari"); -caps.setCapability("os", "OS X"); -caps.setCapability("os_version", "Tahoe"); -caps.setCapability("browser_version", "26.0"); -caps.setCapability("browserstack.user", "YOUR_USERNAME"); -caps.setCapability("browserstack.key", "YOUR_ACCESS_KEY"); -``` - -```js -DesiredCapabilities capabilities = new DesiredCapabilities(); -capabilities.setCapability("browserName", "Safari"); -capabilities.setCapability("browserVersion", "26"); -HashMap ltOptions = new HashMap(); -ltOptions.put("username", ""); -ltOptions.put("accessKey", ""); -ltOptions.put("platformName", "MacOS Tahoe"); -ltOptions.put("visual", true); -ltOptions.put("video", true); -capabilities.setCapability("LT:Options", ltOptions); -``` - -## Hands On Guide - TestMu AI Migration - -Let’s walk through a practical example demonstrating how to migrate a Selenium 4 test to TestMu AI. In this scenario, we launch the latest Chrome browser on a Windows 10 machine. - -**Test Scenario**: - -This test script performs a basic text validation on the website [TestMu AI eCommerce Playground](https://ecommerce-playground.lambdatest.io/) and shows the expected execution results when running the test in the TestMu AI cloud. - -```java -// TextValidationTest.java -import org.openqa.selenium.WebDriver; -import org.openqa.selenium.remote.RemoteWebDriver; -import org.openqa.selenium.JavascriptExecutor; -import org.openqa.selenium.safari.SafariOptions; - -import java.net.URL; -import java.util.HashMap; - -public class TextValidationTest { - -public static void main(String[] args) throws Exception { - -String username = System.getenv("LT_USERNAME") == null ? -"Your LT Username" : System.getenv("LT_USERNAME"); - -String authkey = System.getenv("LT_ACCESS_KEY") == null ? -"Your LT AccessKey\n" : System.getenv("LT_ACCESS_KEY"); - -String GRID_URL = "https://" + username + ":" + authkey + "@hub.lambdatest.com/wd/hub"; - -SafariOptions browserOptions = new SafariOptions(); -browserOptions.setPlatformName("MacOS Tahoe"); -browserOptions.setBrowserVersion("26"); -HashMap ltOptions = new HashMap(); -ltOptions.put("username", ""); -ltOptions.put("accessKey", ""); -ltOptions.put("project", "Text Validation Test"); -ltOptions.put("build", "Text Validation Test Build"); -ltOptions.put("w3c", true); -browserOptions.setCapability("LT:Options", ltOptions); - -WebDriver driver = new RemoteWebDriver(new URL(GRID_URL), browserOptions); - -try { - -driver.get("https://ecommerce-playground.lambdatest.io/"); - -String expectedText = "This is a dummy website for Web Automation Testing"; -boolean isTextPresent = driver.getPageSource().contains(expectedText); - -if (isTextPresent) { -((JavascriptExecutor) driver).executeScript("lambda-status=passed"); -System.out.println("✔ Text validation PASSED"); -} else { -((JavascriptExecutor) driver).executeScript("lambda-status=failed"); -System.out.println("✘ Text validation FAILED"); -} - -} catch (Exception e) { -((JavascriptExecutor) driver).executeScript("lambda-status=pass"); -e.printStackTrace(); -} finally { -driver.quit(); // 🔹 Correctly placed – runs even if test fails -} -} -} - -``` - -```java -// TextValidationTest.java – Selenium 3 Configuration -import org.openqa.selenium.WebDriver; -import org.openqa.selenium.remote.DesiredCapabilities; -import org.openqa.selenium.remote.RemoteWebDriver; -import org.openqa.selenium.JavascriptExecutor; - -import java.net.URL; -import java.util.HashMap; - -public class TextValidationTest { - -public static void main(String[] args) throws Exception { - -String username = System.getenv("LT_USERNAME") == null ? -"Your LT Username" : System.getenv("LT_USERNAME"); - -String authkey = System.getenv("LT_ACCESS_KEY") == null ? -"Your LT AccessKey" : System.getenv("LT_ACCESS_KEY"); - -String GRID_URL = "https://" + username + ":" + authkey + "@hub.lambdatest.com/wd/hub"; - -DesiredCapabilities capabilities = new DesiredCapabilities(); -capabilities.setCapability("browserName", "Safari"); -capabilities.setCapability("browserVersion", "26"); -HashMap ltOptions = new HashMap(); -ltOptions.put("username", ""); -ltOptions.put("accessKey", ""); -ltOptions.put("platformName", "MacOS Tahoe"); -ltOptions.put("visual", true); -ltOptions.put("video", true); -capabilities.setCapability("LT:Options", ltOptions); - -WebDriver driver = new RemoteWebDriver(new URL(GRID_URL), capabilities); - -try { - -driver.get("https://ecommerce-playground.lambdatest.io/"); - -String expectedText = "This is a dummy website for Web Automation Testing"; -boolean isTextPresent = driver.getPageSource().contains(expectedText); - -if (isTextPresent) { -((JavascriptExecutor) driver).executeScript("lambda-status=passed"); -System.out.println("✔ Text validation PASSED"); -} else { -((JavascriptExecutor) driver).executeScript("lambda-status=failed"); -System.out.println("✘ Text validation FAILED"); -} - -} catch (Exception e) { -((JavascriptExecutor) driver).executeScript("lambda-status=pass"); -e.printStackTrace(); -} finally { -driver.quit(); // 🔹 Correctly placed – runs even if test fails -} -} -} -``` - -**Result** - -Visit TestMu AI Web Automation dashboard to view your test execution result. diff --git a/static/docs/bug-tracking-tools.md b/static/docs/bug-tracking-tools.md deleted file mode 100644 index fdb5cac71..000000000 --- a/static/docs/bug-tracking-tools.md +++ /dev/null @@ -1,49 +0,0 @@ -# Integrate with TestMu AI to report and track bugs - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Jira Integration With TestMu AI - -Jira Self Hosted Integration With TestMu AI - -Airtable Integration With TestMu AI - -Notion Integration With TestMu AI - -Linear App Integration With TestMu AI - -Azure DevOps Integration With TestMu AI - -YouTrack Integration With TestMu AI - -Mantis Integration With TestMu AI - -PagerDuty Integration With TestMu AI - -Bugasura Integration With TestMu AI - -Zoho Bug Tracker Integration With TestMu AI - -GoodDay Integration With TestMu AI - -BugHerd Integration With TestMu AI - -ZipBoard Integration With TestMu AI - -Bugzilla Integration With TestMu AI - -Bugsnag Integration With TestMu AI - -Assembla Integration With TestMu AI - -Taiga Integration With TestMu AI - -ServiceNow Integration With TestMu AI - -Zoho Project Integration With TestMu AI - -Rally Integration With TestMu AI - -**Note** - -If your favourite Bug Tracking tool is not in the list above mentioned, you can just give us a **shout** and we will have it ready shortly for you. diff --git a/static/docs/bugasura-integration.md b/static/docs/bugasura-integration.md deleted file mode 100644 index 07f2fe890..000000000 --- a/static/docs/bugasura-integration.md +++ /dev/null @@ -1,56 +0,0 @@ -# Bugasura Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Bugasura is a simple and collaborative bug-tracking platform for modern SaaS teams. It comes with the ability to customize workflows and uses an agile approach to report, track and close issues. - -Using TestMu AI and Bugasura integration, you can file bugs while performing [online browser testing](https://www.testmuai.com) of your web applications from the TestMu AI platform to your Bugasura instance and avoid the hassle of maintaining bugs and tasks manually. - -## Integrating Bugasura With TestMu AI - -**Note** -Before enabling integrations, ensure you have administrator or user level access. - -1. Sign in to your TestMu AI account. Don't have an account, register for free. - -2. Select **Integrations** from the left sidebar and then **Bugasura**. - -3. It will route you to the installing console where you need to enter your Bugasura API Key. - -### Getting The Bugasura User API Key - -1. Login to your Bugasura account. - -2. From the left sidebar menu, click avatar icon. - -3. Scroll down and copy your user API key. - -4. Paste the user API key that you just copied into the provided field and press **Install**. - -The Bugasura will be integrated with your TestMu AI account. - -## One-Click Bug Logging With Bugasura - -1. Once you are in the TestMu AI Dashboard, navigate to **Real Time Testing > Browser Testing** from the left sidebar menu. - -2. To perform live-interactive testing of your website or a web app, enter the web URL, select testing type (Desktop/Mobile), select the browser **VERSION**, **OS**, **RESOLUTION**, and click **START**. - -A cloud-based virtual machine will fire up running real browsers and operating system where you can test websites and web apps for filing bugs. - -3. If you come across any issue or bug, from the left in-session tool bar, click the **Bug** icon to take a screenshot of a web page. - -4. Using the in-built annotation tool bar, you can mark any issues, bugs and tasks on a web page. Once you have marked the bugs, click **Mark as Bug**. - -5. In the Bugasura ticket, fill the bug details like Teams, Summary, Description, Project, Assignee and Sprint. After filling all the bug details, click **CREATE ISSUE** to log the bugs in Bugasura. - -6. Visit TestMu AI Dashboard. From the left sidebar menu, select **Manage > Issue Tracker**. - -8. In Issue Tracker, you'll notice the logged bugs along with their test environment details. To view your logged bugs in Bugasura, click **upper right arrow in square** button. - -It will re-direct you to the Bugasura dashboard where you can find your filed bugs along with their test environment details. - -## Uninstalling The Bugasura Integration - -1. Sign in to your TestMu AI account and visit **Integrations** from the left sidebar menu. - -2. From **My Integrations**, click **REMOVE** under the Bugasura block. diff --git a/static/docs/bugherd-integration.md b/static/docs/bugherd-integration.md deleted file mode 100644 index c420d2362..000000000 --- a/static/docs/bugherd-integration.md +++ /dev/null @@ -1,48 +0,0 @@ -# BugHerd Integration with TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -[BugHerd](https://bugherd.com/) is a website bug tracker that allows you to easily create bug reports and track feedback, all without leaving your website. Developers can pin notes on the UI right on the element where the bug occurs, and also easily collect bug reports from users and team members. Along with each bug report, user technical data is automatically captured so there’s no back and forth, and BugHerd channels all feedback into an integrated Kanban board for easy task management. - -TestMu AI integrates with BugHerd, allowing users to file issues directly from the platform, assign them to team members, and attach screenshots during a test session, with populated fields displayed on the BugHerd platform for each testing instance. - -> BugHerd Integration with TestMu AI is available for freemium as well as premium plan. - -## Integrate BugHerd from your TestMu AI Account - -**Step 1:** Login to your TestMu AI account. You should have Admin or User level access to see and install integrations. - -**Step 2:** Click on Settings -> Integrations -> Bug Tracker. - -**Step 3:** Click on **Connect** button of ‘BugHerd’ block. - -**Step 4:** Now, provide your BugHerd API Token to establish integration with TestMu AI and click on **install** button. - -**Fetch your Bugherd API Token** - -- Visit your BugHerd account -> **Settings** tab. -- You will find the "**API Token**" under the section **Authentication Token**. Copy your API token. - -## Lodge your First Bug - -**Step 1:** Go to your BugHerd Projects, create a new one if you haven't already. - -**Step 2:** Enter your Project Name and the Website URL which you want to test and report bugs. Click on **Create Projects** button. - -**Step 3:** Now click on the **Open Website** button to start testing your website. - -**Step 4:** After detecting any bug on your site, or web-app, click on any of the method from **Tag an element**, **Tag the page**, **Video Feedback** that you want to use for reporting the bug. - -**Step 5:** Now update the comments of the bug and other details and click on **Create Task** button. - -**Step 6:** Go to your dashboard and check a ticket will be created for the same. - -## Uninstall BugHerd Integration - -**Step 1:** Login to your TestMu AI account. You should have Admin or User level access to see and install integrations. - -**Step 2:** Click on Settings -> Integrations -> Bug Tracker. - -**Step 3:** Click on the **Remove** button. - -> That was all you need to know for TestMu AI + BugHerd Integration. Increase your productivity with our integrations. If you still have any questions for us, please feel free to let us know. Our experts are always available on **chat** to help you out with any roadblock. diff --git a/static/docs/bugsnag-integration.md b/static/docs/bugsnag-integration.md deleted file mode 100644 index 6ba3d125f..000000000 --- a/static/docs/bugsnag-integration.md +++ /dev/null @@ -1,66 +0,0 @@ -# Bugsnag Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Bugsnag is an error-monitoring tool that allows your developers to quickly find, prioritize, and reproduce errors. Individual crashes are gathered together by underlying cause and shown as discrete errors in Bugsnag to allow for easy prioritizing; the error inbox can be sorted by event count, impacted users, error severity, or it can be limited to provide a list of errors matching a specified search parameter. - -The TestMu AI and Bugsnag integration allows you to perform one-click bug logging from TestMu AI to your Bugsnag projects while perform [cross browser testing](https://www.testmuai.com/) of your websites (and web apps). - -## Integrating Bugsnag With TestMu AI - -1. Login to your TestMu AI account. Make sure you have admin or user level access before you begin configuring integrations. - -2. From the left menu, select **Integrations** then **Bugsnag**. - -3. Enter your **Bugsnag Auth Token**. - -> -* **What is a Bugsnag Auth Token?** -The Bugsnag Auth token gives you access to information about your organization, projects, issues, and more, allowing you to create custom integrations with your Bugsnag data. - -### Getting the Bugsnag Token - -1. Sign in to your Bugsnag account. Click on the Profile avatar and then **My account settings**. - -2. Click on **Personal auth tokens**. - -3. To generate token, click on **Generate New Token**. - -4. Enter the Bugsnag Auth token name and press **Generate Token**. - -5. Copy the Bugsnag Auth API Token. - -6. Paste the copied Bugsnag Auth Token into the provided field and press **Install**. - -The Bugsnag will be integrated with your TestMu AI account. - -## One-Click Bug Logging With Bugsnag - -1. Choose **Real Time Testing** tab from the left menu. - -2. Specify the test URL, then select the browser, browser versions, operating system, and click **START**. - -A cloud-based virtual machine will be launched, providing you to conduct browser testing of websites and web apps. - -3. If you discover an issue while testing the websites, click the Bug icon to grab a screenshot. - -4. Once you have grabbed the screenshot, you can use the in-built image editor tools to annotate issues or tasks. After annotating the issues (or bugs), click on **Mark As Bug**. - -5. A Bugsnag ticket will be opened. Enter information such as Organization, Project, Release, Summary, and Description. Now, select **Create Issue**. - -Your bugs will be logged right away. - -6. Open the Bugsnag dashboard, you'll find the marked issues. - -7. Select the **Exceptions** tab and you will notice the data you submitted via TestMu AI would already be presented in it. TestMu AI attaches test environment data and related screenshots to tickets automatically. - -## Uninstalling The Bugsnag Integration - -1. Login to your TestMu AI account and visit Integrations from the left menu. - -2. Navigate to 'My Integrations'and click on **REMOVE** under the Bugsnag block. - -The Bugsnag integration will be uninstalled. - -> -For any queries, feel free to drop them at our **24x7 Chat Support** or you could also mail us directly at **[support@testmuai.com](mailto:support@testmuai.com)**. diff --git a/static/docs/bugzilla-integration.md b/static/docs/bugzilla-integration.md deleted file mode 100644 index 8b83dd6d7..000000000 --- a/static/docs/bugzilla-integration.md +++ /dev/null @@ -1,67 +0,0 @@ -# Bugzilla Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Bugzilla is an open-source bug tracking tool that allows you to keep track of software bugs. It is coded in Perl and uses the MYSQL database. By leveraging Bugzilla, users can stay linked with their customers or employees in order to communicate effectively about the product's issues.  - -In this documentation, we look at how to integrate Bugzilla with TestMu AI and perform one-click bug logging right away from the TestMu AI platform to your Bugzilla products. - -## Integrating Bugzilla With TestMu AI Account - -1. Login to your TestMu AI account. To install integrations, make sure you have admin or user level access. - -2. Visit 'Integrations' from the left menu and select **'Bugzilla'** under the **'BUG TRACKER'** category. - ->If you already have a TestMu AI integration with any bug tracking tool then you will be asked to uninstall the existing TestMu AI integration. - -3. You will be asked to enter your **Bugzilla Host**, **Bugzilla Email Address** and **Bugzilla Password**. - ->**Note**: Make sure you have created an account on Bugzilla. - -* What is my **Bugzilla Host**? -Bugzilla host is the domain name of your workspace. Open Bugzilla in any web browser and you find the Bugzilla URL (or domain name) from the browser's address bar. For Example: `https://lambdatest.bugzilla.com`. - -* What is my **Bugzilla Email Address** ? -Bugzilla Email Address is the address through which you sign into your Bugzilla account. - -* What is my **Bugzilla Password** ? -Bugzilla password is the password that you provide while creating your Bugzilla account. - -4. Enter your **Bugzilla Host**, **Bugzilla Email Address** and **Bugzilla Password** in the provided field and click on **'Install'**. - -The Bugzilla will be successfully integrated with your TestMu AI account. You'll also receive the confirmation message and an email of the same. - -## One-Click Bug Logging Through Bugzilla - -To demonstrate, we take an example of Real time testing. - -1. Select the Real Time Testing from the left menu. - -2. Provide your test URL, select the configurations that includes: Browser, Browser versions, OS and Resolution. Click on **'START'**. - -3. A virtual machine will launch where you can perform real time testing of websites and web apps. If you identify a bug, click on the **'Mark as Bug'** icon from the left bar to capture the screenshot. - -4. Once the screenshot is captured, annotate issues or tasks that you have encountered using in-built image editor tools. After annotating the bugs, click on **'Mark As Bug'**. - -5. A Bugzilla ticket will open up. Fill the details like Product, Component, Version, Priority, Operating System and so on. After populating the required fields, click on **'Create Issue"**. - -Once you click on Create Issue, your bugs will be marked successfully. - -6. To view your logged bugs, visit **Issue Tracker** from the TestMu AI left navigation menu. You'll notice the marked bugs along with the Test details. - -7. To view your logged bugs in Bugzilla, click on the right diagonal arrow icon. - -You'll be routed to the Bugzilla page where you can find the logged issue. - -Scroll down to view the complete details of the marked bugs: Screenshot, Test ID, Reproduce on TestMu AI link, Test URL, OS, Browser and Resolution. - -## Uninstalling The Bugzilla Integration - -1. Login to your TestMu AI account and visit Integrations from the left menu. - -2. Navigate to 'My Integrations'and click on **'REMOVE'**. - -The Bugzilla integration will be uninstalled. - -> -That's all! In case you have any doubts or questions for us, feel free to drop them at our **24x7 Chat Support** or you could also mail us directly at **[support@testmuai.com](mailto:support@testmuai.com)**. diff --git a/static/docs/build-inactivity-time.md b/static/docs/build-inactivity-time.md deleted file mode 100644 index 9e9d35f1d..000000000 --- a/static/docs/build-inactivity-time.md +++ /dev/null @@ -1,30 +0,0 @@ -# Setting Up the Build Inactivity Time [Build Splitting] - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Build Splitting (Build Inactivity Time) lets you organize tests more efficiently. Track your progress by identifying tests that run at specific intervals. Set a time interval after which the system waits for the defined time and marks the build as completed. Any new builds you run after this time period are created as a new build. - -## How It Works - -Build Splitting separates tests into distinct builds based on inactivity time. - -Previously, when you ran a build on TestMu AI, all tests merged into the same build, making it difficult to identify tests that ran at specific intervals. Build Splitting logic helps you organize your tests better. - -Build Splitting requires you to set Build Inactivity Time. For instance, if the Build Inactivity Time is set to 5 minutes, the system waits 5 minutes after the last test completes before marking the build as complete. If you run a test before the inactivity time, it runs in the existing build. If you run a test (even with the same name) after the specified inactivity time (i.e., 5 minutes), it appears under a new build. - -## How to Set the Build Inactivity Time - -Follow these steps to configure Build Inactivity Time for your account. - -**Note** -1. By default, the build inactivity time is six hours. -2. Every user within the organization can set their own build inactivity time. - -1. Log in to your TestMu AI account. Don't have an account, sign up for free. - -2. From the top-right corner, click on the Profile avatar, and navigate to **Profile > Product Preferences**. - -3. Under the **Automation Build Preference**, select the **Build Inactivity Time** to choose -your preferences. - -Once you set the build inactivity time, a confirmation notification appears: *Build Time updated successfully.* diff --git a/static/docs/build-ios-app-for-simulator-testing.md b/static/docs/build-ios-app-for-simulator-testing.md deleted file mode 100644 index 2ab83e550..000000000 --- a/static/docs/build-ios-app-for-simulator-testing.md +++ /dev/null @@ -1,164 +0,0 @@ -# Building Your iOS App for Simulator Testing - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -To test your iOS application on TestMu AI iOS Simulators, your app must be built specifically for the **iOS Simulator** platform. A build intended for a physical iPhone will not work on the Simulator, even if the CPU architecture matches. This guide walks you through creating a Simulator-compatible build and verifying it before uploading. - -## Why Simulator Builds Are Different - -iOS apps are compiled with a **Mach-O platform** identifier that tells the system where the binary is meant to run: - -| Platform ID | Platform Name | Target | -|---|---|---| -| **2** | `iphoneos` | Physical iOS devices | -| **7** | `iphonesimulator` | iOS Simulator | - -Even when the CPU architecture (e.g., arm64) matches between device and Simulator builds, the two platforms link against different system libraries. The Simulator will reject a device build, typically resulting in an immediate crash. - -**Common Mistake** -Building your app for a physical device (e.g., selecting **Any iOS Device** in Xcode) and then uploading it to the Simulator will cause an immediate crash with errors like **"App quit unexpectedly."** You must build targeting the **iOS Simulator** SDK. - -## Prerequisites - -- [Xcode](https://apps.apple.com/us/app/xcode/id497799835?mt=12) installed on a Mac (macOS 13 or later recommended) -- Your iOS project source code -- A TestMu AI account with access to iOS Simulator testing - -## Method 1: Building via Xcode (UI) - -This is the most common method. Xcode handles the platform switching automatically based on the **Run Destination** you select. - -**Step 1:** Open your project (`.xcodeproj` or `.xcworkspace`) in Xcode. - -**Step 2:** Click the **scheme selector** in the top toolbar. In the destination dropdown, choose a specific **Simulator** (e.g., *iPhone 15 Pro*) from the **iOS Simulators** section. - -**Important** -Do **not** select **Any iOS Device** or a physical device plugged into your Mac. You must select a Simulator destination. - -**Step 3:** Press **Cmd + B** or go to **Product > Build** to build the application. - -**Step 4:** Locate the generated `.app` file: -- In the **Project Navigator**, expand the **Products** folder and right-click the `.app` file. -- Select **Show in Finder** to open its location. -- Alternatively, go to **File > Project Settings > Derived Data** to find the build output. - -**Step 5:** Compress the `.app` file into a `.zip` archive: -- Right-click the `.app` file in Finder and select **Compress**. - -The resulting `.zip` file is ready to upload to TestMu AI for Simulator testing. - -## Method 2: Building via Command Line (xcodebuild) - -If you use a CI/CD pipeline or prefer the terminal, use `xcodebuild` with the `-sdk iphonesimulator` flag. - -**Step 1:** Open Terminal and navigate to your project directory. - -**Step 2:** Run the following build command: - -```bash -xcodebuild -project YourProject.xcodeproj \ --scheme YourScheme \ --configuration Debug \ --sdk iphonesimulator \ --arch arm64 \ -build -``` - -If your project uses a **workspace** (e.g., with CocoaPods), replace `-project YourProject.xcodeproj` with `-workspace YourProject.xcworkspace`. - -**Key flags explained:** - -| Flag | Description | -|---|---| -| `-sdk iphonesimulator` | Targets the iOS Simulator platform (Platform 7) instead of physical devices | -| `-arch arm64` | Builds for the arm64 architecture (required for modern Mac hardware) | -| `-configuration Debug` | Uses the Debug build configuration (recommended for testing) | - -**Step 3:** Locate the `.app` output from the **Derived Data** directory. You can find the path in the build output, or use: - -```bash -xcodebuild -project YourProject.xcodeproj \ --scheme YourScheme \ --configuration Debug \ --sdk iphonesimulator \ --arch arm64 \ --showBuildSettings | grep "BUILT_PRODUCTS_DIR" -``` - -**Step 4:** Compress the `.app` into a `.zip` file: - -```bash -cd /path/to/build/output/ -zip -r YourApp.zip YourApp.app -``` - -The `.zip` file is now ready for upload. - -## Verifying Your Build - -Before uploading, verify that your `.app` is targeting the correct platform using the `vtool` command. - -**Step 1:** Run the following command: - -```bash -vtool -show-build YourApp.app/YourApp -``` - -Replace the second `YourApp` with the actual binary name inside the `.app` bundle. - -**Step 2:** Check the output: - -- **Correct (Simulator build):** -``` -platform: IOSSIMULATOR -``` - or `platform 7` - -- **Incorrect (Device build):** -``` -platform: IOS -``` - or `platform 2` - -If the platform shows `IOS` or `platform 2`, the build is for physical devices and will not work on the Simulator. Rebuild using one of the methods above. - -**Alternative Verification** -You can also use `otool` to check the architecture: -```bash -lipo -info YourApp.app/YourApp -``` -This displays the architectures included in the binary (e.g., `arm64`). - -## Uploading to TestMu AI - -Once you have a verified Simulator build (`.zip` containing the `.app`): - -1. Log in to your TestMu AI account. -2. Navigate to **App Testing** > **iOS Simulators**. -3. Upload the `.zip` file containing your `.app` build. -4. Select the desired Simulator device and iOS version. -5. Start your testing session. - -## Key Xcode Build Settings - -If your build still targets the wrong platform, check these settings in Xcode under **Build Settings**: - -| Setting | Value for Simulator | -|---|---| -| **Architectures** | `$(ARCHS_STANDARD)` | -| **Base SDK** | iOS | -| **Supported Platforms** | iOS Simulator | -| **Validate Built Product** | No (for Debug/Simulator builds) | - -## Troubleshooting - -### App crashes immediately on the Simulator -Your `.app` is likely built for physical devices (Platform 2). Verify the platform using `vtool` as described above, and rebuild targeting the Simulator SDK. - -### "No .app found inside zip" error -Ensure you are compressing the `.app` bundle directly into a `.zip` file. The `.app` should be at the root level of the archive, not nested inside additional folders. - -### Build fails with architecture errors -Ensure the `-arch arm64` flag is set. Your Simulator build must include the arm64 architecture. - -That's all! In case you have any questions or need any additional information, you could reach out at our **24X7 Chat Support** or mail us directly at support@testmuai.com. diff --git a/static/docs/buildkite-integration-with-testmu.md b/static/docs/buildkite-integration-with-testmu.md deleted file mode 100644 index f4927a380..000000000 --- a/static/docs/buildkite-integration-with-testmu.md +++ /dev/null @@ -1,103 +0,0 @@ -# Triggering tests from Buildkite CI/CD to TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Buildkite is a continuous integration (CI) and continuous deployment (CD) platform that integrates with version control systems to automate the testing and release of software. Unlike traditional CI/CD solutions, Buildkite allows developers to run builds on their own infrastructure, offering a hybrid model that combines the flexibility of self-hosted runners with the convenience of a cloud-based UI. This ensures faster build times, high levels of security, and scalability to meet the demands of projects of any size. - -TestMu AI now integrates with Buildkite to boost your go-to-market delivery. Perform automated cross browser testing with TestMu AI to ensure your development code renders seamlessly through an online [Selenium grid](https://www.testmuai.com/blog/why-selenium-grid-is-ideal-for-automated-browser-testing/) providing 3000+ real browsers running through machines on the cloud. Perform automation testing in parallel with TestMu AI’s Selenium grid to trim down your test cycles drastically. - -## Prerequisites - -1. Buildkite account -2. A GitHub repository. Here is our sample GitHub repository for [Buildkite](https://github.com/LambdaTest/buildkite-selenium-sample). - -## Integrating Buildkite with TestMu AI via UI - -**Step 1:** Login to Buildkite and click the + icon to create a new pipeline - -**Step 2:** Connect your GitHub repository. Fill the name and description fields. - -You can select the GitHub account where your repository is and add the repository to setup the pipeline for it. Provide an appropriate name and description for this pipeline. - -**Step 3:** Add TestMu AI Credentials to Environment Variables - -You need to set your TestMu AI username and accesskey as your environment variables. You can retrieve them from your TestMu AI automation dashboard by clicking on the key icon near the help button or the key icon in the side navbar. The format should be as following: - -``` -LT_USERNAME=YOUR_LAMBDATEST_USERNAME -LT_ACCESS_KEY=YOUR_LAMBDATEST_ACCESS_KEY -``` - -**Step 4:** Add steps to add lambdatest-buildkite plugin - -To add a plugin to a `command` step, use the `plugins` attribute. The `plugins` attribute accepts an array, so you can add multiple plugins to the same step. - -``` -steps: -- command: 'yarn && yarn LT test' -plugins: -- lambdatest/lambdatest~ -``` - -**Step 5:** Add steps for execution - -The commands to run for test execution are to be mentioned as steps here. For example, the steps can be prerequisite package installs, executing the testing code etc. - -In the example repository, there is only a single step - executing the `lambdatest.py` file. That can be done with the following code: - -``` -python3 lambdatest.py -``` - -You can add additional steps by clicking on the `Add` button. Once all the steps are added, you can click `Create Pipeline`. - -**Step 6:** Adding an agent - -The next step is adding an agent. In this example, a local MacOS agent is used. If you have a custom agent already setup, this step is skipped. - -**Step 7:** Create and run a build - -After the pipeline is created, we can create a build. To create a build, click the `New Build` button. In the popup, add the name of the build and the branch to build. Click `Create Build.` - -Now the build should be created and run. After the build runs successfully, you should be able to see your builds on TestMu AI Automation Dashboard. - -That is it! You can also add a webhook to your repository to trigger the build whenever a change is done in the repository. - -Monitor and analyze your test result on the **[TestMu AI automation dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/).** - -## Integrating Buildkite with TestMu AI via YAML - -**Step 1:** Login to Buildkite. - -**Step 2:** Select **Settings** > **YAML Migration** to open the YAML migration settings. Select **Use YAML Steps for New Pipelines**, then confirm the action in the modal. - -**Step 3:** Go to your desired GitHub repository for which you want to build the pipeline. - -**Step 4:** In the root of your repository, create a file named `pipeline.yml` in a `.buildkite` directory. In `pipeline.yml`, define your pipeline steps. - -```bash -steps: -- label: ":docs-testing:" -commands: -- python3 lambdatest.py -``` - -**Step 5:** Now go to your **Buildkite dashboard** and click on the **Pipelines** > select **New pipeline**. - -**Step 6:** Select the appropriate repository from the list of existing ones in your account. - -**Step 7:** Enter your pipeline's details in the respective **Name** and **Description** fields. You can always change these details later from your pipeline's settings. - -**Step 8:** In the Steps editor, ensure there's a step to upload the definition from your repository. - -**Step 9:** Click on the **Create Pipeline** button. - -**Step 10:** On the next page showing your pipeline name, click New Build. In the modal that opens, create a build using the pre-filled details. Select **Create Build**. - -- Run the pipeline whenever you make changes you want to verify. If you want to add more functionality, go back to editing your steps and repeat. - -- If you've configured webhooks, your pipeline will trigger when you push updates to the repository. Otherwise, select New Build in the Buildkite dashboard to trigger the pipeline. - -> Monitor and analyze your test result on the **[TestMu AI automation dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/).** - -Reliably deploy your code at scale using Buildkite integration with TestMu AI, and ensure it looks robust across every browser to provide a seamless user experience to all your visitors. Happy Testing! diff --git a/static/docs/bulk-update-fields.md b/static/docs/bulk-update-fields.md deleted file mode 100644 index ff85b62dd..000000000 --- a/static/docs/bulk-update-fields.md +++ /dev/null @@ -1,17 +0,0 @@ -# Bulk Update System & Custom Fields - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Utilize bulk action feature that streamlines the process of updating system & Custom fields across multiple test cases, enhancing efficiency and organization of Test Cases. With this feature, users can seamlessly modify key system fields such as Priority, Status, Automation Status, Type and all available custom fields for numerous test cases simultaneously, eliminating the need for time consuming individual updates. - -## Steps to Bulk update Fields: - -1. Bulk Select Test cases to update their Fields. - -2. Click on the `Update Fields` option. - -3. Select the fields you want to update the values for. - -4. Update the values of the fields & Click on `Update Fields` to apply the updates. - -A notification will appear at the top right confirming the fields have been Updated. diff --git a/static/docs/button-text-click.md b/static/docs/button-text-click.md deleted file mode 100644 index e0eeb42a9..000000000 --- a/static/docs/button-text-click.md +++ /dev/null @@ -1,74 +0,0 @@ -# Button Click By Text (OCR) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Button Click By Text is an OCR-powered feature that allows you to click UI elements using their visible text or icon description when traditional locator strategies are not feasible. - -Because the feature is powered by an LLM, you can also provide a natural language description of an icon (for example, "label": "search icon" or "label": "three-dot menu"), and the system will intelligently identify and click the matching element on the screen. - -This approach is particularly useful when: - -- Accessibility IDs or XPath locators are unavailable -- The UI is canvas-based or dynamically rendered -- Native locators are unstable or frequently changing - -This feature is supported across both **Android** and **iOS** real and virtual devices via the `lambda_executor` command. - -## Executor Contract - -The feature is exposed via `driver.execute_script` using the `lambda_executor` interface: - -```python -driver.execute_script('lambda_executor: { -"action": "buttonTextClick", -"arguments": { -"label": "Continue" -} -}') -``` - -## Arguments Specification - -### Required Arguments - -| Field | Type | Description | -|-------|------|-------------| -| `label` | String | Full visible text of the button to click, or a description of an icon (e.g., `"search icon"`, `"three-dot menu"`). It is recommended to use the entire string to avoid OCR misses. | - -### Optional Arguments - -| Field | Type | Default | Description | -|-------|------|---------|-------------| -| `timeout` | Number (seconds) | `0` | Max time to wait for text to appear (up to 30 seconds). | -| `operation` | String | `single` | Type of click operation. Supported values: `single`, `double`, `long`. | -| `matchMode` | Enum | `contain` | Text match strategy. Supported values: `exact`, `contain`. | -| `ignoreCase` | Boolean | `true` | Enable case-insensitive matching. | -| `ignoreSpaces` | Boolean | `true` | Ignore extra spaces in the text. | - -### Advanced Arguments - -| Field | Type | Description | -|-------|------|-------------| -| `index` | Number | Click the nth matched button (0-based index). Useful when multiple buttons share the same text. | - -## Full Executor Example - -```python -driver.execute_script('lambda_executor: { -"action": "buttonTextClick", -"arguments": { -"label": "Pay Now", -"timeout": 0, -"matchMode": "contain", -"ignoreCase": true, -"ignoreSpaces": true, -"index": 0 -} -}') -``` - -## Failure States & Errors - -| Scenario | Error Code | Message | -|----------|------------|---------| -| Text not found | `ERROR_TEXT_NOT_FOUND` | Button with text not found | diff --git a/static/docs/bypass-proxy-domains.md b/static/docs/bypass-proxy-domains.md deleted file mode 100644 index 63f147cc0..000000000 --- a/static/docs/bypass-proxy-domains.md +++ /dev/null @@ -1,30 +0,0 @@ -# How to Bypass Proxy Domains - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -When running tests with Dedicated Proxy (DP) enabled, all outgoing requests are routed through the proxy. While this provides secure and controlled network access, it can sometimes cause issues when your tests need to access services running locally, such as localhost endpoints or specific internal domains. - -To solve this, you can use the `bypassProxyDomains` capability. This capability lets you define a list of domains that should bypass the dedicated proxy and instead be resolved directly from the test environment. - -With `bypassProxyDomains`, you can configure specific domains (like localhost) to skip the proxy hop and resolve locally. This ensures that local APIs, servers, or background services started during tests remain accessible. - -## Capability Reference - -| Capability | Type | Required | Description | -| -------------------- | --------- | -------- | ------------------------------------------------------------------------------ | -| `dedicatedProxy` | `boolean` | Yes | Must be set to `true` for `bypassProxyDomains` to take effect. | -| `bypassProxyDomains` | `array` | Optional | List of domains (hostnames) to bypass the dedicated proxy and resolve locally. | - -> `dedicatedProxy: true` is mandatory for bypassProxyDomains to take effect. - -## Example Usage - -```java title="test1.java" -DesiredCapabilities capabilities = new DesiredCapabilities(); -String bypassProxyDomains[] = { "127.0.0.1", "localhost", "todomvc" }; - -capabilities.setCapability("dedicatedProxy", true); -capabilities.setCapability("bypassProxyDomains", bypassProxyDomains); -``` - -> **Note :** Cypress currently does not support bypassing the domains `127.0.0.1` and `localhost` when using the `bypassProxyDomains` capability diff --git a/static/docs/c-with-selenium-running-c-automation-scripts-on-testmu-selenium-grid.md b/static/docs/c-with-selenium-running-c-automation-scripts-on-testmu-selenium-grid.md deleted file mode 100644 index 737721262..000000000 --- a/static/docs/c-with-selenium-running-c-automation-scripts-on-testmu-selenium-grid.md +++ /dev/null @@ -1,183 +0,0 @@ -# Selenium With C# - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Run your C# Selenium tests on the TestMu AI cloud grid across 3000+ browser and OS combinations. The setup is the same for every framework: you connect to the grid and pass your capabilities. This guide covers that shared flow once, then gives you a per-framework quickstart in the tabs below. - -## Prerequisites - -1. [Create a TestMu AI account](https://www.testmuai.com/register/) if you don't have one. -2. Get your **Username** and **Access Key** from the [TestMu AI Dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard). -3. Install the [.NET Core SDK](https://dotnet.microsoft.com/download) 2.1 or later (MSTest needs 3.0.0). -4. Install the [Selenium WebDriver C# bindings](https://www.selenium.dev/downloads/). - -## Set your credentials - -Every framework authenticates the same way: your Username and Access Key are read from environment variables. Set them once. Pick your operating system: - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME=${ YOUR_LAMBDATEST_USERNAME()} -set LT_ACCESS_KEY=${ YOUR_LAMBDATEST_ACCESS_KEY()}`} - -## How the sample test works - -Every framework below connects to the grid and passes your browser and OS choices through an `LT:Options` object: - -```csharp -var browserOptions = new ChromeOptions(); -browserOptions.PlatformName = "Windows 10"; -browserOptions.BrowserVersion = "latest"; - -var ltOptions = new Dictionary -{ -{ "username", LT_USERNAME }, -{ "accessKey", LT_ACCESS_KEY }, -{ "project", "Demo LT" }, -{ "w3c", true } -}; -browserOptions.AddAdditionalOption("LT:Options", ltOptions); -``` - -What changes between frameworks is only the test runner and how you launch it. That is what each tab covers. - -Use the [Capabilities Generator](https://www.testmuai.com/capabilities-generator/) to build an `LT:Options` block for any browser, version, and OS combination. - -## Run a test in your framework - -Each tab lists the framework-specific pieces. Clone the matching repo (it contains the full, ready-to-run project), then build and run. - -NUnit runs from the Visual Studio Test Explorer, or from the CLI on Linux/macOS. - -1. Clone the [sample GitHub project](https://github.com/LambdaTest/CSharp-NUnit-Selenium): - -```bash -git clone https://github.com/LambdaTest/CSharp-NUnit-Selenium -cd CSharp-NUnit-Selenium -``` - -2. Set your browser and OS in the `LT:Options` object: - -```csharp -var browserOptions = new ChromeOptions(); -browserOptions.PlatformName = "Windows 10"; -browserOptions.BrowserVersion = "latest"; - -var ltOptions = new Dictionary -{ -{ "build", "NUnit Build" }, -{ "name", "NUnit Test" }, -{ "w3c", true } -}; -browserOptions.AddAdditionalOption("LT:Options", ltOptions); -``` - -3. Build the solution, then run in the Test Explorer, or on Linux/macOS: - -```bash -nmake clean build -``` - -MSTest requires .NET Core SDK 3.0.0 and runs via `dotnet test` or the Test Explorer. - -1. Clone the [sample GitHub project](https://github.com/LambdaTest/MSTest-Selenium-Sample): - -```bash -git clone https://github.com/LambdaTest/MSTest-Selenium-Sample -cd MSTest-Selenium-Sample -``` - -2. Set your browser and OS in the `LT:Options` object: - -```csharp -var browserOptions = new ChromeOptions(); -browserOptions.PlatformName = "Windows 10"; -browserOptions.BrowserVersion = "latest"; - -var ltOptions = new Dictionary -{ -{ "build", "MSTest Build" }, -{ "name", "MSTest Test" }, -{ "w3c", true } -}; -browserOptions.AddAdditionalOption("LT:Options", ltOptions); -``` - -3. Run the test: - -```bash -dotnet test MS-Test-Cross-Browser.csproj -``` - -SpecFlow is BDD for .NET: you write Gherkin feature files, and step definitions drive WebDriver. Requires SpecFlow installed. - -1. Clone the [sample GitHub project](https://github.com/LambdaTest/SpecFlow-Selenium-Sample): - -```bash -git clone https://github.com/LambdaTest/SpecFlow-Selenium-Sample -cd SpecFlow-Selenium-Sample -``` - -2. Set your browser and OS in the `LT:Options` object: - -```csharp -var browserOptions = new ChromeOptions(); -browserOptions.PlatformName = "Windows 10"; -browserOptions.BrowserVersion = "latest"; - -var ltOptions = new Dictionary -{ -{ "build", "SpecFlow Build" }, -{ "name", "SpecFlow Test" }, -{ "w3c", true } -}; -browserOptions.AddAdditionalOption("LT:Options", ltOptions); -``` - -3. Build the solution, then run in the Test Explorer (use **Run All** for parallel tests). - -xUnit reads its capabilities and environments from a `config.json`, filtered by profile. - -1. Clone the [sample GitHub project](https://github.com/LambdaTest/CSharp-xUnit-Selenium): - -```bash -git clone https://github.com/LambdaTest/CSharp-xUnit-Selenium -cd CSharp-xUnit-Selenium -``` - -2. Set your browsers in `config.json`: - -```json title="config.json" -{ -"server": "hub.lambdatest.com", -"user": "LT_USERNAME", -"key": "LT_ACCESS_KEY", -"capabilities": { -"lt:options": { -"buildName": "xunit build", -"sessionName": "lambdatest xunit sample test", -"visual": "true", -"plugin": "xunit:sample" -} -}, -"environments": [ -{ "browserName": "chrome" }, -{ "browserName": "firefox" }, -{ "browserName": "safari" } -], -"TunnelOptions": { "tunnel": false } -} -``` - -3. Run a single test, or the parallel profile: - -```bash -dotnet clean -dotnet test --filter "profile=single" -dotnet test --filter "profile=parallel" -``` - -## View your results - -Your test results, including video, network logs, and command-by-command execution, appear on the [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build). diff --git a/static/docs/camera-image-injection-on-real-devices.md b/static/docs/camera-image-injection-on-real-devices.md deleted file mode 100644 index f3cbe66cf..000000000 --- a/static/docs/camera-image-injection-on-real-devices.md +++ /dev/null @@ -1,74 +0,0 @@ -# Media Injection on Real Devices - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI's Media Injection empowers you to test various camera-related functionalities (Image Injection and Video Injection) of your app across a vast array of real mobile devices. This innovative feature supports image capture testing, QR code scanning verification and Barcode scanning assessment. - -## Use Cases - -- URL-embedded QR code scans -- Digital check processing -- Real-time video streaming and playback -- Video capture and upload workflows - -## How It Works - -The Image and Video Injection processes leverages **Sensor Instrumentation** technology. When activated for a test session, it injects TestMu AI's proprietary camera module into your application and emulates or supersedes the native Android/iOS camera SDK - -### Key Features - -- Extensive device compatibility (10000+ real devices) -- Comprehensive support for image and video based functionalities -- Effortless integration with existing test environments - -**note** -- For iOS apps signed with Enterprise Certificates, app resigning (a prerequisite for using the Image Injection tool) is not available. - -## Supported Camera APIs - -Below given is the list of Camera APIs which are supported by TestMu AI Image Injection tool. Please ensure that your app uses these APIs only to enable Video Injection. - - | Supported APIs | - |----------------| - | `didFinishPickingMediaWithInfo` API of `UIImagePickerController` iOS SDK class for capturing an image. 📕 For details, please check [Apple documentation](https://developer.apple.com/documentation/uikit/uiimagepickercontrollerdelegate/1619126-imagepickercontroller)| - | `AVCapturePhoto` iOS SDK class for receiving captured photos from `AVCapturePhotoOutput` API. 📕 For details, please check [Apple Documentation](https://developer.apple.com/documentation/avfoundation/avcapturephoto). | - - | Supported APIs | - |----------------| - | `CameraX` API is used. 📕 For details, please check [CameraX documentation](https://developer.android.com/training/camerax) | - | `Camera` API is used. 📕 For details, please check [Camera documentation] (https://developer.android.com/training/camera-deprecated)| - | `Camera2` API is used. 📕 For details, please check [Camera2 documentation](https://developer.android.com/training/camera2)| - | `ACTION_IMAGE_CAPTURE` is standard Intent action that can be sent to have the camera application capture an image and return it. 📕 For details, please check [MediaStore documentation](https://developer.android.com/reference/android/provider/MediaStore)| - -## Supported Camera Video APIs - -Below given is the list of Camera APIs which are supported by TestMu AI Video Injection tool. Please ensure that your app uses these APIs only to enable Image Injection. - - | Supported APIs | - |----------------| - | `captureOutput(_:didOutput:from:)` method of `AVCaptureVideoDataOutputSampleBufferDelegate` protocol. 📕 For details, please check [Apple Documentation](https://developer.apple.com/documentation/avfoundation/avcapturevideodataoutputsamplebufferdelegate) | - - | Supported APIs | - |----------------| - | `ImageReader` class for accessing video frames from the camera output. 📕 For details, please check [ImageReader documentation](https://developer.android.com/reference/android/media/ImageReader) | - -## Image and Video Injection in Real Device App Testing - -This feature helps you inject any video or image which you want your app to click or scan through the device camera. You just need to follow the steps given below: - -**Step 1:** Click on the **Real Devices** > **App Testing** - -**Step 2:** Select your required device and upload your application which supports the camera APIs listed above. - -**Step 3:** Now, hover over your app and click on the **Settings** icon. Toggle the **Image Injection** and **Video Injection** option and click on **Save Changes** button. Click on **Start** to run your app on the selected device. - -**Step 4:** You can see the video and image preview in the Media Injection section. You can -**delete**, **reupload** and **inject** the image or video from here. - -**Step 5:** After the above config, click on the **Start** button to start the session. Now, go to **Device Controls** in the toolbar as shown below. Select Media Injection and click **Inject** to upload the desired video or image from your computer to the app. - -**Step 6:** Now go to camera feature in your app, the camera pops up. When you click the image from your app, you will see a black image. Please proceed with the same by clicking on **Use Photo**. - -**Step 7:** The black image is now replaced with an injected image or video. - -The clicked image is black because the **Cameras** of TestMu AI Real Devices are taped for security reasons. Thus we inject image through **Sensor Instrumentation**. diff --git a/static/docs/camera-image-injection.md b/static/docs/camera-image-injection.md deleted file mode 100644 index e40a707ef..000000000 --- a/static/docs/camera-image-injection.md +++ /dev/null @@ -1,91 +0,0 @@ -# Camera Image Injection - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Camera Image Injection feature allows you to test **image capturing, QR code scanning**, and **barcode scanning** functionalities in your app across **10000+ real devices** on the TestMu AI Real Device Cloud platform. - -This tool is ideal for testing features such as: - -- Scanning QR codes linked to embedded URLs -- Check scanning -- Capturing profile photos -- Uploading images to your app - -## How It Works -TestMu AI utilizes **Sensor Instrumentation** to integrate with various mobile sensors, including the camera. When you enable Image Injection for a session, TestMu AI injects its camera code module into your app through Sensor Instrumentation. This module then mocks or overrides the Android or iOS SDK used in your app. - -### Key Features - -- Compatible with 3000+ real mobile devices -- Supports image capturing, QR code scanning, and barcode scanning -- Seamless integration with your existing app testing workflow - -**note** -- For iOS apps signed with Enterprise Certificates, app resigning (a prerequisite for using the Image Injection tool) is not available. -- For virtual devices this feature is available on Android App Automation Testing. - -## Supported Camera APIs - -Below given is the list of Camera APIs which are supported by TestMu AI Image Injection tool. Please ensure that your app uses these APIs only to enable Image Injection. - - | Supported APIs | - |----------------| - | `didFinishPickingMediaWithInfo` API of `UIImagePickerController` iOS SDK class for capturing an image. 📕 For details, please check [Apple documentation](https://developer.apple.com/documentation/uikit/uiimagepickercontrollerdelegate/1619126-imagepickercontroller)| - | `AVCapturePhoto` iOS SDK class for receiving captured photos from `AVCapturePhotoOutput` API. 📕 For details, please check [Apple Documentation](https://developer.apple.com/documentation/avfoundation/avcapturephoto).| - | `AVMetadataMachineReadableCodeObject` is a subclass of `AVMetadataObject` iOS SDK class for scanning QR/Barcodes. 📕 For more details, please check [AVMetadataMachineReadableCodeObject Documentation](https://developer.apple.com/documentation/avfoundation/avmetadatamachinereadablecodeobject) | - - | Supported APIs | - |----------------| - | `CameraX` API is used. 📕 For details, please check [CameraX documentation](https://developer.android.com/training/camerax) | - | `Camera` API is used. 📕 For details, please check [Camera documentation](https://developer.android.com/training/camera-deprecated)| - | `Camera2` API is used. 📕 For details, please check [Camera2 documentation](https://developer.android.com/training/camera2)| - | `ACTION_IMAGE_CAPTURE` is standard Intent action that can be sent to have the camera application capture an image and return it. 📕 For details, please check [MediaStore documentation](https://developer.android.com/reference/android/provider/MediaStore)| - -## Image Injection in App Automation - -This section explains how to use Image Injection with App automation. This section will also teach you how to use image injection for all the possible test cases for your app along with other useful information. - -### Step 1: Upload image to TestMu AI - -You can use the following curl command to upload any image of your choice to the TestMu AI cloud. Please note that we support only **JPG, JPEG, and PNG formats which can be 10 MB or less**. - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://mobile-mgm.lambdatest.com/mfs/v1.0/media/upload" -F "media_file=@"/Users/macuser/Downloads/image.jpeg"" -F "type="image"" -F "custom_id="SampleImage""`} - -Response of above cURL will be a JSON object containing the `media_url` of the format - lt://MEDIA123456789123456789 as shown below. - -```js -{ -"media_url":"lt://MEDIA123456789123456789", -"name":"IMG_123.jpg", -"status":"success", -"custom_id":"SampleImage" -} -``` - -### Step 2: Enable Image injection in the Appium Session - -You can use the appium capability to turn the image injection on in your application. Just add `"enableImageInjection": True` in the desired capabilities while writing the automation script. By adding this capability the app undergoes Sensor Instrumentation, which was described earlier in the documentation. Also adding media url `"media": "lt://MEDIA123456789123456789"` in the capabilities to inject the image is now optional and can be passed in LambdaHook as mentioned in step 3. - - {`desired_capabilities = { - "enableImageInjection": True, - "media": "lt://MEDIAf446d4170cd946aa9ec307d10cb679b9", #Add media_url here (optional) - }`} - - {`DesiredCapabilities desiredCapabilities = new DesiredCapabilities(); -desiredCapabilities.setCapability("enableImageInjection", "true");`} - -### Step 3: Inject the image - -Refer to the code snippets given below to upload the image with the action `ImageInjection`. Please make sure that the image needs to be injected before the camera is triggered in your application. This is due to the fact that the camera only captures the image injected at the last. - - {`driver.execute_script("lambda-image-injection=media_url") #Add media_url here`} - - {`driver.execute_script("lambda-image-injection=media_url") //Add media_url here`} - -**Use Case**: This feature can be used when you want to test injecting more than one images in single test at different instances. The image injected through above process will replace the image which you put in previous steps. - -### Step 4: Capture the desired image through camera - -You need to first add a logic to inject the uploaded image in the script. Thereafter add the logic to click the camera button in the mobile app. This will show you the last injected image on your mobile app screen. - -> 📕 Check the [documentation for Camera Image Injection](/support/docs/camera-image-injection-on-real-devices/) feature in Manual App Testing on Real devices diff --git a/static/docs/capabilities-for-playwright-android.md b/static/docs/capabilities-for-playwright-android.md deleted file mode 100644 index c09f78a50..000000000 --- a/static/docs/capabilities-for-playwright-android.md +++ /dev/null @@ -1,36 +0,0 @@ -# Capabilities for Playwright Android Testing - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The TestMu AI Capability Generator allows you to automatically create the capabilities class needed to run your -Playwright automation scripts on TestMu AI. Here are a few capabilities that you can configure to run your Playwright -Android tests. - -You can use the [Playwright Android Capability Generator](https://www.lambdatest.com/playwright-android-capability-generator/) to generate capabilities for your tests. - -| Key | Expected Values | Description | Capability | -|-----------------|--------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------| -| platform | android | Specify the platform name | `const capability = { "LT:Options": {"platform": "android",}}` | -| platformVersion | 12 | Specify the platform version | `const capability = { "LT:Options": {"platformVersion": "12",}}` | -| deviceName | Pixel 5 | Specify the device name. | `const capability = { "LT:Options": {"deviceName": "Pixel 5",}}` | -| build | Playwright Android Build | Represent the build number for your test | ```const capability = {"LT:Options": {"build": "",}}``` | -| name | Playwright Android Test | Represents the name of a test | `const capability = {"LT:Options": {"name": "",}}` | -| projectName | Playwright Sample Project | Represents the name of a project | `const capability = {"LT:Options": {"projectName": "",}}` | -| tags | ["tag1", "tag2", "tag3"] | Group your Playwright tests | ``const capability = {"LT:Options": { "tags": ["tag1", "tag2", "tag3"], }}`` | -| buildTags | ["build1", "build2", "build3"] | Group your Playwright builds | `const capability = {"LT:Options": { "buildTags": ["build1", "build2", "build3"] }}` | -| projectName | My Test | Represent the name of your project | `const capability = { "LT:Options": {"projectName": "",}}` | -| network | true/false | Enable network logs | `const capability = { "LT:Options": {"network": true,}}` | -| console | true/false | Enable browser console logs | `const capabilities = { "LT:Options": {"console": true,}}` | -| video | true/false | Enable video recording of the entire screen | `const capability = { "LT:Options": {"video": true,}}` | -| tunnel | true/false | Enable tunnel for local testing | `const capability = { "LT:Options": {"tunnel": true,}}` | -| tunnelName | true/false | Specify tunnel name | `const capability = { "LT:Options": {"tunnelName": "",}}` | -| geoLocation | AR (Argentina) | Specify country code | `const capability = { "LT:Options": {"geoLocation": "AR",}}` | - -## Using REGEX for device name -Regular Expression or REGEX is widely used to make searching/find characters in a string. - -When you run a test on TestMu AI using a specific device, there may be scenarios, in which the particular device that you selected isn’t available. In these scenarios, REGEX will help you widen the search request for devices to run the test on. - -To quote an example, consider you want to run the test on an Pixel. It doesn’t matter which one, but it’s got to be an Pixel. In these cases, we can use REGEX to specify exactly that. You’ll now be allotted any Pixel device that’s available which widens the search of devices considerably. - -Refer [REGEX for App/Browser Automation](/support/docs/regular-expression-appium/) to read more on this. diff --git a/static/docs/capabilities-for-playwright.md b/static/docs/capabilities-for-playwright.md deleted file mode 100644 index 2e5801109..000000000 --- a/static/docs/capabilities-for-playwright.md +++ /dev/null @@ -1,31 +0,0 @@ -# Capabilities for Playwright Testing - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The TestMu AI Capability Generator allows you to automatically create the capabilities class needed to run your -Playwright automation scripts on TestMu AI. Here are a few capabilities that you can configure to run your Playwright -tests. - -| Key | Expected Values | Description |Capability| -|----|----------|----------|------------| -| browserName| Chrome, Microsoft Edge **Playwright Bundled Browsers**: pw-chromium, pw-webkit, and pw-firefox| Specify the browser to test on | `const capability = {"browserName": "pw-webkit"}` | -| browserVersion | Chrome 83 & above, Edge 83 & above | Specify the browser version to test on | `const capability = {"browserVersion": "latest"}` | -| platform | **Windows**: 11, 10, 8, 8.1, 7 **macOS**: Monterey, Big Sur, Catiline, Mojave| Specify the platform name | `const capability = { "LT:Options": {"platform": "Windows 10",}}`| -| build | Playwright Sample Build | Represent the build number for your test | ```const capability = {"LT:Options": {"build": "",}}``` | -| name| Playwright Sample Test| Represents the name of a test| `const capability = {"LT:Options": {"name": "",}}`| -| projectName | Playwright Sample Project| Represents the name of a project | `const capability = {"LT:Options": {"projectName": "",}}`| -| tags| ["tag1", "tag2", "tag3"]| Group your Playwright tests | ``const capability = {"LT:Options": { "tags": ["tag1", "tag2", "tag3"], }}``| -| buildTags| ["build1", "build2","build3"]| Group your Playwright builds| `const capability = {"LT:Options": { "buildTags": ["build1", "build2", "build3"] }}` | -| resolution| String, **Default value**: 1920x1080 **Windows 11 & 10**: `1024x768, 1280x800, 1280x1024, 1366x768, 1440x900, 1680x1050, 1600x1200, 1920x1200, 1920x1080 and 2048x1536` **macOS**: `1024x768, 1280x960, 1280x1024, 1600x1200 and 1920x1080` | Specifying your desktop resolution before initiating the test | `const capabilities = { "LT:Options": {"resolution": "2048x1536",}}`| -| projectName| My Test| Represent the name of your project| `const capability = { "LT:Options": {"projectName": "",}}`| -| network| true/false| Enable network logs| `const capability = { "LT:Options": {"network": true,}}`| -| console| true/false| Enable browser console logs| `const capabilities = { "LT:Options": {"console": true,}}`| -| video| true/false| Enable video recording of the entire screen| `const capability = { "LT:Options": {"video": true,}}`| -| visual| true/false| Captures screenshot for every command| `const capability = { "LT:Options": {"visual": true,}}`| -| tunnel| true/false| Enable tunnel for local testing| `const capability = { "LT:Options": {"tunnel": true,}}`| -| tunnelName | true /false | Specify tunnel name | `const capability = { "LT:Options": {"tunnelName": "",}}`| -| geoLocation | AR (Argentina) | Specify country code | `const capability = { "LT:Options": {"geoLocation": "AR",}}`| -| idleTimeout | number| Specifies the timeout of the commands in seconds. Default value: 300 Max value: 1800 If a value greater than 1800 is added, idleTimeout will be set to 1800.| `const capability = { "LT:Options": {"idleTimeout": "",}}`| -| lambdaMaskCommands | array | `sendType` or `sendFill` Masks the type method of the ElementHandle and Locator class. `sendPress` Masks the press method of the ElementHandle and Locator class. `setHTTPCredentials` Masks the HTTP credentials parameter in the newContext and newPage method of the browser class. `setStorageState` Masks the storage state parameter in the newContext and newPage method of the browser class `setGeolocation` Masks the Geolocation parameter in the newContext and newPage method of the browser class.|`const capability = { "LT:Options": {'lambdaMaskCommands': ["sendType", "sendFill", "sendPress", "setHTTPCredentials", "setStorageState", "setGeolocation"]}}` | -| useSpecificBundleVersion | Boolean | If set **`true`:** TestMu AI will select the playwright server version as per the browser version you have passed in the capabilities. Check the [supported browser version list](/support/docs/playwright-test-execution-setup/#playwright-supported-browsers). If the value set in the `browser version` capability is supported by multiple Playwright versions, TestMu AI checks your Playwright client version and sets the Playwright version accordingly. If set **`false` :** TestMu AI will run your playwright tests with the same version as setup in your local system and the browser version will also be used the compatible one as per the [supported browser version list](/support/docs/playwright-test-execution-setup/#playwright-supported-browsers) and not your defined one. | `const capability = { "LT:Options": {"useSpecificBundleVersion": true,}}` | -| lambdaSetBrowserPosition | | The `lambdaSetBrowserPosition` function is designed to arrange two browser windows on the screen, ensuring each occupies exactly half of the available screen space. This layout facilitates side-by-side browser comparisons or multitasking workflows. OS Supported - Windows Browsers supported : Chrome, Microsoft Edge , pw-chroium, pw-webkit & pw-firefox | `const capability = {action: 'lambdaSetBrowserPosition',}` | diff --git a/static/docs/capabilities-for-puppeteer.md b/static/docs/capabilities-for-puppeteer.md deleted file mode 100644 index 89e6785ef..000000000 --- a/static/docs/capabilities-for-puppeteer.md +++ /dev/null @@ -1,24 +0,0 @@ -# Capabilities for Puppeteer Testing - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The TestMu AI Capability Generator allows you to automatically create the capabilities class needed to run your Puppeteer automation scripts on TestMu AI. Here are a few capabilities that you can configure to run your Puppeteer tests. - -| Key | Expected Values | Description | Capability | -| -------- | -----| ------- | ----------------- | -| browserName | Chrome, Edge | Specify the browser to test on | `const capability = {"browserName": "Chrome"}` -| browserVersion | Chrome 83 & above, Edge 83 & above | Specify the browser version to test on | `const capability = {"browserVersion": "113.0"}` -| platform | **Windows**: 11, 10, 8, 8.1, 7 **macOS**: Monterey, Big Sur, Catiline, Mojave | Specify the platform name | `const capability = { "LT:Options": {"platform": "Windows 10",}}` -| build | Puppeteer Sample Build | Represent the build number for your test | `const capability = { "LT:Options": {"build": "",}}` -| name | Puppeteer Sample Test | Represents the name of a test | `const capability = { "LT:Options": {"name": "",}}` -| resolution | Specifying your desktop resolution before initiating the test | String, **Default value**: 1920x1080 **Windows 11 & 10**: `1024x768, 1280x800, 1280x1024, 1366x768, 1440x900, 1680x1050, 1600x1200, 1920x1200, 1920x1080 and 2048x1536` **macOS**: `1024x768, 1280x960, 1280x1024, 1600x1200 and 1920x1080` | `const capability = {"LT:Options": {"resolution": '1024x768'}}` | -| projectName | Example: My Test | Represent the name of your project | `const capability = { "LT:Options": {"projectName": "",}}` | -| tags | ["tag1", "tag2", "tag3"] | Group your Puppeteer tests |``const capability = {"LT:Options": { "tags": ["tag1", "tag2", "tag3"], }}`` | -| buildTags | ["build1", "build2", "build3"] | Group your Puppeteer builds |`const capability = {"LT:Options": { "buildTags": ["build1", "build2", "build3"] }}` | -| network | true/false | Enable network logs | `const capability = { "LT:Options": {"network": true,}}` | -| console | true/false | Enable browser console logs | `const capabilities = { "LT:Options": {"console": true,}}` | -| video | true/false | Enable video recording of the entire screen | `const capability = { "LT:Options": {"video": true,}}` | -| tunnel | true/false | Enable tunnel for local testing | `const capability = { "LT:Options": {"tunnel": true,}}` | -| tunnelName | true/false | Specify tunnel name | `const capability = { "LT:Options": {"tunnelName": "",}}` | -| geoLocation | AR (Argentina) | Specify country code | `const capability = { "LT:Options": {"geoLocation": "AR",}}` | -| idleTimeout | number| Specifies the timeout of the commands in seconds. Default value: 300 Max value: 1800 If a value greater than 1800 is added, idleTimeout will be set to 1800.| `const capability = { "LT:Options": {"idleTimeout": "",}}`| diff --git a/static/docs/capybara-on-hyperexecute-grid.md b/static/docs/capybara-on-hyperexecute-grid.md deleted file mode 100644 index 7d79c1b30..000000000 --- a/static/docs/capybara-on-hyperexecute-grid.md +++ /dev/null @@ -1,235 +0,0 @@ -# Running Capybara Framework Tests on HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Capybara is a Ruby-based testing framework commonly used with Selenium for end-to-end web application testing. It provides a clean and expressive DSL (Domain-Specific Language) for interacting with web pages and handling common browser interactions. - -HyperExecute is an AI Native Test Orchestration Cloud Platform that empowers you to run **end-to-end** tests **quickly** and **efficiently**. It provides Just-in-Time (JIT) testing infrastructure with fast execution **speeds**, **smart orchestration**, and **detailed logs**. - -This guide details how to execute your **Capybara** framework tests on **HyperExecute** via two different methods: - -- [**Using Local System**](/support/docs/capybara-on-hyperexecute-grid/#1-testing-using-local-system) - You can use your own local machine to execute tests. -- [**Using Gitpod Platform**](/support/docs/capybara-on-hyperexecute-grid/#2-testing-using-gitpod) - Execute tests using GitPod. (Requires a [Gitpod](https://gitpod.io/login/) account) - -## 1. Testing Using Local System - -Follow the step-by-step guide to execute your test on HyperExecute. - -### Prerequisites - -To run the Tests on HyperExecute from your Local System, you are required: - -- Your TestMu AI [Username and Access key](/support/docs/hyperexecute-how-to-get-my-username-and-access-key/) -- [HyperExecute YAML](/support/docs/hyperexecute-yaml-version0.2/) file which contains all the necessary instructions. -- [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) in order to initiate a test execution Job . -- Setup the [Environmental Variable](/support/docs/hyperexecute-environment-variable-setup/) - -### Step 1: Configure Your Test Suite - -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** - -Download or Clone the code sample for the JUnit from the TestMu AI GitHub repository to run the tests on the HyperExecute. - - View on GitHub - -If you are using your own project, make sure you update the **Hub endpoint** in your tests file. - -By setting up the Hub endpoint, you establish the communication channel between your tests and the browser nodes, enabling effective test distribution and execution. - -Configure the desired capabilities based on your test requirements. For example: - -```bash -@caps = { -"browserName"=>lt_browser, -"version"=>lt_browser_version, -"platform"=>lt_os, -"resolution"=>lt_res, -"build"=>"capybara-lambdatest", -"name"=>"single-Test-Jenkins", -"video"=>true, -"network"=>true, -"console"=>true, -"visual"=>true -} -``` - -> You can also use **dev** and **beta** browser versions. To generate capabilities for your test requirements, you can use our inbuilt 🔗 Capabilities Generator. - -### Step 2: Setup the CLI in your Test Suite - -After cloning / downloading the sample repo, you need to setup the CLI and the environment variables. - -#### Download the HyperExecute CLI - -The CLI is used for triggering the tests on HyperExecute. It is recommend to download the CLI binary on the host system and keep it in the root directory of the suite to perform the tests on HyperExecute. - -You can download the CLI for your desired platform from the below mentioned links: - -| Platform | HyperExecute CLI | -| ---------| ---------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| MacOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -#### Setup Environment Variable - -Now, you need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in the [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). - -Run the below mentioned commands in your terminal to setup the CLI and the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -### Step 3: Configure YAML in your Test Suite - -Configure your YAML file as per your use cases using **key value** pairs. - -In this sample YAML file, we have mentioned: - -- **version** of the YAML file -- **Timeouts** for executing your project -- **Mode of execution** is [Autosplit](/support/docs/hyperexecute-auto-split-strategy/). You can also opt for [Matrix](/support/docs/hyperexecute-matrix-multiplexing-strategy/) or [Hybrid](/support/docs/hyperexecute-hybrid-strategy/) mode. -- **Pre and Post** commands -- **Reports and Artefacts** that will be generated after the completion of tests -- and other necessary YAML Parameters - -```yaml ---- -version: "0.1" - -globalTimeout: 90 -testSuiteTimeout: 90 -testSuiteStep: 90 - -runson: linux - -autosplit: true - -retryOnFailure: true -maxRetries: 5 - -concurrency: 2 - -pre: -- bundle install --deployment - -env: -CONFIG_NAME: "linux" - -cacheKey: '{{ checksum "Gemfile.lock" }}' -cacheDirectories: -- ./vendor/bundle - -uploadArtefacts: -- name: report -path: -- cucumber_results.html - -report: true -partialReports: -location: cucumber_results.html -type: html - -testDiscovery: -type: raw -mode: remote -command: snooper --featureFilePaths=features/ --frameWork=java - -testRunnerCommand: bundle exec cucumber "$test" --format html --out cucumber_results.html - -jobLabel: [selenium-capybara, linux, autosplit] -``` - -### Step 4: Execute your Test Suite - -> **NOTE :** In case of MacOS, if you get a permission denied warning while executing CLI, simply run **`chmod u+x ./hyperexecute`** to allow permission. In case you get a security popup, allow it from your **System Preferences** → **Security & Privacy** → **General tab**. - -Run the below command in your terminal at the root folder of the project: - -```bash -./hyperexecute --config RELATIVE_PATH_OF_YOUR_YAML_FILE -``` - -OR use this command if you have not exported your username and access key in the step 2. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE `} - -### Step 5: Monitor the Test Execution - -Visit the [HyperExecute Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) and check your Job status. - -### Step 6: Download Artifacts and Reports - -HyperExecute also facilitates the provision to download the [Artifacts](/support/docs/hyperexecute-artifacts/) and [Reports](/support/docs/hyperexecute-reports/) on your local machine. Click on the corresponding button to download your generated artifacts and reports. - -## 2. Testing Using Gitpod - -You can also use the Gitpod platform to execute our sample repository. It will fetch all the sample codebases and trigger the CLI to execute the tests. - -Follow the below steps to run Test using Gitpod: - -**Step 1:** Click '**Open in Gitpod**' button. You will be redirected to Login/Signup page. This button is configured to redirect you to the Gitpod platform where you will be able to execute our sample repository. - -**Step 2:** Login with TestMu AI credentials. Once logged in, a pop-up confirmation will appear, asking you to **'Proceed'** to the Gitpod editor in a new tab. The current tab will display the HyperExecute Dashboard. - -**Step 3:** Choose your preferred editor (we recommend VS Code Editor) - -**Step 4:** As you are running a sample project, Fetching of the Test Scripts, [HyperExecute YAML](/support/docs/deep-dive-into-hyperexecute-yaml/), [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) and Triggering your tests using the `Execution Command` will be automated. - -**Step 5:** Once you see the `Job Link` in the logs, you can visit the [HyperExecute dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) to see the tests getting executed. - -You can also implement [Secret Keys](/support/docs/hyperexecute-how-to-save-and-manage-secrets/) in your YAML file. - -## Navigation in Automation Dashboard - -Every test run on the HyperExecute has a unique *jobId* associated with it. Each *jobId* can in turn constitute single (or multiple) *groupId*(s). You can visit [HyperExecute Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) for checking the status of the test execution. - -You can seamlessly navigate between JobId's and taskId's. You need to click on the *testID* to navigate from the HyperExecute logs to the Automation Dashboard. - -The snapshot below shows the videos, logs and other meta data for that specific *test_ID* - -> For any query or doubt, please feel free to contact us via **24×7 chat support** or you can also drop a mail to **support@testmuai.com**. -Happy testing! - -## Using the Capybara Agent Skill with TestMu AI - -The [capybara-skill](https://github.com/LambdaTest/agent-skills/tree/main/capybara-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The capybara-skill package includes: - -``` -capybara-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing Capybara Agent Skill - -Install a Capybara Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/capybara-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/capybara-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only capybara-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). diff --git a/static/docs/cerberus-integration.md b/static/docs/cerberus-integration.md deleted file mode 100644 index 328d96525..000000000 --- a/static/docs/cerberus-integration.md +++ /dev/null @@ -1,43 +0,0 @@ -# Run Cerberus Tests On TestMu AI Selenium Grid - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Cerberus Testing is an open-source tool that helps you cover the complete test cycle from a single place. It can be used for both Automation testing, as well as Manual testing. - -TestMu AI now provides an integration with the Cerberus Testing tool, to help you perform cross-browser testing on over 2000 real desktop browsers, mobile viewports, and operating system combinations. With this document, you will learn how to integrate LamndaTest with the Cerberus Testing tool and get started with your first test. - -## Setting Up TestMu AI As The Executor Robot - -Inside your Cerberus Testing tool, you need to set up TestMu AI as your Test Executor. So follow the below steps to set up TestMu AI as your Executor Robot: - -1. Click on 'Create' a new Robot -2. Enter the Robot name as per your choice, such as "TestMu AIChrome85Win". Also choose the platform, browser, browser version and screen size of your choice. -3. You can also add optional desired capabilities of your choice, in the Capabilities tab. To know more about Desired Capabilities, you can refer to our [Selenium Autiomation Capabilities guide](/docs/selenium-automation-capabilities/) on our [Capabilities Generator](https://www.testmuai.com/capabilities-generator/). -4. Then switch to Executors tab, and click on "Add Executor". -5. In this Add Executor window, you need to pass following values to their corresponding options, as stated below: - -``` -Host: hub.lambdatest.com -Port: 80 -Host User: -Host Password: -``` - - > To get your TestMu AI username and access key, please refer our [Capabilities Generator](https://www.testmuai.com/capabilities-generator/). - -Once you are done, just click on the Create button and the TestMu AI Robot executor will be created successfully, and will look something like this: - -## Executing Your Cerberus Test On TestMu AI - -Since you have the TestMu AI Executor ready, you can now execute your tests from Cerberus directly on the TestMu AI platform. To do so: - -1. Select your test from the "Run Test Case" section of Cerberus tool. For this demo purpose, we have used the existing sample test case available, as shown in the image below -2. Then you need to select the environment, and country of your choice, in which you want to run the test case. -3. Then you need to select the executor for your test case, in the Robot Settings section. Here, you need to select the earlier created TestMu AI Executor, from the list of available Robots. -4. You can also fill in optional environment details, as per your choice. Then click on the **Run TestCase (and See Results)** button to execute your test case. - -Now if you open your [Automation dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/timeline), you can see your Cerberus test case running as per the chosen Robot Executor details. - -Once the test is complete, you can view the result of the test on TestMu AI, with all its details. You can view logs, [mark bugs](/docs/mark-as-bug-in-automation-testing/), and do much more for each executed test from your TestMu AI platform. To explore more about the TestMu AI's Automation Dashboard, refer [inside TestMu AI platform guide](/docs/inside-testmu-platform/). - -> Congrats! You have successfully executed your Cerberus test case on the TestMu AI platform, which provides you 3000+ browser and operating system combinations to perform cross-browser testing. If you have any doubt or face any issues, please feel free to contact us via **24/7 chat support**, or you can also mail us at [support@testmuai.com](mailto:support@testmuai.com). Happy Testing!! 🙂 diff --git a/static/docs/change-browser-window-size.md b/static/docs/change-browser-window-size.md deleted file mode 100644 index 2a8076949..000000000 --- a/static/docs/change-browser-window-size.md +++ /dev/null @@ -1,177 +0,0 @@ -# Change Browser Window Size - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -When running Selenium tests on TestMu AI, you can control the browser window resolution to test responsive layouts, verify UI elements at specific breakpoints, and reproduce resolution-dependent bugs. Set the resolution through the `resolution` capability in `LT:Options`. - -## Set Resolution Using the `resolution` Capability - -Pass a resolution string in `LT:Options` to launch the browser at a specific screen size. - -Add the `resolution` capability to your `LT:Options` configuration. The value is a string in the format `widthxheight`. If you do not specify a resolution, TestMu AI defaults to **1024x768**. - -```java title="ResolutionTest.java" -import org.openqa.selenium.remote.RemoteWebDriver; -import org.openqa.selenium.chrome.ChromeOptions; -import java.net.URL; -import java.util.HashMap; - -public class ResolutionTest { -public static void main(String[] args) throws Exception { -ChromeOptions browserOptions = new ChromeOptions(); -browserOptions.setPlatformName("Windows 11"); -browserOptions.setBrowserVersion("latest"); - -HashMap ltOptions = new HashMap<>(); -ltOptions.put("username", System.getenv("LT_USERNAME")); -ltOptions.put("accessKey", System.getenv("LT_ACCESS_KEY")); -ltOptions.put("resolution", "1920x1080"); -ltOptions.put("project", "Resolution Test"); -browserOptions.setCapability("LT:Options", ltOptions); - -RemoteWebDriver driver = new RemoteWebDriver( -new URL("https://hub.lambdatest.com/wd/hub"), browserOptions); -driver.get("https://example.com"); -driver.quit(); -} -} -``` - -```javascript title="resolution-test.js" -const { Builder } = require("selenium-webdriver"); -const chrome = require("selenium-webdriver/chrome"); - -(async function resolutionTest() { -let options = new chrome.Options(); -options.setPlatform("Windows 11"); -options.setBrowserVersion("latest"); - -let ltOptions = { -username: process.env.LT_USERNAME, -accessKey: process.env.LT_ACCESS_KEY, -resolution: "1920x1080", -project: "Resolution Test", -}; - -const driver = await new Builder() -.usingServer("https://hub.lambdatest.com/wd/hub") -.withCapabilities({ ...options.toJSON(), "LT:Options": ltOptions }) -.build(); - -await driver.get("https://example.com"); -await driver.quit(); -})(); -``` - -```python title="resolution_test.py" -import os -from selenium import webdriver -from selenium.webdriver.chrome.options import Options - -options = Options() -options.platform_name = "Windows 11" -options.browser_version = "latest" - -lt_options = { -"username": os.environ["LT_USERNAME"], -"accessKey": os.environ["LT_ACCESS_KEY"], -"resolution": "1920x1080", -"project": "Resolution Test", -} -options.set_capability("LT:Options", lt_options) - -driver = webdriver.Remote( -command_executor="https://hub.lambdatest.com/wd/hub", -options=options, -) -driver.get("https://example.com") -driver.quit() -``` - -```csharp title="ResolutionTest.cs" -using OpenQA.Selenium; -using OpenQA.Selenium.Chrome; -using OpenQA.Selenium.Remote; - -ChromeOptions browserOptions = new ChromeOptions(); -browserOptions.PlatformName = "Windows 11"; -browserOptions.BrowserVersion = "latest"; - -Dictionary ltOptions = new Dictionary(); -ltOptions.Add("username", Environment.GetEnvironmentVariable("LT_USERNAME")); -ltOptions.Add("accessKey", Environment.GetEnvironmentVariable("LT_ACCESS_KEY")); -ltOptions.Add("resolution", "1920x1080"); -ltOptions.Add("project", "Resolution Test"); -browserOptions.AddAdditionalOption("LT:Options", ltOptions); - -IWebDriver driver = new RemoteWebDriver( -new Uri("https://hub.lambdatest.com/wd/hub"), browserOptions); -driver.Navigate().GoToUrl("https://example.com"); -driver.Quit(); -``` - -```php title="ResolutionTest.php" -setPlatform("Windows 11"); -$capabilities->setCapability("browserVersion", "latest"); - -$ltOptions = [ -"username" => getenv("LT_USERNAME"), -"accessKey" => getenv("LT_ACCESS_KEY"), -"resolution" => "1920x1080", -"project" => "Resolution Test", -]; -$capabilities->setCapability("LT:Options", $ltOptions); - -$driver = RemoteWebDriver::create( -"https://hub.lambdatest.com/wd/hub", $capabilities); -$driver->get("https://example.com"); -$driver->quit(); -``` - -```ruby title="resolution_test.rb" -require "selenium-webdriver" - -options = Selenium::WebDriver::Options.chrome( -platform_name: "Windows 11", -browser_version: "latest", -"LT:Options" => { -username: ENV["LT_USERNAME"], -accessKey: ENV["LT_ACCESS_KEY"], -resolution: "1920x1080", -project: "Resolution Test", -} -) - -driver = Selenium::WebDriver.for( -:remote, -url: "https://hub.lambdatest.com/wd/hub", -capabilities: options, -) -driver.navigate.to("https://example.com") -driver.quit -``` - -## Supported Resolutions - -Choose from these predefined screen resolutions when setting the `resolution` capability. - -| Resolution | Aspect Ratio | Common Use | -|---|---|---| -| **1024x768** (default) | 4:3 | Legacy desktops, older monitors | -| **1280x800** | 16:10 | Older laptops | -| **1280x1024** | 5:4 | Standard desktop monitors | -| **1366x768** | 16:9 | Budget laptops, common breakpoint | -| **1440x900** | 16:10 | MacBook and mid-range laptops | -| **1680x1050** | 16:10 | Large desktop monitors | -| **1600x1200** | 4:3 | Large standard monitors | -| **1920x1080** | 16:9 | Full HD monitors and laptops | -| **1920x1200** | 16:10 | Professional desktop monitors | -| **2048x1536** | 4:3 | High-resolution displays | -| **2560x1440** | 16:9 | QHD monitors | - -Supported resolutions are OS-dependent. Use the [Capabilities Generator](https://www.testmuai.com/capabilities-generator/) to verify which resolutions are available for your target OS and browser. diff --git a/static/docs/change-individual-test-details.md b/static/docs/change-individual-test-details.md deleted file mode 100644 index 73c363d27..000000000 --- a/static/docs/change-individual-test-details.md +++ /dev/null @@ -1,95 +0,0 @@ -# Changing Individual Test Details - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -You can modify individual test details from your automation test build while running Selenium tests on TestMu AI. This guide covers marking test status and renaming tests during execution. - -## Mark Test Status as Pass or Fail - -Use the `lambda-status` hook via JavascriptExecutor to set the final status of a test on the TestMu AI dashboard. - -When you run Selenium tests on the TestMu AI grid, a test that your local assertions marked as failed may show as completed on the dashboard. Use the `lambda-status` hook to explicitly set the correct status. - -You can set these status values: `passed`, `failed`, `skipped`, `ignored`, `unknown`, `error`. - -### Code Samples - -```java -// Mark test as passed -((JavascriptExecutor) driver).executeScript("lambda-status=passed"); - -// Mark test as failed -((JavascriptExecutor) driver).executeScript("lambda-status=failed"); -``` - -```javascript -// Mark test as passed -await driver.executeScript('lambda-status=passed'); - -// Mark test as failed -await driver.executeScript('lambda-status=failed'); -``` - -```python -# Mark test as passed -driver.execute_script("lambda-status=passed") - -# Mark test as failed -driver.execute_script("lambda-status=failed") -``` - -```csharp -// Mark test as passed -((IJavaScriptExecutor)driver).ExecuteScript("lambda-status=passed"); - -// Mark test as failed -((IJavaScriptExecutor)driver).ExecuteScript("lambda-status=failed"); -``` - -```php -// Mark test as passed -$driver->executeScript("lambda-status=passed"); - -// Mark test as failed -$driver->executeScript("lambda-status=failed"); -``` - -```ruby -# Mark test as passed -driver.execute_script("lambda-status=passed") - -# Mark test as failed -driver.execute_script("lambda-status=failed") -``` - -Place the `lambda-status` call inside your test's teardown or `@AfterMethod` block so the status is set before the session ends. For the full list of Lambda Hooks, see [Lambda Hooks](/support/docs/lambda-hooks/). - -## Rename Your Test - -Use the `lambda-name` hook to update the test name after execution has started. - -You can rename a running test to reflect dynamic data such as iteration count or data-driven parameters. Pass the new name through JavascriptExecutor: - -```java -((JavascriptExecutor) driver).executeScript("lambda-name=Your_test_name"); -``` - -```javascript -await driver.executeScript('lambda-name=Your_test_name'); -``` - -```python -driver.execute_script("lambda-name=Your_test_name") -``` - -```csharp -((IJavaScriptExecutor)driver).ExecuteScript("lambda-name=Your_test_name"); -``` - -```php -$driver->executeScript("lambda-name=Your_test_name"); -``` - -```ruby -driver.execute_script("lambda-name=Your_test_name") -``` diff --git a/static/docs/charles-proxy.md b/static/docs/charles-proxy.md deleted file mode 100644 index b45766cf5..000000000 --- a/static/docs/charles-proxy.md +++ /dev/null @@ -1,150 +0,0 @@ -# Setting Up Charles Proxy Through TestMu AI Tunnel - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This article will guide you on how to setup Charles Proxy for the tests run on TestMu AI platform through TestMu AI Tunnel. - -## Objective - -By the end of this article, you will be able to : - -- Configure **TestMu AI Tunnel** and setup **Charles Proxy** for your tests through the created tunnel. -- View Network Logs of the tests run on the **TestMu AI Cloud Platform** on Charles Interface. -- Manipulate the **Network Requests** of the tests through **SSL Proxying** on Charles. - -## What is Charles Proxy? - -**Charles proxy** is an interactive **web debugging tool** which acts as an intermediate between a client (mobile or web app) and the internet. It enables the users in configuring the client environment to pass all the networking requests and responses through Charles proxy so that they can inspect and even change the data midstream to test how a app responds. - -In Web and Internet development we are unable to see what is being sent and received between the client and the server. So, it is difficult to determine exactly where the fault is. By using Charles, users can get the idea of what is happening to quickly diagnose and fix problems. - -## Prerequisites - -Before getting started with the setup, you would have to follow these steps: - -1. Download **Charles Proxy** software from [the official website](https://www.charlesproxy.com/) and install it. You can also download from the links below. - - > - [Windows](https://www.charlesproxy.com/download/#) - > - [MacOS](https://www.charlesproxy.com/download/#) - > - [Linux](https://www.charlesproxy.com/download/#) - -2. Download the TestMu AI Tunnel binary file and extract it to setup the tunnel. - - > - [TestMu AI Tunnel for Windows](https://downloads.lambdatest.com/tunnel/v3/windows/64bit/LT_Windows.zip) - > - [TestMu AI Tunnel for Mac](https://downloads.lambdatest.com/tunnel/v3/mac/64bit/LT_Mac.zip) - > - [TestMu AI Tunnel for Linux](https://downloads.lambdatest.com/tunnel/v3/linux/64bit/LT_Linux.zip) - -3. Or you can download the **GUI** tunnel application **Underpass** to setup the tunnel easily without using terminal. - - > - [Underpass for Windows](https://downloads.lambdatest.com/underpass/master/UnderPass%20Setup.exe) - > - [Underpass for Mac](https://downloads.lambdatest.com/underpass/master/UnderPass.dmg) - > - [Underpass for Linux](https://downloads.lambdatest.com/underpass/master/UnderPass.AppImage) - -## Installing Charles Root Certificate - -- Once you install **Charles**, you would have to navigate to **"Install Charles Root Certificate"** under **Help** section. - -- On clicking, you need to grant the access to install the certificate to your system. Then you will be able to view the certificate in **Mac's Keychain Access** as shown - -- Now you need to open the installed certificate and select **"Always Trust"** option under the **Trust** section. This gives permission to the Charles Proxy to access the network resources from the system. Here you would have to grant the access again to save the modified settings. - -## Getting the Port Number - -Now, navigate to the **Proxy Settings** on Charles to view the **Port Number** used by the Charles Proxy in your system. - -> In this case the Port number which is in use is `8888`. We will be using it to configure the TestMu AI Tunnel. - -## Configuring The TestMu AI Tunnel - -Now you need to setup and configure the TestMu AI Tunnel through which you will be able to route the traffic of your tests run using Charles Proxy. You can configure a TestMu AI Tunnel by using the tunnel **binary file** or by downloading the **GUI** application **Underpass** by TestMu AI. - -### Through The Binary File - -To configure the tunnel, you need to download the **TestMu AI's Tunnel Binary File** following the steps mentioned in the Prerequisites before. You can find the shell command in the [TestMu AI Dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard) to setup a basic tunnel. - -But here we need to add the Charles proxy server through which we want to route the traffic of the tunnel. We can do that by appending the `--proxy-host` (localhost) and `--proxy-port` (`8888` in this case) options to the above command. The format of the command would be: - -```bash -./LT -user {user’s login email} -key {user’s access key} \ ---proxy-host localhost \ ---proxy-port 8888 -``` - -> You can get your TestMu AI Access Key from the [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/). - -You need to run the above command from the terminal in the directory where you have extracted the **LT binary file** as shown. Make sure that **Charles Proxy Session** is up and running before you can start the tunnel. Now once the tunnel is setup, you will be able to see **"You can start testing now"** in the terminal as shown. - -### Through Underpass - -Before you can configure and setup the tunnel, you need to download the Underpass application as mentioned in the Prerequisites before. For more details about Underpass, refer to our guide on [getting started with Underpass application](/support/docs/underpass-tunnel-application/). Once you install Underpass, you need to pass your **TestMu AI authentication credentials** as shown below. - -Now to configure the tunnel with Charles Proxy, we need to add proxy to the tunnel. The **proxy host** : `localhost` and **port** : `8888` can be set using **"Show Advanced Configuration option"** in **Underpass**. You can set more advanced options through this if necessary. - -Make sure to also select **Advanced Configuration** and turn on **MITM** (Man in the Middle) mode so that we can see the decrypted network data and perform changes to the data being exchanged. - -On clicking the **Launch** button, you will be able to start the tunnel with the configured proxy setup. Make sure you are running the Charles Software before launching the tunnel using Charles Proxy Port. You will be able to see the status as **"running"** once the tunnel is launched successfully. - -## Steps to download the SSL certificate in the TestMu AI Private Real Devices: - -> Only for **Private Cloud** Users. - -**Step 1:** Launch the session with the tunnel which we have enabled and visit http://chls.pro/ssl site to download the SSL certificate. - -**Step 2:** Visit the settings of the device and search for "CA certificate" - -**Step 3:** Click on CA certificate - -**Step 4:** Confirm installation by clicking "Install Anyway" when prompted. - -**Step 5:** Select the Charles Proxy SSL proxying certificate to install it. - -## Viewing Network Logs Through Charles Proxy - -Once this is taken care of, the setup is now complete. - -Now there are 2 ways of enabling **SSL proxying** without which we will not be able to view the data being exchanged. - -### Method 1: Adding domains - -- Navigate to **SSL Proxying Settings** under **Proxy** section. - -- Then you need to add the host of the site [amazon.com] under **Include** to enable SSH Proxying. - - > There is no need to mention the port number here as it will be allocated by default on adding. - -> We have added `*amazon.com*` so that we target the domain and all traffic that contains it in it's URL. You can also add `*` (wildcard) and it will intercept end to end flow of data. - -**For Automation** - -We need to ensure that the capabilities `network` and `tunnel` are passed as `true` to successfully get the network logs on Charles. On running the tests, you will be able to see the host in **Structure** panel with the **Notes** - **"SSL Proxying enabled for this host"**. - -We can now see the data flowing when we open [Amazon](https://www.amazon.in/) on a real device. - -### Method 2: Right click & enable SSL proxy - -That's it, now all the Network logs will start to show up and we can interact with Charles Proxy. - -## Mocking data - -Charles Proxy offers the ability to mock the data that is being passed to and for. -Let's look at one such example. - -The clean data looks something like this: - -Let's try to change the `astro` to `lambdatest` - -Click on **Tools** on the Nav bar and select **Rewrite** -Now, check the **Enable Rewrite** option. - -Once done, provide a *name* to the new rule you create. -Then add the URL that you want to change and Charles Proxy will pick up all the data and fill up details. - -Click *add* on the lower table where we'll be defining the data that will auto rewrite when the data is flowing. - -Click *save* and we're good to go! - -We can now re-launch the browser after closing the tab (on the real device) so that we can clear the cache on the device. - -Now when we try opening the URL, we can see the `lambdatest` appear in the place of `astro`. - -> That’s all! In case you have any questions or need any additional information, you could reach out at our **[24X7 Chat Support]** or mail us directly at support@testmuai.com. diff --git a/static/docs/chat-agent-api-integration.md b/static/docs/chat-agent-api-integration.md deleted file mode 100644 index 6f91b7920..000000000 --- a/static/docs/chat-agent-api-integration.md +++ /dev/null @@ -1,175 +0,0 @@ -# How to Integrate a Chat Agent API - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The Agent Testing Platform connects to your chat agent over standard HTTP. It sends multi-turn test conversations to your API endpoint and scores the replies, with no code changes to your agent. - -It works with any chatbot provider or framework. You provide the endpoint URL, the authentication, and the request and response shape, and the platform matches whatever structure your API uses. - -## How the Platform Sends Requests to Your Chat Agent - -The platform sends each message as an HTTP POST request with a JSON body. The example below shows the shape, followed by a breakdown of each component. - -```bash title="Example Request" -curl -X POST https://api.examplechatbot.com/chat \ --H "Authorization: Bearer sk-example-a1b2c3d4e5f6" \ --H "Content-Type: application/json" \ --d '{ -"assistantId": "asst_7xG9kPqR2mN4", -"input": "Hi, I need help with my account" -}' -``` - -The field names above (`assistantId`, `input`) are illustrative. Your chat agent may use different field names (for example `message`, `query`, `botId`, `agentId`). The platform adapts to whatever request structure your API expects. - -### Request Components - -| Component | Value in Example | Description | -|-----------|-----------------|-------------| -| **Method** | `POST` | HTTP method. All chat requests use POST. | -| **Endpoint URL** *(Required)* | `https://api.examplechatbot.com/chat` | Your chat agent's API endpoint, the URL that accepts incoming messages. This varies by provider. | -| **Authorization** *(Required)* | `Bearer sk-example-a1b2c3d4e5f6` | Authentication token. The platform uses the token you provide to authenticate each request. The format may vary (Bearer token, API key header, and similar). | -| **Content-Type** *(Required)* | `application/json` | Indicates a JSON-formatted request body. | -| **Additional Headers** *(Optional)* | Any extra `-H "…"` entries | If your chat agent requires extra headers (session ID, API version, workspace ID, custom tracking headers), they are supported. Include as many as needed. | -| **Request Body** | JSON payload | Contains the fields your chat agent expects, typically a bot or assistant identifier and the user message. Field names vary by provider. | - -**Header Flexibility** -The platform sends the exact URL, token, and headers you configure. If your chat agent requires headers beyond Authorization and Content-Type, provide them, and the platform forwards all configured headers with every request. - -## How to Connect Your Chat Agent - -The platform supports three connection methods, based on where your agent runs and how it is reachable. Choose the option that matches your environment. - -### Option A: Public API (Direct) - -The simplest path. If your chat agent exposes a publicly reachable HTTPS endpoint, the platform connects to it directly using the URL, auth headers, and any additional headers you provide. - -**Flow:** `Testing Platform` → `Internet (HTTPS)` → `Your Chat Agent API` - -| You Provide | Details | -|-------------|---------| -| **Endpoint URL** *(Required)* | The public HTTPS URL of your chat agent (for example `https://api.yourcompany.com/chat`). | -| **Auth Headers** *(Required)* | Authorization token, API key, or any authentication headers your API requires. | -| **Additional Headers** *(Optional)* | Any extra headers (session ID, API version, workspace ID). No limit on count. | - -> **Best for:** Production and cloud-hosted bots - -### Option B: Secure Proxy (Private Network) - -If your chat agent is not publicly reachable (it sits behind a corporate firewall, within a VPC, or on an internal network), TestMu AI provides a lightweight proxy agent that you install inside your network. The agent establishes a secure outbound tunnel to TestMu AI, so test traffic reaches your agent without exposing it to the public internet. - -**Flow:** `Testing Platform` → `Secure Tunnel` → `Proxy Agent (Your Network)` → `Your Chat Agent (Internal)` - -| You Provide | Details | -|-------------|---------| -| **Internal Endpoint** *(Required)* | The internal URL or hostname of your chat agent (for example `https://chatbot.internal:8443/chat`). | -| **Auth Headers** *(Required)* | The same authentication headers your chat agent expects. The proxy handles network access, not authentication. Your agent still validates credentials on every request. | -| **Additional Headers** *(Optional)* | Any extra headers your chat agent requires. | -| **Proxy Agent** | Provided by TestMu AI. A lightweight service installed on a machine in your network that can reach the agent. Only outbound connectivity is needed, with no inbound firewall rules. | - -> **Best for:** Enterprise and on-premise deployments - -### Option C: Localhost (Dev / Staging) - -For chat agents running on a local development machine (for example `localhost:3000`), the same proxy agent is installed locally. It creates a secure tunnel from TestMu AI to your machine, so the platform reaches your locally running agent. - -**Flow:** `Testing Platform` → `Secure Tunnel` → `Proxy Agent (Your Machine)` → `localhost:port` - -| You Provide | Details | -|-------------|---------| -| **Local Endpoint** *(Required)* | The localhost URL and port your chat agent runs on (for example `http://localhost:3000/chat`). | -| **Auth Headers** *(If applicable)* | If your local agent enforces authentication, provide the same auth headers. If auth is disabled in dev mode, skip this. | -| **Proxy Agent** | Provided by TestMu AI. Installed on your local machine, running as a background process during testing. | - -> **Best for:** Development and pre-deployment testing - -### Comparison at a Glance - -| | Option A: Public API | Option B: Proxy | Option C: Localhost | -|---|---|---|---| -| **Chat agent reachable from internet?** | Yes | No | No | -| **Proxy agent needed?** | No | Yes (installed in your network) | Yes (installed on your machine) | -| **Auth headers required?** | Required | Required | If enforced | -| **Additional headers supported?** | Yes | Yes | Yes | -| **Firewall changes needed?** | None | None (outbound only) | None | -| **Typical use case** | Production, cloud-hosted | Enterprise, on-premise | Development, staging | - -Whatever the connection method, the platform sends the exact authentication and custom headers you configure with every request. The proxy agent handles network reachability only. It does not bypass or replace your agent's authentication. - -## How the Platform Reads Your Chat Agent's Response - -The platform reads the reply from each response, uses it to continue the conversation, and scores the result. Response shapes vary by provider, and the platform adapts to yours. - -```json title="Example Response" -{ -"id": "1310ab59-9477-4311-8f1b-b3eb81105133", -"input": [ -{ "role": "user", "content": "Hi, I need help with my account" } -], -"output": [ -{ "role": "assistant", "content": "Sure! Could you please provide more details…" } -], -"createdAt": "2025-10-29T06:02:45.616Z", -"cost": 0.0063 -} -``` - -### Common Response Fields - -Most chat agent APIs return some variation of the following. The exact field names and structure depend on your provider. - -| Field (typical) | Type | Description | -|-----------------|------|-------------| -| `id` | string | Unique identifier for the conversation exchange. | -| `input` / `messages` | array or string | Echo of the user message that was sent. May include a `role` and `content` pair, or a plain string. | -| `output` / `response` | array or string | The agent's reply. **This is the primary field the platform evaluates.** May be nested under `role: "assistant"` or returned as a top-level string. | -| `assistantId` / `botId` | string | Identifies which assistant or bot handled the request, if applicable. | -| `createdAt` / `timestamp` | ISO 8601 / epoch | When the exchange occurred. | -| `cost` / `usage` | number or object | Cost or token usage for this exchange, if the provider exposes it. | - -**Provider Flexibility** -The platform is not tied to any specific chat agent provider. Whether your bot runs on a third-party platform or a custom-built API, TestMu AI configures the request and parses the response to match your schema. - -## What You Provide to Connect a Chat Agent - -To connect your agent, provide the following. No code changes or infrastructure changes are required on your side. - -| Item | Required? | Description | -|------|-----------|-------------| -| **Chat agent URL** (API endpoint) | Required | The HTTPS endpoint where your agent receives messages. Example: `https://your-domain.com/chat` | -| **Authentication Credentials** | Required | The token or key needed to access your agent API. This can be a Bearer token, API key, or any other auth mechanism your API uses. | -| **Bot / Assistant Identifier** | Required | The identifier for the specific agent to test, if your platform hosts multiple bots under one API. | -| **Additional Headers** | Optional | Any extra headers your agent requires, such as a session token, API version, workspace ID, or custom tracking header. Provide as many as needed. | -| **Request Body Format** | Optional | If your agent expects a different body structure (field names, extra fields), provide it and TestMu AI matches it exactly. | - -## How the Platform Runs a Chat Evaluation - -Once credentials are configured, The platform runs the evaluation autonomously. - -`Credentials Configured` → `Test Scenarios Generated` → `API Calls Sent to Your Agent` → `Responses Captured` → `Quality Scored` - -- The platform sends requests using the exact URL, token, and headers you provide. -- It runs multi-turn conversations, reading the `output` field from each response to continue the dialogue across exchanges. -- Each completed conversation is scored across the 9 chat and voice quality metrics, and results appear in the dashboard. - -**Security** -All credentials are encrypted at rest and decrypted only at runtime during test execution. They are never exposed in reports or logs. - -## Troubleshoot Chat Agent Connection Failures - -Most connection failures come from authentication, the response path, or network reach. Each has a direct fix. - -- **What you see:** the platform reports an authentication error and no conversation runs. **Why:** the auth header or token is missing, expired, or wrong. **Fix:** confirm the Authorization header and token, and that the same credentials succeed in a direct curl call. - -- **What you see:** requests succeed but the reply is empty or scored as blank. **Why:** the platform is reading the wrong response field. **Fix:** point the response path at the field that holds the reply. For the example above, that is `output`. - -- **What you see:** every request fails to reach a private or local agent. **Why:** the proxy agent is not running, or the internal endpoint is wrong. **Fix:** confirm the proxy agent is installed and connected, and that the internal or localhost URL and port are correct. - -- **What you see:** requests are rejected before a reply. **Why:** the endpoint returns a non-2xx status such as 401 unauthorized, 404 not found, or 429 rate limited. **Fix:** verify the endpoint URL and method, and that the agent accepts the request body shape you configured. - -- **What you see:** the connection fails on a public endpoint. **Why:** the URL is not served over HTTPS, or the TLS certificate is invalid. **Fix:** serve the endpoint over HTTPS with a valid certificate, or use the proxy method for internal agents. - -## Related TestMu AI Guides - -- See how to [test chat agents from the terminal](/support/docs/testmu-a2a-cli/), including custom body templates and response paths. -- See how the platform [runs an evaluation end to end](/support/docs/architecture-and-how-evaluation-works/). diff --git a/static/docs/chat-agent.md b/static/docs/chat-agent.md deleted file mode 100644 index f7b8e2a29..000000000 --- a/static/docs/chat-agent.md +++ /dev/null @@ -1,90 +0,0 @@ -# Chat Agent Testing With TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The Agent Testing Platform tests a chat agent by holding multi-turn conversations with it through its API, following AI-generated scenarios, and scoring each conversation across quality metrics. It covers text chatbots, virtual assistants, IT helpdesks, and FAQ bots. - -You provide the endpoint, the agent's intended behavior, and any requirement documents. The platform generates the scenarios and runs the conversations. No code changes to your agent are required. - -## How Chat Agent Testing Works - -The platform generates 60 to 100+ scenarios from your prompt and uploaded documents, then runs each one as a full conversation. Testing agents send multi-turn HTTP messages to your endpoint, read the reply from each response, and decide the next message based on the scenario goal and prior context. - -A conversation ends when the goal is met, the maximum turn count is reached, or an unrecoverable error occurs. Each completed conversation is then scored. For the request and response mechanics, see how to [connect a chat agent over its API](/support/docs/chat-agent-api-integration/). - -## What Chat Testing Evaluates - -Every chat conversation is scored across the 9 chat and voice quality metrics, such as hallucination, bias, completeness, context awareness, response quality, and conversation flow. Each metric is scored on a 0.0 to 1.0 scale against the thresholds you set. - -Scenarios are drawn from behavioral test categories, including intent recognition, context and memory, multi-turn reasoning, error handling, and compliance. - -## Run a Chat Agent Test - -You can test a chat agent three ways. All three run the same evaluation and produce the same scored results. - -- **Dashboard:** create the agent, upload documents, generate scenarios, and run the evaluation. See how to [run your first evaluation in the UI](/support/docs/testing-your-first-ai-agent/). -- **CLI:** point `testmu-a2a test` at your endpoint for a one-command run, or drive scenarios and suites from a config file. See how to [test chat agents from the terminal](/support/docs/testmu-a2a-cli/). -- **API integration:** configure the endpoint, authentication, and request or response format directly. See how to [connect a chat agent over its API](/support/docs/chat-agent-api-integration/). - -## Read the Results - -Each run produces per-metric scores with pass or fail badges, full transcripts for failing conversations annotated with the evidence that drove the score, and score deltas against previous runs. The scores roll up into a Green, Yellow, or Red go-live verdict. - -## Features - -The chat agent workflow includes the following features. - -**Workflow-Based Test Generation.** Connect knowledge sources and auto-generate scenarios, with no manual scripting. - -- Upload knowledge base documents (PDF, text files). -- Connect Confluence, JIRA, or GitHub as knowledge sources. -- Generate test scenarios automatically from uploaded documents. -- Watch generation progress stream live. - -**Scenario Management.** Build and manage the conversations you test, manually or with AI. - -- Create scenarios manually with title, description, and expected behavior, or generate them from knowledge sources. -- Define custom pass/fail validation criteria per scenario. -- Add special instructions, assign personas, and link test data profiles. - -**Test Suites.** Group related scenarios and track results over time, with run history and status filtering. - -**Endpoint Profiles.** Configure how the platform connects to your agent's API. - -- Import a Postman collection or configure endpoints manually via JSON. -- Set up multi-phase execution: suite setup (login/auth), scenario setup (session creation), and chat. -- Manage static, auto-generated, and extracted variables, with retry, caching, endpoint testing, and import/export. - -**Test Profiles.** Reusable key-value test data with typed fields, default profiles, and JSON import/export, injected at runtime for data-driven testing. - -**Playground.** Interactively test your agent before a full run, with multi-turn chat, cURL connection testing, and request/response schema analysis. - -**Evaluation Execution.** Run evaluations at scale with metric selection, endpoint profile selection, HyperExecute integration with tunnel support for private agents, and real-time streaming over server-sent events. - -**Metric Threshold Configuration.** Set a minimum score (0.0 to 1.0) per metric, configure higher or lower is better, and save named threshold configurations. - -**Go-Live Assessment.** Get a Green, Yellow, or Red production readiness verdict with an overall score (0 to 100), confidence level, dimension scores, scenario coverage, risk assessment, and AI recommendations. - -**Scheduled Runs.** Automate regression coverage with cron-based scheduling, IANA timezones, pause and resume, and run history. - -## Metrics - -Chat agents are evaluated on 9 quality metrics, each scored on a 0 to 100% scale. - -| # | Metric | What it measures | -|---|--------|------------------| -| 1 | Bias Detection | Detects biased, discriminatory, or unfair responses | -| 2 | Hallucination Detection | Identifies false, fabricated, or unsupported information | -| 3 | Completeness | Whether the agent fully addresses the user's question | -| 4 | Context Awareness | How well the agent understands and maintains conversation context | -| 5 | Response Quality | Overall quality, clarity, and helpfulness of responses | -| 6 | Conversation Flow | Natural flow and coherence across multi-turn conversations | -| 7 | User Satisfaction | Estimated end-user satisfaction based on conversation quality | -| 8 | File Handling Quality | Quality of file upload and download interactions, if applicable | -| 9 | File Generation Accuracy | Accuracy of generated files against requirements, if applicable | - -Every evaluation result includes the overall score, per-metric scores with pass or fail badges, detailed analysis, the full multi-turn transcript, identified strengths, areas for improvement, actionable recommendations, and validation criteria results. - -## Related TestMu AI Guides - -- See how to [connect a chat agent over its API](/support/docs/chat-agent-api-integration/). diff --git a/static/docs/chrome-extension.md b/static/docs/chrome-extension.md deleted file mode 100644 index 98ae174cb..000000000 --- a/static/docs/chrome-extension.md +++ /dev/null @@ -1,95 +0,0 @@ -# TestMu AI Chrome Extension - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The [TestMu AI Chrome Extension](https://chrome.google.com/webstore/detail/lambdatest/fjcjehbiabkhkdbpkenkhaahhopildlh) enables you to perform cross-browser testing across 3000+ browser environments on the cloud more quickly and easily. Pick your favorite configurations, frequently used lists, and launch tests directly from your browser tab to save time. - -TestMu AI Chrome Extension provides the following features: - -* **Real Time Testing**: Perform Real-time tests of websites & web apps across 3000+ browsers & operating systems. -* **Screenshot Testing**: [Run Screenshot tests](/support/docs/automated-screenshot-testing/) of websites & web apps across 25 different browsers & operating systems. -* **Schedule Screenshots**: Schedule screenshots with preferred date, time and configurations and capture screenshots automatically. -* **Geolocation Testing**: Test websites and web apps to check how it renders across different geographies. -* **Favorites**: Save your favorites configurations in favorites section and launch tests faster & easily. -* **List**: Save configurations to your lists & initiate tests instantly. - -## How To Add TestMu AI Chrome Extension? - -**Step 1:** Visit [Chrome web store](https://chrome.google.com/webstore/category/extensions) on Chrome browser. - -**Step 2:** Search for **'TestMu AI'** in the search bar. - -**Step 3:** Click on **'Add To Chrome'**. - -**Step 4:** Approve permission request pop up by clicking on **'Add extension'**. - -You have successfully added TestMu AI Chrome extension to your Chrome browser. - -You can also see TestMu AI icon in top-right-hand side of the browser. - -## Setting Up Profile - -**Step 1:** Click on TestMu AI icon in top-right-hand side of the browser. Here you will need to enter your registered **'TestMu AI Email ID'** and **'TestMu AI Access Key'**. You can get your Email address & Access Key from your [TestMu AI Profile](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile) section. - -**Step 2:** Click on **'Apply'**. - -You are all set with TestMu AI Chrome Extension. - -## How To Initiate Real Time Tests? - -**Step 1:** Launch your website that you want to test and click on TestMu AI icon. - -**Step 2:** Select the configurations on which you want to perform live-interactive tests and click on **'START'**. - -You will be routed to TestMu AI Real Time Console where you can perform Real time testing of your websites & web apps. - -For more information, refer to **[Real Time Testing documentation](/docs/getting-started-with-desktop-browser-real-time-testing/)**. - -## How To Capture Screenshots? - -**Step 1:** Launch your website that you want to test and click on TestMu AI icon. - -**Step 2:** Select the configurations on which you want to [generate screenshots](https://www.testmuai.com/full-page-screen-capture) and click on **'CAPTURE'**. - -You will be routed to TestMu AI Screenshot Console where your screenshots will be generated. You can perform multiple operation on the screenshots like download screenshots as a zipped file into your local machine, share the screenshots, perform one-click bug logging and even more. Visit [**TestMu AI Screenshot Testing documentation**](/docs/automated-screenshot-testing/) for more details. - -## How To Add Selected Configurations To Favorites? - -Favorites allow you to perform real-time testing instantly and eliminates the hassle of selecting configurations every time. You will need to add your favorites configurations, and the particular configuration will save in the Favorite section. If you want to retest the same configuration, you can launch it right from your Favourite tab. - -**Step 1**: Once you are in the Real-time console, from the left menu bar, click on the Switch icon and then select **' Add to Favorites'**. - -**Step 2**: Now visit the TestMu AI Chrome Extension Home page and navigate to the **'Favorites'**. You'll find your selected configuration in the Favorites section. - -To retest on the same configuration, select the desired configuration and you will be redirected to a real-time console with the selected configuration. - -## How To Schedule Screenshots? - -**Step 1**: Launch your website that you want to test and click on the TestMu AI icon. Select the configurations on which you want to generate screenshots. After that, click on **'Schedule Screenshot'**. - -**Step 2**: After clicking, a pop-up will appear. Enter the desired time, frequency and the day you want to [schedule a screenshot](https://www.testmuai.com/automated-screenshot-scheduler) and then click on **'Schedule'**. - -## How To Perform Geolocation Testing - -With TestMu AI Chrome Extension, test your website for browser compatibility from different geographies. - -### How To Run Geolocation Tests While Performing Real Time Testing - -**Step 1**: Once you are in the Real-time console, select the Settings icon from the left menu bar and click on the dropdown **'Select a Country'**. - -**Step 2**: Select a country of your preferred choice and you'll be redirected to a respective IP address based on your selection. - -You can now perform the geolocation testing on the selected country. - -### How To Run Geolocation Tests While Performing Screenshot Testing - -**Step 1**: Launch your website that you want to test and click on the TestMu AI icon. Select the configurations on which you want to generate screenshots. After that, click on **'Settings'** icon. - -**Step 2**: Select the dropdown **'Select Country'** and choose your desired country. - -**Step 3**: Click on **'Apply'**. - -**Step 4**: Click on **'CAPTURE'** and your automated screenshots will be generate for the respective country. - -> -That's all! Increase your productivity with TestMu AI Chrome Extension. In case you have any queries, feel free to drop them at our **Customer Chat Support** or you can also mail us at **[support@testmuai.com](mailto:support@testmuai.com)**. diff --git a/static/docs/chrome-os-app-testing.md b/static/docs/chrome-os-app-testing.md deleted file mode 100644 index 3fec0eb31..000000000 --- a/static/docs/chrome-os-app-testing.md +++ /dev/null @@ -1,56 +0,0 @@ -# ChromeOS App Testing - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This document outlines how you can leverage the features like mark as bug, changing IP geolocation, in-depth device logs and network logs etc to ease your Real Time App testing process. - -## Steps to start your ChromeOS App Testing - -**Step 1:** Click on the Real Time option from your TestMu AI dashboard. -**Step 2:** Click on the **Virtual Mobile** option in **App Testing** section. -**Step 3:** Select the **ChromeOS** option, and upload your application to the TestMu AI. (We are using the sample application for the demo purpose) -**Step 4:** Select your screen resolution and click on **Start** button. - -## Device Controls -This transcend basic volume adjustments and lock/unlock functionalities. It empowers you to interact with the emulated ChromeOS device programmatically, automating tasks and integrating them within your testing scripts. This gives you fine-grained control over the device's state, tailoring the testing environment to your specific needs. - -## App Controls -This provide programmatic control over the applications installed within the emulator (e.g., install new app, kill the existing app, or uninstall the app). App Controls give you the power to do it all programmatically. This streamlines your testing process by eliminating the need for manual app manipulation within the emulator. - -## Screenshot -Screenshots remain a cornerstone for capturing static images of the current emulator screen. These screenshots serve as invaluable visual references throughout the testing process, aiding in bug reporting, test case documentation, and visual regression testing. - -## Record Session -This allows you to record a video of your testing session, providing a detailed playback for in-depth bug analysis, test case demonstrations, and comprehensive test case documentation. - -## Gallery -Serves as your central hub for all captured screenshots and recorded sessions from your ChromeOS testing endeavors. This eliminates the need to search for these assets across different locations. The Gallery provides easy access, allowing you to review, download, and organize your testing assets effortlessly. - -## Mark as Bug -It empowers you to streamline bug reporting within the testing environment itself. Encounter a UI glitch? Simply capture a screenshot, optionally annotate it with relevant details using shapes, text, and color coding, and seamlessly report the bug via your preferred integrated project management tool. This integration eliminates the need for context switching and facilitates efficient bug reporting. - -## IP Geolocation -This transcends basic location selection. It allows you to spoof the IP address of your emulated device, simulating various geographical locations across over 45 countries. This enables you to test geo-restricted features, localized content delivery, and the overall user experience for a global audience. - -## Network Throttling -It goes beyond simply simulating different network speeds (3G, 4G). It empowers you to create custom network setups, allowing you to test your app's behavior under a wide range of bandwidth limitations and network configurations. This is crucial for optimizing app performance and ensuring a seamless user experience even in areas with limited or unreliable internet connectivity. - -## Settings -Settings provide a central location to configure various aspects of your emulated ChromeOS environment. Change the language settings to conduct localized testing, or define the timeout scenario to manage test session duration based on your needs. - -## Switch -Switch allows you to seamlessly switch between multiple emulated devices within the same project. This eliminates the need to tear down and rebuild individual emulators, streamlining your testing workflow and enabling you to compare app behavior across different ChromeOS device configurations. - -## End Session -Terminates the current emulator instance, freeing up resources and allowing you to start a fresh session with a clean slate. - -## Advance Tools - -### UI Inspector -This is designed to examine and interact with the user interface (UI) elements of your mobile apps, or desktop applications. It acts like a behind-the-scenes detective, allowing you to delve into the inner workings of your app's interface. - -### DevTools -This provide access to all types of logs (Device, Network, and App) and you can also sort them as per their status (Activity, Warning, Debug, Error). You can also download these logs. - -### Adb Shell -It provides a Unix-like shell environment, allowing you to execute commands on the device itself. It is used for interacting with the Android operating system directly on a connected Android device or emulator. diff --git a/static/docs/chrome-os-testing.md b/static/docs/chrome-os-testing.md deleted file mode 100644 index c03226dca..000000000 --- a/static/docs/chrome-os-testing.md +++ /dev/null @@ -1,23 +0,0 @@ -# ChromeOS Testing - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -ChromeOS is a lightweight, Linux-kernel-based operating system designed for Chromebooks. It prioritizes speed, security, and manageability, with a focus on cloud-based applications and services. This makes it ideal for resource-constrained environments and cloud-centric workflows like ed-tech business. - - ChromeOS Web Browser Testing - Guidebook about the features offered for testing your webapps in ChromeOS - - ChromeOS App Testing - Guidebook about the features offered for testing your apps in ChromeOS - -## Benefits of using ChromeOS Testing - -### Efficiency and Speed - -- **Lightweight OS:** ChromeOS is known for its speed and efficiency. This translates to faster test setup times and smoother testing experiences compared to heavier operating systems. -- **Cloud-based Focus:** Chromebooks prioritize cloud-based apps and services. This allows for quicker test iterations and easier access to testing tools without software installations. - -### Accessibility and Consistency - -- **Standardized Environment:** ChromeOS offers a consistent user experience across different Chromebook models. This reduces variables and helps ensure tests are replicable on various Chrome devices. -- **Real-World Simulation:** Testing on ChromeOS provides a close approximation to how users will experience web apps and Chrome extensions in a real-world setting. diff --git a/static/docs/chrome-os-web-browser-testing.md b/static/docs/chrome-os-web-browser-testing.md deleted file mode 100644 index a24c25e34..000000000 --- a/static/docs/chrome-os-web-browser-testing.md +++ /dev/null @@ -1,63 +0,0 @@ -# ChromeOS Web Browser Testing - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This document outlines how to leverage features like mark as bug, changing IP geolocation, in-depth device logs and network logs, etc for your Real Time Web Browser testing. - -Apart from these features, TestMu AI also offers you the - -## Steps to start your ChromeOS Web Browser Testing - -**Step 1:** Click on the Real Time option from your TestMu AI dashboard. -**Step 2:** Click on the **Virtual Mobile** option in **Web Browser Testing** section. -**Step 3:** Select the **ChromeOS** option, and enter your URL in the space provided. (We are using the TestMu AI website for the demo purpose) -**Step 4:** Select your screen resolution and click on **Start** button. - -## Device Controls -The Device Control feature allows you tp perform certain actions with your device within your current test session: - -- **Volume:** Adjust the device volume directly within your testing session, ensuring optimal audio levels for your tests. -- **Lock Device :** Lock or unlock your device seamlessly during testing sessions without interrupting your workflow. - -## Screenshot -Capture clear screenshots of the current screen on your test device. Easily access and download these screenshots for future reference or bug reporting. - -## Record Session -Record your entire test session for detailed playback. A timer keeps track of recording duration, allowing you to pinpoint specific moments for review. You can click on the stop button to stop the video recording. - -## Gallery -All screenshots and video recordings from test sessions are stored in the Gallery. To access, select Gallery from the left toolbar and you'll find the saved screenshots and videos. To download the screenshots or videos to your local machine, click on the Download icon. You can also mark them as bug by clicking on the Bug icon. - -## Mark as Bug -This feature allows you to mark any bug in your UI during the session. You can mark the bug in your UI via different shapes, texts and of different colors. You can also configure different project management tools with your session to create a ticket for the same. - -You can also share that bug, save the image to the gallery or download that screenshot. - -## IP Geolocation -The IP Geolocation feature enables you to simulate website and mobile experiences from over 45 countries by using local IP addresses. This allows you to test localized features such as pricing, languages, and product offerings. With IP Geolocation enabled, you can verify that your app's traffic is coming from a specific country, allowing for comprehensive global testing. - -## Network Throttling -TestMu AI's network throttling feature empowers you to simulate real-world network conditions (2G, 3G, LTE) during testing. This allows you to assess your application performance under varying internet speeds, ensuring a seamless user experience across different network environments. You can leverage predefined network profiles or craft custom settings for comprehensive testing. - -## Settings -The settings section offer the following features: -- **Idle Timeout :** This feature allows you to set the idle timeout for your test session. The default Idle Timeout is 5 minutes and the maximum limit is 60 minutes. -- **Time Zone :** With this feature, you can select the time zone of your preferred region that align with your testing needs. -- **Keyboard Input** With this feature, you can change the language in your test session for a more localized testing experience. - -## Switch -This feature allows you to switch between devices, browser and their OS versions without exiting your current test session. - -## End Session -By clicking on the end session button, you can close your current test session. - -## Advance Tools - -### Chrome DevTools -Chrome DevTools offer a comprehensive suite of features to inspect and debug web pages. Here's how DevTools can help you with styles, event listeners, DOM breakpoints, properties, and accessibility: - -- **Styles:** Inspect and modify CSS styles directly on the webpage. See applied, inherited, and final computed styles. -- **Event Listeners:** View, disable, and set breakpoints on event listeners associated with elements to debug event handling. -- **DOM Breakpoints:** Pause script execution when specific changes occur in the DOM tree for efficient debugging of dynamic content. -- **Inspect & Modify Element Properties:** View detailed information about each element, including type, class names, IDs, attributes, and JavaScript variable values. Modify properties for testing or temporary adjustments. -- **Accessibility Tools:** Identify and address potential accessibility issues with the built-in audit tool, color contrast checker, and vision deficiency simulator. diff --git a/static/docs/circle-ci-with-hyperexecute.md b/static/docs/circle-ci-with-hyperexecute.md deleted file mode 100644 index dbbebd296..000000000 --- a/static/docs/circle-ci-with-hyperexecute.md +++ /dev/null @@ -1,85 +0,0 @@ -# CircleCI Integration with Hyperexecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -CircleCI allows you to build fully automated pipelines from testing to deployment, freeing up your time to focus on the real work of innovation. Using CircleCI, you can automate your entire testing suite for new commits, reducing the possibility of human error, while also automating deploys with orbs. - -This document will show you how to integrate CircleCI Pipeline with HyperExecute to greatly shorten your test cycles. - -## How To Integrate CircleCI Pipeline with Hyperexecute - -To integrate CircleCI Pipeline with HyperExecute, follow the below steps: - -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** -Download or Clone the code sample from the TestMu AI GitHub repository to run the tests on the HyperExecute. - - View on GitHub - -### 1. Log into your CircleCI account - -### 2. Create a New Project - -Follow these steps to create a new project in CircleCI: - -- In CircleCI, click **Projects** in the sidebar. - -- Find your project and click the blue **Set Up Project** button next to it. - -If you cannot see your project, check that you have selected the correct organization in the top left-hand corner of CircleCI. - -### 3. Specify a Config File - -Once you have set up your project, you will be prompted to provide a config.yml file. - -- From the pop-up window, select your preferred option. You can either: - - - Include a config.yml in the .circleci directory of your repo. - - - Commit a starter CI pipeline to a new branch of your repo. - - - Use an editable config.yml template. - - If you choose the starter CI pipeline, a sample config.yml file is created and committed to a circleci-project-setup branch in your repo. - -Below is a sample of CircleCI YAML created for your reference: - -```bash -version: 2.1 -jobs: -test-java: -docker: -- image: cimg/openjdk:17.0 -steps: -- checkout -- run: -name: "Download HE CLI" -command: wget https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute -- run: -name: "Permissions" -command: chmod u+x hyperexecute -- run: -name: "Runner Command" -command: ./hyperexecute --config yaml/autosplit_linux.yaml -workflows: -build-and-test: -jobs: -- test-java -``` - -- Click the blue **Set Up Project** button. - -CircleCI then uses the specified config.yml file to run your pipeline. You can see the output in the CircleCI dashboard. - -To make changes to your pipeline, either edit the .circleci/config.yml file in your repo, or click the ellipsis next your project in the Dashboard and choose **Configuration File**. - -### 4. Run Your Job -- Once your changes are made and your configuration is valid, you may commit and re-run the pipeline by clicking the **Save and Run** button. - -- A modal will pop up, and you will see the option to commit on the branch you are working from, or you can choose to create a new branch for the commit. - -**Below is an example of a Hyperexecute job that was triggered through the above pipeline:** - -> -**Run your tests at speeds never seen before. Happy testing! :)** diff --git a/static/docs/circleci-integration-with-testmu.md b/static/docs/circleci-integration-with-testmu.md deleted file mode 100644 index 4d626cedb..000000000 --- a/static/docs/circleci-integration-with-testmu.md +++ /dev/null @@ -1,248 +0,0 @@ -# CircleCI Integration With TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -CircleCI is one of the most renowned CI/CD tools due to its easy-to-use interface, simplistic configuration, and huge support for numerous libraries and services. CircleCI ensures that your code deployment is of high quality and in less time. It does so by passing every commit to run through your automated pipeline before pushing it to the respective feature branch. So if anything goes south, then you know instantly where to look for. This helps in preventing a major breakdown by eliminating any bottleneck at an early stage. CircleCI facilitates rapid as well as robust fixes at scale. - -TestMu AI now integrates with CircleCI to boost your go-to-market delivery. Perform automated cross browser testing with TestMu AI to ensure your development code renders seamlessly through an online Selenium grid providing 3000+ real browsers running through machines on the cloud. Perform automation testing in parallel with TestMu AI’s Selenium grid to trim down your test cycles drastically. - -## Prerequisites - -To perform your karma tests with TestMu AI, you would need the following things to be already set up: - -* Global Dependencies - - A Git or GitHub repository - - Download and install node.js and node package manager or npm. - To install node.js with homebrew, use the below command. -``` -$ brew install node -``` - - If you have npm already installed, you may want to upgrade it to the latest version. Here the code you can run in your terminal to upgrade npm. - -``` -npm install npm@latest -g -``` - -* TestMu AI Authentication Credentials - - Be aware of your TestMu AI authentication credentials, i.e., your TestMu AI username, access key, and HubURL. You need to set them up as your environment variables. You can retrieve them from your [TestMu AI automation dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/) by clicking on the key icon near the help button. - -``` -$ export LT_USERNAME= -$ export LT_ACCESS_KEY= -``` - -## Integrating TestMu AI With CircleCI - -To integrate CircleCI with TestMu AI, you need to make minor tweaks in the `.circleci/config.yml` file, which is the configuration file for your CircleCI instance. The changes you need to make will revolve around the environment variables such as access key, username, grid config and so on. Refer to the official CircleCI documentation on environment variables for more information regarding their set up. - -Below is a sample config file for integrating CircleCI with TestMu AI. You can use the sample file to test your own automation test suite by replacing the environment variables according to your need. All the code used for the demonstration of this topic can be found in our GitHub repository. - -``` - -# Javascript Node CircleCI 2.0 configuration file -# Check https://circleci.com/docs/2.0/language-javascript/ for more details - -version: 2 - -jobs: - -build: - -docker: - -# specify the version you desire here - -- image: circleci/node:7.10 - -# Specify service dependencies here if necessary - -# CircleCI maintains a library of pre-built images - -# documented at https://circleci.com/docs/2.0/circleci-images/ - -# the working dir is github repo that you need to fork to become owner. - -working_directory: ~/nightwatch-saple-for-circleci - -steps: - -- checkout - -- run: - -name: "Setup custom environment variables // its your workflow step" - -command: | - -echo 'export LT_USERNAME="{your_lambdatest_username}"' >> $BASH_ENV - -- run: - -name: "Setup custom environment variables" - -command: | - -echo 'export LT_ACCESS_KEY="{your_lambda_access_key}"' >> $BASH_ENV - -- run: # Validating your above mentioned environment variables - -name: "Here is the LT_Username : " - -command: echo ${LT_USERNAME} - -# Download and cache dependencies - -- restore_cache: - -keys: - -- v1-dependencies-{{ checksum "package.json" }} - -# fallback to using the latest cache if no exact match is found - -- run: npm install - -# run tests! - -- run: node_modules/.bin/nightwatch -e chrome // Executing test in bash. -``` - -TestMu AI offers a [Selenium Desired Capabilities Generator](https://www.testmuai.com/capabilities-generator/) to fetch coded values for your desired test configurations. - -For Instance, if you select the below configurations: - -| Capability | Selected Value | -|------------------|-----------------| -| Selenium Version | 3\.11\.0 | -| Resolution | 1280×1024 | -| Operating System | Windows 8\.1 | -| Browser Version | 62\.0 | -| Browser | Mozilla Firefox | - -Then Capabilities Generator at TestMu AI will provide you with the below program: - -``` -var capabilities = { -"build" : "your build name", //You can edit this and assign a build name -"name" : "your test name", // Assign a name to your Test -"platform" : "Windows 8.1", // The operating system on which you want to test your website -"browserName" : "Firefox", // The browser on which you want to test -"version" : "62.0", // The browser version which you've selected to perform the test upon -"resolution" : "1280x1024", // The resolution in which you want to run the test as per your operating system -"selenium_version" : "3.11.0", //The version of Selenium on which the test will run -"visual" : true, -"firefox.driver" : v0.21.0 -} -``` - -For more information, refer to our documentation on [Selenium Automation Capabilities](/docs/selenium-automation-capabilities/). - -## Testing Locally Hosted or Privately Hosted Projects - -To help you perform cross browser testing of your locally stored web pages, TestMu AI provides an SSH(Secure Shell) tunnel connection with the name Lambda Tunnel. With **Lambda Tunnel**, you can execute a test server inside your CircleCI build container to perform automated cross browser testing on browsers offered by Selenium grid on TestMu AI. So you make sure how well your changes look, even before your customers. Curious to know more about Lambda Tunnel? - ->Follow our documentation on Lambda Tunnel to know it all. OS-specific instructions to download and set up tunnel binary are at the following links. ->- [Documentation For Windows User](/docs/local-testing-for-windows/) ->- [Documentation For Mac User](/docs/local-testing-for-macos/) ->- [Documentation For Linux User](/docs/local-testing-for-linux/) - ->Download the binary file of: -> ->- [Lambda Tunnel for Windows](https://downloads.lambdatest.com/tunnel/v3/windows/64bit/LT_Windows.zip) ->- [Lambda Tunnel for Mac](https://downloads.lambdatest.com/tunnel/v3/mac/64bit/LT_Mac.zip) ->- [Lambda Tunnel for Linux](https://downloads.lambdatest.com/tunnel/v3/linux/64bit/LT_Linux.zip) - -The below example of *config.yml* file would demonstrate to you how to leverage TestMu AI’s Selenium grid by performing a browser test through your testing server contained in your CircleCI build. - -``` -# Javascript Node CircleCI 2.0 configuration file -# -# Check https://circleci.com/docs/2.0/language-javascript/ for more details -# -version: 2 -jobs: -build: -docker: -# specify the version you desire here -- image: circleci/node:7.10 - -# Specify service dependencies here if necessary -working_directory: ~/Nightwatch-circleci-selenium - -steps: -- checkout -- run: -name: "Downloading tunnel binary" -command: | -wget https://downloads.lambdatest.com/tunnel/v3/linux/64bit/LT_Linux.zip - -- run: -name: "Extracting tunnel binary" -command: | -sudo apt-get install unzip -unzip LT_Linux.zip - -- run: -name: "Executing tunnel binary" -background: true -command: | -./LT -user ${LAMBDATEST_EMAIL} -key ${LAMBDATEST_KEY} -sleep 40 - -- run: -name: "Setup custom environment variables" -command: | -echo 'export LT_USERNAME="${LAMBDATEST_USERNAME}"' >> $BASH_ENV - -- run: -name: "Setup custom environment variables" -command: | -echo 'export LT_ACCESS_KEY="${LAMBDATEST_ACCESS_KEY}"' >> $BASH_ENV - -- run: # test what branch we're on. -name: "Here is the LT_Username : " -command: echo ${LT_USERNAME} - -# Download and cache dependencies -# - restore_cache: -# keys: -# - v1-dependencies-{{ checksum "package.json" }} -# fallback to using the latest cache if no exact match is found - -- run: npm install -# - save_cache: -# paths: -# - node_modules -# key: v1-dependencies-{{ checksum "package.json" }} - -# run tests! -- run: node_modules/.bin/nightwatch -e chrome -``` - -Once the tunnel is successfully set up. You can add the below code to your capabilities for testing internal servers on your network. - -``` -"desiredCapabilities": { -"tunnel":true -} -``` - -## Parallel Testing - -Parallel Testing is one of the most demanding features of TestMu AI Selenium Grid. By parallel testing, you can run more than one test case simultaneously. This means that Parallel testing would allow you to execute numerous automation test cases altogether. You perform a single test scenario across different browsers or could run different test scenarios across the same browser but with different browser versions. - -To perform automation testing in parallel on Windows, use the below command. - -``` -$ node_modules\.bin\nightwatch -e chrome,edge,firefox tests -``` - -To perform automation testing in parallel on macOS/Linux use the below command. - -``` -$ ./node_modules/.bin/nightwatch -e chrome,edge,firefox tests -``` - -Monitor and analyze your test result on the [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/). - -Deploy your code in a reliable manner at scale using CircleCI integration with TestMu AI, and ensure it looks robust across every browser to provide a seamless user experience to all your visitors. Happy Testing! diff --git a/static/docs/clickup-integration.md b/static/docs/clickup-integration.md deleted file mode 100644 index 89ab341f2..000000000 --- a/static/docs/clickup-integration.md +++ /dev/null @@ -1,60 +0,0 @@ -# ClickUp Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -ClickUp offers a highly customizable cloud-based platform to boost your productivity for both professional and personal life by storing all of your tasks, docs, conversations, timelines, spreadsheets, and more in a single place. You can have different views of the project such as Gantt view, board view, and calendar view. You could also chat with your colleagues, and share documents with the team. ClickUp also offers numerous features for time management, task management, project management, along with effective collaboration, reporting, and integrations to third-party tools such as TestMu AI. - -TestMu AI integration with ClickUp will help you report bugs and UI anomalies directly in your desired project on ClickUp while performing [cross browser testing](https://www.testmuai.com/) with TestMu AI. You can capture a screenshot of the bug, highlight the anomaly, annotate extra information, choose an assignee, provide a suitable title and description to help relay further actions. You can do all that directly from TestMu AI, in just a single click. - -## How To Integrate ClickUp With Your TestMu AI Account? - -**Step 1:** Login to your TestMu AI account. You would need to have Admin or User level access to see and install integrations. - -**Step 2:** Select ‘Integration’ from the left navigation menu bar. This will guide you to a screen where you will find a list of 3rd party applications, available to integrate with your TestMu AI account. - -**Step 3:** Click on the block that says ‘ClickUp’. - -**Step 4:** After you click on the ClickUp icon, you will need to authenticate TestMu AI API with your ClickUp account. If you are not logged into your ClickUp account, then you will be asked to do so for successfully authenticating you ClickUp account with TestMu AI. - -> Note: If you are already logged into your ClickUp account, you will be redirected to the ClickUp instance for authenticating the TestMu AI APIs to fetch necessary details from your ClickUp account. - -**Step 5:** After authentication of your ClickUp account, you will be redirected back into TestMu AI application where you will notice a prompt message indicating that you have successfully integrated your TestMu AI account with your ClickUp instance. You will also notice a green tick and a refresh icon. The refresh button will help you synchronize your ClickUp account with TestMu AI in just a single click. - -## Log Your First Bug Through TestMu AI Integration With ClickUp - -**Step 1:** Go for any of the test from the left navigation menu. For demo, we will be taking **"Real Time Testing"** option. - -**Step 2:** Present a URL of the web-app you need to test in the dialog box. After that, select any configuration for browser and operating system of your choice & hit **'Start'**. - -**Step 3:** After the VM is launched and operable. You can perform testing on your web-app for finding bugs. If a bug gets revealed, then you need to click on the **Bug icon** from the left panel for [capturing a screenshot](https://www.testmuai.com/full-page-screen-capture) of the same. We have highlighted that option with yellow in the below image. - -**Step 4:** After a screenshot is captured, you can annotate any issue or a task with an in-built image editor. Once you are done highlighting the bug, click on the button that says **"Mark as Bug"**. - -**Step 5:** After clicking on **"Mark as Bug"** button a form specific to ClickUp would open up. Fill the fields as per your requirement, and **create the issue**. - -* You can select the Team you wish to assign the bug. -* Specify the Space on which you wish to log the bug. -* Choose a particular Project. -* nDetermine the List on which you wish to involve the UI bug/suggestion. -* Set a Status for the task. -* Assign it to a colleague. -* Provide a task name. -* A relevant Description about the UI observation. - -> Note: After you click on **"Create Bug"**, you will be able to observe it being successfully marked through a single click effort. You will get prompt messages on top of your Virtual Machine indicating the progress of bug logging. After few seconds you will be notified with a prompt message **"Bug successfully marked"** indicating that the screenshot has been pushed to your **ClickUp project**. - -**Step 6:** Now, go to your ClickUp project and you will notice your logged task under the specific project that you assigned it through TestMu AI. - -**Step 7:** Click on the logged task and you will find all the details about the bug that you stated while marking the bug from TestMu AI. You will also find a URL that would route you to the particular screenshot that was logged while creating the issue from TestMu AI. - -## Remove TestMu AI Integration With ClickUp - -> You can work with one integration at a time. So if you would want to integrate to a similar 3rd party application, then you would have to **remove** your current integration. Here is how you can do that. - -**Step 1:** Login to your TestMu AI account. - -**Step 2:** Select **‘Integrations’** from the left-navigation menu. This will guide you to a screen where you will find 3rd party applications, available to integrate with your TestMu AI account. - -**Step 3:** Click on **‘REMOVE’**. You can find the remove button under the ‘ClickUp’ block which would be highlighted with a green tick. - -> That was all you need to know for TestMu AI + ClickUp Integration. Increase your productivity with our integrations. If you still have any questions for us, please feel free to let us know. Our experts are always available on **chat** to help you out with any roadblock regarding our product. Happy testing! 🙂 diff --git a/static/docs/codecept-on-hyperexecute.md b/static/docs/codecept-on-hyperexecute.md deleted file mode 100644 index 9a4cebad2..000000000 --- a/static/docs/codecept-on-hyperexecute.md +++ /dev/null @@ -1,167 +0,0 @@ -# Running Codecept Framework Tests on HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -CodeceptJS is a versatile JavaScript testing framework that supports Puppeteer for end-to-end testing of web applications. It introduces a human-readable syntax and a versatile architecture that allows writing tests in a natural language style. - -HyperExecute is an AI Native Test Orchestration Cloud Platform that empowers you to run **end-to-end** tests **quickly** and **efficiently**. It provides Just-in-Time (JIT) testing infrastructure with fast execution **speeds**, **smart orchestration**, and **detailed logs**. - -This guide details how to execute your **Codecept** framework tests on **HyperExecute** via two different methods: - -- [**Using Local System**](/support/docs/codecept-on-hyperexecute/#1-testing-using-local-system) - You can use your own local machine to execute tests. -- [**Using Gitpod Platform**](/support/docs/codecept-on-hyperexecute/#2-testing-using-gitpod) - Execute tests using GitPod. (Requires a [Gitpod](https://gitpod.io/login/) account) - -## 1. Testing Using Local System - -Follow the step-by-step guide to execute your test on HyperExecute. - -### Prerequisites - -To run the Tests on HyperExecute from your Local System, you are required: - -- Your TestMu AI [Username and Access key](/support/docs/hyperexecute-how-to-get-my-username-and-access-key/) -- [HyperExecute YAML](/support/docs/hyperexecute-yaml-version0.2/) file which contains all the necessary instructions. -- [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) in order to initiate a test execution Job . -- Setup the [Environmental Variable](/support/docs/hyperexecute-environment-variable-setup/) - -### Step 1: Configure Your Test Suite - -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** - -Download or Clone the code sample for the JUnit from the TestMu AI GitHub repository to run the tests on the HyperExecute. - - View on GitHub - -If you are using your own project, make sure you update the **Hub endpoint** in your tests file. - -By setting up the Hub endpoint, you establish the communication channel between your tests and the browser nodes, enabling effective test distribution and execution. - -> You can also use **dev** and **beta** browser versions. To generate capabilities for your test requirements, you can use our inbuilt 🔗 Capabilities Generator. - -### Step 2: Setup the CLI in your Test Suite - -After cloning / downloading the sample repo, you need to setup the CLI and the environment variables. - -#### Download the HyperExecute CLI - -The CLI is used for triggering the tests on HyperExecute. It is recommend to download the CLI binary on the host system and keep it in the root directory of the suite to perform the tests on HyperExecute. - -You can download the CLI for your desired platform from the below mentioned links: - -| Platform | HyperExecute CLI | -| ---------| ---------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| MacOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -#### Setup Environment Variable - -Now, you need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in the [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). - -Run the below mentioned commands in your terminal to setup the CLI and the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -### Step 3: Configure YAML in your Test Suite - -Configure your YAML file as per your use cases using **key value** pairs. - -In this sample YAML file, we have mentioned: - -- **version** of the YAML file -- **Timeouts** for executing your project -- **Mode of execution** is [Autosplit](/support/docs/hyperexecute-auto-split-strategy/). You can also opt for [Matrix](/support/docs/hyperexecute-matrix-multiplexing-strategy/) or [Hybrid](/support/docs/hyperexecute-hybrid-strategy/) mode. -- **Pre and Post** commands -- **Reports and Artefacts** that will be generated after the completion of tests -- and other necessary YAML Parameters - -```yaml ---- -version: "0.1" - -globalTimeout: 90 -testSuiteTimeout: 90 -testSuiteStep: 90 - -runson: linux - -autosplit: true - -retryOnFailure: false -maxRetries: 1 - -concurrency: 2 - -pre: -- npm install - -cacheKey: '{{ checksum "package.json" }}' -cacheDirectories: -- node_modules - -testDiscovery: -type: raw -mode: remote -command: grep -lr 'Scenario' *test.js - -testRunnerCommand: npx codeceptjs run $test --steps - -jobLabel: [puppeteer-CodeceptJS, linux, autosplit] -``` - -### Step 4: Execute your Test Suite - -> **NOTE :** In case of MacOS, if you get a permission denied warning while executing CLI, simply run **`chmod u+x ./hyperexecute`** to allow permission. In case you get a security popup, allow it from your **System Preferences** → **Security & Privacy** → **General tab**. - -Run the below command in your terminal at the root folder of the project: - -```bash -./hyperexecute --config RELATIVE_PATH_OF_YOUR_YAML_FILE -``` - -OR use this command if you have not exported your username and access key in the step 2. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE `} - -### Step 5: Monitor the Test Execution - -Visit the [HyperExecute Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) and check your Job status. - -### Step 6: Download Artifacts and Reports - -HyperExecute also facilitates the provision to download the [Artifacts](/support/docs/hyperexecute-artifacts/) and [Reports](/support/docs/hyperexecute-reports/) on your local machine. Click on the corresponding button to download your generated artifacts and reports. - -## 2. Testing Using Gitpod - -You can also use the Gitpod platform to execute our sample repository. It will fetch all the sample codebases and trigger the CLI to execute the tests. - -Follow the below steps to run Test using Gitpod: - -**Step 1:** Click '**Open in Gitpod**' button. You will be redirected to Login/Signup page. This button is configured to redirect you to the Gitpod platform where you will be able to execute our sample repository. - -**Step 2:** Login with TestMu AI credentials. Once logged in, a pop-up confirmation will appear, asking you to **'Proceed'** to the Gitpod editor in a new tab. The current tab will display the HyperExecute Dashboard. - -**Step 3:** Choose your preferred editor (we recommend VS Code Editor) - -**Step 4:** As you are running a sample project, Fetching of the Test Scripts, [HyperExecute YAML](/support/docs/deep-dive-into-hyperexecute-yaml/), [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) and Triggering your tests using the `Execution Command` will be automated. - -**Step 5:** Once you see the `Job Link` in the logs, you can visit the [HyperExecute dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) to see the tests getting executed. - -You can also implement [Secret Keys](/support/docs/hyperexecute-how-to-save-and-manage-secrets/) in your YAML file. - -## Navigation in Automation Dashboard - -Every test run on the HyperExecute has a unique *jobId* associated with it. Each *jobId* can in turn constitute single (or multiple) *groupId*(s). You can visit [HyperExecute Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) for checking the status of the test execution. - -You can seamlessly navigate between JobId's and taskId's. You need to click on the *testID* to navigate from the HyperExecute logs to the Automation Dashboard. - -The snapshot below shows the videos, logs and other meta data for that specific *test_ID* - -> For any query or doubt, please feel free to contact us via **24×7 chat support** or you can also drop a mail to **support@testmuai.com**. -Happy testing! diff --git a/static/docs/codeception-integration-with-testmu.md b/static/docs/codeception-integration-with-testmu.md deleted file mode 100644 index c8fc30410..000000000 --- a/static/docs/codeception-integration-with-testmu.md +++ /dev/null @@ -1,126 +0,0 @@ -# Selenium With Codeception - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Run Codeception tests on the TestMu AI cloud grid. This guide covers setup, running a sample test, configuring capabilities, and testing locally hosted pages. - -**Sample repo** -All the code used in this guide is available in the sample repository. - - View on GitHub - -## Prerequisites - -Complete the following steps before you begin automation testing with Selenium and Codeception. - -1. Install the latest **PHP** on your system. Use the following commands in the terminal: - - * **MacOS:** Previous versions of **MacOS** have **PHP** installed by default. For the latest **MacOS** versions starting with **Monterey**, download and install **PHP** manually: -```bash -/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" -brew install php -``` - * **Windows:** Download **PHP** from [PHP for Windows](http://windows.php.net/download/) and add it to your system PATH. Refer to the [PHP Windows installation guide](http://php.net/manual/en/install.windows.php) to ensure PHP is accessible through Command Prompt (cmd). - -2. Download **composer** in the project directory ([Linux/MacOS](https://getcomposer.org/download/), [Windows](https://getcomposer.org/doc/00-intro.md#installation-windows)). - - **Note:** To use the **composer** command directly, it either should have been downloaded in the project directory or should be accessible globally which can be done by the command below: -```bash -mv composer.phar /usr/local/bin/composer -``` - -## Step 1: Clone the Sample Project - -Clone the repository and install dependencies. - -```bash -git clone https://github.com/LambdaTest/codeception-selenium-sample -cd codeception-selenium-sample -``` - -Install the composer dependencies: -```bash -composer install -``` - -## Step 2: Set Your Credentials - -Configure your credentials to connect to the TestMu AI Selenium Grid. - -Set TestMu AI `Username` and `Access Key` in environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -## Step 3: Configure Your Test Capabilities - -Define browser, version, and OS settings for your test run. - -In the test script, update your test capabilities. Notice the declaration of class name **"AcceptanceTester"**. This class specifies test configuration, port number, browser name, browser version, and other desired capabilities. - -```php -# Codeception Test Suite Configuration -# -# Suite for acceptance tests. -# Perform tests in browser using the WebDriver or PhpBrowser. -# If you need both WebDriver and PHPBrowser tests - create a separate suite. - -class_name: AcceptanceTester -modules: -enabled: -- WebDriver: -url: 'https://lambdatest.github.io/sample-todo-app/' -host: '{username}:{token}@hub.lambdatest.com' #provide your TestMu AI credentials -port: 80 -browser: chrome - -capabilities: -name: 'Codeception Example' -build: '1.0' -browserName: 'Chrome' # request the latest version of chrome -platform: 'ANY' -version: '71.0' -tunnel: false # sets tunnel value to true for routing traffic through your local machine -``` - -**Capabilities Generator** -Use the TestMu AI [Capabilities Generator](https://www.testmuai.com/capabilities-generator/) to auto-generate the capabilities class for your test requirements. - -## Step 4: Run the Test - -Execute the Codeception test from the command line. - -```bash -./vendor/bin/codecept run --steps -``` - -## Step 5: View Your Results - -Check the test output on the console and the TestMu AI dashboard. - -Visit the [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) to view your test results. The dashboard provides: - -- Text logs for each test step -- Screenshots captured during execution -- Video recordings of the full test session - -## Run Codeception Tests Using Agent Skills - -Use AI coding assistants to generate and run Codeception tests with the TestMu AI Agent Skill. - -The [codeception-skill](https://github.com/LambdaTest/agent-skills/tree/main/codeception-skill) is part of [TestMu AI Agent Skills](https://github.com/LambdaTest/agent-skills/) - structured packages that teach AI coding assistants how to write production-grade test automation. - -Install the skill: - -```bash -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/codeception-skill .claude/skills/ - -# For Cursor / Copilot -cp -r agent-skills/codeception-skill .cursor/skills/ -``` - -Install all available framework skills at once by cloning the repository directly into your tool's skills directory (e.g., `.claude/skills/`, `.cursor/skills/`). diff --git a/static/docs/codeceptjs-with-selenium.md b/static/docs/codeceptjs-with-selenium.md deleted file mode 100644 index ad10a3485..000000000 --- a/static/docs/codeceptjs-with-selenium.md +++ /dev/null @@ -1,167 +0,0 @@ -# CodeceptJS With Selenium - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -CodeceptJS is a modern end-to-end testing framework with a human-readable syntax. It supports Selenium via the WebDriver helper, making it straightforward to run browser automation tests on TestMu AI cloud Selenium Grid across 3000+ browser and OS combinations. - -## Prerequisites - -1. You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** -Download or clone the CodeceptJS Selenium sample from the TestMu AI GitHub repository to run the tests. - - View on GitHub - -```bash -git clone https://github.com/LambdaTest/lambdatest-codeceptjs-sample.git -cd lambdatest-codeceptjs-sample -``` - -2. Install the dependencies. - -```bash -npm install -``` - -3. Get your TestMu AI username and access key from your TestMu AI Profile > Account Settings > Password & Security. - -4. Set your credentials as environment variables. - -**Windows** - -```bash -set LT_USERNAME="YOUR_LAMBDATEST_USERNAME" -set LT_ACCESS_KEY="YOUR_LAMBDATEST_ACCESS_KEY" -``` - -**macOS/Linux** - -```bash -export LT_USERNAME="YOUR_LAMBDATEST_USERNAME" -export LT_ACCESS_KEY="YOUR_LAMBDATEST_ACCESS_KEY" -``` - -## Configure CodeceptJS for TestMu AI - -The `codecept.conf.js` file connects your tests to the TestMu AI Selenium Grid using the `WebDriver` helper. Set `host` to `hub.lambdatest.com` and pass your credentials via environment variables. - -```js title="codecept.conf.js" -exports.config = { -tests: './*_test.js', -output: './output', - -helpers: { -WebDriver: { -url: 'http://google.com/ncr', -browser: 'chrome', -host: 'hub.lambdatest.com', -port: 80, -user: process.env.LT_USERNAME, -key: process.env.LT_ACCESS_KEY, - -desiredCapabilities: { -name: '[CodeceptJS] Automation Sample', -build: '[CodeceptJS] Automation Sample', -platformName: 'Windows 11', -browserName: 'Chrome', -browserVersion: 'dev' -} -}, - -LTHelper: { -require: 'codeceptjs-lambdatest-service', -user: process.env.LT_USERNAME, -key: process.env.LT_ACCESS_KEY, -updateTestName: true -} -}, - -include: { -I: './steps_file.js' -}, - -bootstrap: null, -mocha: {}, -name: 'CodeceptJS' -} -``` - -Use the **[TestMu AI Capabilities Generator](https://www.testmuai.com/capabilities-generator/)** to generate the `desiredCapabilities` object for any browser, OS, or device combination. - -## Use the codeceptjs-lambdatest-service - -The [codeceptjs-lambdatest-service](https://github.com/LambdaTest/codeceptjs-lambdatest-service) package is a CodeceptJS helper that automatically syncs test names and test results with TestMu AI after each test run. It uses CodeceptJS's built-in `_passed` and `_failed` hooks to push the outcome to the TestMu AI platform in real time. - -### Install the Package - -The package is included as a dev dependency in the sample repo. To add it to your own project: - -```bash -npm install codeceptjs-lambdatest-service --save-dev -``` - -### Add the LTHelper to Your Config - -Add the `LTHelper` block inside the `helpers` section of `codecept.conf.js`: - -```js -helpers: { -// ... your WebDriver helper config - -LTHelper: { -require: 'codeceptjs-lambdatest-service', -user: process.env.LT_USERNAME, -key: process.env.LT_ACCESS_KEY, -updateTestName: true -} -} -``` - -### Test Name Updates - -When `updateTestName: true` is set, the service reads the CodeceptJS scenario title and updates the test name on TestMu AI dynamically. This means each test in the TestMu AI Automation Dashboard reflects the exact scenario name from your test file instead of the default session name. - -For example, if your test is: - -```js title="googleTest_test.js" -Feature('GoogleTest'); - -Scenario('test something', async ({ I }) => { -await I.amOnPage('http://google.com/ncr'); -await I.seeInTitle('Google'); -}); -``` - -The session will appear on the dashboard as **test something** under the **GoogleTest** feature. - -### Test Status Updates - -The service uses the `_passed` and `_failed` hooks to mark each session as **Passed** or **Failed** on TestMu AI immediately after the test finishes. This is required for accurate status reporting on the dashboard - without it, all sessions show as **Unknown** regardless of the actual result. - -No additional configuration is needed beyond adding `LTHelper` to your helpers. - -## Run Your Tests - -**Run desktop tests:** - -```bash -npm test -# or -npm run test:desktop -``` - -**Run a specific test file:** - -```bash -npx codeceptjs run --steps googleTest_test.js -``` - -## View Test Results - -Visit the **[TestMu AI Web Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build)** to view your test results. Each test session shows: - -- Test name (synced from your scenario title via `codeceptjs-lambdatest-service`) -- Pass/fail status -- Video recording, screenshots, and network/console logs -- Build grouping based on the `build` capability diff --git a/static/docs/codefresh-ci-integration-with-testmu.md b/static/docs/codefresh-ci-integration-with-testmu.md deleted file mode 100644 index f8158f241..000000000 --- a/static/docs/codefresh-ci-integration-with-testmu.md +++ /dev/null @@ -1,159 +0,0 @@ -# Selenium Testing With Codefresh CI And TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Codefresh is a cloud-native continuous integration and delivery platform that enables teams to quickly and efficiently develop, deploy, and manage cloud-native applications. - -Teams can quickly and easily build, test, and deploy their applications on any cloud platform, including Kubernetes, Docker, and AWS. Our intuitive, easy-to-use UI helps streamline the development process. - -TestMu AI now integrates with Codeship to boost your go-to-market delivery. Perform automated cross browser testing with TestMu AI to ensure your development code renders seamlessly through an online [Selenium grid](https://www.testmuai.com/blog/why-selenium-grid-is-ideal-for-automated-browser-testing/) providing 3000+ real browsers running through machines on the cloud. Perform automation testing in parallel with TestMu AI’s Selenium grid to trim down your test cycles drastically. - -## Prerequisites - -1. Signing details of Codefresh CI -2. A GitHub repository. Here is our sample GitHub repository for [Python Selenium Sample](https://github.com/LambdaTest/python-selenium-sample.git). -3. **TestMu AI Authentication Credentials** - Be aware of your TestMu AI authentication credentials i.e. your TestMu AI username, access key and HubURL. You need to set them up as your environment variables. You can retrieve them from your [TestMu AI automation dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/) by clicking on the key icon near the help button. - -## Integrating Codefresh CI With TestMu AI - -**Step 1:** Sign-in into your codefresh ci account , and create a new pipeline - -**Step 2:** Select your user/git organisation, select repository and then click create. - -**Step 3:** After the creation of pipeline, you must see the workflow YAML got generated, with your selected github repository. - -**Step 4:** You will have option to choose your runtime image as per your requirements, or test scripts. - - Choose your runtime image - - Add your environment variables which needs to be passed to your testscripts. - - Add your execution command in the commands option of your pipeline YAML. - - As the selected repository has scripts which is made to run tests on TestMu AI Selenium Grid, you would need - set the env variables such as ```LT_USERNAME``` and ```LT_ACCESS_KEY``` - - Then to execute the pipeline, click the ```RUN``` button. - -You may take reference from this YAML to run your own test project/scripts -``` -version: "1.0" -stages: -- "clone" -- "build" -- "test" - -steps: -clone: -title: "Cloning repository" -type: "git-clone" -repo: "xyz/python-selenium-sample_2" -revision: "${{CF_BRANCH}}" -git: "github" -stage: "clone" - -test: -title: "Running test" -type: "freestyle" -image: "python:3.9" # Using Python image -working_directory: "${{clone}}" -environment: -- LT_USERNAME=xyz -- LT_ACCESS_KEY=xyz -commands: -- "pip3 install selenium" -- "python3 codefresh-sample.py" -stage: "test" - -``` - -You can see the pipeline running. - -**Step 4:** Login to the TestMu AI Dashboard, Navigate to Automation -> Web Automation. You can see your codefresh pipeline build has been executed on TestMu AI - -Similarly you can run test scripts based on any framework using this above described method. -Here is the selenium test script for your reference. - -``` -import unittest -import os -from selenium import webdriver -from selenium.webdriver.common.by import By -from selenium.webdriver.chrome.options import Options as ChromeOptions - -username = os.getenv("LT_USERNAME") # Replace the username -access_key = os.getenv("LT_ACCESS_KEY") # Replace the access key - -# paste your capability options below -options = ChromeOptions() -options.browser_version = "latest" -options.platform_name = "win10" -lt_options = {} -lt_options["username"] = username -lt_options["accessKey"] = access_key -lt_options["video"] = True -lt_options["resolution"] = "1920x1080" -lt_options["network"] = True -lt_options["build"] = "test_build" -lt_options["project"] = "unit_testing" -lt_options["name"] = "basic_unit_selinium" -lt_options["w3c"] = True -lt_options["plugin"] = "python-python" -options.set_capability("LT:Options", lt_options) - -class FirstSampleTest(unittest.TestCase): -driver = None - -def setUp(self): -self.driver = webdriver.Remote( -command_executor="http://{}:{}@hub.lambdatest.com/wd/hub".format( -username, access_key -), -options=options, -) - -# """ You can write the test cases here """ -def test_demo_site(self): -# try: -driver = self.driver -driver.implicitly_wait(10) -driver.set_page_load_timeout(30) -driver.set_window_size(1920, 1080) - -# Url -print("Loading URL") -driver.get( -"https://stage-lambda-devops-use-only.lambdatestinternal.com/To-do-app/index.html" -) - -# Let's click on a element -driver.find_element(By.NAME, "li1").click() -location = driver.find_element(By.NAME, "li2") -location.click() -print("Clicked on the second element") - -# Let's add a checkbox -driver.find_element(By.ID, "sampletodotext").send_keys("LambdaTest") -add_button = driver.find_element(By.ID, "addbutton") -add_button.click() -print("Added LambdaTest checkbox") - -# print the heading -search = driver.find_element(By.CSS_SELECTOR, ".container h2") -assert search.is_displayed(), "heading is not displayed" -print(search.text) -search.click() -driver.implicitly_wait(3) - -# Let's download the invoice -heading = driver.find_element(By.CSS_SELECTOR, ".container h2") -if heading.is_displayed(): -heading.click() -driver.execute_script("lambda-status=passed") -print("Tests are run successfully!") -else: -driver.execute_script("lambda-status=failed") - -# tearDown runs after each test case -def tearDown(self): -self.driver.quit() - -if __name__ == "__main__": -unittest.main() -``` diff --git a/static/docs/codeship-integration-with-testmu.md b/static/docs/codeship-integration-with-testmu.md deleted file mode 100644 index eaebfe060..000000000 --- a/static/docs/codeship-integration-with-testmu.md +++ /dev/null @@ -1,122 +0,0 @@ -# Selenium Testing With Concourse CI And TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Codeship is a scalable, cloud-based platform for continuous integration that offers GitHub, Bitbucket, and GitLab projects support. There are two versions of Codeship, Codeship Basic and Codeship Pro. Codeship Basic instantly provides robust VMs along with technology stacks being preinstalled to help run your CI/CD pipeline faster. Keep in mind, though, the primary purpose of Codeship Basic is to develop applications with general workflows by UI. It doesn’t offer support for dockers. - -TestMu AI now integrates with Codeship to boost your go-to-market delivery. Perform automated cross browser testing with TestMu AI to ensure your development code renders seamlessly through an online [Selenium grid](https://www.testmuai.com/blog/why-selenium-grid-is-ideal-for-automated-browser-testing/) providing 3000+ real browsers running through machines on the cloud. Perform automation testing in parallel with TestMu AI’s Selenium grid to trim down your test cycles drastically. - -## Prerequisites - -1. Signing details of Codeship -2. A GitHub repository. Here is our sample GitHub repository for [Codeship Basic](https://github.com/LambdaTest/protractor-selenium-codeship-basic). -3. **TestMu AI Authentication Credentials** - Be aware of your TestMu AI authentication credentials i.e. your TestMu AI username, access key and HubURL. You need to set them up as your environment variables. You can retrieve them from your [TestMu AI automation dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/) by clicking on the key icon near the help button. - - * **For Linux/Mac:** - -``` -$ export LT_USERNAME= `{YOUR_LAMBDATEST_USERNAME}` - -$ export LT_ACCESS_KEY= `{YOUR_LAMBDATEST_ACCESS_KEY}` -``` - - * **For Windows:** - -``` -$ set LT_USERNAME= `{YOUR_LAMBDATEST_USERNAME}` - -$ set LT_ACCESS_KEY= `{YOUR_LAMBDATEST_ACCESS_KEY}` -``` - -## Integrating Codeship With TestMu AI - -**Step 1:** Sign up on Codeship with your GitHub account, and create a **"New Project."** - -**Step 2:** Select GitHub as your SCM. - -**Step 3:** Provide your GitHub organization and your GitHub repository, then hit **Connect.** - -**Step 4:** Now, you need to opt for **"Codeship Basic"** as your project type by clicking on **"Select Basic Project."** - -**Step 5:** You will be redirected to a screen of **Project Settings.** Select the technology to prepopulate basic commands with respect to your tests. -For the below demonstration, we will be using Protractor, so you can select Node.js. Now, enter the commands below in setup. - -``` -#nvm install 0.10 -npm install -npm install -g protractor -``` - -**Step 6:** Create a new pipeline in the section **"Configure Test Pipelines."** Use the below command to connect it with your **GitHub repository** by replacing the **GitHubOrganization & reponame** with the input's values. - -``` -cd /home/rof/src/github.com///conf -protractor single.conf.js -``` - -**Step 7:** Provide your TestMu AI authentication credentials in the **Environment Variables** and hit the **"Save and go to Dashboard"** button. - -That is it! Now, any changes you commit in your GitHub repository will trigger the build in Codeship and run your test on TestMu AI Selenium Grid. - -## Testing Locally Hosted or Privately Hosted Projects - -To help you perform [cross browser testing](https://www.testmuai.com) of your locally stored web pages, TestMu AI provides an SSH(Secure Shell) tunnel connection with the name Lambda Tunnel. With Lambda Tunnel, you can execute a test server inside your GitLab CI build a container to perform automated cross browser testing on browsers offered by Selenium grid on TestMu AI. So you make sure how well your changes look, even before your customers. -Curious to know more about Lambda Tunnel? - ->Follow our documentation on Lambda Tunnel to know it all. OS-specific instructions to download and set up tunnel binary are at the following links. -> ->- [Documentation For Windows User](/docs/local-testing-for-windows/) ->- [Documentation For Mac User](/docs/local-testing-for-macos/) ->- [Documentation For Linux User](/docs/local-testing-for-linux/) - ->**Download the binary file of:** -> ->- [Lambda Tunnel for Windows](https://downloads.lambdatest.com/tunnel/v3/windows/64bit/LT_Windows.zip) ->- [Lambda Tunnel for Mac](https://downloads.lambdatest.com/tunnel/v3/mac/64bit/LT_Mac.zip) ->- [Lambda Tunnel for Linux](https://downloads.lambdatest.com/tunnel/v3/linux/64bit/LT_Linux.zip) - -The below example of **pipelinecommand** would demonstrate you to leverage TestMu AI Selenium Grid by performing cross browser testing through the test server contained in your Codeship build. -Create two pipelines with the following commands: - -**Pipeline 1** - -``` -#Downloading tunnel binary - -wget http://downloads.lambdatest.com/tunnel/linux/64bit/LT_Linux.zip - -wget http://downloads.lambdatest.com/tunnel/mac/64bit/LT_Mac.zip - -wget http://downloads.lambdatest.com/tunnel/windows/64bit/LT_Windows.zip -#Extracting tunnel binary -sudo apt-get install unzip -unzip LT_Linux.zip -#Executing tunnel binary -./LT -user -key -``` - -**Pipeline 2** - -``` -sleep 90 -cd /home/rof/src/github.com///conf -protractor local.conf.js -``` - -The setup command will be the same as mentioned earlier under the "Integration With Codeship" section. After then commit any changes into the repository and get your test run using TestMu AI Tunnel. - -## Parallel Testing - -Go to the **Project Settings** screen. Change your pipeline command with the following one & save settings. - -``` -cd /home/rof/src/github.com///conf -protractor parallel.conf.js -``` - -Now, any changes commit in your repository will trigger a build and run automation tests in parallel on TestMu AI Selenium Grid. - -Monitor and analyze your test result on the **[TestMu AI automation dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/).** - -Reliably deploy your code at scale using Codeship integration with TestMu AI, and ensure it looks robust across every browser to provide a seamless user experience to all your visitors. Happy Testing! diff --git a/static/docs/command-annotations.md b/static/docs/command-annotations.md deleted file mode 100644 index 119b029ee..000000000 --- a/static/docs/command-annotations.md +++ /dev/null @@ -1,32 +0,0 @@ -# Command Annotations - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Your test script holds important details about the test, like its description, when different scenarios start and finish, and other data you may want to show on the console for debugging and tracking purposes. - -By using command annotations, you can integrate configurations in your tests that log this information on TestMu AI. These logs are available on the TestMu AI Automation Dashboard alongside the command logs, providing a quick way to search and navigate to a specific test section and troubleshoot any failed tests. - -## Sending Logs to TestMu AI Using JavaScriptExecutor - -Use the JavaScriptExecutor to send annotations directly from your test script to the dashboard. - -You can send annotations to TestMu AI directly from your test script using the annotate action through the JavaScriptExecutor. - -Here is an example written in Java: - -```java -JavascriptExecutor jse = (JavascriptExecutor)driver; -jse.executeScript("lambdatest_executor: {\"action\": \"stepcontext\", \"arguments\": {\"data\": \"\", \"level\": \"\"}}"); -``` - -The arguments passed in the JavaScript method for setting the status and the corresponding reason for the test are `data` and `level`. - -* **data**: Accepts a value in string data type. - -* **level**: Accepts the standard log severity levels: info, debug, warn, and error. This argument is optional with the default value of info. - -## Searching and Filtering - -Locate and filter your annotated logs in the All Commands tab on the Automation Dashboard. - -Once your test script has sent command annotations to TestMu AI, you can locate all the annotations pushed to the logs in the **All Commands** tab on the TestMu AI Automation Dashboard. This search feature is especially useful for long-duration test sessions. Additionally, you can filter these annotated logs based on severity levels and customize the selection according to your logging patterns. diff --git a/static/docs/concurrency-widget.md b/static/docs/concurrency-widget.md deleted file mode 100644 index d9d96ddcd..000000000 --- a/static/docs/concurrency-widget.md +++ /dev/null @@ -1,118 +0,0 @@ -# Understanding Your TestMu AI Concurrency - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Your TestMu AI subscription is designed to be flexible, allowing you and your team to run different types of tests. This guide explains how your concurrency (the number of tests you can run in parallel) is managed. - -## Key Concepts Made Simple - -**Concurrency:** The number of tests you can run at the same time. If you have 10 parallels, you can run 10 tests simultaneously. These are determined by the sum of all subscriptions that have been purchased. - -**Plans (Your Subscriptions):** These are the packages you purchase, like "Web & Mobile Browser Automation on Real Device" or "Hyperexecute Linux" Each plan comes with a specific number of parallels. - -**Intents (Types of Tests):** These are the specific test types you can run, such as Realtime Testing, Desktop Automation, or Mobile Automation. - -## The Most Important Idea: The License Pool - -Think of each Plan you purchase as its own Pool of concurrency. - -All the different test types (Intents) included in that one Plan share the same pool. If you run any test from that Plan, it takes up one slot from the pool, reducing the available slots for all other tests within that same Plan. - -### Understanding the License Pool with an example - -Let's say that you buy a plan Web & Mobile Automation on Real Devices with 10 parallel tests, now all the types of tests that are included in this plan will share the same concurrency pool. - -Now lets say you run 1 web automation test, now the availability of the parallel tests throughout will be 9 (i.e. 10 - 1 = 9 that's now available for any type of test). - -## Decoding the Widget: A Detailed Look - -The widget gives you a real-time view of your license pool usage. Let's break it down using an example. Assuming the current plan consists of Web & Mobile Automation on Real Devices x10 - -### The Main View - -At a glance, you see a list of test types (Intents) and their usage, like: - -This means that for the license pool this test belongs to, 1 parallel is currently being used out of a total of 10. - -### The Expanded View: Running vs. Consumed By Other - -When you click the arrow to expand a section, you get a more detailed breakdown. This is where you can see exactly how your pool is being used. - -**Running:** This shows how many tests of that specific type are active. In the example, Running is 0, meaning no Mobile Browser Automation tests are active right now. - -**Consumed By Other:** This is the most important field for understanding your pool. It shows how many parallels are being used by: -1. Different test types that belong to the same plan. (Running at Group/Sub-Org Level) -2. Same test type run by users in other groups. -3. Same test type run by users in sub-org. - -In this case we are running a Web Automation Desktop Browser. - -**Available:** This is the final count of parallels left in the pool for you to use. - -### Putting it all together with the example: - -- The plan bought (Web & Mobile Automation on Real Devices x10) has a subscription of 10 parallels. -- The Consumed By Other field shows 1. This tells you that a different test from the same plan (in this case, Web Automation Desktop) is using one parallel. -- Since 0 tests of the current type (Mobile Browser Automation) are Running and 1 is Consumed By Other, the total usage for the pool is 1. -- This leaves 9 parallels Available, perfectly matching the 1/10 summary. - -## Scenarios Explained - -### Scenario 1: One Plan, Multiple Tests - -**Setup:** You have one subscription: the "Web & Mobile Automation on Real Devices" with 10 parallels. This plan includes Web Automation Desktop and Mobile Browser Automation etc. - -**Action:** Your team starts running 2 Web Automation Desktop tests. - -**Result on the Widget:** - -When you look at the widget, both sections will reflect this usage: -- Web Automation Desktop will show 2 Running and 8 Available. -- Mobile Browser Automation will show 0 Running, 2 Consumed By Other, and 8 Available. - -**Why?** Because both Intents pull from the same pool of 10. You have 8 parallels left for either test type. - -Web Automation Desktop Testing Widget - -Mobile Browser Automation Testing Widget - -### Scenario 2: The Widget on Different Pages - -**Setup:** You have the same "Web & Mobile Automation on Real Devices" with 10 parallels. - -**Action:** A colleague starts 2 web automation desktop tests. You are on the Real-time Testing page. - -**Result on Your Widget:** -- The Real-time Testing section on your page will show 2 tests as Consumed By Other. -- It will display that only 8/10 parallels are available, even though you haven't run any real-time tests yourself. The widget accurately reflects the total usage of the shared pool. - -### Scenario 3: Multiple Plans with Overlapping Tests - -This is where the system's flexibility shines. - -**Setup:** Your organization has two subscriptions: -- "Web & Mobile Automation on Real Devices" with 10 parallels, which includes Web Automation Desktop and Real Mobile Automation and more. -- "Hyperexecute (Linux Only)" with 5 parallels, which includes Web Automation Desktop and Hyperexecute Linux and more. - -**The Overlap:** Web Automation Desktop is in both plans, giving you a combined total of 15 parallels (10 + 5) specifically for web automation desktop testing. - -**Action:** Your team needs to run a large batch of web automation tests and start 12 of them. - -**Result on the Widget:** The system intelligently uses licenses from both pools to meet the demand. - -- **Web Automation Desktop (common in both plans)** - Example: Running 12 tests out of 15 total. - Shows 3 available slots left, regardless of the type of test you choose next. - -- **Real Mobile Automation (unique to Web & Mobile Automation on Real Devices plan)** - Available only in the Web & Mobile Automation on Real Devices plan. - Even if 12 Web Automation tests are running, the system still shows 3 available slots (15 – 12). - -- **HyperExecute (Linux) (unique to Hyperexecute Linux plan)** - Available only in the HyperExecute (Linux Only) plan. - The system again shows 3 available slots, since concurrency dynamically balances across plans. - -Now the system will intelligently decide your available concurrency based on the type of test that you want to run. -However, your total number of parallel tests is capped at 3 (regardless of type of test you want to run). - -**Why?** The system pools your licenses for any overlapping test types, giving you maximum capacity. However, using these pooled licenses will consume slots from each contributing plan, affecting the availability of other, non-overlapping tests in those plans. diff --git a/static/docs/configure-timezones.md b/static/docs/configure-timezones.md deleted file mode 100644 index 2903f68f9..000000000 --- a/static/docs/configure-timezones.md +++ /dev/null @@ -1,189 +0,0 @@ -# Configure Timezones for Your Tests - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Many applications display dates, schedule events, or trigger time-sensitive logic based on the user's timezone. TestMu AI lets you set the timezone on the test VM using the `timezone` capability in `LT:Options`, so you can validate timezone-dependent behavior without changing your local machine settings. The capability accepts UTC offset strings (e.g., `UTC+05:30`). - -## Set the Timezone Using the `timezone` Capability - -Pass a UTC offset string in `LT:Options` to configure the test VM's system timezone. - -Add the `timezone` capability to your `LT:Options` configuration. The value is a UTC offset string in the format `UTC+HH:MM` or `UTC-HH:MM` (e.g., `UTC+05:30`). The default is `UTC+00:00`. - -```java title="TimezoneTest.java" -import org.openqa.selenium.remote.RemoteWebDriver; -import org.openqa.selenium.chrome.ChromeOptions; -import java.net.URL; -import java.util.HashMap; - -public class TimezoneTest { -public static void main(String[] args) throws Exception { -ChromeOptions browserOptions = new ChromeOptions(); -browserOptions.setPlatformName("Windows 11"); -browserOptions.setBrowserVersion("latest"); - -HashMap ltOptions = new HashMap<>(); -ltOptions.put("username", System.getenv("LT_USERNAME")); -ltOptions.put("accessKey", System.getenv("LT_ACCESS_KEY")); -ltOptions.put("timezone", "UTC-05:00"); -ltOptions.put("project", "Timezone Test"); -browserOptions.setCapability("LT:Options", ltOptions); - -RemoteWebDriver driver = new RemoteWebDriver( -new URL("https://hub.lambdatest.com/wd/hub"), browserOptions); -driver.get("https://example.com"); -driver.quit(); -} -} -``` - -```javascript title="timezone-test.js" -const { Builder } = require("selenium-webdriver"); -const chrome = require("selenium-webdriver/chrome"); - -(async function timezoneTest() { -let options = new chrome.Options(); -options.setPlatform("Windows 11"); -options.setBrowserVersion("latest"); - -let ltOptions = { -username: process.env.LT_USERNAME, -accessKey: process.env.LT_ACCESS_KEY, -timezone: "UTC-05:00", -project: "Timezone Test", -}; - -const driver = await new Builder() -.usingServer("https://hub.lambdatest.com/wd/hub") -.withCapabilities({ ...options.toJSON(), "LT:Options": ltOptions }) -.build(); - -await driver.get("https://example.com"); -await driver.quit(); -})(); -``` - -```python title="timezone_test.py" -import os -from selenium import webdriver -from selenium.webdriver.chrome.options import Options - -options = Options() -options.platform_name = "Windows 11" -options.browser_version = "latest" - -lt_options = { -"username": os.environ["LT_USERNAME"], -"accessKey": os.environ["LT_ACCESS_KEY"], -"timezone": "UTC-05:00", -"project": "Timezone Test", -} -options.set_capability("LT:Options", lt_options) - -driver = webdriver.Remote( -command_executor="https://hub.lambdatest.com/wd/hub", -options=options, -) -driver.get("https://example.com") -driver.quit() -``` - -```csharp title="TimezoneTest.cs" -using OpenQA.Selenium; -using OpenQA.Selenium.Chrome; -using OpenQA.Selenium.Remote; - -ChromeOptions browserOptions = new ChromeOptions(); -browserOptions.PlatformName = "Windows 11"; -browserOptions.BrowserVersion = "latest"; - -Dictionary ltOptions = new Dictionary(); -ltOptions.Add("username", Environment.GetEnvironmentVariable("LT_USERNAME")); -ltOptions.Add("accessKey", Environment.GetEnvironmentVariable("LT_ACCESS_KEY")); -ltOptions.Add("timezone", "UTC-05:00"); -ltOptions.Add("project", "Timezone Test"); -browserOptions.AddAdditionalOption("LT:Options", ltOptions); - -IWebDriver driver = new RemoteWebDriver( -new Uri("https://hub.lambdatest.com/wd/hub"), browserOptions); -driver.Navigate().GoToUrl("https://example.com"); -driver.Quit(); -``` - -```php title="TimezoneTest.php" -setPlatform("Windows 11"); -$capabilities->setCapability("browserVersion", "latest"); - -$ltOptions = [ -"username" => getenv("LT_USERNAME"), -"accessKey" => getenv("LT_ACCESS_KEY"), -"timezone" => "UTC-05:00", -"project" => "Timezone Test", -]; -$capabilities->setCapability("LT:Options", $ltOptions); - -$driver = RemoteWebDriver::create( -"https://hub.lambdatest.com/wd/hub", $capabilities); -$driver->get("https://example.com"); -$driver->quit(); -``` - -```ruby title="timezone_test.rb" -require "selenium-webdriver" - -options = Selenium::WebDriver::Options.chrome( -platform_name: "Windows 11", -browser_version: "latest", -"LT:Options" => { -username: ENV["LT_USERNAME"], -accessKey: ENV["LT_ACCESS_KEY"], -timezone: "UTC-05:00", -project: "Timezone Test", -} -) - -driver = Selenium::WebDriver.for( -:remote, -url: "https://hub.lambdatest.com/wd/hub", -capabilities: options, -) -driver.navigate.to("https://example.com") -driver.quit -``` - -## Common Timezone Values - -Use these UTC offset strings with the `timezone` capability. - -| Value | Region | -|---|---| -| `UTC+00:00` | London, Coordinated Universal Time | -| `UTC-05:00` | New York, Washington D.C. (US Eastern) | -| `UTC-08:00` | Los Angeles, San Francisco (US Pacific) | -| `UTC+01:00` | Berlin, Paris, Amsterdam | -| `UTC+05:30` | Mumbai, Delhi, Bangalore | -| `UTC+09:00` | Tokyo, Osaka | -| `UTC+11:00` | Sydney, Melbourne | -| `UTC+08:00` | Singapore, Hong Kong, Perth | -| `UTC-03:00` | Sao Paulo, Buenos Aires | -| `UTC+03:00` | Moscow, Istanbul | - -The timezone capability is case sensitive. Use the exact format `UTC+HH:MM` or `UTC-HH:MM`. - -## When to Configure Timezones - -Set the timezone capability when your test validates time-sensitive application behavior. - -Consider setting the `timezone` capability in these scenarios: - -- **Scheduled events.** Verify that calendar events, reminders, or notifications fire at the correct local time. -- **Date formatting.** Confirm that dates and timestamps render in the expected format for a given locale. -- **Time-based business logic.** Test features like cutoff times, availability windows, or session expiration that depend on the user's timezone. -- **Cross-timezone consistency.** Run the same test suite across multiple timezones to catch discrepancies in time-dependent logic. - -Combine the `timezone` capability with [geolocation](/support/docs/selenium-geolocation-capabilities/) to simulate a complete locale environment for your tests. diff --git a/static/docs/configure-yaml-for-integrating-gocd-pipelines-with-selenium-grid-online.md b/static/docs/configure-yaml-for-integrating-gocd-pipelines-with-selenium-grid-online.md deleted file mode 100644 index 48122ad33..000000000 --- a/static/docs/configure-yaml-for-integrating-gocd-pipelines-with-selenium-grid-online.md +++ /dev/null @@ -1,28 +0,0 @@ -# Configure YAML For Integrating GoCD Pipelines With TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Through our previous document, you were able to integrate GoCD pipelines with TestMu AI, and executed your first Selenium testing script through GoCD on TestMu AI. However, in our previous chapter we ran the test by copying the YAML from the cloned [repository for GoCD](https://github.com/LambdaTest/nightwatch-gocd-sample). This topic will teach you how you can configure your own YAML through GoCD and then integrate your pipeline with our online Selenium Grid. - -> **Note:** If you are just getting started with GoCD then make sure to visit our documentation to [integrate TestMu AI with GoCD](/docs/gocd-integration-with-testmu/). - -## Configuring YAML through GoCD - -**Step 1:** Visit your GoCD dashboard and use Pipeline as a code. - -**Step 2:** By default, the configuration language will be set as YAML. Under Part 1, mention the link to our [GitHub repository for GoCD](https://github.com/LambdaTest/nightwatch-gocd-sample) and test the connection. - -**Step 3:** Now, in Part 2, make sure to label your Pipeline with a name of your choice. Here we have labelled it as Sample-GoCD-Pipeline. Similarly, in Part 3 for Stage details, provide a label for your Staging environment. - -**Step 4:** In Part 4 i.e. Job & Tasks, you need to specify a Job name as per your choice and enter the below commands. - -``` -npm install -./node_modules/.bin/nightwatch -e chrome -``` - -**Step 5:** Under Advanced setting, you need to specify your TestMu AI authentication credentials as the environment variables. - -**Step 6:** You will find the YAML configuration file generated over the right side panel. In rare cases, if it doesn’t then you need to select the configuration language to first JSON, and then select YAML again. Now, if you scroll to bottom you will find a section to "Add Your Pipeline as Code Definitions to Your SCM Repository". Click on the button to "Download Config". - -Now, you just need to execute the job like we did in the previous chapter when we [integrated GoCD with TestMu AI](/docs/gocd-integration-with-testmu/). That is all you need to do if you wish to configure a private YAML file before you run your Selenium testing scripts over TestMu AI Selenium Grid. diff --git a/static/docs/connect-to-session.md b/static/docs/connect-to-session.md deleted file mode 100644 index a8c990180..000000000 --- a/static/docs/connect-to-session.md +++ /dev/null @@ -1,339 +0,0 @@ -# Connect to a Session - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Connect to a cloud browser session and control it with Puppeteer, Playwright, or Selenium. All three adapters connect to the same TestMu AI Browser Cloud infrastructure - pick the one that matches your existing test framework. - -| | Puppeteer | Playwright | Selenium | -|---|---|---|---| -| **Connection** | WebSocket (CDP) | WebSocket (CDP) | HTTP (WebDriver) | -| **Stealth** | Best | Good | Coming Soon | -| **Return type** | `Browser` | `{ browser, context, page }` | `WebDriver` | -| **Humanized interactions** | `click`, `type` | `click`, `type`, `fill` | Coming Soon | -| **Auto-waiting** | Manual | Built-in | Manual | -| **Profile persistence** | Yes | Yes | Yes | - -**Our recommendation:** Start with **Puppeteer** for most agent use cases. It -has the best stealth support and the simplest return type. Switch to -**Playwright** if you need auto-waiting or `page.fill()`. Use **Selenium** if -you have an existing Selenium test suite you want to run on the cloud. - -## Before You Begin - -Make sure you have completed the following before connecting to a session: - -- **TestMu AI Browser SDK installed** - Run `npm install @testmuai/browser-cloud` if you have not already. -- **Session credentials configured** - Set your `LT_USERNAME` and `LT_ACCESS_KEY` environment variables. -- **Quickstart completed** - If this is your first time using TestMu AI Browser Cloud, [launch your first session](/support/docs/launch-first-session/) first. - -## Basic Usage - -### Puppeteer - -Most Puppeteer scripts start with `puppeteer.launch()` to launch a local -browser. With the TestMu AI Browser SDK, you replace that with -`client.puppeteer.connect()` to connect to a cloud browser instead. Your -subsequent Puppeteer calls work exactly the same as before. - -```typescript -import { Browser } from '@testmuai/browser-cloud'; - -const client = new Browser(); - -// 1. Create a cloud session -const session = await client.sessions.create({ -adapter: 'puppeteer', -lambdatestOptions: { -build: 'My Agent', -name: 'Puppeteer Session', -'LT:Options': { -username: process.env.LT_USERNAME, -accessKey: process.env.LT_ACCESS_KEY, -} -} -}); - -// 2. Connect - returns a standard Puppeteer Browser object -const browser = await client.puppeteer.connect(session); -const page = (await browser.pages())[0]; - -// 3. Use Puppeteer as normal -await page.goto('https://example.com'); -await page.screenshot({ path: 'screenshot.png' }); - -// 4. Clean up -await browser.close(); -await client.sessions.release(session.id); -``` - -The `browser` object returned by `client.puppeteer.connect()` is a standard -Puppeteer `Browser`. Use it exactly as you would with plain Puppeteer - all -existing Puppeteer knowledge applies. - -### Playwright - -Like Puppeteer, the main change is how you connect - replacing -`chromium.launch()` with `client.playwright.connect()`. - -Playwright requires **Node.js 18+**. If you see a version error, -upgrade with `nvm install 18 && nvm use 18`. - -```typescript -import { Browser } from '@testmuai/browser-cloud'; - -const client = new Browser(); - -const session = await client.sessions.create({ -adapter: 'playwright', -lambdatestOptions: { -build: 'My Agent', -name: 'Playwright Session', -'LT:Options': { -username: process.env.LT_USERNAME, -accessKey: process.env.LT_ACCESS_KEY, -} -} -}); - -// Returns browser, context, AND page - all three ready to use -const { browser, context, page } = await client.playwright.connect(session); - -await page.goto('https://example.com'); -await page.screenshot({ path: 'screenshot.png' }); - -await browser.close(); -await client.sessions.release(session.id); -``` - -Notice the key difference from Puppeteer: `client.playwright.connect()` returns -three objects - `browser`, `context`, and `page` - instead of just a browser. -These are standard Playwright objects. Use them exactly as you would with plain -Playwright. - -### Selenium - -Selenium connects to TestMu AI differently from Puppeteer and Playwright. -Instead of WebSocket, it uses the standard **WebDriver protocol over HTTP**, -connecting to TestMu AI's Selenium Hub. - -```typescript -import { Browser } from '@testmuai/browser-cloud'; - -const client = new Browser(); - -const session = await client.sessions.create({ -adapter: 'selenium', -lambdatestOptions: { -build: 'My Agent', -name: 'Selenium Session', -'LT:Options': { -username: process.env.LT_USERNAME, -accessKey: process.env.LT_ACCESS_KEY, -} -} -}); - -// Returns a standard Selenium WebDriver -const driver = await client.selenium.connect(session); - -await driver.get('https://example.com'); -const title = await driver.getTitle(); -console.log('Title:', title); - -await driver.quit(); -await client.sessions.release(session.id); -``` - -The `driver` object is a standard Selenium `WebDriver`. Use it exactly as you -would with plain `selenium-webdriver`. - -## What the SDK Does for You - -When you call `client.puppeteer.connect(session)`, the SDK handles several -things automatically based on your session configuration: - -1. **Stealth check.** If `stealthConfig` is present and `skipFingerprintInjection` is not set, the SDK connects via `puppeteer-extra` with the stealth plugin - which patches 15+ browser fingerprints automatically. - -2. **User-agent.** If `randomizeUserAgent` is enabled, a random realistic user-agent is set on the page. - -3. **Viewport.** If `randomizeViewport` is enabled, ±20px random jitter is added to the viewport dimensions. - -4. **Humanized interactions.** If `humanizeInteractions` is true, `page.click()` and `page.type()` are monkey-patched to add random delays that mimic human behavior. - -5. **Profile loading.** If `profileId` is set, saved cookies are loaded from disk. And when you call `browser.close()`, the profile is automatically saved with the current cookies. - -When you call `client.playwright.connect(session)`: - -1. **Connects** to TestMu AI cloud via `chromium.connect` -2. **Gets or creates** a `BrowserContext` and `Page` -3. **Injects stealth scripts** (if enabled) via `page.addInitScript()` - these run before any page JavaScript: - - Hides `navigator.webdriver` - - Fakes `chrome.runtime` (simulates extensions presence) - - Fakes `navigator.plugins` (3 standard Chrome plugins) - - Sets `navigator.languages = ['en-US', 'en']` - - Patches `permissions.query` for notifications - - Spoofs WebGL vendor/renderer -4. **Auto-applies stealth** to new pages via `context.on('page')` -5. **Patches interactions** (if humanize enabled) - `page.click()`, `page.type()`, and `page.fill()` get random delays -6. **Loads/saves profile** (if `profileId` set) - -When you call `client.selenium.connect(session)`: - -1. **Connects** to the TestMu AI Selenium Hub at `https://hub.lambdatest.com/wd/hub` via HTTP -2. **Reads** `LT_USERNAME` and `LT_ACCESS_KEY` from your environment variables -3. **Builds** W3C capabilities from your session config -4. **Connects** over the standard WebDriver protocol - -The Selenium adapter **ignores** `session.websocketUrl` and builds its own connection. - -## Adding Session Features - -```typescript -const session = await client.sessions.create({ -adapter: 'puppeteer', // or 'playwright' or 'selenium' -stealthConfig: { // Anti-bot detection -humanizeInteractions: true, -randomizeUserAgent: true, -}, -profileId: 'my-app-login', // Persist auth state -tunnel: true, // Access localhost -timeout: 600000, // 10-minute timeout -lambdatestOptions: { ... } -}); -``` - -## Full Working Example - -A complete script that creates a session, scrapes a page title, and cleans up -with proper error handling: - -```typescript -import { Browser } from '@testmuai/browser-cloud'; - -const client = new Browser(); - -async function main() { -const session = await client.sessions.create({ -adapter: 'puppeteer', -lambdatestOptions: { -build: 'Agent Scripts', -name: 'Scrape Example', -'LT:Options': { -username: process.env.LT_USERNAME, -accessKey: process.env.LT_ACCESS_KEY, -} -} -}); - -console.log(`View session: ${session.sessionViewerUrl}`); - -try { -const browser = await client.puppeteer.connect(session); -const page = (await browser.pages())[0]; - -await page.goto('https://news.ycombinator.com'); -const title = await page.title(); -console.log('Page title:', title); - -await browser.close(); -} finally { -await client.sessions.release(session.id); -} -} - -main().catch(console.error); -``` - -A complete script that creates a session, scrapes a page title, and cleans up -with proper error handling: - -```typescript -import { Browser } from '@testmuai/browser-cloud'; - -const client = new Browser(); - -async function main() { -const session = await client.sessions.create({ -adapter: 'playwright', -lambdatestOptions: { -build: 'Agent Scripts', -name: 'Scrape Example', -'LT:Options': { -username: process.env.LT_USERNAME, -accessKey: process.env.LT_ACCESS_KEY, -} -} -}); - -console.log(`View session: ${session.sessionViewerUrl}`); - -try { -const { browser, context, page } = await client.playwright.connect(session); - -await page.goto('https://news.ycombinator.com'); -const title = await page.title(); -console.log('Page title:', title); - -await browser.close(); -} finally { -await client.sessions.release(session.id); -} -} - -main().catch(console.error); -``` - -A complete script that creates a session, scrapes a page title, and cleans up -with proper error handling: - -```typescript -import { Browser } from '@testmuai/browser-cloud'; - -const client = new Browser(); - -async function main() { -const session = await client.sessions.create({ -adapter: 'selenium', -lambdatestOptions: { -build: 'Agent Scripts', -name: 'Scrape Example', -'LT:Options': { -username: process.env.LT_USERNAME, -accessKey: process.env.LT_ACCESS_KEY, -} -} -}); - -console.log(`View session: ${session.sessionViewerUrl}`); - -try { -const driver = await client.selenium.connect(session); - -await driver.get('https://news.ycombinator.com'); -const title = await driver.getTitle(); -console.log('Page title:', title); - -await driver.quit(); -} finally { -await client.sessions.release(session.id); -} -} - -main().catch(console.error); -``` - -Sessions remain active until explicitly released or timed out. -Always call `client.sessions.release()` when finished instead of waiting for -the timeout. - -## When to Choose - -| Use Case | Recommended | Why | -|----------|-------------|-----| -| General agent automation | Puppeteer | Mature, well-documented, large ecosystem | -| Stealth-heavy tasks | Puppeteer | Best stealth plugin support (`puppeteer-extra-plugin-stealth`) | -| Complex form interactions | Playwright | Built-in `page.fill()` method | -| Auto-waiting | Playwright | Automatically waits for elements before interacting | -| Multi-page workflows | Playwright | Better context management for complex navigation | -| Existing Selenium test suite | Selenium | Minimal migration - same WebDriver API | -| Simple return type | Puppeteer | Returns a single `Browser` object (vs Playwright's 3 objects) | diff --git a/static/docs/contacts-on-real-devices-browser.md b/static/docs/contacts-on-real-devices-browser.md deleted file mode 100644 index 2fca24ddc..000000000 --- a/static/docs/contacts-on-real-devices-browser.md +++ /dev/null @@ -1,18 +0,0 @@ -# Access your Contacts on Real Devices (Browser) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -With TestMu AI Real Devices, you can interact with the built-in Contacts app on our real devices to test your app's functionality. To support diverse testing, we've included sample contact data for you to use. - -## Supported Devices -TestMu AI supports access to the dummy Contacts on the following devices: - -- Devices running Android 10 and higher(Except Tabs). -- Devices running iOS 13 and higher. - -## What you can test? -With access to the Contacts app on the remote devices, you can now test scenarios, such as: - -- Access contacts from the Contacts app. -- Add/Save your custom contacts to the Contacts app. -- Edit/delete a contact. diff --git a/static/docs/contacts-on-real-devices.md b/static/docs/contacts-on-real-devices.md deleted file mode 100644 index 37e94936b..000000000 --- a/static/docs/contacts-on-real-devices.md +++ /dev/null @@ -1,18 +0,0 @@ -# Access your Contacts - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -With TestMu AI Real Devices, you can interact with the built-in Contacts app on our real devices to test your app's functionality. To support diverse testing, we've included sample contact data for you to use. - -## Supported Devices -TestMu AI supports access to the dummy Contacts on the following devices: - -- Devices running Android 10 and higher(Except Tabs). -- Devices running iOS 13 and higher. - -## What you can test? -With access to the Contacts app on the remote devices, you can now test scenarios, such as: - -- Access contacts from the Contacts app. -- Add/Save your custom contacts to the Contacts app. -- Edit/delete a contact. diff --git a/static/docs/copy-and-move-support-for-test-cases.md b/static/docs/copy-and-move-support-for-test-cases.md deleted file mode 100644 index fec90053f..000000000 --- a/static/docs/copy-and-move-support-for-test-cases.md +++ /dev/null @@ -1,41 +0,0 @@ -# Copy and Move Support for Test Cases - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The Copy and Move feature for test cases helps streamline the organization of test cases. It allows you to easily transfer test cases between different folders or projects, making it simple to keep everything organized. This feature enhances team collaboration and ensures that test cases are always accessible where needed. - -**How to Copy Multiple Test Cases to other Projects & Folders** - -1. Navigate to the list of test cases in your project. Multi-select the test cases you wish to move or copy to specific folders. Click on the **Bulk Actions** button at the top right of the screen. In the dropdown menu, select **Copy**. - - You can also **select all test cases** in one step by clicking on the top most Tickbox, highlighted below : - -2. In the dropdown menu, select **Project** and **Target Folder** of your choice where you want to copy the selected test cases. - - Test Cases can be Copied/Moved between Projects. - -3. You also have the option to create a new folder or a sub-folder within the main folder by clicking on **Add Folder**. - -4. Add the Name of the Folder to be created. - -5. After selecting or creating the target folder, click on the **Copy** button to complete the process. - -5. A notification will appear at the top right of the screen confirming the test cases have been copied. Click **View Copied Test Cases** to go directly to the new folder and verify the copied test cases. - -6. The test cases will now be displayed in the selected folder. In this case, we have created a subfolder named **Register** under **Services page** folder, the test cases will be reflected in that location. - -**How to Duplicate a Single Test Case** - -You can also duplicate an individual test case directly from the test case list without using Bulk Actions. - -1. Hover over the test case you want to duplicate and click the three-dot menu icon on the right. - -2. Select **Duplicate Test Case** from the dropdown. The test case will be copied following the same flow as bulk copy. - -**How to Move Multiple Test Cases to other Projects & Folders** - -1. Select multiple test cases of your choice, Click on the **Bulk Actions** button located at the top right of the screen. From the dropdown menu, choose **Move**. - -2. In the next dropdown menu, select **Project** and **Target Folder** of your choice and click on **Move Test Cases**. - -3. A notification will appear at the top right confirming the test cases have been moved. Click on **View Moved Test Cases** to navigate to the new folder. diff --git a/static/docs/create-modules.md b/static/docs/create-modules.md deleted file mode 100644 index 66e16ed66..000000000 --- a/static/docs/create-modules.md +++ /dev/null @@ -1,56 +0,0 @@ -# Modules - Creation and Management - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Tired of recreating the same test steps for repetitive testing flows? With Modules in TestMu AI Test Manager you can streamline your workflow by reusing pre-existing test steps across multiple test cases. This powerful feature enables you to encapsulate redundant testing flows into modular, reusable components, saving time and ensuring consistency across your projects. - -## 1.1 Module Creation from existing Test Steps - -To export test steps as a module, -- Navigate to the Test Steps page of your desired Test Case. -- Select the test steps you wish to include in the module and click the `Create a module` button. - - Only **sequential** steps can be clubbed to create a module. - -- Clearly define your new module by entering a concise name and a detailed description. This will help you easily identify and understand its purpose later on. - -## 1.2 Create Module from Modules section - -### Step 1: Go to Modules page - -- Click on `Modules` button in the navbar to go to the [Modules listing page](https://www.testmuai.com/login/?redirectTo=https://test-manager.lambdatest.com/module). -- Click on `Create a Module`. - - When you duplicate a module, its steps, description, and linked projects will also be duplicated. - -### Step 2: Create New Module - -- Click on the `Create a Module` button. -- Define your module and outline the steps you want to include. - -Don't worry if you don't have all the details now; you can always edit or add them later. - - You must provide **Module name** & create at **least one test step** to be able to create a new module. - - Keep in mind Modules can't be deleted but only updated after creation. - -## 2. Update Module -Modules support duplication, editing and versioning; however, a new version is only triggered when specific fields are edited. -- Add a step by clicking on `Add Step` or update a step through edit icon. -- Click on `update` on top right to save changes. - - Only **Test Step** or **Expected Outcome** updation will create a new version of the module. - -## 3. Module Versions - -### Revert to previous Version -- Click on `revert` to revert your module to that version. Reverting to a previous version will create a new version in your module's history, ensuring a clear audit trail of all changes. - -### Compare Versions -Compare your version history side-by-side to easily identify differences and track changes between various iterations of your module. - -- Click on `Compare` & select the version to compare with. - -You will be redirected to the Version Comparison page where you can track changes: - -Project linking is only added for manual tests in test cases. Hence, modules linked to a particular project are only available to be added in a manual test case. However, for KaneAI test cases, all modules in all projects will be available for selection. diff --git a/static/docs/create-projects.md b/static/docs/create-projects.md deleted file mode 100644 index 549943ff3..000000000 --- a/static/docs/create-projects.md +++ /dev/null @@ -1,15 +0,0 @@ -# Create Projects - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Test cases play a pivotal role in ensuring the systematic execution of testing activities. Below are key aspects related to test cases, including creating projects, importing test cases, creating manual test cases, and managing them through the test case repository. - -Creating projects within Test Manager is the foundational step in organizing and managing your testing activities. Projects serve as containers for your test cases, allowing you to categorize and structure your testing efforts effectively. - -**Steps to Create a Project** - -1. Log in to the Test Manager and navigate to the dashboard - [http://test-manager.lambdatest.com/](https://www.testmuai.com/login/?redirectTo=http://test-manager.lambdatest.com/) - -2. Click on the **Create Project** button. - -3. Enter the **Project name**, **Description**, and **Tag(s)**. Be descriptive to ensure clarity for all team members. Click on **Create** to complete the project creation. diff --git a/static/docs/credits-management.md b/static/docs/credits-management.md deleted file mode 100644 index 7ce60cd27..000000000 --- a/static/docs/credits-management.md +++ /dev/null @@ -1,130 +0,0 @@ -# Credits Usage & Management in TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI supports a **credit management system** as an add‑on over your active subscriptions for TestMu AI products. Many products include additional **AI features** that are powered by credits. This guide explains how credits work, how to view usage, and how admins can control consumption. - -## Who gets credits? - -* **Free users:** Receive **1,000 complimentary credits** to try AI‑powered features. -* **Paid users:** Receive **complimentary credits** based on subscription to try AI‑powered features and can purchase credits as an add‑on at any time. Credits are used only when AI features are invoked. - -Once free credits are exhausted, AI features will be unavailable until you **upgrade or purchase credits** from the Billing page. Use this link to buy credits directly: [Upgrade / Buy Credits](https://billing.lambdatest.com/billing/subscriptions?addCredits=true). - -## View your current credit balance - -If you are an **admin** in TestMu AI organization, you can see the current credits and subscription details in [Billing & Subscriptions](https://billing.lambdatest.com/billing/subscriptions). - -## View credit transactions - -**Admins** can view detailed **debits and credits** for your organization - **[Transactions view](https://billing.lambdatest.com/billing/subscriptions?viewCredits=true)**. - -The transactions table shows: - -* **Date** of the event -* **Type** (Debit / Credit) -* **Credits** (Amount of credits) -* **Name** of the user who did the transaction - -## Set credit usage limits (Admins Only) - -Admins can define **org‑wide controls** to prevent unexpected consumption: - -* **Soft Limit** - When the soft limit is reached, users are **notified via email**, but usage **continues**. - -* **Hard Limit** - When the hard limit is reached, **no further credit consumption** is allowed. Users are notified via email. To resume, increase the limit or purchase more credits. - -**Recommendation:** Start with a soft limit (e.g., 70–80% of your monthly plan) and a hard limit (e.g., 90%). - -## AI features that consume credits - -Below are the currently supported features and how they consume credits. - -### SmartUI Visual AI - -TestMu AI **SmartUI Visual AI** simulates human perception for visual regression. Rather than flagging every pixel change, it highlights **meaningful, human‑relevant** differences between baseline and new screenshots. [Learn more](/support/docs/smartui-visual-ai/). - -#### How credits are consumed? - -**Small / Normal Screenshot** - -* A page or screen that fits within a typical device viewport (e.g., one laptop or mobile screen) -* Examples: login page, product details, simple dashboard -* **Consumes \~1–4 credits** (most common) - -**Large Screenshot** -* A long or content‑heavy page beyond the standard viewport (full‑page scroll, data‑heavy dashboards) -* Examples: full home page, long reports, multi‑section dashboards -* **Consumes \~5–8 credits** (less common) - -**In summary:** The bigger and more content‑heavy the screenshot, the more credits it consumes. - -### AI Test Case Generator - -The **AI Test Case Generator** converts diverse inputs (text, PDFs, audio, videos, images, Jira tickets, and more) into **structured, contextual test cases**, accelerating authoring while improving coverage. [Learn more](/support/docs/generate-test-cases-with-ai/). - -#### How credits are consumed - -* **10 credits per scenario** generated. -* When you start generation, credits are tentatively consumed based on **Max Scenarios** you select (default **5**). - Example: Default 5 → **50 credits** tentatively consumed when you begin. -* If fewer scenarios are produced (e.g., **4**), **10 credits are credited back**, so you only pay for what’s actually generated. -* **Regeneration** also consumes credits at **10 credits per scenario**. - ->This ensures fair usage. You are charged only for **actual output** produced by the AI. - -#### Conversational Layer - -The AI Test Case Generator includes a **Conversational Layer** that lets you refine generated scenarios and test cases through natural‑language messages. Credits are consumed based on the scope of each refinement: - -| Action | Credit Cost | -|--------|-------------| -| Edit or modify a **scenario** (including its test cases) | **5 credits** | -| Edit or modify a single **test case** | **1 credit** | - -* Credits are deducted **per message** based on the scope of the change. -* If your message affects multiple scenarios, credits are deducted for each scenario impacted. - -## FAQs - -**Q: What happens when my credits run out?** - -AI features that require credits will be disabled. You can **purchase credits** or **upgrade** [here](https://billing.lambdatest.com/billing/subscriptions?addCredits=true). - -**Q: Who can view balances and transactions?** - -Users with the **Admin** role in your TestMu AI organization. - -**Q: Do unused credits expire?** - -For free users, complimentary credits are provided for one-time usage and do not expire. While for users subscribed to TestMu AI products, the complimentary credits get reset at the beginning of each month. Any credits explicitly purchased, do not expire. - -**Q: Why were credits refunded after generation?** - -When fewer scenarios than the selected Max Scenarios are generated, the difference is **credited back automatically**. - -**Q: Can I cap my organization’s usage?** - -Yes, set **Soft** and **Hard** limits (Admins only). - -**Q: Does the platform offer AI FinOps tools to control and manage AI testing costs?** - -Yes, it provides AI FinOps capabilities for cost control and spend management, enabling organizations to monitor, manage, and optimize costs across AI-driven testing workflows. This includes pre-execution credit controls, operation-level cost tracking, and detailed cost analytics to prevent overspending and ensure financial visibility. - -**Key capabilities include:** - -- Credit reservation system with pre-operation credit checks, reservation IDs per operation, and automatic blocking when credits are insufficient. - -- Cost tracking per operation type, including phone scenario generation, phone call execution, chat scenario generation, chat evaluation, and image analysis. - -- Credit balance management with real-time credit monitoring, consumption tracking, and organization-level cost summaries. - -- Cost metrics and analytics covering operation-level breakdowns, project-level aggregation, and historical cost analysis. - -## What’s next - -Additional AI features will join the credit system shortly. Watch the **TestMu AI Changelog** and Support Docs for updates. - -If you have questions about credits, billing, or limits, reach out to **[support@testmuai.com](mailto:support@testmuai.com)**. diff --git a/static/docs/csharp-with-playwright.md b/static/docs/csharp-with-playwright.md deleted file mode 100644 index cfc11a15f..000000000 --- a/static/docs/csharp-with-playwright.md +++ /dev/null @@ -1,118 +0,0 @@ -# C# with Playwright: Running Your First Test - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Learn how to use Playwright with C# to automate web application testing across 50+ real browsers and operating systems on TestMu AI cloud platform. - -## Prerequisites - -1. You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** -Download or clone the code sample for the Playwright C# from the TestMu AI GitHub repository to run the tests. - - View on GitHub - -```js -git clone https://github.com/LambdaTest/playwright-sample.git -cd playwright-sample -cd playwright-csharp -``` - -2. Restore the .NET dependencies. - -``` -dotnet restore -``` - -3. A TestMu AI Username and Access key. You can get it from your TestMu AI Profile section. Don't have an account, sign up for free. - -4. To run Playwright tests, set your TestMu AI Username and Access key in the Environment Variables. - -## Run your Playwright tests with C# - -Navigate to the `PlaywrightTestSingle.cs` file in the `playwright-csharp` directory. - -```csharp -using Microsoft.Playwright; -using System.Threading.Tasks; -using System; -using System.Collections.Generic; -using Newtonsoft.Json; - -class PlaywrightTestSingle -{ -public static async Task main(string[] args) -{ -using var playwright = await Playwright.CreateAsync(); - -string user, accessKey; -user = Environment.GetEnvironmentVariable("LT_USERNAME"); -accessKey = Environment.GetEnvironmentVariable("LT_ACCESS_KEY"); - -Dictionary capabilities = new Dictionary(); -Dictionary ltOptions = new Dictionary(); - -ltOptions.Add("name", "Playwright Test"); -ltOptions.Add("build", "Playwright C-Sharp tests"); -ltOptions.Add("platform", "Windows 10"); -ltOptions.Add("user", user); -ltOptions.Add("accessKey", accessKey); - -capabilities.Add("browserName", "Chrome"); -capabilities.Add("browserVersion", "latest"); -capabilities.Add("LT:Options", ltOptions); - -string capabilitiesJson = JsonConvert.SerializeObject(capabilities); - -string cdpUrl = "wss://cdp.lambdatest.com/playwright?capabilities=" + Uri.EscapeDataString(capabilitiesJson); - -await using var browser = await playwright.Chromium.ConnectAsync(cdpUrl); -var page = await browser.NewPageAsync(); -try { -await page.GotoAsync("https://www.bing.com"); -await page.Locator("[aria-label='Enter your search term']").ClickAsync(); -await page.FillAsync("[aria-label='Enter your search term']", "LambdaTest"); -await page.Keyboard.PressAsync("Enter"); -var title = await page.TitleAsync(); - -if (title.Contains("LambdaTest")) -{ -// Use the following code to mark the test status. -await SetTestStatus("passed", "Title matched", page); -} -else { -await SetTestStatus("failed", "Title not matched", page); -} -} -catch (Exception err) { -await SetTestStatus("failed", err.Message, page); -} -await browser.CloseAsync(); -} - -public static async Task SetTestStatus(string status, string remark, IPage page) { -await page.EvaluateAsync("_ => {}", "lambdatest_action: {\"action\": \"setTestStatus\", \"arguments\": {\"status\":\"" + status + "\", \"remark\": \"" + remark + "\"}}"); -} -} -``` - -1. Pass the below command in the terminal: - -```js -dotnet build -``` - -2. To run the test, pass the below command: - -```js -dotnet run single -``` - -**Note**: To run sessions on custom devices with emulation, run `dotnet run iphonetest` or `dotnet run ipadtest`. - -## View your test results - -Go to the [TestMu AI Web Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) to see your Playwright C# test results. - -To generate Playwright tests with AI coding assistants, see [Run Tests With Agent Skills](/support/docs/playwright-agent-skills/). diff --git a/static/docs/csv-import.md b/static/docs/csv-import.md deleted file mode 100644 index dd61ccf6e..000000000 --- a/static/docs/csv-import.md +++ /dev/null @@ -1,54 +0,0 @@ -# Import Test Cases from CSV - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -CSV import in Test Manager allows seamless import of existing test cases from various sources, making the transition and consolidation of testing assets effortless. - -## Steps for Importing Test Cases: -Follow these steps to import test cases using a CSV file: - -### 1. Upload CSV file -- **Choose Project:** Select the project into which you want to import the test cases. -- **CSV Upload Option:** If you have created a new project, you’ll see the **Drop your CSV or browse** option. Alternatively, navigate to ` Add Test Case > Import CSV ` to initiate the import. - -- **File Preview:** After selecting your CSV file, a preview will appear displaying the contents of the file to be imported. -- **Next Step:** Click Next to proceed to the Field Mapping stage. - -### 2. Map fields -In this step, you will map the CSV file headers to corresponding fields in Test Manager. This enables Test Manager to understand where each piece of information from the CSV should be placed. -Following entities can be mapped & imported: -- **Test Case Title:** Using ` Title ` option -- **Test Case Steps:** Using ` Steps ` option -- **Test Case Step Outcomes:** Using ` Outcome ` option -- **BDD Scenarios** -- **System Fields** -- **Custom Fields** -- **Folder Names:** Using ` Folder ` option -- **Jira Tickets:** Using ` Link Jira ` option - -#### Prerequisites: -- To map Custom Fields, ensure the custom fields are created and linked to the project before beginning the import process. -- To import Jira Tickets, ensure your Jira instance is connected to Test Manager (refer [here](/support/docs/link-jira-issues-with-test-manager/)). If you forget to connect it before starting the import, don't worry, the test cases will automatically link to the appropriate Jira tickets once the integration is set up. - -**Following format of Test Cases are supported for Importing:** -1. **Test cases use a single row:** Use this option when your steps, outcomes, or BDDs are combined in one row per test case & indexed like: -``` 1. Got to lambdatest.com 2. click on login button ``` -2. **Test cases use a multiple row:** Use this option when your steps, outcomes, or BDDs are present in separate rows. - -Mapping at least one CSV header with the **title** field is mandatory in this step, as the **title** serves as the unique identifier for your test cases. - -### 3. Map values -Once you've mapped the CSV headers to Test Manager fields, it's time to map the CSV field values to the selected fields. - -- **System Fields Values:** Existing values will be automatically mapped. For any new values, you can map them to existing values or create new values using the ` Create new value ` option. -- **Custom Fields Values:** Values will be automatically generated and applied. -- **Folder Names:** If your CSV includes folder paths, you can select the appropriate delimiter to split the folder path when parsing. For example, use ` / ` to split the folder path ` "folder1/folder2/folder3" `. - -### 4. Preview import -- **Preview:** Before finalizing the import, you’ll have a chance to preview the test cases and the mapped CSV values. This allows you to ensure everything is mapped correctly. - -- **Import Test Cases:** After verifying the mappings, click ` Import Test Cases ` to complete the import process. - -By default the Test Cases not having any folder defined for them, go into the **Untitled** folder. - -Let's see the imported data: diff --git a/static/docs/cucumber-on-hyperexecute-grid.md b/static/docs/cucumber-on-hyperexecute-grid.md deleted file mode 100644 index 5f9278eea..000000000 --- a/static/docs/cucumber-on-hyperexecute-grid.md +++ /dev/null @@ -1,244 +0,0 @@ -# Running Cucumber Framework Tests on HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Cucumber is a widely-used testing framework for Java applications, designed to simplify and enhance the testing process for developers. It provides a flexible and powerful platform for running test suites, enabling effective unit testing, integration testing, and end-to-end testing of Java applications - -HyperExecute is an AI Native Test Orchestration Cloud Platform that empowers you to run **end-to-end** tests **quickly** and **efficiently**. It provides Just-in-Time (JIT) testing infrastructure with fast execution **speeds**, **smart orchestration**, and **detailed logs**. - -This guide details how to execute your **Cucumber** framework tests on **HyperExecute** via two different methods: - -- [**Using Local System**](/support/docs/cucumber-on-hyperexecute-grid/#1-testing-using-local-system) - You can use your own local machine to execute tests. -- [**Using Gitpod Platform**](/support/docs/cucumber-on-hyperexecute-grid/#2-testing-using-gitpod) - Execute tests using GitPod. (Requires a [Gitpod](https://gitpod.io/login/) account) - -## 1. Testing Using Local System - -Follow the step-by-step guide to execute your test on HyperExecute. - -### Prerequisites - -To run the Tests on HyperExecute from your Local System, you are required: - -- Your TestMu AI [Username and Access key](/support/docs/hyperexecute-how-to-get-my-username-and-access-key/) -- [HyperExecute YAML](/support/docs/hyperexecute-yaml-version0.2/) file which contains all the necessary instructions. -- [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) in order to initiate a test execution Job . -- Setup the [Environmental Variable](/support/docs/hyperexecute-environment-variable-setup/) - -### Step 1: Configure Your Test Suite - -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** - -Download or Clone the code sample for the JUnit from the TestMu AI GitHub repository to run the tests on the HyperExecute. - - View on GitHub - -If you are using your own project, make sure you update the **Hub endpoint** in your tests file. - -By setting up the Hub endpoint, you establish the communication channel between your tests and the browser nodes, enabling effective test distribution and execution. - -Configure the desired capabilities based on your test requirements. For example: - -```bash -DesiredCapabilities capability = new DesiredCapabilities(); -capability.setCapability(CapabilityType.BROWSER_NAME, browser); -capability.setCapability(CapabilityType.VERSION,version); -capability.setCapability(CapabilityType.PLATFORM, platform); -capability.setCapability("build", "Your Build Name"); -``` - -> You can generate capabilities for your test requirements with the help of our inbuilt 🔗 Capabilities Generator. - -### Step 2: Setup the CLI in your Test Suite - -After cloning / downloading the sample repo, you need to setup the CLI and the environment variables. - -#### Download the HyperExecute CLI - -The CLI is used for triggering the tests on HyperExecute. It is recommend to download the CLI binary on the host system and keep it in the root directory of the suite to perform the tests on HyperExecute. - -You can download the CLI for your desired platform from the below mentioned links: - -| Platform | HyperExecute CLI | -| ---------| ---------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| MacOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -#### Setup Environment Variable - -Now, you need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in the [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). - -Run the below mentioned commands in your terminal to setup the CLI and the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -### Step 3: Configure YAML in your Test Suite - -Configure your YAML file as per your use cases using **key value** pairs. - -In this sample YAML file, we have mentioned: - -- **version** of the YAML file -- **Timeouts** for executing your project -- **Mode of execution** is [Autosplit](/support/docs/hyperexecute-auto-split-strategy/). You can also opt for [Matrix](/support/docs/hyperexecute-matrix-multiplexing-strategy/) or [Hybrid](/support/docs/hyperexecute-hybrid-strategy/) mode. -- **Pre and Post** commands -- **Reports and Artefacts** that will be generated after the completion of tests -- and other necessary YAML Parameters - -```yaml ---- -version: 0.1 -globalTimeout: 90 -testSuiteTimeout: 90 -testSuiteStep: 90 - -runson: linux - -autosplit: true -retryOnFailure: true - -maxRetries: 1 -concurrency: 4 - -env: -# PAT: ${{ .secrets.testKey }} -CACHE_DIR: m2_cache_dir - -# Dependency caching for Windows -cacheKey: '{{ checksum "pom.xml" }}' -cacheDirectories: -- ${CACHE_DIR} - -shell: bash - -pre: -# Download and install packages in the CACHE_DIR. -# Skip execution of the tests in the pre step -- mvn -Dmaven.repo.local=${CACHE_DIR} -Dmaven.test.skip=true clean install - -post: -- cat yaml/linux/cucumber_hyperexecute_autosplit_sample.yaml - -mergeArtifacts: true - -uploadArtefacts: -- name: XmlReports -path: -- target/surefire-reports/testng-results.xml -- name: JsonReports -path: -- target/cucumber-reports/CucumberTestReport.json - -report: true -partialReports: -location: target/cucumber-reports/ -frameworkName: cucumber -type: json - -testDiscovery: -type: raw -mode: remote -#Parallel execution at feature level -#command: grep -rni 'Features' -e 'Feature:' | sed 's/.*://' -command: grep -nri '@' src/main/java/Features --include=\*.feature | sed 's/^.*://' - -testRunnerCommand: mvn test -Dplatname=linux -Dmaven.repo.local=m2_cache_dir -Dcucumber.options="--tags $test" - -jobLabel: [selenium-cucumber-java, linux, autosplit] -``` - -### Step 4: Execute your Test Suite - -> **NOTE :** In case of MacOS, if you get a permission denied warning while executing CLI, simply run **`chmod u+x ./hyperexecute`** to allow permission. In case you get a security popup, allow it from your **System Preferences** → **Security & Privacy** → **General tab**. - -Run the below command in your terminal at the root folder of the project: - -```bash -./hyperexecute --config RELATIVE_PATH_OF_YOUR_YAML_FILE -``` - -OR use this command if you have not exported your username and access key in the step 2. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE `} - -### Step 5: Monitor the Test Execution - -Visit the [HyperExecute Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) and check your Job status. - -### Step 6: Download Artifacts and Reports - -HyperExecute also facilitates the provision to download the [Artifacts](/support/docs/hyperexecute-artifacts/) and [Reports](/support/docs/hyperexecute-reports/) on your local machine. Click on the corresponding button to download your generated artifacts and reports. - -## 2. Testing Using Gitpod - -You can also use the Gitpod platform to execute our sample repository. It will fetch all the sample codebases and trigger the CLI to execute the tests. - -Follow the below steps to run Test using Gitpod: - -**Step 1:** Click '**Open in Gitpod**' button. You will be redirected to Login/Signup page. This button is configured to redirect you to the Gitpod platform where you will be able to execute our sample repository. - -**Step 2:** Login with TestMu AI credentials. Once logged in, a pop-up confirmation will appear, asking you to **'Proceed'** to the Gitpod editor in a new tab. The current tab will display the HyperExecute Dashboard. - -**Step 3:** Choose your preferred editor (we recommend VS Code Editor) - -**Step 4:** As you are running a sample project, Fetching of the Test Scripts, [HyperExecute YAML](/support/docs/deep-dive-into-hyperexecute-yaml/), [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) and Triggering your tests using the `Execution Command` will be automated. - -**Step 5:** Once you see the `Job Link` in the logs, you can visit the [HyperExecute dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) to see the tests getting executed. - -You can also implement [Secret Keys](/support/docs/hyperexecute-how-to-save-and-manage-secrets/) in your YAML file. - -## Navigation in Automation Dashboard - -Every test run on the HyperExecute has a unique *jobId* associated with it. Each *jobId* can in turn constitute single (or multiple) *groupId*(s). You can visit [HyperExecute Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) for checking the status of the test execution. - -You can seamlessly navigate between JobId's and taskId's. You need to click on the *testID* to navigate from the HyperExecute logs to the Automation Dashboard. - -The snapshot below shows the videos, logs and other meta data for that specific *test_ID* - -> -For any query or doubt, please feel free to contact us via **24×7 chat support** or you can also drop a mail to **support@testmuai.com**. -Happy testing! - -## Using the Cucumber Agent Skill with TestMu AI - -The [cucumber-skill](https://github.com/LambdaTest/agent-skills/tree/main/cucumber-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The cucumber-skill package includes: - -``` -cucumber-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing Cucumber Agent Skill - -Install a Cucumber Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/cucumber-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/cucumber-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only cucumber-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). diff --git a/static/docs/cucumber-report.md b/static/docs/cucumber-report.md deleted file mode 100644 index 78e4124c2..000000000 --- a/static/docs/cucumber-report.md +++ /dev/null @@ -1,46 +0,0 @@ -# Cucumber Report - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Cucumber reporting is a way to visualize and analyze test results when using the Cucumber testing framework. Cucumber is widely used for Behavior-Driven Development (BDD), allowing tests to be written in plain language using Gherkin syntax. The reports generated from Cucumber tests provide a readable format for stakeholders to understand the results, which helps in understanding the behavior of the system being tested without requiring technical expertise. - -Cucumber itself provides basic reporting in the command line, but additional plugins and tools can enhance the reporting experience, generating rich HTML or JSON reports. - -## Steps to Generate Cucumber Reports on HyperExecute - -### Step 1: Configure the TestRunner File -In your `TestRunner` file, configure `@CucumberOptions` to specify report formats and output paths. Here’s an example configuration: - -```javascript title="TestRunner.java" -@CucumberOptions( -features = "src/main/java/Features", -glue = {"Steps"}, -tags = {"~@Ignore"}, -format = { -"pretty", -"html:target/cucumber-reports/cucumber-pretty", -"json:target/cucumber-reports/CucumberTestReport.json", -"rerun:target/cucumber-reports/rerun.txt" -},plugin = "json:target/cucumber-reports/CucumberTestReport.json") -``` - -Explanation of plugin Options: - -- **pretty :** Outputs readable format in console. -- **html:target/cucumber-reports/cucumber-pretty :** Generates HTML report in the target directory. -- **json:target/cucumber-reports/CucumberTestReport.json :** Generates JSON report, often required for CI/CD and advanced reporting. -- **rerun:target/cucumber-reports/rerun.txt :** Logs any failed scenarios for rerun. - -### Step 2: Configure the HyperExecute YAML File -In your HyperExecute YAML configuration, define the [`report`](/support/docs/deep-dive-into-hyperexecute-yaml/#report) parameters like this: - -```yaml title="hyperexecute.yaml" -report: true -partialReports: -location: target/cucumber-reports/ -frameworkName: cucumber -type: json -``` - -### Step 3: Execute Your Tests -Run your tests on HyperExecute using the CLI. After your job completes, you can visit the HyperExecute dashboard to download and view the Cucumber report. diff --git a/static/docs/cucumberjs-with-selenium-running-cucumberjs-automation-scripts-on-testmu-selenium-grid.md b/static/docs/cucumberjs-with-selenium-running-cucumberjs-automation-scripts-on-testmu-selenium-grid.md deleted file mode 100644 index 96e53ec04..000000000 --- a/static/docs/cucumberjs-with-selenium-running-cucumberjs-automation-scripts-on-testmu-selenium-grid.md +++ /dev/null @@ -1,107 +0,0 @@ -# Selenium With CucumberJS - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Run CucumberJS tests on the TestMu AI cloud grid. This guide covers setup, running a sample test, configuring capabilities, and testing locally hosted pages. - -**Sample repo** -All the code used in this guide is available in the sample repository. - - View on GitHub - -## Prerequisites - -Complete these steps before running CucumberJS Selenium tests on TestMu AI. - -1. Create a [TestMu AI account](https://www.testmuai.com/register/?redirectTo=https://accounts.lambdatest.com/dashboard) and get your username and access key from the dashboard. -2. Install **NodeJS** v6 or newer from [nodejs.org](https://nodejs.org/en/). -3. Install **npm** from the [official npm website](https://www.npmjs.com/). -4. Download [Selenium JavaScript bindings](https://www.selenium.dev/downloads/) from the official website. - -## Step 1: Clone the Sample Project - -Clone the TestMu AI CucumberJS Selenium sample repository to your local machine. - -```bash -git clone https://github.com/LambdaTest/NodeJs-Cucumber-Selenium -cd NodeJs-Cucumber-Selenium -``` - -Install the required dependencies: -```bash -npm install -``` - -## Step 2: Set Your Credentials - -Set your TestMu AI username and access key as environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -## Step 3: Configure Your Test Capabilities - -Update the capabilities in `conf/single.conf.js` to define the browser and platform settings. - -```js -capabilities: [{ -browserName: 'chrome', -platformName: 'Windows 10', -browserVersion: 'latest', -name: "cucumber-js-single-test", -build: "cucumber-js-LambdaTest-single" -}] -``` - -Generate capabilities for your test requirements with the [Capabilities Generator](https://www.testmuai.com/capabilities-generator/). - -## Step 4: Run the Test - -Execute the test using the following command. - -**Single test:** -```bash -npm run single -``` - -**Parallel tests:** -```bash -npm run parallel -``` - -**All tests:** -```bash -npm run test -``` - -## Step 5: View Your Results - -After running the test, view your results on the [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build). - -The dashboard provides: -- Video recordings of each test session -- Screenshots captured at each step -- Console logs from the browser -- Network logs for debugging -- Detailed command logs - -## Run CucumberJS Tests Using Agent Skills - -Use AI coding assistants to generate and run CucumberJS tests with the TestMu AI Agent Skill. - -The [cucumber-skill](https://github.com/LambdaTest/agent-skills/tree/main/cucumber-skill) is part of [TestMu AI Agent Skills](https://github.com/LambdaTest/agent-skills/) - structured packages that teach AI coding assistants how to write production-grade test automation. - -Install the skill: - -```bash -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/cucumber-skill .claude/skills/ - -# For Cursor / Copilot -cp -r agent-skills/cucumber-skill .cursor/skills/ -``` - -Install all available framework skills at once by cloning the repository directly into your tool's skills directory (e.g., `.claude/skills/`, `.cursor/skills/`). diff --git a/static/docs/custom-dns-map.md b/static/docs/custom-dns-map.md deleted file mode 100644 index afb902fa0..000000000 --- a/static/docs/custom-dns-map.md +++ /dev/null @@ -1,61 +0,0 @@ -# Custom DNS Mapping for Selenium Tests - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Overview - -The `customDnsMap` capability lets you define custom IP addresses for specific domains. - -This is useful when you need to redirect domain requests to different IP addresses for testing purposes, without altering the global or browser-specific configurations. - -The customDnsMap capability uses the following format: - -```bash -"customDnsMap": { -"google.com": "", -"stackoverflow.com": "", -"": "", -"": "", -... -} -``` - -Here `` and `` are the domain names to be redirected, and `` and `` are the corresponding IP addresses. When a domain listed in the `customDnsMap` is accessed, it resolves to the specified IP address, overriding any existing DNS resolution for that domain. - -## Use Cases - -The customDnsMap capability serves a variety of testing purposes. - -- **Domain Redirection for Testing:** Redirect a domain to a different IP address, such as a staging or development server, when testing a web application that needs to connect to a specific backend. - -- **Local Development Testing:** Map a preferred domain to the local IP address to test application behavior when accessed via a domain name instead of localhost. - -- **Simulating Server Failures:** Redirect a domain to an invalid or different IP address to test how an application responds to server failures. - -- **Overriding Default DNS Resolutions:** When a default DNS resolution is set but a specific test scenario requires a different IP address, use customDnsMap to override the resolution for that particular test. - -- **Load Balancer Testing:** Redirect traffic to a specific server to evaluate its performance and functionality in isolation. - -The following example shows how to redirect traffic to a staging server for the domain `example.com`: - -```bash -"customDnsMap": { -"example.com": "192.168.1.100" -} -``` - -With this configuration, whenever a user accesses `example.com`, the request redirects to `192.168.1.100`, which is the staging server. - -## Benefits - -The `customDnsMap` capability offers the following benefits. - -- **Flexibility:** Easily and dynamically redirect traffic to different IP addresses without global configuration changes. - -- **Control:** Gain full control over how domains are resolved, which is essential for testing and debugging. - -- **Efficiency:** Improve testing and development workflows by eliminating the need to switch between different environments manually. - -## Conclusion - -The `customDnsMap` capability improves the efficiency and flexibility of testing and development workflows on the TestMu AI platform. Use it to redirect domains, simulate server failures, and test against specific backend servers. diff --git a/static/docs/custom-headers.md b/static/docs/custom-headers.md deleted file mode 100644 index 5db09b1a6..000000000 --- a/static/docs/custom-headers.md +++ /dev/null @@ -1,253 +0,0 @@ -# Bypass Firewalls with customHeaders and customUrlFilters - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Corporate firewalls keep networks secure but can occasionally interfere with your testing process. The customHeaders and customUrlFilters capabilities give developers precise control over network requests and firewall bypassing during testing. - -This document covers the customHeaders capability, which lets you add custom headers to your tests and bypass firewall restrictions while performing automated browser testing. - -## About CustomHeaders - -Custom headers carry information about the request or response, such as the method, URL, and body content. You can modify the parameters of HTTP requests sent by your tests by manipulating these headers, thereby working around firewall restrictions. - -## How to Use CustomHeaders Capability on TestMu AI - -Add custom headers using the Desired Capabilities class. - -1. Create an instance of the Desired Capabilities class. - -2. Use the customHeaders capability to add your custom headers. - -```java -DesiredCapabilities capabilities = new DesiredCapabilities(); -capabilities.setCapability("customHeaders", new HashMap() {{ -put("headerName", "headerValue"); -}}); -``` - -In the above code snippet, replace "headerName" and "headerValue" with the actual name and value of the header. You can add multiple headers based on your requirements. - -## CustomHeaders: Use Cases and Examples - -Custom headers serve different purposes for web development and network communications. - -1. **User Identification and Session Management**: Send tokens and session IDs to authenticate and identify users. This helps manage user sessions and implement stateless authentication. - -**Example**: The Custom header `X-Session-ID` tracks user sessions. - -```java -X-Session-ID: 1234567890 -``` - -2. **Content Negotiation**: Determine how the client and server decide on the data format to exchange. The "Accept" header specifies the format (like JSON or XML) that the client prefers. - -**Example**: `Accept` header specifies the client-preferred format of the response data. - -```java -Accept: application/json -``` - -3. **Rate Limiting**: APIs use custom headers to provide information about rate limits, including how many requests a client can make in a given time period and when they can make new requests. - -**Example** - -```java -X-RateLimit-Limit: 60 -X-RateLimit-Remaining: 56 -X-RateLimit-Reset: 1372700873 -``` - -4. **Debugging and Performance Tracking**: Some services include custom headers in their responses to provide additional information for debugging or performance tracking, such as server version numbers and execution times. - -**Example** - -```java -X-Session-ID: 1234567890 //custom header X-Session-ID to track user sessions. -``` - -5. **CORS (Cross-Origin Resource Sharing)**: The CORS standard uses custom headers to allow browsers and servers to interact securely with resources from different origins, including headers like "Access-Control-Allow-Origin" and "Access-Control-Allow-Methods". - -**Example** - -```java -X-Session-ID: 1234567890 //custom header X-Session-ID to track user sessions. -``` - -6. **Custom Application Logic**: Use custom headers to implement specific application-level logic, such as determining the response language, enabling or disabling features, or specifying API version numbers. - -**Example** - -```java -X-Session-ID: 1234567890 //custom header X-Session-ID to track user sessions. -``` - -7. **Bypassing Firewalls or Proxies**: In some cases, use custom headers to bypass certain network restrictions, such as firewalls or proxy servers. Always do this responsibly and in accordance with security policies. - -**Example** - -```java -X-Session-ID: 1234567890 //custom header X-Session-ID to track user sessions. -``` - -8. **Server Health and Status Information**: Some applications use custom headers to provide health and status information about the server or application for monitoring purposes. - -**Example** - -```java -X-Session-ID: 1234567890 //custom header X-Session-ID to track user sessions. -``` - -9. **SEO Optimization**: Custom headers like canonical and pagination headers guide search engines and optimize SEO. - -**Example** - -```java -X-Session-ID: 1234567890 //custom header X-Session-ID to track user sessions. -``` - -10. **A/B Testing**: Use custom headers to control or track A/B testing, where different versions of a service are tested against each other. - -**Example** - -```java -X-Session-ID: 1234567890 //custom header X-Session-ID to track user sessions. -``` - -## CustomHeader Capability: Your Key to Bypass Firewalls - -The customHeaders capability lets you inject custom headers into your HTTP requests to bypass firewalls or simulate specific client behavior. - -Custom headers are an integral part of HTTP requests and responses. They can carry authentication tokens, user agents, API versioning, and more. By modifying these headers in your tests, you can adjust the network behavior of the browser and ensure compatibility with restricted environments. - -## Targeted Control with customUrlFilters - -The customUrlFilters capability, used together with customHeaders, lets you specify exactly which URLs should receive the custom headers. This ensures that headers are only applied to requests matching your defined filters. - -### Key Behavior - -- If customHeaders are defined without customUrlFilters, the headers apply globally to all outgoing network requests. -- If customUrlFilters are provided, the customHeaders only apply to requests matching the filter criteria. -- Filters can be exact URLs or regular expressions, providing flexible targeting. - -### Implementation Example - -```java -DesiredCapabilities capabilities = new DesiredCapabilities(); - -Map headers = new HashMap<>(); -headers.put("WebView", "Enable"); -headers.put("X-Custom-Token", "secure-token-123"); - -List urlFilters = Arrays.asList( -"https://www.xhaus.com/headers", -"https://.*\\.example\\.com/.*" -); - -capabilities.setCapability("customHeaders", headers); -capabilities.setCapability("customUrlFilters", urlFilters); -``` - -### Behavior of This Example - -The headers `WebView: Enable` and `X-Custom-Token: secure-token-123` are only applied to: - -- `https://www.xhaus.com/headers` -- Any subpaths of domains like `https://api.example.com/`, etc. - -A request to `https://lambdatest.github.io/sample-todo-app/` will not contain any of the custom headers since it is not listed in customUrlFilters. - -## A Responsible Approach to Bypassing Firewalls - -While the ability to add and control custom headers is useful, always follow your organization's security and compliance policies. These capabilities are designed to facilitate secure, realistic testing - not to bypass security controls inappropriately. - -## Use Cases - -The following examples show common use cases for custom headers. - -**User Identification and Session Management** - -Send tokens or session IDs with headers like: - -```java -X-Session-ID: 1234567890 -``` - -**Content Negotiation** - -Specify expected response formats: - -```java -Accept: application/json -``` - -**Rate Limiting** - -Get limits and usage from APIs: - -```java -X-RateLimit-Remaining: 10 -``` - -**Debugging and Performance Tracking** - -Include trace info or timing metrics: - -```java -X-Execution-Time: 150ms -``` - -**CORS (Cross-Origin Resource Sharing)** - -Enable cross-origin requests: - -```java -Access-Control-Allow-Origin: * -``` - -**Custom Application Logic** - -Pass app-level config: - -```java -X-App-Version: v2.3.1 -``` - -**Bypassing Firewalls/Proxies** - -Mask the request with common headers: - -```java -User-Agent: Mozilla/5.0 (Windows NT 10.0...) -``` - -**Server Health** - -Return backend state: - -```java -X-Server-Status: All systems operational -``` - -**SEO Optimization** - -Guide search engines: - -```java -Link: ; rel="canonical" -``` - -**A/B Testing** - -Track experimental groups: - -```java -X-Experiment-ID: variant_b -``` - -## Conclusion - -The customHeaders and customUrlFilters capabilities let you simulate request scenarios, bypass firewalls responsibly, and selectively apply network rules - providing a flexible, developer-centric approach to browser testing. - -By turning obstacles like firewalls into controllable conditions, TestMu AI simplifies testing and enhances the realism and effectiveness of your QA process. - -Happy testing! diff --git a/static/docs/custom-speclow-report.md b/static/docs/custom-speclow-report.md deleted file mode 100644 index 4247c0a08..000000000 --- a/static/docs/custom-speclow-report.md +++ /dev/null @@ -1,53 +0,0 @@ -# Custom Specflow Report - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This guide provides step-by-step instructions on how to generate a custom SpecFlow report in TestMu AI HyperExecute. By following the specified changes in the YAML file and modifying the test script, users can receive detailed test reports via email. - -## Prerequisites -- A valid TestMu AI account. -- A configured HyperExecute environment. -- A SpecFlow-based test suite. - -## Configuration Changes -To enable custom SpecFlow reporting, make the following changes to your YAML configuration file and test script. - -### Step 1: Modify the YAML Configuration File -Add the following settings in your YAML file to enable reporting and configure partial reports: - -```yaml -report: true -partialReports: -location: YOUR_REPORTS_LOCATION -type: xml -frameworkName: specflow-custom -email: -to: [YOUR_EMAIL_ID] -metaInfo: [ "project-name:YOUR_CUSTOM_PROJECT_NAME", "project-env: YOUR_CUSTOM_PROJECT_ENV" ] -``` - -Explanation: -- `report: true` → Enables the reporting feature. -- `partialReports.location` → Specifies the directory where the partial reports will be stored. -- `partialReports.type: xml` → Configures the report format as XML. -- `partialReports.frameworkName: specflow-custom` → Sets the framework name. -- `email.to` → Defines the recipient email address where the reports will be sent. -- `email.metaInfo` → Adds metadata, including project name and environment details, to the report. - -### Step 2: Modify the Test Script -Add the following capability in your test script to ensure the correct execution status is recorded: - -`((IJavaScriptExecutor)driver).ExecuteScript($"lambda-info=suitename:YOUR_SUITE_NAME:passed");` - -Explanation: -- This script sends test execution information (suite name and status) to TestMu AI. -- Replace `YOUR_SUITE_NAME` with the appropriate suite name. -- The passed status can be dynamically changed based on the test result. - -### Step 3: Report Generation and Email Notification -Once the tests are executed, the custom SpecFlow report will be generated and sent to the configured email address. The report will include: - -- Tag-based test summaries and details. -- Suite-based test summaries and details. -- Number of tests passed, failed, skipped, or with other statuses. -- Meta information such as project name and environment. diff --git a/static/docs/cyp-environment.md b/static/docs/cyp-environment.md deleted file mode 100644 index 4709da190..000000000 --- a/static/docs/cyp-environment.md +++ /dev/null @@ -1,68 +0,0 @@ -# Environment Variables Support for Cypress Tests - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -You can specify the environment variables that you want to use for your Cypress tests via the configuration file, via the `cypress.env.json` file, or via the TestMu AI Cypress CLI. However, if we set environment variables via both TestMu AI CLI and `cypress.env.json`, the `cypress.env.json` file will be ignored and only the variables set via CLI will be set into environment variables. If you want to understand these methods, go through the document below. - -### 1. Via the Configuration File - -a. **Cypress 9** - -A sample `cypress.json` file: - -```bash -{ -...... -"env":{ -"CYPRESS_BASE_URL":"https://example.cypress.io/", -"ACTIONS_URL": "commands/actions", -"WINDOW_URL": "commands/window" -}, -...... -} -``` - -b. **Cypress 10** - -A sample `cypress.config.js` file: - -```bash -module.exports = defineConfig({ -env: { -'CYPRESS_BASE_URL':'https://example.cypress.io/', -'ACTIONS_URL' : 'commands/actions', -'WINDOW_URL': 'commands/window' -}, -``` - -You can use both of these in your test specifications file. An example of a test specifications file is: - -```bash -describe('Sample test', () => { -it('test case - actions', () => { -cy.visit(Cypress.env('CYPRESS_BASE_URL') + Cypress.env('ACTIONS_URL')) -cy.wait(3000) -}) -it('test case - window', () => { -cy.visit(Cypress.env('CYPRESS_BASE_URL') + Cypress.env('WINDOW_URL')) -cy.wait(3000) -}) -}) -``` -### 2. Via the `cypress.env.json` File - -```bash -{ -"CYPRESS_BASE_URL":"https://example.cypress.io/", -"ACTIONS_URL" : "commands/actions", -"WINDOW_URL": "commands/window" -} -``` - -### 3. Via the TestMu AI Cypress CLI - -You can also add your environment variables via the parameter `--envs` in the TestMu AI Cypress CLI. - -```bash -lambdatest-cypress run --envs "CYPRESS_BASE_URL=https://example.cypress.io/,ACTIONS_URL=commands/actions,WINDOW_URL=commands/window" -``` diff --git a/static/docs/cyp-multi-reporters.md b/static/docs/cyp-multi-reporters.md deleted file mode 100644 index c38d47222..000000000 --- a/static/docs/cyp-multi-reporters.md +++ /dev/null @@ -1,84 +0,0 @@ -# Multi Reporters Support - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -With TestMu AI, you can generate multiple reporters supported by Cypress for your tests. If you want to do so, follow the steps below and download the generated reports automatically. - -#### Step 1 - -Provide the path where your report artefacts will be generated by the test script in the `lambdatest-config.json` file. In the same file, add the path of the reporter config file that contains the configurations that your reporters require. - -The example `lambdatest-config.json` file2 shown below uses 2 reporters, `mochawesome` and `mocha-junit-reporter`. - -```bash -"run_settings":{ -..... -"reporter_config_file": "reporter-config_mochawesome_junit.json", -"downloads": "./cypress/results", -..... -} -``` - -#### Step 2 - -Create a file with the name `reporter-config_mochawesome_junit.json` and copy the contents of the code block below. - -```bash -{ -"reporterEnabled": "mochawesome,mocha-junit-reporter", -"mochawesomeReporterOptions": { -"reportDir": "cypress/results/json", -"overwrite": true, -"html": true, -"json": true -}, -"mochaJunitReporterReporterOptions": { -"mochaFile": "cypress/results/my-test-output.xml", -"toConsole": true -} -} -``` - -#### Step 3 - -Run your test in sync mode by using the code `lambdatest-cypress run --sync=true`. - -After all the tests are run, the generated reports will be automatically downloaded at the location `lambdatest-artefacts`. - -****Note**** -The location of `lambdatest-artefacts` is relative to the path from where you ran the command. - -**Sample Output**: - -```bash -Waiting for build to finish... -┌─────────┬───────────────────────────────────┬─────────────┬──────────┬──────────┬─────────┐ -│ (index) │ Spec │ Status │ Platform │ Browser │ Version │ -├─────────┼───────────────────────────────────┼─────────────┼──────────┼──────────┼─────────┤ -│ 0 │ 'cypress_env_params_test_spec.js' │ 'completed' │ 'win10' │ 'Chrome' │ '109.0' │ -│ 1 │ 'cypress_env_params_test_spec.js' │ 'completed' │ 'win10' │ 'Chrome' │ '109.0' │ -└─────────┴───────────────────────────────────┴─────────────┴──────────┴──────────┴─────────┘ -{ completed: 2 } -Creating directories -Directory created lambdatest-artefacts/f60aa4f9-4fca-46aa-b862-e4a0746a2eea -Extracted 14 entries for NPE6A-VMB8F-GVMDY-AG782 -Extracted 14 entries for 0OETA-BPSP3-XZVX4-EAPWB -``` - -The reports are grouped on the basis of your session_id with a designated _browser name_, _browser version_, and a _test id_. - -## Send Reports to ReportPortal.io - -By default the **Cypress-Multi-Reporter** mechanism above generates **mochawesome**. To send results to [**ReportPortal.io**](http://reportportal.io/) instead, create a separate reporter config file and point `reporter_config_file` at it. Make sure you have a [ReportPortal.io](http://reportportal.io/) account first. - -- Navigate to [ReportPortal.io](http://reportportal.io/) and log in, then open your **Report Portal Profile**. - -- Copy your Report Portal credentials. - -- In your Cypress project, create a new file defining the ReportPortal configuration/credentials. - -- Set that file name in the `reporter_config_file` capability of `lambdatest-config.json`. - -- Add the ReportPortal dependency (`@reportportal/agent-js-cypress`) to your `lambdatest-config.json` or `package.json`. - -- The integration is done. Open the Dashboard to see the results. diff --git a/static/docs/cypress-agent-skills.md b/static/docs/cypress-agent-skills.md deleted file mode 100644 index a88b8d0ec..000000000 --- a/static/docs/cypress-agent-skills.md +++ /dev/null @@ -1,108 +0,0 @@ -# Run Your Cypress Tests Using Agent Skills - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Instead of manually setting up your Cypress project, you can use **Agent Skills** to let your AI coding assistant generate production-ready Cypress test automation for you. - -The [cypress-skill](https://github.com/LambdaTest/agent-skills/tree/main/cypress-skill) is part of [TestMu AI Agent Skills](https://github.com/LambdaTest/agent-skills/) - structured packages that teach AI coding assistants how to write production-grade test automation code. - -## What Are Agent Skills? - -Agent Skills are self-contained packages of instructions, code patterns, debugging guides, and CI/CD configurations for specific testing frameworks. Once installed, your AI assistant works like a **Senior QA automation architect** with knowledge of: - -- Proper project structure for each framework -- Correct dependency versions and configurations -- Both local and TestMu AI cloud execution patterns -- Common pitfalls and debugging approaches -- CI/CD integration with GitHub Actions - -## Prerequisites - -- A TestMu AI account. If you don't have one, sign up for free. -- Your TestMu AI **Username** and **Access Key** from the [Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/). -- An AI coding assistant: [Claude Code](https://claude.ai/code), [GitHub Copilot](https://github.com/features/copilot), [Cursor](https://cursor.sh/), [Gemini CLI](https://github.com/google-gemini/gemini-cli), or any compatible tool. - -## Installing the Cypress Agent Skill - -Clone the repository and copy the Cypress skill into your AI tool's skills directory: - -```bash -git clone https://github.com/LambdaTest/agent-skills.git - -# For Claude Code -cp -r agent-skills/cypress-skill .claude/skills/ - -# For Cursor -cp -r agent-skills/cypress-skill .cursor/skills/ - -# For GitHub Copilot -cp -r agent-skills/cypress-skill .github/skills/ - -# For Gemini CLI -cp -r agent-skills/cypress-skill .gemini/skills/ -``` - -If you prefer installing **all available framework skills**, clone the repository directly into your tool's skills directory (e.g., `.claude/skills/`, `.cursor/skills/`). - -## Set Up Your Authentication - -Configure your TestMu AI credentials as environment variables: - -```bash -export LT_USERNAME="YOUR_LAMBDATEST_USERNAME" -export LT_ACCESS_KEY="YOUR_LAMBDATEST_ACCESS_KEY" -``` - -## Running Your First Test Using Agent Skills - -Once installed, simply ask your AI assistant to write tests in natural language: - -``` -"Write Cypress E2E tests for the login page and run them on TestMu AI cloud using Chrome and Firefox" -``` - -``` -"Set up Cypress component tests for the React dashboard and upload screenshots on failure" -``` - -``` -"Run Cypress tests across multiple browsers in parallel on TestMu AI" -``` - -The Agent Skill automatically handles: - -1. **Project setup** - Creates the right folder structure, dependencies, and config files -2. **Test type detection** - Routes to E2E (`cypress/e2e/`), Component (`cypress/component/`), or API tests -3. **Execution target** - Configures local or TestMu AI cloud execution via `lambdatest-config.json` -4. **Best practices** - Uses `data-cy` selectors, `cy.intercept()` for network, `cy.session()` for auth - -## What's Included in the Cypress Skill - -``` -cypress-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -| Section | Details | -|---------|---------| -| **Project Setup** | Dependencies, versions, configuration files, project structure | -| **Core Patterns** | Complete, runnable code examples with Cypress chaining best practices | -| **Cloud Integration** | TestMu AI `lambdatest-config.json` and `npx lambdatest-cypress run` setup | -| **CI/CD Integration** | GitHub Actions workflows with reporting and parallel execution | -| **Debugging Guide** | 15+ common issues mapped to solutions | -| **Best Practices** | Actionable recommendations for production code | - -## Supported AI Tools - -| Tool | Type | Installation Path | -|------|------|-------------------| -| Claude Code | CLI | `.claude/skills/` | -| GitHub Copilot | Extension | `.github/skills/` | -| Cursor | IDE | `.cursor/skills/` | -| Gemini CLI | CLI | `.gemini/skills/` | -| Codex CLI | CLI | `.codex/skills/` | -| OpenCode | CLI | `.opencode/skills/` | -| Claude.ai | Web | Settings > Features > Skills | diff --git a/static/docs/cypress-cli-commands.md b/static/docs/cypress-cli-commands.md deleted file mode 100644 index c826629d3..000000000 --- a/static/docs/cypress-cli-commands.md +++ /dev/null @@ -1,136 +0,0 @@ -# List of TestMu AI Cypress CLI Commands - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The TestMu AI-Cypress CLI provides a set of commands to streamline and enhance your testing workflows on the TestMu AI platform. Below is a comprehensive list of supported commands and their usage. - -## General Syntax -To use any command in the TestMu AI-Cypress CLI, follow this basic syntax: -```bash -lambdatest-cypress [options] -``` - -| Command / Arg | Purpose | -| -------------- | ------------------------------------------------------ | -| `--help` | To get information of all the commands that can be run | -| `init` | To create an initial config file | -| `run` | To run tests on TestMu AI | -| `build-info` | To get information about the build | -| `build-stop` | To stop all the tests in the build | -| `generate-report` | To generate the test session report | -| `--vip, --vi-project` | To set the smart ui project name. The acceptable data type is `string` | -| `--vib , --vi-build` | To set the smart ui build name. The acceptable data type is `string` | -| `--vibase , --vi-base` | To set that build as baseline for smart ui. The acceptable data type is `boolean` | - -## `init` command -The `init` command to generate a initial lambdatest configuration file. - -``` -lambdatest-cypress init -``` - -On running the above command, it will generate `base_reporter_config.json`,`custom_support_file.js` and `lambdatest-config.json` files. - -- **`base_reporter_config.json`**: This is the configuration file for mochawesome reporter, that TestMu AI uses to generate mochawesome reports which in turn is used to generate the commands section on the TestMu AI dashboard. For Cypress 10, the absence of this file may lead to the commands section not being visible on the dashboard. - -- **`custom_support_file.js`**: By default, Cypress automatically captures screenshots when a test fails. For Cypress 10, in order to make the screenshot visible with the failed tests on our dashboard, we recommend you move and import this file as recommended. - -- **`lambdatest-config.json`**: This file contains configurations like TestMu AI credentials, capabilities, run settings etc., that are required to run the test. - -## `run` command - -**Note** -For detailed examples of each Cypress flag, please visit our guide [Configuring Cypress Test Execution](/support/docs/run-settings/). - -To start running the test build, you can use the given-below command. - -```bash -lambdatest-cypress run -``` - -Given below are the additional flags available with the `run` command. - -| Flag | Purpose | Type | -|------|---------|------| -| `--version` | Show version number | Boolean | -| `--help` | Show help | Boolean | -| `--ccf, --cypress-config-file` | Path of the config file | String | -| `--user, --username` | LambdaTest username | String | -| `--ak, --access_key` | LambdaTest access key | String | -| `--lcf, --lambdatest-config-file` | Path of the LambdaTest config file | String | -| `-s, --specs` | Path of the spec file, directory, or pattern | String | -| `--env, --environment` | Specify environment name | String | -| `--bn, --build-name` | Set build name | String | -| `-t, --tags` | Run tests with specific tags | String | -| `-p, --parallels` | Number of parallel sessions | String | -| `--envs, --env-variables` | Set environment variables before test execution | String | -| `--tun, --tunnel` | Enable LambdaTest tunnel | String | -| `--tname, --tunnel_name` | Set LambdaTest tunnel name | String | -| `--brs, --browsers` | Run tests on specified browsers in format `platform:browser:version` | String | -| `--bi, --build-identifier` | Set build identifier or build counter | String | -| `--if, --ignore_files` | Files to ignore in project zip | String | -| `--sync, --sync-mode` | Enable sync mode to wait for test completion | String | -| `--autostart, --tat` | Enable tunnel auto start | String | -| `--headless, --headless-mode` | Run tests in headless mode | Boolean | -| `--net, --network` | Capture network logs | String | -| `--eof, --exit-on-failure` | Exit with code 1 on failure | String | -| `--cy, --cypress_settings` | Pass Cypress settings | String | -| `--geo, --geo_location` | Pass geo country code | String | -| `--sof, --stop_on_failure` | Stop other tests if any test in session fails | Boolean | -| `--ra, --reject_unauthorized` | Reject self-signed certificates in external requests | Boolean | -| `--bt, --build-tags` | Set build tags | String | -| `--sys-envs, --sys-env-variables` | Set system environment variables during test run | String | -| `--envfl, --env-file` | Path of `.env` file | String | -| `--npm-f, --npm-force` | Force npm install | Boolean | -| `--npm-lpd, --legacy-peer-deps` | Use legacy peer dependencies during npm install | Boolean | -| `--vip, --vi-project` | Set Visual UI project name | String | -| `--vib, --vi-build` | Set Visual UI build name | String | -| `--vibase, --vi-base` | Set Visual UI baseline build | Boolean | -| `--res, --resolution` | Set machine resolution | String | -| `--dp, --dedicated_proxy` | Enable dedicated proxy | Boolean | -| `--npm_tun, --npm_via_tunnel` | Install npm packages behind private VPN (increases build duration) | Boolean | -| `--md, --max_duration` | Stop test if running longer than specified minutes | String | -| `--cmd_log, --command_log` | Show command logs on dashboard | String | -| `--ret_fail, --retry_failed` | Retry failed tests in a new build | Boolean | -| `--net_http2, --network_http2` | Capture HTTP2 network logs | Boolean | -| `--net_ws, --network_ws` | Bypass WebSocket calls for network logs | Boolean | -| `--node18, --useNode18` | Use Node.js v18 for Cypress runtime | Boolean | -| `--nodeV, --useNodeVersion` | Specify Node.js version for Cypress runtime | String | -| `--net_sse, --network_sse` | Bypass SSE events for network logs | Boolean | -| `--cypress_accessibility, --accessibility` | Enable accessibility testing for Cypress | Boolean | -| `--tz, --timezone` | Set custom timezone in machine | String | -| `--reg, --region` | Set data center region (e.g., us, eu, ap) | String | -| `--pC, --privateCloud` | Set custom private cloud | String | - -## `build-info` command -You can use the `build-info` command to get information on the build. - -```bash -lambdatest-cypress build-info -``` - -Given below are the additional arguments available with the `build-info` command. - -| Flag | Purpose | Type | -| --------| -----------| -----| -| `--id, --build-id` | Build Identifier | String, Required | -| `--user, --username` | Your TestMu AI username | String | -| `--ak, --access_key` | Your TestMu AI access key | String | - -## `build-stop` command -You can use the `build-stop` command to stop all the test in the build. - -```bash -lambdatest-cypress build-stop -``` - -Given below are the additional arguments available with the `build-stop` command. - -| Flag | Purpose | -| -------------------- | --------- | -| `--id, --session_id` | Identifies the session | -| `--sls, --stop_last_session` | Stop the last test session | - -- `--stop_last_session` - -- `--session_id` diff --git a/static/docs/cypress-detailed-command-logs.md b/static/docs/cypress-detailed-command-logs.md deleted file mode 100644 index 8973e167c..000000000 --- a/static/docs/cypress-detailed-command-logs.md +++ /dev/null @@ -1,146 +0,0 @@ -# Detailed Command Logs for Cypress - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The **Detailed Command Logs** feature provides a comprehensive record of all Cypress commands and their results, both in the console and in a file. This functionality is ideal for debugging and troubleshooting, enabling you to pinpoint specific logs quickly and effectively. - -The logs are presented in an easy-to-read, human-readable format using the [cypress-terminal-report](https://www.npmjs.com/package/cypress-terminal-report) plugin. Below are the steps to implement this feature for Cypress versions below and above 10. - -## Prerequisite: Install Cypress and Cypress Terminal Report -Before proceeding, ensure that you have installed Cypress and the cypress-terminal-report plugin. - -In your `package.json` or `lambdatest-config.json` file, add the appropriate version of [cypress-terminal-report](https://www.npmjs.com/package/cypress-terminal-report) as a dependency: - -### For Cypress < 10: - -```javascript -"cypress-terminal-report": "4.1.3" -``` - -### For Cypress >= 10: - -```javascript -"cypress-terminal-report": "^5.3.2" -``` - -- In the `lambdatest-config.json`, enable detailed command logs by adding the following setting: -> **NOTE:-** You will be able to see this tab only when you use this capability **detailed_command_logs** in run_settings in lambdatest-config.json - -```javascript -"run_settings": { -"detailed_command_logs": true, -"downloads": "./cypress/results" -} -``` - -## For Cypress v9 and previous versions. - -### Step 1: Configure the Plugin -- Open the `cypress/plugins/index.js` file in your project. -- Add the following code to install and configure the cypress-terminal-report plugin: - -```javascript -const installLogsPrinter = require('cypress-terminal-report/src/installLogsPrinter') - -module.exports = (on, config) => { -// `on` is used to hook into various events Cypress emits -// `config` is the resolved Cypress config - -installLogsPrinter(on, { -printLogsToFile: 'always', -outputRoot: 'cypress/results/detailCommandLogs', -outputTarget: { -'detailCommandLogs.json': 'json', -}, -}) -} -``` - -### Step 2: Enable Logs in the Console (Optional) -To display detailed logs in the terminal, update the `installLogsPrinter` with the `printLogsToConsole: 'always'` code: - -```javascript -module.exports = (on, config) => { -installLogsPrinter(on, { -printLogsToConsole: 'always', // Enables logs in the terminal -printLogsToFile: 'always', -outputRoot: 'cypress/results/detailCommandLogs', -outputTarget: { -'detailCommandLogs.json': 'json', -}, -}); -}; -``` - -### Step 3: Install Logs Collector -- Navigate to `cypress/support/index.js`. -- Add the following code to install the log collector: - -```javascript -const installLogsCollector = require('cypress-terminal-report/src/installLogsCollector') - -installLogsCollector() -``` - -## For Cypress v10 and later versions. - -### Step 1: Configure the Plugin -- Open `cypress.config.js` in your project. -- Add the following code to configure the plugin: - -```javascript -const { defineConfig } = require("cypress"); -const installLogsPrinter = require("cypress-terminal-report/src/installLogsPrinter"); -module.exports = defineConfig({ -e2e: { -setupNodeEvents(on, config) { -// implement node event listeners here -installLogsPrinter(on, { -printLogsToFile:"always", -outputRoot: 'cypress/results/detailCommandLogs', -outputTarget: { -'detailCommandLogs.json': 'json', -} -}); -}, -}, -}); -``` - -### Step 2: Enable Logs in the Console (Optional) -If you need logs in the terminal, update the code like this: - -```javascript -const { defineConfig } = require("cypress"); -const installLogsPrinter = require("cypress-terminal-report/src/installLogsPrinter"); -module.exports = defineConfig({ -e2e: { -setupNodeEvents(on, config) { -// implement node event listeners here -installLogsPrinter(on, { -printLogsToConsole: 'always' -printLogsToFile:"always", -outputRoot: 'cypress/results/detailCommandLogs', -outputTarget: { -'detailCommandLogs.json': 'json', -} -}); -}, -}, -}); -``` - -### Step 3: Install Logs Collector -- Open `cypress/support/e2e.js`. -- Add the following code to install the log collector: - -```javascript -import installLogsCollector from 'cypress-terminal-report/src/installLogsCollector' - -installLogsCollector() -``` - -## View Generated Logs -Once the Cypress tests are executed, you can view the detailed command logs in the **"Detailed Command Logs"** tab on the test details page. - -For more information, visit [Artefacts For A Cypress Project](/support/docs/download-artefacts-cypress/) diff --git a/static/docs/cypress-integration-with-hyperexecute.md b/static/docs/cypress-integration-with-hyperexecute.md deleted file mode 100644 index 113293829..000000000 --- a/static/docs/cypress-integration-with-hyperexecute.md +++ /dev/null @@ -1,112 +0,0 @@ -# Cypress Integration with HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -In the realm of the software development and testing, developers and QA engineers embarked on a quest to ensure the reliability and efficiency of their applications. - -They needed a robust end-to-end testing framework that can help them to deal with issues like flaky tests, time travel debugging, parallel test execution, etc. Hence they turned to Cypress, that provides the following advantages: - -- **Parallel Test Execution :** Cypress supports parallel test execution, enabling faster test runs and optimized resource utilization. -- **End-to-End Test Execution :** Cypress allows you to write and run end-to-end tests for web applications. You can easily run these tests locally on your machine verify the checks. -- **Dashboard and Reports :** Cypress offers a Cloud Based Dashboard, a centralized location for all the reports and tests result visualizations. -- **Flaky Test Detections :** Cypress is a great framework option for addressing flaky tests and providing a stable testing environment. - -However, for larger projects and complex testing scenarios, certain challenges emerged, including **Test Parallelization**, **CI Pipeline Infrastructure**, **Fail Fasts mechanism**, **Detailed Reports** etc. Here comes a need for an effective companion to augment the capabilities of Cypress. - -Currently HyperExecute support these Operating Systems: - -- Linux -- Windows -> Windows 10, Windows 11 -- macOS -> Monterra, Ventura - -## Advantages of Integrating Cypress with HyperExecute - -**HyperExecute**, a powerful platform with features designed to seamlessly complement Cypress and address its limitations via: - -### Intelligently Orchestrate Your Tests - -Cypress excels in parallel testing, but managing large test suites within complex CI pipelines can be challenging. HyperExecute's [**AI Native orchestration**](/support/docs/hyperexecute-test-splitting-and-multiplexing/) intelligently distributes tests across available resources, optimizing execution speed and ensuring CI/CD stability. - -### Comprehensive Artifacts of Your Job - -While Cypress offers basic [**test reports**](/support/docs/cypress-mochaawesome-report/), detailed logs and [**artifacts**](/support/docs/hyperexecute-artifacts/) can be crucial for debugging and analysis. HyperExecute captures every step of your test run, providing comprehensive artifacts for in-depth troubleshooting and failure identification. - -### Fail Fast Mechanism - -Cypress prioritizes fast execution, but catching flaky tests early on can save time and prevent unnecessary re-runs. HyperExecute's smart "[**Fail Fast**](/support/docs/hyperexecute-failfast/)" mechanism identifies flaky tests quickly, stopping irrelevant test executions and focusing resources on reliable tests. - -### Mute Your Flaky Tests - -Flaky tests can disrupt your testing workflow. HyperExecute allows you to temporarily [**mute unreliable tests**](/support/docs/hyperexecute-test-muting/) while investigating the root cause, eliminating wasted resources and maintaining testing stability. - -### Smart Caching - -Repeatedly fetching data during tests can slow down execution. HyperExecute's [**intelligent caching**](/support/docs/deep-dive-into-hyperexecute-yaml/#cachekey) stores frequently accessed data, significantly speeding up test runs and optimizing resource utilization. - -### Detailed Reports of Your Job - -Beyond basic pass/fail information, HyperExecute generates [**detailed reports**](/support/docs/cypress-mochaawesome-report/) with performance metrics, logs, and screenshots for each test case. This granularity provides deeper insights into your testing process and facilitates informed decision-making. - -### Auto Retries - -Occasional test failures happen. HyperExecute's configurable [**retry mechanism**](/support/docs/deep-dive-into-hyperexecute-yaml/#retryonfailure) allows you to automatically re-run flaky or failed tests based on defined criteria, ensuring accurate results and preventing false negatives. - -### Prioritizing your Jobs - -In complex testing scenarios, prioritizing critical tests is essential. HyperExecute lets you define [**job priorities**](/support/docs/hyperexecute-prioritize-tests/), ensuring high-priority tests get executed first, maximizing efficiency and meeting your testing goals. - -### Run Background Services - -Cypress focuses on browser interactions. HyperExecute empowers you to run [**background services**](/support/docs/hyperexecute-background-services/) like database calls or API tests alongside your Cypress tests, streamlining your testing workflow and reducing overall execution time. - -### Archive Your Jobs - -Maintaining a clean testing history is crucial. HyperExecute lets you [**archive your jobs**](/support/docs/hyperexecute-jobs-archiving/) with their artifacts and reports, providing a comprehensive audit trail and valuable data for future analysis. - -### Secure Upload, Zero Hassle - -HyperExecute streamlines your CI/CD workflows by eliminating the need for manual test script uploads. All you have to do is just configure the [**`sourcePayload`**](/support/docs/hyperexecute-how-to-configure-sourcePayload/) parameter in your HyperExecute YAML file, and it will securely access your test scripts directly from your Git repository using secure access tokens. - -### Incremental Updates for Seamless Testing - -Optimize your testing workflows with HyperExecute's intelligent [**Incremental Code Updates**](/support/docs/deep-dive-into-hyperexecute-yaml/#differentialupload) feature. This feature significantly reduces upload times by only transferring necessary files for subsequent test runs. - -If your codebase changes are less than 75%, HyperExecute intelligently identifies and uploads only the updated or added parts, efficiently mapping the remainder from previous uploads. This approach minimizes wait times and streamlines your testing cycles, leading to faster results and a smoother overall testing experience. - -## Using the Cypress Agent Skill with TestMu AI - -The [cypress-skill](https://github.com/LambdaTest/agent-skills/tree/main/cypress-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The cypress-skill package includes: - -``` -cypress-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing Cypress Agent Skill - -Install a Cypress Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/cypress-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/cypress-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only cypress-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). diff --git a/static/docs/cypress-mochaawesome-report.md b/static/docs/cypress-mochaawesome-report.md deleted file mode 100644 index e466a0a9c..000000000 --- a/static/docs/cypress-mochaawesome-report.md +++ /dev/null @@ -1,44 +0,0 @@ -# Consolidated Mochawesome Report for Cypress - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The Mochawesome reporter is a custom Cypress reporter designed to generate standalone HTML reports. These reports are clean, modern, and interactive, making it easier to visualize test results. With features like filtering tests and displaying stack traces for failures, the Mochawesome report is an essential tool in Cypress for clear and concise test insights. - -## Generating a Consolidated HTML Report -Follow these steps to generate a consolidated HTML report using Mochawesome: - -### Step 1: Update Your Cypress Configuration -In your Cypress configuration file `cypress.config.js`, add the following code to enable Mochawesome as a reporter: - -```javascript title="cypress.config.js" -"reporter": "cypress-multi-reporters", -"reporterOptions": { -"reporterEnabled": [ -"mochawesome" -], -"mochawesomeReporterOptions": { -"reportDir": "cypress/results", -"overwrite": true, -"html": false, -"json": true -} -} -``` - -- The `overwrite` parameter should be set to `true` to ensure the report is replaced with the latest run results. -- The `html` option should be set to `false` because we will be merging JSON files later, and the mocha-merge utility does not support HTML files. -- Ensure the `reportDir` path is set to `"cypress/results"`. This path is used to generate logs that will be visible on the dashboard. **Do not change this path.** - -### Step 2: Configure the HyperExecute YAML File -In your HyperExecute YAML configuration, define the [`report`](/support/docs/deep-dive-into-hyperexecute-yaml/#report) parameters like this: - -```yaml title="hyperexecute.yaml" -report: true -partialReports: -frameworkName: cypress -location: cypress/results -type: html -``` - -### Step 3: Execute Your Tests -Run your Cypress tests on HyperExecute using the CLI. After your job completes, you can visit the HyperExecute dashboard to download and view the consolidated Mochawesome report. diff --git a/static/docs/cypress-testing-using-webkit.md b/static/docs/cypress-testing-using-webkit.md deleted file mode 100644 index b01a15fa3..000000000 --- a/static/docs/cypress-testing-using-webkit.md +++ /dev/null @@ -1,196 +0,0 @@ -# Cypress Testing Using WebKit - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -WebKit is a web browser engine based on KHTML that displays and interacts with web pages. It is open-source and used by many web browsers like Apple's Safari and Google's Chrome. - -TestMu AI lets you perform Cypress testing using WebKit, Safari's browser engine. By testing your web apps in WebKit, you can get an idea of how your website will run in Safari. - -## Prerequisites - -**Sample repo** - -Before we get started, make sure to clone the **TestMu AI's sample Cypress Cloud repo**, used in this document. View on GitHub - -You can run Cypress test in WebKit on the TestMu AI platform in a few simple steps: - -1. Clone the `LambdaTest-Cypress-Cloud` GitHub repo and navigate to the cloned directory. - -```bash -git clone https://github.com/LambdaTest/Cypress-Cloud.git -cd Cypress-Cloud -``` - -2. To run Cypress tests in WebKit, set your TestMu AI username and access key in the environment variables. You can get them from the TestMu AI Automation Dashboard. - -**Windows** - -```js -set LT_USERNAME="YOUR_LAMBDATEST_USERNAME" -set LT_ACCESS_KEY="YOUR_LAMBDATEST_ACCESS_KEY" -``` - -**macOS/Linux** - -```js -export LT_USERNAME="YOUR_LAMBDATEST_USERNAME" -export LT_ACCESS_KEY="YOUR_LAMBDATEST_ACCESS_KEY" -``` - -3. Install Node.js version 12 or higher. You can download it from the [official Node.js website](https://nodejs.org/en/download/). - -## Running Your First Test in WebKit - -Following are the steps to run your first Cypress test in WebKit on the TestMu AI platform. - -1. Install the TestMu AI-Cypress CLI using the below command. - -```bash -npm install -g lambdatest-cypress-cli -``` - -2. Clone the Cypress kitchen sink repo using the following command. - -```bash -# Clone the kitchen sink repo -git clone https://github.com/cypress-io/cypress-example-kitchensink.git - -# Go to the cloned directory -cd cypress-example-kitchensink -``` - -```bash -# Clone the kitchen sink repo -https://github.com/cypress-io/cypress-example-kitchensink/tree/ab10094ef7b199ae7febafec413a0626414bcd3c - -# Go to the cloned directory -cd cypress-example-kitchensink -``` - -Once you clone the kitchen sink repo, below will be the structure of your Cypress project. - -```bash -app -... -cypress -|-- fixtures -|-- e2e -|-- support -cypress.config.js -... -``` - -3. Install the npm dependencies by passing the below command. - -```bash -npm install -``` - -4. Create `lambdatest-config.json` file that contains configurations like auth, capabilities, test settings, etc. which need to be successfully executed at LambaTest. - -Use `init` command to generate the sample configuration files. - -```bash -lambdatest-cypress init -``` - -Once you run the above command, below is the project structure for the `lambdatest-config.json` file. - -```js -{ -"lambdatest_auth": { -"username": "", -"access_key": "" -}, -"browsers": [ -{ -"browser": "Webkit", -"platform": "Windows 11", -"versions": [ -"latest" -] -}, -{ -"browser": "Webkit", -"platform": "Windows 10", -"versions": [ -"latest" -] -} -{ -"browser": "Webkit", -"platform": "MacOS Monterey", -"versions": [ -"latest" -] -}, -{ -"browser": "Webkit", -"platform": "MacOS Big sur", -"versions": [ -"latest" -] -}, - -], -"run_settings": { -"cypress_config_file": "cypress.config.js", -"reporter_config_file": "base_reporter_config.json", -"build_name": "build-name", -"parallels": 1, -"specs": "./*.cy.js", -"ignore_files": "", -"network": false, -"headless": false, -"npm_dependencies": { -"cypress": "10.8.0" -"playwright-webkit": "^1.28.1", -"mochawesome": "7.0.1" - -} -}, -"tunnel_settings": { -"tunnel": false, -"tunnel_name": null -} -} -``` - -5. Pass the below command to run the test. - -```bash -lambdatest-cypress run -``` - -6. Visit [TestMu AI Automation dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard) to view your test results. The CLI also has a link to view the Cypress test build. - -## Testing Locally Hosted or Privately Hosted Projects - -To tests locally hosted websites on the TestMu AI platform, you need to setup [TestMu AI tunnel](/docs/testing-locally-hosted-pages/), and execute commands using the CLI, or [Download UnderPass](/docs/underpass-tunnel-application/), our GUI based desktop app. Once you have the TestMu AI tunnel or Underpass set up and started, you can use Cypress to test locally hosted websites. - -Now you need to activate the tunnel capability in the `lambdatest-config.json` file under the section "tunnel_settings" as shown below: - -```json -"tunnel_settings": { -"tunnel": true, -"tunnel_name": "LT_Tunnel" -} -``` - -You can provide the name of the **TestMu AI tunnel** as per your requirements. - -## Limitations - -* WebKit only supports the latest version. - -* The following dependencies must be in `lambdatest-config.json` file. - -```js -"npm_dependencies": { -"cypress": "10.8.0", -"playwright-webkit": "^1.28.1" -} -``` - -* Works only with Cypress **v10.8.0**. -* Supported on **Windows** - 11 and 10, and **macOS** - Monterey and Big Sur. diff --git a/static/docs/cypress-v10-accessibility-test.md b/static/docs/cypress-v10-accessibility-test.md deleted file mode 100644 index 61ea80f1f..000000000 --- a/static/docs/cypress-v10-accessibility-test.md +++ /dev/null @@ -1,105 +0,0 @@ -# Automation Tests with Accessibility Tool using Cypress v10 - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This guide takes you step-by-step through the process of using TestMu AI's Accessibility Tool with Cypress v10 and above to run automated tests for your website. - -> **NOTE:** This documentation is applicable for **Cypress v10** and **later versions**. - -## Prerequisites - -- Your [TestMu AI Username and Access key](/support/docs/using-environment-variables-for-authentication-credentials/) -- Setup your local machine as per your testing framework. - -## Step-by-Step Guide to Trigger Your Test - -### Step 1: Setup Your Test Suite - -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**sample repo** -Download or Clone the code sample from the TestMu AI GitHub repository to run your tests. - - View on GitHub - -Configure the desired capabilities based on your test requirements. For example: - -```yaml reference title="lambdatest-config.json" -https://github.com/LambdaTest/lambdatest-accessibility-cypress-v10/blob/main/lambdatest-config.json -``` - -### Step 2: Establish User Authentication - -Now, you need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in the [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). - -Run the below mentioned commands in your terminal to setup the CLI and the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -### Step 3: Update the Necessary Capabilities and Configurations in your project - -#### 1. Install the Cypress CLI - -- If you are passing npm dependencies through **package.json**, then add the below mentioned line there: - -```javascript reference title="package.json" -https://github.com/LambdaTest/lambdatest-accessibility-cypress-v10/blob/main/package.json -``` - -- If you are using **lambdatest-config.json** to pass the dependency, then pass the given line there: - -```javascript title="lambdatest-config.json" -"run_settings": { -//... -"npm_dependencies": { -"cypress": "10.0.0", -"lambdatest-cypress-cli": "^3.0.30" -}, -//... -} -``` - -- if you are not passing npm dependency in **lambdatest-config.json** you can run - -```bash -npm i lambdatest-cypress-cli -``` - -#### 2. In the `e2e.js` file - -Add this import statement in your `e2e.js` file to import the acceessibility scanner dependency - -```javascript -import 'lambdatest-cypress-cli/accessibility/scanner' -``` - -#### 3. In the `cypress.config.js` file - -Add this template in your config file. - -```javascript reference title="cypress.config.js" -https://github.com/LambdaTest/lambdatest-accessibility-cypress-v10/blob/main/cypress.config.js -``` - -#### 4. Add the capabilities in the `lambdatest-config.json` file - -To enable the accessibility testing within your automated test suite, set the `accessibility: true` in your configuration file. You can also define other settings capabilities as described below. - -```javascript title="lambdatest-config.json" -"accessibility" : true, // Enable accessibility testing -"accessibility.wcagVersion": "wcag21a", // Specify WCAG version (e.g., WCAG 2.1 Level A) -"accessibility.bestPractice": false, // Exclude best practice issues from results -"accessibility.needsReview": true // Include issues that need review -``` - -### Step 4: Execute and Monitor your Test - -Now execute your tests and visit the [Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard). Click on the Accessibility tab and check the report generated. - -```bash -lambdatest-cypress-cli run -``` diff --git a/static/docs/cypress-v10-hyprerexecute-accessibility-tests.md b/static/docs/cypress-v10-hyprerexecute-accessibility-tests.md deleted file mode 100644 index abddfe2b7..000000000 --- a/static/docs/cypress-v10-hyprerexecute-accessibility-tests.md +++ /dev/null @@ -1,117 +0,0 @@ -# Accessibility Testing on HyperExecute using Cypress v10 - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This document provides a detailed, step-by-step walkthrough for executing automated accessibility tests using TestMu AI's Accessibility Tool on HyperExecute. - -This documentation is applicable for **Cypress v10** and **later versions**. - -Please reach out to our support team to enable the Accessibility feature for your organization. - -## Prerequisites -- [HyperExecute YAML](/support/docs/hyperexecute-yaml-version0.2/) file which contains all the necessary instructions. -- [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) in order to initiate a test execution Job . -- Your [TestMu AI Username and Access key](/support/docs/using-environment-variables-for-authentication-credentials/) -- Setup your local machine as per your testing framework. - -## Step-by-Step Guide to Trigger Your Test - -### Step 1: Setup Your Test Suite - -To configure and test the system, you may use your existing project. For demonstration purposes, this guide utilizes a sample repository. - -**sample repo** -Download or Clone the code sample from the TestMu AI GitHub repository to run your tests. - - View on GitHub - -#### Download HyperExecute CLI - -The *HyperExecute CLI* is used for triggering tests on HyperExecute. It is recommend to download the HyperExecute CLI binary on the host system to perform the tests on HyperExecute. The CLI download site for various platforms is displayed below: - -| Platform | HyperExecute CLI download location | -| ---------| --------------------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| macOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -#### Setup Environment Variable -Export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in the [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). - -Run the below mentioned commands in your terminal to setup the CLI and the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -### Step 2: Update the Necessary Capabilities and Configurations in your project - -#### 1. Install the Cypress CLI -You will have to pass the `npm i lambdatest-cypress-cli` command in your [pre steps](/support/docs/deep-dive-into-hyperexecute-yaml/#pre) to install the TestMu AI Cypress CLI which will be used to execute your Cypress tests. - -```yaml title="hyperexecute.yaml" -pre: -- npm install -- npm install cypress --save-dev -# highlight-next-line -- npm i lambdatest-cypress-cli -``` - -#### 2. Add the capabilities in the YAML file - -To enable the accessibility testing within your automated test suite, set the `accessibility: true` in the [cypressOps](/support/docs/deep-dive-into-hyperexecute-yaml/#cypressops) flag of your YAML file. You can also define other settings capabilities as described below. - -```yaml title="hyperexecute.yaml" -cypressOps: -accessibility: true #Enable accessibility testing -accessibilityWcagVersion: "wcag21a" #Specify WCAG version (e.g., WCAG 2.1 Level A) -accessibilityBestPractice: false #Exclude best practice issues from results -accessibilityNeedsReview: true #Include issues that need review -``` - -#### 3. In the `e2e.js` file - -Add this import statement in your `e2e.js` file to import the acceessibility scanner dependency - -```javascript title="cypress/support/e2e.js" -import 'lambdatest-cypress-cli/accessibility/scanner' -``` - -#### 4. In the `cypress.config.js` file - -Add this code snippet in your `cypress.config.js` file. - -```javascript title="cypress.config.js" -const { defineConfig } = require("cypress"); -const lambdatestAccessibility = require('lambdatest-cypress-cli/accessibility/plugin'); - -module.exports = defineConfig({ -...// -e2e: { -setupNodeEvents(on, config) { -// implement node event listeners here -lambdatestAccessibility(on, config); -return config; -}, -}, -...// -}); -``` - -### Step 3: Execute and Monitor your Test - -Run the below command in your terminal at the root folder of the project: - -> **NOTE :** In case of macOS, if you get a permission denied warning while executing CLI, simply run **`chmod u+x ./hyperexecute`** to allow permission. In case you get a security popup, allow it from your **System Preferences** → **Security & Privacy** → **General tab**. - -```bash -./hyperexecute --config RELATIVE_PATH_OF_YOUR_YAML_FILE -``` - -OR use this command if you have not exported your username and access key in the step 1. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE `} - -Now visit the [Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard) and click on the Accessibility tab to check the report generated. diff --git a/static/docs/cypress-v9-accessibility-test.md b/static/docs/cypress-v9-accessibility-test.md deleted file mode 100644 index 278c5da18..000000000 --- a/static/docs/cypress-v9-accessibility-test.md +++ /dev/null @@ -1,115 +0,0 @@ -# Automation Tests with Accessibility Tool using Cypress v9 - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This document walks you through the step by step guide for executing the automation tests of your website with TestMu AI's Accessibility Tool using Cypress v9 and below. - -> **NOTE:** This documentation is applicable for **Cypress v9** and **previous versions**. - -## Prerequisites - -- Your [TestMu AI Username and Access key](/support/docs/using-environment-variables-for-authentication-credentials/) -- Setup your local machine as per your testing framework. - -## Step-by-Step Guide to Trigger Your Test - -### Step 1: Setup Your Test Suite - -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**sample repo** -Download or Clone the code sample from the TestMu AI GitHub repository to run your tests. - - View on GitHub - -Configure the desired capabilities based on your test requirements. For example: - -```yaml reference title="lambdatest-config.json" -https://github.com/LambdaTest/lambdatest-accessibility-cypress-v9/blob/main/lambdatest-config.json -``` - -### Step 2: Establish User Authentication - -Now, you need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in the [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). - -Run the below mentioned commands in your terminal to setup the CLI and the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -### Step 3: Update the Necessary Capabilities and Configurations in your project - -#### 1. Install the Cypress CLI - -- If you are passing npm dependencies through **package.json**, then add the below mentioned line there: - -```javascript title="package.json" -// ... -"devDependencies": { -"@cypress/eslint-plugin-dev": "5.0.0", -// highlight-next-line -"lambdatest-cypress-cli": "^3.0.30" -}, -``` - -- If you are using **TestMu AI-config.json** to pass the dependency, then pass the given line there: - -```javascript title="lambdatest-config.json" -"run_settings": { -//... -"npm_dependencies": { -"cypress": "10.0.0", -"lambdatest-cypress-cli": "^3.0.30" -}, -//... -} -``` - -- if you are not passing npm dependency in **TestMu AI-config.json** you can run - -```bash -npm i lambdatest-cypress-cli -``` - -#### 2. In the `support/index.js` file - -Add this import statement in your `index.js` file to import the acceessibility scanner dependency - -```javascript -import 'lambdatest-cypress-cli/accessibility/scanner' -``` - -#### 3. In the `cypress/plugins/index.js` file - -Add this template in your `index.js` file. - -```javascript title="index.js" -const lambdatestAccessibility = require('lambdatest-cypress-cli/accessibility/plugin'); - -module.exports = (on, config) => { -// `on` is used to hook into various events Cypress emits -// `config` is the resolved Cypress config - -lambdatestAccessibility(on, config); -return config; -} - -``` - -#### 4. Add the capabilities in the `lambdatest-config.json` file - -To enable the accessibility testing within your automated test suite, set the `accessibility: true` in your configuration file. You can also define other settings capabilities as described below. - -```javascript title="lambdatest-config.json" -"accessibility" : true, // Enable accessibility testing -"accessibility.wcagVersion": "wcag21a", // Specify WCAG version (e.g., WCAG 2.1 Level A) -"accessibility.bestPractice": false, // Exclude best practice issues from results -"accessibility.needsReview": true // Include issues that need review -``` - -### Step 4: Execute and Monitor your Test - -Now execute your tests and visit the [Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard). Click on the Accessibility tab and check the report generated. diff --git a/static/docs/cypress-v9-hyprerexecute-accessibility-tests.md b/static/docs/cypress-v9-hyprerexecute-accessibility-tests.md deleted file mode 100644 index c53591296..000000000 --- a/static/docs/cypress-v9-hyprerexecute-accessibility-tests.md +++ /dev/null @@ -1,111 +0,0 @@ -# Accessibility Testing on HyperExecute using Cypress v9 - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This document provides a detailed, step-by-step walkthrough for executing automated accessibility tests using TestMu AI's Accessibility Tool on HyperExecute using Cypress v9 and below. - -This documentation is applicable for **Cypress v9** and **previous versions**. - -Please reach out to our support team to enable the Accessibility feature for your organization. - -## Prerequisites -- [HyperExecute YAML](/support/docs/hyperexecute-yaml-version0.2/) file which contains all the necessary instructions. -- [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) in order to initiate a test execution Job . -- Your [TestMu AI Username and Access key](/support/docs/using-environment-variables-for-authentication-credentials/) -- Setup your local machine as per your testing framework. - -## Step-by-Step Guide to Trigger Your Test - -### Step 1: Setup Your Test Suite - -To configure and test the system, you may use your existing project. For demonstration purposes, this guide utilizes a sample repository. - -**sample repo** -Download or Clone the code sample from the TestMu AI GitHub repository to run your tests. - - View on GitHub - -#### Download HyperExecute CLI - -The *HyperExecute CLI* is used for triggering tests on HyperExecute. It is recommend to download the HyperExecute CLI binary on the host system to perform the tests on HyperExecute. The CLI download site for various platforms is displayed below: - -| Platform | HyperExecute CLI download location | -| ---------| --------------------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| macOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -#### Setup Environment Variable -Export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in the [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). - -Run the below mentioned commands in your terminal to setup the CLI and the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -### Step 2: Update the Necessary Capabilities and Configurations in your project - -#### 1. Install the Cypress CLI -You will have to pass the `npm i lambdatest-cypress-cli` command in your [pre steps](/support/docs/deep-dive-into-hyperexecute-yaml/#pre) to install the TestMu AI Cypress CLI which will be used to execute your Cypress tests. - -```yaml title="hyperexecute.yaml" -pre: -- npm install -- npm install cypress@9.6.1 --save-dev -# highlight-next-line -- npm i lambdatest-cypress-cli -``` - -#### 2. Add the capabilities in the YAML file - -To enable the accessibility testing within your automated test suite, set the `accessibility: true` in the [cypressOps](/support/docs/deep-dive-into-hyperexecute-yaml/#cypressops) flag of your YAML file. You can also define other settings capabilities as described below. - -```yaml title="hyperexecute.yaml" -cypressOps: -accessibility: true #Enable accessibility testing -accessibilityWcagVersion: "wcag21a" #Specify WCAG version (e.g., WCAG 2.1 Level A) -accessibilityBestPractice: false #Exclude best practice issues from results -accessibilityNeedsReview: true #Include issues that need review -``` - -#### 3. In the `support/index.js` file - -Add this import statement in your `support/index.js` file to import the acceessibility scanner dependency - -```javascript title="cypress/support/index.js" -import 'lambdatest-cypress-cli/accessibility/scanner' -``` - -#### 4. In the `plugins/index.js` file - -Add this code snippet in your `cypress/plugins/index.jss` file. - -```javascript title="cypress/plugins/index.js" -const lambdatestAccessibility = require('lambdatest-cypress-cli/accessibility/plugin'); - -module.exports = (on, config) => { -// `on` is used to hook into various events Cypress emits -// `config` is the resolved Cypress config -lambdatestAccessibility(on, config); -return config; -}; -``` - -### Step 3: Execute and Monitor your Test - -Run the below command in your terminal at the root folder of the project: - -> **NOTE :** In case of macOS, if you get a permission denied warning while executing CLI, simply run **`chmod u+x ./hyperexecute`** to allow permission. In case you get a security popup, allow it from your **System Preferences** → **Security & Privacy** → **General tab**. - -```bash -./hyperexecute --config RELATIVE_PATH_OF_YOUR_YAML_FILE -``` - -OR use this command if you have not exported your username and access key in the step 1. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE `} - -Now visit the [Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard) and click on the Accessibility tab to check the report generated. diff --git a/static/docs/cypressv10-on-hyperexecute.md b/static/docs/cypressv10-on-hyperexecute.md deleted file mode 100644 index ad7adb036..000000000 --- a/static/docs/cypressv10-on-hyperexecute.md +++ /dev/null @@ -1,433 +0,0 @@ -# Run Automation Tests on HyperExecute using Cypress v10 - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -> **NOTE :** This documentation is applicable for **Cypress v10** and **later versions**. - -HyperExecute is a smart test orchestration platform that allows you to run end-to-end Cypress tests as quickly as possible by providing a test infrastructure with optimal speed, test orchestration, and detailed execution logs. - -This guide will cover the basics of getting started with Cypress v10 testing on the HyperExecute. - -## Running Cypress v10 Tests on HyperExecute - -Cypress Tests can be executed on HyperExecute using 2 ways: - -1. [**Using Local System**](/support/docs/cypressv9-on-hyperexecute#1-testing-using-local-system) - Requires [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) to execute tests from your Local System. -2. [**Using Gitpod**](/support/docs/cypressv9-on-hyperexecute#2-testing-using-gitpod) - Execute tests using GitPod. (Requires a [Gitpod](https://gitpod.io/login/) account) - -## 1. Testing Using Local System - -### Prerequisites: - -To run the Tests on HyperExecute from your Local System, you are required: - -- [HyperExecute YAML](/support/docs/hyperexecute-yaml-version0.2/) file which contains all the necessary instructions. -- [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) in order to initiate a test execution Job . -- Your TestMu AI [Username and Access key](/support/docs/hyperexecute-how-to-get-my-username-and-access-key/) -- Setup the Environmental Variable - -**Sample repo** -You can either download or clone the code samples of Cypress v9 from the **TestMu AI's GitHub Repository** to quickly run your tests. View on GitHub - -#### Download HyperExecute CLI - -The *HyperExecute CLI* is used for triggering tests on HyperExecute. It is recommend to download the HyperExecute CLI binary on the host system to perform the tests on HyperExecute. The CLI download site for various platforms is displayed below: - -| Platform | HyperExecute CLI download location | -| ---------| --------------------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| macOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -#### Setup Environment Variable -Export the environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in the [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). -Run the below mentioned commands in the terminal to setup the CLI and the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -### Execution Methods: - -Cypress Tests can be executed on HyperExecute using 2 methods: - -### 1. Execution Using Auto-Split - -The Auto-Split mechanism enables you to run tests at predefined concurrency levels and distribute them across available infrastructure. Concurrency can be achieved at various levels, including file, module, test suite, test, and scenario. - -> To learn more about Auto-Split, refer to the [Auto-Split Getting Started Guide](/support/docs/getting-started-with-hyperexecute/#smart-auto-test-splitting) - -#### Core - -Auto-split YAML file in the repo contains the following configuration: - -```yaml -globalTimeout: 90 -testSuiteTimeout: 90 -testSuiteStep: 90 -``` - -The **`runson`** key determines the platform (or operating system) on which the tests are executed. Here we have set the target OS as Windows. - -```yaml -runson: win -``` - -**`auto-split`** is set to true in the YAML file. - -```yaml -autosplit: true -``` - -**`retryOnFailure`** is set to true, instructing HyperExecute to retry failed command(s). The retry operation is carried out till the number of retries mentioned in *maxRetries* are exhausted or the command execution results in a *Pass*. In addition, the concurrency (i.e. number of parallel sessions) is set to 4. - -```yaml -retryOnFailure: true -maxRetries: 1 -concurrency: 1 -``` - -**`cypress`** is set to true in the YAML file. - -```yaml -cypress: true -``` -Cypress Build name and tags are added in the YAML file. - -```yaml -cypressOps: -Build: "Hyperexecute Cypress Sample Build" -Tags: ["Hyperexecute","Cypress", "Windows", "Autosplit"] -BuildTags: ["Hyperexecute-Cypress"] -``` - -#### Pre Steps - -Steps (or commands) that must run before the test execution are listed in the *pre* run step. - -```yaml -pre: -- npm install -``` - -#### Post Steps - -Steps (or commands) that need to run after the test execution are listed in the *post* step. In the example, we *cat* the contents of *yaml/.hyperexecute_autosplit.yaml* - -```yaml -post: -- cat yaml/win/.hyperexecute_autosplit.yaml -``` - -The **`testDiscovery`** directive contains the command that gives details of the mode of execution, along with detailing the command that is used for test execution. Here, we are fetching the list of class names that would be further passed in the *testRunnerCommand* - -```yaml -testDiscovery: -type: raw -mode: static -command: ls cypress/e2e/2-advanced-examples | sed -n 1,'1p' -``` - -Running the above command on the terminal will give a list of scenarios present in the *feature* files: - -* actions.cy.js - -The **`testRunnerCommand`** contains the command that is used for triggering the test. The output fetched from the *testDiscoverer* command acts as an input to the *testRunner* command. - -```yaml -testRunnerCommand: npx cypress run --spec ./cypress/e2e/2-advanced-examples/$test --browser=chrome-95.0 --headed --config video=false -``` - -#### Test Execution - -Run the following command on the terminal to trigger the tests as per your machine. - -```bash -./hyperexecute --config yaml/win/.hyperexecute_autosplit.yaml --force-clean-artifacts --download-artifacts -``` - -```bash -./hyperexecute --config yaml/mac/.hyperexecute_autosplit.yaml --force-clean-artifacts --download-artifacts -``` - -```bash -./hyperexecute --config yaml/linux/.hyperexecute_autosplit.yaml --force-clean-artifacts --download-artifacts -``` - -Visit [HyperExecute Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/hyperexecute) to check the status of execution - -#### Sample YAML File for Auto-Split - -```yaml ---- -version: 0.1 - -runson: win -cypress: true - -autosplit: true -concurrency: 2 - -retryOnFailure: true -maxRetries: 1 - -pre: -- npm install -- npm install cypress --save-dev - -cacheKey: '{{ checksum "package.json" }}' -cacheDirectories: -- node_modules -- cypressCache - -env: -CYPRESS_CACHE_FOLDER: cypressCache - -testDiscovery: -mode: static -type: raw -command: ls cypress/e2e/2-advanced-examples - -testRunnerCommand: npx cypress run --spec ./cypress/e2e/2-advanced-examples/$test --browser=chrome-95.0 --headed --config video=false - -cypressOps: -Build: "Hyperexecute Cypress Sample Build" -Tags: ["Hyperexecute","Cypress", "Windows", "Autosplit"] -BuildTags: ["Hyperexecute-Cypress"] -Network: true -FullHar: true - -post: -- cat yaml/win/.hyperexecute_autosplit.yaml - -jobLabel: [cypress-v10, win, autosplit] -``` - -### 2. Execution Using Matrix - -Matrix-based test execution is used for running the same tests across different test (or input) combinations. The Matrix directive in HyperExecute YAML file is a *key:value* pair where value is an array of strings. Also, the *key:value* pairs are opaque strings for HyperExecute. - -> To learn more about Auto-Split, refer to the [Matrix Getting Started Guide](/support/docs/hyperexecute-matrix-multiplexing-strategy/) - -#### Core - -In the current example, matrix YAML file in the repo contains the following configuration: - -```yaml -globalTimeout: 100 -testSuiteTimeout: 90 -testSuiteStep: 90 -``` - -The target platform is set to Win. Please set the **`runson`** key to mac if the tests have to be executed on the macOS platform. - -```yaml -runson: win -``` - -The **`matrix`** constitutes of the following entries - *files*. The entries represent the test file names in the test code. - -```yaml -matrix: -os: [win] -browser: ["chrome-95.0","chrome-96.0","chrome-97.0","chrome-98.0","chrome-99.0","chrome-100.0","chrome-101.0","chrome-102.0","chrome-103.0","chrome-104.0","chrome-105.0"] -files: ["actions.cy.js"] -``` - -The **`testSuites`** object contains a list of commands (that can be presented in an array). In the current YAML file, commands for executing the tests are put in an array (with a '-' preceding each item). The Maven command *mvn test* is used to run tests located in the current project. In the current project, parallel execution is achieved at the *class* level. The *maven.repo.local* parameter in Maven is used for overriding the location where the dependent Maven packages are downloaded. - -```yaml -testSuites: -- npx cypress run --spec ./cypress/e2e/2-advanced-examples/$files --browser=$browser --headed --config video=false -``` - -**`cypress`** is set to true in the YAML file. - -```yaml -cypress: true -``` -Cypress build name and tags are added in the YAML file. - -```yaml -cypressOps: -Build: "Hyperexecute Cypress Sample Build" -Tags: ["Hyperexecute","Cypress", "Windows", "Matrix"] -BuildTags: ["Hyperexecute-Cypress"] -``` -#### Pre Steps - -Steps (or commands) that must run before the test execution are listed in the *pre* run step. - -```yaml -pre: -- npm install -``` - -#### Post Steps - -Steps (or commands) that need to run after the test execution are listed in the *post* step. In the example, we *cat* the contents of *yaml/.hyperexecute_matrix.yaml* - -```yaml -post: -- cat yaml/win/.hyperexecute_matrix.yaml -``` - -#### Test Execution - -Run the following command on the terminal to trigger the tests as per your machine. - -```bash -./hyperexecute --config yaml/win/.hyperexecute_matrix.yaml --force-clean-artifacts --download-artifacts -``` - -```bash -./hyperexecute --config yaml/mac/.hyperexecute_matrix.yaml --force-clean-artifacts --download-artifacts -``` - -```bash -./hyperexecute --config yaml/linux/.hyperexecute_matrix.yaml --force-clean-artifacts --download-artifacts -``` - -Visit [HyperExecute Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/hyperexecute) to check the status of execution - -#### Sample YAML File for Matrix -```yaml ---- -version: 0.1 - -runson: ${matrix.os} -cypress: true - -parallelism: 1 - -retryOnFailure: false -maxRetries: 2 - -cacheKey: '{{ checksum "package.json" }}' -cacheDirectories: -- node_modules - -pre: -- npm install -- npm install cypress --save-dev - -matrix: -os: [win] -browser: ["chrome-103.0","chrome-104.0","chrome-105.0"] -files: ["actions.cy.js"] - -testSuites: -- npx cypress run --spec ./cypress/e2e/2-advanced-examples/$files --browser=$browser --headed --config video=false - -cypressOps: -Build: "Hyperexecute Cypress Sample Build" -Tags: ["Hyperexecute","Cypress", "Windows", "Matrix"] -BuildTags: ["Hyperexecute-Cypress"] -Network: true -FullHar: true - -post: -- cat yaml/win/.hyperexecute_matrix.yaml - -jobLabel: [cypress-v10, win, matrix] -``` - -## 2. Testing Using Gitpod - -Follow the below steps to run Cypress Tests on HyperExecute using Gitpod: - -**Step 1:** Click the **Open in Gitpod** button (You will be redirected to Login/Signup page). - -**Step 2:** Login with TestMu AI credentials. You will be redirected to HyperExecute dashboard with pop-up confirming to **'Proceed'** to Gitpod editor in the new tab and current tab will show hyperexecute dashboard. - -**Step 3:** You will need to login into your Gitpod account, or create a new one. - (If you are creating a new account, choose your preferred editor. We recommend VS Code) - -**Step 4:** As you are running a sample project, Fetching of the Test Scripts, [HyperExecute YAML](/support/docs/deep-dive-into-hyperexecute-yaml/), [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) and Triggering your tests using the `Execution Command` will be automated. - -**Step 5:** Once you see the `Job Link` in the logs, you can visit the [HyperExecute dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) to see the tests getting executed. - -## Additional Details - -### Secrets Management - -If you want to use any secret keys in the YAML file, this can be set by clicking on the **Secrets** button on the dashboard. All you need to do is create an environment variable that uses the secret key: - -```yaml -env: -PAT: ${{ .secrets.testKey }} -``` - -### HAR Logs - -HAR files are a standardized format for archiving HTTP Archive data. They provide a detailed record of all network requests and responses made by a web browser, including information like URLs, headers, timings, and response data. - -HAR logs offer a comprehensive view of your application's network activity during testing. This can be helpful for: - -- Debugging network-related issues -- Analyzing performance bottlenecks -- Diagnosing Load Order Issues - -Network Logs are disabled by default. To enable it, pass the `Network` and `FullHar` capability to `true` in the [`cypressOps`](/support/docs/deep-dive-into-hyperexecute-yaml/#cypressops) flag. - -```yaml -cypressOps: -Network: true -FullHar: true -``` - -You can download network logs via the Automate Dashboard. You can visualize HAR files using the [HAR Viewer](http://www.softwareishard.com/har/viewer/). - -### Detailed Cypress Command Logs -> To enable it for your organization, please contact us via **24×7 chat support** or you can also drop a mail to **support@testmuai.com**. - -The **Detailed Command Logs** feature can be used to generate a comprehensive record of all Cypress commands and their results, both in the console and in a file. This can be extremely useful for debugging and troubleshooting Cypress tests because it narrows down the logs to reveal specific information. The logs are presented in a human-readable format, making them simple to read and comprehend. - -Prerequisite to get the detailed logs in HyperExecute: - -```yaml -pre: -- npm install cypress-terminal-report --save-dev -``` - -For in detailed information on how to configure it properly, visit [Cypress Detailed Command Logs](/support/docs/cypress-detailed-command-logs/#for-cypress-v10-and-later-versions) documentation. - -## Using the Cypress Agent Skill with TestMu AI - -The [cypress-skill](https://github.com/LambdaTest/agent-skills/tree/main/cypress-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The cypress-skill package includes: - -``` -cypress-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing Cypress Agent Skill - -Install a Cypress Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/cypress-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/cypress-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only cypress-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). diff --git a/static/docs/cypressv9-on-hyperexecute.md b/static/docs/cypressv9-on-hyperexecute.md deleted file mode 100644 index 0bf9c7576..000000000 --- a/static/docs/cypressv9-on-hyperexecute.md +++ /dev/null @@ -1,411 +0,0 @@ -# Run Automation Tests Using Cypress v9 - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -> **NOTE :** This documentation is applicable for **Cypress v9** and **previous versions**. - -HyperExecute is a smart test orchestration platform that allows you to run end-to-end Cypress tests as quickly as possible by providing a test infrastructure with optimal speed, test orchestration, and detailed execution logs. - -This guide will cover the basics of getting started with Cypress v9 testing on the HyperExecute. - -## Running Cypress v9 Tests on HyperExecute - -Cypress Tests can be executed on HyperExecute using 2 ways: - -1. [**Using Local System**](/support/docs/cypressv9-on-hyperexecute#1-testing-using-local-system) - Requires [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) to execute tests from your Local System. -2. [**Using Gitpod**](/support/docs/cypressv9-on-hyperexecute#2-testing-using-gitpod) - Execute tests using GitPod. (Requires a [Gitpod](https://gitpod.io/login/) account) - -## 1. Testing Using Local System - -### Prerequisites: - -To run the Tests on HyperExecute from your Local System, you are required: - -- [HyperExecute YAML](/support/docs/hyperexecute-yaml-version0.2/) file which contains all the necessary instructions. -- [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) in order to initiate a test execution Job . -- Your TestMu AI [Username and Access key](/support/docs/hyperexecute-how-to-get-my-username-and-access-key/) -- Setup the Environmental Variable - -**Sample repo** -You can either download or clone the code samples of Cypress v9 from the **TestMu AI's GitHub Repository** to quickly run your tests. View on GitHub - -#### Download HyperExecute CLI - -The *HyperExecute CLI* is used for triggering tests on HyperExecute. It is recommend to download the HyperExecute CLI binary on the host system to perform the tests on HyperExecute. The CLI download site for various platforms is displayed below: - -| Platform | HyperExecute CLI download location | -| ---------| --------------------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| macOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -#### Setup Environment Variable -Export the environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in the [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). -Run the below mentioned commands in the terminal to setup the CLI and the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -### Execution Methods: - -Cypress Tests can be executed on HyperExecute using 2 methods: - -### 1. Execution Using Auto-Split - -The Auto-Split mechanism enables you to run tests at predefined concurrency levels and distribute them across available infrastructure. Concurrency can be achieved at various levels, including file, module, test suite, test, and scenario. - -> To learn more about Auto-Split, refer to the [Auto-Split Getting Started Guide](/support/docs/hyperexecute-auto-split-strategy/) - -#### Core - -Auto-split YAML file (*yaml/.hyperexecute_autosplit.yaml*) in the repo contains the following configuration: - -```yaml -globalTimeout: 90 -testSuiteTimeout: 90 -testSuiteStep: 90 -``` - -The **`runson`** key determines the platform (or operating system) on which the tests are executed. Here we have set the target OS as Windows. - -```yaml -runson: win -``` - -Auto-split is set to true in the YAML file. - -```yaml -autosplit: true -``` - -**`retryOnFailure`** is set to true, instructing HyperExecute to retry failed command(s). The retry operation is carried out till the number of retries mentioned in **`maxRetries`** are exhausted or the command execution results in a *Pass*. In addition, the **`concurrency`** (i.e. number of parallel sessions) is set to 2. - -```yaml -retryOnFailure: true -maxRetries: 1 -concurrency: 2 -``` -**`cypress`** is set to true in the YML file. - -```yaml -cypress: true -``` - -Cypress Build name and tags are added in the YML file. - -```yaml -cypressOps: -Build: "Hyperexecute Cypress Sample Build" -Tags: ["Hyperexecute","Cypress", "Windows", "Autosplit", "V9"] -BuildTags: ["Hyperexecute-Cypress"] -``` - -#### Pre Steps - -Steps (or commands) that must run before the test execution are listed in the *pre* run step. - -```yaml -pre: -- npm install -``` - -#### Post Steps - -Steps (or commands) that need to run after the test execution are listed in the *post* step. In the example, we *cat* the contents of *yaml/.hyperexecute_autosplit.yaml* - -```yaml -post: -- cat yaml/.hyperexecute_autosplit.yaml -``` - -The **`testDiscovery`** directive contains the command that gives details of the mode of execution, along with detailing the command that is used for test execution. Here, we are fetching the list of class names that would be further passed in the *testRunnerCommand* - -```yaml -testDiscovery: -type: raw -mode: static -command: ls cypress/integration | sed -n 1,'10p' -``` - -Running the above command on the terminal will give a list of scenarios present in the *feature* files: - -* actions.spec.js -* aliasing.spec.js -* assertions.spec.js -* connectors.spec.js -* cookies.spec.js -* local_storage.spec.js -* navigation.spec.js -* querying.spec.js -* sample.spec.js -* spies_stubs_clocks.spec.js - -The **`testRunnerCommand`** contains the command that is used for triggering the test. The output fetched from the *testDiscoverer* command acts as an input to the *testRunner* command. - -```yaml -testRunnerCommand: npx cypress run --spec ./cypress/integration/$test --browser=chrome-95.0 --headed --config video=false -``` - -#### Test Execution - -Run the following command on the terminal to trigger the tests as per your machine. - -```bash -./hyperexecute --config yaml/win/.hyperexecute_autosplit.yaml --force-clean-artifacts --download-artifacts -``` - -```bash -./hyperexecute --config yaml/mac/.hyperexecute_autosplit.yaml --force-clean-artifacts --download-artifacts -``` - -```bash -./hyperexecute --config yaml/linux/.hyperexecute_autosplit.yaml --force-clean-artifacts --download-artifacts -``` - -Visit [HyperExecute Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/hyperexecute) to check the status of execution - -#### Sample YAML File for Auto-Split - -```yaml ---- -version: 0.1 -globalTimeout: 90 -testSuiteTimeout: 90 -testSuiteStep: 90 -retryOnFailure: true -runson: win -cypress: true -maxRetries: 1 -concurrency: 2 -autosplit: true -# cacheKey: '{{ checksum "package.json" }}' -# cacheDirectories: -# - node_modules -# - cypressCache -env: -CYPRESS_CACHE_FOLDER: cypressCache -testDiscovery: -mode: static -type: raw -command: ls cypress/integration -testRunnerCommand: npx cypress run --spec ./cypress/integration/$test --browser=chrome-95.0 --headed --config video=false -cypressOps: -Build: "Hyperexecute Cypress Sample Build" -Tags: ["Hyperexecute","Cypress", "Windows", "Autosplit", "V9"] -BuildTags: ["Hyperexecute-Cypress"] -post: -- cat ./yaml/win/.hyperexecute_matrix.yaml -pre: -- npm install - -jobLabel: [cypress-v9, win, autosplit] -``` - -### 2. Execution Using Matrix - -Matrix-based test execution is used for running the same tests across different test (or input) combinations. The Matrix directive in HyperExecute YAML file is a *key:value* pair where value is an array of strings. Also, the *key:value* pairs are opaque strings for HyperExecute. - -> To learn more about Auto-Split, refer to the [Matrix Getting Started Guide](/support/docs/hyperexecute-matrix-multiplexing-strategy/) - -#### Core - -In the current example, matrix YAML file in the repo contains the following configuration: - -```yaml -globalTimeout: 90 -testSuiteTimeout: 90 -testSuiteStep: 90 -``` - -The target platform is set to Win. Please set the **`runso`n** key to **`mac`** if the tests have to be executed on the macOS platform. - -```yaml -runson: win -``` - -The **`matrix`** constitutes of the following entries - *classname*. The entries represent the class names in the test code. - -```yaml -matrix: -os: [win] -browser: ["chrome-95.0","chrome-96.0","chrome-97.0","chrome-98.0","chrome-99.0","chrome-100.0","chrome-101.0","chrome-102.0","chrome-103.0","chrome-104.0","chrome-105.0"] -files: ["actions.spec.js", "aliasing.spec.js", "assertions.spec.js", "connectors.spec.js", "cookies.spec.js", "local_storage.spec.js", "navigation.spec.js", "querying.spec.js"] -``` - -The **`testSuites`** object contains a list of commands (that can be presented in an array). In the current YAML file, commands for executing the tests are put in an array (with a '-' preceding each item). The Maven command *mvn test* is used to run tests located in the current project. In the current project, parallel execution is achieved at the *class* level. The *maven.repo.local* parameter in Maven is used for overriding the location where the dependent Maven packages are downloaded. - -```yaml -testSuites: -- npx cypress run --spec ./cypress/integration/$files --browser=$browser --headed --config video=false -``` - -#### Pre Steps - -Steps (or commands) that must run before the test execution are listed in the *pre* run step. - -```yaml -pre: -- npm install -``` - -#### Post Steps - -Steps (or commands) that need to run after the test execution are listed in the *post* step. In the example, we *cat* the contents of *yaml/.hyperexecute_matrix.yaml* - -```yaml -post: -- cat yaml/.hyperexecute_matrix.yaml -``` - -#### Test Execution - -Run the following command on the terminal to trigger the tests as per your machine. - -```bash -./hyperexecute --config yaml/win/.hyperexecute_matrix.yaml --force-clean-artifacts --download-artifacts -``` - -```bash -./hyperexecute --config yaml/mac/.hyperexecute_matrix.yaml --force-clean-artifacts --download-artifacts -``` - -```bash -./hyperexecute --config yaml/linux/.hyperexecute_matrix.yaml --force-clean-artifacts --download-artifacts -``` - -Visit [HyperExecute Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/hyperexecute) to check the status of execution: - -#### Sample YAML File for Matrix - -```yaml ---- -version: 0.1 -globalTimeout: 90 -testSuiteTimeout: 90 -testSuiteStep: 90 -retryOnFailure: false -runson: ${matrix.os} -cypress: true -maxRetries: 2 -parallelism: 1 -concurrency: 2 -cacheKey: '{{ checksum "package.json" }}' -cacheDirectories: -- node_modules -pre: -- npm install -matrix: -os: [win] -browser: ["chrome-104.0","chrome-105.0"] -files: ["actions.spec.js", "aliasing.spec.js"] -testSuites: -- npx cypress run --spec ./cypress/integration/$files --browser=$browser --headed --config video=false -cypressOps: -Build: "Hyperexecute Cypress Sample Build" -Tags: ["Hyperexecute","Cypress", "Windows", "Matrix", "V9"] -BuildTags: ["Hyperexecute-Cypress"] -post: -- cat ./yaml/win/.hyperexecute_autosplit.yaml - -jobLabel: [cypress-v9, win, matrix] -``` - -## 2. Testing Using Gitpod - -Follow the below steps to run Cypress Tests on HyperExecute using Gitpod: - -**Step 1:** Click the **Open in Gitpod** button (You will be redirected to Login/Signup page). - -**Step 2:** Login with TestMu AI credentials. You will be redirected to HyperExecute dashboard with pop-up confirming to **'Proceed'** to Gitpod editor in the new tab and current tab will show hyperexecute dashboard. - -**Step 3:** You will need to login into your Gitpod account, or create a new one. - (If you are creating a new account, choose your preferred editor. We recommend VS Code) - -**Step 4:** As you are running a sample project, Fetching of the Test Scripts, [HyperExecute YAML](/support/docs/deep-dive-into-hyperexecute-yaml/), [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) and Triggering your tests using the `Execution Command` will be automated. - -**Step 5:** Once you see the `Job Link` in the logs, you can visit the [HyperExecute dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) to see the tests getting executed. - -## Additional Details - -### Secrets Management - -If you want to use any secret keys in the YAML file, this can be set by clicking on the **Secrets** button on the dashboard. All you need to do is create an environment variable that uses the secret key: - -```yaml -env: -PAT: ${{ .secrets.testKey }} -``` - -### HAR Logs -HAR files are a standardized format for archiving HTTP Archive data. They provide a detailed record of all network requests and responses made by a web browser, including information like URLs, headers, timings, and response data - -HAR logs offer a comprehensive view of your application's network activity during testing. This can be helpful for: - -- Debugging network-related issues -- Analyzing performance bottlenecks -- Diagnosing Load Order Issues - -Network Logs are disabled by default. To enable it, pass the `Network` and `FullHar` capability to `true` in the [`cypressOps`](/support/docs/deep-dive-into-hyperexecute-yaml/#cypressops) flag. - -```yaml -cypressOps: -Network: true -FullHar: true -``` - -You can download network logs via the Automate Dashboard. You can visualize HAR files using the [HAR Viewer](http://www.softwareishard.com/har/viewer/). - -### Detailed Cypress Command Logs -> To enable it for your organization, please contact us via **24×7 chat support** or you can also drop a mail to **support@testmuai.com**. - -The **Detailed Command Logs** feature can be used to generate a comprehensive record of all Cypress commands and their results, both in the console and in a file. This can be extremely useful for debugging and troubleshooting Cypress tests because it narrows down the logs to reveal specific information. The logs are presented in a human-readable format, making them simple to read and comprehend. - -Prerequisite to get the detailed logs in HyperExecute: - -```yaml -pre: -- npm install cypress-terminal-report --save-dev -``` - -For in detailed information on how to configure it properly, visit [Cypress Detailed Command Logs](/support/docs/cypress-detailed-command-logs/#for-cypress-v9-and-previous-versions) documentation. - -## Using the Cypress Agent Skill with TestMu AI - -The [cypress-skill](https://github.com/LambdaTest/agent-skills/tree/main/cypress-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The cypress-skill package includes: - -``` -cypress-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing Cypress Agent Skill - -Install a Cypress Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/cypress-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/cypress-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only cypress-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). diff --git a/static/docs/dark-mode-on-browser.md b/static/docs/dark-mode-on-browser.md deleted file mode 100644 index 59d727e06..000000000 --- a/static/docs/dark-mode-on-browser.md +++ /dev/null @@ -1,31 +0,0 @@ -# Dark Mode - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI’s **Dark Mode** feature allows you to manually toggle between light and dark themes on **real iOS and Android devices** during manual testing sessions. -This enables testers to validate UI appearance, color contrast, and theme switching for both mobile apps and web applications without writing automation scripts. - -## Use Cases - -- Validate app UI elements render correctly in dark mode. -- Test color contrast and readability in dark themes. -- Verify icon and image visibility on dark backgrounds. -- Ensure theme inheritance across multiple app screens. -- Check browser and web application dark mode behavior. - -## Supported Devices - -| Device Type | Supported OS Version | -|-------------------------|----------------------| -| iPhone / iPad | iOS 13 and above | -| Android Phones/Tablets | Android 11 and above | - -## Steps to enable Dark Mode in Manual Testing - -**Step 1:** Start a manual testing session on a real iOS or Android device in **App Live** or **Browser Live**. - -**Step 2:** Go to the **Device Control tab** and turn the toggle **On** for Dark Mode. - -- Dark mode changes are applied at the **system/browser level** and may require refreshing the page for full effect. -- Some legacy websites may not fully support dark mode. -- Web applications must implement dark mode CSS for consistent theming beyond browser UI. diff --git a/static/docs/dark-mode-on-real-devices.md b/static/docs/dark-mode-on-real-devices.md deleted file mode 100644 index 7b497e495..000000000 --- a/static/docs/dark-mode-on-real-devices.md +++ /dev/null @@ -1,31 +0,0 @@ -# Dark Mode on Real Devices - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI’s **Dark Mode** feature allows you to manually toggle between light and dark themes on **real iOS and Android devices** during manual testing sessions. -This enables testers to validate UI appearance, color contrast, and theme switching for both mobile apps and web applications without writing automation scripts. - -## Use Cases - -- Validate app UI elements render correctly in dark mode. -- Test color contrast and readability in dark themes. -- Verify icon and image visibility on dark backgrounds. -- Ensure theme inheritance across multiple app screens. -- Check browser and web application dark mode behavior. - -## Supported Devices - -| Device Type | Supported OS Version | -|-------------------------|----------------------| -| iPhone / iPad | iOS 13 and above | -| Android Phones/Tablets | Android 11 and above | - -## Steps to enable Dark Mode in Manual Testing - -**Step 1:** Start a manual testing session on a real iOS or Android device in **App Live** or **Browser Live**. - -**Step 2:** Go to the **Device Control tab** and turn the toggle **On** for Dark Mode. - -- Dark mode changes are applied at the **system/browser level** and may require refreshing the page for full effect. -- Some legacy websites may not fully support dark mode. -- Web applications must implement dark mode CSS for consistent theming beyond browser UI. diff --git a/static/docs/dashboards-custom-widgets.md b/static/docs/dashboards-custom-widgets.md deleted file mode 100644 index 6ef1aa245..000000000 --- a/static/docs/dashboards-custom-widgets.md +++ /dev/null @@ -1,270 +0,0 @@ -# Custom Widgets in TestMu AI Analytics Dashboards - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Custom widgets in TestMu AI Analytics Dashboards allow you to create personalized visualizations of your test data. This feature enables you to gain insights into your testing process and make data-driven decisions. - -## Prerequisites - -Before creating a custom widget, ensure you have: - -1. An active LambdaTest account with access to Insights -2. Executed tests on the LambdaTest platform to generate data -3. Access to the Custom Widgets feature - -## Common Steps for Creating Custom Widgets - -The following steps are common to all custom widget types. After completing these steps, proceed to the widget-specific configuration based on your chosen visualization type. - -### Step 1: Navigate to Insights - -1. Log in to your LambdaTest account. -2. Navigate to the **Insights** section from the left sidebar or visit [https://analytics.lambdatest.com/](https://analytics.lambdatest.com/). -3. Click on the **Dashboards** tab. - -### Step 2: Create a New Dashboard or Select Existing - -1. Click on the **+ Create New** button located at the top right of the dashboard list. -2. From the dropdown menu, select **Custom Widgets**. - -### Step 3: Choose a Product - -Select the product for which you want to create the widget. Available options include: - -- **Web Automation**: Visualize web testing data -- **App Automation**: Analyze mobile testing data -- **HyperExecute**: Monitor HyperExecute test runs -- **Real Time Testing**: Visualize real-time testing data -- **Real Device Testing**: Analyze real device testing data -- **Smart UI Testing**: Visualize Smart UI testing data - -### Step 4: Select Data Source - -Choose the data source that will power your widget: - -- **Tests**: Visualize test execution data including pass/fail rates, test counts, and execution times -- **Auto Heal**: Visualize Auto Heal statistics and trends (available for Web Automation and HyperExecute only) - -### Step 5: Configure Filters (Optional) - -Use filters to narrow down your data based on specific requirements: - -1. Click **+ Add a Filter** to add filter criteria. -2. Select a **Key** from the dropdown (e.g., browser, OS, resolution, custom data). -3. Choose an **Operator** (equals, contains, greater than, etc.). -4. Select or enter the **Value** to filter by. -5. Add multiple filters as needed to refine your dataset. - -Use filters to focus on specific release versions, custom parameters, or other attributes. This helps create more targeted and actionable visualizations. - -### Next Steps: Widget-Specific Configuration - -After completing the common steps above, proceed to configure your specific widget type: - -- **[Heatmap Widgets](/docs/analytics-heatmap-widgets)**: Create heatmap visualizations to identify patterns across multiple dimensions -- **[Bar and Line Chart Widgets](/docs/analytics-bar-line-widgets)**: Create bar and line charts to visualize trends and comparisons -- **[Table Widgets](/docs/analytics-table-widgets)**: Create structured tables with multiple columns and aggregations - -## Features of Custom Widgets - -### Supported Products - -- **Web Automation**: Create custom widgets to visualize web testing data, including test execution times, pass/fail rates, and more. -- **App Automation**: Visualize mobile testing data, including device usage, OS versions, and app performance metrics. -- **HyperExecute**: Create custom widgets to monitor your HyperExecute test runs, including execution times, pass/fail rates, and more. -- **Real Time Testing**: Visualize real-time testing data, including browser and OS usage, test execution times, and more. -- **Real Device Testing**: Create custom widgets to monitor your real device testing data, including device usage, OS versions, and app performance metrics. -- **Smart UI Testing**: Visualize your Smart UI testing data, including test execution times, pass/fail rates, and more. - -### Advanced Filters and Options - -- **Select Keys**: Choose the keys you want to visualize in your custom widget. You can select multiple keys to create a more comprehensive view of your data. -- **Choose Operators**: Select the operators you want to apply to your data. This allows you to filter and manipulate your data to create the desired visualization. -- **Select Values**: Choose the values you want to visualize in your custom widget. This allows you to create a more focused view of your data. - -## Available Visualization Types - -Custom widgets support multiple visualization types, each optimized for different use cases: - -- **[Heatmap Widgets](/docs/analytics-heatmap-widgets)**: Visualize data density and relationships across multiple dimensions using color intensity and box sizes -- **[Bar Chart Widgets](/docs/analytics-bar-line-widgets)**: Compare values across categories using rectangular bars -- **[Line Chart Widgets](/docs/analytics-bar-line-widgets)**: Track trends and changes over time using connected data points -- **[Table Widgets](/docs/analytics-table-widgets)**: Display structured data with multiple columns, aggregations, and grouping options -- **Pie Chart**: Visualize the distribution of values across categories (documentation coming soon) -- **Billboard**: Display key metrics prominently on your dashboard (documentation coming soon) - -## Available Keys for Custom Widgets - -The following keys are available for configuring custom widgets across all visualization types. The availability of specific keys depends on the selected product and data source. These keys can be used for filtering, grouping, axis configuration, and column definitions depending on your widget type. - -Choose keys that represent categorical dimensions (like browser, OS, project name) for grouping and comparisons, and numeric or countable fields for aggregations. For time-based trends, use timestamp fields. - -Web Automation Keys - -The following keys are available for Web Automation widgets: - -| Key | Description | Recommended Use | -|-----|-------------|-----------------| -| `browser` | Browser name (Chrome, Firefox, Edge, Safari, etc.) | Grouping, filtering, X-axis | -| `browser_version` | Browser version number | Grouping, filtering | -| `build_name` | Build name identifier | Grouping, filtering, row identifier | -| `build_status` | Status of the build | Grouping | -| `buildtag_name` | Build tag name | Grouping, filtering | -| `create_timestamp` | Test creation timestamp | X-axis (time trends) | -| `custom_data.isFlakyTest` | Flaky test indicator from custom data | Grouping, filtering | -| `custom_data.product` | Product identifier from custom data | Grouping, filtering | -| `device` | Device name/type | Grouping, filtering | -| `duration` | Test execution duration | Y-axis (with aggregation), column (with aggregation) | -| `end_time` | Test end timestamp | X-axis (time trends) | -| `failure_category` | Category of test failures | Grouping | -| `os` | Operating system (Windows, macOS, Linux) | Grouping, filtering, X-axis | -| `os_version` | Operating system version | Grouping, filtering | -| `product` | Product name | Grouping, filtering | -| `project_name` | Project name | Grouping, filtering, row identifier | -| `resolution` | Screen resolution | Grouping, filtering | -| `start_time` | Test start timestamp | X-axis (time trends) | -| `status` | Test status (passed, failed, etc.) | Grouping (most common) | -| `test_id` | Test identifier | Y-axis (with Count aggregation), column (with Count aggregation) | -| `test_name` | Test case name | Row identifier, filtering | -| `test_type` | Type of test | Grouping, filtering | -| `testtag_name` | Test tag name | Grouping, filtering | -| `username` | User who executed the test | Grouping, filtering | - -**Custom Data Keys**: Any custom data keys you've defined in your test capabilities can also be used. - -App Automation Keys - -The following keys are available for App Automation widgets: - -| Key | Description | Recommended Use | -|-----|-------------|-----------------| -| `app_name` | Application name | Row identifier, grouping | -| `app_type` | Type of application | Grouping | -| `brand` | Device brand name | Grouping, filtering | -| `build_name` | Build name identifier | Row identifier, grouping | -| `build_status` | Status of the build | Grouping | -| `build_type` | Type of build | Grouping | -| `create_timestamp` | Test creation timestamp | X-axis (time trends) | -| `custom_data.isFlakyTest` | Flaky test indicator from custom data | Grouping, filtering | -| `device` | Device name/model | Row identifier, grouping | -| `duration` | Test execution duration | Y-axis (with aggregation), column (with aggregation) | -| `end_time` | Test end timestamp | X-axis (time trends) | -| `failure_category` | Category of test failures | Grouping | -| `os` | Mobile OS (Android, iOS) | Grouping, filtering, X-axis | -| `os_version` | OS version number | Grouping, filtering | -| `product` | Product name | Grouping, filtering | -| `project_name` | Project name | Row identifier, grouping | -| `start_time` | Test start timestamp | X-axis (time trends) | -| `status` | Test status (passed, failed, etc.) | Grouping (most common) | -| `test_id` | Test identifier | Y-axis (with Count aggregation), column (with Count aggregation) | -| `test_name` | Test case name | Row identifier, filtering | -| `test_type` | Type of test | Grouping, filtering | -| `username` | User who executed the test | Grouping, filtering | - -**Custom Data Keys**: Any custom data keys you've defined in your test capabilities can also be used. - -HyperExecute Keys - -The following keys are available for HyperExecute widgets: - -| Key | Description | Recommended Use | -|-----|-------------|-----------------| -| `app_name` | Application name (for app tests) | Row identifier, grouping | -| `app_type` | Type of application | Grouping | -| `browser` | Browser name | Grouping, filtering | -| `browser_version` | Browser version number | Grouping, filtering | -| `build_name` | Build name identifier | Row identifier, grouping | -| `build_status` | Status of the build | Grouping | -| `buildtag_name` | Build tag name | Grouping, filtering | -| `create_timestamp` | Test creation timestamp | X-axis (time trends) | -| `custom_data.product` | Product identifier from custom data | Grouping, filtering | -| `device` | Device name/type | Grouping, filtering | -| `duration` | Test execution duration | Y-axis (with aggregation), column (with aggregation) | -| `end_time` | Test end timestamp | X-axis (time trends) | -| `failure_category` | Category of test failures | Grouping | -| `job_created_at` | Job creation timestamp | X-axis (time trends) | -| `job_labels` | Job labels | Grouping | -| `job_status` | Status of the HyperExecute job | Grouping | -| `job_test_type` | Type of test in the job | Grouping | -| `os` | Operating system | Grouping, filtering | -| `os_version` | Operating system version | Grouping, filtering | -| `product` | Product name | Grouping, filtering | -| `project_name` | Project name | Row identifier, grouping | -| `resolution` | Screen resolution | Grouping, filtering | -| `stage_name` | Stage name in the job | Grouping | -| `stage_status` | Status of the stage | Grouping | -| `stage_type` | Type of stage | Grouping | -| `start_time` | Test start timestamp | X-axis (time trends) | -| `status` | Test status (passed, failed, etc.) | Grouping (most common) | -| `test_id` | Test identifier | Y-axis (with Count aggregation), column (with Count aggregation) | -| `test_name` | Test case name | Row identifier, filtering | -| `test_type` | Type of test | Grouping, filtering | -| `testtag_name` | Test tag name | Grouping, filtering | -| `username` | User who executed the test | Grouping, filtering | - -**Custom Data Keys**: Any custom data keys you've defined in your test capabilities can also be used. - -## Use Cases for Custom Widgets - -Custom widgets can be used for various purposes, including: -- **Tracking Test Execution**: Visualize the number of tests executed, passed, and failed over time to monitor your testing progress. -- **Analyzing Test Performance**: Identify trends in test execution times and pinpoint areas for improvement. -- **Monitoring Browser and OS Usage**: Gain insights into the most popular browsers and operating systems used in your tests to optimize your testing strategy. -- **Comparing Test Results**: Compare test results across different environments or configurations to identify discrepancies and ensure consistent performance. -- **Visualizing Test Coverage**: Create visualizations to understand the coverage of your tests across different features or components of your application. - -For detailed widget-specific use cases and examples, see: -- [Heatmap Widget Use Cases](/docs/analytics-heatmap-widgets#use-cases-for-heatmap-widgets) -- [Bar and Line Chart Use Cases](/docs/analytics-bar-line-widgets#use-cases-for-bar-and-line-chart-widgets) -- [Table Widget Use Cases](/docs/analytics-table-widgets#use-cases-for-table-widgets) - -## Private/Dedicated Device Widgets - -If your organization uses **Private Real Devices** (dedicated devices), you can create custom widgets to track usage and performance specifically for those devices. All private device widgets use the filter `is_dedicated = Equals = true` to scope data to dedicated devices only. - -Since the custom widget builder scopes data per product, you will need to create **separate widgets for each product** (e.g., App Automation and Real Device) to get a complete view. - -### Recommended Widget Configurations - -#### Testing by Device (Table) - -Get a per-device breakdown showing test count and total duration on each dedicated device. - -| Property | Value | -|----------|-------| -| Visualization | Table | -| Column | `device` | -| Measures | `test_id` (Count), `duration` (Sum) | -| Filter | `is_dedicated` = Equals = `true` | - -Create one widget for **App Automation** and one for **Real Device** testing. - -#### Testing by Type: Table - -Get a product-level summary (e.g., "App Automation Real Device", "Manual App Testing Real Device") for dedicated devices. - -| Property | Value | -|----------|-------| -| Visualization | Table | -| Column | `product` | -| Measures | `test_id` (Count), `duration` (Sum) | -| Filter | `is_dedicated` = Equals = `true` | - -Create one widget for **App Automation** and one for **Real Device** testing. - -#### Testing by Type: Bar Chart - -Same data as the table above, rendered as a bar chart for visual comparison. - -| Property | Value | -|----------|-------| -| Visualization | Bar | -| X-axis | `product` | -| Y-axis (Measure) | `duration` (Sum) | -| Filter | `is_dedicated` = Equals = `true` | - -Create one widget for **App Automation** and one for **Real Device** testing. - -The screenshot below shows a sample Private Real Device dashboard with all six widgets configured: - -> If you have any questions or need assistance with creating custom widgets, please reach out to our support team at [support@testmuai.com](mailto:support@testmuai.com) or visit our [Support Center](https://www.testmuai.com/support/). diff --git a/static/docs/data-retention-policy.md b/static/docs/data-retention-policy.md deleted file mode 100644 index 21ddd700a..000000000 --- a/static/docs/data-retention-policy.md +++ /dev/null @@ -1,137 +0,0 @@ -# Data Retention Policy - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This page provides a comprehensive overview of how long your data is retained across all TestMu AI products. Unless stated otherwise, the default retention period is **60 days** from the date of creation. - -**Enterprise Retention** -Enterprise customers can request extended retention periods for most data types. Contact your account manager or [support@testmuai.com](mailto:support@testmuai.com) for details. - - Web Automation - App Automation - Real Device Cloud - Real-Time Testing - HyperExecute - SmartUI - Test Manager - KaneAI - Accessibility - Analytics / Insights - Web Scanner - Platform - -## Web Automation - -| Data Type | Retention Period | Notes | -|-----------|-----------------|-------| -| Builds & Tests | 60 days | Build and test metadata, status, and execution data | -| Videos | 60 days | Test session video recordings | -| Logs | 60 days | Console, network, and command logs | -| Screenshots | 60 days | Captured during test execution | - -## App Automation - -| Data Type | Retention Period | Notes | -|-----------|-----------------|-------| -| Builds & Tests | 60 days | Build and test metadata, status, and execution data | -| Videos | 60 days | Test session video recordings | -| Logs | 60 days | Console, network, and command logs | -| Screenshots | 60 days | Captured during test execution | - -## Real Device Cloud - -| Data Type | Retention Period | Notes | -|-----------|-----------------|-------| -| Uploaded apps (APK/IPA) | 60 days | Apps uploaded for testing | -| App Profiling data | 60 days | Performance profiling reports | - -## Real-Time Testing - -| Data Type | Retention Period | Notes | -|-----------|-----------------|-------| -| Uploaded apps (APK/IPA) | 60 days | Apps uploaded for manual testing | - -## HyperExecute - -| Data Type | Retention Period | Notes | -|-----------|-----------------|-------| -| Test execution logs & records | 60 days | Execution data and test results | -| Test scripts | 60 days | Deleted permanently after retention period | -| Job artifacts & reports | 60 days | Stored automatically after job completion | -| Shareable links | 2-60 days | Configurable via `linkValidity` in YAML; default is 2 days | - -Use the `linkValidity` key in your HyperExecute YAML to set how long shareable artifact links remain active (2-60 days). - -## SmartUI - -| Data Type | Retention Period | Notes | -|-----------|-----------------|-------| -| Projects / Builds | Retained until deleted | Stays active as long as the account is active | -| Screenshots (baseline, compared, captured) | 1 year | Baseline, comparison, and captured screenshots | -| Shareable links | Retained until build/project deleted | Linked to the lifecycle of the parent build/project | -| Audit logs | 60 days | Dependent on internal log retention policies | - -## Test Manager - -| Data Type | Retention Period | Notes | -|-----------|-----------------|-------| -| Projects | No retention limit | Retained as long as the account is active | -| Test Cases | No retention limit | Retained as long as the account is active | -| Test Runs | No retention limit | Retained as long as the account is active | -| Insights | No retention limit | Retained as long as the account is active | - -Test Manager entities (projects, test cases, test runs) have no retention limit. However, if a test case is linked to an automation run, the **linked automation data** inherits the retention policy of the corresponding automation product (e.g., Web Automation = 60 days). - -## KaneAI - -| Data Type | Retention Period | Notes | -|-----------|-----------------|-------| -| Sessions | No retention limit | Includes generated code, runs, and version history | -| Modules | No retention limit | Reusable test blocks | -| Databases | No retention limit | Database connections | -| Variables | No retention limit | Variables and secrets | - -## Accessibility - -| Data Type | Retention Period | Notes | -|-----------|-----------------|-------| -| Automation scan reports (App/Web) | 60 days | Higher retention available on request for Enterprise | -| Manual scan reports (App/Web) | 60 days | Higher retention available on request for Enterprise | - -## Analytics / Insights - -| Data Type | Retention Period | Notes | -|-----------|-----------------|-------| -| Dashboards | No retention limit | User-created dashboards are retained indefinitely | -| Build & Test Insights data | 60 days | Historical analytics data | -| App Profiling reports | 60 days | Performance profiling data | -| Shareable dashboard links | 7, 15, 30 days, or Never | Links set to "Never" require a password for access | - -## Web Scanner - -| Data Type | Retention Period | Notes | -|-----------|-----------------|-------| -| Projects (visual / accessibility) | Retained until deleted | Active as long as the account is active | -| Schedules (visual / accessibility) | 1 year | Dependent on job execution infrastructure | - -## Platform - -| Data Type | Retention Period | Notes | -|-----------|-----------------|-------| -| Audit logs | 60 days | Enterprise customers can request higher retention | - -## Key Points - -- **Default retention**: 60 days from the date of creation unless otherwise specified. -- **Permanent data**: Test Manager, KaneAI, and Analytics dashboards have no retention limit and are retained as long as your account is active. -- **Extended retention**: SmartUI screenshots (1 year) and Web Scanner schedules (1 year) have longer retention periods. -- **Enterprise flexibility**: Enterprise customers can negotiate custom retention periods for most data types. -- **Linked data**: When test management entities reference automation data, the automation data follows its own product's retention policy. -- **Deletion**: Data is permanently deleted after the retention period expires and cannot be recovered. - -## Need Extended Retention? - -If your organization requires longer data retention for compliance, audit, or operational needs, contact our team: - -- **Email**: [support@testmuai.com](mailto:support@testmuai.com) -- **Enterprise inquiries**: Reach out to your account manager diff --git a/static/docs/datadog-integration.md b/static/docs/datadog-integration.md deleted file mode 100644 index 6fe4dd933..000000000 --- a/static/docs/datadog-integration.md +++ /dev/null @@ -1,46 +0,0 @@ -# Datadog Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Datadog is a cloud application monitoring and analytics tool. You can use it to compute performance metrics and event monitoring for infrastructure and cloud services. In addition, Datadog can monitor services such as servers, databases and tools. With TestMu AI and Datadog integration, you can log bugs while perform [cross browser testing](https://www.testmuai.com/) of your websites (and web apps) from the TestMu AI platform to Datadog. - -## Integrating Datadog With TestMu AI Account - -1. To begin, sign in to your TestMu AI account. To setup integrations, make sure you have admin or user level access. - -2. Visit **Integrations** from the left menu and select **DATADOG** from the ANALYTICS section. - ->**Note**: For logging bugs to Datadog, make sure you a Datadog account. - -3. It will redirect you to a Datadog OAuth authorization page. Click **Authorize**. - -The Datadog will be successfully integrated with your TestMu AI account. - -## One-Click Bug Logging With Datadog - -1. Select **Real Time Testing** from the left menu. - -2. Enter the test URL, select browser, browser versions, OS and resolution and press **START**. - -3. A cloud-based virtual machine will be launched, allowing you to conduct real-time testing of websites. After you've identified the issues, click on **Mark as Bug** icon in the left menu to capture a screenshot. - -4. After you've grabbed the screenshot, use the in-built image editor tools to mark any issues or tasks you've observed. After annotating the bugs, click on **Mark As Bug**. - -5. A Datadog ticket will open up. Fill the details like Service, Team, Assignee, Severity, State, Summary and Description. Then click on **Create Issue** and your bugs will be marked right away. - -6. To view your marked bugs, visit **Issue Tracker** from the TestMu AI left navigation menu. You'll notice the marked bugs along with the Test environment details. - -7. Click on the right diagonal arrow icon to view the logged issues in Datadog. - -You'll be taken to the Datadog interface where you can find your marked issues right away. - -## Uninstalling The Datadog Integration - -1. Login to your TestMu AI account and visit Integrations from the left menu. - -2. Navigate to 'My Integrations'and click on **REMOVE** under the Datadog block. - -The Datadog integration will be uninstalled. - -> -For any queries, feel free to drop them at our **24x7 Chat Support** or you could also mail us directly at **[support@testmuai.com](mailto:support@testmuai.com)**. diff --git a/static/docs/debugging-espresso-tests.md b/static/docs/debugging-espresso-tests.md deleted file mode 100644 index 3b2f7f3f8..000000000 --- a/static/docs/debugging-espresso-tests.md +++ /dev/null @@ -1,86 +0,0 @@ -# Debugging Your Espresso Tests - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Learn how to debug Espresso tests for your mobile applications to resolve different kinds of bugs for your failed tests/builds. - -## Objective - -By the end of this guide, you will be able to: - -* Resolve different kinds of bugs due to which your test/build failed. -* Understand the reasoning behind the same. - -## Debugging Tests - -TestMu AI offers a variety of logs to help you locate the root cause: - -* Instrumentation Logs -* Device Logs -* Network Logs -* Video Recording - -### Instrumentation Logs - -Instrumentation Logs are generated by the Espresso Test Runner. It generates the records of your test executions which are used to troubleshoot errors when a failure occurs. - -You can also download the logs by clicking on **Download Instrumentation logs**. - -#### Retrieving Consolidated Instrumentation Logs -In-depth insights into your test runs are crucial for effective debugging and optimization. When a more thorough understanding is needed, consolidated instrumentation logs play a pivotal role. Below, you'll find specific curl commands tailored to different scenarios. - -#### Non-Sharded Builds - -To access consolidated instrumentation logs for a non-sharded build: - -```bash -curl -u "YOUR_LAMBDATEST_USERNAME:YOUR_LAMBDATEST_ACCESSKEY" \ ---location 'https://mobile-api.lambdatest.com/mobile-automation/api/v1/framework/builds//logs/instrumentation' -``` -Replace `` with your actual build ID. -#### Sharded Builds -For sharded builds, use the following curl command: -```bash -curl -u "YOUR_LAMBDATEST_USERNAME:YOUR_LAMBDATEST_ACCESSKEY" \ ---location 'https://mobile-api.lambdatest.com/mobile-automation/api/v1/framework/jobs//logs/instrumentation' -``` -Replace `` with your actual job ID. - -### Device Logs - -Device Logs are generated by the Android Logcat. They are extremely helpful to see all the actions that are being performed on the device. - -It comes in really handy when an application crashes or any other failure occurs related to the application. - -Device Logs are disabled by default & can be enabled by passing the `deviceLog` capability. - -KEY|VALUE|DESCRIPTION ---|--|-- -| deviceLog | TYPE: BOOLEAN DEFAULT: False `deviceLog = false` OR `deviceLog = true` | Enable Device logs | - -### Network Logs `Will Be Available Soon` - -Network Logs record Network traffic data in detail which will show us the latency & HTTP requests. They’ll come in handy when we need to test our application on different network parameters. - -They’re disabled by default & can be enabled using the capability mentioned below: - -KEY|VALUE|DESCRIPTION ---|--|-- -| network | TYPE: BOOLEAN DEFAULT: FALSE `network = TRUE` OR `network = FALSE` | Enable network logs | - -### Video Recording - -All tests executed on the TestMu AI Real Devices are recorded & the recording is viewable when you click on a Test. The video is downloadable as well so you can use it as you like. - -By viewing this video recording, you can see where the test failed & the point from where it could not continue executing. - -The video capability is enabled by default as mentioned below. - -KEY|VALUE|DESCRIPTION ---|--|-- -| video | TYPE: BOOLEAN DEFAULT: TRUE `video= TRUE` OR `video = FALSE` | Video recording of the complete screen | - -Refer to our [Espresso testing documentation](/support/docs/getting-started-with-espresso-testing/#capabilities-supported) for more capabilities. - -> -If you still have any questions for us, please feel free to let us know via our **24X7 Chat Portal** or mail us to support@testmuai.com diff --git a/static/docs/debugging-options.md b/static/docs/debugging-options.md deleted file mode 100644 index 6cdc4d967..000000000 --- a/static/docs/debugging-options.md +++ /dev/null @@ -1,113 +0,0 @@ -# Debugging Options for Automation Testing - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI supports different automation capabilities for enabling debugging log options. This documentation explains how to use various logging options for your failed tests and fix them quickly. - -## Command Logs - -Command logs record every step executed in your test for troubleshooting failed steps. - -Command logs are a record of your test. Use them to identify all the steps executed in the test and troubleshoot errors for failed steps. Command logs are accessible from the TestMu AI Automation dashboard and via the REST API. - -## Visual Logs - -Visual logs capture screenshots at every Selenium command to help identify rendering issues. - -Visual logs capture screenshots of web pages automatically generated at every Selenium command executed by your code. Visual logs help debug the exact step and how the page rendered when a failure occurred. They also help identify layout or design-related issues with your web pages on different browsers. - -## Video Logs - -Video logs record the entire test run on the remote machine for playback. - -The TestMu AI Selenium Grid records every test run exactly as executed on the remote machine. This feature is useful when a browser test fails so that you can see the entire test in action. Access these videos from the TestMu AI Automation dashboard for each session. - -You can also navigate the video via command logs to view the step behavior at the time of command execution for the failed command. - -## Selenium Logs - -Selenium logs capture information about every Selenium command and browser preferences. - -Selenium logs capture information about every Selenium command in your test script, along with browser preferences. These logs provide insight into any errors that occurred during the execution of your tests. - -They contain details about Selenium internal logs and downstream and upstream commands that Grid received. - -## Appium Logs - -Appium logs provide a detailed record of your mobile test session for troubleshooting. - -The TestMu AI Appium server generates Appium logs, which contain a detailed record of your test session for mobile tests. If a test fails, the logs aid in troubleshooting any errors. - -Appium logs are similar to Selenium logs but have a different log structure standard to the Appium framework. - -## Console Logs - -Console logs capture browser console output at various test steps for JavaScript and DOM debugging. - -Console logs capture browser console output at various steps of the tests to troubleshoot JavaScript/DOM issues. You can retrieve console logs using both the TestMu AI Automation dashboard and REST API. - -## Network Logs - -Network logs capture browser performance data like traffic, latency, and HTTP requests in HAR format. - -Network Logs capture the browser's performance data, such as network traffic, latency, HTTP requests, and responses in the HAR format. Access network logs using the TestMu AI Automation dashboard and the REST API. - -## Terminal Logs - -Upload terminal logs from the client machine to view stack traces for specific test sessions. - -Using the Session API, you can upload the terminal logs (logs printed on the client machine) for specific tests/sessions. This helps you view the stack traces in case of failure and pinpoint actual issues in the code. - -## Telemetry Logs - -Use Selenium 4 and OpenTelemetry to trace requests and event logs for each session. - -Selenium 4 and OpenTelemetry support lets you view the tracing of requests and event logs for each request, which helps you troubleshoot sessions from the Selenium internals perspective. It collects all the events inside the Selenium services to triage the issue with Selenium servers. - -For more details, visit [Debugging in Selenium using Telemetry Logs](/support/docs/selenium-telemetry-logs/) - -## Enhanced Analytics - -Use analytics widgets to track test health, trends, browser distribution, and error insights. - -### Test Case Health Snapshot - -The Test Case Health Snapshot widget allows you to quickly analyze the flakiness of the tests executed on the platform. Track the performance of the tests by viewing the count of tests that are consistently passing or failing. - -### Test Summary - -The Test Summary widget lets you track the total tests run on the platform grouped by their status and includes information about errors like test case errors, idle timeouts, queue timeouts, and Lambda errors. - -### Test Browser - -The Test Browser widget displays the overall distribution of tests on various browsers on the platform. Click on a particular browser to see the specific versions where the tests ran. - -### Test OS - -The Test OS widget displays the overall distribution of tests on various operating systems. Click on a particular OS to see the specific versions where the tests ran. - -### Test Trends - -The Test Trends widget allows you to analyze the trends of tests executed on the platform over a period of time. Compare the current trends with previous trends by clicking on the legends at the bottom of the graph. - -### Test Device Coverage - -The Test Device Coverage widget is an App Automation-specific widget that displays the overall distribution of tests on various mobile devices on the platform. Click on a particular mobile device to see the specific models where the tests ran. - -### Test Status Ratio - -The Test Status Ratio widget provides information about the quality of your current test cases based on the status marked on the platform. Filter the information by clicking on the relevant legends at the bottom of the graph. - -### Error Insights - -The Error Stats widget allows you to quickly analyze the tests categorized by their status on the TestMu AI platform, including Test Case Errors, Idle Timeout, Queue Timeout, and Lambda Error. Filter the graph by clicking on the relevant legends at the bottom. - -### Concurrency Trends - -The Concurrency Trends widget allows you to view the tests kept in the queue and the tests in a running state. Filter the information by clicking on the relevant legends at the bottom of the graph. - -## HyperExecute - -View detailed analytics for your HyperExecute test runs. - -For details about HyperExecute Analytics, refer to our tutorial - [HyperExecute Analytics Modules](/support/docs/analytics-modules-hyperexecute/). diff --git a/static/docs/dedicated-proxy.md b/static/docs/dedicated-proxy.md deleted file mode 100644 index 02ee2243a..000000000 --- a/static/docs/dedicated-proxy.md +++ /dev/null @@ -1,27 +0,0 @@ -# Dedicated Proxy - IP Whitelisting - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This article will guide you on how does TestMu AI cloud access network restricted resources using a dedicated proxy server. - -## Prerequisites - -- Client network resources are publicly discoverable (clientapplication.private.com) but access is restricted to the dedicated proxy. -- The **dedicated proxy IP** needs to be whitelisted by the client’s network administrators. - -## IP whitelisting - -IP whitelisting can help the client to have control over who has the access to resources hosted on their network. IP whitelist is an approved list of IP addresses or IP domains that have permission to access the resource domain. Reserved for only trusted users, it is only set and updated by the client network administrators. With dedicated proxy solution, **only a single IP** needs to be whitelisted which saves the hassle of whitelisting multiple IP ranges. - -## Dedicated Proxy Server of TestMu AI - -TestMu AI provides a dedicated proxy server solution that can be whitelisted to provide access to the client’s network-restricted resources. - -### How does it work? - -1. The test script from the client network machine makes a call to the TestMu AI cloud platform to start the test session. -2. TestMu AI cloud then allocates a device to the user with the specified capabilities to run the user’s test on the TestMu AI cloud. -3. This allocated device makes all the network requests via the dedicated Proxy allocated to the client. The dedicated proxy checks whether resources can be attained through the public internet or need to be fetched from the client’s network. -4. The dedicated proxy accesses the network restricted resources by reaching out to the client’s network via the whitelisted IP. - -> That’s all! In case you have any questions or need any additional information, you could reach out at our **[24X7 Chat Support]** or mail us directly at support@testmuai.com. diff --git a/static/docs/deep-dive-into-hyperexecute-yaml.md b/static/docs/deep-dive-into-hyperexecute-yaml.md deleted file mode 100644 index 90b8f03cd..000000000 --- a/static/docs/deep-dive-into-hyperexecute-yaml.md +++ /dev/null @@ -1,1519 +0,0 @@ -# Deep Dive into HyperExecute YAML - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This document contains detailed explanations for all YAML flags, providing an in-depth understanding of each configuration parameter. - -## Mandatory Parameters - -### `version` -The version of HyperExecute YAML being used to run the tests. Currently there are two supported versions are [0.1](/support/docs/deep-dive-into-hyperexecute-yaml/#hyperexecute-yaml-parameters) and [0.2](/support/docs/hyperexecute-yaml-version0.2/). -```yaml -version: 0.1 -``` - -### `runson` -In this flag, you will define your required Operating System on which you want to run your tests. -> Currently we support linux, mac (macOS Monterey), mac13 (macOS Ventura), win (Windows 10) and win11 (Windows 11). - -```yaml -runson: linux # mac, mac13, win, win11 -``` - -If you want to run a multi OS job, you can use [matrix method](/support/docs/hyperexecute-matrix-multiplexing-strategy/) as shown below - -```yaml -runson: ${matrix.os} -matrix: -os: [linux, mac, mac13, win, win11] -``` - -### `pre` -All actions you need to perform before each test execution, such as installing dependencies. You’ll ideally want to use this parameter to "pre" run simple commands like `npm install`, `yarn install`, `mvn install` etc - -```yaml -pre: -- npm install -- mvn install -``` - ->📘 Refer to [globalPre](/support/docs/deep-dive-into-hyperexecute-yaml/#globalpre) command to perform a common global setup for all your tasks, such as installing dependencies or configuring environments. - -## AutoSplit Mode Parameters - -### `autosplit` -Auto-Split mode automatically splits your [`scenarios`](/support/docs/hyperexecute-status/#3-stage-level-status) among the concurrent number of [`tasks`](/support/docs/hyperexecute-status/#2-task-level-status) and executes them parallelly. This ensures optimal utilization of resources and significantly reduces the overall execution time of your test suite. - -For instance, if you have a concurrency of 10 and you want to run 50 tests in total, AutoSplit mode will distribute these 50 tests on 10 VMs in the most efficient manner possible to reduce your overall job execution time. - -> **Note:** In [`local mode`](/support/docs/deep-dive-into-hyperexecute-yaml/#mode), these commands will be smartly distributed among the VMs using history data, such that each VM (`task`) gets to run for almost the same amount of time to reduce your total [`job`](/support/docs/hyperexecute-guided-walkthrough/#jobs-page) time. - -```yaml -autosplit: true -``` - -> 📕 Take a closer look at the [AutoSplit mode](/support/docs/hyperexecute-auto-split-strategy/). - -### `concurrency` -This indicates the total number of concurrent sessions that can run in parallel for processing your test-cases. Your job will have as many virtual machines [(`tasks`)](/support/docs/hyperexecute-status/#2-task-level-status) as you have defined for this flag. - -> if you are using the [AutoSplit strategy](/support/docs/deep-dive-into-hyperexecute-yaml/#autosplit) then it is mandatory for you to define the concurrency. - -```yaml -concurrency: 10 -``` -**Pro Tip** -- After analyzing your test cases and usage patterns, the platform will automatically recommend the optimal concurrency settings tailored to your needs. These recommendations are displayed in the banner on the left-hand side. [Learn more here.](/support/docs/hyperexecute-how-to-find-correct-concurrency/) -- You can see the overall concurrency trends using our [analytics widgets.](/support/docs/analytics-modules-resource-utilization/#concurrency-trends) - -### `testDiscovery` -This is used to locate or discover relevant tests via class names, filters, file names, etc. -```yaml -testDiscovery: -type: raw -mode: local -command: grep 'public class' src/test/java/hyperexecute/*.java | awk '{print$3}' -``` - -It contains the following attributes: -#### `type` -```yaml -#(Recommended). When we are passing a command to discover tests. -type: raw #or - -#(Advanced). For more advanced use cases. -type: automatic -``` - -**`type: raw`** - -- **Purpose:** Perform a basic test discovery based on the provided command. -- **Functionality:** Directly executes the specified command and displays the discovered tests. -- **Limitations:** Doesn't utilize any built-in logic or advanced discovery capabilities. -- **Suitable for:** Simple test discovery scenarios where the command directly identifies the desired tests. - -**`type: automatic`** - -- **Purpose:** Utilize backend logic to discover tests using external tools. -- **Functionality:** Relies on a backend tool, such as Snooper, to perform test discovery. -- **Limitations:** Cannot be used directly with a command-based approach. -- **Suitable for:** Complex test discovery scenarios where advanced logic or external tools are required. - -In summary, `type:raw` is a basic and straightforward approach for discovering tests based on a specified command, while `type:automatic` provides more flexibility and advanced capabilities by leveraging external tools and backend logic. - -#### `mode` -This attribute defines where the test discovery occurs and how it is executed. HyperExecute supports two discovery modes: `local` and `remote`. - -> **NOTE:** The earlier `dynamic` discovery mode has been deprecated. Use `remote` instead for all new and existing YAML configurations. - -```yaml -#test discovery happens on machine where CLI is running -mode: local #or - -# test discovery happens in designated HyperExecute VMs -mode: remote -``` - -**`mode: local`** - -- **Purpose:** Test discovery is performed locally on the machine where the CLI is running. -- **Use Case:** Ideal for small projects or when tests need to be discovered locally. -- **Limitations:** Requires dependencies installed locally and doesn’t support matrix-based distributions. Debugging logs are generated locally, limiting visibility. - -**`mode: remote`** - -The `remote` discovery mode addresses the limitations of `local` modes. Instead of running test discovery on your local machine (local), this mode centralizes the process by using a dedicated remote Virtual Machines. - -This setup helps to ease the discovery process and makes it more efficient, especially for complex test setups. Additionally, it fully supports [matrix-based testing](/support/docs/hyperexecute-matrix-multiplexing-strategy/), which allows you to discover and manage tests across different configurations more effectively. - -> **NOTE :** [`type`](/support/docs/deep-dive-into-hyperexecute-yaml/#type) is not required with remote discovery. - -#### Key Features: -- **Centralized Discovery Tasks:** Tests are discovered remotely in a Virtual Machines designed for this purpose. Discovery tasks focus solely on identifying the tests without executing them, optimizing the discovery process. - -- **Orchestration Support:** Once tests are identified, orchestration algorithms ensure they are efficiently distributed across Virtual Machines, reducing idle time and improving resource utilization. - -- **Matrix Support:** Fully supports matrix configurations, allowing individual test discovery for each matrix combination, particularly useful for YAML 0.2 runners. - -- **Optimized Caching:** Remote Discovery Mode reduces redundant cache operations by performing them once in the discovery task, and sharing the cache across all Execution Tasks. This speeds up the overall process and minimizes resource wastage. - -- **Code Caching:** For users cloning their codebase via Git, Remote Discovery caches the code during the discovery task, reducing Git rate limits and accelerating the execution tasks. - -#### `command` -The command that fetches the list of test scenario that would be further executed using the value passed in `testRunnerCommand` -```yaml -command: grep 'public class' src/test/java/hyperexecute/*.java | awk '{print$3}' -``` - -- Test orchestration will happen with [`mode: local`](/support/docs/deep-dive-into-hyperexecute-yaml/#mode) only. -- 📕 Learn how to perform [dependent test discovery](/support/docs/hyperexecute-how-to-perform-dependent-test-based-discovery/). - -### `testRunnerCommand` -The `testRunnerCommand` used to run a single test entity in isolation. This entity could be a file, module, feature, or scenario. It is defined in the YAML file and tells the system how to run the test entity. - -```yaml -testRunnerCommand: mvn test -Dcucumber.options="$test" -Dscenario="$test" -DOs="win 10" - -# This command runs the test using Maven and passes in the options for Cucumber, the scenario to run, and the operating system to use. -``` - -## Matrix Mode Parameters - -### `matrix` -The [matrix mode](/support/docs/hyperexecute-matrix-multiplexing-strategy/) allows you to run the same set of tests across multiple combinations of environments, configurations, or parameters. This is particularly useful for ensuring that your software works correctly under different conditions, such as various operating systems, browser versions, or dependency files. - -```yaml -runson: ${matrix.os} - -matrix: -os: ["win", "mac", "linux"] -version: ["latest, dev"] -browser: ["Chrome", "Firefox"] -files: ["@File1","@File2"] -``` - -### `exclusionMatrix` -The [exclusion matrix](/support/docs/hyperexecute-matrix-multiplexing-strategy/#exclusion-in-matrix-strategy) allows you to omit specific combinations of parameters from being tested. For instance, if Safari on Windows is not a valid combination, you can exclude it using the exclusionMatrix parameter, ensuring that invalid or irrelevant test combinations are skipped. - -```yaml -#runson defines the OS of your test execution node. -runson: ${matrix.os} - -matrix: -# Browser version(s) separated by Comma -version: ["latest"] -# OS name sepataed by Coma -os: ["win", "mac", "linux"] -# Browser name separated by Comma -browser: ["Chrome", "Firefox", "Safari"] -# Test Files separated by Comma -files: ["@File1","@File2","@File3","@File4"] - -exclusionMatrix: -- os: [win] -browser: ["Safari"] -files: ["@File2","@File3"] -``` - -### `combineTasksInMatrixMode` - -In matrix mode, the [concurrency](/support/docs/deep-dive-into-hyperexecute-yaml/#concurrency) flag is not recognized. Therefore, in order to use the limited concurrencies that come with your license for a matrix-mode job, you need to set `combineTasksInMatrixMode` to `true`. This will run the (matrix-multiplied) combinations as scenarios in the number of HyperExecute machines that was specified in concurrency, as opposed to using one machine per matrix combination. - -For example, the below-mentioned YAML snippet will generate a total of 8 scenarios, and since the concurrency is set to 2, these 8 scenarios will run in parallel on 2 HyperExecute machines. In each machine (let's say each has 4 scenarios to execute), they will be running sequentially only. - -> **Total Scenarios =** *[Entries in os List] x [Entries in browser List] x [Entries in Files List]* - -```yaml -concurrency: 2 -combineTasksInMatrixMode: true - -matrix: -os: ["mac", "linux"] -browser: ["edge", "brave"] -files: ["Test1", "Test2"] -``` - -### `testSuites` -A command to run the tests that were mentioned in the scenario key for [matrix](/support/docs/deep-dive-into-hyperexecute-yaml/#matrix) based test execution. - -```yaml -testSuites: - mvn test -Dtest=$files -``` - -## Hybrid Mode Parameters - -### `parallelism` -`parallelism` defines the number of virtual machines [('tasks')](/support/docs/hyperexecute-status/#2-task-level-status) to be spawned in the case of hybrid mode. If you are not defining the parallelism, then you must define operating system-specific parallelism (win, mac, and  Linux). If both are defined, then preference will be given to OS-based parallelism. - -```yaml -runson: ${matrix.os} -parallelism: 2 - -matrix: -os: [win, mac] -version: [1, 2, 3] -browser: [chrome] -``` - -In the above example we have total of 3 combinations for each os i.e. -**For win:** - Version: 1, Browser: Chrome - Version: 2, Browser: Chrome - Version: 3, Browser: Chrome -**For mac:** - Version: 1, Browser: Chrome - Version: 2, Browser: Chrome - Version: 3, Browser: Chrome - -So each combination will run on a parallelism provided in the yaml. Here all combinations will run on a parallelism of 2. - -#### Platform specific parallelism -> `macParallelism`, `winParallelism`, and `linuxParallelism` - -If you want to provide different parallelism for macOS, linux and windows. If any of the specific operating system parallelism is not present it will consider the `parallelism` as the default value. - -```yaml -parallelism: 2 -linuxParallelism: 2 -winParallelism: 1 -macParallelism: 3 - -matrix: -os: [win, mac] -version: [1, 2, 3] -browser: [chrome] -``` -In the above example linux combinations will run on a parallelism on 2, `windows` combinations will run on a parallelism on 1, and `mac` combinations will run on a parallelism defined of 3. - -### `testRunnerCommand` -The `testRunnerCommand` used to run a single test entity in isolation. This entity could be a file, module, feature, or scenario. It is defined in the YAML file and tells the system how to run the test entity. - -```yaml -testRunnerCommand: mvn test -Dcucumber.options="$test" -Dscenario="$test" -DOs="win 10" - -# This command runs the test using Maven and passes in the options for Cucumber, the scenario to run, and the operating system to use. -``` - -#### Platform specific `testRunnerCommand` - -In [hybrid mode](/support/docs/hyperexecute-hybrid-strategy/), you can run your tests on multiple operating system using the same yaml. You can provide different `testRunnerCommand` for macOS, linux and windows. - -If any of the specific operating system `testRunnerCommand` is not present it will consider the `testRunnerCommand` as the default value. - -```yaml -testRunnerCommand: mvn test -Dcucumber.options="$test" -Dscenario="$test" -DOs="win 10" -linuxTestRunnerCommand: mvn test `-Dcucumber.options="$test"` `-Dscenario="$test"` `-DOs="linux"` -winTestRunnerCommand: mvn test `-Dcucumber.options="$test"` `-Dscenario="$test"` `-DOs="win 10"` -macTestRunnerCommand: mvn test -Dcucumber.options="$test" -Dscenario="$test" -DOs="mac" -``` - -## Basic Parameters - -### `cacheKey` -It is a unique identifier that enables HyperExecute to store and retrieve cached results efficiently. HyperExecute generates a hash value for the files specified in the `cacheKey` and checks the database to see if an entry with this hash already exists. - -If an entry exists and the dependencies haven't changed, HyperExecute reuses the cached directives. If no entry exists, a new one is created. Upon successful job completion, the directives are cached for future use. This process avoids redundant downloads of dependencies, ensuring faster test execution. - -```yaml -cacheKey: '{{ checksum "pom.xml" }}' -``` - -If you also use Windows, you can use **forward slashes** to specify the cache file's path if your file is inside one of the directories, as demonstrated below: - -```yaml -cacheKey: {{ checksum "dir1/dir2/package-lock.json" }} -``` - -### `cacheDirectories` -It is used to cache files that do not change frequently, such as dependency files for your tests (e.g., node_modules, .m2). By caching these files, HyperExecute can significantly speed up your test execution time in subsequent runs. - -```yaml -cacheDirectories: -- .m2 -``` - -In [version 0.2 YAML](/support/docs/hyperexecute-yaml-version0.2/), the support for caching is by default, you don't have to specify any directories to cache for faster performance. For example, in Maven, we cache the entire .m2 directory in the home folder so that subsequent tasks run faster. - -If you add the `cacheDirectories` and `cacheKey` keys in your YAML file, then the default caching gets disabled and preference is given your specified cache. - -```yaml -cacheKey: '{{ checksum "pom.xml" }}' -cacheDirectories: -- .m2 -``` - -### `env` -This is helpful to set environment variables on the machine and use it in your code or install dependencies to run your test cases. - -```yaml -env: -USERNAME: abc -PLATFORM: windows -``` - -#### Special Character Encoding - -HyperExecute VMs support special character encoding, ensuring tests with multilingual content and special characters (e.g., ñ, á, é, í, ó, ú, ü, ¿, ¡) execute correctly. You can configure this using the `env` and `pre` flags in your YAML file. - -```yaml -env: -LANG: es_ES.UTF-8 -LC_ALL: es_ES.UTF-8 - -pre: -- sudo locale-gen es_ES.UTF-8 -- sudo update-locale LANG=es_ES.UTF-8 -``` - -If you are using Java-based frameworks, you can also set the encoding via `JAVA_TOOL_OPTIONS`: -```yaml -env: -JAVA_TOOL_OPTIONS: "-Dfile.encoding=UTF-8" -``` - -### `runtime` - -The `runtime` flag is used to: -- Download and install the dependent language and framework that is needed to execute your tests. -- You can provide the language and the version you want to be installed. - -Current Languages Supported: **maven, java, dotnet, node**, **ruby**, **android-sdk**, **katalon** and **python** - -> 📕 Check the [supported versions](/support/docs/hyperexecute-supported-languages-and-packages/) of all the languages along with the respective operating system. - -```yaml -runtime: -- language: java -version: "11" -``` - -- You can also install **gauge** and **gradle** only as an additional frameworks or tools independent of the language defined. - -```yaml -runtime: -- language: java -version: 11 -addons: -- name: "gauge" -version: "1.4.0" -- name: "gradle" -version : "7.0" -``` - -- You can also provide **multiple languages** and their **specified versions** that you want installed on your machine. - -```yaml -runtime: -- language: java -version: '17' -addons: -- name: gauge -version: '1.5.6' -- name: gradle -version: '7.6' -- language: python -version: '3.10' -- language: node -version: '16' -``` - -### `retryOnFailure` -The `retryOnFailure` enables automatic retries for failed test scenarios only when the [testRunnerCommand](/support/docs/deep-dive-into-hyperexecute-yaml/#testrunnercommand) itself fails. When set to `true`, the system will retry failed commands based on the [`maxRetries`](/support/docs/deep-dive-into-hyperexecute-yaml/#maxretries) value defined. The default setting is `false`. - -```yaml -retryOnFailure: true -``` - -#### 🔍 Important Behavior Notes: -- Retries are triggered only if the test command fails. -- Retries are not triggered if the command exits successfully but the test results are marked as failed by the user. - -#### ✅ Use Case: -This feature helps avoid re-running the entire job to determine if a test failure is due to a genuine issue or a transient problem (e.g., flaky tests, network hiccups). By enabling `retryOnFailure`, only the failing command is retried, allowing for quicker feedback and more efficient test execution. - -If you are using Maven, ensure that the `testFailureIgnore` flag in your `pom.xml` is set to `false`. When `testFailureIgnore` is set to `true`, Maven treats test failures as ignored, causing HyperExecute’s retry mechanism to not trigger, even if `retryOnFailure` is enabled. - -```java -false -``` - -### `maxRetries` -The `maxRetries` key defines the number of retries allowed for a failed test scenario. It is used in conjunction with the `retryOnFailure` key. When `retryOnFailure` is set to true, `maxRetries` specifies how many times each test will be retried upon failure. - -With `maxRetries` and `retryOnFailure`, you don't need to rerun the entire job to retry failed test scenarios. Instead, your tests are automatically retried immediately after a failure, allowing you to determine if they pass in subsequent attempts. - -> You can set a value between 1 and 5 for `maxRetries`, indicating the maximum number of retries for your tests. - -```yaml -retryOnFailure: true -maxRetries: 2 -``` - -### `post` -This parameter is used for executing actions after every test execution, such as printing an output file or uploading a report via a curl API request. It's ideal for performing post-run tasks. - -```yaml -post: -- echo /output/output.log -- curl https://www.example.com -``` - ->📘 Refer to [globalPost](/support/docs/deep-dive-into-hyperexecute-yaml/#globalpost) command to perform a common global setup for all your tasks, such as clean up tasks and or killing the environments. - -### `report` -This allows you to generate a consolidated report across the VMs. To use this feature, provide `report: true`, and the relative path of the data where it is expected to store and generate the reports after your test execution. You can also define the type and frameworkName of the report inside `partialReports` as shown in below example. - -```yaml -report: true -partialReports: -location: target/surefire-reports/html -type: html -frameworkName: extent -``` - -It helps you to view the report on the dashboard itself. You can download the report either from the [jobs detail page](/support/docs/hyperexecute-guided-walkthrough/#job-details-page) or you can pass[`-–download-report`](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/#--download-report) flag in the job triggering command from [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/). - -If you want to generate multiple reports of different types [supported by HyperExecute](/support/docs/hyperexecute-job-reports/): - -```yaml -partialReports: -- location: reports/json -type: json -frameworkName: extent-native - -- location: target/surefire-reports -type: html -frameworkName: testng -``` - -Set `defaultReport` as false in the [`framework`](/support/docs/hyperexecute-yaml-version0.2/#framework) if you are using [`YAML version 0.2`](/support/docs/hyperexecute-yaml-version0.2/) and you want to generate a report using `partialReports` as shown below. -```yaml -framework: -name: maven/testng -defaultReports: false -``` - -#### Regex Support for Dynamic Report Paths -You can now use **glob-style wildcards `(**)`** in the location field of `partialReports` to dynamically match nested folders and custom report paths. This is especially useful when your test reports are generated inside directories that include date stamps, environment names, or other runtime variables. - -Example: -```yaml title="hyperexecute.yaml" -partialReports: -location: reports/**/cucumber/ -# location: target/abc/**/pqr/**/cucumber-reports -frameworkName: cucumber -type: json -``` - -Ensure that your Testrunner.java file should have the complete path from where partial reports will be added. Example: - -```java title="Testrunner.java" -plugin = { -"json:target/abc/mno/pqr/xyz/cucumber-reports/CucumberTestReport.json", -"json:target/abc/its/pqr/abc/cucumber-reports/CucumberTestReport.json" -} -``` - -- 📕 Take a closer look at the [HyperExecute Reports](/support/docs/hyperexecute-reports/) -- Understand how you can [email the generated report](/support/docs/hyperexecute-email-reports/) -- How you can generate [different types of report](/support/docs/hyperexecute-job-reports/) based on your requirements - -### `errorCategorizedOnFailureOnly` - -The `errorCategorizedOnFailureOnly` flag allows you to control the behavior of error categorization after your job is executed based on the status of your stage. By default, error categorization is applied to each stage, regardless of it's status. This means that error categorization is generated for every stage, whether it succeeds or fails. - -When you enable this flag as mentioned below, the error categorization will only be generated for stages that are not passed. - -```yaml -errorCategorizedOnFailureOnly: true -``` - -### `errorCategorizedReport` - -The `errorCategorizedReport` flag is used to export a RCA report that has been generated by the failed tests. - -```yaml -errorCategorizedReport: -enabled: true -``` - -> Refer to the [Error Categorized Report documentation](/support/docs/error-categorization-report/) for detailed instructions on generating this RCA report. - -### `jobLabel` -The `jobLabel` YAML key is used to add tags or labels to jobs. This allows you to search your jobs using the labels or tags assigned to them. - -#### Prioritize Your Job Pipeline - -To prioritize your jobs, you need to add the required priority to the jobLabel key in the YAML file e.g `high`, `low`, and `'medium'`. With `'high'` priority jobs triggered first, followed by `'medium'` priority jobs and finally `'low'` priority jobs. The values are case insensitive and the default priority is 'medium'. - -You can also use it along with your existing job labels like this: - -```yaml -jobLabel: ['chrome', 'linux', 'low'] -``` - -## Advanced Parameters - -### `mergeArtifacts` -It is used to combine the artifacts created under each task to one. -```yaml -mergeArtifacts: true -``` - -### `uploadArtefacts` -It uploads the artifacts generated from running the tests, these artifacts are uploaded, preserved and can be downloaded once the job ends. - -```yaml -uploadArtefacts: - -# Will upload your reports from target/site/** to HyperExecute as FinalReport -- name: FinalReport -path: -- target/site/** - -# Will upload your Surefire reports from target/surefire-reports/**to HyperExecute as Surefire Report -- name: Surefire Report -path: -- target/surefire-reports/** -``` - -> You can use the flag `--download-artifacts-zip` to download the single zip file for all the artifacts that are generated. - -**note** -The uploadArtefact flag is not currently supported for tests running with the **Espresso** or **XCUI** frameworks. Please be aware of this limitation when configuring your tests. - -### `globalPre` -The `globalPre` flag allows you to define a pre-execution step that runs once before any of your tasks starts. This flag ensures that all necessary setup tasks, such as installing dependencies, configuring environments, or initializing resources, are completed before test execution begins. - -#### Functionality -- Runs before any test execution starts, ensuring the environment is properly configured. -- Executes on a separate machine (VM) or the local machine, based on the [test discovery mode](/support/docs/deep-dive-into-hyperexecute-yaml/#mode) selected. -- Useful for setup tasks, such as fetching credentials, initializing databases, or downloading required files. - -#### Limitations -- This feature is supported in YAML version 0.1 and 0.2 only. -- `remote` mode is not supported for **XCUI** and **Espresso** framework jobs. Default mode for these frameworks is `local`. - -```yaml title="hyperexecute.yaml" -globalPre: -mode: remote #local or remote -commands: -- "echo 'Setting up environment'" -- "apt-get update && apt-get install -y curl" -- "curl -X POST https://api.example.com/init" -runson: win -cache: true -``` - -#### Parameters -| Parameter | Type | Description | -|-----------|------|-------------| -| mode | string | Defines where the pre-step commands will be executed. Options: [local or remote](/support/docs/deep-dive-into-hyperexecute-yaml/#mode). Default `mode` is `remote`.| -| commands | list | List of shell commands to execute before test execution begins. | -| runson | string | It specifies the operating system on which all the task would run in case of `remote mode`. By default, it is set to `linux`. | -| cache | boolean | It is used to cache the payload after all commands have executed. It is useful in cases like:     - If you want to made some modification in the payload at runtime by executing some set of commands.     - If the payload is fetched from git source. By caching the payload, git rate limiting can be avoided as the payload will only be fetched once per job with `cache: true`. | - -#### Difference between `globalPre` and `pre` flags -| Scenario | globalPre | pre | -|----------|-----------|-----| -|Purpose | Global setup (e.g., install dependencies, initialize environment) | Task-specific setup (e.g., prepare test data) | -|Execution Frequency | Executes once per entire test execution | Executes once per task | -|Execution Location | Separate VM or local machine | Inside the task environment | -|Example Usage | `apt-get update`, `docker pull` | `export ENV=staging` | - -### `globalPost` -The `globalPost` flag defines a post-execution step that runs once after all tasks have completed. This step ensures that cleanup tasks, such as removing temporary files, logging results, or notifying external systems, are performed after test execution. - -#### Functionality -- Runs after all test execution is completed, ensuring final cleanup and reporting. -- Executes on a separate machine (VM) or the local machine, based on the mode selected. -- Useful for cleanup tasks, such as deleting test artifacts, summarizing reports, or deallocating cloud resources. - -#### Limitations -- This feature is supported in YAML version 0.1 and 0.2 only. -- `remote` mode is not supported for **XCUI** and **Espresso** framework jobs. Default mode for these frameworks is `local`. - -```yaml title="hyperexecute.yaml" -globalPost: -mode: remote #local or remote -commands: -- "echo 'Cleaning up test environment'" -- "rm -rf /tmp/test-results" -- "curl -X POST https://api.example.com/cleanup" -runson: linux -``` - ->📘 `globalPost` can also download every task's artifacts onto a VM, run custom commands against them, upload the processed output to the dashboard, and email the result. See [Download and Process Job Artifacts in Global Post](/support/docs/hyperexecute-global-post-artifacts/). - -#### Parameters -| Parameter | Type | Description | -|-----------|------|-------------| -| mode | string | Defines where the post-step commands will be executed. Options: [local or remote](/support/docs/deep-dive-into-hyperexecute-yaml/#mode). Default `mode` is `remote`. | -| commands | list | List of shell commands to execute after test execution completes. | -| runson | string | It specifies the operating system on which all the task would run in case of `remote mode`. By default, it is set to `linux`. | - -#### Difference between `globalPost` and `post` flags -| Scenario | globalPost | post | -|----------|------------|------| -|Purpose | Global cleanup (e.g., remove logs, finalize reports) | Task-specific cleanup (e.g., delete temporary files) | -|Execution Frequency | Executes once after all tasks complete | Executes once per task | -|Execution Location | Separate VM or local machine | Inside the task environment | -|Example Usage | `rm -rf /logs`, `curl -X POST …` | `rm -rf temp/*` | - -### `captureScreenRecordingForScenarios` -If this key is set to true, it will record whole scenario execution, and then video is accessible from your HyperExecute dashboard. -This can be majorly used for non selenium based tests to have the recorded video of the whole scenario. -```yaml -captureScreenRecordingForScenarios: true -``` - -**info** -To avoid test failures, ensure the `video` capability is `false` before enabling the `captureScreenRecordingForScenarios` flag in your YAML file. If both `video` capability and `captureScreenRecordingForScenarios` flag is set to `true` simultaneously, the tests will fail. To prevent this conflict, always pass the capability `video: false` in your tests. - -### `skipArtifactStageIfNoTest` -It is used to gracefully handle the scenario stage inside the tasks in which there are no selenium/cdp tests executed. Instead of marking the artifact stage as failed (if artifacts are not found at the defined location, HyperExecute marks it as failed) and hence the task and job as failed, the stage would be marked as skipped and hence task and job as skipped. - -```yaml -skipArtifactStageIfNoTest: true -``` - -### `globalTimeout` - -The `globalTimeout` value determines the maximum duration (in minutes) of a Task(VM) . It can be set between 1 and 150 minutes, and has a default value of 90 minutes. - -For example, if you set the `globalTimeout` to 120 minutes, a Task(VM) that exceed this duration will be automatically terminated. If you’re expecting that running all your test-cases despite parallelism is going to take more than 90 mins, set it to an appropriate value, for example, 120. If you have tests that run for longer than the maximum limit of 150 minutes, you need to get in touch with our support team. - -```yaml -globalTimeout: 90 -``` - -### `failFast` -FailFast feature allows you to stop running your tests in case there is a series of tests failure. failFast flag can be used to abort your job if the number of consecutive test failures for that job(jobID) across multiple tasks equals or exceeds the **maxNumberOfTests** value. - -```yaml -failFast: -maxNumberOfTests: 2 -``` -In case retryOnFailure flag is enabled in yaml, then there will be multiple retries for a failed test. In that case, only the failure of last retry will be considered to increment the test failure count. - -To apply `failFast` at the **scenario** level instead of the default test level, set the `level: scenario` in the YAML configuration. - -```yaml -failFast: -maxNumberOfTests: 2 -level: scenario -``` - -> **NOTE :** You can specify `failFast` at either the test level or scenario level, depending on your requirements. Both configurations will work, but can lead to unexpected results, only one should be used at a time. - -### `base` -This feature allows you to use one YAML as a base for another YAML file. By inheriting the configurations of the base (or parent) YAML file, you don't need to specify those fields in the inheriting (child) file again. [Learn more](/support/docs/hyperexecute-inherit-config/). - -```yaml -base: -yamls: -- ./ -- ./ -``` - -### `sourcePayload` -You can use this flag to use to define the source of your test scripts so that the scripts can directly be fetched from your git directly for execution on HyperExecute. Your test code is directly sourced from your Git provider with the help of secure access tokens and only your HyperExecute YAML file is encrypted and uploaded through the HyperExecute CLI. To learn more about how this feature works, go through [this page](/support/docs/hyperexecute-how-to-configure-sourcePayload/). - -```yaml -sourcePayload: -platform: git -link: https://-------- -ref: master -accessToken: -commit: -``` - -### `hostsOverride` -This field allows you to add domain mappings which are required to be added in the local DNS entry represented by the /etc/hosts file. This is required if you want to map a custom domain name to an ip (local or otherwise) so that any requests on this domain name resolves to the provided ip on HyperExecute machines where your tests are running. For instance, suppose for a domain "example.com", global DNS is having the entry which resolves the network call to the public IP address. With hostsOverride flag you can explicitly route the network call to a local IP address, instead of allowing the call to go through a public IP address. - -```yaml -hostsOverride: -- host: example.com -ip: 127.0.0.1 -- host: example1.com -ip: 127.0.0.1 -``` - -### `frameworkStatusOnly` -Set status of scenario based on framework tests status. For instance, if you run a job where the framework uses lambda hooks to mark the status of tests and you want the status of the scenario to be based on the status of the tests, you need to set `frameworkStatusOnly : true` in the yaml. This flag will consider the status of the tests and will use this status to mark the status of scenarios giving you the ability to control the status of stages based on test status. - -### `testSuiteTimeout` -It is used to set the timeout on all scenario stages in a task. If defined, your complete test suite should get executed within this time. Its max value is 150 minutes. If you want to increase this time beyond the default maximum limit of 150 minutes, you need to contact the support team. -```yaml -testSuiteTimeout: 30 -``` -In the above example, your scenario stage should finish within 30 mins otherwise the `task` and Job would get marked as **timed out** after 30 minutes of execution. - -### `testSuiteStep` -It is used to timeout individual scenario stages in a task. Each scenario in separate tasks should get completed before this time. If you want to increase this time beyond the default maximum limit of 150 minutes, you need to contact the support team. - -```yaml -testSuiteStep: 15 -``` -In the above example, if any of your scenario stages take more than 15 minutes to execute, then it will get **timed out** . - -### `retryOptions` - -Regular Expressions provides more granular control over when test retries are triggered. You can achieve this through **`retryOptions`** flag. - -- You can specify precise error patterns using regular expressions to determine which errors should initiate retries. -- It works seamlessly with Cypress, CDP, and Selenium framework tests. -- It is supported in both YAML 0.1 and 0.2 - -#### Configuration: - -- Set `retryOnFailure: true` to activate the retry feature. -- Indicate the maximum number of retry attempts with `maxRetries: `. -- Within the `retryOptions` section, create an `errorRegexps` array to list the regular expressions that represent the errors you want to trigger retries. - -```yaml -retryOnFailure: true -maxRetries: 3 -retryOptions: -errorRegexps: ["org.openqa.selenium.NoSuchElementException"] -``` - -### `preDirectives` -This is an advanced version of `pre` where you can control **how** your pre commands should be executed in a parallel HyperExecute `Task`. If both pre and preDirectives flags are provided at the same time, then the precedence is given to the preDirectives flag. -preDirectives currently has the ability to take the following additional inputs: -- `maxRetries`: You can retry the commands that failed in the pre-step by using this directive. The numerical value assigned to this field determines the amount of times you can retry the failed pre commands. -- `commands`: The actual commands that needs to run like `npm install` or `mvn install` -- `shell`: shell to execute the commands under. This is typically helpful if you want to run your pre commands in a specific shell. For example, `powershell` for Windows or `bash` for Linux and MacOS. -- `workingDirectory`: This specifies the location of the directory in which all test discovery and execution commands will run, as well as the location of any files or directories that are created as a result of the command execution. - -```yaml -preDirectives: -commands: ['mvn -Dmaven.repo.local=./.m2 dependency:resolve', sleep 60] -maxRetries: 0 -shell: "" -workingDirectory: "" -``` - -### `postDirectives` -This is an advanced version of `post` where you can control "how" your post commands should be executed in a parallel HyperExecute Executor. If both post and postDirectives flags are provided at the same time, then the precedence is given to the postDirectives flag. - -postDirectives currently has the ability to take the following additional inputs: - -- `commands`: actual commands that needs to run like `echo /output/output.log` -- `shell`: shell to execute the commands under. This is typically helpful if you want to run your post commands in a specific shell. For example, `powershell` for Windows or `bash` for Linux and MacOS. (Coming Soon) -- `workingDirectory`: This specifies the location of the directory in which all test discovery and execution commands will run, as well as the location of any files or directories that are created as a result of the command execution. - -```yaml -postDirectives: -commands: [cat yaml/linux/v1/testng_hyperexecute_autosplit_sample.yaml] -shell: "" -workingDirectory: "" -``` - -### `alwaysRunPostSteps` - -**Problem :** Test scenarios failing led to the cancellation of post-steps, being unable to upload reports, and other actions that you need to perform after all test executions. - -**Solution :** The `alwaysRunPostSteps` flag ensures that post-steps execute even if the scenario stage fails. - -```yaml -alwaysRunPostSteps: true -``` - -### `project` - -This flag is used to set the Name of your Projects which would later allow you to see all jobs of that Project at one place. To know more about Projects [click here](/support/docs/hyperexecute-projects). - -**note** -- Once you have created a project, you can pass in your project ID and project name to trigger your job on the same project. -- If you are using [**Project Level Secrets**](/support/docs/hyperexecute-how-to-manage-project-level-secrets/), you must include the id field in your project configuration. - -```yaml -project: -name: 'demo-project' #project name -id: '1abc2345-56de-7890-f1cg-h2ij34k56789' #project id -``` - -### `differentialUpload` - -When you are working with relatively large codebases, and constantly updating and upgrading your test scripts. Getting them onto the HyperExecute platform for every run might consume extra time. - -To overcome this challenge, you can use `differentialUpload` flag, which is used to minimize the time taken to upload the codebase, especially when there are incremental changes. - -This flag optimizes codebase uploads by **fetching** only the parts of the codebase that have been **updated** or **newly added**, significantly **reducing upload times**. - -- **enabled (boolean):** Set to true to activate the optimization, and false to maintain the default behavior. - -- **ttlHours (integer):** Specifies the Time-To-Live (TTL) for the uploaded code. Users can control the duration for which the optimized upload remains active, with valid values ranging from 1 hour to 360 hours. - -When you pass the `differentialUpload` flag, it ensures that whenever you upload the **same codebase a second time with less than 75% changes, only the modified or new parts are fetched**. The rest is mapped to the previously uploaded version of the codebase. This approach is beneficial in scenarios where network issues or a large codebase contribute to slow upload times. - -> **NOTE:** The default value for **ttlHours** is 60 hours - -```yaml -project: -name: XYZ Name -differentialUpload: -enabled: #true/false -ttlHours: #int value, with possible range of values [1 hour to 360 hours] -``` - -If the project flag is not passed then the name for the project will be set to **"Default Project"**. - -By activating this feature, you can experience a significant reduction in upload times, enhancing the efficiency of running and debugging multiple jobs on HyperExecute. - -### `linkValidity` - -The **`linkValidity`** flag is used to customize the duration of validity for report and artifact URLs generated after job completion. Currently URLs are inherently valid for 48 hours post-generation but you can tailor the validity period to align with the specific requirements, ensuring you can access the associated resources within the defined timeframe. - -> **Note:** If `linkValidity` is not defined or 0, then the default link validity to access the artifacts and reports is 2 days. The maximum validity is 60 days. - -```yaml -report: true - -partialReports: -location: target/surefire-reports/html -type: html -frameworkName: extent -email: -to: -- johndoe@example.com -linkValidity: 5 -``` - -```yaml -uploadArtefacts: -- name: ExecutionSnapshots -path: -- -email: -to: -- -linkValidity: 5 -``` - -### `stripParentDirectory` -When mergeArtifacts is true, providing this flag will strip the parent directory from all the downloaded artifacts and place the artifacts in the base path. -```yaml -stripParentDirectory: true -``` - -### `generateArtifactAfterEveryStage` -With this flag artifacts will be generated after every stage,without this flag artifacts are generated after every task and avoid any overrides for artifacts of the same name. -``` bash -generateArtifactAfterEveryStage: true -``` - -### `taskIdentifierInNonConflictingArtifacts` -This flag when set to **true** specifies that the task ID should be included in the non-conflicting artifacts. The non-conflicting artifacts are those artifacts that can be safely shared between tasks. By default, the task ID is not included in the non-conflicting artifacts. - -- When the `taskIdentifierInNonConflictingArtifacts` flag is set to **true** and the `mergeArtifacts` flag is also set to **true**, the task ID will be appended to all the filenames of artifacts. This is done to ensure that the artifacts can be uniquely identified. - -- For example, if you have a task that creates a file called index.html, and the `taskIdentifierInNonConflictingArtifacts` flag is set to **true**, the file will be renamed to **index.html.task-id**. This ensures that the file can be uniquely identified, even if it is merged with other files called index.html from other tasks. - -```yaml -mergeArtifacts: true -taskIdentifierInNonConflictingArtifacts: true -``` - -- If the `taskIdentifierInNonConflictingArtifacts` flag is set to **false** and the `mergeArtifacts` flag is set to **true**, the task ID will only be appended to the filename if the filename already exists. This is done to avoid appending the task ID to the filename of an artifact that already exists. - -- For example, if you have a task that creates a file called index.html, and the `taskIdentifierInNonConflictingArtifacts` flag is set to **false**, the file will not be renamed. However, if you have another task that also creates a file called index.html, the second task will rename its file to **index.html.task-id** - -```yaml -mergeArtifacts: true -taskIdentifierInNonConflictingArtifacts: false -``` - -### `smartGrid` -If enabled, it will try to reduce the browser setup time [currently only for windows]. Default value is false. - -```yaml -smartGrid: true -``` - -After a test is run, it caches the browser name, version, and all the capabilities used for this particular state and test case. -When running a test case, it tries to fetch from cache, which browser was used for the next stage in the previous run, and launches that browser in the background. This makes sure that, when the next test is run, the browser session is already running. - -If you feel that browser setup time is more than expected, you should enable this feature. However, please note that it will reduce the browser setup time of the next test only if the current test takes enough time to launch the browser for the next test in background. - -This would work only for `version 0.1` and local discovery mode. - -As this is dependent on the previous run of a job, any change in the browser capability or test name or test order might render the cache invalid. And browser setup time reduction might not be visible in this run, rather, it would appear in the next run. - -> **Note** : Currently this is available for Windows - edge, firefox, chrome - recent 5 versions. Linux and MAC are not supported currently. - -### `scenarioCommandStatusOnly` -When enabled, it will mark the scenario as passed even if no test is associated with it or will mark the status of scenario based on the status of the scenario executed. -This can be utilised for advanced use cases where teams want to define status in HyperExecute platform based on there needs of custom frameworks. - -There are two scenarios associated with it: - -#### Scenario 1: When Sessions do not open up for certain scenarios: -- For certain scenarios that do not open up a session hence no tests are associated with that scenario, because of this the scenario gets marked as skipped and the task as partially completed. Need a way to mark those scenarios as completed/passed instead of skipped and see a green tick instead of a grey tick. - -**Solution :** When `scenarioCommandStatusOnly` is set `true` in YAML, it will mark the task as passed even though no test is associated to it. In the given screenshot **task 8** is passed even though no test is associated to it. - -#### Scenario 2: When test cases are run n no of times: -- Some times a user might runs some test cases n no of times inherently due to framework retries etc. Those test cases are considered as separate entries for us, and if one fails, scenario is marked as failed and hence the task and job is marked as fail. Need a way to update status of the task as if a test case first fails and then passes, it should be shown as passing and with green tick. Currently it shows as failed. - -**Solution :** When `scenarioCommandStatusOnly` is set `true` in YAML, it will mark the status of scenario as passed. - - Test | Edge Case | Working Case | -|:---|:--|:---| -| Test 1 | Passes | Passes | -| Test 2 | Failed | Failed | -| Test 2 Retry - (Actually Test 3 for us) | Failed | Passes | -| Test 3 | Passes | Passes | -| Final Status of Stage | Failed | Success | - -As seen in the screenshot, when one of the tests is marked as failed while the other tests are marked as passed, the overall scenario is marked as passed. - -### `cypress` -This option signifies that the test suite is a cypress test suite and should be pass as true in case of cypress suite. - -```yaml -cypress: true -``` - -### `cypressOps` -This parameter is used to pass cypress specific options from the CLI. [Learn more](/support/docs/cypressv10-on-hyperexecute/) - -Additional options include: -- **Dedicated Proxy**: This is a boolean parameter to be passed in cypressOps. When enabled user’s traffic will be passed to a dedicated proxy server, this feature has to be enabled from customer support team. -- **Network**: This is a boolean parameter which when set to true enables network proxy logs for the test. -- **GeoLocation**: This parameter is used to pass the test traffic through a specific geolocation. Value to be passed here is geolocation code -- **FullHar**: This flag generates the HAR logs that offer a comprehensive view of your application's network activity during testing. -- **reporterConfigFile**: If your Cypress project does not have a custom reporter configured, `reporterConfigFile` will automatically enable command logs using the mochawesome reporter by default. -- **ProjectName**: You can define your project name to autocreate your project in the HyperExecute dashboard. - -```yaml -cypressOps: -Build: "Hyperexecute Cypress Sample Build" -Tags: ["Hyperexecute","Cypress", "Windows", "Matrix"] -BuildTags: ["Hyperexecute-Cypress"] -Network: true -FullHar: true -geoLocation: IN -reporterConfigFile: "reporter_config.json", -ProjectName: "Cypress_Project" -``` - -### `shell` -Shell defines the shell/terminal type on which all of the commands defined under pre, post, testRunnerCommand mentioned in yaml should run. If shell is also defined under PreDirectives then for PreDirectives command that shell would be used. -```yaml -shell: bash -``` - -### `dataJsonPath` -DataJsonPaths helps to distribute data/configs over the VMs. In this you can create a json files and put configurations/data required for your suite as json array inside the file. For example: - -```yaml -[ -{ -"Username": "user1", -"Password": "pass1" -}, -{ -"Username": "user2", -"Password": "pass2" -} -] -``` - -To access the data from the JSON files, there are primarily 2 methods: - -#### 1. By reading the JSON file - -In the JSON file, we have a data object and not an array of objects, hence you can directly read the data from the file. - -```yaml -{ -"Username": "user1", -"Password": "pass1" -} -``` - -#### 2. Using via env variables - -You can use the env variables to access the defined parameters as: - -```yaml -STATIC_DATA_1_ -``` -> **NOTE:-** In the above syntax, **1** represents the file passed in the yaml file and not the data object within the file. - -For instance, to access data from **file1.json**, the syntax would be: -``` -STATIC_DATA_1_Username -``` - -Similarly, to access the data from the second file i.e **file2.json**: -``` -STATIC_DATA_2_Username -``` - -### `dataJsonBuilder` -The `dataJsonBuilder` flag in the HyperExecute YAML file is used to facilitate efficient and distributed test execution by processing JSON data. It reads JSON data from a specified file, applies optional filters, and indexes the JSON objects based on a defined key. This ensures that the data is prepared and distributed according to the testing requirements, optimizing test execution across multiple environments. - -```yaml title="hyperexecute.yaml" -dataJsonBuilder: -path: -index: -filters: -query: -attributes: -- key: -values: [, , ...] -``` - -#### Configuration -- **path (required) :** Specifies the path to the JSON file containing the data. Example: -```yaml -path: data/file/abc.json -``` -- **index (optional) :** Specifies the JSON key on which the indexing is required. Example: -```yaml -index: username -``` -- **filters (optional) :** Applies filtering on the JSON objects before indexing. Either `query` or `attributes` can be used, if both are mentioned then the `attributes` will have higher precedence. - - **query:** These are used for more complex filtering operations. If both attributes and queries are provided, attributes take precedence. This means that if there’s a conflict, the filter based on attributes will be used. Example: -```yaml -query: (username == "abc" or username == "bcd") -``` - - **attributes:** These are key-value pairs used for filtering. You can provide a list of attributes with their corresponding values, and the filtering will include only those JSON objects where each specified attribute matches one of the provided values. Attributes are applied with an "AND" logic between different keys and an "OR" logic within the same key. Example: -```yaml -attributes: -- key: username -value: ["abc", "bcd"] -- key: tags -values: ["@x", "@y"] -``` - -#### Working of `dataJsonBuilder` - -- The **input JSON data** should follow this format: -```javascript title="sample.json" -[ -{ -"accesskey": "jhscuystc7ewgucu79as8yc9", -"username": "abc", -"tags": "@x" -}, -{ -"accesskey": "cjdy87328yeiqhd93urd28hh", -"username": "bcd", -"tags": "@y" -}, -{ -"accesskey": "jhscuystc7ewgucu79as8yc9", -"username": "abc", -"tags": "@y" -}, -{ -"accesskey": "cjdy87328yeiqhd93urd28hh", -"username": "bcd", -"tags": "@a" -} -] -``` - -- **`dataJsonBuilder`** flag passed in the HyperExecute YAML file: - -```yaml title="hyperexecute.yaml" -dataJsonBuilder: -path: sample.json -filters: -attributes: -- key: username -values: ["abc", "bcd"] -- key: tags -values: ["@x", "@y"] -``` - -- The **filtered JSON data** will be: -```javascript title="sample.json" -[ -{ -"accesskey": "jhscuystc7ewgucu79as8yc9", -"username": "abc", -"tags": "@x" -}, -{ -"accesskey": "cjdy87328yeiqhd93urd28hh", -"username": "bcd", -"tags": "@y" -}, -{ -"accesskey": "jhscuystc7ewgucu79as8yc9", -"username": "abc", -"tags": "@y" -} -] -``` -The objects with **usernames** `"abc"` and `"bcd"` and **tags** `"@x"` and `"@y"` are selected as they meet the criteria defined by the attributes filter. - -#### Indexing and Test Case Distribution: -After filtering, the JSON data is indexed to distribute test cases. Here’s how the distribution works: -- **VM Allocation:** Each filtered JSON object corresponds to a separate VM. If the filtered JSON contains 3 objects, 3 VMs are allocated. -- **Test Case Distribution:** Test cases are then distributed across these VMs. For each VM, test cases are distributed based on the `username` present in the filtered JSON objects. All test cases related to the same `username` are assigned to the VMs containing that `username`. - -So as per the above filtered JSON data: - -- **VM1** will receive all test cases related to `username: "abc"` -```javascript -{ -"accesskey": "jhscuystc7ewgucu79as8yc9", -"username": "abc", -"tags": "@x" -} -``` - -- **VM2** will receive all test cases related to `username: "bcd"`. -```javascript -{ -"accesskey": "cjdy87328yeiqhd93urd28hh", -"username": "bcd", -"tags": "@y" -} -``` - -- **VM3** will handle the JSON object: -```javascript -{ -"accesskey": "jhscuystc7ewgucu79as8yc9", -"username": "abc", -"tags": "@y" -} -``` - -Test cases related to `username: "abc"` will be split between **VM1** and **VM3**, while test cases related to `username: "bcd"` will be handled by **VM2**. - -#### Use Cases -- **Filtering and Indexing :** When both filtering and indexing are required, filters are applied first, followed by indexing of the filtered results. -```yaml -dataJsonBuilder: -path: data/file/abc.json -index: username -filters: -#either query or attributes can be used, if both are mentioned then the attributes will have higher precedence. -query: (username == "abc" or username == "bcd") -attributes: -- key: username -values: ["abc", "bcd"] -``` - -- **Only Filtering :** If only filtering is required without indexing, omit the index key. In this case, no indexing will occur, and each JSON object will be distributed in a round-robin manner among parallel executions. -```yaml -dataJsonBuilder: -path: data/file/abc.json -filters: -query: (username == "abc" or username == "bcd") -``` - -- **Only Indexing :** If only indexing is required without filtering, omit the filters section. -```yaml -dataJsonBuilder: -path: data/file/abc.json -index: username -``` - -You can access the data on the machines using environment variables: `${STATIC_DATA_1_username}` - for the **username** key - -### `strict` -If strict is set to true in yaml then the variables used in yaml must be present in either vars or environment variables on the user machine. If not present then only a warning is displayed. But in case of strict: true, the execution will throw an error. - -### `cacheTestURL` -It enables you to cache static test files in the browser. -```yaml -cacheTestURL: true -``` - -In case your test, loads some URLs which can be cached but the server or browser’s configuration prevents these static files from being cached in the browser, you can enable `cacheTestURL` in these cases. - -For instance, -if your test loads jquery static library multiple times and for some reason it is getting downloaded each time. Enable `cacheTestURL` to cache it. It will decrease file load time from the second time onwards. - -> **Note**: These cached resources are not yet shared across VMs. So, each VM has its own copy of cache. - -### `afterAll` -> This flag is deprecated, instead use [`globalPost`](/support/docs/deep-dive-into-hyperexecute-yaml/#globalpost) - -It is used to run commands after the job has finished. Currently only local directive is allowed, means that all the commands would be run on the same host on which HyperExecute CLI was run. Running commands in `afterAll` on HyperExecute VMs(remote commands) is not yet supported. Users will have access to all the artifacts when these commands would be run. -```yaml -afterAll: -local: -commands: -- stitch-artifacts.sh artifacts-directory -``` - -For instance you want to further process the artifacts and create a custom PDF. You can use `afterAll` for this purpose wherein custom commands can be invoked. Other use cases can be in case you : -- Want to run some commands after the job is finished. -- Want to run these commands from the same host from which hyperexecute-cli is run. - -## `beforeAll` -> This flag is deprecated, instead use [`globalPre`](/support/docs/deep-dive-into-hyperexecute-yaml/#globalpre) - -BeforeAll is used for running pre operations like discovery and payload_update. It can be executed either on local system or on hyperexecute beforeAll VM. -```yaml -beforeAll: -location: local/remote -type: discovery/update_payload -local: -commands: -pip install -r requirements.txt -``` -We can choose to run on local/remote(on HYP Vms) from the location command. Type can be either discovery/update_payload (type of operation to be performed) and commands will have all the commands that needs to be run for that operation. - -### `afterEachScenario` -The `afterEachScenario` flag is used to define tasks that should be executed after each test scenario. This can be particularly useful for cleanup activities, such as clearing cookies, resetting databases, or capturing logs, ensuring that each test scenario starts with a clean state. This flag allows you to specify a list of commands or scripts that will run sequentially after the completion of each scenario. This ensures that any necessary teardown steps are consistently applied, maintaining the integrity and isolation of your test scenarios. - -```yaml -afterEachScenario: -- echo "Running a script" -- pwsh run.test -``` - -### `buildConfig` - -This is used to manage hyperlink behavior based on test status. Here's a breakdown of the parameters within buildConfig: - - - **`buildPrefix`**: This parameter sets a custom prefix for dynamically generated build names. The format employs ${name} as a placeholder, which will be replaced with a specific value during configuration. - - - **`buildName`**: This parameter allows you to define a specific name for the build. Similar to buildPrefix, ${name} acts as a placeholder for a custom value. - -```yaml -buildConfig: -buildPrefix: myCustomBuildPrefix-${name} -buildName : "name=${name}" -``` - - > The value for `${name}` in the above command can be passed through the vars command either by the [CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/#--vars) as mentioned below or through [YAML](/support/docs/deep-dive-into-hyperexecute-yaml/#vars). - -**Dynamic Build Naming via CLI** - -If you prefer to set `buildPrefix` and `buildName` values through the command-line interface (CLI), the following commands can be used: - -To set `buildPrefix`: -```yaml ---labels buildPrefix --vars "name=xyz" -``` - -To set `buildName`: - -```yaml ---labels buildName --vars "name=xyz" -``` - -These commands utilize `--labels` to specify the parameter being configured and `--vars "name=xyz"` to define the value to be replaced for `${name}`. - -#### Key Pointers - -- #### Build Configuration Handling: - - If `buildConfig` is not provided, then the `build_id` column within the job table remains empty. When you specify the `buildConfig`, it populates the `build_id` column with the corresponding value. - -- #### Build Configuration Precedence: - - When both `buildConfig` and the `build` capability are defined, `buildConfig` takes priority. - -- #### Priority of Build Naming Parameters: - - If both `buildConfig.buildName` and `buildConfig.buildPrefix` are specified, preference is given to `buildPrefix`. - -- #### Association with Build Name: - - Defining `buildName` associates test results with the designated name, enabling organized tracking and management. - -- #### Dynamic Build Creation: - - Specifying `buildPrefix` results in the creation of a new build for each executed job. The build name format follows **buildPrefix-{jobID}** to ensure uniqueness. - -- #### Compatibility of Build Naming Variables: - - Both `buildPrefix` and `buildName` are compatible with vars and can be used together or independently based on your requirements. - -### `captureCSVResult` -By enabling the boolean yaml directive `​​captureCSVResult`, you can get the statistics of the time taken by Selenium Commands run across the job. It will gather the time taken by the Selenium commands, calculate the aggregated info for the statistics like Average, 95, 99 percentiles etc and publish it in the form of a separate artifact. -```yaml -captureCSVResult: true -``` - -Below mentioned custom Lambda hooks are also required to be added in the Selenium script: -lambda-start-timer=<some_label> -lambda-end-timer=<some_label> -These custom lambda hooks are basically used to track the amount of time taken by the Selenium command. - -Above performance stats artifact would be of the CSV format with headers as Label, Average, Min, Max, Median, P95, P99. -Here is a brief information about these headers: -1. **Label**: Identifier passed in the custom lambda hook to identify the Selenium command -2. **Average**: Average time taken by the Selenium command across the whole job -3. **Min**: Minimum time taken by the Selenium command across the whole job -4. **Max**: Maximum time taken by the Selenium command across the whole job -5. **Median**: Median of the time taken by the Selenium command across the whole job -6. **P95**: 95th percentile of the time taken by the Selenium command across the whole job -7. **P99**: 99th percentile of the time taken by the Selenium command across the whole job - -Here is the sample code showing how we can use the above mentioned Lambda hooks: -```yaml -((JavascriptExecutor) driver).executeScript("lambda-start-timer=t1"); - -/* Add 5 items in the list */ -Integer item_count = 15; -for (int count = 1; count <= item_count; count++) -{ -/* Enter the text box for entering the new item */ -elem_new_item.click(); -elem_new_item.sendKeys("Adding a new item " + count + Keys.ENTER); -Thread.sleep(2000); -} -Object t1 = ((JavascriptExecutor) driver).executeScript("lambda-end-timer=t1"); -System.out.println(t1); -``` - -> **Note**: t1 will denote the time taken by each selenium command between start and end. - -### `matrixEnvPrefix` -When we run a job in matrix mode, we set the keys with their resolved value as env variables in the scenario being run. The keys are not prefixed and hence, in some cases, we had found out that some variables like "os" can affect your test runs (for example in dotnet build commands). So, if one has an `os` key in `matrix`, it may affect `dotnet build` command if one doesn’t set the matrixEnvPrefix: true in yaml to have the "os" key of matrix available to us as `HE_ENV_os - -### `dynamicAllocation` -When we set dynamicAllocation true, the test cases are distributed among parallels at the runtime. This is valid in case of local test discovery. In this case, the parallels are utilised in the most optimised manner. - -```yaml -dynamicAllocation: true -``` - -### `collectLocalGitData` -It is a default flag which captures the git information of your repository that you have used to trigger the job in HyperExecute which are later utilized in AI-based Root Cause Analysis (RCA) generation. In order to turn off the collection of any git information after executing your HyperExecute jobs, you can define the value `false`. - -```yaml -collectLocalGitData: false, -``` - -### `background` -This feature allows you to run long-running **Application Server** tasks like *"running a webapp"* or a *"my-sql database server"* in the background. - -To enable this feature from the YAML file you can use either of the below mentioned methods: - -#### 1. Using `backgroundDirectives` - -```yaml -backgroundDirectives: -shell: bash -commands: -- name: YOUR_SERVICE_NAME -command: npx static-server -- name: Database -command: mysql-server -``` - -It contains the following attributes: - -- The **`shell`** property defines the terminal it should use to run the background service. - -- The **`command`** property specifies the command to be executed in the background. - -- The **`name` **property specifies the name of your Background Service. - -#### 2. Using `background` - -```yaml -background: -- npx static-server -- mysql-server -``` - -> To learn more about it, refer to the [Background Service](/support/docs/hyperexecute-background-services/) page. - -### `vars` -This method allows you to name your variables. This will make the process of modifying the YAML file easier. You can use these variables in the YAML file as ${your_variable_name}. -Below example shows how to use vars keyword and how to use the variables define under it at other places in yaml. - -```yaml -vars: -test: e2e -framework: net5.0 -runtime: win-x64 -testPath: ${test}\bin\build\${framework}\${runtime} -preDirectives: -Commands: -- echo ${framework} -``` - -> You can also use the vars method using the [CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/#--vars). - -### `tunnel` -Indicates whether to enable a tunnel for accessing your applications which are locally hosted or behind a firewall. The cli will launch a tunnel as sub process if tunnel is set to true -```yaml -tunnel: true -``` - -### `tunnelOpts` -The options to use when running the [tunnel](/support/docs/deep-dive-into-hyperexecute-yaml/#tunnel). - -| Options | Type | Description | -|---------|------|-------------| -|`args`| List of strings | The arguments to pass to your tunnel. For a list of arguments checkout our [tunnel flags](/support/docs/lambda-tunnel-modifiers/) | -|`preOnly`| Boolean | Should the tunnel be enabled for only the pre step. Default false. | -|`postOnly`| Boolean | Should the tunnel be enabled for only the post step. Default false. | -|`global`| Boolean | Should the tunnel be enabled for all the steps? Default true. | -|`systemProxy`| Boolean | Should the tunnel be OS system wide? Default false. | -|`checkTunnelOnFailure`| Boolean | Check tunnel on failure adds a check on our system to check the tunnel connection if a test fails and the tunnel is set to true. This option will retry the test 2 times if tunnel connection is flaky. | -|`--expose`| | This flag takes arguments in the form of `service_name:host_name_host_port`. When you trigger a test, there are environment variables exposed in the machine with the variable name :- 1. `service_name_PROXY_HOST` 2. `service_name_PROXY_PORT` | - -```yaml -tunnel: true -tunnelOpts: -args: -- "--verbose" -``` - -```yaml -tunnel: true -tunnelOpts: -preOnly: true -#------OR------ -postOnly: true -#------OR------ -global: true -#------OR------ -systemProxy: true -``` - -```yaml -pre: -- echo %LT_PROXY_PORT% -- echo %LT_PROXY_HOST% -- echo %MYSQL_PROXY_HOST% -- echo %MYSQL_PROXY_PORT% - -tunnel: true -tunnelOpts: -args: -- "--expose mysql:localhost:3306" -``` - -### `tunnelNames` -Specify the list of already running tunnel names which you want to use for accessing your applications which are locally hosted or behind a firewall. - -```yaml -tunnelNames: ["lambdatest_tunnel"] -``` - -### `testRunnerExecutor` -When utilizing the `testRunnerCommand` to execute a job on a Windows Virtual Machine, the default behavior is to run the command in PowerShell. However, in situations where test names include special characters, you may encounter an error like below. - -To address this, it is necessary to include this specific flag to switch the test execution from powershell to the command line as intended. - -```yaml -testRunnerExecutor: bat -``` - -### `workingDirectory` - -The `working directory` specifies the location of the directory in which all test discovery and execution commands will run, as well as the location of any files or directories that are created as a result of the command execution. If the `workingDirectory` option is not specified, then the working directory will be the directory where the YAML file is located. - -```yaml -framework: -name: maven/testng -discoveryType: method -workingDirectory: src/main -flags: -- "-Dplatname=win" -``` - -### `syncStart` -By default, parallel tasks in autosplit or matrix mode begin execution as soon as the VM is allocated. However, in certain situations (e.g., benchmarking or comparative analysis), you may want all VMs to start tasks at the same time. - -To support this, HyperExecute provides the `syncStart` directive in the job YAML. - -```yaml -execution: -syncStart: true -``` - -- When `syncStart` is enabled, the job ensures that all VMs are allocated and ready before starting scenario execution i.e. tests. -- The system waits for up to 15 minutes for all required VMs to become available. - - If all VMs are allocated within this window → tasks start together. - - If some VMs are still pending after 15 minutes → the job proceeds with the allocated VMs. - -> This feature is supported only in [`autosplit`](/support/docs/deep-dive-into-hyperexecute-yaml/#autosplit) and [`matrix`](/support/docs/deep-dive-into-hyperexecute-yaml/#matrix) mode and not in hybrid mode diff --git a/static/docs/default-list-of-mobile-and-desktop-browsers-for-automated-screenshot-testing.md b/static/docs/default-list-of-mobile-and-desktop-browsers-for-automated-screenshot-testing.md deleted file mode 100644 index e17e61915..000000000 --- a/static/docs/default-list-of-mobile-and-desktop-browsers-for-automated-screenshot-testing.md +++ /dev/null @@ -1,33 +0,0 @@ -# Default List Of Mobile And Desktop Browsers For Automated Screenshot Testing - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The default list of Mobile and Desktop browsers for [Automated Screenshot Testing](/docs/automated-screenshot-testing/) includes: - -## Default Desktop Browsers - -| S. NO. | OPERATING SYSTEM | BROWSER | BROWSER VERSION | -|--------|-------------------|-------------------|-----------------| -| 1 | Windows 10 | Google Chrome | 71 | -| 2 | Windows 10 | Google Chrome | 70 | -| 3 | Windows 10 | Internet Explorer | 11 | -| 4 | Windows 10 | Opera | 56 | -| 5 | macOS High Sierra | Safari | 11 | -| 6 | macOS Sierra | Safari | 10.1 | -| 7 | Windows 10 | Firefox | 61 | -| 8 | Windows 10 | Firefox | 60 | - -## Default Mobile Browsers - -| S. NO. | DEVICE OPERATING SYSTEM | DEVICE BRAND | DEVICE MODEL | -|--------|-------------------------|--------------|---------------| -| 1 | Android | Samsung | Galaxy Note 9 | -| 2 | Android | Samsung | Galaxy S8 | -| 3 | Android | Google | Pixel XL | -| 4 | Android | Google | Pixel | -| 5 | iOS | iOS 12.0 | iPhone XR | -| 6 | iOS | iOS 12.0 | iPhone XS | -| 7 | iOS | iOS 12.0 | iPhone XS Max | -| 8 | Windows 10 | Firefox | 60 | - ->For more information, refer to [List of Browsers](https://www.testmuai.com/list-of-browsers). diff --git a/static/docs/defect-analysis-prediction.md b/static/docs/defect-analysis-prediction.md deleted file mode 100644 index 4cbe12a68..000000000 --- a/static/docs/defect-analysis-prediction.md +++ /dev/null @@ -1,132 +0,0 @@ -# Defect Analysis and Prediction - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Modern test automation generates thousands of results per day. Without intelligent analysis, teams waste significant time manually triaging failures, chasing flaky tests, and diagnosing root causes. TestMu AI's AI-powered defect analysis and prediction capabilities automatically classify, analyze, and surface actionable insights from your test execution data, helping you move from reactive debugging to proactive quality improvement. - -This page provides an overview of the AI/ML-based capabilities available across the TestMu AI platform for defect analysis and prediction, and how they work together to improve test suite reliability. - -## How It Works - -TestMu AI applies machine learning and AI models across your test execution history to: - -1. **Detect**: Identify flaky and unreliable tests automatically using execution pattern analysis -2. **Classify**: Categorize failures by type (environment, script, application, network) using AI models -3. **Diagnose**: Pinpoint root causes of failures using LLM-powered analysis -4. **Predict**: Surface early warning signals through smart tags that flag tests trending toward failure - -These capabilities work across **Web Automation**, **App Automation**, and **HyperExecute**. - -## Flaky Test Detection - -Flaky tests, tests that produce inconsistent pass/fail results without any code change, are one of the biggest threats to test suite reliability. They erode team confidence in test results and waste time on false investigations. - -TestMu AI uses machine learning algorithms to automatically identify flaky tests by analyzing historical execution patterns across your test runs. - -### What It Does - -- Analyzes test execution history to detect inconsistent pass/fail patterns -- Identifies the specific commands or steps within a test that cause flakiness -- Tracks flakiness trends over time (Passed, Failed, Flaky categorization) -- Helps distinguish between genuine failures and environmental noise - -### How to Use It - -Flaky test detection is available in two places: - -| Location | What You Get | -|----------|--------------| -| [**Test Intelligence: Flaky Tests Detection**](/docs/test-intelligence-flakiness-test-detection/) | Deep-dive into individual flaky tests, flakiness sources, and command-level analysis | -| [**Insights: Flaky Tests Analytics**](/docs/analytics-modules-test-intelligence-flaky-test-analytics/) | Dashboard widgets showing flakiness trends, distribution, and team-level patterns | - -## Failure Categorization AI - -When tests fail, the first question is always: *"Is this a real bug, a test script issue, or an environment problem?"* Manually answering this across hundreds of failures is unsustainable. - -TestMu AI's Failure Categorization AI automatically classifies test failures into categories based on execution data, environment parameters, browser/OS combinations, and failure signatures. - -### What It Does - -- Automatically categorizes failures by type (application defect, script error, environment issue, etc.) -- Analyzes parameters like browser, OS, device, and failure patterns to determine category -- Helps teams prioritize: focus on application defects first, fix script issues separately -- Reduces manual triage time significantly - -### How to Use It - -- [**Failure Categorization AI**](/docs/analytics-test-failure-classification/): Available in Insights dashboards for Web Automation, App Automation, and HyperExecute -- [**Error Categorization Report**](/docs/error-categorization-report/): Structured failure reports for HyperExecute jobs with multiple error types - -**Prerequisite:** Add the `remark` capability in your test scripts to provide additional context to the AI model, which improves categorization accuracy. - -## AI Root Cause Analysis (AI RCA) - -Once a failure is detected and categorized, the next step is understanding *why* it happened. TestMu AI's AI RCA uses LLM-powered analysis to automatically diagnose failed tests and provide actionable fix recommendations. - -### What It Does - -- Analyzes failed test logs, screenshots, and execution data using advanced AI -- Distinguishes **primary root causes** from cascading symptoms -- Provides **actionable fix recommendations**, not just error descriptions, but specific steps to resolve -- Generates **error timelines** showing the chronological sequence of events leading to the failure -- Available across multiple surfaces for different use cases - -### Where It's Available - -| Surface | Use Case | Details | -|---------|----------|---------| -| [**Insights: AI RCA**](/docs/analytics-ai-root-cause-analysis/) | Analyze any failed test from dashboards | LLM-powered, credits-based (15-25 credits per analysis) | -| [**HyperExecute: AI Native RCA**](/docs/ai-powered-test-failure-analysis/) | Automatic RCA for HyperExecute job failures | Integrated into job results, analyzes logs automatically | -| [**SmartUI: Visual RCA**](/docs/smartui-root-cause-analysis/) | Diagnose visual regression failures | Identifies underlying causes of visual mismatches | - -## Smart Tags - -Smart Tags provide an early warning system by automatically labeling tests based on their execution patterns. Rather than waiting for tests to fail consistently, Smart Tags proactively surface tests that are trending in a concerning direction. - -### Available Tags - -| Tag | What It Means | Why It Matters | -|-----|---------------|----------------| -| **Flaky** | Inconsistent results across multiple executions | Unreliable signal, needs investigation or quarantine | -| **Always Failing** | Consistently failing across multiple executions | Likely a real defect or broken test, high priority | -| **New Failures** | Recently started failing after previously passing | Potential regression, investigate recent changes | - -### What It Does - -- Automatically assigned by the system based on execution pattern analysis -- Visible on dashboard widget drilldowns in Insights -- Helps teams prioritize which failures to investigate first -- Surfaces regression signals early before they become widespread - -### How to Use It - -- [**Smart Tags: Test Intelligence**](/docs/analytics-smart-tags-test-intelligence/): Available in Insights dashboards, requires minimum 10 test runs - -## Putting It All Together - -These capabilities are designed to work as a pipeline. Each stage feeds into the next: - -``` -Test Execution -↓ -Smart Tags (early warning: flaky, always failing, new failure) -↓ -Flaky Test Detection (deep analysis of inconsistent tests) -↓ -Failure Categorization AI (classify: app bug vs script vs environment) -↓ -AI Root Cause Analysis (diagnose root cause + recommended fix) -``` - -### Recommended Workflow - -1. **Set up Insights dashboards** with Flaky Tests and Failure Categorization widgets to monitor your test suite health continuously -2. **Review Smart Tags** on drilldowns to catch early warning signals, especially "New Failures" which may indicate regressions -3. **Use Flaky Test Detection** to identify and quarantine unreliable tests, improving the signal-to-noise ratio of your test results -4. **Apply AI RCA** to diagnosed failures to get actionable fix recommendations without manually reading through logs - -### Related Resources - -- [Test Intelligence Overview](/docs/test-intelligence-overview/) -- [Command Error Logs Analytics](/docs/test-intelligence-command-error-logs-analytics/) -- [Insights Best Practices](/docs/insights-best-practices/) diff --git a/static/docs/delete-testmu-account.md b/static/docs/delete-testmu-account.md deleted file mode 100644 index 86244ec33..000000000 --- a/static/docs/delete-testmu-account.md +++ /dev/null @@ -1,51 +0,0 @@ -# Deleting Your TestMu AI Account - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This documentation will guide you through the steps to delete your TestMu AI account. - -## Deleting Freemium Account - -If you are a TestMu AI freemium user, you can follow the below steps to delete your account. - -**Step 1**: Login to your TestMu AI account. Navigate to the upper-right corner and click on **'Profile'**. - -**Step 2**: From the dropdown, click on **Account Settings**. - -**Step 3:** To delete your TestMu AI account, head to the **Click Here** option. - -**Step 4**: An email notification will be sent to your registered e-mail address. An email will consist of the further instruction you need to follow for deleting your TestMu AI account. - -Read the instructions and reply as stated in the e-mail. - -> -I understand that the deletion is irrecoverable and I wish to delete my TestMu AI account. - -**Step 5**: Once your request is processed, your TestMu AI account will be deleted. - -## Deleting Premium Account - -If you are a premium user and wish to delete your account, you will need to cancel your current plan subscription. Once your plan gets expired, you can follow the above-mentioned steps to delete your account. For canceling the current plan subscription, refer to [Manage Subscriptions](/docs/manage-subscriptions/). - -## Deleting Premium Account Of A Team Member - -If you are a member of the TestMu AI organization's team, you will need to seek permission from the team's administrator to remove you from the team. Once removed, you can follow the steps mention in _Deleting Premium Account_ to delete your account. - -### Removing Member From The Team - -As an admin, you can remove the members from an organization. - -**Step 1**: Login to your TestMu AI account. Navigate to the upper-right corner and click on **'Profile'**. - -**Step 2**: From the dropdown, click on **Account Settings**. - -**Step 3**: Navigate to the user that you want to delete. Click on the **three dots** next to the user. - -**Step 4**: From the dropdown, Click on **'Delete'**. - -**Step 5**: A popup box will appear, Click on **'Delete'**. - - The user will be removed successfully. - -> -In case you have any questions or need any additional information regarding TestMu AI account deletion, you can contact at our **24/7 Chat Support** or mail us at [**support@testmuai.com**](mailto:support@testmuai.com). diff --git a/static/docs/deque-integration.md b/static/docs/deque-integration.md deleted file mode 100644 index 4df97c191..000000000 --- a/static/docs/deque-integration.md +++ /dev/null @@ -1,59 +0,0 @@ -# Deque axe Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Deque's axe is a world-class digital accessibility toolkit that allows you to integrate functionality into your tests to scan content and generate an a11y score. - -Learn how to automate your Web Accessibility tests with TestMu AI and Deque using Webdriver IO framework. - -1. Ensure you have the below dependency for running accessibility tests. - -```js -"@axe-core/webdriverio": "4.1.2-alpha.106" -//**or the latest version -``` - -2. The TestMu AI Service, which you probably have in your project. - -```js -"@wdio/lambdatest-service" -``` - -3. The next step will be to ensure that our TestMu AI configuration file contains the desired browser configurations and TestMu AI Username and Access Key. - -```js -// Specify LambdaTest Username and Access Key. -user: process.env.LT_USERNAME, -key: process.env.LT_ACCESS_KEY, -``` - -4. In the `wdio.conf` file below, we build an axeWdio object, which then creates a new AxeWebdriverIO instance that uses the current browser (or client) object from WDIO. This injects axe-core into the current page, and when analyze() is invoked, axe-core scans the current page content. - -```js -const AxeWebdriverIO = require('@axe-core/webdriverio').default; - -before: function (capabilities, specs, browser) { -const axeWdio = new AxeWebdriverIO({ -client: browser -}) - -// Configure Command to run axe -browser.addCommand('getAxeResults', async () => { -return axeWdio.analyze() -.then(async (result) => { - -return result -}) -.catch(err => { -console.log(err) -}) -}) - -}, -``` - - 5. Now call the below method in your tests - -```js -browser.getAxeResults() -``` diff --git a/static/docs/desired-capabilities-in-appium.md b/static/docs/desired-capabilities-in-appium.md deleted file mode 100644 index 83fc1e499..000000000 --- a/static/docs/desired-capabilities-in-appium.md +++ /dev/null @@ -1,94 +0,0 @@ -# Desired Capabilities - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Desired capabilities in [Appium](https://www.testmuai.com/appium) are essential for guiding the Appium server to create the correct test automation environment. Represented as a set of key-value pairs within a JSON object, these capabilities allow [QA automation](https://www.testmuai.com/blog/qa-automation/) engineers to specify the desired mobile platform and configuration for their test sessions. - -By setting these parameters, QAs can ensure that the Appium server accurately targets the intended test environment, facilitating effective and efficient [mobile app testing](https://www.testmuai.com/mobile-app-testing). - -> **Note:** With TestMu AI [capabilities generator](https://www.testmuai.com/capabilities-generator/) run your App Automation leveraging Appium Desktop. - -## Mandatory capabilities -| KEY | VALUES | CAPABILITY DESCRIPTION | SUPPORTED ON | -|-----|------- | ---------------------- |--------------| -| user | TYPE: STRING | Your `LT_USERNAME` | | -| accessKey | TYPE: STRING | Your `LT_Access_Key`| | -| isRealMobile | TYPE: BOOLEAN DEFAULT: FALSE`isRealMobile=TRUE` OR`isRealMobile=FALSE` | This capability is used to run the test on Real Device with True value. | | -| deviceName | TYPE: STRING `iPhone 13` | Name of the device. | | -| platformName | TYPE: STRING `ios` | Name of the OS. | | -| platformVersion | TYPE: STRING `14` | OS version. | | - -## Debugging Options -| KEY | VALUES | CAPABILITY DESCRIPTION | SUPPORTED ON | -|-----|--------| -----------------------|--------------| -| video | TYPE: BOOLEAN DEFAULT: TRUE `video=TRUE` OR `video=FALSE` | Video recording of the complete screen. | | -| devicelog | TYPE: BOOLEAN DEFAULT: FALSE `devicelog=TRUE` OR `devicelog=FALSE` | Enable Device logs. | | -| network | TYPE: BOOLEAN DEFAULT: FALSE `network=TRUE` OR `network=FALSE` | Enable Network logs. | | -| console | TYPE: Boolean DEFAULT: FALSE `console: true` | Used to fetch the console logs.| | - -## Test configuration -| KEY | VALUES | CAPABILITY DESCRIPTION | SUPPORTED ON | -|-----|--------| -----------------------|--------------| -| build | TYPE: STRING DEFAULT: Untitled `build=iOS Small Run` | You can group your tests like a job containing multiple tests. | | -| name | TYPE: STRING DEFAULT: TestID of the Test. In case the Name is not passed. `name=iphone 6 Small Run` | Name of your test. | | -| project | Will remain blank in case 'project' is not passed in capability. `project=Small Run` | You can group your builds like a project containing multiple jobs. | | -| queueTimeout | TYPE: Integer DEFAULT: 600 `queueTimeout=300` | This capability can be used to modify the Queue timeout value within a range. queueTimeout Range : 300-900. | | -| idleTimeout | TYPE: Integer DEFAULT: 120 `idleTimeout=120` | This capability can be used to modify the timeout value. | | -| deviceOrientation | TYPE: STRING DEFAULT: PORTRAIT `deviceOrientation=portrait` OR `deviceOrientation=landscape` | Change the screen orientation of the device. | | -| orientation | TYPE: STRING `orientation=auto` | This capability allows the app or browser to automatically adjust its display based on the physical orientation of the device. | | -| newCommandTimeout | TYPE: STRING DEFAULT: 60 `newCommandTimeout=60` | How long (in seconds) Appium will wait for a new command from the client before assuming the client quit and ending the session. | | -| automationName | TYPE: STRING DEFAULT (Android): UiAutomator2 DEFAULT (iOS): XCUITest `automationName = UiAutomator2` | Choose which automation engine you'd like to use. Android - `UiAutomator2`, `Flutter`. iOS - `XCUITest`, `Flutter`. | | -| eventTimings | TYPE: BOOLEAN DEFAULT: FALSE `true` | Enable or disable the reporting of the timings for various Appium-internal events (e.g., the start and end of each command, etc.). To enable, use true. The timings are then reported as events property on response to querying the current session. See the event timing docs for the the structure of this response. | | -| otherApps | TYPE: ARRAY OF STRINGS DEFAULT: [ ] or Empty Array `"otherApps":` `["lt://APP1002211081648217405891389",` `"lt://APP1002211081648217429465823"]` | Accepts a list of App URLs returned after uploading an app on the TestMu AI servers. Conditions to be satisfied:1. App should also be passed if "otherApps" is passed. 2. Length of app URL array ≤ 3.3. At max 3 other apps can be installed.4. App should not be present inside 'otherApp' array.5. No duplicates in ‘otherApp’ array. | | -| globalHttpProxy | TYPE: BOOLEAN DEFAULT: FALSE globalHttpProxy = true | This is only for Android specific applications and devices. If the customer’s app requires Proxy to enable Data Transfer that is not possible otherwise, they can reach out to the CS & LT will enable it for them from BE.This is not available for customers directly unless set from BE by LT. | | -| region | TYPE: STRING DEFAULT: Location of the nearest data center `region = US` | 1. By default, it picks the location of the nearest data center.2. If you would like to change the region of the Data Center manually, you can pass this capability. 3. Currently, only three regions are supported: Europe - "EU", USA - "US" and Asia-Pacific - "AP" | | -| waitForIdleTimeout | TYPE: BOOLEAN `waitForIdleTimeout = 0` | 1. Sets the timeout for waiting for the user interface to go into an idle state before starting a UI Automator action.2. It changes the timeout of the configuration in UIAutomator and could help to make interaction speed faster.| | -| privateCloud | TYPE: BOOLEAN DEFAULT: FALSE `privateCloud: True`| If your organization has opted for Private Cloud, then using this flag will fetch the devices from your Private Cloud. If your organization has not opted for Private Cloud, your test can fail if this capability is passed.| | -| w3c | TYPE: Boolean DEFAULT: FALSE `w3c: true` | Appium will use the W3C webdriver protocol when it's set to true.| | -| autoLaunch | TYPE: Boolean DEFAULT: TRUE `autoLaunch: false`| If the user doesn't pass this, the app will start on boot. Else, it will not open on boot.| | - -### Android -| KEY | VALUES | CAPABILITY DESCRIPTION | SUPPORTED ON | -|-----|--------| -----------------------|--------------| -| playStoreLogin | TYPE: STRINGS OF HASHMAP DEFAULT: FALSE `"playStoreLogin":``{"email":user@gmail.com","password":"ejndfdfs"}` | The capability is used to login to Play Store on Android devices. Conditions to be specified: 1. The account should not be using any kind 2 factor authentication, otherwise the test can fail. 2. The account should not require you to login using a trusted device using popup or OTP. 3. Since we clean the Device after each session, please make sure you are logging into the account during each session.| | -| appActivity | TYPE: String `com.lambdatest.proverbial/.home`| Launch the activity directly on the device using the App Activity. **Only for Android**.| | -| appPackage | TYPE: String `com.lambdatest.proverbial`| Launch the app directly on the device using the App Package. **Only for Android**.| | - -### iOS -| KEY | VALUES | CAPABILITY DESCRIPTION | SUPPORTED ON | -|-----|--------| -----------------------|--------------| -| resignApp | TYPE: BOOLEAN DEFAULT: True `resignApp = true` | 1. By default, if this capability is not passed, your app will be re-signed. 2. You can pass "false" as a capability to prevent your apps from being re-signed. 3. This is only for iOS-specific applications and devices | | -| bundleId | TYPE: String `com.whatsapp`| Launch the app directly on the device using the Bundle ID. **Only for iOS**.| | -| autoDismissAlerts | TYPE: BOOLEAN DEFAULT: False `autoDismissAlerts=TRUE` OR `autoDismissAlerts=FALSE` | Appium capability to Dismiss alerts/popups on iOS Devices. | | -| autoAcceptAlerts | TYPE: BOOLEAN DEFAULT: True `autoAcceptAlerts=TRUE` OR `autoAcceptAlerts=FALSE` | Appium capability to Accept alerts/popups on iOS Devices. | | -| noReset | TYPE: BOOLEAN DEFAULT: False `true` | Don't reset app state before this session. See [here](https://appium.readthedocs.io/en/stable/en/writing-running-appium/other/reset-strategies/) for more details. | | -## Real User conditions -| KEY | VALUES | CAPABILITY DESCRIPTION | SUPPORTED ON | -|-----|--------| -----------------------|--------------| -| language | TYPE: STRING `fr` | Language to set for iOS (XCUITest driver only) and Android. | | -| locale | TYPE: STRING `fr_CA, CA` | Locale to set for iOS (XCUITest driver only) and Android. fr_CA format (language code and country name abbreviationfor iOS). CA format (country name abbreviation) for Android. For more info, [click here.](/support/docs/list-of-supported-locales/) | | -| disableAnimation | TYPE: BOOLEAN DEFAULT: False `disableAnimation = true` | Disable all kinds of animations on the complete device [including all applications] that is running the test.| | -| lambdaMaskCommands | TYPE: Array of Strings DEFAULT: Empty Array `["setValues", "setCookies", "getCookies"]` | Used to make sure that the values aren't shown on the device. The text could be a Password.| | -| timezone | TYPE: String `timezone=UTC+13:00``timezone=UTC-0400` | Used to set the timezone on the device using the UTC Time. If this capability is not passed, the device will auto update the timezone based on the physical location of the device.| | -| geoLocation | TYPE: String `geoLocation=US``geoLocation=FR` | Used to change the geolocation of the device to the location provided. If this capability is not passed, the device will auto update the Geo Location based on the physical location of the device.| | -| enableImageInjection | TYPE: Boolean DEFAULT: false `enableImageInjection: true` | Used to enable the Image Injection for the app. Must send "media" capability along with this capability.| | -| media | TYPE: STRING `media: Path to the image`|Used to provide the media to upload the file to the app for Image Injection. If this capability is not passed, then the no media will be sent to the device. Must send "enableImageInjection" capability along with this capability.| | -| enableBluetooth | TYPE: Boolean DEFAULT: false `enableBluetooth: true` | Used to enable the bluetooth functionality during the session.| | -| location | TYPE: HashMap Python example: `location: {"lat": "28.6235192", "long": "77.3662948"}` For examples in other languages, please head over to [Capability Generator](https://www.testmuai.com/capabilities-generator/) | Used to change the GPS Geolocation of the device to the location provided. If this capability is not passed, the device will auto update the Geo Location based on the physical location of the device. It needs to be passed in a HashMap structure with `lat` & `long` being passed separately. | | - -## Device configuration -| KEY | VALUES | CAPABILITY DESCRIPTION | SUPPORTED ON | -|-----|--------| -----------------------|--------------| -| autoGrantPermissions | TYPE: BOOLEAN `autoGrantPermissions=TRUE` OR `autoGrantPermissions=FALSE` | Have Appium automatically determine which permissions your app requires and grant them to the app on install. Defaults to false. If noReset is true, this capability doesn't work. | | -| proxyUrl | TYPE: STRING `proxyUrl: 192.168.1.1:8080`| Used to change the Proxy Address of the device. Data will pass through the specified proxy address. If this capability is not passed, then there'll be no proxy being added to the device.| | - -## Network configuration -| KEY | VALUES | CAPABILITY DESCRIPTION | SUPPORTED ON | -|-----|--------| -----------------------|--------------| -| tunnel | TYPE: BOOLEAN `tunnel=TRUE` OR `tunnel=FALSE` | To test local applications with TestMu AI.| | -| tunnelName | TYPE: STRING `tunnelName=RabbitHole` | Name of the tunnel. | | -| dedicatedProxy | TYPE: BOOLEAN `dedicatedProxy=TRUE` OR `dedicatedProxy=FALSE` | Dedicated Proxy. | | -| blockDomains | TYPE: Array of Strings `"blockDomains": ["www.facebook.com", "www.amazon.com"] ` | It is used to block the mentioned domains on the device.| | - -> Got any questions? -> Please reach out at our **24x7 Chat Support** or you could also mail us at [support@testmuai.com](https://support.lambdatest.com/). diff --git a/static/docs/developer-tools.md b/static/docs/developer-tools.md deleted file mode 100644 index b9cad2c86..000000000 --- a/static/docs/developer-tools.md +++ /dev/null @@ -1,33 +0,0 @@ -# Developer Tools - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -> Pace up your cross browser debugging while performing live, interactive testing on mobile and desktop browsers at TestMu AI platform with our native browser tools. - -## How To Launch Developer Tools On Desktop? - -**Step 1:** Login to your TestMu AI account. Go to **"Real Time Test"** from the left navigation menu. - -**Step 2:** Select the test configuration of your choice. You can select from a wide variety of desktop browsers & their various versions to perform a test session. You may also choose from multiple operating system & screen resolutions for evaluating the RWD(Responsive Web Design) of the website when rendered across devices of different screen sizes. - -**Step 3:** Provide the URL to test and hit `Start` to launch your desired VM. - -**Step 4:** Once the desired VM is launched and operable, right-click on the webpage that gets loaded inside the VM and click on "**Inspect Element**". - -**Step 5:** As you click on "Inspect Element", the developer tools would pop up at the bottom of the virtual machine. - -## How To Launch Developer Tools On Mobile Browsers? - -> **Note:** Developer tools for android devices are not supported for android versions below 7.0. - -**Step 1:** Login to your TestMu AI account. Go to **"Real Time Test"** from the left navigation menu. - -**Step 2:** By default, you will be viewing the desktop machines and browsers. Click on the **mobile icon** to view mobile browsers and devices. - -**Step 3:** Provide the URL to test and select the device and configuration you want to test upon from our collection of android and iOS devices. Hit `Start` to launch your desired VM. - -**Step 4:** Once the desired VM is launched and operable, you will notice controller buttons on the left-side of the screen. Click the button with tools icon to open developer tools. - -**Step 5:** As you click on the **tools icon**, the developer tools would be launched for the webpage you are on, inside the virtual machine. - -That was all from us about our native browser developer tools. If you still have any questions for us, please feel free to let us know. Our experts are always available on chat to help you out with any roadblock regarding our product. Happy debugging! diff --git a/static/docs/device-passcode-appautomation.md b/static/docs/device-passcode-appautomation.md deleted file mode 100644 index c642e3ccc..000000000 --- a/static/docs/device-passcode-appautomation.md +++ /dev/null @@ -1,70 +0,0 @@ -# Device Passcode in App Automation on Real Devices - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -For certain use cases, you may need to perform tests on devices that are **passcode-protected**. Apps handling sensitive data (for example, **banking apps**, **enterprise apps**, or apps distributed through **MDM**) often enforce device passcodes for enhanced security. - -TestMu AI’s **Device Passcode in Automation** enables you to run such automated tests seamlessly on real devices. - -> To enable it for your organization, please contact us via **24×7 chat support** or you can also drop a mail to **support@testmuai.com**. - -## Use Cases - -- Validate apps requiring a system passcode prompt before login or secure workflows. -- Ensure authentication flows function correctly when the device is protected by a passcode. -- Automate enterprise test cases where passcodes are enforced by MDM policies. - -## Supported Devices - -### iOS Devices - -| Device Model | iOS Version | -|-----------------------|-------------| -| iPhone 16 | 18 | -| iPhone 16 Pro | 18 | -| iPhone 15 | 17 | -| iPhone 14 | 16 | -| iPhone 13 | 15 | -| iPad Air 11 (2024) | 18 | - -> We are actively working on expanding coverage to all supported iOS devices on TestMu AI Real Device Cloud. - -### Android Devices -Available on **all real devices** running on **version 11 and above**. - -## Default Passcode -- **iOS:** `123456` -- **Android:** `1234` - -## Capability to Enable Passcode - -Use the `enablePasscode` capability to configure passcode-protected devices during automation testing. - -| Capability | Type | Default | Description | -|-------------------|---------|---------|-------------| -| **enablePasscode** | Boolean | `false` | Enable passcode devices for running tests. Use `true` to start the session with a system passcode configured. | - -```python -{ -"platformName": "ios", -"deviceName": "iPhone 16", -"platformVersion": "18", -"isRealMobile": True, -#highlight-next-line -"enablePasscode": True -} -``` - -```python -{ -"platformName": "android", -"deviceName": "Samsung Galaxy S25", -"platformVersion": "15", -"isRealMobile": True, -#highlight-next-line -"enablePasscode": True -} -``` - -- Passcode entry screens are **not visible** in the stream for security reasons. You may see a blank screen briefly during automation. -- Enabling passcode for iOS Devices may increase the setup time of your test by **25–30 seconds** compared to regular sessions. diff --git a/static/docs/device-passcode-on-realdevice.md b/static/docs/device-passcode-on-realdevice.md deleted file mode 100644 index 6041d8a28..000000000 --- a/static/docs/device-passcode-on-realdevice.md +++ /dev/null @@ -1,60 +0,0 @@ -# Device Passcode on Real Devices - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -> To enable it for your organization, please contact us via **24×7 chat support** or you can also drop a mail to **support@testmuai.com**. - -TestMu AI’s **Device Passcode** feature allows you to test applications that require a device passcode as part of their security or authentication flow. -This is especially useful for apps in **Banking/Finance**, **Enterprise**, or those distributed via **MDM (Mobile Device Management)** that mandate passcode entry for sensitive workflows. - -## Use Cases - -- Verify app behavior when the device requires a passcode for login or secure actions. -- Test app workflows that trigger a system passcode prompt (e.g., after inactivity, or accessing secure sections). -- Validate app compliance with enterprise policies that enforce passcodes. -- Reproduce real-world scenarios for apps with heightened security requirements. - -## Supported Devices - -### iOS Devices - -| Device Model | iOS Version | -|-----------------------|-------------| -| iPhone 16 | 18 | -| iPhone 16 Pro | 18 | -| iPhone 15 | 17 | -| iPhone 14 | 16 | -| iPhone 13 | 15 | -| iPad Air 11 (2024) | 18 | - -> We are actively working on expanding coverage to all supported iOS devices on TestMu AI Real Device Cloud. - -### Android Devices -Available on **all real devices** running on **version 11 and above**. - -## Default Passcode -- **iOS:** `123456` -- **Android:** `1234` - -## Accessing Device Passcode in Manual Testing - -**Step 1:** (**iOS only**) On the App Testing Dashboard, click **Advanced Settings** and enable the **Device Passcode** filter. This will refine the iOS device list to show only those with passcode support. - -**Step 2:** Start a manual testing session on your selected device (iOS or Android). -- On **iOS** → Open the **iOS Settings** panel and toggle **Passcode** on. - -- On **Android** → Open the **Device Controls** panel and toggle **Passcode** on. - -Setup may take **15–30 seconds** depending on the platform. - -**Step 4:** Trigger the workflow inside your app that requires the device passcode. -- A system passcode screen will appear. -- Since passcode entry is sensitive, the keypad or the screen may not be streamed and could appear as black in your session view. -- Use your system keyboard to enter the default passcode: - - iOS: `123456` - - Android: `1234` - -**Step 5:** Continue your app testing once the **passcode** is validated. - -- Passcode input is **not visible** in the stream for security reasons. -- You may experience a blacked-out screen briefly during passcode entry. diff --git a/static/docs/devrev-integration.md b/static/docs/devrev-integration.md deleted file mode 100644 index 49bea73a6..000000000 --- a/static/docs/devrev-integration.md +++ /dev/null @@ -1,70 +0,0 @@ -# DevRev Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -DevRev is a customer relationship management (CRM) system that seamlessly integrates customer support conversations and tickets with developer issues and product enhancements. - -With the help of TestMu AI and DevRev Integration, you can file bugs while performing live-interactive testing of your web applications from the TestMu AI platform to your DevRev instance, thus saving time and effort. - -## Integrating DevRev With TestMu AI - -**Note** -Before enabling TestMu AI and DevRev integration, ensure you have administrator or user level access. - -1. Sign in to your TestMu AI account. Don't have an account, register for free. - -2. Select **Integrations** from the left sidebar. Then choose **DevRev** from the BUG TRACKER section. - -3. It will route you to the TestMu AI installing console where you need to enter DevOrg URL and DevRev Personal Access Token. - -### Getting The DevOrg URL - -1. Login to your DevRev account. - -2. Click on the avatar icon on the top-right and select **Settings**. - -3. Navigate to **General**. - -4. You will notice the DevOrg URL. Copy this URL and paste it in the TestMu AI installing console. - -### Getting The DevRev Personal Access Token - -1. Click on the avatar icon on the top-right and select **Settings**. - -2. From the Personal Access Tokens, click **New Token**. - -3. Enter the token name and choose its validity. Then click **Generate**. - -4. Copy the generated DevRev Personal Access Token. - -5. Paste the copied API token in the TestMu AI installing console. - -It will now integrate your TestMu AI account with DevRev. - -## One-Click Bug Logging With DevRev - -1. Once you are in the TestMu AI Dashboard, navigate to **Real Time Testing > Browser Testing** from the left sidebar menu. - -2. Enter the web URL that you wish to test, select testing type (Desktop/Mobile), select the browser **VERSION**, **OS**, **RESOLUTION**, and click **START**. - -A cloud-based real operating system will fire up running real browsers where you can test websites and web apps for logging bugs. - -3. If you come across any issue, click the **Bug** icon from the left in-session tool bar to capture screenshot of a web page. - -4. Using the in-built annotation tool bar, mark any issues, and tasks on a web page. Once you have marked the bugs, click **Mark as Bug** icon under the INTEGRATIONS section. - -5. To log a bug in DevRev, fill out the required information in the DevRev ticket, such as Type, Part, Priority, Summary and Description. Click **Mark As Bug**. - -It will log your bugs to the DevRev instance. - -6. Navigate to the TestMu AI Dashboard and go to **Manage > Issue Tracker**. - -8. In Issue Tracker, you will see the bugs that you just logged in DevRev along with their associated test environment details. To view your logged bugs, click on the **upper right arrow in the square button**. - -It will redirect to the DevRev instance where you can view your filed bugs along with their test environment details such as Test ID, Reproduce on TestMu AI link, Test URL, Operating System, Browser, and Resolution. - -## Uninstalling The DevRev Integration - -1. Sign in to your TestMu AI account and visit **Integrations** from the left sidebar menu. - -2. From **My Integrations**, click **REMOVE** under the DevRev block. diff --git a/static/docs/devtools-on-real-devices-app.md b/static/docs/devtools-on-real-devices-app.md deleted file mode 100644 index 562dbb6aa..000000000 --- a/static/docs/devtools-on-real-devices-app.md +++ /dev/null @@ -1,48 +0,0 @@ -# DevTools On Real Devices - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -> To enable it for your organization, please contact us via **24×7 chat support** or you can also drop a mail to **support@testmuai.com**. - -Testing hybrid apps or mobile websites that rely on embedded web views often requires advanced debugging capabilities. With TestMu AI, you can inspect and debug these web views directly on **real Android and iOS devices** using familiar tools like [**Chrome DevTools**](https://developer.chrome.com/docs/devtools/) and [**Safari Web Inspector**](https://developer.apple.com/safari/tools/#current). - -This enables seamless inspection of DOM elements, console logs, network activity, and JavaScript execution giving developers the same debugging power they enjoy on desktop browsers. - -## Access DevTools on TestMu AI Real Devices - -**Step 1:** Log into your TestMu AI dashboard and navigate to **Real Devices** > **App Testing**. - -**Step 2:** Choose a real Android or iOS device, upload your app (APK/IPA) or open a browser URL, and click **Start** to launch your session. - -**Step 3:** Once your app or mobile website loads, go to the screen with the WebView content. - -**Step 4:** In the session toolbar, click on the **Chrome DevTools** button. - -**Step 5:** A debugging window will open: -- On **Android**, this launches Chrome DevTools. -- On **iOS**, you can choose between Safari Web Inspector or Chrome DevTools. - -Use the DevTools interface to inspect HTML structure, debug JavaScript, monitor network requests, and view console logs in real time. - -## Supported Platforms - -| Platform | Debugging Tools Available | OS Versions Supported | -|----------|---------------------------|------------------------| -| Android | Chrome DevTools | Android 9 and above | -| iOS | Safari Web Inspector, Chrome DevTools | iOS 13 and above | - -With Hybrid application, release builds must enable the ability to inspect web views in your application. - -- On Android 4.4*, call WebView.setWebContentsDebuggingEnabled. -- On iOS 16.4+, set isinspectable to true on on the Webkit/WKWebView instance. - -## Use Cases - -- Inspect embedded content in hybrid mobile apps -- Debug JavaScript errors or performance issues -- Test mobile websites inside native wrappers -- Validate responsive design and layout within WebViews -- Troubleshoot third-party widget rendering or data fetching -- Test ad placements or dynamic content in app containers -- Analyze network requests, responses, and timings to optimize your network. -- View and edit DOM elements. diff --git a/static/docs/disable-screenshot-block.md b/static/docs/disable-screenshot-block.md deleted file mode 100644 index 7713c3cb5..000000000 --- a/static/docs/disable-screenshot-block.md +++ /dev/null @@ -1,52 +0,0 @@ -# Disable Screenshot Block for Android - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Introduction - -If your app doesn't allow screenshot to be taken of the app and you want to test your app's performance without any hindrance, you can use this tool. You can test your apps with this tool in our app-live tests and app automation tests. -This document details all you need to know about the tool. - -TestMu AI now supports disable screenshot block for Android real devices. This new feature allows you to test your application on our platform without any issue of video coming black in between or screenshots of the app not getting captured(due to app's properties) - -**OS Version:** Android Disable Screenshot Unblock is supported for all os versions. - -This feature is in the **beta phase**. To enable it, please reach out to our 24x7 Chat Support by clicking on **24x7 Chat Support** or you could also mail us at [support@testmuai.com](https://support.lambdatest.com/). - -## Disable Screenshot Block in Real Time App Testing - -For manual testing, **disable screenshot block** toggle will be seen as soon as the user uploads any app. - -After uploading your application, please wait for 3-5 minutes(depends of app file size) for the application file patching process to complete. - -## Disable Screenshot Block in App Automation - -For app automation also, **disable screenshot block** is enabled as soon as the user uploads any app. - -### Using desired Capabilities - -You can use the appium capability to turn the **Disable Screenshot Block** on in your application. Just add `"enableScreenshotUnblock": true` in the desired capabilities while writing the automation script. - -```java -DesiredCapabilities capabilities = new DesiredCapabilities(); -capabilities.setCapability("build","Java TestNG Android"); -capabilities.setCapability("name",platform+" "+device+" "+version); -capabilities.setCapability("deviceName", device); -capabilities.setCapability("platformVersion",version); -capabilities.setCapability("platformName", platform); -capabilities.setCapability("isRealMobile", true); -capabilities.setCapability("app", "APP_URL"); //Enter your app (.apk) url -capabilities.setCapability("deviceOrientation", "PORTRAIT"); -capabilities.setCapability("console", true); -capabilities.setCapability("network", false); -capabilities.setCapability("visual", true); -capabilities.setCapability("devicelog", true); -capabilities.setCapability("enableScreenshotUnblock",true); // for screenshot unblock -``` - -## Important Notes - -**Disable Screenshot Block** will only work for applications that were uploaded after 28th June 2023. - -> Got any questions? -> Please reach out at our **24x7 Chat Support** or you could also mail us at [support@testmuai.com](https://support.lambdatest.com/). diff --git a/static/docs/docker-tunnel-extension.md b/static/docs/docker-tunnel-extension.md deleted file mode 100644 index 0417f5105..000000000 --- a/static/docs/docker-tunnel-extension.md +++ /dev/null @@ -1,79 +0,0 @@ -# TestMu AI Docker Tunnel Extension - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI Docker Tunnel Extension allows you to perform cross browser testing of locally hosted websites and web applications across 3000+ browser and OS environments, thereby eliminating the need to manually launch the tunnel through the CLI. This allows for a seamless and hassle-free experience when establishing a secure connection for testing locally hosted websites on the TestMu AI platform. - -## Prerequisites - -1. [Install the Docker Desktop](https://www.docker.com/products/docker-desktop) based on your operating system. - -2. A TestMu AI account. Don't have an account, sign up on LambdaTest. - -## Setting up the TestMu AI Docker Tunnel Extension - -Below are the steps to configure the TestMu AI Docker Tunnel Extension: - -1. Launch the Docker Desktop and and navigate to the left sidebar under the Extensions category. From here, select the **+ Add Extensions** button to proceed further. - -2. It will redirect you to the Extensions Marketplace. Search for the *TestMu AI Tunnel* Extension and click on the **Install** button to begin the installation process. - -3. Upon the successful installation of the TestMu AI Docker Tunnel Extension, you can go to TestMu AI Docker Tunnel page either via clicking on *Open* button from the browse list or by navigating to the *TestMu AI Tunnel* from the Extensions category. - -4. Next, proceed to the TestMu AI Docker Tunnel page and click on the **Setup Tunnel** to configure the tunnel. - -5. In the *Basic* tab, you need to provide your TestMu AI Username, Access Token, and preferred Tunnel Name. Then, click on **Launch Tunnel**. - -*The respective fields can be reset using the **RESET** option, if required.* - -It will launch the TestMu AI Tunnel where you can notice different running Tunnel logs. - -*In case you wish to launch another tunnel, click on **+ Configure New Tunnel**.* - -## Local Testing with TestMu AI Docker Tunnel Extension - -Visit the [TestMu AI Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build), you'll find the Active Tunnel icon on right-side corner. Now that you have set up your TestMu AI Docker Tunnel, you can begin local testing of your website and web pages. - -Head over to our documentation to get started: [Testing Locally Hosted Pages](/support/docs/testing-locally-hosted-pages/). - -## Advanced Configurations in TestMu AI Docker Tunnel Extension - -TestMu AI Docker Tunnel also allows you to create a secure tunnel with some Advanced Configurations. - -To do so, click on the **Advanced Configurations** tab, and an entire list of settings will pop up. Here, you can add Environment, Local File Server Directory, Info API Ports, and more. - -Furthermore, when enabling **Show Proxy Config** toggle button, you can see the options to add a Proxy Host, Proxy Port, Proxy User, and Proxy Password for a remote server, office network, or for your local machine. - -Following are the features that you can leverage using TestMu AI Docker Tunnel Extension. - -* **DNS Servers**: To add a list of DNS servers separated by commas. - -* **Environment**: Specify the type of test environment, whether you are running tests on stage, pre-prod, or on prod. - -* **Local File Server Directory**: To access a local folder or directory on your local machine. - -* **Info API Ports**: It shows the tunnel API server based on the specified port. - -* **Bypass Hosts**: Specify any hosts you want to bypass from the tunnel. - -* **Log File Path**: Specify the location of tunnel log file. - -* **Connection Mode**: It lets you choose the mode of connection. - -* **Server Domain Name**: Specify your domain name of the server. - -* **Shared Tunnel**: To share the tunnel with your teammates. - -* **Verbose**: Enable you view the detailed logs. - -* **MITM**: To test websites using self-signed certificates on your local machine or internal network. - -* **Ingress Only**: It routes the incoming traffic or inbound requests via the specified proxy. - -## Uninstalling the TestMu AI Docker Tunnel Extension - -Here are steps to uninstall the TestMu AI Docker Tunnel Extension. - -1. From the TestMu AI Docker Tunnel Extension page, click the three dots menu from the left sidebar and select **Manage**. - -2. In the Manage tab, navigate to the three dots menu and click **Uninstall**. diff --git a/static/docs/docker-tunnel.md b/static/docs/docker-tunnel.md deleted file mode 100644 index 5b282edc1..000000000 --- a/static/docs/docker-tunnel.md +++ /dev/null @@ -1,104 +0,0 @@ -# Docker Tunnel - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -With the help of the Docker Tunnel, i.e., Docker and TestMu AI Tunnel, you can perform live interactive and automated cross-browser testing of your locally hosted webpage on the TestMu AI platform, with the help of 3000+ real browsers (desktop and mobile) and operating systems. - -With Docker Tunnel, you will also be able to create multiple instances of TestMu AI Tunnel. - -## Prerequisites To Get Started With TestMu AI Docker Tunnel - -1. Download and set up Docker Desktop from [official Docker website](https://www.docker.com/products/docker-desktop). -2. You also need your TestMu AI Username and Access Key. You can get these from your [TestMu AI's profile](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). - -## How To Use Docker With TestMu AI Tunnel - -Initially, you need to pull the [Docker hub image of Lambda Tunnel](https://hub.docker.com/r/lambdatest/tunnel). You can use the below command to raise the pull request. - -`docker pull lambdatest/tunnel:latest` - -In order to pull the other docker images from the hub, use the below command: - -* **Legacy Alpine:** -`docker pull lambdatest/tunnel:legacy-alpine` - -* **Alpine:** -`docker pull lambdatest/tunnel:alpine` - -* **Legacy:** -`docker pull lambdatest/tunnel:legacy` - -* **Burrow:** -`docker pull lambdatest/tunnel:burrow` - -This image supports all the [tunnel modifier flags](/docs/lambda-tunnel-modifiers/) available in the tunnel command-line binary and passes them as the entry point. Let us see some of them: - -* **Version Checking**: You can check the installed version of the TestMu AI Tunnel using the below command. - -`docker run -it lambdatest/tunnel --version` - -* **Basic Authentication and Login**: You can login into your TestMu AI Tunnel with Docker image, with the help of below command: - -`docker run -it --name lt lambdatest/tunnel --user johndoe --key XXXXXXXXXXXX` - -* **Capturing logs in mounted volume on host machine**: You can capture these logs with the help of below command: - -`docker run -it -v /mydir:/logs lambdatest/tunnel --user joendoe --key XXXXXXXX --logFile /logs/tunnel.log` - -* **Getting Tunnel Status**: You can fetch the current tunnel status using the Info API on the tunnel. - -Suppose the InfoAPI is available on the host over port 13001, then you need to expose the docker container port to host in the form 13001:8000. Use the below command to infuse the InfoAPI in the current docker image: - -`docker run -it -p 13001:8000 lambdatest/tunnel --user johndoe --key XXXXXXX --infoAPIPort 8000` - -Now, in order to fetch the current tunnel status, just execute the below command: - -`curl http://127.0.0.1:13001/api/v1.0/info` - -* You can also use **Proxy running on host machine at port 8082 having foo and bar as username and key** as per the example shown below: - -`docker run -it lambdatest/tunnel --user johndoe --key XXXXXXX --proxy-host host.docker.internal --proxy-port 8082 --proxy-user foo --proxy-pass bar` - -* You can also use Proxy running on another docker container within same default bridge network, assuming the container in which proxy is running has IP 172.17.0.2. - -> Note: The IP can be found from inside of the container or by inspecting the network to which container is attached. - -Consider the below example: - -`docker run -it lambdatest/tunnel --user johndoe --key XXXXXXX --proxy-host 172.17.0.2 --proxy-port 8082 --proxy-user foo --proxy-pass bar` - -* You can also use Proxy running on another docker container within same custom bridge network, assuming the container in which proxy is running has IP 172.17.0.2. - -> Note: When you create custom network, containers can reach each other using container names due to automatic service discovery. Assuming that custom-network already exists and container named proxy-service has a proxy server running. - -Consider the below example: - -`docker run -it lambdatest/tunnel --user johndoe --key XXXXXXX --proxy-host proxy-service --proxy-port 8082 --proxy-user foo --proxy-pass bar` - -* In order to stop the Docker tunnel, you can use the above mentioned InfoAPI, according to the command shown below: - -`curl -X DELETE http://127.0.0.1:13001/api/v1.0/stop` - -## How To Make Use Of TestMu AI Tunnel With Docker Image For Development - -By default TestMu AI tunnel can no longer access web apps running on host machines or other docker containers using *localhost* or *127.0.0.1* when you run it using docker container. This means that the test scripts need to be modified in a way that they can access the target web app, according to the docker network topology and host operating system. - -**For Linux Machines:** - -On Linux, containers can run in a special network mode called host. This network mode makes the container use host's network stack and doesn't create an isolated one for the containers. Running the following command makes the tunnel containers run with host networking and can access the host's network. The test scripts can access services running on the host machine using the localhost. - -`docker run -it --network host lambdatest/tunnel --user johndoe --key XXXXXXXXXXXX` - -**For Mac and Windows Machines:** - -Unfortunately, on both mac and windows, host networking mode is not available due to docker-machine implementation. The recommended approach to access services on the host machine is to use a special hostname *host.docker.internal*, which resolves to the host machine. - -> You can find more details on these in Docker's networking documentation for Docker Desktop on Mac and Windows respectively. - -The test scripts need to use this special hostname to access the web services running on the host machine. The most idiomatic way of testing docker infra is creating a custom bridge network and access services using their container names. This method works on all operating systems. - -## Conclusion - -Test your locally hosted pages and privately hosted pages at TestMu AI [Selenium Test Automation](https://www.testmuai.com/selenium-automation/) Platform using Docker Tunnel, via SSH based integration tunnel. If you still have any doubts or questions, please don't hesitate to contact us. Our experts are available on **24/7 chat support**. You can also drop us a mail at [support@testmuai.com](mailto:support@testmuai.com). - -Happy testing! ?? diff --git a/static/docs/download-artefacts-cypress.md b/static/docs/download-artefacts-cypress.md deleted file mode 100644 index f298ed557..000000000 --- a/static/docs/download-artefacts-cypress.md +++ /dev/null @@ -1,61 +0,0 @@ -# Downloading Artefacts For A Cypress Project - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This article guides you on how to download the artefacts for a Cypress project from TestMu AI Cloud machine. - -**Sample repo** - -In this demo we are demonstrating using TestMu AI's sample Cypress Cloud repo. You can find all the resources used for this article in the linked repo. View on GitHub - -## Downloading The Data From The TestMu AI Cloud Machine - -Assuming that you have run a Cypress test on lambdatest platform ( if not, this article [here](/support/docs/getting-started-with-cypress-testing/) will guide you in running your first Cypress test on TestMu AI), you need to follow these steps below: - -### Update TestMu AI Cypress CLI - -Before getting started, you would have to update the **lambdatest-cypress cli** using the command below: - -```bash -npm install -g lambdatest-cypress-cli -``` - -### Add downloads Capability - -Add **"downloads"** capability in the `lambdatest-config.json` file and define the files/directories that you wanted to download in this capability. You can mention multiple file paths in this capability separated by a comma. Refer to the below example for the same. - -### Add --sync Flag - -Append `-sync` flag in the run command as mentioned below and run your test on the TestMu AI platform. - -```bash -lambdatest-cypress run --sync=true -``` - -Once the test gets executed, you will find a `build_id` on the console corresponding to the executed test. - -### Get session_id In lambdatest_run.json - -Go to your `lambdatest_run.json` file and copy the `session_id`. - -```json title=lambdatest_run.json -{ -"build_id": "7724863", -"session_id": "6c899177-047e-4432-a9a7-17776e23aae3" -} -``` - -Now pass the session_id by running the following command. - -```json -lambdatest-cypress generate-report --session_id 6c899177-047e-4432-a9a7-17776e23aae3 -``` - -### View The Downloaded Folder - -- A new folder with the name **lambdatest-artifacts** will get created in the **root** directory of your project and all the files will be downloaded in this folder. - -- The file name will be created with the session_id combined with the browser name, browser version, and the test id corresponding to it. - -**Support** -That's all! In case you have any questions or need any additional information, you could reach out at our **[24X7 Chat Support]** or mail us directly at support@testmuai.com. diff --git a/static/docs/download-files-using-testmu-selenium-grid.md b/static/docs/download-files-using-testmu-selenium-grid.md deleted file mode 100644 index 763c32087..000000000 --- a/static/docs/download-files-using-testmu-selenium-grid.md +++ /dev/null @@ -1,607 +0,0 @@ -# Download Files Using TestMu AI Selenium Grid - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -While performing Selenium test automation, you may want to test the download functionality of your web-application or website. With TestMu AI Selenium Grid, you can test the download feature on 3000+ real browsers for mobile and desktop. You can download a file inside the test machine through your Selenium test automation script by Base64 encryption and decryption. - -TestMu AI Selenium Grid provides an encoded string of base64 that you can use to download any file inside the virtual machine triggered through your Selenium testing scripts. TestMu AI provides three main flags using JavascriptExecutor to: - -- Check whether the downloaded file exists in the test machine. - -```javascript -((JavascriptExecutor) driver).executeScript("lambda-file-exists=file-name.file_format"); -``` -- Retrieve file metadata such as md5 code, modified time, name, and size. -```javascript -((JavascriptExecutor) driver).executeScript("lambda-file-stats=file-name.file_format"); -``` -- Download file content using base64 encoding. -```javascript -((JavascriptExecutor) driver).executeScript("lambda-file-content=file-name.file_format"); -``` -- Lambda File List Custom Executor -```javascript -print driver.execute_script("lambda-file-list={match string with filename}"); -ie:. -print driver.execute_script("lambda-file-list=sample"); -Response: List of files in downloads dir starting with sample -``` ->Note: Include the file format with the file name in the JavaScriptExecutor. - -## Downloading File Using Selenium with Java - -Use the lambda-file hooks in Java to verify, inspect, and download files from the test VM. - -```java -import java.io.FileOutputStream; -import java.io.OutputStream; -import java.net.MalformedURLException; -import java.net.URL; -import java.util.Base64; -import java.util.HashMap; -import java.util.concurrent.TimeUnit; -import org.openqa.selenium.By; -import org.openqa.selenium.JavascriptExecutor; -import org.openqa.selenium.NoSuchElementException; -import org.openqa.selenium.WebElement; -import org.openqa.selenium.chrome.ChromeOptions; -import org.openqa.selenium.remote.DesiredCapabilities; -import org.openqa.selenium.remote.RemoteWebDriver; -import org.openqa.selenium.remote.SessionId; -import org.testng.Assert; -import org.testng.annotations.AfterTest; -import org.testng.annotations.BeforeTest; -import org.testng.annotations.Test; - -public class PullFileSelenium { -public String username = "YOUR_LAMBDATEST_USERNAME"; -public String accesskey = "YOUR_LAMBDATEST_ACCESS_KEY"; -public RemoteWebDriver driver; -public String gridURL = "@hub.lambdatest.com/wd/hub"; -String status = "passed"; - -@BeforeTest -public void setUp() throws Exception { - -ChromeOptions options = new ChromeOptions(); - -HashMap ltOptions = new HashMap(); -ltOptions.put("version", "123.0"); -ltOptions.put("platform", "Linux"); -ltOptions.put("build", "Download functionality test"); -ltOptions.put("name", "LT Test"); -ltOptions.put("network", true); // To enable network logs -ltOptions.put("visual", true); -ltOptions.put("video", true); // To enable video recording` -ltOptions.put("console", true); // To capture console logs -options.setCapability("lt:Options", ltOptions); - -try { - -driver = new RemoteWebDriver(new URL("https://" + username + ":" + accesskey + gridURL), options); - -} catch (MalformedURLException e) { -System.out.println("Invalid grid URL"); -} catch (Exception e) { -System.out.println(e.getMessage()); -} -} - -@Test() -public void fileDownload() throws Exception { -try { - -driver.get("https://file-examples.com/wp-content/storage/2017/02/file_example_XLSX_10.xlsx"); - -Thread.sleep(4000); - -Assert.assertEquals( -((JavascriptExecutor) driver).executeScript("lambda-file-exists=file_example_XLSX_10.xlsx"), -true); // file exist check - -System.out.println( -((JavascriptExecutor) driver).executeScript("lambda-file-stats=file_example_XLSX_10.xlsx")); // retrieve -// file -// stats - -String base64EncodedFile = ((JavascriptExecutor) driver) -.executeScript("lambda-file-content=file_example_XLSX_10.xlsx").toString(); // file content download -System.out.println(base64EncodedFile); - -byte[] data = Base64.getDecoder().decode(base64EncodedFile); -OutputStream stream = new FileOutputStream("file_example_XLSX_10.xlsx"); -stream.write(data); - -} catch (NoSuchElementException e) { -System.out.println(e.getMessage()); - -SessionId id = driver.getSessionId(); -System.out.println("Failed test session id: " + id.toString()); -} - -} - -@AfterTest -public void tearDown() throws Exception { -if (driver != null) { -((JavascriptExecutor) driver).executeScript("lambda-status=" + status); -driver.quit(); -} -} -} -``` - -```java -import java.io.FileOutputStream; -import java.io.OutputStream; -import java.net.MalformedURLException; -import java.net.URL; -import java.util.Base64; -import java.util.concurrent.TimeUnit; -import org.openqa.selenium.By; -import org.openqa.selenium.JavascriptExecutor; -import org.openqa.selenium.NoSuchElementException; -import org.openqa.selenium.WebElement; -import org.openqa.selenium.firefox.FirefoxOptions; -import org.openqa.selenium.remote.DesiredCapabilities; -import org.openqa.selenium.remote.RemoteWebDriver; -import org.openqa.selenium.remote.SessionId; -import org.testng.Assert; -import org.testng.annotations.AfterTest; -import org.testng.annotations.BeforeTest; -import org.testng.annotations.Test; - -public class DownloadCheck { -public String username = "user_name"; -public String accesskey = "access_Key"; -public RemoteWebDriver driver; -public String gridURL = "@hub.lambdatest.com/wd/hub"; -String status = "passed"; - -@BeforeTest -public void setUp() throws Exception { - -FirefoxOptions options = new FirefoxOptions(); -options.addPreference("browser.download.folderList", 2); -options.addPreference("browser.download.dir", "D:\\Downloads"); -options.addPreference("browser.download.useDownloadDir", true); -options.addPreference("browser.helperApps.neverAsk.saveToDisk", "image/jpeg"); - -DesiredCapabilities capabilities = new DesiredCapabilities(); -capabilities.setCapability("browserName", "firefox"); -capabilities.setCapability("version", "65"); -capabilities.setCapability("platform", "WIN10"); -capabilities.setCapability("build","Download functionality test"); -capabilities.setCapability("name", "sample test"); -capabilities.setCapability("network", true); // To enable network logs -capabilities.setCapability("visual", true); -capabilities.setCapability("video", true); // To enable video recording` -capabilities.setCapability("console", true); // To capture console logs -capabilities.setCapability("selenium_version","3.4.0"); - -capabilities.merge(options); - -try { - -driver = new RemoteWebDriver(new URL("https://" + username + ":" + accesskey + gridURL), capabilities); - -} catch (MalformedURLException e) { -System.out.println("Invalid grid URL"); -} catch (Exception e) { -System.out.println(e.getMessage()); -} -} - -@Test() -public void fileDownload() throws Exception { -try { - -driver.get("https://www.w3schools.com/tags/tryit.asp?filename=tryhtml5_a_download"); -driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS); -driver.switchTo().frame("iframeResult"); - -WebElement element = driver.findElement(By.xpath("//a[@href='/images/myw3schoolsimage.jpg']")); -element.click(); - -Thread.sleep(4000); - -Assert.assertEquals(((JavascriptExecutor) driver).executeScript("lambda-file-exists=myw3schoolsimage.jpg"), true); //file exist check - -System.out.println(((JavascriptExecutor) driver).executeScript("lambda-file-stats=myw3schoolsimage.jpg")); //retrieve file stats - -String base64EncodedFile = ((JavascriptExecutor) driver).executeScript("lambda-file-content=myw3schoolsimage.jpg").toString(); // file content download -System.out.println(base64EncodedFile); - -byte[] data = Base64.getDecoder().decode(base64EncodedFile); -OutputStream stream = new FileOutputStream("myw3cImage.jpg"); -stream.write(data); - -} catch (NoSuchElementException e) { -System.out.println(e.getMessage()); - -SessionId id = driver.getSessionId(); -System.out.println("Failed test session id: " + id.toString()); -} - -} - -@AfterTest -public void tearDown() throws Exception { -if (driver != null) { -((JavascriptExecutor) driver).executeScript("lambda-status=" + status); -driver.quit(); -} -} -} -``` - -## Downloading File Using Selenium with Python - -Use the lambda-file hooks in Python to verify and download files from the test VM. - -```python -import unittest -import time -import base64 -from selenium import webdriver -from selenium.webdriver.common.keys import Keys - -class LTAutomate(unittest.TestCase): - -def setUp(self): - -# username: Username can be found at automation dashboard -username="user_name" -# accessToken: AccessToken can be generated from automation dashboard or profile section -accessToken="access_Key" - -# gridUrl: gridUrl can be found at automation dashboard -gridUrl = "hub.lambdatest.com/wd/hub" - -desired_cap = { -'platform' : "win10", -'browserName' : "chrome", -'version' : "79.0", -# Resolution of machine -"resolution": "1024x768", -"name": "sample test", -"build": "Download functionality test", -"selenium_version" : "3.4.0", -"network": True, -"video": True, -"visual": True, -"console": True, -} - -# URL: https://{username}:{accessToken}@mobile-hub.lambdatest.com/wd/hub -url = "https://"+username+":"+accessToken+"@"+gridUrl - -print("Initiating remote driver on platform: "+desired_cap["platform"]+" browser: "+desired_cap["browserName"]+" version: "+desired_cap["version"]) -self.driver = webdriver.Remote( -desired_capabilities=desired_cap, -command_executor= url -) - -def test_download(self): - -driver = self.driver -print("Driver initiated successfully. Navigate url") -driver.get("https://www.w3schools.com/tags/tryit.asp?filename=tryhtml5_a_download") - -time.sleep(5) -driver.switch_to.frame('iframeResult') - -elem = driver.find_element_by_xpath("//a[@href='/images/myw3schoolsimage.jpg']") -elem.click() - -#file exists check -exists_status = driver.execute_script('lambda-file-exists=myw3schoolsimage.jpg') -print(exists_status) - -# get file stats -file_properties = driver.execute_script('lambda-file-stats=myw3schoolsimage.jpg') -print(file_properties) - -# download file-base64 -file_content = driver.execute_script('lambda-file-content=myw3schoolsimage.jpg') -print(file_content) - -data = base64.b64decode(file_content) -f = open("myw3cImage.jpg", "wb") -f.write(data) -driver.execute_script("lambda-status=passed") - -def tearDown(self): -""" -Quit selenium driver -""" -self.driver.quit() - -if __name__ == "__main__": -unittest.main() -``` - -## Downloading File Using Selenium with Node.js - -Use the lambda-file hooks in Node.js to verify and download files from the test VM. - -```javascript -const webdriver = require('selenium-webdriver'); - -const USERNAME = 'user_name'; - -dashboard or profile section -const KEY = 'access_Key'; - -// gridUrl: gridUrl can be found at automation dashboard -const GRID_HOST = 'hub.lambdatest.com/wd/hub'; - -const fs = require('fs'); - -function downloadFeature() { - -// Setup Input capabilities -const capabilities = { -platform: 'windows 10', -browserName: 'chrome', -version: '79.0', -resolution: '1280x800', -network: true, -visual: true, -console: true, -video: true, -name: 'Test 1', // name of the test -build: 'NodeJS build' // name of the build -} - -const gridUrl = 'https://' + USERNAME + ':' + KEY + '@' + GRID_HOST; - -const driver = new webdriver.Builder() -.usingServer(gridUrl) -.withCapabilities(capabilities) -.build(); - -driver.get('https://www.w3schools.com/tags/tryit.asp?filename=tryhtml5_a_download').then(function() { -driver.switchTo().frame("iframeResult").then(function() { -driver.findElement(webdriver.By.xpath("//a[@href='/images/myw3schoolsimage.jpg']")).click().then(function() { -driver.getTitle().then(function(title) { -// check if file exists -driver.executeScript('lambda-file-exists=myw3schoolsimage.jpg').then(function(file_exists){ -console.log(file_exists); -})// get file stats -driver.executeScript('lambda-file-stats=myw3schoolsimage.jpg').then(function(file_properties) { -console.log(file_properties); -}) -// get file base64 -driver.executeScript('lambda-file-content=myw3schoolsimage.jpg').then(function(get_file_content) { -fs.writeFile('myfile.jpg', get_file_content, {encoding: 'base64'}, function(err) { -console.log(get_file_content.toString('base64')); - -}); -}) - -driver.quit(); - -}); -}); -}); -}) -.catch(function(err){ -console.log("test failed with reason "+err) -driver.executeScript('lambda-status=failed'); -driver.quit(); -}); -} -downloadFeature(); -``` - -## Downloading File Using Selenium with C# - -Use the lambda-file hooks in C# to verify and download files from the test VM. - -```csharp -using System; -using OpenQA.Selenium; -using OpenQA.Selenium.Remote; -using NUnit.Framework; -using System.Threading; -using System.Collections.Generic; -using System.Linq.Expressions; -using System.IO; -​ -namespace NUnitSelenium -{ -[TestFixture("chrome", "84.0", "Windows 10")] - -[Parallelizable(ParallelScope.Children)] -public class NUnitSeleniumSample -{ -public static string LT_USERNAME = Environment.GetEnvironmentVariable("LT_USERNAME") ==null ? "your username" : Environment.GetEnvironmentVariable("LT_USERNAME"); -public static string LT_ACCESS_KEY = Environment.GetEnvironmentVariable("LT_ACCESS_KEY") == null ? "your accessKey" : Environment.GetEnvironmentVariable("LT_ACCESS_KEY"); -public static bool tunnel = Boolean.Parse(Environment.GetEnvironmentVariable("LT_TUNNEL")== null ? "false" : Environment.GetEnvironmentVariable("LT_TUNNEL")); -public static string build = Environment.GetEnvironmentVariable("LT_BUILD") == null ? "your build name" : Environment.GetEnvironmentVariable("LT_BUILD"); -public static string seleniumUri = "https://hub.lambdatest.com:443/wd/hub"; -​ -​ -ThreadLocal driver = new ThreadLocal(); -private String browser; -private String version; -private String os; -​ -public NUnitSeleniumSample(String browser, String version, String os) -{ -this.browser = browser; -this.version = version; -this.os = os; -} -​ -[SetUp] -public void Init() -{ - -DesiredCapabilities capabilities = new DesiredCapabilities(); -capabilities.SetCapability(CapabilityType.BrowserName, browser); -capabilities.SetCapability(CapabilityType.Version, version); -capabilities.SetCapability(CapabilityType.Platform, os); -capabilities.SetCapability("visual", true); -capabilities.SetCapability("network", true); -capabilities.SetCapability("console", true); - -​ -if (tunnel) -{ -capabilities.SetCapability("tunnel", tunnel); -} -if (build != null) -{ -capabilities.SetCapability("build", build); -} - -capabilities.SetCapability("user", LT_USERNAME); -capabilities.SetCapability("accessKey", LT_ACCESS_KEY); -​ -capabilities.SetCapability("name", -String.Format("{0}:{1}", -TestContext.CurrentContext.Test.ClassName, -TestContext.CurrentContext.Test.MethodName)); -driver.Value = new RemoteWebDriver(new Uri(seleniumUri), capabilities, TimeSpan.FromSeconds(600)); -Console.Out.WriteLine(driver); -} -​ -[Test] -public void Todotest() -{ -{ -try -{ -Console.WriteLine("Navigating to todos app."); -driver.Value.Navigate().GoToUrl("https://www.w3schools.com/tags/tryit.asp?filename=tryhtml5_a_download"); -driver.Value.SwitchTo().Frame("iframeResult"); -driver.Value.FindElement(By.XPath("//a[@href='/images/myw3schoolsimage.jpg']")).Click(); -​ - -Console.WriteLine(((IJavaScriptExecutor) driver.Value).ExecuteScript("lambda-file-stats=myw3schoolsimage.jpg")); -​ -String base64EncodedFile = ((IJavaScriptExecutor)driver.Value).ExecuteScript("lambda-file-content=myw3schoolsimage.jpg").ToString(); -Console.WriteLine(base64EncodedFile); -byte[] data = System.Convert.FromBase64String(base64EncodedFile); -base64EncodedFile = System.Text.ASCIIEncoding.ASCII.GetString(data); -Console.WriteLine(data); -} -catch(Exception e) -{ -Console.WriteLine(e); -} -} -} -​ -[TearDown] -public void Cleanup() -{ -bool passed = TestContext.CurrentContext.Result.Outcome.Status == NUnit.Framework.Interfaces.TestStatus.Passed; -try -{ - -((IJavaScriptExecutor)driver.Value).ExecuteScript("lambda-status=" + (passed ? "passed" : "failed")); -} -finally -{ - -// Terminates the remote webdriver session -driver.Value.Quit(); -} -} -} -} -``` - -## Downloading File Using Selenium with Ruby - -Use the lambda-file hooks in Ruby to verify and download files from the test VM. - -```ruby -require 'selenium-webdriver' -require 'test/unit' - -class LtTest < Test::Unit::TestCase -""" -LambdaTest selenium automation sample example -Configuration ----------- -username: Username can be found at automation dashboard -accessToken: AccessToken can be generated from automation dashboard or profile section - -Result -------- -Execute Ruby Automation Tests on LambdaTest Distributed Selenium Grid -""" - -def setup -""" -Setup remote driver -Params ----------- -platform : Supported platform - (Windows 10, Windows 8.1, Windows 8, Windows 7, macOS High Sierra, macOS Sierra, OS X El Capitan, OS X Yosemite, OS X Mavericks) -browserName : Supported platform - (chrome, firefox, Internet Explorer, MicrosoftEdge) -version : Supported list of version can be found at https://www.testmuai.com/capabilities-generator/ - -Result -------- -""" -username= "YOUR_LAMBDATEST_USERNAME" -accessToken= "YOUR_LAMBDATEST_ACCESS_KEY" -gridUrl = "hub.lambdatest.com/wd/hub" - -caps = { -:browserName => "chrome", -:version => "80.0", -:platform => "win10", -:name => "LambdaTest ruby google search name", -:build => "LambdaTest ruby google search build", -:network => false, -:visual => false, -:video => true, -:console => false -} - -puts (caps) -# URL: https://{username}:{accessToken}@hub.lambdatest.com/wd/hub -@driver = Selenium::WebDriver.for(:remote, -:url => "https://"+username+":"+accessToken+"@"+gridUrl, -:desired_capabilities => caps) -end - -def test_Login -""" -Setup remote driver -Params ----------- -Execute test: Download File from remote browser Sample in Ruby -Result -------- -File exists -""" -puts("Navigate the URL") -sleep(10) -@driver.get("https://www.w3schools.com/tags/tryit.asp?filename=tryhtml5_a_download") -@driver.switch_to.frame('iframeResult') -elem = @driver.find_element(:xpath, "//a[@href='/images/myw3schoolsimage.jpg']") -elem.click -fileExist= @driver.execute_script("lambda-file-exists=myw3schoolsimage.jpg") -puts(fileExist) -fileGet = @driver.execute_script('lambda-file-stats=myw3schoolsimage.jpg') -puts(fileGet) -fileCon = @driver.execute_script('lambda-file-content=myw3schoolsimage.jpg') -puts(fileCon) - -end - -def teardown -""" -Quit selenium driver -""" -@driver.quit -end -end -``` -Got questions? Give a shout to us. We are here for you 24/7. diff --git a/static/docs/drone-ci-integration-with-testmu.md b/static/docs/drone-ci-integration-with-testmu.md deleted file mode 100644 index bc2c85649..000000000 --- a/static/docs/drone-ci-integration-with-testmu.md +++ /dev/null @@ -1,229 +0,0 @@ -# Drone CI Integration with TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Drone CI is one of the most popular modern open-source CI/CD tools. It not only provides a continuous integration with your projects but also provides a distributed CI/CD pipeline, which is not possible with some other tools like Jenkins. So Drone is the first choice for many teams as it is a purely distributed, cloud-native, DevOps friendly, team-oriented, and highly scalable open-source CI/CD tool. - -TestMu AI now integrates with Drone CI to boost your go-to-market delivery. You can perform automated cross-browser testing with TestMu AI to ensure seamless code render through an online Selenium grid. TestMu AI provides 3000+ real browsers running through machines, on the cloud. You can perform automation testing in parallel with TestMu AI’s Selenium grid which in turn can help you to drastically trim down your test cycles. - -## Prerequisites For Integrating Drone CI With Our Online Selenium Grid - -In order to perform your tests with TestMu AI, you would need the below things to be already set up: - -1. You need to get a VPN setup from TestMu AI as Drone is not publicly accessible. Please contact our customer support or mail us at support@testmuai.com. - -2. Some Global Dependencies, such as: - - * A [Git or GitHub](https://github.com/) repository on which you want to activate the CI pipeline - - * You need to [download and install node.js](https://nodejs.org/en/) and node package manager or npm. Use the below command to install node.js with homebrew. - -``` -$ brew install node -``` - - * If already installed, update it to the latest version, using the below command. - -For this article, we have cloned and used this [Github repository](https://github.com/LambdaTest/webdriverio-selenium-sample/blob/master/.drone.yml). - -3. You also need your TestMu AI authentication credentials, which includes your TestMu AI username, and access key. These will be set up as the Secrets for configuring the pipeline. To get your Username and Access Key, go to your [TestMu AI automation dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/) and click on the "key" icon left to the help button, on the top right corner. - -## Activating A Github Repository For CI/CD - -To activate and integrate your Github repository with Drone CI on TestMu AI, you need to first setup your VPN port and activate it, since Drone is not publicly accessible. Once the VPN is activated, here is the stepwise illustration of how to activate your git repo for CI/CD: - -**Step 1:** Enter URL `https://drone.lambdatest.io/` on any browser. - -**Step 2:** Authorize your GitHub account to access drone. - -**Step 3:** Click on the Sync button to sync all the repositories. - -**Step 4:** You will see the repository which you want to use after the sync is completed. - -**Step 5:** You need to activate the repository for which you want to integrate Drone CI, by clicking on the Activate button. - -You may not be able to activate it for CI/CD. (This is because the drone admin user is robot-lt and this user should be the admin of the repository to activate it for CI/CD). You can try by clicking on the repository and then clicking on the "Activate Repository" button. - -After clicking on "Activate Repository," it might happen that the repository is not activated and states that "There was a problem" - -To resolve this error, kindly add the robot-lt user as admin of the repository and authorize it to make it activated or ask the owner of the repository to make it active. - -**Step 6:** After enabling your repository, the Settings tab will appear. Check the Trusted checkbox in the Project Settings row (this might not occur if the OAuth is already trusted) and select the Private radio button in the Project visibility row and enter the name of your YAML configuration file (in which pipelines will be defined) in the text box in Configuration row. Here we have entered the ".drone.yml" file name, as we are integrating Drone CI. - -**Step 7:** Add the Secrets that are required to execute the Drone CI. These secrets are sensitive information for the repository, such as passwords, tokens, keys, etc. You can add your own secrets in the repository secrets on the Drone server by using the drone-cli. Also, you can add these secrets from drone UI in the Settings as shown below. - -(Note: Do not forget to check this allow pull request checkbox if you have any build step to be executed on the pull request.) - -Congratulations!! Now you have successfully activated your repository for CI/CD. - -## Running Tests Using Drone CI with TestMu AI - -Now we will see an example in which we will be creating a TestMu AI Tunnel and install node in it. For this, make a YAML configuration file as see shown below: - -``` -kind: pipeline -name: Build - -trigger: -event: -- push - -steps: -- name: Tunnel -image: lambdatest/tunnel -environment: -PASSWORD: -from_secret: LT_ACCESS_KEY -USERNAME: -from_secret: LT_USERNAME -commands: -- ls -- apt-get update && \ -- apt-get upgrade -y && \ -- apt-get install -y wget unzip -- wget https://downloads.lambdatest.com/tunnel/alpha/linux/64bit/LT_Linux.zip && \ -- unzip LT_Linux.zip && \ -- rm LT_Linux.zip && \ -- chmod +x /LT -- ls -- /LT -user $USERNAME -key $PASSWORD & - -- name: Test -image: node -environment: -PASSWORD: -from_secret: LT_ACCESS_KEY -USERNAME: -from_secret: LT_USERNAME -commands: -- export LT_USERNAME=$USERNAME -- export LT_ACCESS_KEY=$PASSWORD -- npm install -``` - -Let’s try to understand what’s written in this YAML file by deconstructing it into multiple steps. - -* First of all, we are defining what type of pipeline is this. - -``` -kind: pipeline -name: Build -``` - -We are making a docker pipeline so here the type defined will be docker. (Note: To make another type of pipeline you may check drone [official documentation](https://docs.drone.io/) and steps will be almost similar.) We will also give a name to our pipeline. Here, we have given the name "Build". - -* Now we are defining when this pipeline should be executed. - -``` -trigger: -event: -- push -``` - -Here the pipeline will be triggered when the event will be a push event received through the webhook. This can be changed to pull events, commit events, etc as well. (Note: To see the full list of ways and events to restrict over pipeline execution kindly refer to official documentation) - -* Now we will be defining the pipeline steps by using the steps key in the YAML file. steps: Under this, our steps will be listed - -**Step I:** - -``` -- name: Tunnel -image: lambdatest/tunnel -environment: -PASSWORD: -from_secret: LT_ACCESS_KEY -USERNAME: -from_secret: LT_USERNAME -commands: -- ls -- apt-get update && \ -- apt-get upgrade -y && \ -- apt-get install -y wget unzip -- wget https://downloads.lambdatest.com/tunnel/alpha/linux/64bit/LT_Linux.zip && \ -- unzip LT_Linux.zip && \ -- rm LT_Linux.zip && \ -- chmod +x /LT -- ls -- /LT -user $USERNAME -key $PASSWORD & -``` - -Here we are giving a name to our step, i.e. "Tunnel". Image is used for builder containers and commands are the steps to be executed to connect to the TestMu AI Tunnel, build our source code, and make it executable. Here the LT_ACCESS_KEY and LT_USERNAME are fetched from Secrets. - -**Step II:** - -``` -- name: SampleTest -image: node -environment: -PASSWORD: -from_secret: LT_ACCESS_KEY -USERNAME: -from_secret: LT_USERNAME -commands: -- export LT_USERNAME=$USERNAME -- export LT_ACCESS_KEY=$PASSWORD -- npm install -``` - -In this step, we are just installing node on our TestMu AI Tunnel. - -Similarly, we can also write a Test to deploy our executable formed in the build step to s3 using the s3 plugin. (To see the full list of supported plugins and their documentation refer to this [link](https://docs.aws.amazon.com/s3/index.html).) - -``` -- name: upload -image: plugins/s3 -settings: -bucket: lambda-devops-use-only -region: us-east-1 -access_key: -from_secret: aws_access_key_id -secret_key: -from_secret: aws_secret_access_key -source: mobile-node-remote-client -target: /magicleap/LMRC/latest/ -acl: public-read -``` - -Here AWS credentials are fetched through the secrets. - -Now push your code and enjoy the CI/CD pipeline. - -## Execution Of The Drone CI Pipeline - -Below is the step-by-step execution of the Drone CI pipeline. - -1. Edit the ".drone.yml" and add the code as discussed above. - -2. Once the edit is complete, click on "Commit Changes" to save and commit the changes made. Drone CI is activated as soon as this step is done since we have used "push event" for triggering the Drone CI. - -3. As soon as the Drone CI is triggered (through push event in this case), a new activity can be seen in the "ACTIVITY FEED" tab. The current status of this activity will be "running" or "executing". - -4. The series of execution of steps includes cloning, followed by the provided tests in the .drone.yml file. For example, we have provided two tests by name "Tunnel" and "SampleTest". Hence the series of steps to be executed will be: - - * Clone - * Tunnel - * SampleTest - -Drone provides functionality of default clone in its workspace, and we are using the same. -If you want to restrict the default clone in the pipeline and want to use your custom clone then you can add below-mentioned YAML snippet to restrict the clone. - -``` -clone: -disable: true -``` - -5. Now based on the tests in the .drone.yml file, this activity can pass or fail. If passed, a green tick will appear on the left of the activity name. Else if failed, a red cross will appear instead, as shown below. - -6. You can view the status and details about each activity by clicking on it. After opening an activity, you can click on any of the passed or failed test cases to view the log. For example, in the below image, we have clicked on the failed test "SampleTest" and its log has appeared on its right. - -7. A successful activity will have a green tick on its left in the ACTIVITY FEED as well on the left of each step and tests. - -## Parallel Testing - -Parallel Testing is one of the most demanding features of TestMu AI Selenium Grid. By parallel testing, you can run more than one test case, simultaneously. This means that Parallel testing would allow you to execute numerous automation test cases altogether. So you execute a single test scenario across different browsers or could run different test scenarios across the same browser but with different browser versions. This would significantly trim down the time taken on your cross-browser testing activities. For running test in parallel, add the below command when you update the pipeline’s yaml file, i.e. `.drone.yml file`. - -``` -npm test -``` - -Deploy your code in a reliable manner at scale using Drone CI integration with TestMu AI, and ensure it looks robust across every browser to provide a seamless user experience to all your visitors. Happy Testing! diff --git a/static/docs/enterprise-ready.md b/static/docs/enterprise-ready.md deleted file mode 100644 index a26ced86b..000000000 --- a/static/docs/enterprise-ready.md +++ /dev/null @@ -1,290 +0,0 @@ -# Enterprise Readiness - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI is built for enterprise-scale testing. This page provides a comprehensive overview of how TestMu AI meets the requirements of enterprise buyers across security, compliance, access control, deployment, integrations, and more. - - Single Sign-On - Audit Logs - RBAC - Product Security - Deployment Options - Team Management - Integrations - Test Data Generation - Reporting & Analytics - SLA & Support - Data Protection & GDPR - Change Management - Product Assortment - -## Single Sign-On - -TestMu AI supports enterprise Single Sign-On (SSO) to centralize user authentication through your organization's identity provider. SSO eliminates the need for separate credentials and enforces your authentication policies across all TestMu AI products. - -**Capabilities:** -- **SAML 2.0** based authentication with any compatible IdP -- **Just-in-Time (JIT) provisioning**: automatically create user accounts on first login -- **SCIM provisioning**: automated user lifecycle management (create, update, deactivate) -- **Supported identity providers**: Okta, Azure AD, PingOne, JumpCloud, and any SAML 2.0 compliant IdP - -**Documentation:** -- [Getting Started with SSO](/support/docs/single-sign-on/) -- [SCIM Overview](/support/docs/scim/) -- [Okta SCIM Integration](/support/docs/scim/okta/) -- [Azure AD SCIM Integration](/support/docs/scim/azure/) -- [PingOne SCIM Integration](/support/docs/pingone-scim/) -- [JumpCloud SCIM Integration](/support/docs/jumpcloud-scim/) - -## Audit Logs - -TestMu AI provides comprehensive audit logging to track user and organization-level activity. Audit logs are essential for compliance, security investigations, and operational visibility. - -**Capabilities:** -- Track user actions across the platform (logins, test executions, configuration changes) -- Filter logs by user, action type, date range, and resource -- Export logs for external analysis or compliance reporting -- Retention: 60 days (extended retention available for Enterprise customers) - -**Documentation:** -- [Audit Logs](/support/docs/audit-logs/) -- [Data Retention Policy](/support/docs/data-retention-policy/) - -## Role-Based Access Control - -TestMu AI provides granular RBAC to control who can access, view, and manage resources across the platform. Administrators can define custom roles with fine-grained permissions at both the product and entity level. - -**Capabilities:** -- **Custom roles** with configurable permissions -- **Product-level access control**: restrict access to specific products (Web Automation, App Automation, HyperExecute, etc.) -- **Entity-level permissions**: control access to projects, test cases, test runs, and configurations -- **Organization-level role assignment**: manage roles centrally for all users -- **Pre-built roles**: Admin, Manager, User, and Guest with sensible defaults - -**Documentation:** -- [Roles and Permissions (RBAC)](/support/docs/rbac-roles-and-permissions/) - -## Product Security - -TestMu AI maintains enterprise-grade security across the platform with industry-standard certifications and encryption. - -**Certifications & Compliance:** -- **SOC 2 Type II** certified: security, availability, processing integrity, confidentiality, and privacy (report available under NDA) -- **ISO 27001** certified: information security management -- **GDPR** compliant: data privacy and protection -- **HIPAA** compliant: safeguards for protected health information (PHI) - -**Encryption:** -- **Data in transit**: TLS 1.2 or higher for all communications -- **Data at rest**: AES-256 encryption for all stored data -- **Secrets management**: HashiCorp Vault with AES-256-GCM encryption, zero-knowledge architecture -- **Credentials**: Multi-layer bcrypt hashing, no plaintext storage - -**Infrastructure Security:** -- Clean VM/device per test session, no data leakage between sessions -- Immediate data deletion from VMs/devices after test completion -- Network isolation and secure service-to-service authentication -- Private tenant deployment for AI features (Azure OpenAI) - -**Resources:** -- [Security & Trust Page](https://www.testmuai.com/trust/security#platform-security) -- [Secrets Management](/support/docs/hyperexecute-how-to-save-and-manage-secrets/) - -## Deployment Options - -TestMu AI offers flexible deployment options to meet your organization's infrastructure and compliance requirements. - -| Deployment | Description | Best For | -|-----------|-------------|----------| -| **SaaS Cloud** | Fully managed cloud platform, no infrastructure to maintain | Most organizations | -| **Private Cloud** | Dedicated cloud infrastructure with isolated resources | Organizations requiring data isolation | -| **On-Premise** | Self-hosted deployment within your own infrastructure | Highly regulated industries, air-gapped environments | - -**Key features across all deployments:** -- Private real device cloud (dedicated devices for your org) -- Network whitelisting and tunnel support for internal applications -- Custom data retention policies - -**Documentation:** -- [HyperExecute Private Cloud Setup](/support/docs/hyperexecute-private-cloud-setup/) -- [Network Whitelisting Guide](/support/docs/network-whitelisting-and-tunnel-guide/) -- [Public IP Ranges](/support/docs/testmu-public-ip/) - -### Data residency & deployment isolation - -TestMu AI supports regional data residency and tenant isolation for regulated deployments. Each organization's data is isolated by tenant ID within its own VPC, with production segregated from non-production. Private-cloud and on-premise (including Private Real Device Cloud) keep test data, app binaries and session content inside the customer boundary. Test execution can run fully air-gapped; note that AI-agent features require connectivity to TestMu AI APIs. Regional residency options (e.g. US, EU) and sovereign-cloud deployments are available on enterprise contracts. - -## Team Management - -TestMu AI provides comprehensive team management capabilities to organize users, control access, and manage testing at scale across your organization. - -**Capabilities:** -- **Organizations**: top-level entity for managing users, billing, and settings -- **Sub-Organizations**: create separate sub-orgs for departments, teams, or business units with independent analytics and concurrency tracking -- **Groups**: organize users into groups for easier access management and resource allocation -- **User Management**: invite, manage, and remove users with role-based permissions - -**Documentation:** -- [Team Management](/support/docs/team-management/) -- [Sub Organizations](/support/docs/sub-organizations/) -- [Account Management](/support/docs/account-management/) - -## Integrations - -TestMu AI integrates with the tools your enterprise already uses, from project management and CI/CD to communication and reporting. - -**Project Management:** -- **Jira**: native app with bidirectional sync, AI test generation from tickets -- **Azure DevOps**: native app with work item linking and AI generation - -**Source Control & CI/CD:** -- **GitHub App**: PR-based test triggering, auto test generation, RCA, status checks -- **Jenkins, GitHub Actions, GitLab CI, CircleCI, Azure Pipelines**: full CI/CD integration -- **HyperExecute CLI**: integrate test execution into any pipeline - -**Communication:** -- **Slack**: build and dashboard notifications -- **Microsoft Teams**: build and dashboard notifications - -**APIs:** -- RESTful APIs for test management, execution, and artifact retrieval -- Webhook support for real-time event notifications - -**Documentation:** -- [Integrations Overview](/support/docs/integrations-with-ci-cd-tools/) -- [GitHub App Integration](/support/docs/github-app-integration/) - -## Test Data Generation - -TestMu AI provides built-in capabilities to generate, manage, and protect the test data your teams use across automation. Teams can author synthetic datasets for data-driven tests, mask sensitive payloads in network logs and recordings, and parameterize tests so the same flow runs across environments and inputs without rework. - -**Capabilities:** -- **Synthetic data generation**: author datasets in KaneAI Test Manager, populate values manually, autofill with AI based on parameter names, or import from CSV. Default datasets are immutable and every dataset carries full version history with revert/restore support -- **Data masking**: mask sensitive payloads (passwords, tokens, API keys, auth headers, cookies) in HTTP network logs via the `network.mask` Selenium capability. Mask credentials, geolocation, and storage state captured in HyperExecute test recordings (Lambda Masking) -- **Parameterization**: pass dynamic values into test cases at runtime using variables, secrets (encrypted), smart variables (resolved at execution), parameters, and datasets to reuse the same test across environments such as staging, pre-prod, and prod - -**Documentation:** -- [Test Data Generation Overview](/support/docs/test-data-generation/) -- [KaneAI Datasets](/support/docs/kane-ai-using-datasets/) -- [KaneAI Variables & Parameters](/support/docs/kaneai-variables-and-parameters/) -- [Network Data Masking for Selenium](/support/docs/network-data-masking/) -- [HyperExecute Lambda Masking](/support/docs/hyperexecute-release-notes-2024/#version-231) - -## Reporting & Analytics - -TestMu AI provides enterprise-grade reporting and analytics with AI-powered insights to help QA managers and executives make data-driven decisions. - -**Capabilities:** -- **Pre-built dashboard templates**: get started instantly with common views -- **Custom dashboards**: build your own with heatmap, bar, line, table, pie, and billboard widgets -- **AI Test Intelligence**: Smart Tags (Flaky, Always Failing, New Failures), Flaky Test Detection, Failure Categorization AI -- **AI Root Cause Analysis (AI RCA)**: LLM-powered failure diagnosis with actionable fix recommendations -- **Build Insights and Comparison**: compare builds side-by-side to track quality trends -- **Sub-Organization analytics**: drill down into team-level and sub-org performance -- **Dashboard sharing**: shareable links with configurable expiry and password protection -- **Data export**: export up to 1,000 records for external analysis - -**Documentation:** -- [Insights Overview](/support/docs/analytics-overview/) -- [Custom Widgets](/support/docs/dashboards-custom-widgets/) -- [AI Root Cause Analysis](/support/docs/analytics-ai-root-cause-analysis/) -- [Defect Analysis and Prediction](/support/docs/defect-analysis-prediction/) - -### Machine-readable outputs - -TestMu AI emits JUnit XML and JSON for CI, NDJSON event streams for agents, and framework report schemas (Allure, Extent, Cucumber, Mochawesome). OpenTelemetry spans are available via Selenium telemetry logs. SARIF is not emitted natively; security findings export as JSON via REST/MCP and can be transformed to SARIF where a consumer requires it. - -## SLA & Support - -TestMu AI provides enterprise support with dedicated resources and guaranteed response times. - -**Support Tiers:** - -| Feature | Standard | Enterprise | -|---------|----------|-----------| -| Support channels | Email, Chat | Email, Chat, Phone, Dedicated Slack | -| Response time | Business hours | Priority SLA (custom) | -| Customer Success Manager | Shared | Dedicated CSM | -| Onboarding | Self-serve + docs | White-glove onboarding | -| Training | Documentation | Custom training sessions | -| Audit log retention | 60 days | Extended (custom) | -| Data retention | 60 days | Custom retention policies | - -**Contact:** -- Email: [support@testmuai.com](mailto:support@testmuai.com) -- Community: [community.testmuai.com](https://community.testmuai.com/) -- Documentation: [testmuai.com/support/docs](https://www.testmuai.com/support/docs/) - -## Data Protection & GDPR - -TestMu AI is GDPR compliant and provides comprehensive data protection controls for enterprise customers. - -**Data Protection:** -- **Data Processing Agreement (DPA)** available with Standard Contractual Clauses -- **Customer as Data Controller**: you retain ownership and control of your test data -- **TestMu AI as Data Processor**: we process data only as instructed -- **No PII/PHI storage** by default: system does not collect or require identifiable personal data - -**Data Retention:** -- Default 60-day retention across all products -- Custom retention available for Enterprise customers -- Immediate VM/device cleanup after each test session -- Data export capabilities for compliance needs - -**Privacy Controls:** -- Right to data deletion -- Data portability via APIs and exports -- Configurable retention periods -- Encrypted data storage and transmission - -**Documentation:** -- [Data Retention Policy](/support/docs/data-retention-policy/) -- [Privacy Policy](https://www.testmuai.com/legal/privacy) -- [Data Processing Agreement](https://www.testmuai.com/resources/LambdaTest_DPA_with_SCC_V2.4.pdf) - -## Change Management - -TestMu AI provides tools for controlled test execution management and scheduled rollouts, ensuring enterprises can manage changes systematically. - -**Capabilities:** -- **Scheduled Workflows**: run test suites on a schedule (one-time, recurring, with expiry dates) without CI/CD -- **Connected Workflows (Test Chains)**: chain workflows across projects for sequential execution, run regression only if smoke passes -- **Workflow Variables**: parameterize runs to reuse the same configuration across environments (staging, pre-prod, prod) -- **Feature flags**: enterprise customers can control feature availability at the org level -- **Version control integration**: GitHub and Bitbucket integration for branch-level test execution - -**Documentation:** -- [HyperExecute Workflows](/support/docs/hyperexecute-workflows/) -- [Test Chains](/support/docs/hyperexecute-test-chains/) - -## Product Assortment - -TestMu AI offers multiple product tiers designed for different organizational needs, from individual developers to large enterprises. - -| Tier | Target | Key Differentiators | -|------|--------|-------------------| -| **Free** | Individual developers, evaluation | Limited sessions, basic features | -| **Starter** | Small teams | Increased concurrency, core features | -| **Professional** | Growing teams | Full feature access, parallel execution | -| **Enterprise** | Large organizations | SSO, RBAC, Private Cloud, Dedicated Support, Custom Retention, On-Premise | - -**Enterprise-exclusive features:** -- Single Sign-On (SSO) and SCIM provisioning -- Role-Based Access Control (RBAC) with custom roles -- Private Cloud and On-Premise deployment -- Dedicated Customer Success Manager -- Custom data retention policies -- Extended audit log retention -- Org-level secrets management -- Priority support SLA - -**For pricing details**: [lambdatest.com/pricing](https://www.testmuai.com/pricing) - -## Need More Information? - -If you're evaluating TestMu AI for your enterprise and need additional details on security, compliance, architecture, or deployment options: - -- **Enterprise Sales**: Contact your account manager -- **Security Questionnaire**: Request via [support@testmuai.com](mailto:support@testmuai.com) -- **SOC 2 Report**: Available under NDA. Contact enterprise sales -- **Architecture Diagram**: Available upon request with NDA in place diff --git a/static/docs/environment-variables-supported-by-lambda-tunnel.md b/static/docs/environment-variables-supported-by-lambda-tunnel.md deleted file mode 100644 index d00abe365..000000000 --- a/static/docs/environment-variables-supported-by-lambda-tunnel.md +++ /dev/null @@ -1,21 +0,0 @@ -# Environment Variables Supported By TestMu AI Tunnel - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -You can leverage these environment variables to define your proxy settings. If these proxy environment variables are set then the tunnel will automatically detect them. You can switch command-line flags through an environment variable or .lt.yaml==$0 file with the TestMu AI tunnel. -To configure the .lt.yaml, you can refer to the section [Leverage .lt.yaml file](/docs/advanced-tunnel-features/#leverage-ltyaml-file) of the documentation [Advanced TestMu AI Tunnel Features](/docs/advanced-tunnel-features/). - -### What are Environment Variables? - -Environment variables can be used in place of command-line flags. Each operating system has its compatibility in terms of environment variables. Command-line flags restrict environment variables from being bypassed. When available, the command-line flags always take priority. - -Below is the following list of environment variables used by the TestMu AI Tunnel - -| ENVIRONMENT VARIABLE | DESCRIPTION | OPERATING SYSTEM | -| ---------------------------------- | --------------------------------------------------------------------------------------------------------- | ------------------------------- | -| `http_proxy``HTTP_PROXY` | TestMu AI tunnel uses the HTTP proxy `http://hostname:port` | Windows, Mac OS, Linux | -| `https_proxy``HTTPS_PROXY` | TestMu AI tunnel uses the HTTPS proxy `https://hostname:port` | Windows, Mac OS, Linux | -| `no_proxy``NO_PROXY` | Comma separated list of hosts to bypass proxy when using tunnel. Wild cards is supported for sub-domains. | Windows, Mac OS, Linux | -| `LT_USERNAME``LT_ACCESS_KEY` | TestMu AI UsernameTestMu AI Access Key | Windows, Mac OS, Linux, FreeBSD | - -That was all you need to know for TestMu AI tunnel proxies. In case you have any questions, feel free to share them with us. Our experts are available on **24/7 chat support**. You can also drop us a mail at **[support@testmuai.com](mailto:support@testmuai.com)**. Happy testing! 🙂 diff --git a/static/docs/error-categorization-report.md b/static/docs/error-categorization-report.md deleted file mode 100644 index f8c1a5171..000000000 --- a/static/docs/error-categorization-report.md +++ /dev/null @@ -1,20 +0,0 @@ -# Error Categorization Report - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The Error Categorization Report is a specialized report that helps simplify the process of identifying and categorizing errors in your HyperExecute job. By organizing test failures with similar characteristics, this report provides a clear and structured overview of problematic stages and associated tests for efficient error tracking and analysis. - -You can generate and download other reports as well, along with the error-report. - -## Configure YAML file to Generate the Error Categorization Report - -This report is generated when your job includes multiple error categories. To enable the error categorization report, simply pass the [`errorCategorizedReport`](/support/docs/deep-dive-into-hyperexecute-yaml/#errorcategorizedreport) flag to your YAML file: - -```yaml title="hyperexecute.yaml" -errorCategorizedReport: -enabled: true -``` - -This report does not depend on the [`report: true`](/support/docs/deep-dive-into-hyperexecute-yaml/#report) flag. It will generate the error report even if you do not pass the `report: true` flag in your YAML file. - -Now trigger your job on HyperExecute, and if your job fails (encounters any failed tests), it will generate the Error Categorization Report. This report list down all the error summaries along with the specific details. You can also check for the particular error for the specific test. diff --git a/static/docs/error-handling-kaneai.md b/static/docs/error-handling-kaneai.md deleted file mode 100644 index 5224822c5..000000000 --- a/static/docs/error-handling-kaneai.md +++ /dev/null @@ -1,157 +0,0 @@ -# Error Handling in Authoring - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This page provides a comprehensive reference for common errors encountered when authoring automated test cases for both web and app platforms. Each table below lists error categories, describes what they mean, gives examples of instructions that are not accepted, and suggests how to write correct instructions. - -Use this guide to quickly identify, understand, and correct authoring issues to ensure your test cases run smoothly. - -## Error Categories for Web and App Authoring - -### Unsupported Operation -**Description:** This command is not supported while authoring test cases. It applies to actions that cannot be executed in the authoring environment. -**Common Authoring Error:** Draw signature or Take a screenshot, Hover over the settings icon -**Suggested Ways:** Use supported web actions e.g., Use tap or click instead of hover or Try performing using slash commands in the prompt box. - -### Invalid Parameter -**Description:** The value provided is not valid for this command. The value provided is not valid for this command. This may happen if the input is missing, outside the supported range, or formatted incorrectly. -**Common Authoring Error:** Switch to tab with index 0 or Swipe top left. -**Suggested Ways:** Use: scroll up, scroll down, scroll left, or scroll right. You can also use pixel value and percentage, scroll until. - -### Nested Conditional -**Description:** Conditional logic cannot be placed inside another conditional. To avoid errors, break down the flow into separate, standalone conditions instead of nesting them. -**Common Authoring Error:** If X is visible then if Y is not visible then click it. -**Suggested Ways:** Use separate, simple conditions. E.g., If X is visible then click it. - -### Missing Context -**Description:** The system cannot determine the required state or context. This usually occurs when the instruction relies on information that hasn’t been defined, provided, or established earlier in the test. -**Common Authoring Error:** After clicking login, verify 'redirect happens' or Verify the checkbox you ticked earlier is now unchecked. -**Suggested Ways:** Assert only on current, observable states. - -### Ambiguous Relationship -**Description:** The specified property or relationship is too vague for the system to validate. Use clear and precise terms to ensure accurate validation. -**Common Authoring Error:** - -1. Check page loaded properly without errors -2. Assert if input field is editable - -**Suggested Ways:** Use specific, testable properties (e.g., visible, enabled, disabled). - -### Invalid Assertion -**Description:** The assertion provided is either not supported or is too broad to be validated. Use only allowed assertion operators and ensure the condition is specific and measurable. -**Common Authoring Error:** - -1. Assert all products are visible -2. Check all products have price - -**Suggested Ways:** Assert each product individually or specify a count. - -### Excessive Queries -**Description:** The instruction attempts to check too many elements at once. Reduce the number of queries to stay within the supported limit. -**Common Authoring Error:** Assert login, signup, user, password, continue, email is visible -**Suggested Ways:** Split into multiple instructions. - -### Instruction is Too Complex -**Description:** The instruction contains too many actions or conditions to be processed at once. Simplify it by breaking it into smaller, more manageable steps. -**Common Authoring Error:** Assert user is logged in and profile is updated and dashboard is loaded -**Suggested Ways:** Break into smaller steps. - -### Unspecified Element -**Description:** The target element is not clearly defined. Provide a specific identifier to ensure the system can locate the correct element. -**Common Authoring Error:** Click it, Type in input field or Tap it. -**Suggested Ways:** Specify the exact element, e.g., click 'Submit'. - -### Vague Instruction -**Description:** The instruction is unclear or incomplete. -**Common Authoring Error:** Waiting, qwertyu, Perform login on this page. -**Suggested Ways:** Provide a specific action or check. - -### Attribute Not Allowed -**Description:** Attribute used is not supported (e.g., button has color #333 on mobile). -**Common Authoring Error:** Using blocked or unsupported attributes. -**Suggested Ways:** Remove it or use a supported attribute. - -### Invalid Variable Syntax -**Description:** The variable is written in an incorrect format, such as using extra or mismatched braces, spaces. -**Common Authoring Error:** Wrong variable like ``${{name}}`` or ``${ user }``. -**Suggested Ways:** Always use ``${var}`` or ``{{var}}`` format. - -### Time Sensitive Assertion -**Description:** The instruction attempts to validate temporary UI elements such as toasts, spinners, or banners. These elements are short-lived and may lead to unreliable results; use stable post-conditions instead. -**Common Authoring Error:** Example: assert toast says "Forecast job deleted". -**Suggested Ways:** Instead, check something stable (e.g., row is gone, job is absent). - -### Captcha Blocked - -**Description:** If CAPTCHA is detected on screen, the test flow will be halted immediately, as interacting with CAPTCHA (e.g., solving puzzles, entering characters) is not supported. -**Common Authoring Error:** Selecting CAPTCHA checkbox or entering CAPTCHA text. -**Suggested Ways:** Generally, CAPTCHA is not present in application used for testing. If you encounter a CAPTCHA, try testing the application where CAPTCHA is disabled or not triggered. - -### Loop Detected - -**Description:** The same operation is repeated 3 or more times without any state change, indicating a possible infinite loop. -**Common Authoring Error:** Repeating the same action (e.g., clicking) without checking state. -**Suggested Ways:** Review and update the flow to ensure that each iteration results in a state change to avoid loops. - -### Persistent Loader -**Description:** If a loading spinner or progress indicator stays visible for too long (specifically, after waiting multiple times or for more than 30 seconds), it usually means the system is stuck and not progressing as expected. -**Common Authoring Error:** Waiting forever for the loader to go away without handling timeouts. -**Suggested Ways:** Implement a timeout to stop waiting after a reasonable period. Use checks that confirm when the page or content has fully loaded instead of relying only on the loader disappearing. - -### Dead Click - -**Description:** If you keep clicking the same element but nothing happens (no change on the screen, no popup, and no response), this means the click isn’t having any effect. -**Common Authoring Error:** Repeatedly clicking on an element that isn’t interactive or is disabled. -**Suggested Ways:** Make sure the element you want to click can actually be clicked and will trigger some action before clicking it. stop the flow if clicks don’t cause any response. - -### Navigation Stuck - -**Description:** If you try to navigate multiple times but nothing changes on the screen, and the system can’t confirm that the navigation succeeded, it means the navigation is stuck. -**Common Authoring Error:** repeatedly navigating without checking if the page or state actually changed. -**Suggested Ways:** check the navigation to ensure it is successful and the page or state changes as expected. - -### Lambda Error - -**Description:** This error indicates an internal issue within the system. it may be caused by problems such as creating input parameters, accessing secrets, or other internal issues. -**Common Authoring Error:** Errors related to parameter configuration, secret management, or system-level issues. -**Suggested Ways:** Please attempt to resolve the issue on your side. if the problem continues, contact support at support@testmuai.com for assistance. - -### Reached Maximum Step Limit - -**Description:** During the generative step, if the number of steps reaches the maximum limit of 50, this error occurs. -**Common Authoring Error:** Flows that are too long or contain loops causing excessive steps. -**Suggested Ways:** Simplify the flow by breaking it into smaller parts and ensure there are no infinite loops or unnecessary steps. - -### While Loop: Maximum Iterations Reached - -**Error Code:** `LOOP_MAX_LIMIT_REACHED` -**Error Message:** *While loop exceeded maximum iterations. This may be due to an issue with the loop condition or the actions within the loop.* -**Description:** A [While Loop](/support/docs/kaneai-while-loops/) ran the hard cap of **30 iterations per execution** without its condition ever becoming false. This usually means a variable referenced in the condition is never updated inside the body, or the UI state being checked is never reached. -**Common Authoring Error:** A loop condition such as `{{counter}} < 10` where no body step increments `{{counter}}`, or `while the spinner is visible` where the underlying action never dismisses the spinner. -**Suggested Ways:** Review the loop condition and body to confirm the condition can become false. Add or correct a body step that advances the condition toward termination (for example, increment the counter, or interact with the element whose state is checked). If your scenario legitimately requires more than 30 iterations, contact your TestMu AI support representative. - -### While Loop: Infinite Loop Detected - -**Error Code:** `INFINITE_LOOP_DETECTED` -**Error Message:** *The while loop appears to be an infinite loop. Please review the loop condition and the actions within the loop to ensure that the loop will terminate properly.* -**Description:** KaneAI detected a While Loop condition / body combination that cannot terminate, typically a condition that is independent of anything the body changes, or a comparison that is always true (for example, `1 == 1`). -**Common Authoring Error:** A While Loop whose body does not touch any value referenced by the condition, or a condition built from constants only. -**Suggested Ways:** Ensure the body contains at least one step that changes a value referenced by the condition: increment a counter, click a control that updates UI state, or wait for a status transition. If the condition uses constants, rewrite it so it depends on a variable or UI state that evolves during the loop. - -### While Loop: Both Operands Are Parameters - -**Error Code:** `BOTH_OPERANDS_AS_PARAMETERS` -**Error Message:** *Both operands in the while loop condition are parameters. Only one operand should be a parameter.* -**Description:** Both sides of a [While Loop](/support/docs/kaneai-while-loops/) condition are test parameters (for example, `${max_retries} > ${default_retries}`). Parameter values are fixed for the lifetime of a run, so such a condition cannot change between iterations. It would either loop forever or never enter. -**Common Authoring Error:** Comparing two dataset parameters directly in the loop condition, such as `${threshold} > ${limit}`. -**Suggested Ways:** Replace one operand with a runtime‑updated value: a counter variable incremented inside the body, a value read from the UI via a query, or a literal. Valid examples include `{{counter}} < ${max_retries}`, `${status} == "ready"`, and `{{cart_empty}} == false`. - -### While Loop: Cannot Be Created via Natural Language - -**Error Code:** `WHILE_NOT_SUPPORTED_VIA_NL` -**Error Message:** *Looping is supported via slash commands only. Type / and select While Loop to add a loop.* -**Description:** A natural‑language step was used to describe a loop (for example, *"repeat until the cart is empty"*, *"while the spinner is visible, do X"*, or *"keep clicking Next"*). The natural‑language pipeline does not expand these phrases into loops. [While Loops](/support/docs/kaneai-while-loops/) can only be added through the **/** slash command menu. -**Common Authoring Error:** Typing looping phrases like *"keep clicking Load more until no more results appear"* as a regular step instead of creating a While Loop block. -**Suggested Ways:** Remove the looping phrase from the plain‑English step. Open the slash menu (`/`), select **While Loop**, enter the loop condition, and add the per‑iteration action as a body step inside the loop. - -> _Have any feedback or request? Reach out to us via support@testmuai.com and we would be happy to hear from you._ diff --git a/static/docs/error-messages.md b/static/docs/error-messages.md deleted file mode 100644 index 0f5db727e..000000000 --- a/static/docs/error-messages.md +++ /dev/null @@ -1,655 +0,0 @@ -# Error Messages During Test Execution - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Below is a list of error messages that may occur during test execution, along with their causes and solutions. - -## Authentication Error at the Time of Test Execution - -This error occurs when your Username or Access Key is missing or invalid. - -This happens when the Username or Access Key you passed is missing or invalid. - -### How to Resolve? - -1. Go to the TestMu AI [Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard). -2. Click the key icon at the top-right of the dashboard. -3. Copy your username and access key. - -## Max Duration Exceeded Error - -The test was terminated because it exceeded the maximum allowed duration. - -The test was terminated because it exceeded the maximum duration allowed (default is 1800 seconds). - -### Possible Cause - -This might result from several issues: - -* Your test is too long. A single test case may contain multiple tests. -* The test gets stuck in an endless loop, continuously sending commands. - -### Possible Solution - -* Break your test into smaller, atomic tests. -* Use the maxDuration desired capability option to set how long you want to wait for your test to complete. -* Check your test for endless loops. - -## Test Cancellation - Status: Error - -A queued test gets cancelled before execution starts. - -When a test is placed in a queue but gets cancelled before execution. - -### Possible Cause - -This might occur due to various reasons: - -* The connection between your local machine and the TestMu AI cloud server is aborted. -* High latency may also cause your test script to get cancelled. -* You cancel the test manually after placing it in the queue. - -## Exceeded Queue Limit Error - -Your test runner closed the connection before a new session became available. - -Your test runner started a new test on TestMu AI, but then closed the connection before a new test session became available. - -### Possible Cause - -This might occur due to several things: - -* Client timeout with browser tests. Make sure to set the connection timeout in your test runner or framework high enough, as older browsers take a while to start up. Wait for at least a couple of minutes. -* You might be running too many tests at once. If you exceed the total concurrent test limit, tests are queued per your plan. If this queueing takes too long, your test runner might disconnect before the test starts. - -### Possible Solutions - -* Increase the connection timeout setting in your test runner or framework. For example, for WebdriverIO, set `connectionRetryTimeout: 210000`. -* Make sure you are not exceeding the total allowed concurrent test limit for your TestMu AI Automation plan. - -## Lambda Error - -The grid failed to recognize your test input. - -You may encounter a Lambda Error with the below message. - -> **Lambda Error:** Uh Oh! Looks like our Grid failed to recognize your test input. - -### Possible Cause - -There could be several reasons behind this error. The most common ones include: - -* Infrastructure Unavailability: With too many incoming requests, the cloud server may fail to allocate a VM for your request. -* Incorrect Data Type for Desired Capabilities: Sending a string input for an integer data type capability triggers a Lambda Error. -* Excessive web-traffic spikes may also cause Lambda Error. - -## Element Click Intercepted - 400 - -Another element is obscuring the element you want to click. - -The Element Click command could not complete because another element is obscuring the target element. - -{`org.openqa.selenium.ElementClickInterceptedException: element click intercepted: Element ... is not clickable at point (161, 562). Other element would receive the click: ... -`} - -### Possible Cause - -* This usually occurs when the target element is overlaid by another element in the web page. - -### Possible Solutions - -* There are multiple ways to solve this issue. Check StackOverflow for more information. - -## Element Not Interactable - 400 - -The element is not ready for pointer or keyboard interaction. - -A command could not complete because the element is not pointer- or keyboard interactable. - -### Possible Cause - -1. The element has not rendered yet. - -2. The element has rendered but is not in the visible part of the screen. - -### Possible Solutions - -For 1. -> Use implicit or explicit wait: - - Implicit wait: - - {`driver.manage().timeouts().implicitlyWait(50, TimeUnit.SECONDS);`} - - - Explicit wait: - - {`WebDriverWait wait=new WebDriverWait(driver, 20); element1 = wait.until(ExpectedConditions.elementToBeClickable(By.className("fa-stack-1x")));`} - -For 2. -> Scroll to the element. Based on the Selenium version, this can be handled in different ways. For more information, refer to this link. - -## Insecure Certificate - 400 - -Navigation triggered a certificate warning due to an expired or invalid TLS certificate. - -Navigation caused the user agent to hit a certificate warning, usually resulting from an expired or invalid TLS certificate. - -### Possible Cause - -* SSL works through a combination of programs and encryption/decryption routines on the web server and browser. -* When a secure connection is not established between the server and client due to the certificate, an SSL certificate error appears. -* If you send an HTTPS request in the browser and get a prompt such as "This connection is untrusted" or "The site's security certificate is not trusted" (which varies by browser), this indicates an SSL certificate error. - -### Possible Solutions - -You can adjust your script to handle the SSL exception automatically using Selenium WebDriver. - -* For understanding and handling SSL errors, check this site for more information. - -## Invalid Argument - 400 - -The arguments passed to a command are invalid or malformed. - -The arguments passed to a command are either invalid or malformed. -Example: - -You cannot set a window size to a negative value: - -{`from selenium import webdriver from selenium.common import exceptions session = webdriver.Firefox() - try: - session.set_window_size(-10, 0) - except - exceptions.InvalidArgumentException as e: print(e.message) `} - -Output: - -{`InvalidArgumentException: Expected -10 to be >= 0`} - -### Possible Cause - -* The invalid argument error is a WebDriver error that occurs when the arguments passed to a command are invalid or malformed. -* Invalid argument errors are similar to TypeErrors in JavaScript. They can occur for many APIs when the input value is not the expected type or is malformed. - -### Possible Solutions - -* Check the values in your input and verify they are valid. - -## Invalid Cookie Domain - 400 - -You attempted to set a cookie under a domain different from the current page. - -An illegal attempt was made to set a cookie under a different domain than the current page. - -### Possible Cause - -* The invalid cookie domain error occurs when you attempt to set a cookie under a different domain than the current browsing context. WebDriver does not permit setting cookies for other domains. -Example: -If the current domain is 'example.com', you cannot add a cookie for 'example.org': - -{`from selenium import webdriver - from selenium.common import exceptions - session = webdriver.Firefox() - session.get("https://example.com/") - try: - cookie = {"name": "foo", - "value": "bar", - "domain": "example.org"} - session.add_cookie(cookie) - except exceptions.InvalidCookieDomainException as e: print(e.message) -`} - -Output: - -{`InvalidCookieDomainException: https://example.org/ -`} - -## Invalid Element State - 400 - -The element is in a state that does not support the requested operation. - -A command could not complete because the element is in an invalid state, e.g. attempting to clear an element that is not both editable and resettable. - -### Possible Cause - -* When you try to perform an operation that does not apply to the element, it throws InvalidElementStateException. -* If a textbox is disabled and you try to type into it, it throws an exception. -* If a radio button, checkbox, or other web element is disabled and you try to click it, it throws an exception. -* If an element supports only click events and you try to perform type events, it throws an exception. - -### Possible Solutions - -* Verify you are performing the correct operation based on the element state. -* If the element is clickable, perform a click. If it supports type events, use sendkeys. -* If the element is disabled, enable it first before performing operations. - -For more, refer to this link (learn-automation.com). - -## Invalid Selector - 400 - -The provided selector strategy is unknown or incorrect. - -Argument was an invalid selector. - -{`org.openqa.selenium.InvalidSelectorException: invalid selector -`} - -### Possible Cause - -* The invalid selector error occurs when an element retrieval command uses an unknown web element selector strategy. - -* The available selector strategies are CSS, link text, partial link text, tag name, and XPath. Any other selector strategy triggers this error. - -### Possible Solutions - -* Check punctuation such as @, ', and []. -* Make sure there is only one field name with that path. Otherwise, use the contains() method. - -## Session Not Generated - Invalid Session ID 404 / Session Not Created 500 - -The session either does not exist, is not active, or could not be created. - -### Invalid Session ID - 404 - -Occurs if the given session ID is not in the list of active sessions, meaning the session either does not exist or is not active. - -{`ERROR webdriver: Request failed with status 404 due to invalid session id: invalid session -`} - -### Possible Cause - -* As of 04/2021, sync mode is no longer supported starting from Node.js v16 due to changes in Chromium. -* Use async to solve this error. -* The WebdriverIO site has an official guideline for this: WebdriverIO - -### Session Not Created - 500 - -A new session could not be created. - -{`org.openqa.selenium.SessionNotCreatedException: Message: Could not start a new session. Response code 500. Message: session not created -`} - -### Possible Cause - -* Incompatibility between the versions of binaries being used. - -### Possible Solutions - -* Ensure the Chrome version and the JDK/driver versions are compatible. Refer to this link: stackoverflow - -## JavaScript Error - 500 - -A user-supplied script failed to execute in the browser. - -The JavaScript error occurs when a script supplied by the user fails to execute. - -Example: - -{`from selenium import webdriver - from selenium.common import exceptions - session = webdriver.Firefox() - try: - session.execute_script("return foo") - except exceptions.JavascriptException as e: - print(e.message) -`} - -Output: - -{`JavascriptException: ReferenceError: foo is not defined -`} - -### Possible Cause - -* The underlying cause of the execution error is often supplied in the error message, along with a stacktrace from the JavaScript engine in the browser. - -### Possible Solutions - -* Check for invalid declarations and definitions in your code. - -## Move Target Out of Bounds - 500 - -The target for mouse interaction is outside the browser viewport. - -The target for mouse interaction is not in the browser's viewport and cannot be brought into view. - -Example: - -{`MoveTargetOutOfBoundsException: Element cannot be scrolled into view -`} - -### Possible Cause - -* Selenium replicates exact human behavior, so if code failure indicates an element is not visible, the element is actually not visible. - -### Possible Solutions - -* For a detailed explanation, refer here. - -## No Such Alert - 404 - -You attempted to operate on a modal dialog when none was open. - -An attempt was made to operate on a modal dialog when one was not open. - -{`Exception in thread "main" org.openqa.selenium.UnhandledAlertException: Unexpected modal dialog (text: You need to use IE 6.0 for viewing this application. Else some features may not work): You need to use IE 6.0 for viewing this application. Else some features may not work -`} - -### Possible Cause/Explanation - -Refer to these links for possible explanations: -* no-alert-is-present-no-modal-dialog-found-webdriver-unable-to-catch-js-error -* selenium-webdriver-unexpected-modal-dialog-alert - -## No Such Cookie - 404 - -No cookie matching the given path name was found in the current browsing context. - -No cookie matching the given path name was found among the associated cookies of the current browsing context's active document. - -Example: - -{`org.openqa.selenium.UnableToSetCookieException: Unable to set cookie (WARNING: The server did not provide any stacktrace information) -`} - -### Possible Cause - -* Creating a cookie before navigating to the site. - -### Possible Solutions - -* For a detailed explanation, refer here. - -## No Such Element - 404 - -The element could not be located on the page using the given search parameters. - -An element could not be located on the page using the given search parameters. - -Example: - -{`exception selenium.common.exceptions.NoSuchElementException(msg=None, screen=None, stacktrace=None) -`} - -### Possible Cause - -This occurs mainly in two cases: - -* When using `webdriver.find_element_by_*("expression") //example : my_element = driver.find_element_by_xpath("xpath_expression")` - -* When using `element.find_element_by_*("expression")//example : my_element = element.find_element_by_*("expression")` - -There are different possibilities for this error. Check the link below for more information. - -### Possible Solutions - -* For a detailed explanation, refer here. - -## No Such Frame - 404 - -The command to switch to a frame could not find the specified frame. - -A command to switch to a frame could not be satisfied because the frame could not be found. - -Example: - -{`selenium.common.exceptions.NoSuchFrameException: Message: no such frame -`} - -### Possible Cause - -* To switch to the desired frame, use WebDriverWait for the frame to become available and then switch to it. - -### Possible Solutions -Implement suitable edits to either CSS selector or XPATH: - -* `CSS Selector`: `WebDriverWait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR,"iframe[id^='layui-layer-iframe'][src^='fangyuan']")))` -* `XPATH`: `WebDriverWait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH,"//iframe[starts-with(@id,'layui-layer-iframe') and starts-with(@src,'fangyuan')]")))` - -## No Such Window - 404 - -The command to switch to a window could not find the specified window. - -A command to switch to a window could not be satisfied because the window could not be found. - -Example: - -{`webdriver: Request failed with status 404 due to no such window: Currently focused window has been closed. -`} - -### Possible Solution - -There are multiple approaches to this problem: -* In export configuration, to delete everything at the end, set `killinstances` to `true`. -* Check all required configurations for the corresponding web browser driver (especially for InternetExplorerDriver). -* Enabling protected mode for all zones often resolves this. - -For more, refer to Stack Overflow. - -## No Such Shadow Root - 404 - -The element does not have a shadow root attached. - -The element does not have a shadow root. - -### Explanation - -For a `GET` request of Element Shadow Root with URI template `/session/{session id}/element/{element id}/shadow`, if `shadow root` is null, the error code `no such shadow root` is returned. - -For an explanation on locator strategy, refer to this link. - -## Stale Element Reference - 404 - -The referenced element is no longer attached to the DOM. - -A command failed because the referenced element is no longer attached to the DOM. -Example: - -{`StaleElementReferenceException: The element reference of e75a1764-ff73-40fa-93c1-08cb90394b65 is stale either the element is no longer attached to the DOM, it is not in the current frame context, or the document has been refreshed -`} - -### Possible Cause - -* The stale element reference error occurs because the referenced web element is no longer attached to the DOM. - -* Every DOM element is represented in WebDriver by a unique identifying reference, known as a web element. The web element reference is a UUID used to execute commands targeting specific elements, such as getting an element's tag name and retrieving a property off an element. - -* When an element is no longer attached to the DOM (it has been removed from the document or the document has changed), it is said to be stale. Staleness occurs, for example, when you have a web element reference and the document it was retrieved from navigates. - -### Possible Solutions -Common solutions include: - -* Refreshing the webpage: -`driver.navigate().refresh(); - driver.findElement(By.xpath("xpath here")).click();` - -* Using 'try-catch' block within 'for loop': - -{`for(int i=0; i<=2;i++){ - try{ - driver.findElement(By.xpath("xpath here")).click(); - break; - } - catch(Exception e){ - Sysout(e.getMessage()); - } -}`} - -* Using ExpectedConditions.refreshed: - -Wait for the element until it is available. Use ExpectedConditions.refreshed to avoid StaleElementReferenceException and retrieve the element again. This method updates the element by redrawing it so you can access the referenced element. - -`wait.until(ExpectedConditions.refreshed(ExpectedConditions.stalenessOf("table")));` - -* Using POM (lazy initialization): -In POM, use initElements() which loads the element but does not initialize elements. initElements() takes the latest address. - -For a detailed explanation: -* Link 1 -* Link 2 - -## Unsupported Operation - 500 - -The requested operation is not supported for the given class or data structure. - -A command that should execute properly cannot be supported for some reason. - -Example: - -{`import java.util.Arrays; -import java.util.List; -public class UnsupportedOperationExceptionExample { - public static void main(String[] args) { - String array[] = {"a", "b", "c"}; - List list = Arrays.asList(array); - list.add("d"); - } -} -`} - -(for Java) - -### Possible Cause - -An UnsupportedOperationException is thrown when a requested operation cannot be performed because it is not supported for that class. One common cause is using the `asList()` method of `java.util.Arrays`. Since this method returns a fixed-size unmodifiable `List`, the `add()` or `remove()` methods are unsupported. Trying to add or remove elements from such a `List` throws the `UnsupportedOperationException`. - -Other cases where this exception can occur include: - -Using wrappers between collections and primitive types. -Trying to remove elements using an `Iterator`. -Trying to add, remove, or set elements using `ListIterator`. - -### Possible Solutions - -* Resolve the `UnsupportedOperationException` by using a mutable collection, such as `ArrayList`, which can be modified. Do not attempt to modify an unmodifiable collection or data structure. - -Refer to this explanation from Rollbar for more info: Link - -## Unknown Method - 405 - -The request matched a known URL but used an unsupported HTTP method. - -The requested command matched a known URL but did not match any method for that URL. - -Example: - -{`exception selenium.common.exceptions.NoSuchElementException(msg=None, screen=None, stacktrace=None) -`} - -### Possible Cause - -The unknown method error occurs when the driver does not recognize the HTTP request method used for the endpoint. - -* WebDriver provides a largely REST-ish API and not all endpoints have `GET`, `POST`, and `DELETE` methods. This error occurs when you call an endpoint with an HTTP request method it does not support. - -Refer to this example: Unknown Method error - -## Unknown Error - 500 - -An unspecified error occurred in the driver while processing a command. - -The unknown element error occurs when an unspecified error occurs in the driver while processing a command. - -An unknown error is usually specific to a particular driver, so read the error message for an indication of what happened. - -## Unknown Command - 404 - -The driver does not recognize the command or HTTP endpoint. - -The unknown command error occurs when the driver does not recognize the command or HTTP endpoint. - -Example: -The /session/{session id}/foo endpoint does not exist, and returns an unknown command error with a 404 Not Found HTTP status code. - -{`% curl -i -d '{}' http://localhost:4444/session/foo -HTTP/1.1 404 Not Found -Connection: close -Content-Type: application/json; charset=utf-8 -Cache-Control: no-cache -Content-Length: 113 -Date: Fri, 30 Mar 2018 15:30:51 GMT -{"value":{"error":"unknown command","message":"POST /session/asd did not match a known command","stacktrace":""}} -`} - -## Script Timeout - 500 - -A user-provided script did not complete before the session's script timeout expired. - -An operation did not complete before its timeout expired. - -The script timeout error occurs when a script you provided did not complete before the session's script timeout duration expired. - -The script timeout duration is a configurable capability. You can change how long the driver waits before interrupting an injected script. The driver waits 30 seconds by default before interrupting the script and returning a script timeout error, but you can extend, limit, or set this to indefinite. - -If you set the session script timeout duration to indefinite by using a `null` value, you risk putting the session into a non-recoverable state. Use this with caution. - -Example: - -{`from selenium import webdriver -from selenium.common import exceptions -session = webdriver.Firefox() -try: - session.execute_script(""" - let [resolve] = arguments; - window.setTimeout(resolve, 35000); - """) -except exceptions.ScriptTimeoutException as e: - print(e.message) -`} - -Output: - -{`ScriptTimeoutException: Timed out after 35000 ms`} - -* A workaround is to use capabilities to extend the session's default script timeout. - -## Unable to Set Cookies - 500 - -A command to set a cookie's value could not be satisfied. - -`org.openqa.selenium.UnableToSetCookieException: Unable to set cookie (WARNING: The server did not provide any stacktrace information)` - -Example: - -{`System.setProperty("webdriver.edge.driver","C:\\Program Files\\Latest Webdriver\\MicrosoftWebDrive.exe" ); -EdgeDriver = new EdgeDriver(); -Thread.sleep(2000); -Cookie cookie = new Cookie("Testing", "11111"); -EdgeDriver.manage().addCookie(cookie); -EdgeDriver.get("https://www.google.ca/?gws_rd=ssl"); // The link is an example -`} - -### Explanation - -This error results from creating a cookie before navigating to the site. - -If you need to create a cookie on the domain `www.example.com`, first navigate to some page on that domain, create the cookie, and then start your test. - -Navigate to a page you know will not exist on the domain, e.g. `www.example.com/this404page`, then create the cookie. It loads faster since it is an error page with minimal content. After creating the cookie on the 404 page, start the test. - -## Unable to Capture Screen - 500 - -A screen capture could not be completed. - -Possible explanations: (For Appium) - -## Unexpected Alert Open - 500 - -A command could not execute because the remote end encountered an unexpected alert. - -`org.openqa.selenium.UnhandledAlertException: unexpected alert open` - -Example: - -{`org.openqa.selenium.UnhandledAlertException: unexpected alert open - (Session info: chrome=38.0.2125.111) - (Driver info: chromedriver=2.9.248315,platform=Windows NT 6.1 x86) (WARNING: The server did not provide any stacktrace information) - Command duration or timeout: 16 milliseconds: null - Build info: version: '2.42.2', revision: '6a6995d', time: '2014-06-03 17:42:30' - System info: host: 'Casper-PC', ip: '10.0.0.4', os.name: 'Windows 7', os.arch: 'x86', os.version: '6.1', java.version: '1.8.0_25' - Driver info: org.openqa.selenium.chrome.ChromeDriver -`} - -### Explanation - -This happens because of the default behavior of the driver when it encounters an alert. The default behavior was set to "ACCEPT", so the alert was closed automatically, and the switchTo().alert() could not find it explicitly. - -The solution is to modify the default behavior of the driver to ("IGNORE"), so that it does not close the alert but just ignores it. - -Then use try-catch to catch any errors. diff --git a/static/docs/error-reports.md b/static/docs/error-reports.md deleted file mode 100644 index 2c62ba35c..000000000 --- a/static/docs/error-reports.md +++ /dev/null @@ -1,20 +0,0 @@ -# Getting The JavaScript Error Report Of Your Website - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -[LT Browser](https://www.testmuai.com/lt-browser/) monitors web-page errors and services that fail to load and alerting you when the issue occurs and generates a javascript error report of the particular website. For example, if your URL is not rendered correctly, it will throw an error message. - -To get started, you need to download the executable file from here. - -Download LT Browser For Windows - -Download LT Browser For macOS - -Download LT Browser For Linux - -To get the error report of your unrendered URL, navigate to the **'Error Report'** button in the URL bar as highlighted below. - -Upon clicking the Error Report button, your Error Report will be generated. - -> -That's all!. In case you come across any doubts, feel free to reach out through our Customer Chat Support or you can also drop a mail to [support@testmuai.com](mailto:support@testmuai.com).Happy testing! diff --git a/static/docs/espresso-automation-on-emulators-simulators.md b/static/docs/espresso-automation-on-emulators-simulators.md deleted file mode 100644 index c6474ac97..000000000 --- a/static/docs/espresso-automation-on-emulators-simulators.md +++ /dev/null @@ -1,3 +0,0 @@ -# Espresso Automation using Emulators and Simulators on TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). diff --git a/static/docs/espresso-env-variables-settings.md b/static/docs/espresso-env-variables-settings.md deleted file mode 100644 index 8145c3672..000000000 --- a/static/docs/espresso-env-variables-settings.md +++ /dev/null @@ -1,112 +0,0 @@ -# Setting Up Espresso Environment Variables - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This feature allows you to dynamically set and test environment variables during Espresso test execution on TestMu AI. - -## Step 1: Create Variables in Your Test Suite -Define environment variables in your Espresso test suite to fetch the variable values during execution. - -```bash title="Sample Script" -String envVar = InstrumentationRegistry.getArguments().getString(ENV_VAR); -``` - -``` bash title="Examples" -String stage = InstrumentationRegistry.getArguments().getString(“STAGE”); -String prod = InstrumentationRegistry.getArguments().getString(“PROD”); -``` - -## Step 2: Upload Your Application - -To begin testing, upload your Android application (.apk file) to TestMu AI's servers. You'll use our **REST API** for this process. - -- **Authentication :** You'll need your TestMu AI Username and AccessKey. Combine them in the format `Username:AccessKey`. -- **Uploading the App :** Use **cURL command** to send a request to our API. The request should include the path to your application file (**appFile**). - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" --location --request POST 'https://manual-api.lambdatest.com/app/uploadFramework' --form 'appFile=@""' --form 'type="espresso-android"'`} - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" --location --request POST "https://manual-api.lambdatest.com/app/uploadFramework" --form "appFile=@""" --form "type=\"espresso-android\""`} - -- Provide the path of your android application in the above URL in place of `` -- Response of above cURL will be a **JSON** object containing the `App URL` of the format - `lt://APP123456789123456789` and will be used in the last step. - -## Step 3: Upload Your Test Suite - -Upload your Espresso test suite (.apk) file to TestMu AI servers using our REST API. - -The following sample cURL command shows how to upload a test suite: - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" --location --request POST 'https://manual-api.lambdatest.com/app/uploadFramework' --form 'appFile=@""' --form 'type="espresso-android"'`} - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" --location --request POST "https://manual-api.lambdatest.com/app/uploadFramework" --form "appFile=@""" --form "type=\"espresso-android\""`} - -- Provide the path of your android application in the above URL in place of `` -- Response of above cURL will be a **JSON** object containing the `App URL` of the format - `lt://APP123456789123456789` and will be used in the last step. - -## Step 4: Executing The Test - -- You will need **base64 encoded authentication** in order to execute your Espresso automation test suite. Enter your `username:accesskey` in **[Basic Authentication Header Generator](https://mixedanalytics.com/knowledge-base/api-connector-encode-credentials-to-base-64/)** to generate your auth token. - -Take note of the base64 encoded authentication which needs to be added in the next step. - -{`${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}`} - -- Once you have uploaded your app and test suite, you can execute your test by running the following command: - -> Enter your **BASIC_AUTH_TOKEN**, **APP_ID** (generated in the first step) and **TEST_SUITE_ID** (generated in the second step) in the below command. - -```bash -curl --location --request POST 'https://mobile-api.lambdatest.com/framework/v1/espresso/build' \ ---header 'Authorization: Basic BASIC_AUTH_TOKEN' \ ---header 'Content-Type: application/json' \ ---data-raw '{ -"app" : "APP_ID", -"testSuite": "TEST_SUITE_ID", -"device" : ["Galaxy S21 5G-12"], -"queueTimeout": 10800, -"IdleTimeout": 150, -"deviceLog": true, -"network": false, -"build" : "Proverbial-Espresso" -# highlight-start -"envVariables":{ //setting up environment variables -"STAGE":"stg1", -"PROD":"prod1" -} -# highlight-end -}' -``` - -## Run your test in HyperExecute -To execute your test suite in HyperExecute, configure your YAML file by specifying the `` and ``. - -```yaml ---- -version: "0.2" -concurrency: 4 -runson: android -autosplit: true -maxRetries: 2 -# highlight-start -env: -STAGE: stg1 -PROD: prod1 -# highlight-end -framework: -name: "android/espresso" -args: -reservation: false -buildName: "Test Espresso Sharding" -appId: -testSuiteAppId: -deviceSelectionStrategy: any -devices: [".*"] -shards: -mappings: -- name: shard1 -strategy: "only-testing/skip-testing" -values: ["/"] -- name: shard2 -strategy: "only-testing/skip-testing" -values: ["/", "/"] -``` diff --git a/static/docs/espresso-junit-report.md b/static/docs/espresso-junit-report.md deleted file mode 100644 index 32f16d0a8..000000000 --- a/static/docs/espresso-junit-report.md +++ /dev/null @@ -1,39 +0,0 @@ -# Espresso JUnit XML Reports - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Introduction - -JUnit reports provide a detailed summary of test execution, allowing you to better understand your test outcomes. This document aims to guide you through the process of retrieving JUnit reports for Espresso tests executed on the TestMu AI platform. - -## Objective - -### By the end of this document, you should be able to: - -1. Fetch JUnit reports for non-shard Espresso builds. - -2. Fetch JUnit reports for shard builds, both for individual shards and all shards collectively. - -## JUnit report APIs - -**Non-shard build :** -To fetch the JUnit report for a `non-shard` build, you can use the following cURL command: - -{`curl --location "https://mobile-api.lambdatest.com/mobile-automation/api/v1/framework/builds//report/?encoder=false" \ ---header 'Authorization: Basic '`} - -**Shard build (For single shard):** -To fetch the JUnit report for a `single shard` in a shard build,use: - -{`curl --location 'https://mobile-api.lambdatest.com/mobile-automation/api/v1/framework/jobs//report/?shard=&encoder=false' \ ---header 'Authorization: Basic '`} - -**Shard build (For all the shards):** -To fetch the JUnit reports for `all shards` in a shard build, use: - -{`curl --location 'https://mobile-api.lambdatest.com/mobile-automation/api/v1/framework/jobs//report/?encoder=false' \ ---header 'Authorization: Basic '`} - -- Authenticate the API using your TestMu AI username and access key, and replace `build_id`, `job_id` and `shard_id` for which you want to fetch report. -- It is recommended to run the sharding test(via HyperExecute CLI) in the verbose mode i.e. with the **--verbose** flag. This allows the shard ID(task ID) and build ID(Job ID) to be displayed in the logs and then they can be used to fetch the above reports. -- In case the report is not a valid XML format, the `encoder=true` parameter can be utilized to prevent the decoding of certain characters. Decoding is usually performed at the server's end to enhance the readability of the report. diff --git a/static/docs/espresso-mockwebserver-localhost.md b/static/docs/espresso-mockwebserver-localhost.md deleted file mode 100644 index e587490f3..000000000 --- a/static/docs/espresso-mockwebserver-localhost.md +++ /dev/null @@ -1,80 +0,0 @@ -# Testing with MockWebServer & Localhost - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI supports testing apps that use MockWebServer or similar localhost-based mock servers in Android Espresso tests. - -## Why Special Configuration is Needed - -When network capture (`network: true`) is enabled, HTTP requests from the device route through proxy. Localhost requests fail because the proxy runs on the host machine, not the device, so `http://localhost:{port}` resolves to the host's localhost instead of the device's. - -Platform provides two solutions: **Localhost Bypass** and **Port Forwarding**. - -## Option 1: Localhost Bypass - -Works when network is set to `true` and application uses standard HTTP libraries.Localhost network calls will not be captured. - -```json -{ -"app": "lt://APP_ID", -"testSuite": "lt://TESTSUITE_ID", -"device": ["Galaxy S21-12", "Pixel 6-13"], -"build": "MockWebServer Test", -"network": true, -"localhost": true -} -``` - -| Capability | Data Type | Description | -|------------|-----------|-------------| -| `localhost` | Boolean | Bypass proxy for `localhost`/`127.0.0.1` requests. Default: `false` | - -### Supported HTTP Libraries - -| Library | Notes | -|---------|-------| -| **Java HttpURLConnection** | `URL.openConnection()` is intercepted | -| **OkHttp** | `proxy()` and `proxySelector()` are intercepted | -| **Retrofit** | Uses OkHttp internally | -| **Volley** | Uses HttpURLConnection internally | - -**Requirements** -- `network: true` is required for `localhost: true` to work. -- `localhost` capability cannot be used together with `portForwarding`. -- Supported only on real android devices right now. - -## Option 2: Port Forwarding - -Best when localhost network logs need to be captured and there are no port conflicts. - -```json -{ -"app": "lt://APP_ID", -"testSuite": "lt://TESTSUITE_ID", -"device": ["Galaxy S21-12", "Pixel 6-13"], -"build": "MockWebServer Test", -"network": true, -"portForwarding": { -"ports": [9091, 9092] -} -} -``` - -| Capability | Data Type | Description | -|------------|-----------|-------------| -| `portForwarding` | Object | Configure port forwarding for localhost services on the device | -| `portForwarding.ports` | Array | Ports to forward (max 5 unique ports, must be 1024–65535) | - -Port forwarding works at the network level, so **all HTTP libraries are supported**. - -- Ports must be in the range 1024–65535. Privileged ports (1–1023) are blocked. -- Maximum 5 unique ports. No duplicate ports allowed. -- Invalid port formats (e.g., strings like `"abc"`) are rejected. -- Cannot be used together with `localhost`. -- `network: true` is **not** required for port forwarding. It works independently. - -## Troubleshooting - -- **Connection errors**: Ensure `network: true` is set when using `localhost: true`. For `portForwarding`, `network: true` is optional. -- **Cannot use localhost and portForwarding together**: These are mutually exclusive. Pick one. -- **Port validation errors**: Ports must be 1024–65535, max 5 unique ports, no duplicates allowed. diff --git a/static/docs/espresso-supported-capabilities.md b/static/docs/espresso-supported-capabilities.md deleted file mode 100644 index c63feda3b..000000000 --- a/static/docs/espresso-supported-capabilities.md +++ /dev/null @@ -1,37 +0,0 @@ -# Espresso Supported Capabilities - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This document provide details about the features and capabilities supported for Espresso Framework on TestMu AI. - -> The capabilities listed in this document are also supported for [Virtual Devices](/support/docs/app-automation-on-emulators-simulators/) - -| Capability Name | Data Type | Description | -|------|-----------|-------------| -| app | String | Enter the app id generated while uploading the app to the platform. Example:`lt://APP123456789123456789` | -| testSuite | String | Enter the test suite id generated while uploading the test suite to the platform. Example: `lt://APP123456789123456789` | -| device | String | Enter the device name and os version in `DeviceName-OSVersion` format. Example: `"device" : ["Galaxy S20-10","Pixel 3 XL-9"]` | -| video | Boolean | When you pass the `video: true`, it will generate the video for all the tests that are executed. | -| queueTimeout | Integer | Total execution time of the build. Example: `queueTimeout: 300`. Maximum value is `10800`| -| idleTimeout | Integer | Enter the time in seconds for maximum running time of a test in a build. Example: `idleTimeout: 120`| -| deviceLog | Boolean | To generate the device logs, pass the value as `deviceLog: true` | -| network | Boolean | To generate the network logs , pass the value as `network: true` | -| network.har | Boolean | To generate the network logs in .har format, pass the value as `network.har: true` only works with `network:true` | -| build | String | To set the Espresso build name. Example: `build: My Espresso Build`. | -| geoLocation | String | Set the geolocation [country code](/support/docs/appium-ip-geolocation/#supported-ip-geolocations) if you want to enable the same in your test. Example - `geoLocation: FR`| -| gpsLocation | Decimal | The capability should be defined to override the device’s default GPS coordinates and should be provided as a comma-separated string. Example: `"location": {"lat": "-50","long": "150.028333"}`| -| enableBluetooth | Boolean | Used to enable the bluetooth functionality during the session. Example: `enableBluetooth: true`| -| tunnel, tunnelName | Boolean | To activate the tunnel mode, pass the value as `tunnel: true` and provide the name of your tunnel as `tunnelName: NewTunnel` | -| disableAnimation | Boolean | Set this to `true` if you want to disable animations for espresso tests. **Default:** `false` | -| clearPackageData | Boolean | Set this to `true` if you want to clear the app data after each test has completed running. **Default**: `false` | -| singleRunnerInvocation | Boolean | Set this to `true` if you want to execute test cases together with a `single test runner invocation`.This can help significantly speed up the test execution time. By default, each test case is invoked separately. **Default:** `false` | -| testRemarks | Boolean | Set to `true` to surface failure remarks at the top of the dashboard when a test fails. Has no effect on passing tests. **Default**: `false`. | -| retries | Integer |Defines the number of times a test should automatically retry if it fails. Maximum allowed value: 5.| -| region | String | Set the region for the test execution data center. Supported values: `US`, `EU`, `AP`. **Default:** Nearest data center. Example: `region: EU` | -| networkProfile | String | Set a predefined [network throttling](/support/docs/app-auto-network-throttling/) profile during test execution. Requires `network: true`. **Default:** `null`. Example: `networkProfile: 2g-gprs-good` | -| uploadMedia | String | Upload media files to the device for testing. Provide the media URL generated after [uploading the file](/support/docs/upload-media/). **Default:** `null`. Example: `uploadMedia: lt://MEDIA123456789` | - -Ensure that the latitude is between -90 and 90, and the longitude is between -180 and 180. Otherwise, an error will occur like "Invalid GPS location: Latitude must be between -90 and +90, Longitude must be between -180 and +180." - -**Important** -You can either use **GeoLocation** or **Tunnel** in single execute command. They both are mutually exclusive. diff --git a/static/docs/espresso-visual-regression.md b/static/docs/espresso-visual-regression.md deleted file mode 100644 index 368b3554b..000000000 --- a/static/docs/espresso-visual-regression.md +++ /dev/null @@ -1,419 +0,0 @@ -# Getting Started With Visual Regression Testing Using Espresso On SmartUI Real Devices - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Espresso is a widely-used testing framework for Android, designed to simplify the process of writing reliable and efficient UI tests. It allows developers to create automated tests that simulate user interactions within an app, ensuring that the app's UI behaves as expected. - -In this documentation, you will learn step-by-step how to perform regression testing and find Visual UI Regression bugs easily with the help of Smart Testing. - -## Prerequisites - -- Your TestMu AI [Username and Access key](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/security). -- Access to an **Android** app (.apk) and an **Espresso Test** app (.apk file). -- Go to [`TestMu AI SmartUI`](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) and login along with your credentials. - -If you do not have any **Android** app (.apk) and an **Android Test** app (.apk) file, you can run your sample tests on TestMu AI by using our sample :link: [Android app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/espresso/proverbial_android-app.apk) and a sample :link: [Espresso Test](https://prod-mobile-artefacts.lambdatest.com/assets/docs/espresso/proverbial_android_espressotest-app.apk). - -## Step 1: Create a SmartUI Project - -The first step is to create a project with the application in which we will combine all your **builds** run on the project. -To create a SmartUI Project, follow these steps: - -1. Go to [Projects page](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) -2. Click on the `new project` button -3. Select the platform as Real Device for executing your `Espresso` tests. -4. Add name of the project, approvers for the changes found, tags for any filter or easy navigation. -5. Click on the **Submit**. - -## Step 2: Set up your Authentication - -Now, you need to export your environment variables LT_USERNAME and LT_ACCESS_KEY that are available in the [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). - -Run the below mentioned commands in your terminal to setup the CLI and the environment variables. - -{`export LT_USERNAME="${YOUR_LAMBDATEST_USERNAME}" -export LT_ACCESS_KEY="${YOUR_LAMBDATEST_ACCESS_KEY}"`} - -{`set LT_USERNAME="${YOUR_LAMBDATEST_USERNAME}" -set LT_ACCESS_KEY="${YOUR_LAMBDATEST_ACCESS_KEY}"`} - -## Step 3: Update your App Configurations - -> It should be in your app configurations then only you can perform the Espresso integration. - -- In your `build.gradle` file, add the [lambdatest-espresso](https://central.sonatype.com/artifact/io.github.lambdatest/lambdatest-espresso) dependency: - -```bash -implementation 'io.github.lambdatest:lambdatest-espresso:1.0.1' -``` - -- Initialize this dependency in your test classes. You can do it as follows: - -```bash java -import io.github.lambdatest.LTApp; -public class BrowserTest { - -LTApp smartUIApp = new LTApp(); - -@Test -public void checkBrowserPageIsOpened() throws InterruptedException { -String response = smartUIApp.screenshot("LT-Espresso-Test"); -} -} -``` - -Now build your application. - -## Step 4: Upload your Application - -To begin testing, upload your Android application (.apk file) to TestMu AI's servers. You'll use our **REST API** for this process. - -- **Authentication :** You'll need your TestMu AI Username and AccessKey. Combine them in the format `Username:AccessKey`. -- **Uploading the App :** Use **cURL command** to send a request to our API. The request should include the path to your application file (**appFile**). - -{`curl -u "${YOUR_LAMBDATEST_USERNAME}:${YOUR_LAMBDATEST_ACCESS_KEY}" --location --request POST 'https://manual-api.lambdatest.com/app/uploadFramework' --form 'appFile=@""' --form 'type="espresso-android"'`} - -{`curl -u "${YOUR_LAMBDATEST_USERNAME}:${YOUR_LAMBDATEST_ACCESS_KEY}" --location --request POST "https://manual-api.lambdatest.com/app/uploadFramework" --form "appFile=@""" --form "type=\"espresso-android\""`} - -- Provide the path of your android application in the above URL in place of `` -- Response of above cURL will be a **JSON** object containing the `App URL` of the format - `lt://APP123456789123456789` and will be used in the last step. - -## Step 5: Upload Your Test Suite - -Upload your Espresso test suite (.apk) file to TestMu AI servers using our REST API. - -The following sample cURL command shows how to upload a test suite: - -{`curl -u "${YOUR_LAMBDATEST_USERNAME}:${YOUR_LAMBDATEST_ACCESS_KEY}" --location --request POST 'https://manual-api.lambdatest.com/app/uploadFramework' --form 'appFile=@""' --form 'type="espresso-android"'`} - -{`curl -u "${YOUR_LAMBDATEST_USERNAME}:${YOUR_LAMBDATEST_ACCESS_KEY}" --location --request POST "https://manual-api.lambdatest.com/app/uploadFramework" --form "appFile=@""" --form "type=\"espresso-android\""`} - -- Provide the path of your android application in the above URL in place of `` -- Response of above cURL will be a **JSON** object containing the `App URL` of the format - `lt://APP123456789123456789` and will be used in the last step. - -## Step 6: Executing The Test - -- You will need **base64 encoded authentication** in order to execute your Espresso automation test suite. Enter your `username:accesskey` in **[Basic Authentication Header Generator](https://mixedanalytics.com/knowledge-base/api-connector-encode-credentials-to-base-64/)** to generate your auth token. - -Take note of the base64 encoded authentication which needs to be added in the next step. - -{`${YOUR_LAMBDATEST_USERNAME}:${YOUR_LAMBDATEST_ACCESS_KEY}`} - -- Once you have uploaded your app and test suite, you can execute your test by running the following command: - -> Enter your **BASIC_AUTH_TOKEN**, **APP_ID** (generated in the fourth step) and **TEST_SUITE_ID** (generated in the fifth step) in the below command. - -```javascript -curl --location 'https://mobile-api.lambdatest.com/framework/v1/espresso/build' \ ---header 'Authorization: Basic BASIC_AUTH_TOKEN' \ ---header 'Content-Type: application/json' \ ---data '{ -"app": "APP_ID", //enter your app-id -"testSuite": "TEST_SUITE_ID", //enter your test-suite id -"device": [ -"Galaxy.*" -], -"smartUI.project": "Espresso-SmartUI-Project", -"smartUI.build": "Espresso-SmartUI-Build", // Optional -"smartUI.cropNavigationBar" : true, // Optional (By default false) -"smartUI.cropStatusBar" : true, // Optional (By default true) -"queueTimeout": 300, -"IdleTimeout": 30, -"deviceLog": true, -"network": false, -"visual": true, -"build": "Proverbial-Espresso-Test", -"singleRunnerInvocation": false -}' -``` - -> You can check the executed builds over at [TestMu AI SmartUI](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/). - -## Types of Sharding -Sharding can be categorized into two types: - -**Manual Sharding**: This feature enables users to manually specify shards along with their definitions, offering multiple strategies such as class, package, and more. By doing so, users can have greater flexibility and control over the sharding process. - -Refer to the sample `.yaml` file here - -```bash title="SampleYamlFile.yaml" -version: "0.2" -concurrency: 2 -runson: android -autosplit: false -globalTimeout: 180 #MAXQUEUETIMEOUT - -framework: -name: "android/espresso" -args: -buildName: "Espresso" -video: true -deviceLog: true - -# You can use either the appId (lt://APP1234567) or provide the path of the application using appPath. Both examples are given below. - -#highlight-next-line -appPath: Proverbial.apk - -testSuitePath: ProverbialExpressoTest.apk -# We have used the appPath and testSuitePath here. - -#highlight-next-line -appId: lt://APP1010461471690377432133206 -testSuiteAppId: lt://APP10104592261690377454846669 -# We have used the appId and testSuiteAppID here. - -deviceSelectionStrategy: all -devices: ["Galaxy.*", "Pixel.*"] - -smartUI: -project: "Espresso-SmartUI-Project" - -shards: -mappings: -- name: shard1 -strategy: "class" -values: ["com.lambdatest.proverbial.BrowserTest"] -# The strategy for this shard is based on "class". -# This shard will run all tests from the class com.lambdatest.proverbial.BrowserTest. -- name: shard2 -strategy: "package" -values: ["com.lambdatest.proverbial"] -# The strategy for this shard is based on "package". -# This shard will run all tests that belong to the package com.lambdatest.proverbial.l. -- name: shard3 -strategy: "skipClass" -values: ["com.lambdatest.proverbial.BrowserTest"] -# The strategy for this shard is to skip a specific class. -# This shard will avoid running tests from the class com.lambdatest.proverbial.BrowserTest. -``` - -**When shards are added** - -If you are using the `deviceSelectionStrategy: all`,then in that case all the specified shards will be executed on every device available. -**For example:** If there are 3 shards, 2 devices in list (d1, d2) and test suites contains total 10 test cases. -Then, d1 has 3 shards, i.e., total 3 devices of d1 configuration will be used. 10 test cases distributed among these 3 shards and similarly d2 also has 3 shards, i.e., total 3 devices of d2 configuration will be used. 10 test cases will be distributed among these 3 shards. - -If you are using the `deviceSelectionStrategy: any`, then in that case all the mentioned shards will be executed on just one device from the provided list. -**For example:** If there are 2 shards mentioned in `.yaml`and 3 devices mentioned, the system will create 2 shards. These shards might use any 2 devices from the given 3 configurations. It's also possible that the same device configuration could be used for both shards. In this setup, test cases will be distributed between these shards. - -**Auto Sharding** : The system intelligently determines the distribution of tests across devices, employing specific criteria to optimize the testing process. - -Refer to the sample `.yaml` file here - -```bash title="SampleYamlFile.yaml" -version: "0.2" -concurrency: 2 -runson: android -autosplit: true -globalTimeout: 180 #MAXQUEUETIMEOUT - -framework: -name: "android/espresso" -args: -buildName: "Espresso" -video: true -deviceLog: true - -# You can use either the appId (lt://APP1234567) or provide the path of the application using appPath. Both examples are given below. - -#highlight-next-line -appPath: Proverbial.apk - -testSuitePath: ProverbialExpressoTest.apk -# We have used the appPath and testSuitePath here. - -#highlight-next-line -appId: lt://APP1010461471690377432133206 -testSuiteAppId: lt://APP10104592261690377454846669 -# We have used the appId and testSuiteAppID here. - -smartUI: -project: "Espresso-SmartUI-Project" - -deviceSelectionStrategy: all -devices: ["Galaxy.*", "Pixel.*"] - -``` - -**When shards aren't added** - -If you are using the `deviceSelectionStrategy: all`, then in that case the tests will be executed on all mentioned devices in `.yaml` based on the concurrency. -**For example:** If the concurrency is 3, 2 devices in list (d1, d2) and test suites contains total 10 test cases. -Then, d1 has 3 shards, i.e., total 3 devices of d1 configuration will be used. 10 test cases distributed among these 3 shards and similarly d2 also has 3 shards, i.e., total 3 devices of d2 configuration will be used. 10 test cases will be distributed among these 3 shards. - -If you are using the `deviceSelectionStrategy: any`, then in that case all the specified tests will be executed on each device from the provided list, considering the concurrency setting. -**For example:** If the concurrency is set to 2 and 3 devices mentioned, the system will create 2 shards. These shards might use any 2 devices from the given 3 configurations. It's also possible that the same device configuration could be used for both shards. In this setup, test cases will be distributed between these shards. - -## Smart Crop With SmartUI - -The all-new **Real Device mobile notification status bar and navigation bar crop** feature in SmartUI allows you to take your visual regression testing workflows to the next level. With Smart Crop, you can crop the status bar and navigation bar or footer from screenshots, enabling them to focus solely on the core UI elements during visual comparisons. - -By leveraging machine learning algorithms, it accurately detects and crops the status bar and navigation bar from screenshots. With precise image processing techniques, SmartUI precisely identifies the location of status bar elements. By excluding it from visual comparisons, the focus is solely on critical UI elements. - -| Original Screenshot | Cropped Screenshot | -|---------------------|--------------------| -| | | - -## Best Practices - -### Project and Build Naming - -- Use descriptive, consistent names for projects and builds -- Include app version or release info in build names -- Avoid special characters that might cause issues -- Use consistent naming conventions across test runs - -**Example:** -```json -{ -"smartUI.project": "MyApp-VisualTests", -"smartUI.build": "Release-1.0.0" -} -``` - -### Device Selection - -- Test on devices that match your user base -- Include multiple device configurations for comprehensive coverage -- Use device patterns for consistent testing (e.g., `Galaxy.*`, `Pixel.*`) - -### Smart Crop Configuration - -- Enable `cropStatusBar` to focus on core UI elements -- Enable `cropNavigationBar` for Android devices -- Test cropped screenshots to ensure important content isn't removed - -### Test Organization - -- Use sharding for parallel test execution -- Group related tests in same build -- Use meaningful test names for better organization - -### App and Test Suite Management - -- Upload apps and test suites before execution -- Use app IDs (`lt://APP...`) for faster execution -- Keep app and test suite versions synchronized - -### App and Test Suite Management - -- Upload apps and test suites before execution -- Use app IDs (`lt://APP...`) for faster execution -- Keep app and test suite versions synchronized - -## Troubleshooting - -### Issue: Screenshots Not Captured - -**Symptoms**: Tests run but no screenshots appear in SmartUI dashboard - -**Possible Causes**: -- `visual: true` not set in request -- Incorrect project name -- Network connectivity issues -- Authentication issues - -**Solutions**: -1. Verify `visual: true` is set in API request: -```json -{ -visual": true, -"smartUI.project": "ProjectName" -} -``` - -2. Check project name matches exactly (case-sensitive) - -3. Verify authentication token is correct: - - Generate base64 encoded token from username:accesskey - - Ensure token is included in Authorization header - -4. Check network connectivity to TestMu AI - -### Issue: Project Not Found" Error - -**Symptoms**: Error indicating SmartUI project cannot be found - -**Possible Causes**: -- Project name typo or mismatch -- Project deleted -- Wrong account credentials - -**Solutions**: -1. Verify project exists in SmartUI dashboard -2. Copy project name directly from dashboard -3. Check credentials match the account with the project -4. Ensure project name is in API request - -### Issue: App Upload Fails - -**Symptoms**: App upload returns error or fails - -**Possible Causes**: -- Invalid APK file -- File size too large -- Network issues -- Authentication problems - -**Solutions**: -1. Verify APK file is valid and not corrupted -2. Check file size limits -3. Retry upload with stable network connection -4. Verify authentication credentials - -### Issue: Test Execution Fails - -**Symptoms**: Test suite execution fails or times out - -**Possible Causes**: -- Invalid test suite APK -- Device not available -- Timeout settings too low -- Test suite errors - -**Solutions**: -1. Verify test suite APK is valid -2. Check device availability -3. Increase `queueTimeout` and `IdleTimeout`: -```json -{ -queueTimeout: 600, -IdleTimeout: 60 -} -``` - -4. Review device logs for test errors - -### Issue: Screenshots Show Incorrect Content - -**Symptoms**: Screenshots captured but show wrong screen or state - -**Possible Causes**: -- App state issues -- Timing problems -- Navigation issues - -**Solutions**: -1. Ensure app is in correct state before test execution -2. Add appropriate waits in test code -3. Verify test navigation flow - -### Getting Help - -If you encounter issues not covered here: - -- Review the [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) for detailed solutions -- Check [SmartUI Configuration Options](/support/docs/smartui-sdk-config-options) documentation -- See [Handling Dynamic Data](/support/docs/smartui-handle-dynamic-data) for dynamic content issues -- Visit [TestMu AI Support](https://www.testmuai.com/support) for additional resources -- Contact support at support@testmuai.com or use [24/7 Chat Support](https://www.testmuai.com/support) - -## Additional Resources - -- [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) -- [SmartUI Configuration Options](/support/docs/smartui-sdk-config-options) -- [Handling Dynamic Data](/support/docs/smartui-handle-dynamic-data) -- [Baseline Management](/support/docs/smartui-baseline-management) -- [Running Your First Project](/support/docs/smartui-running-your-first-project) -- [Appium Hooks Documentation](/support/docs/smartui-appium-hooks) diff --git a/static/docs/executing-tesbo-automation-script-over-online-selenium-grid.md b/static/docs/executing-tesbo-automation-script-over-online-selenium-grid.md deleted file mode 100644 index f0c42a336..000000000 --- a/static/docs/executing-tesbo-automation-script-over-online-selenium-grid.md +++ /dev/null @@ -1,145 +0,0 @@ -# Selenium With Tesbo - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Run [Tesbo](https://tesbo.io/) keyword-driven automation scripts on the TestMu AI cloud grid. Tesbo is a hybrid framework built on Selenium that lets you write automation scripts in plain English using a keyword-driven approach. - -**Sample repo** -All the code used in this guide is available in the sample repository. - - View on GitHub - -## Prerequisites - -Make sure you have the following set up before you start. - -1. [Create a TestMu AI account](https://www.testmuai.com/register/) if you don't have one. -2. Get your **Username** and **Access Key** from the [TestMu AI Dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard). -3. Install [Java Development Kit (JDK)](https://www.oracle.com/java/technologies/downloads/) 1.6 or higher. -4. Install an IDE such as Eclipse or IntelliJ. - -## Step 1: Clone the Sample Project - -Pull the sample repo to your local machine and navigate into the project directory. - -```bash -git clone https://github.com/LambdaTest/LamdaTest_Tesbo_Demo -cd LamdaTest_Tesbo_Demo -``` - -## Step 2: Set Your Credentials - -Add your TestMu AI credentials as environment variables so the test can authenticate with the grid. - -Visit the [TestMu AI Dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard), navigate to the left sidebar, and select **Credentials**. Copy your **Username** and **Access Key**, then set them as environment variables: - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -## Step 3: Configure Your Test Capabilities - -Define the browser, version, and OS for your test run. - -Update `config.json` to point to the TestMu AI Selenium Grid. Set the Selenium address, enable the grid, and define your desired capabilities: - -```json -{ -"run": { -"baseUrl": "https://www.seleniumeasy.com/test/", -"seleniumAddress": "https://{userName}:{ApiKey}@hub.lambdatest.com/wd/hub", -"by": { -"tag": ["FO1"] -}, -"browser": { -"name": ["chrome"] -}, -"capabilities": { -"chrome": { -"build": "Tesbo_With_TestMuAI", -"name": "Tesbo", -"platformName": "Windows 10", -"browserName": "Chrome", -"browserVersion": "latest" -} -}, -"parallel": { -"status": false, -"count": "3" -}, -"retryAnalyser": { -"count": "0" -}, -"highlightElement": true, -"IsGrid": true -} -} -``` - -Key settings: -- **seleniumAddress** - Points to the TestMu AI hub URL with your credentials -- **IsGrid** - Set to `true` to run on the cloud grid -- **capabilities** - Define browser, version, and platform - -Use the [Capabilities Generator](https://www.testmuai.com/capabilities-generator/) to auto-generate capabilities for any browser, version, and OS combination. - -## Step 4: Run the Test - -Trigger the test from your IDE. - -Hit the run button in your IDE. Your Tesbo test scripts execute on the TestMu AI Selenium Grid. - -## Step 5: Run Tests in Parallel - -Run multiple Tesbo tests simultaneously across different browsers to reduce execution time. - -Set the parallel status to `true` and specify the count in `config.json`: - -```json -"parallel": { -"status": true, -"count": "3" -} -``` - -To run across multiple browsers, add more capabilities: - -```json -"capabilities": { -"chrome": { -"build": "Tesbo_With_TestMuAI", -"name": "Tesbo", -"platformName": "Windows 10", -"browserName": "Chrome", -"browserVersion": "latest" -}, -"firefox": { -"build": "Tesbo_With_TestMuAI", -"name": "Tesbo", -"platformName": "Windows 10", -"browserName": "Firefox", -"browserVersion": "latest" -}, -"safari": { -"build": "Tesbo_With_TestMuAI", -"name": "Tesbo", -"platformName": "MacOS Catalina", -"browserName": "Safari", -"browserVersion": "latest" -} -} -``` - -## Step 6: View Your Results - -Check the Automation Dashboard to see exactly what happened during your test. - -Visit the [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) to see your test results. Each session includes: - -- **Video recording** of the full test execution -- **Screenshots** captured at each step -- **Console logs** from the browser -- **Network logs** for every request and response -- **Selenium command logs** showing each driver action diff --git a/static/docs/export-test-cases.md b/static/docs/export-test-cases.md deleted file mode 100644 index 6c304c837..000000000 --- a/static/docs/export-test-cases.md +++ /dev/null @@ -1,37 +0,0 @@ -# Export Test Cases - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Export your test cases in bulk as a CSV file for tracking, analysis, and sharing with your team. You can select which fields to include, choose how steps are structured in the output, and download the file in a few clicks. - -## How to Export - -### Step 1: Select Test Cases -Navigate to the **Test Case Listing** page, bulk select the test cases you want to export, and click the **Bulk Action** button. - -### Step 2: Choose Export to CSV -Click **Export to CSV** from the dropdown menu. - -### Step 3: Select Fields for Export -Choose which test case fields to include in the exported CSV file. You can select or deselect individual fields, or use the **Select All** checkbox to include everything. - -Available fields include: Test Case ID, Test Case Title, Project Name, Folder Name, Description, Priority, Type, Status, Tags, Preconditions, Manual Steps, Expected Outcomes, BDD Scenarios, KaneAI Steps, and more. - -The ability to export **KaneAI Steps** is being rolled out gradually and may not be available on all accounts yet. -- Only the **natural language instruction** of each KaneAI step is exported. Steps such as Execute JavaScript, Execute API, etc. will not include the underlying code in the CSV. -- If you edit or modify KaneAI steps, it may take up to **1 minute** for the changes to reflect in the exported CSV. - -### Step 4: Choose CSV Layout -Select a **CSV Layout** to control how steps are structured in the exported file: - -| Layout | Description | -|---|---| -| **Single Row** | Steps, outcomes, KaneAI steps, and BDD scenarios are included in the same row as the test case, using a numbered format (e.g., `1. Step one 2. Step two`). | -| **Multiple Rows** | Each step, outcome, KaneAI step, or BDD scenario is placed in a separate row without numbered prefixes. | - -### Step 5: Download the CSV -Click **Export to CSV** to download the file. - -Here is an example of an exported CSV file with single row CSV layout: - -The exported file includes all the metadata fields that were selected during export. diff --git a/static/docs/extended-debugging-options.md b/static/docs/extended-debugging-options.md deleted file mode 100644 index 7fd02519f..000000000 --- a/static/docs/extended-debugging-options.md +++ /dev/null @@ -1,370 +0,0 @@ -# Extended Debugging Options for Selenium Testing - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI's Extended Debugging Options let you debug and optimize your Selenium tests by giving you granular control over network behavior and system performance. These debugging features allow you to intercept and modify network requests, simulate various CPU and network conditions, and download network analysis files within your test automation workflow. - -With Extended Debugging Options, you can replicate real-world scenarios such as slow networks, resource-constrained devices, API failures, and connectivity issues, ensuring your application delivers a solid user experience across diverse conditions and environments. - -## How Extended Debugging Options Help You - -Use these features to control network requests, test performance under constraints, and analyze network traffic. - -Extended Debugging Options on TestMu AI provide capabilities for advanced testing and debugging scenarios: - -- **Network Request Control**: Intercept and modify outgoing requests to test API failures, redirects, and mock responses without setting up backend infrastructure. -- **Performance Testing**: Simulate various CPU and network throttling conditions to understand how your application performs on low-end devices or poor network connections. -- **Network Analysis**: Download HAR (HTTP Archive) files to perform detailed analysis of network traffic, load times, and resource optimization. -- **Realistic Test Scenarios**: Create test conditions that mirror real-world user experiences, including offline modes, slow connections, and server errors. - -## Available Extended Debugging Methods - -Reference these methods to intercept requests, throttle resources, and download HAR files. - -TestMu AI supports the following extended debugging methods: - -| Method | Description | -|--------|-------------| -| `lt:intercept:redirect` | Redirect network requests to different URLs | -| `lt:intercept:response` | Mock responses for intercepted requests | -| `lt:intercept:error` | Simulate error responses for requests | -| `lt:throttle:cpu` | Simulate different CPU performance levels | -| `lambda-throttle-network` | Configure and simulate network conditions | -| `lt:downloadHAR` | Download HTTP Archive files for analysis | - -## 1. Intercept Network Requests - -Modify outgoing network requests to test redirect, mock, and error scenarios. - -TestMu AI provides three methods to intercept and modify network requests, enabling you to test how your application behaves under different network conditions, mock API responses, and simulate error scenarios. - -### Method 1: Redirect Requests (`lt:intercept:redirect`) - -Redirect outgoing requests to a different URL using the `lt:intercept:redirect` command. - -#### Parameters - -| Parameter | Type | Required | Description | -|-----------|------|----------|-------------| -| `url` | String | Yes | URL pattern to intercept. | -| `redirectUrl` | String | Yes | Target URL to redirect the request to. | - -#### Example Usage - -**Python:** -```python -driver.execute_script("lt:intercept:redirect", { -"url": "https://www.google.com", -"redirectUrl": "https://www.bing.com" -}) -driver.get("https://www.google.com") -``` - -**Node.js:** -```javascript -await driver.executeScript("lt:intercept:redirect", { -url: "https://www.google.com", -redirectUrl: "https://www.bing.com" -}); -await driver.get("https://www.google.com"); -``` - -**Response:** -```json -{ -"status": "success", -"message": "Requests to 'https://www.google.com' will be redirected to 'https://www.bing.com'" -} -``` - -### Method 2: Mock Response (`lt:intercept:response`) - -Mock a custom response for the intercepted URL using the `lt:intercept:response` command. - -#### Parameters - -| Parameter | Type | Required | Description | -|-----------|------|----------|-------------| -| `url` | String | Yes | URL pattern to intercept. | -| `response` | Object | Yes | Response object containing status, headers, and body. | -| `response.status` | Integer | No | HTTP status code (default: 200). | -| `response.headers` | Object | No | Custom response headers as key-value pairs. | -| `response.body` | String | No | Response body content (use JSON string for JSON responses). | - -#### Example Usage - -**Python:** -```python -driver.execute_script("lt:intercept:response", { -"url": "https://www.amazon.com", -"response": { -"status": 200, -"headers": { -"Content-Type": "application/json", -"keyheader": "valueheader" -}, -"body": "{\"keybody\":\"valuebody\"}" -} -}) -driver.get("https://www.amazon.com") -``` - -**Node.js:** -```javascript -await driver.executeScript("lt:intercept:response", { -url: "https://jsonplaceholder.typicode.com/todos/1", -response: { -status: 200, -headers: { -"Content-Type": "application/json" -}, -body: JSON.stringify({ -id: 999, -title: "Custom mocked response", -completed: true -}) -} -}); -await driver.get("https://jsonplaceholder.typicode.com/todos/1"); -``` - -**Response:** -```json -{ -"status": "success", -"message": "Mock response configured for the specified URL" -} -``` - -### Method 3: Simulate Error Response (`lt:intercept:error`) - -Inject error responses to test how your application handles failures using the `lt:intercept:error` command. - -#### Parameters - -| Parameter | Type | Required | Description | -|-----------|------|----------|-------------| -| `url` | String | Yes | URL pattern to intercept. | -| `error` | String | Yes | Error type to simulate. See supported error types below. | - -#### Example Usage - -**Python:** -```python -driver.execute_script("lt:intercept:error", { -"url": "https://www.testmuai.com", -"error": "TimedOut" -}) -driver.get("https://www.lambdatest.com") -``` - -**Node.js:** -```javascript -await driver.executeScript("lt:intercept:error", { -url: "https://example.com/images/*", -error: "Failed" -}); -await driver.get("https://example.com/images/photo.jpg"); -``` - -**Response:** -```json -{ -"status": "success", -"message": "Error 'TimedOut' configured for the specified URL" -} -``` - -#### Supported Error Types - -| Error Type | Description | -|------------|-------------| -| `Failed` | Generic network failure | -| `Aborted` | Request was aborted | -| `TimedOut` | Request timed out | -| `AccessDenied` | Access to resource denied | -| `ConnectionClosed` | Connection closed unexpectedly | -| `ConnectionReset` | Connection reset by peer | -| `ConnectionRefused` | Connection refused by server | -| `ConnectionAborted` | Connection aborted | -| `ConnectionFailed` | Connection failed to establish | -| `NameNotResolved` | DNS name resolution failed | -| `InternetDisconnected` | Internet connection lost | -| `AddressUnreachable` | Network address unreachable | - -## 2. Throttle CPU Performance - -Simulate slower CPU to measure application performance under resource constraints. - -The `lt:throttle:cpu` method simulates lower or higher CPU usage on the testing device, allowing you to measure your application's performance under resource constraints. - -### Parameters - -| Parameter | Type | Required | Description | -|-----------|------|----------|-------------| -| `rate` | Integer | Yes | Rate of slowdown. Example: `2` equals 2x slowdown, `4` equals 4x slowdown. | - -### Example Usage - -**Python:** -```python -driver.execute_script("lt:throttle:cpu", {"rate": 4}) -driver.get("https://lambdatest.com") -``` - -**Node.js:** -```javascript -await driver.executeScript("lt:throttle:cpu", { rate: 4 }); -await driver.get("https://www.wikipedia.org"); -``` - -**Response:** -```json -{ -"status": "success", -"message": "CPU throttled to 4x slowdown." -} -``` - -**CPU Throttling Rates** -- A rate of `1` means no throttling (normal CPU performance) -- A rate of `2` means 2x slower than normal -- A rate of `4` means 4x slower than normal -- Higher values simulate lower-end devices or heavy CPU load scenarios - -## 3. Throttle Network Conditions - -Simulate various network speeds and latency to test application behavior on different connections. - -The `lambda-throttle-network` method enables you to simulate various network conditions including slower speeds, high latency, and offline modes. This helps ensure your application performs well across different connection types. - -### Parameters - -| Parameter | Type | Required | Description | -|-----------|------|----------|-------------| -| `download` | Integer | Conditional | Download speed in kb/s (required for custom configuration). | -| `upload` | Integer | Conditional | Upload speed in kb/s (required for custom configuration). | -| `latency` | Integer | Conditional | Round Trip Time (RTT) in milliseconds (required for custom configuration). | - -Alternatively, you can pass a predefined network profile name as a string (e.g., `"Regular 3G"`, `"Offline"`). - -### Custom Network Configuration - -**Python:** -```python -driver.execute_script("lambda-throttle-network", { -"download": 1000, -"upload": 750, -"latency": 20 -}) -driver.get("https://lambdatest.com") -``` - -**Node.js:** -```javascript -await driver.executeScript("lambda-throttle-network", { -download: 1000, -upload: 500, -latency: 40 -}); -await driver.get("https://www.cnn.com"); -``` - -**Response:** -```json -{ -"status": "success", -"message": "Network conditions set with 1000 kb/s download, 750 kb/s upload, and 20 ms latency." -} -``` - -### Using Predefined Network Profiles - -**Python:** -```python -driver.execute_script("lambda-throttle-network", "Offline") -driver.get("https://lambdatest.com") -``` - -**Node.js:** -```javascript -await driver.executeScript("lambda-throttle-network", "Regular 3G"); -await driver.get("https://www.nytimes.com"); -``` - -### Predefined Network Profiles - -TestMu AI provides predefined network profiles for quick testing across common connection types: - -| Profile | Download Speed | Upload Speed | Latency (ms) | -|---------|----------------|--------------|--------------| -| `offline` | 0 kb/s | 0 kb/s | 0 | -| `GPRS` | 50 kb/s | 20 kb/s | 500 | -| `Regular 2G` | 250 kb/s | 50 kb/s | 300 | -| `Good 2G` | 450 kb/s | 150 kb/s | 150 | -| `Regular 3G` | 750 kb/s | 250 kb/s | 100 | -| `Good 3G` | 1 Mb/s | 750 kb/s | 40 | -| `Regular 4G` | 4 Mb/s | 3 Mb/s | 20 | -| `DSL` | 2 Mb/s | 1 Mb/s | 5 | -| `WiFi` | 30 Mb/s | 15 Mb/s | 2 | -| `online` | No Restrictions | No Restrictions | No Restrictions | - -## 4. Download HAR File - -Download network activity data in HAR format for detailed performance analysis. - -The `lt:downloadHAR` method downloads network activity data in HAR (HTTP Archive) format, enabling detailed analysis of network performance, resource loading times, and HTTP transactions. - -### Parameters - -| Parameter | Type | Required | Description | -|-----------|------|----------|-------------| -| `job_id` | String | Yes | Unique job identifier for the test session. | -| `output_file` | String | Yes | Filename to save the HAR file. | - -### Example Usage - -**Python:** -```python -driver.execute_script("lt:downloadHAR", { -"job_id": "123456", -"output_file": "network.har" -}) -``` - -**Node.js:** -```javascript -await driver.executeScript("lt:downloadHAR", { -job_id: "123456", -output_file: "network.har" -}); -``` - -**Response:** -```json -{ -"status": "success", -"message": "HAR file downloaded as 'network.har'." -} -``` - -**HAR File Analysis** -HAR files can be analyzed using tools like: -- Chrome DevTools (Network tab - right-click - "Save all as HAR") -- Online HAR analyzers -- Performance monitoring tools - -These files contain detailed information about request/response headers, timing data, cookies, and more. - -## Best Practices - -Follow these guidelines when using extended debugging options in your tests. - -When using Extended Debugging Options on TestMu AI, consider the following best practices: - -- **Use Wildcards Wisely**: When intercepting requests, use specific URL patterns to avoid unintended interceptions. -- **Test Incrementally**: Start with mild throttling conditions and gradually increase constraints to identify performance breaking points. -- **Combine Methods**: Use multiple methods together (e.g., network throttling + CPU throttling) to simulate realistic low-end device scenarios. -- **Analyze HAR Files**: Download HAR files for failed tests to identify network-related issues and performance bottlenecks. - -By using these Extended Debugging Options, you can create test scenarios that validate your application's behavior under diverse real-world conditions, ensuring a reliable user experience. diff --git a/static/docs/extent-report.md b/static/docs/extent-report.md deleted file mode 100644 index bbefd2e58..000000000 --- a/static/docs/extent-report.md +++ /dev/null @@ -1,90 +0,0 @@ -# Extent Report - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Extent Reports is a powerful reporting library used in test automation frameworks to generate visually appealing and detailed test reports. It provides insights into the status of each test case, including whether they passed, failed, or were skipped, along with additional information such as logs, screenshots, and system/environment details. This makes it especially popular in Selenium, Appium, and API testing frameworks. - -## Steps to Generate Extent Reports `(Version <= 2)` on HyperExecute -Follow these steps to enable Extent Reports for your HyperExecute job: - -### Step 1: Add Dependency -If using Maven, add the following dependency to your `pom.xml` file: - -```xml title="pom.xml" - -com.relevantcodes -extentreports -2.41.2 - -``` - -### Step 2: Create an Extent Report Listener -Create a class, e.g., `ExtentReportListenerV2.java`, to initialize and flush Extent Reports during test execution. This listener will log each test case’s status to the report. - -```java title="ExtentReportListenerV2.java" -import com.relevantcodes.extentreports.ExtentReports; -import com.relevantcodes.extentreports.ExtentTest; -import com.relevantcodes.extentreports.LogStatus; -import org.testng.ITestContext; -import org.testng.ITestListener; -import org.testng.ITestResult -public class ExtentReportListenerV2 implements ITestListener { -private static ExtentReports extent; -private static ThreadLocal test = new ThreadLocal<>() -@Override -public void onStart(ITestContext context) { -// Initialize ExtentReports with the report path -extent = new ExtentReports("extent-report.html", true); -extent.addSystemInfo("Environment", "QA").addSystemInfo("User", "Tester"); -} -``` - -## Steps to Generate Extent Reports `(Version > 2)` on HyperExecute -Follow these steps to enable Extent Reports for your HyperExecute job: - -### Step 1: Add Dependency -If using Maven, add the latest extentreports dependency to `pom.xml` file: - -```xml title="pom.xml" - -com.aventstack -extentreports -5.0.9 - -``` - -### Step 2: Create an Extent Report Listener -For Extent Reports > 2, use `ExtentHtmlReporter` to generate and customize the HTML report. Create `ExtentReportListener.java`: - -```java title="ExtentReportListener.java" -import com.aventstack.extentreports.ExtentReports; -import com.aventstack.extentreports.ExtentTest; -import com.aventstack.extentreports.reporter.ExtentHtmlReporter; -import com.aventstack.extentreports.reporter.configuration.Theme; -import org.testng.ITestContext; -import org.testng.ITestListener; -import org.testng.ITestResult -public class ExtentReportListener implements ITestListener { -private static ExtentReports extent; -private static ThreadLocal test = new ThreadLocal<>() -@Override -public void onStart(ITestContext context) { -ExtentHtmlReporter htmlReporter = new ExtentHtmlReporter("extent-report.html"); -htmlReporter.config().setTheme(Theme.STANDARD); -htmlReporter.config().setDocumentTitle("Test Report"); -htmlReporter.config().setReportName("Automation Test Results") -extent = new ExtentReports(); -extent.attachReporter(htmlReporter); -} -``` - -## Configure the HyperExecute YAML File -In your HyperExecute YAML configuration, define the [`report`](/support/docs/deep-dive-into-hyperexecute-yaml/#report) parameters like this: - -```yaml -report: true -partialReports: -type: json -location: reports/json -frameworkName: extent -``` diff --git a/static/docs/fastlane-with-testmu.md b/static/docs/fastlane-with-testmu.md deleted file mode 100644 index 7eda3b6df..000000000 --- a/static/docs/fastlane-with-testmu.md +++ /dev/null @@ -1,68 +0,0 @@ -# Integration With fastlane Plugin - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -fastlane is a CI/CD tool to automate beta deployments and releases for Android and iOS apps. It is an open-source platform for Android and iOS app deployment. It will take care of all the tasks like testing, generating screenshots, dealing with code signing, and releasing your application. Provides integration support with all the major CI systems. Adding the TestMu AI fastlane plugin in Fastfile will help you upload the app to the server and run tests on TestMu AI real devices. - -## Prerequisites - -**Github Sample** - -To get started, please try our **[fastlane Sample](https://github.com/LambdaTest/fastlane-android-sample)** GitHub repository.. - -```bash -git clone https://github.com/LambdaTest/fastlane-android-sample -``` - -To upload an app to TestMu AI for app testing, you would need the following things : - -* Make sure you have fastlane installed on the machine. Please refer fastlane docs for installation. - -* TestMu AI Credentials - - * Please ensure that you have your TestMu AI credentials handy, including your TestMu AI username and access key. You will need to set them up in the Fastfile. To retrieve these credentials, simply click on the key icon located near the help button on your TestMu AI automation [dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build). - -``` bash -LT_USERNAME="YOUR_LAMBDATEST_USERNAME" -LT_ACCESS_KEY="YOUR_LAMBDATEST_ACCESS_KEY" -``` - -## Installing TestMu AI fastlane Plugin - -**Step 1:** Add TestMu AI [fastlane](https://rubygems.org/gems/fastlane-plugin-lambdatest) plugin by running. -``` -fastlane add_plugin lambdatest -``` - -**Step 2:** Add the below action in fastfile in desired lane to upload the app to TestMu AI. -``` -upload_to_lambdatest( -lt_username: ENV["LT_USERNAME"], -lt_access_key: ENV["LT_ACCESS_KEY"], -file_path: "app_file_path" -) -``` -or if you want to used custom_id. - -``` -upload_to_lambdatest( -lt_username: ENV["LT_USERNAME"], -lt_access_key: ENV["LT_ACCESS_KEY"], -file_path: "", -custom_id: " -) -``` -**_NOTE:_** custom_id is an optional field. - -**Step 3:** Once the app is uploaded to TestMu AI, this action will set an identifier to the uploaded app as APP_URL in the environment variable and can be accessed in the test. - -``` -String app = System.getenv("APP_URL"); -capabilities.setCapability("app", app); -``` - -**Step 4:** Execute the below command to upload app and run tests. - -``` -fastlane -``` diff --git a/static/docs/favro-integration.md b/static/docs/favro-integration.md deleted file mode 100644 index 8bbeb9d49..000000000 --- a/static/docs/favro-integration.md +++ /dev/null @@ -1,83 +0,0 @@ -# Favro Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Favro is a project management and collaboration tool set up by serial deep tech entrepreneurs. It works under an agile methodology that helps in planning, execution, collaboration, and creation to enhance the workflow of your organization. You can create your dashboard where you can track your various projects, interact, respond to ideas, handle projects with your teammates using multiple kanbans, timelines, and sheets. - -The TestMu AI Favro Integration allows you to create a task directly to your specified Favro dashboard from TestMu AI platform. Share your UI observations and input with your teammates anytime, by [capturing a screenshot](https://www.testmuai.com/full-page-screen-capture), even in the middle of your test session. You can annotate the screenshot & highlight your issue or input. The fields populated by you when marking as bug through TestMu AI are displayed as information on Favro for that testing instance. - -## How To Integrate Favro With Your TestMu AI Account? - -**Step 1:** Login to your TestMu AI account. You would need to have Admin or User level access to see and install integrations. - -**Step 2:** Select **'Integration'** from the left navigation menu bar. This will guide you to a screen where you will find a list of 3rd party applications, available to integrate with your TestMu AI account. - -**Step 3:** Click on **'ADD'** under the block that says 'Favro'. - -**Step 4:** Once you click on add, you'll redirected to the below screen. This will ask you to enter your e-mail address and API token. - ->* **What is my Favro E-mail? ** ->In this field you need to specify the **email id** through which you sign into your **Favro** account. ->* ** What is an API Token?** ->When two different applications interact with each other, a server-to-server communication is established. API token is needed for authenticating user identity from one server to another, by fetching user-owned resources available on the server from where the communication is initiated. API token has been a necessity for maintaining security and data integrity of any application. They are just as vital to an end user as they are to an application. API tokens are relevant if an end user logs in an application through a 2-step verification process. ->If you are logging into **Favro** for the first time, then you can generate a token for yourself by just entering a name of your choice. - -**Step 5:** Login your favro account and navigate to **'My Profile'** on the left. - -**Step 6:** Navigate to **'API Token'** on the left in the My Profile section. - -**Step 7:** Click on the **'Create New Token'** button. - -**Step 8:** Enter a name in the provided field and hit the **'Create Token'** button. - -**Step 9:** Once your API token is generated, copy the same by clicking on**'Copy to clipboard'**. - -**Step 10:** Then enter your e-mail address and paste the API token into the field provided at TestMu AI & hit the **'Install'** button. If you are using **'Self hosted' **Favro instance, instead of API token, you would just have to add your favro login password. - -If you are not logged into Favro then you will be redirected to the below screen. Kindly login to your account via gmail or create your account by given option. - -Go to **'Integrations'** again and you will be able to notice a **'green tick'** indicating that Favro is successfully installed. You are all set to experience **'one-click bug logging'** to share your issues directly from your TestMu AI account to your teammates on Favro project. - -## How To Log Your First Bug Through Favro Integration? - ->**Note:** If you are using Favro for the first time, then make sure to create a project for yourself. It is a pre-requisite in order to push screenshots from your TestMu AI account. - -**Step 1:** Go for any of the test from the left navigation menu. For demo, we will be taking '**Real Time Test**' option. - -**Step 2:** Present a URL of the web-app you need to test in the dialog box. After that, select any configuration for browser and operating system of your choice & hit '**Start**'. - -**Step 3:** After the VM is launched and operable. You can perform testing on your web-app for finding bugs. If a bug gets revealed, then you need to click on the **'Bug icon'** from the left panel for capturing a screenshot of the same. - -**Step 4:** After a screenshot is captured, you can annotate any issue or a task with an **'in-built image editor'**. Once you are done highlighting the bug, click on the button that says **'Mark as Bug'**. - -**Step 5:** After clicking on **'Mark as Bug'** button a form would open up. Fill the fields as per your requirement. ->* You can select which **organization** should the ticket go under. -* You can choose from the **collection** to create dashboards to manage your tasks. -* You can choose from the** widgets** to design your workspace. -* You can specify the testing phase in the **widget column**. -* You can give suitable** title** for your test. -* You also get to post a **description** to help relate the cause of the issue. - -At the bottom of the form you will find a button **'Create Issue'**. - -**Step 6:** After you click on **'Create Issue'**, you will be able to observe it being successfully marked through a single click effort. You will get **'prompt messages'** on top of your Virtual Machine indicating the progress of bug logging. After few seconds you will be notified with a prompt message **'Bug successfully marked'** indicating that the screenshot has been pushed to your workspace. - -**Step 7:** Visit your dashboard. You will be able to notice the logged issue right away! All the data you provided through TestMu AI would already be presented in it. TestMu AI automatically includes test environment details and related screenshots in the screenshot as attachments. - -Click on the **'Testing'** Widget Column and you will find all the data that you provided through TestMu AI already presented in it. - -TestMu AI automatically includes test environment details and related screenshots in the card as attachments. - -You can add your team members by navigating to **'Members'** on the top-left. - -## How To Remove Favro Integration? - ->You can work with one integration at a time. So if you would want to integrate to a similar 3rd party application, then you would have to **'remove'** your current integration. Here is how you can do that. - -**Step 1:** Login to your TestMu AI account. - -**Step 2:** Select 'Settings' from the left navigation menu bar & click on 'Integrations'. This will guide you to a screen where you will find 3rd party applications, available to integrate with your TestMu AI account. - -**Step 3:** Click on **'Remove'**. You can find the Remove button right under the **'Favro'** block which would be highlighted with a green tick. - -That was all you need to know for TestMu AI + Favro Integration. Increase your productivity with our integrations. If you still have any questions for us, please feel free to let us know. Our experts are always **available on chat** to help you out with any roadblock regarding our product. Happy testing! diff --git a/static/docs/filter-your-selenium-tests.md b/static/docs/filter-your-selenium-tests.md deleted file mode 100644 index ff1812284..000000000 --- a/static/docs/filter-your-selenium-tests.md +++ /dev/null @@ -1,468 +0,0 @@ -# Organizing Your Selenium Tests and Builds - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Once your Selenium tests are running on TestMu AI, the automation dashboard gives you several ways to keep them organized. You can filter tests, group them with custom tags, tag and split builds, edit individual test details during a run, and share results with your team. This document covers each of these. - -On each of the tabs available on the automation dashboard (Timeline, Automation Logs, and Analytics), a _filter toolbar_ helps you filter your tests based on selected values. The sections below start with the filter toolbar of each tab, then cover tagging, build splitting, editing test details, and sharing. - -## Filtering Tests on Timeline - -Use the filter toolbar on the Timeline tab to narrow down tests by date, user, build, and status. - -When you visit the [Timeline](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/timeline) on your automation dashboard, you can see a **Filter** icon that allows you to use different filters. - -### Understanding the Filter Toolbar - -Now let us see each component of this filter toolbar one by one. - -### 1. Date - -This option filters automation tests on your timeline based on a range of dates. Select a start date and an end date to filter tests that ran in this period. By default, it shows the range from the beginning until the current date. - -To filter tests based on custom dates, first select the starting date from the calendar window, then select the ending date. You can see a selected range after you click on the starting date. - -For example, to view tests that ran between 1st Feb - 3rd Feb, click on 1st Feb first and then on 3rd Feb. - -Once the ending date is selected, the tests are automatically sorted based on the selected custom dates. - -### 2. Users - -You might have multiple team members added to your TestMu AI account. To filter tests by the member who ran them, use this option in the filter toolbar. Click on the _Users_ option to see a list of members added to your account. Select the members whose tests you want to filter, and the tests filter automatically. You can select multiple members. - -### 3. Build - -Use this feature to see selected builds on your Timeline. Click on this option to see a list of all builds that have run. Select one or more as needed. - -### 4. Status - -This option on the filter toolbar helps you filter automation tests based on the current status of the test: - -* Running -* Queued -* Idle Timeout -* Queue Timeout -* Completed -* Passed -* Failed -* Error -* Stopped -* Cancelled -* Lambda Error -* Allocating VM - -Click on this option to see a list of all the above statuses. Select one or more to filter tests that have these statuses. - -### 5. Clear Filters - -Use this option to clear all applied filters. Clicking this option returns the Timeline to the default state with all filters removed. - -## Filtering Tests on Automation Logs - -Use the filter toolbar on the Automation Logs tab to filter by build, date, users, status, and tags. - -When you visit the [Automation Logs](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/logs) on your automation dashboard, you also see a filter toolbar. - -The components of this toolbar are: - -1. Build -2. Date -3. Users -4. Status -5. Tags -6. Clear Filters - -The Build, Date, Users, Status, and Clear Filters components behave the same as in the Timeline. Let us look at the Tags component in detail. - -### Filter Tests Using Tags - -You can filter your Selenium tests on TestMu AI using [custom tags](#group-tests-using-custom-tags). Click on the Tags option to select the tag by which you want to filter tests. You must have used the custom tags feature earlier to use this filter. - -Here is an example showing tests filtered using the _Demo1_ custom tag: - -You can also filter tests based on multiple tags. Select multiple custom tags at once from the filter toolbar: - -## Filtering Tests on Analytics - -Use the filter toolbar on the Analytics tab to filter by tags, users, environment, and date range. - -When you visit the [Analytics](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/analytics) on your automation dashboard, you also see a filter toolbar. - -The components of this toolbar are: - -1. Tags -2. Users -3. Environment -4. Date -5. Analytics Tab Timeline -6. Clear Filters - -The Tags, Users, Date, and Clear Filters behave the same as above. Let us look at the Environment and Analytics Tab Timeline components in detail. - -### Filter Tests Using Environment - -The Environment filter filters tests based on the browser and/or operating system used in the test. Refer to our [list of browsers](https://www.testmuai.com/list-of-browsers/) to understand the options available in this filter. - -### Filter Tests Using Analytics Tab Timeline - -The Analytics Tab Timeline option helps you quickly filter tests based on preset time periods: - -* TODAY -* THIS WEEK -* THIS MONTH - -Select any of these, and the tests that ran in that period get filtered out. - -By default, the _THIS WEEK_ filter is selected. You can use the Date filter instead to filter tests in a custom date period. If you select the Date filter, this filter is automatically removed. - -## Group Tests Using Custom Tags - -Group your automation tests with custom tags so you can view and filter them together on the dashboard. - -TestMu AI allows you to group your automation tests using custom tags. This section shows how to create custom tags and use them. For demonstration purposes, we use a [sample TestNG project](https://github.com/LambdaTest/Java-TestNG-Selenium) to run on the TestMu AI platform. - -### Create Custom Tags on the Selenium Grid - -You can create a custom tag while writing your Selenium automation tests. When you create the [Desired Capabilities](/docs/selenium-automation-capabilities/) via code, add the below lines of code. - -1. Create a String array that contains the names of your custom tags, separated by a comma. - -```javascript -// In case for just 1 tag, just add 1 element in the array -String[] customTags = {"Custom Tag"}; - -// In case for multiple tags, add them in the array separated by comma -String[] customTags = {"Tag 1", "Tag 2", "Tag 3", ...}; -``` - -2. Now add this custom tag in your Desired Capabilities instance: - -```javascript -DesiredCapabilities caps = new DesiredCapabilities(); -. -. - -// To create custom tags -caps.setCapability("tags", customTags); -``` - -For example: - -You have successfully created the custom tags. Let us now see how to view and group tests based on custom tags. - -### View Custom Tags on Your Timeline - -Navigate to [Timeline](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/timeline/) view of your automation dashboard, and toggle the **Build View** to **Test View**: - -You can see the applied custom tags below the tests in this Test View. - -### View Custom Tags on Your Automation Logs - -Navigate to [Automation Logs](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/logs) of your automation dashboard, and you can see the applied custom tags below the tests in the left panel. - -### Filter Tests Using Custom Tags - -You can filter tests on your automation dashboard with these custom tags. - -Navigate to [Automation Logs](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/logs) of your automation dashboard, and you can see a filter by name **Tags** in the filter toolbar. - -Click on it and select the Tag by which you want to filter the tests on your dashboard. For example, we have filtered the test using _Tag2_ custom tag: - -You can filter tests by selecting multiple custom tags at once from the filter toolbar: - -## Group and Filter Builds Using Build Tags - -Tag your builds so you can group and filter test builds on the automation dashboard. - -With TestMu AI, you can group your test builds with Build tags. This section shows how to create Build tags and use them to group or filter your test builds on the TestMu AI Automation Dashboard. - -### Create Build Tags - -While writing your automation tests, add another capability when you create the [Desired Capabilities](/docs/selenium-automation-capabilities/) via code. Follow the below steps: - -1. Create an array of Strings that contains your Build tags, each separated by a comma. - -```java -// For example, when you have only 1 tag -String[] buildTagList = {"Build Tag"}; - -// For example, when you have multiple tags -String[] buildTagList = {"Tag 1", "Tag 2", "Tag 3", ...}; -``` - -2. Now add this Build tag array in your Desired Capabilities instance. For example, while creating a sample Desired Capabilities instance in Java, the code will be: - -```java -// Creating the Build Tags -String[] buildTagList = {"Tag1", "Tag2", "Tag3", "BuildTagRishabh"}; - -DesiredCapabilities caps = new DesiredCapabilities(); -caps.setCapability("browser", "Safari"); -caps.setCapability("version", "13"); -caps.setCapability("platform", "macos Catalina"); -caps.setCapability("build", "Build Tags Demo"); -caps.setCapability("name", "Sample Test"); - -// To create custom tags -caps.setCapability("buildTags", buildTagList); - -System.out.println("Desired Caps: " + caps); -driver = new RemoteWebDriver(new URL("https://" + username + ":" + authkey + hub), caps); -``` - -You have successfully created the Build tags. Run your test and navigate to [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/timeline/) to see the tags in that build: - -### Guidelines for Creating Build Tags - -While creating Build Tags, follow the below guidelines: - -* Add a maximum of 5 custom tags to a build. -* You can update the existing build by specifying different tags. -* If you change a tag name or number of tags, no new build is created. The existing build is updated with the new tag to avoid unnecessary build creation. - -### Filter Tests Using Build Tags - -To group or filter the tests on your automation dashboard with Build tags, navigate to [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/timeline/), and you can see a filter by name **Build Tags** on the right corner of the filter toolbar. - -Select the Build tag by which you want to filter your test. For example, if we choose _Tag1_, only the test builds with the tag "Tag1" appear: - -You can choose multiple Build tags to filter your tests, as shown in the image below: - -## Split Builds with Build Inactivity Time - -Set a build inactivity time so tests that run after a defined idle period start a new build automatically. - -Build Splitting (Build Inactivity Time) lets you organize tests more efficiently. Set a time interval after which the system waits for the defined time and marks the build as completed. Any new builds you run after this time period are created as a new build. - -### How It Works - -Previously, when you ran a build on TestMu AI, all tests merged into the same build, making it difficult to identify tests that ran at specific intervals. Build Splitting logic helps you organize your tests better. - -Build Splitting requires you to set Build Inactivity Time. For instance, if the Build Inactivity Time is set to 5 minutes, the system waits 5 minutes after the last test completes before marking the build as complete. If you run a test before the inactivity time, it runs in the existing build. If you run a test (even with the same name) after the specified inactivity time (i.e., 5 minutes), it appears under a new build. - -### Set the Build Inactivity Time - -**Note** -1. By default, the build inactivity time is six hours. -2. Every user within the organization can set their own build inactivity time. - -1. Log in to your TestMu AI account. Don't have an account? [Sign up for free](https://www.testmuai.com/register/). - -2. From the top-right corner, click on the Profile avatar, and navigate to **Profile > Product Preferences**. - -3. Under the **Automation Build Preference**, select the **Build Inactivity Time** to choose your preferences. - -Once you set the build inactivity time, a confirmation notification appears: *Build Time updated successfully.* - -## Edit Individual Test Details - -Mark a test's final status or rename it during execution using JavascriptExecutor hooks. - -You can modify individual test details from your automation test build while running Selenium tests on TestMu AI. This section covers marking test status and renaming tests during execution. - -### Mark Test Status as Pass or Fail - -When you run Selenium tests on the TestMu AI grid, a test that your local assertions marked as failed may show as completed on the dashboard. Use the `lambda-status` hook to explicitly set the correct status. - -You can set these status values: `passed`, `failed`, `skipped`, `ignored`, `unknown`, `error`. - -```java -// Mark test as passed -((JavascriptExecutor) driver).executeScript("lambda-status=passed"); - -// Mark test as failed -((JavascriptExecutor) driver).executeScript("lambda-status=failed"); -``` - -```javascript -// Mark test as passed -await driver.executeScript('lambda-status=passed'); - -// Mark test as failed -await driver.executeScript('lambda-status=failed'); -``` - -```python -# Mark test as passed -driver.execute_script("lambda-status=passed") - -# Mark test as failed -driver.execute_script("lambda-status=failed") -``` - -```csharp -// Mark test as passed -((IJavaScriptExecutor)driver).ExecuteScript("lambda-status=passed"); - -// Mark test as failed -((IJavaScriptExecutor)driver).ExecuteScript("lambda-status=failed"); -``` - -```php -// Mark test as passed -$driver->executeScript("lambda-status=passed"); - -// Mark test as failed -$driver->executeScript("lambda-status=failed"); -``` - -```ruby -# Mark test as passed -driver.execute_script("lambda-status=passed") - -# Mark test as failed -driver.execute_script("lambda-status=failed") -``` - -Place the `lambda-status` call inside your test's teardown or `@AfterMethod` block so the status is set before the session ends. For the full list of Lambda Hooks, see [Lambda Hooks](/support/docs/lambda-hooks/). - -### Rename Your Test - -You can rename a running test to reflect dynamic data such as iteration count or data-driven parameters. Pass the new name through JavascriptExecutor: - -```java -((JavascriptExecutor) driver).executeScript("lambda-name=Your_test_name"); -``` - -```javascript -await driver.executeScript('lambda-name=Your_test_name'); -``` - -```python -driver.execute_script("lambda-name=Your_test_name") -``` - -```csharp -((IJavaScriptExecutor)driver).ExecuteScript("lambda-name=Your_test_name"); -``` - -```php -$driver->executeScript("lambda-name=Your_test_name"); -``` - -```ruby -driver.execute_script("lambda-name=Your_test_name") -``` - -## Share Test Results - -Retrieve a test's Session ID and build a shareable URL for its logs or execution video. - -TestMu AI lets you share individual test results with team members. Retrieve the Session ID from your test script, then build a shareable URL for automation logs or execution videos. - -### Get the Session ID - -Every test session on TestMu AI has a unique Session ID. Use the code below to retrieve it in your preferred language: - -```java -import org.openqa.selenium.remote.SessionId; - -SessionId session = ((RemoteWebDriver) driver).getSessionId(); -System.out.println("Session ID: " + session.toString()); -``` - -```javascript -const session = await driver.getSession(); -console.log("Session ID: " + session.getId()); -``` - -```python -session_id = driver.session_id -print("Session ID: " + session_id) -``` - -```csharp -var sessionId = ((RemoteWebDriver)driver).SessionId; -Console.WriteLine("Session ID: " + sessionId); -``` - -```php -$sessionId = $driver->getSessionID(); -echo "Session ID: " . $sessionId; -``` - -```ruby -session_id = driver.session_id -puts "Session ID: #{session_id}" -``` - -Once you have the Session ID, share the automation logs URL with your colleague: - -``` -https://automation.lambdatest.com/logs/?sessionID=YOUR_SESSION_ID -``` - -### Share Your Test Execution Video - -You can share a video recording of any test execution. Build the URL in the following format: - -``` -https://automation.lambdatest.com/public/video?testID={testid/sessionid}&auth=AUTH_TOKEN -``` - -#### Get Your TestID or SessionID - -Go to the **Automation Dashboard** and click on the test whose ID you need. Click the **"i"** button on the status bar to access the test metadata. - -After clicking the **"i"** button, you will see the test metadata including the SessionID. - -For example, if your SessionID is `HJKXM-RHZL1-SVPWY-AB8X6`, the URL becomes: - -``` -https://automation.lambdatest.com/public/video?testID=HJKXM-RHZL1-SVPWY-AB8X6&auth=AUTH_TOKEN -``` - -#### Generate the AUTH_TOKEN - -Create the AUTH_TOKEN by computing an MD5 hash of your `username:access_key` string. - -```java -MessageDigest m = MessageDigest.getInstance("MD5"); -String s = "username:access_key"; -m.update(s.getBytes(), 0, s.length()); -System.out.println("MD5: " + new BigInteger(1, m.digest()).toString(16)); -``` - -```javascript -var crypto = require('crypto'); -var token = crypto.createHash('md5').update("username:access_key").digest("hex"); -console.log("AUTH_TOKEN: " + token); -``` - -```python -import hashlib -token = hashlib.md5("username:access_key".encode('utf-8')).hexdigest() -print("AUTH_TOKEN: " + token) -``` - -```csharp -byte[] inputBytes = System.Text.Encoding.ASCII.GetBytes("username:access_key"); -byte[] hashBytes = System.Security.Cryptography.MD5.Create().ComputeHash(inputBytes); -StringBuilder sb = new StringBuilder(); -for (int i = 0; i < hashBytes.Length; i++) -{ -sb.Append(hashBytes[i].ToString("X2")); -} -Console.WriteLine("AUTH_TOKEN: " + sb.ToString()); -``` - -```php -$token = md5("username:access_key"); -echo "AUTH_TOKEN: " . $token; -``` - -```ruby -require 'digest' -token = Digest::MD5.hexdigest("username:access_key") -puts "AUTH_TOKEN: #{token}" -``` - -For example, if the generated AUTH_TOKEN is `331k534uf3toef`, the final URL becomes: - -``` -https://automation.lambdatest.com/public/video?testID=HJKXM-RHZL1-SVPWY-AB8X6&auth=331k534uf3toef -``` - -You must be logged into TestMu AI to access the sharing URL. - ->You can now filter, tag, split, edit, and share your Selenium tests and builds from the automation dashboard. If you have any questions, share them with us through our **24/7 chat support** or by mailing us at [support@testmuai.com](mailto:support@testmuai.com). diff --git a/static/docs/flock-integration.md b/static/docs/flock-integration.md deleted file mode 100644 index 6fa34c3ad..000000000 --- a/static/docs/flock-integration.md +++ /dev/null @@ -1,65 +0,0 @@ -# Flock Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Flock is a communication and collaboration platform for cross-functional teams. Using Flock, you can streamline all communication and decision-making processes. Flock provides top-notch features such as easy file sharing and management, simple onboarding, 24x7 support, and data security & privacy. - -Integrating Flock with TestMu AI, you can post bugs or issues to Flock's team channels directly from the TestMu AI platform and collaborate faster on websites and web apps issues. - -## Integrating Flock With Your TestMu AI Account - -1. Login to your TestMu AI account. Before configuring integrations, ensure you have administrator or user access. - -2. Select **Integrations** from the TestMu AI left menu. - -3. From the COMMUNICATION category, select **FLOCK**. - -4. It will redirect to the console where you will need to enter **Webhook URL**. - -### Generating The Webhook URL - -1. Login to your Flock account. - -2. From the left menu, click **Webhooks**. - -3. Select **Add** under Incoming Webhooks. - -4. Select the channel where you wish to post the message, provide a Webhook name, and choose an icon. - -5. After that, click **Save and Generate URL**. - -Paste the Webhook URL in the provided field and click **Install**. - -You now have successfully integrated Flock with your TestMu AI account. - -## One-Click Bug Logging With Flock Integration - -1. From the TestMu AI dashboard, navigate to **Real Time Testing** from the left menu. - -2. Specify the test URL, select the browser, browser versions, operating system, and click **START**. - -It will redirect you to the cloud-based virtual machine where you can perform web testing of websites (and web apps.) - -3. If you come across any UI glitch or issue, click the **Bug** icon from the left in-built tools to capture the screenshot of the web page. - -4. To annotate any issues or task, you can use the in-built image editor tools from the left menu. After annotating, click **Mark as Bug**. - -5. On clicking Mark as Bug, a ticket will show up. Fill the details like Assignee, Issue Type, Priority, Summary, and Description. Now click **CREATE ISSUE**. - ->**Note**: By default, the *Send to Flock* checkbox is marked as checked. - -Your bugs will be logged to the selected Flock's channel. - -6. You can view logged issues in your Flock's workspace. Marked bugs can be found in the set Flock channel along with test environment details. - -## Uninstalling The Flock Integration - -1. Login to your TestMu AI account. - -2. Select **Integrations** from the TestMu AI left menu. - -3. Click **REMOVE** adjacent to the Flock logo in the **MY INTEGRATION** category. - -Alternatively, you can remove the integration by navigating to Flock under the COMMUNICATION category and clicking **REMOVE**. - ->If you have any questions or need further assistance regarding TestMu AI + Flock integration, feel free to reach out to us at our 24x7 Chat Support or email us at [support@testmuai.com](mailto:support@testmuai.com). diff --git a/static/docs/flutter-supported-capabilities.md b/static/docs/flutter-supported-capabilities.md deleted file mode 100644 index 890f6c278..000000000 --- a/static/docs/flutter-supported-capabilities.md +++ /dev/null @@ -1,23 +0,0 @@ -# Flutter Supported Capabilities - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This document provide details about the features and capabilities supported for Espresso Framework on TestMu AI. - -| Capability Name | Data Type | Description | -|------|-----------|-------------| -| app | String | Enter the app id generated while uploading the app to the platform. Example:`lt://APP123456789123456789` | -| testSuite | String | Enter the test suite id generated while uploading the test suite to the platform. Example: `lt://APP123456789123456789` | -| device | String | Enter the name and os version of the device in "DeviceName-OSVersion" format. Example: `Pixel 6-12` or `Galaxy S21 Ultra 5G-11`. You can also choose to use regular expression for device name such as `Pixel.*-12`. | -| video | Boolean | Generate video for all the tests that have run. Example: `true`. | -| queueTimeout | Integer | Total execution time of the build. Example: `queueTimeout: 300`. Maximum value is `10800`| -| testTimeout | Integer | Max wait time for test execution (default value - `900` secs) | -| deviceLog | Boolean | To generate the device logs. Example: `true`. | -| build | String | Set the name of your Flutter test build. Example: `My Flutter Build`. | -| autoGrantPermissions: | Boolean | Boolean value to automatically grant the required permissions based on the [Android manifest](https://developer.android.com/guide/topics/manifest/manifest-intro) in the app’s **.apk** Example: `true`. | -| build | String | To set the Espresso build name. Example: `build: My Espresso Build`. | -| geoLocation | String | Set the geolocation [country code](/support/docs/appium-ip-geolocation/#supported-ip-geolocations) if you want to enable the same in your test. Example - `geoLocation: FR`| -| gpsLocation | Decimal | The capability should be defined to override the device’s default GPS coordinates and should be provided as a comma-separated string. Example: `"location": {"lat": "-50","long": "150.028333"}`| -| enableBluetooth | Boolean | Used to enable the bluetooth functionality during the session. Example: `enableBluetooth: true`| - -Ensure that the latitude is between -90 and 90, and the longitude is between -180 and 180. Otherwise, an error will occur like "Invalid GPS location: Latitude must be between -90 and +90, Longitude must be between -180 and +180." diff --git a/static/docs/fogbugz-integration.md b/static/docs/fogbugz-integration.md deleted file mode 100644 index 0245060c1..000000000 --- a/static/docs/fogbugz-integration.md +++ /dev/null @@ -1,59 +0,0 @@ -# FogBugz Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -FogBugz is a web-based project management tool with features like bug and issue tracking, discussion boards, wikis, customer relationship management, and more. It can help you manage your projects and communicate with your team. FogBugz is designed to streamline the software development process by enabling teams to easily track and manage issues, prioritize tasks, and communicate with team members. - -TestMu AI's integration with FogBugz allows you to avoid the hassle of maintaining issues and tasks manually and instead log bugs in a single click while performing [cross browser testing](https://www.testmuai.com) of your website from the TestMu AI platform to your FogBugz instance. - -## Integrating FogBugz With TestMu AI - -> **Note:** Before enabling integrations, ensure you have administrator or user-level access. - -**Step 1:** Sign in to your TestMu AI account. Don't have an account, register for free. - -**Step 2:** Select **Integrations** from the left sidebar and then FogBugz. - -**Step 3:** It will route you to the installing console window, where you are required to enter your **FogBugz API Token**. - -## Getting The FogBugz User API Token - -**Step 1:** Login to your **FogBugz** account. - -**Step 2:** From the left sidebar menu, hover on the **avatar icon** and click the **User Options** tab. - -**Step 3:** After clicking on **User Options**, scroll down to the section of **API Tokens** and click on **Create API Token**. - -**Step 4:** Enter your **FogBugz Org URL** in the required field. Copy the generated **API Token** and paste it into the **FogBugz API Token** field and click **Install**. - -After installing, **FogBugz will be integrated** with your TestMu AI account. - -## One-Click Bug Logging With Google Chat - -To demonstrate, we take an example of Real-time testing. - -**Step 1:** Select Real-Time Testing from the left menu. - -**Step 2:** Provide your test URL, and select the configurations: Browser, Browser versions, OS, and Resolution. Click on **Start**. - -Upon clicking, a cloud-based virtual machine will fire up, running real browsers and operating systems where you can test websites and web apps for filing bugs. - -**Step 3:** Click on the **Bug** icon from the left in-session toolbar to take a screenshot of any bug you encounter. - -**Step 4:** After marking the bugs, click on the **Bug** icon in front of **FogBugz** under the integrations section. - -**Step 5:** In the FogBugz ticket, fill the bug details like **Project, Priority, Description, Project, Assignee, and Sprint**, and click **Mark As Bug** to log the bugs in FogBugz. - -**Step 6:** To view the logged issues, visit TestMu AI Dashboard and head to **Manage > Issue Tracker** from the left navigation bar. - -**Step 7:** Inside the **Issue Tracker**, you'll notice the logged bugs along with their test environment details. Click on the **upper right arrow** in the square button to view your logged bugs in FogBugz. - -**Step 8:** Upon clicking, you will be redirected to the FogBugz dashboard to find your **logged bugs**. - -## Uninstalling The FogBugz Integration - -**Step 1:** Sign in to your TestMu AI account and visit Integrations from the left sidebar menu. - -**Step 2:** From the My Integrations section, click the remove option under the FogBugz block. - -> That was all you need to know for TestMu AI + Google Chat Integration. Increase your productivity with our integrations. If you still have any questions for us, please feel free to let us know. Our experts are always available on **chat** to help you out with any roadblock regarding our product. Happy testing! diff --git a/static/docs/full-page-screenshots.md b/static/docs/full-page-screenshots.md deleted file mode 100644 index 493d4610e..000000000 --- a/static/docs/full-page-screenshots.md +++ /dev/null @@ -1,26 +0,0 @@ -# Capturing Full-Page Screenshots Of Your Websites - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -[LT Browser](https://www.testmuai.com/lt-browser/) lets you capture full-page screenshots of your pre-installed mobile devices, desktops, and laptops. Now you can capture your website's long content with the full-page screenshot feature and compare it across multiple devices. - -To get started, you need to download the executable file from here. - -Download LT Browser For Windows - -Download LT Browser For macOS - -Download LT Browser For Linux - -**Step 1:** To [capture full page screenshots](https://www.testmuai.com/full-page-screen-capture), navigate to the settings button from the top-left and click on the **'Screenshot Full Page'** button. - -**Step 2:** Click on the **'Screenshot'** icon. - -Upon clicking on the Screenshot icon, you will receive a prompt message stating **'Capturing screenshot in progress!'**. - -After the screenshots are taken, you will again receive a prompt message confirming the same. - -**Step 3:** Then, you will be guided to the page where you can download your full-page screenshots into the LT Browser by pressing the **'Save'** button or you can download your screenshots into your local system by clicking on the **'Arrow'** button. - -> -That's all! You have successfully captured your Full page Screenshots. In case you come across any doubts, feel free to reach out through our **24/7 chat support** or you can also drop a mail to [support@testmuai.com](mailto:support@testmuai.com).Happy testing! diff --git a/static/docs/generate-multiple-lighthouse-reports.md b/static/docs/generate-multiple-lighthouse-reports.md deleted file mode 100644 index 7a80ce7c0..000000000 --- a/static/docs/generate-multiple-lighthouse-reports.md +++ /dev/null @@ -1,35 +0,0 @@ -# Generating Multiple Lighthouse Reports Using Lambda Hooks - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI lets you perform Selenium automation while generating multiple Lighthouse reports through Lambda Hooks. This guide covers the hook structure, expected behavior, and conditions under which the reports are generated or errors are thrown. - -## Hook Structure - -Use the JavaScriptExecutor hook to trigger a Lighthouse report for any URL during your test session. - -To generate a Lighthouse report, use the following *JavaScriptExecutor* hook: - -```java -JavascriptExecutor jse = (JavascriptExecutor)driver; -jse.executeScript("lambdatest_executor: {\"action\": \"generateLighthouseReport\", \"arguments\": {\"url\": \"\"}}"); -``` - -## Generating Lighthouse Report Using Lambda Hooks - -Execute the hook multiple times with different URLs to generate reports for each. - -- You can execute the hook multiple times with a configurable URL. -- If the Lighthouse report execution fails, a *200* status response with the message *Failed to generate Lighthouse report. Please try again.* is sent. The test execution continues. -- For browsers other than Chrome, a 4XX error is thrown stating *Lighthouse report is supported only for Chrome browser.* -- If you generate Lighthouse reports using Lambda hook, turn off the `performance:true` capability. -- Successful generation of the report returns a *200* status and a response body containing: - -``` java -{ -"message": "Lighthouse report generated successfully", -"data": "" -} -``` - -- Lighthouse reports for all hooks executed are visible on the TestMu AI Web Automation Dashboard under the **Performance** tab. diff --git a/static/docs/generate-test-cases-with-ai.md b/static/docs/generate-test-cases-with-ai.md deleted file mode 100644 index 80378352a..000000000 --- a/static/docs/generate-test-cases-with-ai.md +++ /dev/null @@ -1,440 +0,0 @@ -# AI Test Case Generator - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## What is the AI Test Case Generator? -The **AI Test Case Generator** is an intelligent feature within TestMu AI Test Manager and KaneAI that allows users to convert various requirement formats (like text, PDFs, audio, videos, images, Jira tickets and more) into structured, contextual software test cases. It significantly accelerates the test case creation process while improving coverage and quality. - -This feature is designed to save time, improve test coverage, and streamline the test design process for both manual and automated testing workflows. - -## Highlights - -### Conversation Layer -Refine, correct, expand, and restructure your AI-generated test scenarios and test cases using natural language, no manual editing required. The Conversation Layer brings a **chat-driven workflow** directly into your generation session, letting you iterate in real time without starting over. - -- **Natural language control**: Describe changes in plain English, and the AI applies them instantly. -- **Live streaming updates**: Watch scenarios and test cases update in real time as the AI processes your request. -- **Precise targeting**: Use `@` references to target specific scenarios or test cases for modification. -- **Context-aware**: The AI retains your full session context including uploaded files, linked issues, and prior instructions. -- **File-based refinement**: Upload updated specs or requirements mid-session and ask the AI to fill coverage gaps. - -### Custom Instructions - -Custom instructions allow you to define persistent guidelines that the AI follows during test case generation. These instructions ensure that generated test cases align with your team's testing standards, naming conventions, and quality requirements. - -When you provide custom instructions, they are included as additional context alongside your prompt and requirements. The AI uses these instructions to shape the structure, language, coverage, and style of the generated test cases. - -Custom instructions are supported at two levels: - -| Level | Scope | Use Case | -|-------|-------|----------| -| **Project-Level** | Applies to all generation sessions within a specific project | Project-specific conventions, domain terminology, or testing focus areas | -| **Organization-Level** | Applies across all projects in your organization | Company-wide testing standards, compliance requirements, or formatting guidelines | - -When both levels are configured, the AI combines organization-level and project-level instructions, with project-level instructions taking precedence in case of conflicts. - -### Project-Level Custom Instructions - -Project-level instructions are specific to a single project and are applied whenever test cases are generated within that project. - -**To configure project-level custom instructions:** - -1. Navigate to your project in **Test Manager**. -2. Open the project settings. -3. Locate the **Project Instructions** section. -4. Enter your instructions in the text field. -5. Save your changes. - -**Example project-level instructions:** -``` -- All test cases must include preconditions for user authentication state. -- Use the naming convention: [Module]_[Feature]_[Scenario]_[Positive/Negative]. -- Include API response codes in expected results for all API-related test cases. -``` - -### Organization-Level Custom Instructions - -Organization-level instructions apply globally across all projects within your organization. They are ideal for enforcing company-wide standards. - -**To configure organization-level custom instructions:** - -1. Navigate to the **Settings** in Test Manager sidebar. -2. Locate the **Org Instructions** configuration section. -3. Enter your organization-wide instructions. -4. Save your changes. - -**Example organization-level instructions:** -``` -- Include accessibility testing considerations where applicable. -- Severity and priority must be assigned to every test case. -- Test data should use anonymized values, never production data. -``` - -### Smart Context: Memory Layer - -The **Memory Layer** is an intelligent context retrieval system that enhances AI test case generation by leveraging your existing test case repository. It ensures that newly generated test cases are aware of your current test coverage, avoid duplicates, and build upon your existing work. - -### How It Works - -When you initiate a test case generation session, the following process occurs automatically: - -1. **Input Analysis**: The AI analyzes your prompt, requirements, and custom instructions. -2. **Context Retrieval**: The system queries a vector database to retrieve the most relevant existing test cases from your test case repository. -3. **Context-Aware Generation**: The retrieved content is embedded into the prompt sent to the AI, enabling it to generate test cases that are aware of your existing coverage, non-duplicative, and complementary to your current test suite. - -### Benefits - -- **Duplicate Prevention**: The AI recognizes test cases that already exist in your repository and avoids regenerating them. -- **Contextual Relevance**: New test cases are informed by the language, structure, and patterns used in your existing test suite. -- **Incremental Improvement**: Each generation session builds upon your accumulated testing knowledge. - -The Memory Layer operates automatically in the background, you can choose to enable or disable Memory enhancement feature before triggering a session. - -## Generate Test Cases: Step-by-Step - -### Step 1: Open the AI Test Case Generator -You can access the AI Test Case Generator from two entry points: - -**From Test Manager:** -Navigate to the **Test Case Listing** page within your project and click on `Generate With AI`. An input box will appear where you can provide your requirements. - -**From KaneAI:** -Navigate to the **Agent** page in KaneAI and select `Generate Scenarios`. This opens the same AI Test Case Generator interface. - -### Step 2: Enter Your Requirements -Start by entering your product or feature requirements in the input box. - -We support multiple input formats, including: -- **Textual requirements** -- **Jira/Azure DevOps links** (e.g., epics, stories, tasks) -- **PDFs** -- **Images** -- **Audio** (recordings or uploads) -- **Videos** -- **Spreadsheets** (CSV or XLSX) -- **Documents** -- **JSON or XML** - -**Public URLs Only** -Currently, the AI Test Case Generator can only access **publicly available URLs**. Websites behind a VPN, corporate proxy, firewall, or those requiring authentication to reach the landing page cannot be analyzed. - -**Support for private URLs is coming soon.** - -#### Add Input Requirements - -##### Text Input - -- Type your requirement directly into the input box. - -##### Record Audio -- Click the mic icon to begin recording, as shown in the picture. - - Audio recording is not supported in Firefox, ARC and Opera browsers. - -- Click the tick icon to confirm, or the cross to discard. - -##### Upload Files -- Click on the **pin** icon 📎, then select `Upload from device`. -- You can select a maximum of 10 files from your device. - -- Uploaded files will appear within the input box. - -##### Add Issue Links -- Integrate your TestMu AI account with Jira/Azure DevOps. Follow the steps described in the following link to integrate Jira/Azure DevOps: -[TestMu AI Jira Integration](/support/docs/jira-integration/#how-to-establish-integration-with-jira-from-your-lambdatest-account) / [TestMu AI Azure DevOps Integration](/support/docs/link-ado-issues-with-test-manager/) - -- After integration is completed, select `Link Jira Issues` / `Link Azure DevOps Issues`. - -- Enter your Jira/Azure DevOps issue ID or link and press Enter. - - Only issues from the connected Jira/Azure DevOps project can be linked here. - -- The linked issue will then appear inside the input box. - -For linked Jira issues, attachments such as images and other files are also fetched and used as context for test case generation, in addition to the textual content. The same limits that apply to direct file uploads apply here: a maximum of 10 files with a file size limit of 50 MB per file. Azure DevOps integration fetches textual content only. - -##### Configure Generation Settings -Before triggering a generation session, you can configure the following settings to control the AI's output: - -| Setting | Description | -|---------|-------------| -| **Max Test Scenarios** | Set the maximum number of test scenarios the AI will generate in the session | -| **Max Test Cases per Scenario** | Set the upper limit for test cases within each individual scenario | -| **Memory Enhancement** | Toggle to enable or disable the [Smart Context: Memory Layer](#smart-context-memory-layer), which retrieves relevant existing test cases from your repository to improve generation quality | -| **Project Instructions** | View and verify the [project-level custom instructions](#project-level-custom-instructions) that will be applied to the session | - -Configure these settings before pressing Enter to ensure the AI generates test cases aligned with your session requirements. Organization-level instructions, if configured, are automatically applied in addition to project instructions. - -### Step 3: Generate Test Cases -- Once all your input requirements are added (along with any custom instructions), press **Enter** to start generating test cases. -- The AI analyzes your input, retrieves relevant context from your existing test case repository via the [Smart Context: Memory Layer](#smart-context-memory-layer), and generates test cases that are both comprehensive and non-duplicative. - -To end the test case generation while the Agent is thinking, click on the **Stop Generating** button. - -### Step 4: Review Test Cases Across Scenarios -- Test cases are grouped into high-level, logical test scenarios. Each scenario represents a theme or functional area for easier navigation and categorization. - -- Scenarios are **labeled** with tags such as `Must have`, `Should have`, and `Could have`, indicating their relative importance as determined by the Agent. - -- Individual test cases are further categorized with tags: - - **Positive**: Valid test cases expected to pass. - - **Negative**: Invalid or failure cases designed to test robustness. - - **Edge**: Corner cases that may be overlooked in testing flows. - -### Step 5: View and Edit Test Case Details -Click on any test case to explore its full details, including: -- Test Case Title -- Description -- Pre-conditions -- Priority -- Test Steps and Expected Outcomes - -You can **edit** any part of the test case if you want to add more context or align it with your test strategy. - -## Conversation Layer: Refine Your Test Cases - -After generating your initial test scenarios and cases, you can use the **Conversation Layer** to iteratively refine them through natural language. Instead of manually editing each test case or regenerating everything from scratch, simply describe what you want changed in the chat interface, and the AI applies the modifications in real time. - -#### Conversational Workspace - -When the Conversation Layer is active, the workspace is divided into two primary panels: - -**Chat Interface (Left Panel)** - -The left panel is your chat interface for refining test output: -- **Message input**: Type your refinement instructions in natural language at the bottom of the panel. -- **Conversation history**: All messages and AI responses are preserved within the session. -- **@ Autocomplete**: Type `@` to trigger an autocomplete dropdown for referencing specific scenarios or test cases. -- **File attachments**: Attach files directly within the conversation using the attachment button. -- **Confirmation prompts**: The AI asks for confirmation when a change affects more than 5 test cases. - -**Live Output View (Right Panel)** - -The right panel is the primary workspace showing your generated test scenarios and test cases: -- **Real-time updates**: Affected scenarios and test cases update with a streaming animation as changes are processed. -- **Scenario cards**: Each scenario is displayed as an expandable card containing its test cases. -- **Visual indicators**: Scenarios and test cases being modified are visually highlighted during streaming. -- **Full test case details**: Expand any test case to view its steps, expected results, preconditions, and other fields. - -**Session Context Drawer** - -To review the context used for your current generation session, click the **pencil icon** located just above the chat input box at the bottom left. This opens a side drawer displaying: -- **Attached Files**: All files uploaded during the session. -- **Organization Instructions**: Organization-level custom instructions applied to the session. -- **Project Instructions**: Project-level custom instructions applied to the session. -- **Linked Issues**: Jira or Azure DevOps issues linked to the session. - -Use the context drawer to verify that the AI is working with the correct inputs. You can review linked issues, attached files, and the custom instructions being applied before sending refinement messages. - -#### Conversational Refinement - -**How It Works:** - -1. **You send a message** in the chat interface describing what you want to change. -2. **The AI interprets your intent** and determines which scenarios or test cases to modify. -3. **Changes stream in real time** to the live output view on the right. -4. **You review and iterate**: send follow-up messages to fine-tune further. - -#### @ Referencing System - -The `@` referencing system allows you to precisely target specific scenarios or test cases in your messages. - -| Reference | Example | -|-----------|---------| -| Scenario | `@S1`, `@S3` | -| Test Case | `@S1.C1`, `@S2.C4` | - -**Using Autocomplete:** - -1. Type `@` in the conversation input. -2. An autocomplete dropdown appears showing all available scenarios and test cases. -3. Start typing to filter the list (e.g., `@S2` filters to scenarios, `@S2.*` filters to test cases in Scenario 2). -4. Click or press **Enter** to select the reference. - -**What You Can Do with Conversational Refinement:** - -The Conversation Layer transforms how you refine test scenarios. Instead of manually editing each test case, describe what you need in natural language, and watch your test suite reshape itself in real time. - -#### Reorganize Test Scenarios - -Restructure how your test cases are grouped and organized. - -| Action | Example Prompt | -|--------|----------------| -| Group by feature or module | Reorganize all test cases by feature, create separate scenarios for User Authentication, Shopping Cart, Checkout, and Order History | -| Split by test type | Split @S1 into three separate scenarios: one for positive flows, one for negative/error cases, and one for boundary conditions | -| Group by platform or environment | These test cases are mixed. Group them by platform, create one scenario for Web Desktop, one for Mobile Web, and one for Mobile App | -| Create user journey scenarios | Create end-to-end user journey scenarios. Combine login, product search, add to cart, and checkout into a complete "Customer Purchase Journey" scenario | - -#### Expand Test Coverage - -Add new test cases or scenarios to improve coverage. - -| Action | Example Prompt | -|--------|----------------| -| Add missing test types | Add negative test cases to @S1 covering invalid email formats, passwords shorter than 8 characters, usernames with special characters, and empty field submissions | -| Add security tests | We need security coverage for @S3. Add test cases for SQL injection, XSS attacks, CSRF validation, and session hijacking across all input fields | -| Add platform-specific tests | Add mobile-specific test cases to @S2, include touch gestures, pinch to zoom, orientation changes, and offline mode scenarios | -| Fill coverage gaps from uploaded files | I've attached the updated requirements document. Review it against our current test cases and create new ones for any features or user stories we haven't covered yet | -| Add new scenarios | Create a new scenario for the password reset flow with test cases covering successful reset, expired reset links, invalid tokens, rate limiting, and already-used tokens | - -#### Remove Unnecessary Tests - -Clean up your test suite by removing low-value or out-of-scope test cases. - -| Action | Example Prompt | -|--------|----------------| -| Remove by priority | We're focusing on critical paths for this release. Delete all Low priority test cases from @S2 and keep only High and Medium priority ones | -| Remove by test type | Remove all test cases that only verify UI cosmetics like text labels, button colors, and spacing. Keep the functional validation and business logic tests | -| Keep specific functionality only | We're deprecating the wishlist feature. Keep only the shopping cart test cases and remove everything related to wishlist functionality | -| Remove by user role | Our next sprint focuses only on logged-in users. Remove all guest user and anonymous browsing test cases, keep authenticated user workflows | - -#### Combine and Merge Tests - -Reduce redundancy by merging overlapping test cases. - -| Action | Example Prompt | -|--------|----------------| -| Merge duplicate tests | I notice @S1.C3 and @S1.C5 both test login validation with different edge cases. Merge them into a single comprehensive test case that includes all the validation scenarios from both | -| Create end-to-end flows | Combine the separate test cases for user registration, email verification, and first-time login into one end-to-end "New User Onboarding Journey" test | -| Consolidate compatibility tests | We have separate test cases for Chrome, Firefox, Safari, and Edge. Merge these into a single cross-browser compatibility test with browser-specific validation steps | - -#### Standardize Naming and Formatting - -Apply consistent naming conventions and formatting across test cases. - -| Action | Example Prompt | -|--------|----------------| -| Enforce naming conventions | Rename all test cases to follow this format: [Feature] - [Scenario Type] - Description. For example: [Login] - Positive - Successful authentication with valid credentials | -| Align terminology | Our team uses "save" instead of "submit". Replace every instance of "submit" with "save" across all test case titles, steps, and expected results in @S2 | -| Standardize priorities | Update priority to High for all test cases that involve payment processing, checkout, or financial transactions | -| Add tags | Tag all critical user journey test cases with "smoke-suite" so we can easily filter them for our smoke testing runs | - -#### Enhance Test Case Details - -Add preconditions, cleanup steps, or additional context to test cases. - -| Action | Example Prompt | -|--------|----------------| -| Add preconditions | Add a precondition to all test cases in @S1 and @S3 stating: "User must be logged in with a verified email account and have at least one saved payment method" | -| Add cleanup steps | All test cases that create new users or test data need cleanup. Add a final step to delete the test account and associated data to prevent database pollution | -| Add specific steps to all test cases | Add these two steps to all test cases in @S2: Step 1 - Clear browser cache and cookies. Step 2 - Verify the homepage loads successfully before proceeding with the test | -| Add steps to specific test cases | In @S1.C3 and @S1.C5, add a step after login to verify the user's dashboard displays their last login timestamp and session information | -| Add steps at specific position | In @S2.C2, add a step between step 3 and step 4 to take a screenshot of the shopping cart and verify the item quantity matches what was added | - -#### Make Targeted Modifications - -Precisely modify specific scenarios or test cases using `@` references. - -| Action | Example Prompt | -|--------|----------------| -| Modify scenario scope | Update @S3 to focus only on admin dashboard functionality and role-based access control. Remove all end-user and guest user test cases from this scenario | -| Update specific test case | In @S2.C4, the expected result needs updating. Change it to "Account is locked for 30 minutes after 3 consecutive failed login attempts" instead of just showing an "Invalid credentials" error | -| Expand test case validation | Expand @S1.C2 to include password strength validation: minimum 8 characters, at least one uppercase letter, one number, one special character, and cannot contain the username | - -**Pro Tips for Effective Refinement** -- **Use `@` references for precision**: Target specific scenarios (`@S1`) or test cases (`@S2.C3`) to ensure the AI modifies exactly what you intend -- **Combine multiple actions**: Request several changes in one message: "Remove all Low priority test cases from @S1, add 2 boundary tests for the email field, and rename remaining test cases to include the module name" -- **Provide updated context mid-session**: Upload revised specifications or link Jira/Azure DevOps tickets at any point to help the AI identify coverage gaps and generate test cases aligned with the latest requirements -- **Start broad, then refine**: First organize scenarios at a high level, then drill down to individual test cases - -#### Re-Generation Settings - -The Test Case Generator provides several controls that work alongside the Conversation Layer: - -- **Max Scenarios**: Control the maximum number of scenarios generated in a session. -- **Max Test Cases per Scenario**: Set the upper limit for test cases within each individual scenario. -- **Regenerate Scenarios**: Regenerate all scenarios utilizing existing context and to regenerate fresh Scenarions and Test Cases. - -Regeneration replaces all current scenarios. To keep specific scenarios intact and only modify others, use targeted conversational refinements instead. - -#### Credit Usage - -The Conversation Layer consumes AI credits based on the scope of each refinement. - -| Action | Credit Cost | -|--------|-------------| -| Edit or modify a **scenario** (including its test cases) | **5 credits** | -| Edit or modify a single **test case** | **1 credit** | - -- Credits are deducted **per message** based on the scope of the change. -- If your message affects multiple scenarios, credits are deducted for each scenario impacted. -- Your remaining credit balance is visible in your account dashboard. - -#### Current Limitations - -The following capabilities are planned for future releases: - -- **Undo and versioning**: No rollback mechanism for individual conversation-driven changes. -- **Step-level referencing**: You cannot reference individual steps within a test case. Instead, reference the test case and describe the change. -- **Coverage queries**: Asking analytical questions (e.g., "Do we have tests for invalid amounts?") without requesting a change is not supported. - -## Save or Automate Test Cases - -Once you have generated and refined your test scenarios and test cases, it's time to save your work. Select the test cases you want to keep and choose one of the following options: - -#### Option 1: Create Test Cases -Select the desired test cases and click **Create** to save them directly to your **Test Case Repository** in TestMu AI Test Manager. These saved test cases can then be assigned to test runs, shared with teams, or used in manual test planning. Any files attached during the generation session will also be saved as attachments on the created test cases. - -#### Option 2: Create and Automate with KaneAI -Select the desired test cases and click **Create and Automate** to save the test cases to your repository and submit them to KaneAI for automated authoring. Clicking this button opens the **Create and Automate Test Cases** dialog, where you configure automation settings before KaneAI begins authoring. The dialog displays the number of distinct test cases selected and how many are automation-ready. - -The dialog is organized into three platform tabs — **Desktop Browser**, **Mobile Browser**, and **Mobile App**. Each tab contains three sections: - -**Test Settings** - -Configure how KaneAI authors the test cases. You can set the **Agent Concurrency** (number of concurrent KaneAI agents, from 1 to 105), toggle **Generate data dynamically** to enable runtime data generation for adaptive tests, and toggle **Dismiss pop-ups** to automatically handle pop-ups and notifications during execution. - -**Test Configurations** - -Select the target environment for your tests. Mobile Browser and Mobile App tabs allow you to choose specific device, OS, and version combinations for test execution. - -**Advanced Settings** - -Configure environment-level settings for your test run. All platforms support **Network Configuration** to connect via a tunnel or proxy, and **Timezone** to set the execution timezone. Each platform also includes additional settings specific to that environment. For a full reference of available options per platform, see [Advanced Settings in KaneAI](/support/docs/kaneai-advanced-settings/). - -The **Create and Automate** option requires KaneAI access. If you do not have KaneAI enabled on your account, use the **Create** option to save test cases manually. - -**Experimental Feature** -Create and Automate is currently an experimental feature. Review your test steps once the test cases have been created and automated. - -## Troubleshooting - -| Issue | Resolution | -|-------|------------| -| Conversation Layer panel not visible | Ensure you have generated at least one set of scenarios. The Conversation Layer activates after the initial generation completes. Refresh the page if needed. | -| Changes not appearing in Live Output View | Check your internet connection. Large changes may take longer to process. If the issue persists, send a new message. The AI resumes from the current state. | -| AI modified the wrong scenario or test case | Use explicit Scenario (`@S1`) or Test Case (`@S1.C1`) references to avoid ambiguity. Rephrase your message with a direct reference and send it again. | -| File upload fails | Verify the file does not exceed the 50 MB size limit and that you have not exceeded the 10-file session limit. | -| Credits deducted but no changes appeared | This can occur if the AI could not interpret your request or if there was a connection interruption. Check the conversation history for an error message. | -| Regeneration removed refined scenarios | Regeneration replaces all current output. Use targeted conversational messages instead to preserve specific scenarios. | - -## Video Walkthrough - -Watch the video below for a quick walkthrough of the AI Test Case Generator in action. - -## Frequently Asked Questions - -**Can I undo a change made through the Conversation Layer?** - -Undo and versioning are not available in the current release. If a change is not what you expected, send a follow-up message to revert or adjust the change. - -**Does the Conversation Layer work with manually created test cases?** - -The Conversation Layer operates within AI Test Case Generator sessions only. It refines scenarios and test cases generated by the AI within that session. Manually created test cases in Test Manager are not accessible through the Conversation Layer. - -**How many messages can I send in a single session?** - -There is no fixed limit on the number of messages per session. Each message consumes credits based on the scope of the change. - -**What happens if I close the browser mid-session?** - -Your session context, including generated scenarios and conversation history, is preserved. You can return to the session and continue refining from where you left off. - -**Can I export the refined test cases?** - -Yes. After refining your test cases through the Conversation Layer, you can export or save them to your Test Manager project using the same export workflows available in the Test Case Generator. - -**Is the Conversation Layer available for all plans?** - -The Conversation Layer is available to all users with access to Test Manager Premium or KaneAI plans. Credit consumption applies based on your plan's AI credit allocation. diff --git a/static/docs/genrocket-integration.md b/static/docs/genrocket-integration.md deleted file mode 100644 index a1c50cb72..000000000 --- a/static/docs/genrocket-integration.md +++ /dev/null @@ -1,106 +0,0 @@ -# GenRocket Integrates With TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -With TestMu AI's and GenRocket's integration, users utilizing TestMu AI HyperExecute will gain effortless access to GenRocket's capabilities for generating the required synthetic test data in both quantity and diversity. This integration empowers users to define authentic test data using GenRocket and then seamlessly automate the testing process with the aid of HyperExecute. - -This collaboration delivers substantial advantages to customers using both platforms, resulting in quicker and more comprehensive testing procedures. As a direct consequence, testing cycles will be notably shortened, contributing to an expedited pace of software delivery. - -## Prerequisites - -1. Ensure you have a TestMu AI account. If you do not have an account, register for free. - -2. Ensure you have a GenRocket account. - -## Creating User In GenRocket - -GenRocket is sold to enterprise (of all sizes) customers. It is not available to "unaffiliated" individuals and/or general public. Access to GenRocket is provided during a qualified Proof of Concept (POC) phase. - -## Setting Up A Project for Data Generation - -As described above, users need to setup the test data project ("Model" and "Design") for the test data needed. - -For the purpose of this integration, we have used a [sample project with Java + TestNG](https://github.com/prateekLambda/GenRocket-HyperExecute-Sample). In the sample project test case we are opening the browser and calling the GenRocket API to get the username and web application URL (the generated data) and opening those URLs in the browser as a sample test case. - -In order to generate data using GenRocket, you can follow the steps given below: - -**Step 1:** Users need to create the HyperExecute Machine path profile in the Genrocket Dashboard. - ->**Note:** Users can Generate a Scenario through GenRocket Dashboard by [following this documentation](https://genrocket.freshdesk.com/support/solutions/folders/19000164896). - -**Step 2:** Although "Scenarios" can be used to design the volume and variety of data, in practice, most customers who deal with complex data use G-Cases. Learn about G-Cases [here](https://genrocket.freshdesk.com/support/solutions/folders/19000164899). - -**Step 3:** Genrocket User profile and scenario file should be downloaded and placed in the project. - -**Step 4:** All the GenRocket dependency jars are added in the pom.xml of the project and they get automatically downloaded when the project is run. Follow the [steps in the link](https://genrocket.freshdesk.com/support/solutions/articles/19000114946-how-to-integrate-genrocket-with-a-java-maven-project) to add the dependency. - -**Step 5:** Users can call GenRocket methods to access the Genrocket API to generate data while running test suite in Hyper Execute - -The following code snippet shows how to use the GenRocket API to generate the data: - -## Setup On HyperExecute - -Till now you have learnt how to set up your generators on GenRocket. Now, in order to run those tests on Hyperexecute, please follow the steps given below: - -**Step 1:** You can login or create your free TestMu AI account here. - -**Step 2:** Once logged in, click on **HyperExecute** on the Left Menu Bar. - -**Step 3:** New users can avail a free trial of HyperExecute on this page by clicking **Try Now**. - -**Step 4:** Once done, you are all set to run your sample project on HyperExecute. - -## Setting Up A Sample Project - -**Step 1:** Before proceeding further you need to access your **TestMu AI Username** and **Access Key**, which you can find in the Profile section. **Profile section** can be accessed from the top right side of the page as shown below: - -**Step 2:** For the purpose of this integration, we have used the same [sample project](https://github.com/prateekLambda/GenRocket-HyperExecute-Sample) which you would have downloaded already as part of GenRocket Setup. - -**Step 3:** Next, you need to download the **HyperExecute CLI** binary based on your OS: - -* **For Windows:** Download for Windows [here](https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe). - -* **For Linux:** Download for Linux [here](https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute). - -* **For Darwin:** Download for Darwin [here](https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute). - -**Step 4:** Once downloaded, make sure the Hyperexecute binary is at the root directory of your project. - -## Running Your First Test With Data From GenRocket on HyperExecute - -**Step 1:** In order to run your tests in HyperExecute, you need a basic settings file, known as **HyperExecute YAML** configuration file. This file contains a list of settings and commands that one can execute on HyperExecute. [Learn more](/support/docs/deep-dive-into-hyperexecute-yaml/). - ->**Note:** The HyperExecute YAML configuration file is already present in the sample project that you have downloaded. - -Following two commands have been already added in the "[pre](/support/docs/deep-dive-into-hyperexecute-yaml/#14-predirectives-or-pre)" section of this HyperExecute Yaml. - -* In Hyperexecute, every time the tests are run, a new environment is created, and gets destroyed as soon as the tests are completed. - -* To configure Genrocket on HyperExecute we need to create a directory in the HyperExecute Machine with the name **.genrocket** which is done by the following command. - -```bash -mkdir /home/ltuser/.genrocket -``` - -* After creating the directory, we also need to copy the Genrocket profile **.grp** file to the .genrocket folder which is done by the following command. - -```bash -cp ******Profile.grp /home/ltuser/.genrocket -``` - -* Once done, the [pre section](/support/docs/deep-dive-into-hyperexecute-yaml/#14-predirectives-or-pre) inside the HyperExecute YAML will look like as shown below (already present inside the sample project). - ->**Note:** You will need to create a HyperExecute YAML configuration file specific to your project if you are not using the given sample project. In that case, the pre-section would remain the same however, other commands will need relevant changes. [Learn More](/support/docs/deep-dive-into-hyperexecute-yaml/). - -**Step 2:** Once you have the HyperExecute YAML ready, you just need to execute the HyperExecute CLI command. This command would trigger the CLI which will initiate a Job on HyperExecute Platform to run your test cases. - -* Open the terminal and navigate to the project root directory. -* Run the following command with your **TestMu AI Username** and **AccessKey**. (as noted in step 4.1 above) - -```bash -./hyperexecute --user USERNAME --key ACCESS_KEY --config hyperexecute.yaml -``` - -**Step 3:** You have successfully setup genrocket and executed your first job on HyperExecute. Once a Job is executed, you should be able to see the test results & logs on the HyperExecute Portal as shown below: - -To learn more about HyperExecute visit our [documentation](/support/docs/getting-started-with-hyperexecute/) or contact our support team from the in app support section. diff --git a/static/docs/gesture-support-ios.md b/static/docs/gesture-support-ios.md deleted file mode 100644 index 3005165ec..000000000 --- a/static/docs/gesture-support-ios.md +++ /dev/null @@ -1,39 +0,0 @@ -# iOS Gestures on Real Devices - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -When testing iOS apps on **TestMu AI Real Devices**, certain native gestures like double‑tap, triple‑tap, or multi‑finger interactions may be required to complete test flows. - -To make this easier, TestMu AI provides keyboard shortcuts that simulate these **gestures** without needing physical multi‑touch input. These shortcuts work on both **macOS** and **Windows**, allowing you to trigger gestures instantly during manual sessions. - -# Supported Gestures & Shortcuts - - iOS Gesture - Shortcut - - Two-Finger Triple-Tap - ⌘ + ⇧ - - Single-Finger Double-Tap - ⌘ - - Single-Finger Triple-Tap - ⌥ - - iOS Gesture - Shortcut - - Two-Finger Triple-Tap - Ctrl + Shift - - Single-Finger Double-Tap - Ctrl - - Single-Finger Triple-Tap - Alt - -## Additional Links - -- [Manual Testing on Real Devices](/support/docs/app-testing-on-real-devices/) -- [Apple Pay Manual Testing Guide](/support/docs/apple-pay-manual/) -- [Assistive Touch](/support/docs/assistive-touch-on-real-ios-devices/) diff --git a/static/docs/getting-started-with-agent-testing-platform.md b/static/docs/getting-started-with-agent-testing-platform.md deleted file mode 100644 index a0e059cac..000000000 --- a/static/docs/getting-started-with-agent-testing-platform.md +++ /dev/null @@ -1,70 +0,0 @@ -# Getting Started With the Agent Testing Platform - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI [Agent Testing](https://www.testmuai.com/agent-testing/) Platform validates AI agents the way real users interact with them. It deploys 15+ autonomous AI testing agents to evaluate any chat, voice, phone, or image agent against standardized quality metrics, then returns a Green, Yellow, or Red production readiness verdict before and after you deploy. - -It is the first platform to test chat, voice, phone, and image agents in one place. Connecting an agent takes three inputs and no SDK: upload the documents that describe it, define its intended behavior, and point the platform at its endpoint or phone number. - -## Why AI Agents Need a Dedicated Testing Platform - -Traditional QA tests deterministic software: a selector either matches or it does not. AI agents return a different answer every run, so the old assertions no longer apply. Five gaps appear when teams ship agents without agent-specific testing. - -- **Manual testing does not scale.** A QA engineer reviews 50 to 200 scenarios a week. Agents need thousands to cover the input space, so coverage stays a fraction of what production sees. - -- **Non-determinism breaks scripted checks.** The same prompt produces different reasoning each run. There is no fixed string to assert against, so pass/fail scripts miss real failures. - -- **Failure is asymmetric.** A broken button is a bug. A hallucinating healthcare bot or a biased financial agent is a liability and a compliance exposure. - -- **Feedback loops are slow.** Days-long manual review blocks iteration and delays every release that changes the model or the prompt. - -- **Evaluation is subjective.** Reviewers disagree on whether a response is complete or biased. Scores drift between people and cannot be audited. - -## Start by Understanding the Platform - -Learn what the platform tests and how it scores an agent before you run anything. - - Architecture & How Evaluation Works - The five-phase evaluation loop and the 15+ testing agents that score your agent. - -## Run Your First Test - -Pick the interface that matches how you work and run an evaluation end to end. - - Quickstart (UI) - Create an agent, generate scenarios, and run an evaluation from the dashboard. - - Quickstart (CLI) - Test chat and phone agents from your terminal and wire them into CI/CD. - - Quickstart (API) - Connect a chat agent over its API: request format, authentication, and response schema. - - Quality Dimensions - How scoring, thresholds, and the Green, Yellow, or Red go-live verdict work. - -## Test by Agent Type - -Go deeper into the workflow and metrics for your specific agent type. - - Chat Agent Testing - How multi-turn chat conversations are simulated and scored. - - Phone Agent Testing - Real calls, voice and noise simulation, and the two phone directions. - - Inbound Phone Agent Testing - The full workflow for an agent that answers calls. - - Outbound Phone Agent Testing - The workflow for an agent that places calls. - - Image Analyzer Testing - Score generated images against a prompt, brand rules, and technical specs. - -## Get Help - -Find answers to common questions about the platform. - - Agent Testing FAQs - Answers on multilingual support, model governance, and data connectors. diff --git a/static/docs/getting-started-with-appium-testing.md b/static/docs/getting-started-with-appium-testing.md deleted file mode 100644 index 9f0c85709..000000000 --- a/static/docs/getting-started-with-appium-testing.md +++ /dev/null @@ -1,53 +0,0 @@ -# Getting Started with Appium Testing - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Appium is an open-source framework that allows you to automate tests for mobile applications. It works across different platforms like Android and iOS, enabling you to write tests once and run them on various devices. - -Appium leverages the WebDriver protocol, similar to how Selenium automates web applications. This makes it compatible with [popular programming languages](/support/docs/appium-languages-and-frameworks/) for writing test scripts. - -Here is a list of languages and frameworks that are supported by the TestMu AI to run Appium automation tests on [TestMu AI Real Device Cloud Platform](https://www.testmuai.com/real-device-cloud). - - Java - - JUnit - - JBehave - - Cucumber - - TestNG - - JavaScript - - WebDriverIO - - Mocha - - Python - - Behave - - Robot - - Gauge - - PyTest - - Ruby - - Cucumber - - RSpec - - PHP - - Behat - - C# - - NUnit - - We support all languages and frameworks that are compatible with Appium, so in case your favorite isn't in the table.Don't worry, you can still run the test. Contact Us for any help. - -We support all languages and frameworks that are compatible with Selenium, so in case your favorite isn't in the table.Don't worry, you can still run the test. Contact Us for any help. diff --git a/static/docs/getting-started-with-cypress-testing.md b/static/docs/getting-started-with-cypress-testing.md deleted file mode 100644 index 4dc31e68a..000000000 --- a/static/docs/getting-started-with-cypress-testing.md +++ /dev/null @@ -1,246 +0,0 @@ -# Getting Started With Cypress Testing - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Cypress is a modern web front-end testing tool built with JavaScript Mocha. It operates directly on the browsers without the need for Selenium. Its unique DOM manipulation technique makes it a very developer and QA-friendly tool. - -Integrating TestMu AI with Cypress allows you to perform Cypress testing across 40+ browser versions on cloud. - -In this guide, learn how to get started with Cypress testing on the TestMu AI platform. We will use Cypress kitchen sink sample app to execute our tests. - -## Prerequisites - -**Sample repo** - -Before we get started, make sure to clone the TestMu AI's sample Cypress Cloud repo, used in this document. View on GitHub - -You can run your first Cypress test on the TestMu AI platform in a few simple steps: - -1. Clone the TestMu AI-Cypress-Cloud GitHub repo and navigate to the cloned directory. - -```bash -git clone https://github.com/LambdaTest/Cypress-Cloud.git -cd Cypress-Cloud -``` - -2. To run Cypress tests, you will need to set your TestMu AI username and access key in the environment variables. You can get them from the TestMu AI Automation Dashboard. - -**Windows** - -```js -set LT_USERNAME="YOUR_LAMBDATEST_USERNAME" -set LT_ACCESS_KEY="YOUR_LAMBDATEST_ACCESS_KEY" -``` - -**macOS/Linux** - -```js -export LT_USERNAME="YOUR_LAMBDATEST_USERNAME" -export LT_ACCESS_KEY="YOUR_LAMBDATEST_ACCESS_KEY" -``` -3. Install Node.js version 12 or higher. You can download it from the [official Node.js website](https://nodejs.org/en/download/). - -## Running Your First Test - -Following are the steps to run your first Cypress test on the TestMu AI platform. - -1. Install the TestMu AI-Cypress CLI using the below command. - -```bash -npm install -g lambdatest-cypress-cli -``` - -2. Clone the Cypress kitchen sink repo using the following command. - -```bash -# Clone the kitchen sink repo -git clone https://github.com/cypress-io/cypress-example-kitchensink.git - -# Go to the cloned directory -cd cypress-example-kitchensink -``` - -```bash -# Clone the kitchen sink repo -git clone https://github.com/cypress-io/cypress-example-kitchensink.git - -# Go to the cloned directory -cd cypress-example-kitchensink - -# Checkout to this commit -git checkout ab10094ef7b199ae7febafec413a0626414bcd3c -``` - -Once you clone the kitchen sink repo, below will be the structure of your Cypress project. - -```bash -... -cypress -|-- fixtures -|-- e2e -|-- support -cypress.config.js -... -``` - -```bash -... -cypress -|-- fixtures -|-- integration -|-- plugins -|-- support -cypress.json -... -``` - -3. Install the npm dependencies by passing the below command. - -```bash -npm install -``` - -4. Create `lambdatest-config.json` file that contains configurations like auth, capabilities, test settings, etc. which need to be successfully executed at LambaTest. - -Use `init` command to generate the sample configuration files. - -```bash -lambdatest-cypress init --cv=10 -``` - -```bash -lambdatest-cypress init -``` - -Once you run the above command, below is the project structure for the `lambdatest-config.json` file. - -```js -{ -"lambdatest_auth": { -"username": "", -"access_key": "" -}, -"browsers": [ -{ -"browser": "Chrome", -"platform": "Windows 10", -"versions": [ -"latest-1" -] -}, -{ -"browser": "Firefox", -"platform": "Windows 10", -"versions": [ -"latest-1" -] -} -], -"run_settings": { -"cypress_config_file": "cypress.config.js", -"reporter_config_file": "base_reporter_config.json", -"build_name": "build-name", -"parallels": 1, -"specs": "./*.cy.js", -"ignore_files": "", -"network": false, -"headless": false, -"npm_dependencies": { -"cypress": "10.0.0" -} -}, -"tunnel_settings": { -"tunnel": false, -"tunnel_name": null -} -} -``` - -```bash -app -{ -"lambdatest_auth": { -"username": "", -"access_key": "" -}, -"browsers": [ -{ -"browser": "Chrome", -"platform": "Windows 10", -"versions": [ -"latest-1" -] -}, -{ -"browser": "Firefox", -"platform": "Windows 10", -"versions": [ -"latest-1" -] -} -], -"run_settings": { -"cypress_config_file": "cypress.json", -"reporter_config_file": "base_reporter_config.json", -"build_name": "build-name", -"parallels": 1, -"specs": "./*.spec.js", -"ignore_files": "", -"network": false, -"headless": false, -"npm_dependencies": { -"cypress": "9.0.0" -} -}, -"tunnel_settings": { -"tunnel": false, -"tunnel_name": null -} -} -``` - -5. Pass the below command to run the test. - -```bash -lambdatest-cypress run -``` - -6. Visit TestMu AI Automation dashboard to view your test results. The CLI also has a link to view the Cypress test build. - -## Run Your Tests in Parallel - -Once your first test runs, you can execute multiple Cypress tests at once, in either of two ways. - -**Using the CLI:** pass the `--parallels` flag with the number of sessions: - -```bash -lambdatest-cypress run --parallels 5 -``` - -**Using `lambdatest-config.json`:** set the `parallels` key under `run_settings`: - -```json -{ -"run_settings": { -"parallels": 5 -} -} -``` - -**NOTE** -The number of parallel tests running at a time depends on the concurrency plan of your TestMu AI account. If you request more parallel tests than your plan allows, the extra tests are queued and run as running tests finish. For example, on a 5-session plan, running 50 tests runs 5 at a time and queues the other 45. - -## Testing Locally Hosted or Privately Hosted Projects - -To tests locally hosted websites on the TestMu AI platform, you need to setup [TestMu AI tunnel](/docs/testing-locally-hosted-pages/), and execute commands using the CLI, or [Download UnderPass](/docs/underpass-tunnel-application/), our GUI based desktop app. Once you have the TestMu AI tunnel or Underpass set up and started, you can use Cypress to test locally hosted websites. - -Now you need to activate the tunnel capability in the `lambdatest-config.json` file under the section "tunnel_settings" as shown below: - -```json -"tunnel_settings": { -"tunnel": true, -"tunnel_name": "LT_Tunnel" -} -``` - -You can provide the name of the **TestMu AI tunnel** as per your requirements. diff --git a/static/docs/getting-started-with-desktop-browser-real-time-testing.md b/static/docs/getting-started-with-desktop-browser-real-time-testing.md deleted file mode 100644 index 3a94fab53..000000000 --- a/static/docs/getting-started-with-desktop-browser-real-time-testing.md +++ /dev/null @@ -1,30 +0,0 @@ -# Getting started with Desktop Web Browser Real Time Testing - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Real-Time Desktop Web Browser Testing on TestMu AI allows you to test your websites and web applications directly on live desktop browsers. This approach allows developers, testers, and QA professionals to interact with their web applications across various browser environments, operating systems, and versions without the need to configure complex infrastructure on local machines. - -## Launch the Virtual Machine - -- Navigate to the Real Time Testing section. -- Select **Desktop** from the **Web Browser Testing** section. -- To test a website, enter its URL and select your **Operating System**, **browser**, **browser version**, and **screen resolution**. -- Afterwards click on the **Start** button to initiate the testing process. - -## Understanding the Dashboard -The dashboard for real-time desktop browser testing is designed to simplify the testing process with an intuitive user interface. Here’s a breakdown of its key components: - -- **Operating System Selection Panel :** Choose from different OS versions of Windows and macOS, such as Windows 11, Windows 10..., macOS Sequoia, macOS Sonoma, macOS Ventura,... Each selection expands to show supported versions. - -- **Browser and its Version Selection Panel :** Lists the available web browsers and their corresponding versions. Versions are displayed with release dates for easy reference. Clicking on a browser filters the versions available for the selected OS. - -- **Screen Resolution Options :** Shows the available resolutions for the selected browser version. This allows you to test responsive design scenarios and see how your application behaves across various screen sizes. - -- **URL Input Field :** Enter the URL of your web application which you want to test in the real time console of TestMu AI. - -### Additional Options -- The top panel contains advanced settings, including the ability to enable a [Tunnel](/support/docs/real-time-locally-hosted-pages/) for testing internal or locally hosted applications, as well as a toggle for Private Cloud access if applicable. -- [Chrome Extensions](/support/docs/pre-loaded-chrome-extension/) dropdown allows for adding extensions during a test session, making it suitable for testing add-on functionalities. -- The Start button triggers the virtual machine with the chosen settings. - -These features provide flexibility for comprehensive browser testing, making it easier to identify issues and ensure a consistent user experience across different configurations. diff --git a/static/docs/getting-started-with-espresso-testing.md b/static/docs/getting-started-with-espresso-testing.md deleted file mode 100644 index 3306ee0af..000000000 --- a/static/docs/getting-started-with-espresso-testing.md +++ /dev/null @@ -1,203 +0,0 @@ -# Espresso Testing On TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Espresso is a widely-used testing framework for Android, designed to simplify the process of writing reliable and efficient UI tests. It allows developers to create automated tests that simulate user interactions within an app, ensuring that the app's UI behaves as expected. With its straightforward API and synchronization capabilities, Espresso provides a robust solution for validating the functionality and performance of Android applications. - -In this documentation, you will learn how to trigger a automation script of Java for application testing with Appium on TestMu AI, set the desired capabilities for appium testing, and other advanced features of TestMu AI. - -## Prerequisites - -- Your TestMu AI [Username and Access key](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/security). -- Access to an **Android** app (.apk) and an **Espresso Test** app (.apk file). - -If you do not have any **Android** app (.apk) and an **Espresso Test** app (.apk) file, you can run your sample tests on TestMu AI by using our sample :link: [Android app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk) and a sample :link: [Espresso Test](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android_expressotest.apk). - -## Running Your First Test: A Step-by-Step Guide - -### Step 1: Upload Your Application - -To begin testing, upload your Android application (.apk file) to TestMu AI's servers. You'll use our **REST API** for this process. - -- **Authentication :** You'll need your TestMu AI Username and AccessKey. Combine them in the format `Username:AccessKey`. -- **Uploading the App :** Use **cURL command** to send a request to our API. The request should include the path to your application file (**appFile**). - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" --location --request POST 'https://manual-api.lambdatest.com/app/uploadFramework' --form 'appFile=@""' --form 'type="espresso-android"'`} - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" --location --request POST "https://manual-api.lambdatest.com/app/uploadFramework" --form "appFile=@""" --form "type=\"espresso-android\""`} - -- Provide the path of your android application in the above URL in place of `` -- Response of above cURL will be a **JSON** object containing the `App URL` of the format - `lt://APP123456789123456789` and will be used in the last step. - -### Step 2: Upload Your Test Suite - -Upload your Espresso test suite (.apk) file to TestMu AI servers using our REST API. - -The following sample cURL command shows how to upload a test suite: - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" --location --request POST 'https://manual-api.lambdatest.com/app/uploadFramework' --form 'appFile=@""' --form 'type="espresso-android"'`} - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" --location --request POST "https://manual-api.lambdatest.com/app/uploadFramework" --form "appFile=@""" --form "type=\"espresso-android\""`} - -- Provide the path of your android application in the above URL in place of `` -- Response of above cURL will be a **JSON** object containing the `App URL` of the format - `lt://APP123456789123456789` and will be used in the last step. - -### Step 3: Executing The Test - -- You will need **base64 encoded authentication** in order to execute your Espresso automation test suite. Enter your `username:accesskey` in **[Basic Authentication Header Generator](https://mixedanalytics.com/knowledge-base/api-connector-encode-credentials-to-base-64/)** to generate your auth token. - -Take note of the base64 encoded authentication which needs to be added in the next step. - -{`${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}`} - -- Once you have uploaded your app and test suite, you can execute your test by running the following command: - -* On Windows, run the following `curl` command from **Command Prompt (cmd.exe)**. The command is formatted for the Windows CMD terminal and may require different syntax if executed from PowerShell or another shell. -* Before running the command, replace `BASIC_AUTH_TOKEN`, `APP_ID`, and `TEST_SUITE_ID` with your actual LambdaTest credentials and resource IDs obtained from the above curl commands. - -```bash -curl --location --request POST 'https://mobile-api.lambdatest.com/framework/v1/espresso/build' \ ---header 'Authorization: Basic BASIC_AUTH_TOKEN' \ ---header 'Content-Type: application/json' \ ---data-raw '{ -"app": "APP_ID", -"testSuite": "TEST_SUITE_ID", -"device": ["Galaxy S21 5G-12"], -"queueTimeout": 10800, -"IdleTimeout": 150, -"deviceLog": true, -"network": false, -"build": "Proverbial-Espresso", -"geoLocation": "FR" -}' -``` - -```bash -curl --location --request POST "https://mobile-api.lambdatest.com/framework/v1/espresso/build" ^ ---header "Authorization: Basic BASIC_AUTH_TOKEN" ^ ---header "Content-Type: application/json" ^ ---data "{\"app\":\"APP_ID\",\"testSuite\":\"TEST_SUITE_ID\",\"device\":[\"Pixel 6-12\"],\"queueTimeout\":10800,\"IdleTimeout\":150,\"deviceLog\":true,\"network\":false,\"build\":\"Proverbial-Espresso\",\"geoLocation\":\"FR\"}" -``` - -```bash -curl --location --request POST 'https://mobile-api.lambdatest.com/framework/v1/espresso/build' \ ---header 'Authorization: Basic BASIC_AUTH_TOKEN' \ ---header 'Content-Type: application/json' \ ---data-raw '{ -"app": "APP_ID", -"testSuite": "TEST_SUITE_ID", -"device": ["Galaxy S21 5G-12.0"], -"isVirtualDevice": true, -"queueTimeout": 10800, -"IdleTimeout": 150, -"deviceLog": true, -"network": false, -"build": "Proverbial-Espresso", -"geoLocation": "FR" -}' -``` - -```bash -curl --location --request POST "https://mobile-api.lambdatest.com/framework/v1/espresso/build" ^ ---header "Authorization: Basic BASIC_AUTH_TOKEN" ^ ---header "Content-Type: application/json" ^ ---data "{\"app\":\"APP_ID\",\"testSuite\":\"TEST_SUITE_ID\",\"device\":[\"Pixel 6-12.0\"],\"isVirtualDevice\":true,\"queueTimeout\":10800,\"IdleTimeout\":150,\"deviceLog\":true,\"network\":false,\"build\":\"Proverbial-Espresso\",\"geoLocation\":\"FR\"}" -``` - -### Step 4: View Test Execution - -Once you have run your tests, you can view the test execution along with logs. You will be able to see the test cases passing or failing. You can view the same at TestMu AI Automation. - -## Running Tests in Parallel - -You can run tests in parallel on multiple devices by passing the device name in comma separated format in the execute command as show below: - -* On Windows, run the following `curl` command from **Command Prompt (cmd.exe)**. The command is formatted for the Windows CMD terminal and may require different syntax if executed from PowerShell or another shell. -* Before running the command, replace `BASIC_AUTH_TOKEN`, `APP_ID`, and `TEST_SUITE_ID` with your actual LambdaTest credentials and resource IDs obtained from the above curl commands. - -```bash -curl --location --request POST 'https://mobile-api.lambdatest.com/framework/v1/espresso/build' \ ---header 'Authorization: Basic BASIC_AUTH_TOKEN' \ ---header 'Content-Type: application/json' \ ---data-raw '{ -"app": "APP_ID", -"testSuite": "TEST_SUITE_ID", -"device": ["Galaxy S20-10","Galaxy S20-10","Redmi Note 9-10","Galaxy S10+-10","Galaxy S7 edge-8","Galaxy S9+-8"], -"queueTimeout": 10800, -"IdleTimeout": 150, -"deviceLog": true, -"network": false, -"build": "Proverbial-Espresso", -"geoLocation": "FR" -}' -``` - -```bash -curl --location --request POST "https://mobile-api.lambdatest.com/framework/v1/espresso/build" ^ ---header "Authorization: Basic BASIC_AUTH_TOKEN" ^ ---header "Content-Type: application/json" ^ ---data "{\"app\":\"APP_ID\",\"testSuite\":\"TEST_SUITE_ID\",\"device\":[\"Galaxy S20-10\",\"Galaxy S20-10\",\"Redmi Note 9-10\",\"Galaxy S10+-10\",\"Galaxy S7 edge-8\",\"Galaxy S9+-8\"],\"queueTimeout\":10800,\"IdleTimeout\":150,\"deviceLog\":true,\"network\":false,\"build\":\"Proverbial-Espresso\",\"geoLocation\":\"FR\"}" -``` - -```bash -curl --location --request POST 'https://mobile-api.lambdatest.com/framework/v1/espresso/build' \ ---header 'Authorization: Basic BASIC_AUTH_TOKEN' \ ---header 'Content-Type: application/json' \ ---data-raw '{ -"app": "APP_ID", -"testSuite": "TEST_SUITE_ID", -"device": ["Galaxy S20-10.0","Galaxy S20-10.0","Redmi Note 9-10.0","Galaxy S10+-10.0","Galaxy S7 edge-8.0","Galaxy S9+-8.0"], -"isVirtualDevice": true, -"queueTimeout": 10800, -"IdleTimeout": 150, -"deviceLog": true, -"network": false, -"build": "Proverbial-Espresso", -"geoLocation": "FR" -}' -``` - -```bash -curl --location --request POST "https://mobile-api.lambdatest.com/framework/v1/espresso/build" ^ ---header "Authorization: Basic BASIC_AUTH_TOKEN" ^ ---header "Content-Type: application/json" ^ ---data "{\"app\":\"APP_ID\",\"testSuite\":\"TEST_SUITE_ID\",\"device\":[\"Galaxy S20-10.0\",\"Galaxy S20-10.0\",\"Redmi Note 9-10.0\",\"Galaxy S10+-10.0\",\"Galaxy S7 edge-8.0\",\"Galaxy S9+-8.0\"],\"isVirtualDevice\":true,\"queueTimeout\":10800,\"IdleTimeout\":150,\"deviceLog\":true,\"network\":false,\"build\":\"Proverbial-Espresso\",\"geoLocation\":\"FR\"}" -``` - -## Using the Espresso Agent Skill with TestMu AI - -The [espresso-skill](https://github.com/LambdaTest/agent-skills/tree/main/espresso-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The espresso-skill package includes: - -``` -espresso-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing Espresso Agent Skill - -Install a Espresso Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/espresso-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/espresso-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only espresso-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). diff --git a/static/docs/getting-started-with-flutter-dart-android-automation.md b/static/docs/getting-started-with-flutter-dart-android-automation.md deleted file mode 100644 index f3e41f0f4..000000000 --- a/static/docs/getting-started-with-flutter-dart-android-automation.md +++ /dev/null @@ -1,229 +0,0 @@ -# Flutter Dart Testing On TestMu AI - Android - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Flutter, an open-source UI toolkit created by Google, is a popular choice among developers to build natively compiled applications for mobile, web, and desktop from a single codebase. With Dart as its programming language, Flutter enables fast development of beautiful apps with a highly productive, extensible and open-source set of features. - -In this guide, we will explore how to run your first Flutter Dart test on an Android device using the TestMu AI Real Device Cloud. By combining the capabilities of Flutter Dart and TestMu AI, you can ensure the proper functioning of your app across different devices and make your app more reliable and robust. - -## Prerequisites for Getting Started - -1. Flutter SDK installed on your system -2. You will need a TestMu AI username and access key. To obtain your access credentials, [purchase a plan](https://billing.lambdatest.com/billing/plans) or access the [automation dashboard](https://www.testmuai.com/login/?redirectTo=https://appautomation.lambdatest.com/). -3. Access to an android **Sample** app (.apk) and an **Sample Test Suite** app (.apk file). - -If you do not have any **Flutter Android** app (.apk) and an **Flutter Test Suite** app (.apk) file, you can run your sample tests on TestMu AI by using our sample :link: [Android app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/sample-flutter-app.apk) and a sample :link: [Test Suite](https://prod-mobile-artefacts.lambdatest.com/assets/docs/sample-flutter-testsuite.apk). - -## Run Your First Test - -### Step 1: Create your Android Flutter app and test suite for testing -For testing, you need to build a Flutter app and test suite. You can create Flutter applications and test suites using either Flutter cli or Gradlew. The steps below demonstrate how to create apks with Gradlew. - -- Create an instrumentation test file in your application's directory `android/app/src/androidTest/java/com/example/lambdatestSampleApp/`. Replace **com**, **example**, and **lambdatestSampleApp** values with those from your app's package name. - -```java title="SampleTest.java" -package com.example.lambdatestSampleApp; -import androidx.test.rule.SampleTestRule; -import dev.flutter.plugins.integration_test.FlutterTestRunner; -import org.junit.Rule; -import org.junit.runner.RunWith; -import com.example.lambdatestSampleApp.Sample; -@RunWith(FlutterTestRunner.class) -public class SampleTest { -@Rule -public SampleTestRule rule = new SampleTestRule<>(Sample.class, true, false); -} -``` - -- Update your application's `lambdatestSampleApp/android/app/build.gradle` file to use androidx's version of `AndroidJUnitRunner` and include the `androidx` libraries as dependencies. - -```java title="build.gradle" -android { -... -defaultConfig { -... -testInstrumentationRunner "androidx.test.runner.AndroidJUnitRunner" -} -} -dependencies { -testImplementation 'junit:junit:4.12' -androidTestImplementation 'androidx.test:runner:1.2.0' -androidTestImplementation 'androidx.test.espresso:espresso-core:3.2.0' -} -``` - -- Use the following `Gradle` commands to build an instrumentation `test.apk` file(test suite) using the `Sample.java` created in the `androidTest` directory as mentioned in step 1. - -```java title="Terminal" -//Go to the android folder which contains the "gradlew" script used for building Android apps from the terminal -pushd android -//Build an Android test APK (uses the Sample.java file created in step 1) -./gradlew app:assembleAndroidTest -//Build a debug APK by passing the integration test file -./gradlew app:assembleDebug -Ptarget="INTEGRATION_TEST_FILE_PATH" -//Go back to the root of the project -popd -``` - -Avoiding this step might result in **No Tests Ran** issue on the dashboard - -To create APKs with optional Flutter parameters, first run the Flutter tests in verbose mode with the flutter cli. This allows you to see the Gradle command used internally to build the APKs. - -For example, to use `--no-sound-null-safety` in your tests, run the following command. - -```bash -flutter run -v --no-sound-null-safety -``` - -Next, look for gradlew execution in the logs. The above command generates a gradlew command in the logs that looks something like the following. To build your apk files, replace the parameter `YOUR_APP_PATH` with your actual path of the application in the following command: - -```bash -gradlew --full-stacktrace --info -Pverbose=true -Ptarget-platform=android-arm64 -Ptarget=YOUR_APP_PATH/lib/main.dart -Pbase-application-name=android.app.Application -Pdart-obfuscation=false -Pextra-front-end-options=--no-sound-null-safety -Ptrack-widget-creation=true -Ptree-shake-icons=false -Pfilesystem-scheme=org-dartlang-root assembleDebug -``` - -### Step 2: Upload Your Application - -Upload your **android** application (.apk file) to the TestMu AI servers using our **REST API**. You need to provide your **Username** and **AccessKey** in the format `Username:AccessKey` in the **cURL** command for authentication. Make sure to add the path of the **appFile** in the cURL request. Here is an example cURL request to upload your app using our REST API: - -**Using App File:** - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" \\ ---location --request POST 'https://manual-api.lambdatest.com/app/uploadFramework' \ ---form 'appFile=@"/Users/macuser/Downloads/sample-flutter-app.apk"' \ ---form 'type="flutter-android"'`} - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" --location --request POST "https://manual-api.lambdatest.com/app/uploadFramework" --form "appFile=@"C:/Users/winuser/Downloads/proverbial_android.apk"" --form "type=\"flutter-android\""`} - -Response of above cURL will be a **JSON** object containing the `App URL` of the format - ``lt://APP123456789123456789123456789`` and will be used in the last step. - -### Step 3: Uploading Test Suite - -Upload your **test suite** (.apk file) to the TestMu AI servers using our **REST API**. You need to provide your **Username** and **AccessKey** in the format `Username:AccessKey` in the **cURL** command for authentication. Make sure to add the path of the **appFile** in the cURL request. Here is an example cURL request to upload your app using our REST API: - -**Using App File:** - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" \\ ---location --request POST 'https://manual-api.lambdatest.com/app/uploadFramework' \ ---form 'appFile=@"/Users/macuser/Downloads/sample-flutter-testsuite.apk"' \ ---form 'type="flutter-android"'`} - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" --location --request POST "https://manual-api.lambdatest.com/app/uploadFramework" --form "appFile=@"C:/Users/winuser/Downloads/proverbial_android_expressotest.apk"" --form "type=\"flutter-android\""`} - -Response of above cURL will be a **JSON** object containing the `App URL` of the format - ``lt://APP123456789123456789123456789`` and will be used in the next step. - -### Step 4: Executing The Test - -#### Basic Authentication - -You will need base64 encoded authentication in order to execute your Espresso automation test suite. You need to enter your username:accesskey **[here](https://mixedanalytics.com/knowledge-base/api-connector-encode-credentials-to-base-64/)** in order and click on encode to generate the base64 authentication. Take note of the **base64** encoded authentication which needs to be added in the next step. - -{`${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}`} - -#### Execute Command - -Once you have uploaded your app and test suite, you need to you can execute your test by running the following command: - -Make sure to enter your **basic authentication**, **app url** (generated in the first step) and **testSuite url** (generated in the second step) in the below command. - -```bash -curl --location --request POST 'https://mobile-api.lambdatest.com/framework/v1/flutter/android/build' \ ---header 'Authorization: Basic ' \ ---header 'Content-Type: application/json' \ ---data-raw '{ -"app" : "lt://APP_ID", -"testSuite": "lt://TestSuite_ID", -"device" : ["Galaxy S21 5G-12"], -"deviceLog": true, -"network": false, -"build" : "Sample-Flutter" -}' -``` - -```bash -curl --location --request POST "https://mobile-api.lambdatest.com/framework/v1/flutter/android/build" --header "Content-Type: application/json" --header "Authorization: Basic " --data-raw "{\"app\" : \"lt://APP_ID\",\"testSuite\": \"lt://APP_ID\",\"device\" : [\"Pixel 6-12\"], \"deviceLog\": true,\"network\": false,\"build\" : \"Sample-Flutter\"}" -``` - -### Step 5: View Test Execution - -Once you have run your tests, you can view the test execution along with logs. You will be able to see the test cases passing or failing. You can view the same at [TestMu AI Automation](https://www.testmuai.com/login/?redirectTo=https://appautomation.lambdatest.com/build). - -## Running Tests in Parallel - -You can run tests in parallel on multiple devices by passing the device name in comma separated format in the execute command as show below: - -```bash -curl --location --request POST 'https://mobile-api.lambdatest.com/framework/v1/flutter/android/build' \ ---header 'Authorization: Basic ' \ ---header 'Content-Type: application/json' \ ---data-raw '{ -"app" : "app_id", -"testSuite": "testsuite_id", -"device" : ["Galaxy S22 5G-12","Galaxy S24-14","Pixel 7-13","Galaxy S10+-10"], -"queueTimeout": 10800, -"testTimeout": 900, -"deviceLog": true, -"build" : "Sample-Flutter", -}' -``` - -Do note that Flutter builds when run in parallel, result in separate builds being generated for each parallel run. - -## Auto-Grant Permissions for Android Apps - -While testing Android apps, user might need to handle various pop-ups or dialogs asking for permissions like contacts, notifications, photos, etc. - -To simplify this process, set Flutter's `autoGrantPermissions` parameter to automatically grant the required permissions based on the [Android manifest](https://developer.android.com/guide/topics/manifest/manifest-intro) in the app’s **.APK** file.Please find a example `cURL` request to enable `autoGrantPermissions`: - -```bash -curl --location --request POST 'https://mobile-api.lambdatest.com/framework/v1/flutter/android/build' \ ---header 'Authorization: Basic ' \ ---header 'Content-Type: application/json' \ ---data-raw '{ -"app" : "app_id", -"testSuite": "testsuite_id", -"device" : ["Galaxy S22 5G-12"] -"queueTimeout": 10800, -"testTimeout": 900, -"deviceLog": true, -"build" : "Sample-Flutter", -"autoGrantPermissions" : true -}' -``` - -## Using the Flutter Testing Agent Skill with TestMu AI - -The [flutter-testing-skill](https://github.com/LambdaTest/agent-skills/tree/main/flutter-testing-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The flutter-testing-skill package includes: - -``` -flutter-testing-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing Flutter Testing Agent Skill - -Install a Flutter Testing Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/flutter-testing-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/flutter-testing-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only flutter-testing-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). diff --git a/static/docs/getting-started-with-flutter-dart-ios-automation.md b/static/docs/getting-started-with-flutter-dart-ios-automation.md deleted file mode 100644 index 0fd1df10e..000000000 --- a/static/docs/getting-started-with-flutter-dart-ios-automation.md +++ /dev/null @@ -1,149 +0,0 @@ -# Flutter Dart Testing On TestMu AI - iOS - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Flutter, an open-source UI toolkit created by Google, is a popular choice among developers to build natively compiled applications for mobile, web, and desktop from a single codebase. With Dart as its programming language, Flutter enables fast development of beautiful apps with a highly productive, extensible and open-source set of features. - -In this guide, we will explore how to run your first Flutter Dart test on an iOS device using the TestMu AI Real Device Cloud. By combining the capabilities of Flutter Dart and TestMu AI, you can ensure the proper functioning of your app across different devices and make your app more reliable and robust. - -## Prerequisites for Getting Started - -1. Flutter SDK installed on your system -2. You will need a TestMu AI username and access key. To obtain your access credentials, [purchase a plan](https://billing.lambdatest.com/billing/plans) or access the [automation dashboard](https://www.testmuai.com/login/?redirectTo=https://appautomation.lambdatest.com/). -3. Access to a Flutter **Sample** test suite (.zip). - -If you do not have any **Flutter iOS** test suite (.zip), you can run your sample tests on TestMu AI by using our sample :link: [Flutter iOS](https://prod-mobile-artefacts.lambdatest.com/assets/docs/Flutter_iOS.zip). - -## Run Your First Test - -### Step 1: Upload Your Test Suite - -Upload your **iOS** test suite (.zip file) to the TestMu AI servers using our **REST API**. You need to provide your **Username** and **AccessKey** in the format `Username:AccessKey` in the **cURL** command for authentication. Make sure to add the path of the **appFile** in the cURL request. Here is an example cURL request to upload your app using our REST API: - -**Using App File:** - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" \\ ---location --request POST 'https://manual-api.lambdatest.com/app/uploadFramework' \ ---form 'appFile=@"/Users/macuser/Downloads/flutter-ios.zip"' \ ---form 'type="flutter-ios"' ---form 'name="Sample App"' ---form 'custom_id="Sample"'`} - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" --location --request POST "https://manual-api.lambdatest.com/app/uploadFramework" --form "appFile=@"C:/Users/winuser/Downloads/flutter-ios.zip"" --form "type=\"flutter-ios\"" --form "name=\"Sample App\"" --form "custom_id=\"Sample\""`} - -Response of above cURL will be a **JSON** object containing the `App id` of the format - ``lt://APP123456789123456789123456789`` and will be used in the last step. - -### Step 2: Executing The Test - -#### Basic Authentication - -You will need base64 encoded authentication in order to execute your Espresso automation test suite. You need to enter your username:accesskey **[here](https://mixedanalytics.com/knowledge-base/api-connector-encode-credentials-to-base-64/)** in order and click on encode to generate the base64 authentication. Take note of the **base64** encoded authentication which needs to be added in the next step. - -{`${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}`} - -#### Execute Command - -Once you have uploaded your test suite, you can execute your test by running the following command: - -Make sure to enter your **basic authentication** and **app id** (generated in the first step) in the below command. - -```bash -curl --location --request POST 'https://mobile-api.lambdatest.com/framework/v1/flutter/ios/build' \ ---header 'Authorization: Basic ' \ ---header 'Content-Type: application/json' \ ---data-raw '{ -"testSuite": "lt://App_id", -"device": ["iPhone.*-16"], -"video": true, -"deviceLog": true, -"queueTimeout": 9000, -"network": true, -"build": "Flutter iOS" -}' -``` - -```powershell -curl --location --request POST "https://mobile-api.lambdatest.com/framework/v1/flutter/ios/build" --header "Content-Type: application/json" --header "Authorization: Basic " --data-raw "{\"testSuite\": \"lt://APP_ID\",\"device\" : [\"Pixel 6-12\"], \"video\": true, \"deviceLog\": true, \"queueTimeout\" : \"9000\",\"idleTimeout\" : \"600\"\"network\": true,\"build\" : \"Flutter iOS\"}" -``` - -### Step 3: View Test Execution - -Once you have run your tests, you can view the test execution along with logs. You will be able to see the test cases passing or failing. You can view the same at [TestMu AI Automation](https://www.testmuai.com/login/?redirectTo=https://appautomation.lambdatest.com/builds). - -## Capabilities Supported - -The following capabilities are supported: - -1. **testSuite:** Enter the app id generated while uploading the app. Example:`lt://APP123456789123456789123456789` -2. **device:** Enter the name and os version of the device in "DeviceName-OSVersion" format. Example: `iPhone 14-16` or `iPhone 15 Pro-17`. You can also choose to use regular expression for device name such as `iPhone.*-16`. -3. **video:** Generate video for all the tests that have run. Example: `true`. -4. **queueTimeout:** Enter the time in seconds for total execution time of the build. Example: `9000`. Minimum queueTimeout: `1800`, Maximum queueTimeout: `10800` -5. **network:** Boolean value to generate network logs. Example: `"network": true`. -6. **build:** Set the name of the Flutter iOS test build. Example: `My Flutter iOS`. -7. **deviceLog:** Boolean value to generate device logs. Example: `true`. -8. **tunnel:** Boolean value to run build via Tunnel. Example: `true`. -9. **tunnelName:** Name of the tunnel to be used in conjunction with `tunnel` capability. -10. **region:** String to device the region from which device will be allocated. Example - `eu`,`us`,`ap`. -11. **gpsLocation:** This capability should be defined to override the device’s default GPS coordinates and should be provided as a comma-separated string. Example: `"location": {"lat": "-50","long": "150.028333"}`. -12. **enableBluetooth:** This capability is used to enable the bluetooth functionality during the session. Example: `enableBluetooth: true` - -Ensure that the latitude is between -90 and 90, and the longitude is between -180 and 180. Otherwise, an error will occur like "Invalid GPS location: Latitude must be between -90 and +90, Longitude must be between -180 and +180." - -## Running Tests in Parallel - -You can run tests in parallel on multiple devices by passing the device name in comma separated format in the execute command as show below: - -```bash -curl --location --request POST 'https://mobile-api.lambdatest.com/framework/v1/flutter/build' \ ---header 'Authorization: Basic ' \ ---header 'Content-Type: application/json' \ ---data-raw '{ -"testSuite": "lt://App_id", -"device": ["iPhone.*-16","iPhone 14-16","iPhone 15-17"], -"video": true, -"deviceLog": true, -"queueTimeout": 9000, -"network": true, -"build": "Flutter iOS - Parallel" -}' -``` - -Do note that Flutter builds when run in parallel, result in separate builds being generated for each parallel run. - -## Using the Flutter Testing Agent Skill with TestMu AI - -The [flutter-testing-skill](https://github.com/LambdaTest/agent-skills/tree/main/flutter-testing-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The flutter-testing-skill package includes: - -``` -flutter-testing-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing Flutter Testing Agent Skill - -Install a Flutter Testing Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/flutter-testing-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/flutter-testing-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only flutter-testing-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). diff --git a/static/docs/getting-started-with-hyperexecute.md b/static/docs/getting-started-with-hyperexecute.md deleted file mode 100644 index 3cabe187f..000000000 --- a/static/docs/getting-started-with-hyperexecute.md +++ /dev/null @@ -1,117 +0,0 @@ -# Getting Started with HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Welcome to HyperExecute, **an AI-Native Test Orchestration Cloud Platform**, designed to accelerate your testing process, enabling test automation teams worldwide to achieve an accelerated time to market by providing **JIT (Just-in-time) testing infrastructure**. - -If you want to learn more about HyperExecute and how it can enable your team, then you are the right place. This guide will help you take your first steps with HyperExecute, from understanding the basics to running your first test. - -## What is HyperExecute? - -HyperExecute by TestMu AI is a cutting-edge, AI-powered platform designed for DevOps teams and QA engineers to significantly accelerate software testing pipelines. HyperExecute **replaces traditional hub-and-node test grids** with advanced orchestration techniques, **eliminating network latency and optimizing resource usage**. With HyperExecute: - - - **Slash your time-to-market** with parallel execution that distributes tests optimally across available resources improving efficiency. - - **Make data-driven testing decisions** using historical performance trends that reveal patterns and help optimize your testing strategy - - **Never worry about infrastructure constraints** with dynamic scaling to handle peak testing demands, running 10 tests or 10,000. - - **Enable multiple teams using different frameworks** to run tests simultaneously across various environments on same license. - -Whether testing web or mobile applications, HyperExecute empowers your team to deliver higher-quality software rapidly and reliably. - -## Compatible with All Major Frameworks -HyperExecute seamlessly supports all major test automation frameworks as shown below. Regardless of your existing testing stack, it will integrate smoothly to enhance your workflows without disruption. - - In case your favorite framework or tool isn't listed here, just give us a Shout Out Here. - In case of any other problems, feel free to Contact Us. - -## Let's Run your First Test -Ready to experience the power of HyperExecute? Getting started is easy. We’ve simplified the process so you can effortlessly launch your first test with three flexible approaches and experience the speed by yourself: - -**RUN SAMPLE TEST** - - Execute tests directly from the HyperExecute Portal with sample setup - perfect for new users who want to see results immediately. - - Learn More - - Run tests from your own machine using the HyperExecute CLI with sample project and preset configuration file. - - Learn More - - Run your tests in a fully managed online development environment, without any local installations or configurations. - - Learn More - -## Watch HyperExecute in Action - -Get a comprehensive overview of HyperExecute's capabilities by watching our introductory webinar. - -## Next Steps - -Now that you've set up HyperExecute and run your first test, explore these resources to make the most of the platform: - - Running Your First Job on HyperExecute - A step-by-step walkthrough of how to run your first job on the HyperExecute Platform. - - HyperExecute CLI - Understand how you can initiate jobs using our CLI from your local setup or CI/CD pipelines. - - HyperExecute YAML - Learn about the different configurations & creating a YAML suited for your needs. - - Status - Everything you need to know about HyperExecute status and what does it implies. - - Key Features - Learn about the different features and how you can use them in your project. - - HyperExecute vs Traditional Test grids - Understand how we expedite test execution as compared to traditional grids. - - HyperExecute Private Cloud - Learn how to setup HyperExecute Private Cloud for your organization. - - Integrations - Know how to setup with your CI/CD, Project management tools and TestMu AI products. - -**We’re Already in Your Favorite Cloud!** -Deploy our cloud-native test orchestration platform directly from your trusted marketplace: -- [Microsoft Azure Marketplace](https://azuremarketplace.microsoft.com/en-us/marketplace/apps/lambdatestinc1584019832435.hyper_execute?exp=ubp8&tab=Overview) -- [Amazon Web Services Marketplace](https://aws.amazon.com/marketplace/pp/prodview-35cneaxfou2eq) -- [Google Cloud Provider Marketplace](https://console.cloud.google.com/marketplace/product/hyperexecute-privatecloud/lambdatest?pli=1) - -## Using the HyperExecute Agent Skill with TestMu AI - -The [hyperexecute-skill](https://github.com/LambdaTest/agent-skills/tree/main/hyperexecute-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The hyperexecute-skill package includes: - -``` -hyperexecute-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing HyperExecute Agent Skill - -Install a HyperExecute Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/hyperexecute-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/hyperexecute-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only hyperexecute-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). diff --git a/static/docs/getting-started-with-kane-ai.md b/static/docs/getting-started-with-kane-ai.md deleted file mode 100644 index 3e6a2ae92..000000000 --- a/static/docs/getting-started-with-kane-ai.md +++ /dev/null @@ -1,111 +0,0 @@ -# Introduction to KaneAI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Testing shouldn’t slow innovation. KaneAI makes test automation as natural as writing a To Do list with no steep learning curves, no endless script maintenance. - -With KaneAI you can: - -- **Go from ideas to scripts in seconds** – Turn Jira tickets, PRDs, or plain text into ready-to-run test cases. - -- **Remove coding barrier** – Author tests with natural language, or let KaneAI write automation scripts for you. - -- **Scale without friction** – Run tests across browsers, devices, and environments with TestMu AI’s cloud. - -- **Make testing Adaptive & intelligent** – KaneAI auto-heals tests when your UI changes, so your team keeps moving. - -**KaneAI isn’t just a tool it’s how modern teams test smarter and ship faster.** - -## Why KaneAI? - -KaneAI transforms the way teams approach test automation by bridging the gap between natural language requirements and executable test code. Built for modern development workflows, KaneAI empowers teams to plan, author, and evolve comprehensive test suites without the traditional barriers of complex scripting or framework expertise. - -### Current Testing Scenario Challenge - -Traditional test automation often creates bottlenecks in development cycles. Teams face several persistent challenges: - -- **Technical Barriers:** Writing and maintaining test scripts requires specialized programming knowledge, limiting who can contribute to test creation. -- **Framework Complexity:** Managing multiple testing frameworks, languages, and integrations creates overhead and maintenance burden. -- **Scalability Issues:** As applications grow, test suites become increasingly difficult to organize, update, and execute consistently. -- **Security Concerns:** Handling authentication credentials and sensitive test data safely across different environments. -- **Integration Friction:** Connecting testing workflows with CI/CD pipelines often requires significant configuration and ongoing maintenance. - -### The KaneAI Solution - -KaneAI addresses these challenges through an intelligent, unified platform that democratizes end-to-end test automation while maintaining enterprise-grade capabilities. - -- **Intelligent Input Processing:** KaneAI offers dual flexibility in test authoring by intelligently generating structured test cases by transforming various input formats such as text, JIRA tickets, PDFs, images, audio, videos, and spreadsheets, or manually author tests using conversational language for precise control. Whether you need rapid test generation from existing documentation or prefer hands-on test crafting, KaneAI's advanced processing capabilities streamline both approaches. - -- **Unified Test Management:** Consolidate your entire testing ecosystem within a single platform. KaneAI provides centralized control over test cases, execution plans, and results analysis, eliminating the fragmentation common in traditional testing tools. Teams benefit from: - - - Complete visibility into test coverage and execution status - - Streamlined collaboration between development, QA, and business teams - - Simplified reporting and metrics tracking across all test activities - -- **Multi-Language & Framework Support:** KaneAI supports all prominent frameworks and languages, offering the flexibility to export test code in the language and framework that you and your team prefers. - -- **Comprehensive API Testing:** Create robust API test suites with intelligent validation and seamless integration capabilities. KaneAI understands API contracts and automatically generates meaningful assertions, reducing the manual effort typically required for thorough API testing coverage. - -- **Secret Management:** Securely handle authentication credentials and sensitive test data across environments, so secrets never need to be hard-coded into your tests. - -- **CI/CD Integration:** Connect seamlessly with your existing continuous integration and deployment pipelines. KaneAI's integration capabilities ensure automated test execution becomes a natural part of your development workflow, supporting consistent quality gates across multiple environments and deployment stages. - -## Use Cases Across Business Domains - -### Application Suites - -KaneAI accelerates testing and validation across complex application suites by auto-generating test cases, simulating user journeys, detecting integration failures, and continuously validating performance and reliability across modules and releases. - -### Corporate Legal Management - -KaneAI automates validation of legal workflows by testing contract lifecycle systems, clause extraction accuracy, compliance workflows, and document management reliability to help reduce legal risk and operational errors. - -### Customer Service, Support & CRM/CX - -KaneAI improves quality of CRM and CX platforms by autonomously testing customer journeys, chatbot accuracy, ticket workflows, integrations, and performance under high-volume scenarios to ensure seamless omnichannel experiences. - -### Digital Commerce - -KaneAI validates end-to-end eCommerce flows including browsing, search, cart, payments, offers, and order tracking by generating AI-driven test scenarios that detect checkout failures, latency issues, and conversion blockers. - -### Finance - -KaneAI strengthens financial systems by testing transaction integrity, reconciliation workflows, regulatory compliance, security vulnerabilities, and system resilience through AI-generated tests and real-world transaction simulations. - -### Human Resources / Recruiting - -KaneAI ensures HR and recruiting platforms perform reliably by testing applicant workflows, onboarding journeys, payroll accuracy, integrations with background checks, and system scalability during high hiring demand. - -### IT Operations / Service Desk - -KaneAI enhances ITSM and service desk platforms by validating incident, change, and asset workflows, chatbot accuracy, monitoring integrations, and system performance to improve uptime and operational efficiency. - -### Marketing - -KaneAI validates marketing automation platforms by testing campaign workflows, personalization engines, data integrations, email/SMS delivery, segmentation accuracy, and analytics reliability to maximize campaign performance. - -### Sales & Lead Management - -KaneAI ensures sales platforms function flawlessly by testing lead capture, routing, scoring, CRM synchronization, forecasting accuracy, and pipeline health across channels and integrations. - -### Security & Fraud Detection - -KaneAI proactively tests security and fraud systems by simulating attack vectors, anomaly scenarios, false-positive/false-negative conditions, and performance under threat load to strengthen detection accuracy and resilience. - -### Sourcing & Procurement - -KaneAI validates procurement workflows by testing vendor onboarding, purchase approval flows, contract compliance, invoice matching, and ERP integrations to reduce errors and cycle times. - -### Supply Chain - -KaneAI tests supply chain platforms by simulating demand spikes, inventory fluctuations, logistics delays, supplier failures, and integration accuracy to ensure system resilience and real-time visibility. - -### Workplace & Digital Workplace - -KaneAI strengthens digital workplace platforms by testing collaboration tools, access controls, identity flows, remote work scenarios, and performance across devices to ensure secure and seamless employee experiences. - -## Getting Started -If you are new to KaneAI, you can learn more about how to use KaneAI by following guides: - -## How to Instruct KaneAI -Learn what is the best way to tell KaneAI what it needs to do: diff --git a/static/docs/getting-started-with-mobile-app-real-time-testing.md b/static/docs/getting-started-with-mobile-app-real-time-testing.md deleted file mode 100644 index 0718c16be..000000000 --- a/static/docs/getting-started-with-mobile-app-real-time-testing.md +++ /dev/null @@ -1,29 +0,0 @@ -# Getting started with Mobile App Real Time Testing - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Real-Time Mobile App Testing allows you to test mobile applications directly on real devices in real time. It ensures that your app functions as intended, performs optimally, and provides a seamless user experience across various mobile operating systems such as Android and iOS. With TestMu AI, developers, testers, and QA professionals can interact with their mobile applications as end users would, on actual devices, without needing to manage an in-house device lab. - -## Launch the Virtual Machine - -- Navigate to the Real Time Testing section. -- Select **Virtual Mobile** from the **App Testing** section. -- To test your application, enter its URL or upload your application from your local machine and select your **Device** - Andoroid, iOS, or [ChromeOS](/support/docs/chrome-os-testing/), and its version. -- Afterwards click on the **Start** button to initiate the testing process. - -## Dashboard Walkthrough -The Real-Time Testing dashboard provides an intuitive interface for managing your testing sessions: - -- **Operating System Tabs :** Located at the top, this section allows you to toggle between different operating systems (Android, and iOS). - -- **Device Selection Panel :** Allows you to filter devices by operating system (Android or iOS), manufacturer, and model. Lists popular devices, including the latest models, to test on. - -- **Upload Apps :** In the dashboard, there is a section for uploading your mobile application. You can either: - - **Provide a URL :** Use this if the app is hosted online or accessible via a direct link. - - **Upload from Local System :** Use this option to upload the application file (APK/IPA) from your computer. - -- **Tunnel :** For testing on environments that are not publicly accessible, configure the Tunnel settings to enable testing on staging or local sites. - -- **Switch to Real Devices :** If you need to perform tests on physical devices, click on the Switch to Real Devices link to access TestMu AI's real device cloud. - -- **Start Button :** Located on the right side of the URL field, this button initializes the testing session once you have chosen your device and browser configuration. diff --git a/static/docs/getting-started-with-mobile-browser-real-time-testing.md b/static/docs/getting-started-with-mobile-browser-real-time-testing.md deleted file mode 100644 index d0d488f25..000000000 --- a/static/docs/getting-started-with-mobile-browser-real-time-testing.md +++ /dev/null @@ -1,29 +0,0 @@ -# Getting started with Mobile Web Browser Real Time Testing - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Real-Time Mobile Web Browser Testing allows you to test websites and web applications across different mobile browsers and operating systems, such as Android, iOS, and ChromeOS, in real-time. This process helps ensure that your web applications are fully functional, visually consistent, and optimized for various mobile devices, enhancing the user experience across different platforms. Using TestMu AI, developers, testers, and QA professionals can validate their web applications without the need for physical devices, all from a unified, user-friendly interface. - -## Launch the Virtual Machine - -- Navigate to the Real Time Testing section. -- Select **Virtual Mobile** from the **Web Browser Testing** section. -- To test a website, enter its URL and select your **Device** - Andoroid, iOS, or [ChromeOS](/support/docs/chrome-os-testing/), and its version. -- Afterwards click on the **Start** button to initiate the testing process. - -## Understanding the Dashboard -The TestMu AI dashboard is designed to simplify the testing process. Here's a breakdown of its key components: - -- **Operating System Tabs :** Located at the top, this section allows you to toggle between different operating systems (Android, iOS, and ChromeOS). - -- **Browser Selection :** Below the OS tabs, a dropdown lets you choose the browser you want to test on. Available options may vary depending on the selected OS. - -- **Device Selection Pane :** This panel lists the available devices manufacturers and models. You can quickly search for a device using the search bar, or filter by manufacturer (e.g., Samsung, Google, Apple). The list also displays the OS version supported for each device. - -- **URL Field :** The top portion of the dashboard provides a field to enter the target URL. - -- **Switch to Real Devices :** If you need to perform tests on physical devices, click on the Switch to Real Devices link to access TestMu AI's real device cloud. - -- **Tunnel :** For testing on environments that are not publicly accessible, configure the Tunnel settings to enable testing on staging or local sites. - -- **Start Button :** Located on the right side of the URL field, this button initializes the testing session once you have chosen your device and browser configuration. diff --git a/static/docs/getting-started-with-selenium-4.md b/static/docs/getting-started-with-selenium-4.md deleted file mode 100644 index 0189605fc..000000000 --- a/static/docs/getting-started-with-selenium-4.md +++ /dev/null @@ -1,57 +0,0 @@ -# Getting Started With Selenium 4 On TestMu AI Platform - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Introduction - -Now run automated tests with Selenium 4 using the TestMu AI Selenium Grid across 3000+ browsers and operating systems. With this TestMu AI and Selenium 4 integration, TestMu AI not only allows you to perform Selenium 4 testing on an online automation cloud, but also allows you to perform cross-browser testing at scale. - -This Selenium 4 documentation will act as your step-by-step guide on your journey to test using Selenium 4 on TestMu AI. - ->**Note**: As of now, the sample test script is only available in Java. But worry not! We are working on adding the test scripts on other languages too. - -## Running Your First Test - -As a part of this step-by-step guide, we will run a Selenium 4 test that perform the following action: - -1. Open **google.com.** -2. Enter a search string TestMu AI in the browser. -3. Click the **Google Search** button. -4. Assert the page URL. -5. Mark the status of the test as either passed or failed based on the assertion. - -### Step 1 - OS-device-browser Combination - -To select the OS-device-browser combination for running the test, use TestMu AI's [capability generator](https://www.testmuai.com/capabilities-generator/). After selecting the combination, replace the capabilities given in the sample test script with the chosen combination. - -### Step 2 - Understanding the Sample Script - -As the sample test script is only available in Java, we recommend using it to run your first Selenium 4 test on TestMu AI. The offered test script uses the updated Action class features and the relative locators of Selenium 4. - -**Relative Locators** helps users locate any complex UI elements, elements that might be relevant to a critical UI element, in fewer lines of code. Selenium 4 has the following relative locators: `above`, `below`, `toLeftOf`, `toRightOf`, and `near`. For the sake of this test, the script uses the `toLeftof` locator to locate the Google Search button on the Google home page. - -Some other use cases for Relative Locators are: - -- In the case of an e-commerce website, relative locators can help identify a particular button if there are multiple buttons with the same name on a single website page. -- UI elements that do not include unique IDs -- UI elements with complex CSS or XPath expression. - -Selenium 4 comes with a changed syntax of the **Action class**. It is used for handling mouse and keyboard events on a specific element. For this test, the script uses the `moveToElement(onElement).click()` method to simulate the click action of your mouse. - -Along with these features, Selenium 4 offers other CDP features that TestMu AI supports. Check them out now. - -### Step 3 - Writing the Sample Test Script - -In your Java project, create a new file JavaSample.java and copy the following code to this file: - ->**Note**: As of now, the sample test script is only available in Java. But worry not! We are working on adding the test scripts on other languages too. - ->**Warning**: Ensure that you include the driver.quit() statement at the end of the test script to avoid timeout issues. If you skip adding the statement, the test continues to run leading to a timeout. - -### Step 4 - Run the Sample Test Code - -You can run the file using the Run command in your IDE after the test script is created. - -### Step 5 - View the Test Results - -After running the test successfully, the dashboard and the command line will view your result. In addition, you get access to text logs, video recordings ,and screenshots of your test execution. diff --git a/static/docs/getting-started-with-testmu-automation.md b/static/docs/getting-started-with-testmu-automation.md deleted file mode 100644 index df6c530b5..000000000 --- a/static/docs/getting-started-with-testmu-automation.md +++ /dev/null @@ -1,45 +0,0 @@ -# Get Started With Selenium Testing - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Run your Selenium tests on TestMu AI's cloud grid of 3000+ browser and OS combinations. Point your tests at the hub URL, add capabilities, and get parallel execution, video recordings, logs, and screenshots for every session - no infrastructure to maintain. - -## Before You Begin - -1. You need a TestMu AI account. [Sign up free](https://www.testmuai.com/register/) if you don't have one. -2. Visit the [TestMu AI Dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard) and navigate to the left sidebar and select **Credentials**. Your **Username** and **Access Key** will be visible there, along with options to copy them for use. -3. Set them as environment variables: - -```bash -export LT_USERNAME="YOUR_USERNAME" -export LT_ACCESS_KEY="YOUR_ACCESS_KEY" -``` - -```powershell -set LT_USERNAME="YOUR_USERNAME" -set LT_ACCESS_KEY="YOUR_ACCESS_KEY" -``` - -Your tests connect to the grid at: - -``` -https://{username}:{accessKey}@hub.lambdatest.com/wd/hub -``` - -## Choose How to Start - -### Run Your First Test - -New to cloud testing? Clone a sample project, set credentials, and run your first Selenium script on the grid. - -### Migrate Your Test Suite - -Already have Selenium tests? Change the hub URL, add credentials, and run them on the cloud. - -### Explore the Dashboard - -See where test results live - video playback, command logs, screenshots, and analytics. - -### Configure Capabilities - -Control which browser, version, OS, and advanced features your tests use. diff --git a/static/docs/getting-started-with-xcuitest.md b/static/docs/getting-started-with-xcuitest.md deleted file mode 100644 index 7bc5fe726..000000000 --- a/static/docs/getting-started-with-xcuitest.md +++ /dev/null @@ -1,201 +0,0 @@ -# XCUI Testing on TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Developed by Apple, XCUITest is a framework for user-interface (UI) testing for iOS applications. It is built on top of XCTest, an integrated test framework in Apple's Xcode IDE. TestMu AI lets you perform automated app testing of your iOS apps using XCUITest across 10000+ real devices and OS combinations. - -## Prerequisites - -- Your TestMu AI [Username and Access key](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/security). -- Access to an **iOS** app (.ipa) and an **XCUI Test** app (.ipa file). - -If you do not have any **iOS** app (.ipa) and an **XCUI Test** app (.ipa) file, you can run your sample tests on TestMu AI by using our sample :link: [iOS app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_ios.ipa) and a sample :link: [XCUI Test](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_ios_xcuitest.ipa). - -## Running Your First Test: A Step-by-Step Guide - -### Step 1: Upload Your Application - -To begin testing, upload your iOS application (.ipa file) to TestMu AI's servers. You'll use our **REST API** for this process. - -- **Authentication :** You'll need your TestMu AI Username and AccessKey. Combine them in the format `Username:AccessKey`. -- **Uploading the App :** Use **cURL command** to send a request to our API. The request should include the path to your application file (**appFile**). - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" --location --request POST 'https://manual-api.lambdatest.com/app/uploadFramework' --form 'appFile=@""' --form 'type="xcuit-ios"'`} - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/uploadFramework" -F "appFile=@""" -F "type="xcuit-ios""`} - -- Provide the path of your iOS application in the above URL in place of `` -- Response of above cURL will be a **JSON** object containing the `App URL` of the format - `lt://APP123456789123456789` and will be used in the last step. - -### Step 2: Upload Your Test Suite - -Upload your iOS test suite (.ipa) file to TestMu AI servers using our REST API. - -The following sample cURL command shows how to upload a test suite: - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" --location --request POST 'https://manual-api.lambdatest.com/app/uploadFramework' --form 'appFile=@""' --form 'type="xcuit-ios"'`} - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" --location --request POST "https://manual-api.lambdatest.com/app/uploadFramework" --form "appFile=@""" --form "type=\"xcuit-ios\""`} - -- Provide the path of your iOS test suite in the above URL in place of `` -- Response of above cURL will be a **JSON** object containing the `App URL` of the format - `lt://APP123456789123456789` and will be used in the last step. - -### Step 3: Executing The Test - -- You will need **base64 encoded authentication** in order to execute your XCUITest automation test suite. Enter your `username:accesskey` in **[Basic Authentication Header Generator](https://mixedanalytics.com/knowledge-base/api-connector-encode-credentials-to-base-64/)** to generate your auth token. - -Take note of the base64 encoded authentication which needs to be added in the next step. - -{`${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}`} - -- Once you have uploaded your app and test suite, you can execute your test by running the following command: - -* On Windows, run the following `curl` command from **Command Prompt (cmd.exe)**. The command is formatted for the Windows CMD terminal and may require different syntax if executed from PowerShell or another shell. -* Before running the command, replace `BASIC_AUTH_TOKEN`, `APP_ID`, and `TEST_SUITE_ID` with your actual LambdaTest credentials and resource IDs obtained from the above curl commands. - -```bash -curl --location --request POST 'https://mobile-api.lambdatest.com/framework/v1/xcui/build' \ ---header 'Authorization: Basic BASIC_AUTH_TOKEN' \ ---header 'Content-Type: application/json' \ ---data-raw '{ -"app": "APP_ID", -"testSuite": "TEST_SUITE_ID", -"device": ["iPhone 11-14"], -"video": true, -"queueTimeout": 10800, -"idleTimeout": 150, -"devicelog": true, -"network": false, -"build": "Proverbial-XCUITest" -}' -``` - -```bash -curl --location --request POST "https://mobile-api.lambdatest.com/framework/v1/xcui/build" ^ ---header "Authorization: Basic BASIC_AUTH_TOKEN" ^ ---header "Content-Type: application/json" ^ ---data "{\"app\":\"APP_ID\",\"testSuite\":\"TEST_SUITE_ID\",\"device\":[\"iPhone 15-17\"],\"video\":true,\"queueTimeout\":10800,\"idleTimeout\":150,\"devicelog\":true,\"network\":false,\"build\":\"Proverbial-XCUITest\"}" -``` - -```bash -curl --location --request POST 'https://mobile-api.lambdatest.com/framework/v1/xcui/build' \ ---header 'Authorization: Basic BASIC_AUTH_TOKEN' \ ---header 'Content-Type: application/json' \ ---data-raw '{ -"app": "APP_ID", -"testSuite": "TEST_SUITE_ID", -"device": ["iPhone 11-14.0"], -"isVirtualDevice": true, -"video": true, -"queueTimeout": 10800, -"idleTimeout": 150, -"devicelog": true, -"network": false, -"build": "Proverbial-XCUITest" -}' -``` - -```bash -curl --location --request POST "https://mobile-api.lambdatest.com/framework/v1/xcui/build" ^ ---header "Authorization: Basic BASIC_AUTH_TOKEN" ^ ---header "Content-Type: application/json" ^ ---data "{\"app\":\"APP_ID\",\"testSuite\":\"TEST_SUITE_ID\",\"device\":[\"iPhone 15-17.0\"],\"isVirtualDevice\":true,\"video\":true,\"queueTimeout\":10800,\"idleTimeout\":150,\"devicelog\":true,\"network\":false,\"build\":\"Proverbial-XCUITest\"}" -``` - -### Step 4: View Test Execution - -Once you have run your tests, you can view the test execution along with logs. You will be able to see the test cases passing or failing. You can view the same at TestMu AI Automation. - -## Running Tests in Parallel - -You can run tests in parallel on multiple devices by passing the device name in comma separated format in the execute command as show below: - -* On Windows, run the following `curl` command from **Command Prompt (cmd.exe)**. The command is formatted for the Windows CMD terminal and may require different syntax if executed from PowerShell or another shell. -* Before running the command, replace `BASIC_AUTH_TOKEN`, `APP_ID`, and `TEST_SUITE_ID` with your actual LambdaTest credentials and resource IDs obtained from the above curl commands. - -```bash -curl --location --request POST 'https://mobile-api.lambdatest.com/framework/v1/xcui/build' \ ---header 'Authorization: Basic BASIC_AUTH_TOKEN' \ ---header 'Content-Type: application/json' \ ---data-raw '{ -"app": "APP_ID", -"testSuite": "TEST_SUITE_ID", -"device": ["iPhone 11-14","iPhone 12 Pro-15","iPhone X-13"], -"video": true, -"queueTimeout": 10800, -"idleTimeout": 150, -"devicelog": true, -"network": false, -"build": "Proverbial-XCUITest" -}' -``` - -```bash -curl --location --request POST "https://mobile-api.lambdatest.com/framework/v1/xcui/build" ^ ---header "Authorization: Basic BASIC_AUTH_TOKEN" ^ ---header "Content-Type: application/json" ^ ---data "{\"app\":\"APP_ID\",\"testSuite\":\"TEST_SUITE_ID\",\"device\":[\"iPhone 11-14\",\"iPhone 12 Pro-15\",\"iPhone X-13\"],\"video\":true,\"queueTimeout\":10800,\"idleTimeout\":150,\"devicelog\":true,\"network\":false,\"build\":\"Proverbial-XCUITest\"}" -``` - -```bash -curl --location --request POST 'https://mobile-api.lambdatest.com/framework/v1/xcui/build' \ ---header 'Authorization: Basic BASIC_AUTH_TOKEN' \ ---header 'Content-Type: application/json' \ ---data-raw '{ -"app": "APP_ID", -"testSuite": "TEST_SUITE_ID", -"device": ["iPhone 11-14.0","iPhone 12 Pro-15.0","iPhone X-13.0"], -"isVirtualDevice": true, -"video": true, -"queueTimeout": 10800, -"idleTimeout": 150, -"devicelog": true, -"network": false, -"build": "Proverbial-XCUITest" -}' -``` - -```bash -curl --location --request POST "https://mobile-api.lambdatest.com/framework/v1/xcui/build" ^ ---header "Authorization: Basic BASIC_AUTH_TOKEN" ^ ---header "Content-Type: application/json" ^ ---data "{\"app\":\"APP_ID\",\"testSuite\":\"TEST_SUITE_ID\",\"device\":[\"iPhone 11-14.0\",\"iPhone 12 Pro-15.0\",\"iPhone X-13.0\"],\"video\":true,\"queueTimeout\":10800,\"isVirtualDevice\":true,\"idleTimeout\":150,\"devicelog\":true,\"network\":false,\"build\":\"Proverbial-XCUITest\"}" -``` - -## Using the XCUITest Agent Skill with TestMu AI - -The [xcuitest-skill](https://github.com/LambdaTest/agent-skills/tree/main/xcuitest-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The xcuitest-skill package includes: - -``` -xcuitest-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing XCUITest Agent Skill - -Install a XCUITest Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/xcuitest-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/xcuitest-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only xcuitest-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). diff --git a/static/docs/ghost-inspector-integration-with-testmu.md b/static/docs/ghost-inspector-integration-with-testmu.md deleted file mode 100644 index 42d1d3493..000000000 --- a/static/docs/ghost-inspector-integration-with-testmu.md +++ /dev/null @@ -1,100 +0,0 @@ -# Ghost Inspector Integration With TestMu AI’s Cloud - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI provides support with Selenium WebDriver for pacing the execution of your automation test scripts. TestMu AI is a cloud-based, cross browser testing tool, providing a [Selenium Grid](https://www.testmuai.com/selenium-automation/) of 3000+ browsers and browser versions running through real operating systems to speed up automation testing of your web-app or website. This topic will help you to automate your website testing using Ghost Inspector on TestMu AI. - -## What is Ghost Inspector? - -Ghost Inspector is a codeless automation tool that lets a software tester automate the process of writing test scripts. - -This integration between TestMu AI and Ghost Inspector enables you to automate the cross-browser testing process through the Selenium Grid of more than 2,000 browsers. - -As a result, you’ll be able to expand your test coverage, analyze and extract insightful test reports, and enhance your team collaboration. - -# How To Integrate TestMu AI With Ghost Inspector? - -* All the test suites will be reflected in the Ghost Inspector dashboard. - -* Click on the desired test suite. -* Click on the desired test case under test suites. - -* Once you have selected the desired test case, click on **More** -> **Export Suite** and select the format as New Selenium IDE (.side). - -### Setup Selenium IDE Side Runner: - -* Node.js programming language (version 8 or 10). -* NPM or Node.js Package Manager. -* Selenium IDE Plugin, for either Google Chrome and Mozilla Firefox. -* Selenium IDE (SIDE) Runner. You can download SIDE runner using below code: - -```bash -npm install -g selenium-side-runner -``` -* You also need your TestMu AI authentication credentials, which includes your TestMu AI username, and access key. These will be set up as the Secrets for configuring the pipeline. To get your Username and Access Key, go to your TestMu AI automation dashboard and click on the "key" icon left to the help button, on the top right corner. - -* Open Selenium IDE and click on ‘Open an existing project'. - -* Right click on the test suite and go to Settings. Test Suite Settings. - -* In the Settings window, tick the Run in Parallel checkbox. Test Suite Properties. - -* Save the SIDE project. -* We have successfully activated Parallel Testing in Selenium IDE. Now we can execute tests Selenium IDE tests in parallel on TestMu AI Selenium Cloud Grid. - -## Executing Selenium IDE Tests on TestMu AI Selenium Cloud Grid - -* Set the Desired Capabilities in Selenium. For this article, we have set the Chrome browser (version 72.0) for Windows 10, to run our Selenium IDE tests. -* The TestMu AI Selenium Desired Capabilities Generator is used to generate the capabilities. Below are the desired capabilities used in this article for the required test combination, provided in Python Language. - -```bash -capabilities = { -"platform" : "Windows 10", -"browserName" : "Chrome", -"version" : "72.0" -} -``` - -In the string format, it will be represented as: -```bash -**"browserName='chrome' version='72.0' platform='Windows 10'"** -``` - -* Get the Grid URL for executing tests on TestMu AI. Get your username and access key from your TestMu AI's Profile section (as shown in the prerequisites), and replace it in the below URL. - -```bash -https://user-name:access-key@hub.lambdatest.com/wd/hub -``` - -* Specify the desired capabilities along with the Grid URL. Replace the desired capabilities and the Grid URL from the above steps in the below code: - -```bash -selenium-side-runner --server --capabilities -``` -* Mention the number of parallel processes to be executed at a time. You can use any of the below syntax to specify the count of parallel processes to be executed at the above mentioned Grid URL: - -```bash -selenium-side-runner --w --server -https://user-name:access-key@hub.lambdatest.com/wd/hub -``` - -OR - -```bash -selenium-side-runner --max-workers --server -https://user-name:access-key@hub.lambdatest.com/wd/hub -``` - -For this article, we have set the number of parallel processes as '4'. - -* Now we have all the components of the SIDE Runner command to execute the tests. So we will run the Selenium IDE tests on TestMu AI Selenium Cloud Grid. Navigate to the directory where the .SIDE file is located (here it is located at "TestMu AI-IDE.side") and execute the below command in the terminal, after replacing the GRID URL: - -```bash -selenium-side-runner -w 4 "LambdaTest-IDE.side" --server -https://user-name:acces-key@hub.lambdatest.com/wd/hub -c "browserName='chrome' -version='72.0' platform='Windows 10'" -``` - -* We have successfully executed all the steps to run Selenium IDE tests on TestMu AI Selenium Cloud Grid. Now you can see our tests being executed along with their status in the Automation Timeline of your TestMu AI profile. - -* Below are the execution snapshots of the SIDE Runner and the automation timeline instance, that shows the successful execution of the desired tests: test execution snapshots automation test timeline. diff --git a/static/docs/github-actions-with-hyperexecute.md b/static/docs/github-actions-with-hyperexecute.md deleted file mode 100644 index e274cb322..000000000 --- a/static/docs/github-actions-with-hyperexecute.md +++ /dev/null @@ -1,84 +0,0 @@ -# GitHub Actions Pipeline Integration with Hyperexecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -GitHub Actions is a continuous integration and continuous delivery (CI/CD) platform that allows you to automate your build, test, and deployment pipeline. You can create workflows that build and test every pull request to your repository, or deploy merged pull requests to production. - -This document will show you how to integrate GitHub Actions Pipeline with HyperExecute to greatly shorten your test cycles. - -## How To Integrate GitHub Actions Pipeline with Hyperexecute - -To integrate GitHub Actions Pipeline with HyperExecute, follow the below steps: - -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** -Download or Clone the code sample from the TestMu AI GitHub repository to run the tests on the HyperExecute. - - View on GitHub - -### 1. Log into your GitHub account -- Navigate to the main page of the repository. -- Under your repository name, click **Actions**. - -### 2. Create a New Workflow -In the left sidebar, click the **New workflow** button. - -### 3. Create the GitHub Actions work-flow YAML file: - -To create the GitHub Actions pipeline YAML file, follow the sample command below: - -```bash -name: HyperExecute -on: -workflow_dispatch: -inputs: -username: -required: true -description: LT Username -accessKey: -description: LT Access Key -required: true - -jobs: -HyperExecute: -runs-on: ubuntu-latest -timeout-minutes: 15 -strategy: -fail-fast: false -steps: -- name: Checkout sources -uses: actions/checkout@v2 - -- name: Download CLI and Setting Environment Variables -shell: bash -run: | -curl https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute -o hyperexecute -chmod +x hyperexecute - -- name: Executing the Job -shell: bash -run: | -./hyperexecute --user ${{ github.event.inputs.username }} --key ${{ github.event.inputs.accessKey }} --download-artifacts --config yaml/autosplit_linux.yaml -echo "Test completion" -``` -- **On:** - - **Workflow_dispatch:** In the **workflow_dispatch** section, you should declare the pre-defined variables that will be used before running the GitHub Actions PipeLine as an input. -- **Jobs:** - In the Jobs Section, declare the workflow of the pipeline execution. - - **runs-on**: Runs-on contains the value of the OS flavor you would like to execute the GitHub Actions Pipeline on. -- **Steps**: -In the steps section, you should declare the execution commands. - - The first step in the above sample YAML changes the path of the root directory. - - In the second step it downloads the HyperExecute CLI binary. - - The third step is the execution command which executes The Hyperexecute CLI binary. This contains TestMu AI username, access key, and path of the Yaml created for Hyperexecute. You can find more information on this [here.](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) - -### 4. Run the Workflow -To run the new pipeline that you just created, click the **Run workflow** button on the workflow page. - -**Below is an example of how a test that is run on the GitHub Actions pipeline gets executed:** - -**Below is an example of a Hyperexecute job that was triggered through the above pipeline:** - -> -**Run your tests at speeds never seen before. Happy testing! :)** diff --git a/static/docs/github-app-integration.md b/static/docs/github-app-integration.md deleted file mode 100644 index 1404a3219..000000000 --- a/static/docs/github-app-integration.md +++ /dev/null @@ -1,313 +0,0 @@ -# TestMu AI GitHub App Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -**Your pull requests should deliver proof, not just code.** - -The TestMu AI Cloud GitHub App brings KaneAI, our AI testing agent, directly into your GitHub workflow. When a developer opens a pull request, KaneAI analyzes the changes, generates targeted test cases, executes them at scale on HyperExecute, and reports back with AI-powered Root Cause Analysis. The entire cycle runs in minutes, and every update appears right inside your PR. - -No context switching. No manual test authoring. No waiting. - -## Key Capabilities - -#### Automated AI Test Authoring - -Automatically generate test coverage for every pull request. KaneAI reads your code diff, PR description, README, and custom agent instructions to create automated tests that reflect actual business logic, not generic boilerplate. - -#### Smart Intelligence - -KaneAI scans your existing test inventory in TestMu AI Test Manager to find semantically similar test cases relevant to the PR changes. These existing tests are added alongside the newly generated test cases into the test run and executed on HyperExecute, maximizing test coverage by combining new and proven tests in a single validation cycle. - -#### Seamless Execution - -Tests run automatically and seamlessly on TestMu AI's HyperExecute infrastructure, in parallel across browsers, devices, and platforms. Time to first signal is measured in minutes, not hours. - -#### In-Depth Insights - -Detailed execution reports enriched with AI-powered Root Cause Analysis, offering actionable recommendations. When tests fail, KaneAI analyzes logs, screenshots, network traces, and stack traces to surface the specific root cause, directly in your PR comment. - -## Prerequisites - -Before implementing the TestMu AI Cloud GitHub App in your development workflow, ensure your organization has the following: - -#### Account Requirements - -- **TestMu AI Account**: An active TestMu AI account with appropriate licensing. Sign up now if you don't have an account yet. - -- **KaneAI Access**: This integration requires KaneAI to be enabled on your TestMu AI account. New signups can start a free trial to explore KaneAI's key workflows. - -Start KaneAI Free Trial → - -#### Repository Requirements - -- **GitHub Repository Access**: Administrative access to the GitHub repositories where you want to install the app -- **Repository README** *(recommended)*: A descriptive `README.md` in your repository improves test quality. KaneAI uses it to understand your application's purpose, architecture, and key user workflows. - -## Installation - -Follow these steps to install and authorize the TestMu AI Cloud GitHub App for your organization. - -#### Step 1: Access GitHub Marketplace - -Navigate to the [TestMu AI Cloud GitHub App](https://github.com/apps/lambdatest-ai-cloud) on GitHub Marketplace and click **Install** to begin the installation process. - -#### Step 2: Configure Repository Access - -During installation, you'll need to specify which repositories should have access to the app: - -- **Organization-wide Installation**: Select **All repositories** to enable the app across your entire GitHub organization -- **Selective Installation**: Choose **Only select repositories** and specify individual repositories for more granular control - -After making your selection and clicking the **Install and Authorize** button, you will be redirected to TestMu AI where you will be asked to login. - -Once done, you will be redirected to the Integrations page. - -**Best Practice** -For pilot programs or initial rollouts, we recommend installing on a select subset of repositories first. Once your team is comfortable with the workflow, you can expand access organization-wide. - -#### Step 3: Verify Successful Installation - -Confirm the installation was successful by: - -1. Navigating to your GitHub organization's **Settings** page -2. In left side menu under **Integrations**, click **Applications**. -3. Selecting **Installed GitHub Apps** from the left sidebar -4. Verifying that **TestMu AI Cloud** appears in the list of installed applications - -You can also click on the app to review and modify repository access permissions at any time. - -#### Uninstalling the GitHub App - -If you need to remove the TestMu AI Cloud GitHub App from your organization: - -1. Navigate to the [TestMu AI Cloud GitHub App](https://github.com/apps/lambdatest-ai-cloud) on GitHub Marketplace -2. Click on the **Configure** button to access the app settings -3. Scroll down to the bottom of the page to find the **Danger Zone** section -4. Click on the **Uninstall** button to remove the app from your organization -5. Confirm the uninstallation when prompted - -Uninstalling the GitHub App will stop all AI-powered test generation workflows on your pull requests. This action cannot be undone, and you'll need to reinstall the app to restore functionality. - -## Repository Configuration - -After installing the GitHub App, each repository requires a configuration file to connect your GitHub workflow with your TestMu AI Test Manager environment. This configuration defines where tests should be stored, who should be assigned, and which environment to test against. - -#### Configuration File Setup - -Create a `.lambdatest/config.yaml` file in the root directory of your repository with the following structure: - -```yaml -# LambdaTest AI Cloud Configuration -project_id: "your_project_id" -folder_id: "your_folder_id" -assignee: your_user_id -configuration_id: configuration_id -test_url: "https://your-deployed-app-url.com/" -tunnel_name: "your_tunnel_name" # Optional: set if using the same tunnel across PRs -``` - -#### Configuration Parameters - -| Parameter | Description | -|-----------|-------------| -| `project_id` | The unique identifier for your TestMu AI Test Manager project | -| `folder_id` | The folder where generated test cases will be organized | -| `assignee` | The TestMu AI user ID who will be assigned to test runs for executions | -| `configuration_id` | The target testing environment (browser, OS, device configurations) | -| `test_url` | The base URL of your application under test (your staging or testing environment URL) | -| `tunnel_name` | *(Optional)* The name of the LambdaTest tunnel to use for testing. Set this when the same tunnel is reused across all PRs. Can be overridden per PR using the `--tunnel` flag in the trigger comment. | - -After installing the GitHub App, you are redirected to the [integration settings page](https://integrations.lambdatest.com/githubci/install) where all configuration values (project ID, folder ID, assignee, and environment ID) are displayed with a **copy button**. Use these to populate your `.lambdatest/config.yaml` file directly. - -> **Note:** All configuration IDs can also be retrieved programmatically from the [TestMu AI Test Manager API Documentation](https://www.testmuai.com/support/api-doc/?key=test-management). - -**Maximize AI Context** -KaneAI uses your `README.md` and `agent.md` to improve test quality. Keep your README comprehensive (app overview, key workflows, tech stack) and use `agent.md` for custom instructions like testing priorities, scenarios to cover or skip, and domain-specific rules. - -#### Repository Structure - -Your final repository structure should look like this: - -``` -your-repo/ -├── .lambdatest/ -│ └── config.yaml # LambdaTest configuration -└── agent.md # Optional file for custom instructions to enhance responses -├── src/ # Your application source code -├── README.md # Detailed project documentation (used by AI) -└── ... other project files -``` - -#### Configuration Best Practices - -- **Environment Segregation**: Use separate `project_id` and `folder_id` values for different branches (e.g., staging vs. production) to maintain test organization -- **Team Assignment**: Configure `assignee` to route test runs to the appropriate QA team member or use a shared team account for visibility -- **Dynamic URLs**: For teams with ephemeral preview environments, consider parameterizing `test_url` or updating it per deployment -- **Version Control**: Commit `.lambdatest/config.yaml` to your repository so all team members use consistent configuration - -## Try It Out: Sample Repository - -Explore the GitHub App workflow using our public sample repository: - -**[ai-native-quality-validation-sample](https://github.com/LambdaTest/ai-native-quality-validation-sample)** - -Browse the open pull requests to see: -- AI-generated test cases -- Test execution results -- Root Cause Analysis comments - -#### Quick Start with Your Own Fork - -To experience the full workflow hands-on, fork the sample repository and run the GitHub App in your own environment: - -1. **Fork the repository**: Click **Fork** on the [sample repository](https://github.com/LambdaTest/ai-native-quality-validation-sample) to create a copy under your GitHub account. -2. **Install the GitHub App**: Install the [TestMu AI Cloud GitHub App](https://github.com/apps/lambdatest-ai-cloud) on your forked repository (see [Installation](#installation) above). -3. **Configure your credentials**: Add the `.lambdatest/config.yaml` file with your TestMu AI project ID, folder ID, and other configuration values (see [Repository Configuration](#repository-configuration)). -4. **Set up GitHub Pages & Actions**: Enable GitHub Pages and GitHub Actions in your forked repository to handle deployment and workflows. -5. **Trigger the workflow**: Open a pull request in your fork and comment `@TestMuAI Validate this PR` to see the full AI testing pipeline in action. - -## Triggering Test Generation - -With the GitHub App installed and your repository configured, you're ready to start generating AI-powered tests for your pull requests. - -#### Initiating the Workflow - -The test generation workflow is triggered through a simple comment on any pull request: - -1. **Create a Pull Request**: Push your feature branch and open a PR as you normally would. -2. **Add a Trigger Comment**: Post a command in the PR comments section. - -#### Available Commands - -| Command | Description | -|---------|-------------| -| `@TestMuAI Validate this PR` | Triggers the full AI testing workflow: analysis, generation, execution, and reporting | -| `@KaneAI Validate this PR` | Alias for the above | - -#### Optional Parameters - -You can extend any trigger command with optional parameters to customize test execution for a specific PR: - -| Parameter | Description | -|-----------|-------------| -| `--url ` | Overrides the `test_url` set in `.lambdatest/config.yaml` for this PR. Useful for ephemeral preview environments where the deployment URL changes per PR. | -| `--tunnel ` | Routes test traffic through a named LambdaTest tunnel. Required when testing against a locally hosted or privately accessible environment that is not publicly reachable. | - -**Example:** - -``` -@TestMuAI Validate this PR --url https://preview-123.your-app.com --tunnel my-tunnel-name -``` - -Both parameters are independent. Use `--url` alone to override the URL, `--tunnel` alone for private environments, or combine them when both apply. - -#### Setting Up Tunnel Testing - -When your application runs on a local machine, private network, or staging environment not accessible from the internet, use the `--tunnel` flag to route test traffic through a secure LambdaTest tunnel. - -**Before triggering the workflow with `--tunnel`:** - -1. Start a LambdaTest tunnel on the machine that can reach your application. The tunnel must be active at the time the workflow is triggered. -2. Note the tunnel name you assigned when starting the tunnel. -3. Pass that exact tunnel name as the `--tunnel` value in your PR comment. - -For complete setup instructions, refer to the [KaneAI Tunnel Support documentation](https://www.testmuai.com/support/docs/kane-ai-geolocation-tunnel-proxy/#tunnel-support). - -#### What Happens Next - -After you post the trigger comment, KaneAI immediately begins working: - -1. **Code Analysis**: The AI examines all code changes in the pull request, including file modifications, additions, and deletions. -2. **Context Gathering**: Your repository's README.md, PR title, description, comments, and AGENT.md are analyzed to understand application context. -3. **Test Strategy**: Based on the changes and context, the AI determines which areas require testing and what scenarios to cover. -4. **Test Generation**: Intelligent test cases are created with appropriate assertions, validations, and edge case handling. - -**Fast Feedback** -The first signal, test plan and progress tracker, appears in your PR within approximately **1 minute** of triggering the workflow. Full execution and reporting typically completes within minutes, depending on test volume and complexity. - -## Live PR Updates, Step by Step - -From the moment you trigger the workflow, KaneAI posts real-time updates directly in your pull request. Every phase, from analysis through execution to reporting, is visible without leaving GitHub. Here is what happens at each stage. - -#### Step 1: Real-Time Progress Tracker - -As soon as the workflow begins, KaneAI posts a comprehensive progress tracker comment to your PR. This dynamic comment serves as your single source of truth for the entire testing operation. - -**The Progress Tracker includes:** - -- **Current Workflow Status**: Real-time updates on which phase is currently executing -- **Test Case Pipeline**: Progress through analysis, generation, authoring, and code generation phases -- **Intelligent Test Case Suggestion**: AI identifies existing semantically similar test cases from your project and adds them to the test run alongside newly generated tests -- **Test Run Management**: Execution status, including configuration, triggering, monitoring, and completion -- **Reporting Status**: Final report generation and PR approval recommendation - -The tracker automatically updates as each stage completes, no manual refreshes required. - -#### Step 2: Automated AI Test Authoring - -Once KaneAI completes test generation, a detailed comment lists every test case that was created. These are not generic tests. They reflect the specific code changes in your PR and your application's business context. - -Each entry includes: - -- **Test Case ID**: A clickable link (e.g., TC-14822) that opens the test case directly in TestMu AI Test Manager. Any user in your organization with Test Manager access can open these links. -- **Status**: The current authoring state of each test case. When a test case is in **Authoring** status, the link redirects to the live KaneAI session where the test is being authored in real time. - -**Access Permissions** -- **Test Case links** (TC-XXXXX) can be accessed by any user in your organization who has TestMu AI Test Manager access. -- **Authoring status links** can only be accessed by the user who integrated the GitHub App, as they redirect to that user's live KaneAI session. - -This comment updates dynamically as test authoring progresses, so you can monitor the transition from conceptual test cases to executable automation. - -#### Step 3: Smart Intelligence (Similar Test Detection) - -KaneAI does not rely solely on newly generated tests. It scans your existing test inventory in TestMu AI Test Manager to find semantically similar test cases that are relevant to the PR changes. These existing tests are added on top of the newly generated test cases into the test run for execution on HyperExecute. This is **Smart Intelligence** at work. - -- **Maximizes coverage**: Combines AI-generated tests with proven existing test cases, ensuring both new and established scenarios are validated in a single run. -- **Leverages your test library**: Your team's existing test cases become active participants in every PR validation, not just historical records. -- **Builds institutional knowledge**: Every test run enriches the system's understanding of your project, making future test suggestions progressively more accurate. - -#### Step 4: Seamless Execution on HyperExecute - -When test execution begins, a dedicated comment provides live status updates directly in your PR: - -- **Test Run Configuration**: Execution environment, browser matrix, and parallel execution settings -- **Real-Time Execution Status**: Live updates as tests run, including pass/fail counts and completion percentage -- **HyperExecute Dashboard Link**: Direct access to detailed logs, screenshots, video recordings, and network traces - -Tests run in parallel across browsers, devices, and operating systems on TestMu AI's HyperExecute infrastructure, delivering results at scale without queuing delays. - -Test run link in this comment can be accessed by any user in your organization who has TestMu AI Test Manager access. - -#### Step 5: In-Depth Insights and Root Cause Analysis - -Upon completion, KaneAI generates a final report that goes far beyond simple pass/fail metrics: - -- **Executive Summary**: High-level test coverage and success rate for quick stakeholder review -- **Detailed Test Results**: Pass/fail status for each individual test case with failure details -- **PR Approval Recommendation**: Based on test results and failure severity, KaneAI suggests whether the PR should be approved, requires changes, or needs further investigation - -For any failing test, KaneAI performs automated **Root Cause Analysis (RCA)** by correlating: - -- Screenshots and video replay at the point of failure -- DOM state and selector changes -- Network request logs and timing -- Stack traces and error messages - -The RCA summary is posted directly in your PR, along with a clear recommendation: approve, request changes, or investigate further. Your team gets a readable diagnosis, not a raw stack dump. - -Test run link in this comment can be accessed by any user in your organization who has TestMu AI Test Manager access. - -## Why This Changes Your Workflow - -The TestMu AI Cloud GitHub App eliminates the gap between code change and validated confidence. Instead of treating testing as a separate phase that happens after development, every pull request becomes a self-validating artifact. - -- **Automated AI Test Authoring**: KaneAI handles test design, script generation, and execution. Your team reviews results, not writes boilerplate. -- **Smart Intelligence**: AI finds existing test cases semantically relevant to your PR changes and adds them alongside newly generated tests, maximizing coverage with every validation run. -- **Seamless Execution**: Tests run automatically inside your PR workflow. Developers, QA engineers, and stakeholders all see the same information in the same place, no context switching required. -- **In-Depth Insights**: AI Root Cause Analysis transforms cryptic test failures into actionable diagnoses with recommended fixes. - -Every test run, result, and AI recommendation is permanently recorded in your PR history, providing a complete audit trail for compliance and retrospective analysis. - -## Video Walkthrough - -Watch the video below for a complete walkthrough of the TestMu AI Cloud GitHub App in action, from installation to AI-powered test generation and reporting. diff --git a/static/docs/github-integration.md b/static/docs/github-integration.md deleted file mode 100644 index cf482f6b7..000000000 --- a/static/docs/github-integration.md +++ /dev/null @@ -1,92 +0,0 @@ -# GitHub Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -GitHub is one of the most popular community among software developers. It helps you with version control by providing a code hosting platform for all. From open source to business, one can share their codes, discover new ideas and discuss about roadblocks, all in order to deliver a better software. GitHub also serves as a project management tool. It allows a team to collaborate from all around the globe. - -The TestMu AI GitHub Integration allows you to create a card directly in your repository from TestMu AI platform. Push an issue to your respective repository anytime, even in the middle of your test session. The fields populated by you when marking as bug through TestMu AI are displayed as information on the bug in GitHub repository for that testing instance. - -**GitHub Integration with TestMu AI, like all of our other integrations to 3rd party applications, is available as freemium as well as premium plan.** - -## How To Integrate GitHub With Your TestMu AI Account? - -**Step 1:** Login to your TestMu AI account. You would need to have Admin or User level access to see and install integrations. - -**Step 2:** Select **Integrations** from the left navigation menu bar. This will guide you to a screen where you will find a list of 3rd party applications, available to integrate with your TestMu AI account. - -**Step 3:** Click on **ADD** under the block that says Github. - -**Step 4:** Step 4: Once you click on install, youll be redirected to a screen. There would be three options available for you to choose from in order to authenticate GitHub. - -### How To Integrate GitHub With Your TestMu AI Account Via OAuth? - -**Step 1**: Select the radio button on **OAuth** and click on **Install**. - -You will be redirected to GitHub login page where you need to provide your login credentials. - -**Step 2**: Enter your GitHub login credentials and click on **Sign in**. - -**Step 3**: Youll be redirected to the OAuth permission page. Click on **Authorize TestMu AI**. - -> -**What is OAuth?** -The purpose of OAuth is to bypass the authentication when two different applications interact using their respective APIs. GitHub APIs use OAuth 2.0s authorization code grant flow for generating access tokens on the users behalf. - -Thats it! Go to Integrations again and you will be able to notice a green tick indicating that GitHub is successfully installed. You can now experience bug logging in a fly from any of your running test sessions in TestMu AI to your repository directly with a single click. - -### How To Integrate GitHub With Your TestMu AI Account Via Cloud? - -**Step 1**: Select the radio button on **Cloud**. This will ask you to enter your API token. - -> -**What is an API Token?** -Access tokens are strings with an authorization key required to access an API. They are issued to the client-server and are usually opaque. They are used for requesting access to protected, user-specific resources. Access tokens are vital from a security point of view & can be generated in different formats, depending upon security requirements specified on the resource server. - -**Step 2**: Log in to your Github account and navigate to the **Settings**. - -**Step 3**: Click on the **Developer Settings** from the bottom-left. - -**Step 4**: Click on the **Personal access tokens** from the left menu and click on **Generate new token**. - -**Step 5**: Provide a token description, select the checkboxes for **repo** & **user** as scopes. These scopes will define the access to authenticate API over Basic OAuth for their respective personal token. After selecting the scopes click on **Generate token**. - -**Step 6**: Your personal access token will be generated. Copy the generated token & store it safely with you, as you will not be able to see this token again. - -**Step 7**: Paste the copied token into TestMu AI GitHub Authentication and click on **Install**. - -Go to Integrations again and you will be able to notice a green tick indicating that GitHub is successfully installed. You can now experience one-click bug logging from any of your running test sessions in TestMu AI to your GitHub repository directly with a single click. - -## Logging Your First Bug Through GitHub Integration - -**Step 1:** Go for any of the test from the left navigation menu. For demo, we will be taking **"Real Time Test"** option. - -**Step 2:** Present a URL of the web-app you need to test in the dialog box. After that, select any configuration for browser and operating system of your choice & hit **Start.** - -**Step 3**: After the VM is launched and operable. You can perform testing on your web-app for finding bugs. If a bug gets revealed, then you need to click on the **Bug icon** from the left panel for capturing a screenshot of the same. - -**Step 4:** After a screenshot is captured, you can annotate any issue or a task with an **in-built image editor.** Once you are done highlighting the bug, click on the button that says **"Mark as Bug".** - -**Step 5**: After clicking on **"Mark as Bug"** button a GitHub specific form would open up. Fill the fields as per your requirement. - - - You can select which repo should the observation be marked under. - - You can select a relevant label for the same UI observation. - - You can select the Issues title. - - You also get to post a description to help relate the cause of the issue or the card. - -**Step 6**: Click on **"Create Issue"** & observe it being successfully marked through a single click effort. Once you hit the button, you will get prompt messages on top of your Virtual Machine indicating the progress of bug logging. You will get a prompt message in few seconds mentioning **"Bug successfully marked"** indicating that the bug has been pushed to the desired GitHub repository. - -**Step 7:** Visit your GitHub repository. You will be able to notice the bug logged right away under your recent activity. - -Click on the **description** of bug and you will be redirected to the detail page of the bug. You will find all the data that you provided through TestMu AI already presented in the bug. TestMu AI automatically includes test environment details and related screenshots in the GitHub repository as attachments. - -## How To Remove GitHub Integration? - -You can work with one integration at a time. So if you would want to integrate to some 3rd party application of the similar type, other than the GitHub then you would have to remove your present integration. Here is how you can do it. - -**Step 1:** Login to your TestMu AI account. - -**Step 2:** Select **Integrations** from the left navigation menu bar. This will guide you to the same screen from where you initiated GitHub installation. You will also find other third party applications, available to integrate with your TestMu AI account. - -**Step 3:** To remove the GitHub integration, click on **REMOVE** under the category My Integrations. - -That was all you need to know for TestMu AI + GitHub Integration. Increase your productivity with our integrations. If you still have any questions for us, please feel free to let us know. Our experts are always available on available on **chat** to help you out with any roadblock regarding our product. **Happy testing!** diff --git a/static/docs/github-repo-based-oauth-integration.md b/static/docs/github-repo-based-oauth-integration.md deleted file mode 100644 index 502f5937a..000000000 --- a/static/docs/github-repo-based-oauth-integration.md +++ /dev/null @@ -1,33 +0,0 @@ -# GitHub Repo-Based OAuth Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI allows you to integrate with GitHub using repo-based OAuth to log issues directly to your GitHub repository from the TestMu AI platform. - -## Integrating Your GitHub Account Using Repo-Based OAuth - -1. Log in to your TestMu AI account. Ensure that you have either admin or user level access to view and install third-party integrations. - -2. Navigate to **Integrations** from the left-sidebar under **Settings**. - -3. Click on Project Management, then click **Connect** on the GitHub card. - -4. Select instance as **Repo-based OAuth**. - -5. Click **Install** to install the TestMu AI App. - -6. Choose the installation location. Instead of granting access to all GitHub repositories, you also have the option to select a specific repository for installing the TestMu AI App. - -After selecting the installation location, click **Request** to get the approval for the repository authorization. - -8. Once your installation request is approved, navigate back to the GitHub setup dashboard and click **Authorize**. - -9. Click **Continue** to authorize the GitHub repository. - -10. After you complete the authorization, GitHub will install the TestMu AI App. - -11. Navigate back to the Integrations console and you'll notice your TestMu AI account will be integrated with GitHub using repository-based OAuth authentication. - -You can now start testing your website or mobile app testing and log bugs directly from the TestMu AI platform to your GitHub repository. - -Looking to get started? Check out this documentation on [real-time testing on desktop browsers](/support/docs/getting-started-with-desktop-browser-real-time-testing/). diff --git a/static/docs/github-with-hyperexecute.md b/static/docs/github-with-hyperexecute.md deleted file mode 100644 index 6096fdc7f..000000000 --- a/static/docs/github-with-hyperexecute.md +++ /dev/null @@ -1,65 +0,0 @@ -# GitHub CI Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -GitHub is one of the best solutions for supporting continuous integration for Git projects for clients of all sizes. Now, quickly deploy your code and extensively test it to ensure a reliable delivery for your easy release. - -This document will show you how to use HyperExecute Integration with GitHub Continuous Integration. We will use the WebdriverIO framework to execute our automation script. - -## Prerequisites For Integrating GitHub CI With HyperExecute - -To perform your WebdriverIO tests with HyperExecute, you would need the following essentials to be already set up. - -1. **Global Dependencies** - -* A Git or GitHub repository. -* Download and install node.js and node package manager or npm. -* To install node.js with homebrew use the command below. - -``` -$ brew install node -``` - -* If you already have npm set up, you might want to update to the most recent version. To upgrade npm, run the following code in your terminal. - -``` -npm install npm@latest –g -``` - -* Install WebdriverIO with the npm command below. - -``` -npm install webdriverio -``` - -2. **TestMu AI Authentication Credentials** - -Keep your TestMu AI login, access key, and HubURL handy for authentication purposes. They must be configured as environment variables. You can find them on your TestMu AI automation dashboard by clicking on the key symbol next to the help button. - -* **For Linux/Mac** - -``` -$ export LT_USERNAME= -$ export LT_ACCESS_KEY= -``` - -* **For Windows** - -``` -$ set LT_ACCESS_KEY= -$ set LT_ACCESS_KEY= -``` - -## Integrating HyperExecute With GitHub CI - -To integrate GitHub CI with HyperExecute, you need to make use of the Hyperexecute CLI. HyperExecute CLI is a command line client to interact and run tests on HyperExecute and provides a host of other useful features that accelerate test execution. - -For triggering tests using HyperExecute CLI, you have to download the HyperExecute CLI binary corresponding to the platform (or OS) from where the tests are triggered: Below are the HyperExecute CLI download links for different operating systems: - -| OS | HyperExecute CLI download link | -| ---------| --------------------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| macOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -HyperExecute CLI provides different command-line flags that let you customize the HyperExecute experience. Once you download the HyperExecute CLI in your machine, run the command *hyperexecute.exe --help* on the terminal to explore the different flags. diff --git a/static/docs/gitlab-ci-integration-with-testmu.md b/static/docs/gitlab-ci-integration-with-testmu.md deleted file mode 100644 index 744756564..000000000 --- a/static/docs/gitlab-ci-integration-with-testmu.md +++ /dev/null @@ -1,177 +0,0 @@ -# GitLab CI Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -GitLab is one of the finest tools which facilitates continuous integration for Git projects from small to enterprise level clients. Now, deploy your code rapidly and test it thoroughly for a robust delivery in your go-to-market launch. - -TestMu AI now integrates with GitLab CI to boost your go-to-market delivery. Perform automated cross browser testing with TestMu AI to ensure your development code renders seamlessly through an online Selenium grid providing 3000+ real browsers running through machines, on our cloud servers. Perform automation testing in parallel with TestMu AI’s Selenium grid to drastically trim down your test cycles. - -This topic will demonstrate how you can leverage TestMu AI Integration with GitLab CI. We will be running our automation script using the Protractor framework. - -## Prerequisites For Integrating GitLab CI With Our Online Selenium Grid - -In order to perform your protractor tests with TestMu AI, you would need the below things to be already set up. - -1. **Global Dependencies** - -* A Git or GitHub repository. -* Download and install node.js and node package manager or npm. -* To install node.js with homebrew use the command below. - -``` -$ brew install node -``` - -* If you have npm already installed, you may want to upgrade it to latest version. Here the code you can run in your terminal to upgrade npm. -* Upgrade npm to latest version using the below command. - -``` -npm install npm@latest –g -``` - -* Install protractor using npm with the below command. - -``` -npm install –g protractor -``` - -2. **TestMu AI Authentication Credential** - -Be aware of your TestMu AI authentication credentials i.e. your TestMu AI username, access key and HubURL. You need to set them up as your environment variables. You can retrieve them from your TestMu AI automation dashboard by clicking on the key icon near the help button. - -* **For Linux/Mac** - -``` -$ export LT_USERNAME= -$ export LT_ACCESS_KEY= -``` - -* **For Windows** - -``` -$ set LT_ACCESS_KEY= -$ set LT_ACCESS_KEY= -``` - -## Integrating TestMu AI With GitLab CI - -To integrate GitLab CI with TestMu AI you need to make minor tweaks in the .gitlab-ci.yml which is the configuration file for your GitLab CI instance. The changes you need to make will revolve around the environment variables such as access key, username, grid config and so on. - -Below is a sample config file for integrating GitLab CI with TestMu AI. You can use the sample file to test your own automation test suite by replacing the environment variables according to your need. All the code used for the demonstration of this topic can be find in our [GitHub repository](https://github.com/LambdaTest/protractor-selenium-sample-gitlab). - -``` -//Sample Configuration For GitLab CI - -# Choose language and version -image: node:7.10 - -# setup required installers -before_script: -- | -cd ~/protractor-selenium-sample-gitlab -echo 'export LT_USERNAME="{your_lambdatest_username}"' >> ~/.bashrc -echo 'export LT_ACCESS_KEY="{your_lambda_access_key}"' >> ~/.bashrc -npm install -npm install -g protractor - -job: -script: -#Running test! -protractor single.conf.js -``` - -TestMu AI offers a [Selenium Desired Capabilities Generator](https://www.testmuai.com/capabilities-generator/) to fetch coded values for your desired test configurations. For Instance, if you select the below configurations: - -| FIELDS | SELECTED VALUES | -| ------- | -------------- | -| Operating System | Windows 8.1 | -| Browser | Mozilla Firefox | -| Browser Version | 62.0 | -| Resolution | 1280×1024 | -| Selenium Version | 3.11.0 | - -Then Capabilities Generator at TestMu AI will provide you with the below program: - -``` -// Basic Test Configurations For JavaScript - -var capabilities = { -"build" : "your build name", //You can edit this and assign a build name -"name" : "your test name", // Assign a name to your Test -"platform" : "Windows 8.1", // The operating system on which you want to test your website -"browserName" : "Firefox", // The browser on which you want to test -"version" : "62.0", // The browser version which you've selected to perform the test upon -"resolution" : "1280x1024", // The resolution in which you want to run the test as per your operating system -"selenium_version" : "3.11.0", //The version of Selenium on which the test will run -"visual" : true, -"firefox.driver" : v0.21.0 -} -``` - -> For more information, refer to our documentation on Selenium Automation Capabilities. - -## Testing Locally Hosted or Privately Hosted Projects - -To help you perform cross browser testing of your locally stored web pages, TestMu AI provides an SSH(Secure Shell) tunnel connection with the name Lambda Tunnel. With Lambda Tunnel, you can execute a test server inside your GitLab CI build a container to perform automated cross browser testing on browsers offered by Selenium grid on TestMu AI. So you make sure how well your changes look, even before your customers. -Curious to know more about Lambda Tunnel? - ->Follow our documentation on Lambda Tunnel to know it all. OS-specific instructions to download and set up tunnel binary are at the following links. -> ->- [Documentation For Windows User](/docs/local-testing-for-windows/) ->- [Documentation For Mac User](/docs/local-testing-for-macos/) ->- [Documentation For Linux User](/docs/local-testing-for-linux/) - ->**Download the binary file of:** -> ->- [Lambda Tunnel for Windows](https://downloads.lambdatest.com/tunnel/v3/windows/64bit/LT_Windows.zip) ->- [Lambda Tunnel for Mac](https://downloads.lambdatest.com/tunnel/v3/mac/64bit/LT_Mac.zip) ->- [Lambda Tunnel for Linux](https://downloads.lambdatest.com/tunnel/v3/linux/64bit/LT_Linux.zip) - -The below example of **.gitlab-ci.yml** file would demonstrate you on how to leverage TestMu AI’s Selenium grid by performing a browser test through your testing server contained in your GitLab CI build. - -``` -// Example of .gitlab-ci.yml for Lambda Tunnel - -image: node:7.10 - -before_script: -- | -cd /builds/qarachit/protractor-selenium-sample-gitlab -wget https://downloads.lambdatest.com/tunnel/v3/linux/64bit/LT_Linux.zip -#Extracting tunnel binary -sudo apt-get install unzip -unzip LT_Linux.zip -#Executing tunnel library -./LT -user ${LAMBDATEST_EMAIL} -key ${LAMBDATEST_KEY} & -sleep 30 -#Installing Dependencies -echo 'export LT_USERNAME="{your_lambdatest_username}"' >> ~/.bashrc -echo 'export LT_ACCESS_KEY="{your_lambda_access_key}"' >> ~/.bashrc -npm install -npm install -g protractor - -job: -script: -#Running Test! -protractor single.conf.js -``` - -Once, the tunnel is successfully set up. You can add the below code to your capabilities for testing internal servers on your network. - -``` -"desiredCapabilities": { -"tunnel":true -} -``` - -## Parallel Testing - -Parallel Testing is one of the most demanding features of TestMu AI Selenium Grid. By parallel testing, you can run more than one test case, simultaneously. This means that Parallel testing would allow you to execute numerous automation test cases altogether. So you execute a single test scenario across different browsers or could run different test scenarios across the same browser but with different browser versions. - -``` -protractor parallel.conf.js -``` - -Monitor and analyze your test result on the [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/). - -Deploy your code in a reliable manner at scale using GitLab CI integration with TestMu AI, and ensure it looks robust across every browser to provide a seamless user experience to all your visitors. Happy Testing! diff --git a/static/docs/gitlab-integration-with-hyperexecute.md b/static/docs/gitlab-integration-with-hyperexecute.md deleted file mode 100644 index 8dc548e77..000000000 --- a/static/docs/gitlab-integration-with-hyperexecute.md +++ /dev/null @@ -1,73 +0,0 @@ -# GitLab Integration with HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -GitLab is a web-based Git repository that provides free open and private repositories, issue-following capabilities, and wikis. It is a complete DevOps platform that enables professionals to perform all the tasks in a project, from project planning and source code management to monitoring and security. - -This document will show you how to integrate GitLab Pipeline with HyperExecute to greatly shorten your test cycles. - -## How To Integrate GitLab Pipeline with Hyperexecute - -To integrate GitLab Pipeline with HyperExecute, follow the below steps: - -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** -Download or Clone the code sample from the TestMu AI GitHub repository to run the tests on the HyperExecute. - - View on GitHub - -### 1. Log into your GitLab account - -### 2. Create a New Project - -To create a new project and connect your external repository to GitLab CI/CD, click the **Run CI/CD for external repository** tab on the bottom right. - -### 3. Connect Your External Repository: - -On GitLab, you can either connect your external repository from [GitHub](https://www.github.com) or by URL. In this example, we will connect our external repository by URL. - -To connect your external repository by URL, fill in the required information in the form below and click the **Create Project** button at the bottom of the page. - -### 4. Configure the Pipeline -- To configure the new pipeline that you just created, click the **Configure Pipeline** button in the center of the page. - -- Click on the **+Set up CI/CD** button as shown below. - -- Create a new `.gitlab-ci.yml` file at the root of the repository. - -Below is a sample of GitLab YAML created for your reference: - -```bash -## Define the image to use (adjust for macOS if needed) -image: ubuntu:latest - -## Define pipelines (can have multiple pipelines) -pipelines: -## Default pipeline (can be named differently) -default: -## Branches to trigger this pipeline on (adjust as needed) -branches: -- master - -## Define steps in the pipeline -steps: -## Download Hyperexecute CLI (descriptive name) -- name: Download Hyperexecute CLI -script: | -wget https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute -chmod u+x hyperexecute - -## Run Hyperexecute tests (descriptive name) -- name: Run Hyperexecute Tests -script: | -./hyperexecute --user --key --config -``` - -### 5. Run Your Job -- To commit your changes and run your job, click the **Commit Changes** button. - -**Below is an example of a Hyperexecute job that was triggered through the above pipeline:** - -> -**Run your tests at speeds never seen before. Happy testing! :)** diff --git a/static/docs/gitlab-integration.md b/static/docs/gitlab-integration.md deleted file mode 100644 index 70039dcc3..000000000 --- a/static/docs/gitlab-integration.md +++ /dev/null @@ -1,100 +0,0 @@ -# GitLab Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - ->GitLab is a web-based Git-repository manager with the purpose of facilitating entire DevOps lifecycle. It helps in the collaboration of different teams working on a project. Ensuring Development, QA, Security, Operations and Product team to sync simultaneously on the same project. GitLab helps in bringing your product a lot faster in the market by reducing the time taken to complete a release cycle in a very significant manner. - -The TestMu AI GitLab Integration allows you to create a card directly in your repository from TestMu AI platform. Push an issue to your respective repository anytime, even in the middle of your test session. The fields populated by you when marking as a bug through TestMu AI are displayed as information on the bug in GitLab repository for that testing instance. - -**GitLab Integration with TestMu AI, like all of our other integrations to 3rd party applications, is available for freemium as well as premium plan.** - -## How To Integrate GitLab With Your TestMu AI Account? - -**Step 1:** Login to your TestMu AI account. You would need to have Admin or User level access to see and install integrations. - -**Step 2:** Select **‘Integrations’** from the left navigation menu bar. This will guide you to a screen where you will find a list of 3rd party applications, available to integrate with your TestMu AI account. - -**Step 3:** Here under the ‘Project Management’ Section, you’ll find GitLab. Click on Gitlab block. - -**Step 4:** Once you click on install, you’ll be redirected to a screen. There would 2 options available for you to choose in order to **authenticate GitLab.** - -### 1. Via OAuth - -1. Select the radio button on **‘Via OAuth’** and hit **‘Next’.** - -2. You will be redirected to GitLab login page where you will be asked to provide your login credentials. - -3. If you are new to GitLab and want to sign up then you can do so with your GitHub account as well. Once you click on the **‘GitHub’** button you will be redirected to an authorization page. There you will authorize GitLab APIs to fetch your user owned resources from GitHub APIs. - -4. After you login, you will have to authorize TestMu AI to use your account. Click on the green button that says **‘Authorize’.** - -* **Why do you need to login?** - -The purpose of logging in is to bypass the authentication that is applied when two different applications interact using their respective APIs. GitLab APIs uses OAuth 2.0’s [authorization code grant flow](https://tools.ietf.org/html/rfc6749#section-4.1) for generating access tokens on user’s behalf. - -* **What is an Access token?** - -Access tokens are strings with authorization key required to access an API. They are issued to the client server and are usually opaque. They are used for requesting access to protected, user-specific resources. Access tokens are vital from a security point of view & can be generated in different formats, depending upon security requirements specified on the resource server. - -That’s it! Go to Integrations again and you will be able to notice a **green tick** indicating that GitLab is successfully **installed**. You can now experience bug logging in a fly from any of your running test session in TestMu AI to your repository directly by a single click. - -**Note:** Make sure you already have a project in your Gitlab account before you integrate to avoid any issues - -### 2. Using Personal Access Token - -1. Select the radio button on **‘Cloud’** and click **‘Install’**. Here you’ll need to enter your Gitlab API token to validate integration with TestMu AI. - -2. Login to your Gitlab account. Go to the profile icon on the top-right corner and click on ‘Settings’. - -3. Click the ‘Access Tokens’ from the menu on the left. - -4. **"Add a personal access token"** by providing a name for the application. Keep the name that is easy for you to remember as you access token will be derived from it. You can also set an expiry date for that token. Also, select the checkboxes for **‘api’, ‘read_user’ & read_repository** as scopes. These scopes will define the access to authenticate API over Basic OAuth for their respective personal token. After selecting the scopes hit the green button to **"Create personal access token".** - -* **What is a Personal Access Token?** - -In scenarios where OAuth is excessively utilized, Personal Access Tokens comes to aid in accessing the API. While generating a token, make sure you provide a description that is easy to remember, as your Personal Access Token will be derived from it. Treat this token just as you would treat your own password! Access tokens are strings with authorization key required to access an API. They are issued to the client server and are usually opaque. They are used for requesting access to protected, user-specific resources. Access tokens are vital from a security point of view & can be generated in different formats, depending upon security requirements specified on the resource server. - -After you click on **"Create personal access token",** you will get a prompt message saying **"Your new personal access token has been created".** You will find it under the highlighted field. - -Your **API token** is now ready. Copy the token & store it safely with you, as you will not be able to find it again. Paste the copied token into TestMu AI GitLab Authentication. - -That’s it! Go to Integrations again and you will be able to notice a **green tick** indicating that GitLab is successfully installed. You can now experience bug logging in a fly from any of your running test session in Lambdatest to your repository directly by a single click. - -## Logging Your First Bug Through GitLab Integration? - -**NOTE:** You need to have a project created on Gitlab before you perform one click bug logging through TestMu AI. If a project is not created then you will face the below error message when logging a bug. - -**Step 1:** Go for any of the test from the left navigation menu. For demo, we will be taking **"Real Time Test"** option. - -**Step 2:** Present a URL of the web-app you need to test in the dialog box. After that, select any configuration for browser and operating system of your choice & click **‘Start‘.** - -**Step 3:** After the VM is launched and operable. You can perform testing on your web-app for finding bugs. If a bug gets revealed, then you need to click on the **Mark as Bug** from the left panel for [capturing a screenshot](https://www.testmuai.com/full-page-screen-capture) of the same. We have highlighted that option with yellow in the below image. - -**Step 4:** After a screenshot is captured, you can annotate any issue or a task with an in-built image editor. Once you are done highlighting the bug, click on the button that says **"Mark as Bug".** - -**Step 5:** After clicking on **"Mark as Bug"** button a GitLab specific form would open up. Fill the fields as per your requirement. - - - You can select which **Projects** should the observation be marked under. - - You can select a relevant **Label** for the same UI observation. - - You can give a title to the bug through the field **Title.** - - You also get to post a **Description** to help relate the cause of the issue or the card. - -**Step 6:** Click on **"Create Bug"** & observe it being successfully marked through a single click effort. Once you hit the button, you will get **prompt messages** on top of your Virtual Machine indicating the progress of bug logging. You will get a prompt message in few seconds mentioning **"Bug marked successfully"** indicating that the bug has been pushed to the desired GitLab repository. - -**Step 7:** Visit the project you marked your bugs to. You will notice that the number of issues has increased! - -**Step 8.** Click on **‘Issues’,** in the next scene, you’ll find your bug along with the list of all the issues logged in the project. - -**Step 9.** Click on the description of bug and you will be redirected to the detail page of the bug. You will find all the data that you provided through TestMu AI already presented in the bug. TestMu AI automatically includes test environment details and related screenshots in the repository as attachments. - -## How To Uninstall GitLab Integration? - -You can work with one integration at a time. So if you would want to integrate to some 3rd party application of the similar type, other than the GitLab then you would have to uninstall your present integration. Here is how you can do it. - -**Step 1:** Login to your TestMu AI account. - -**Step 2:** Select **‘Settings’** from the left navigation menu bar & click on **‘Integrations’.** This will guide you to the same screen from where you initiated **GitLab installation.** You will find Gitlab in the ‘My Integrations’ section. - -**Step 3:** To remove the GitLab integration, click on **‘Remove’.** You can find the **Remove** button right next to **GitLab** icon - -That was all you need to know for TestMu AI + GitLab Integration. Increase your productivity with our integrations. If you still have any questions for us, please feel free to let us know. Our experts are always available on **chat** to help you out with any roadblock regarding our product. Happy testing! diff --git a/static/docs/gocd-integration-with-hyperexecute.md b/static/docs/gocd-integration-with-hyperexecute.md deleted file mode 100644 index 7f2a35e3b..000000000 --- a/static/docs/gocd-integration-with-hyperexecute.md +++ /dev/null @@ -1,116 +0,0 @@ -# GoCD Integration With HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -GoCD, short for "Go Continuous Delivery," is an open-source continuous integration and continuous delivery (CI/CD) server that helps automate and streamline the software development and release process. It is designed to facilitate the efficient and reliable delivery of software from development to production. - -This document will show you how to integrate GoCD with HyperExecute to greatly shorten your test cycles. - -## How to Integrate with HyperExecute - -### Pre-requisite: - -- [Download](https://www.gocd.org/download/#osx) the GoCD Server and Agent and unzip the folder. - -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** -Download or Clone the code sample from the TestMu AI GitHub repository to run the tests on the HyperExecute. - - View on GitHub - -### Step 1: Start the GoCD Server - -- Go to the gocd/bin folder and run the following commands: - - - To start the Server -```bash -./bin/go-server start -``` - - - To start the Agent -```bash -./bin/go-agent start -``` -- Now type ``http://localhost:8153/`` go in your browser to open the GoCD GUI. - -- Click on the New Pipeline Button. - -### Step 2: Configure the GoCD Pipeline - -#### Part 1: Material - -- Select the **Material Type**. For the demo purposes, we will be using the Git. - -- Enter the **Repository URL** from which you want to set up the project. - -- Enter your **Repository branch** name. - -- Enter the **Username** and **Password** of your GitHub. (It is required in case the access to the repo needed authentication). - -#### Part 2: Pipeline Name - -- Enter your **Pipeline Name**. - -#### Part 3: Stage Details - -A Stage is a group of Jobs and a Job is a work which needs to be executed. - -- Enter your **Stage Details**. - -#### Part 4: Job and Tasks - -- Enter the **Job Name**. - -- Enter the below mentioned script in the space provided. - -```bash -curl -O https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute -chmod +x hyperexecute -./hyperexecute --user --key --config -``` - -> Since my default agent is mac I am using darwin CLI in the cURL command. Kindly change it according to your system default agent. -- **Mac** -> darwin -- **Linux** -> linux -- **Windows**-> windows - -- Click on **Save + Run This Pipeline** button. - -### Job Running - -Now the Job is executed and running. You can check the status of Job in the: - -- #### GoCD GUI - -- #### GoCD Console - -- #### HyperExecute Dashboard - -After the build is successful, go to your HyperExecute Dashboard and check the output of the triggered Job. - -- #### Job Status Passed - -## Sample Workflow File - -```xml - - - - # Adjust for macOS if needed - - - - - - - - - # Adjust for macOS if needed - - - - - - -``` diff --git a/static/docs/gocd-integration-with-testmu.md b/static/docs/gocd-integration-with-testmu.md deleted file mode 100644 index 58af1f8ea..000000000 --- a/static/docs/gocd-integration-with-testmu.md +++ /dev/null @@ -1,48 +0,0 @@ -# GoCD Integration With TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -GoCD offers a free and open-source platform to help you configure robust CI/CD pipelines with ease. The intuitive UI offers an end-to-end view wherein you can visualize the entire migration path to production in a collective view. That way, you can easily pinpoint the inefficient areas to optimize the CI/CD process. By leveraging GoCD, you can achieve faster feedback loops through its modelling constructs which makes your complex CI/CD pipeline looks a lot simpler. You also get features like dependency management & parallel execution to help you deliver better products quicker. - -TestMu AI integration with GoCD will help your perform Selenium test automation for your code changes on more than 3000+ real browsers for both mobile and desktop. You can trigger your Selenium testing scripts directly from GoCD instance and they would be executed over your TestMu AI automation dashboard. - -## Prerequisites - -1. Copy the YAML configuration file from our [GitHub repository for GoCD](https://github.com/LambdaTest/nightwatch-gocd-sample/blob/master/nightwatch-gocd-sample.gocd.yaml). - -2. **TestMu AI Authentication Credentials** - Be aware of your TestMu AI authentication credentials i.e. your TestMu AI username, access key and HubURL. You need to set them up as your environment variables. You can retrieve them from your [TestMu AI automation dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/) by clicking on the key icon near the help button. - - * **For Linux/Mac:** - - $ export LT_USERNAME= `{YOUR_LAMBDATEST_USERNAME}` - - $ export LT_ACCESS_KEY= `{YOUR_LAMBDATEST_ACCESS_KEY}` - - * **For Windows:** - - $ set LT_USERNAME= `{YOUR_LAMBDATEST_USERNAME}` - - $ set LT_ACCESS_KEY= `{YOUR_LAMBDATEST_ACCESS_KEY}` - -3. Make sure to install GoCD in your system. - -## Configuring GoCD Pipeline For TestMu AI Selenium Grid - -**Step 1:** Visit your GoCD Dashboard and create a new pipeline by selecting the option to "Use Pipeline as Code". - -**Step 2:** Scroll-down to the section to "Register Your Pipelines as Code Repo with GoCD" and untick the box for using the same SCM repository. - -**Step 3:** Provide the [TestMu AI GitHub repository for GoCD](https://github.com/LambdaTest/nightwatch-gocd-sample) as the repository URL. Test you connection and provide a label for this repository. In our case, we have provided the label as Sample-Pipeline, you may keep something else for your instance. - -**Step 4:** After providing the GitHub repository, you will need to Scan it. Hit the highlighted button to Scan repository and you will notice that a respective YAML definition file would be through the repository. - -**Step 5:** After scanning the repository, you need to scroll to bottom and hit finish. Make sure to specify the username and password under the advanced setting, for private repositories. - -**Step 6:** Once you hit the finish button. Under the Config repositories, you will find the details of the repository as specified by you. - -**Step 7:** Now, if you visit your GoCD dashboard, you will find the pipeline generated and the job in running state. - -**Step 8:** If you open the pipeline and refer to the console then you will notice the current job status as passed. - -Great work! You have successfully integrated your GoCD instance with your TestMu AI account. Follow our another documentation, if you wish to configure a private YAML and then integrate your GoCD pipeline with TestMu AI. diff --git a/static/docs/goodday-integration.md b/static/docs/goodday-integration.md deleted file mode 100644 index 764c4fd62..000000000 --- a/static/docs/goodday-integration.md +++ /dev/null @@ -1,70 +0,0 @@ -# GoodDay Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -> GoodDay is a comprehensive work management platform that offers tools for project management, task organization, and productivity growth with features that boost transparency, agility, and motivation. With GoodDay as the bug tracking tool, project teams can easily avoid the hassle of digging through bug logs for hours. Bugs are a part of the software development process, which is unlikely to change. As software projects grow in features, their complexity also increases, which increases the number of bugs. The GoodDay bug tracker lets users capture, log, and prioritize the bugs that require fixing on priority. GoodDay integrates with the tools you use daily, letting you work faster and deliver more quality software on time. The GoodDay bug tracker template has a Kanban board to represent all the relevant information about each bug. It shows who is responsible for fixing the bug and when, so you can assign the bug to the person responsible. You can even set a deadline or schedule the task to have all the bugs resolved. - -The TestMu AI GoodDay integration helps you in filing bug-related issues to your project in GoodDay directly from the TestMu AI platform. With one-click integration, you can push annotated bug issues to the project of your choice, assign them to the required teammate, and attach necessary screenshots. You can do all that while in the middle of a test session in the TestMu AI platform. The fields populated by you when marking as a bug through TestMu AI are displayed as information on the GoodDay platform for a testing instance. - -> GoodDay Integration with TestMu AI, like all of the integrations to 3rd party applications, is available for freemium as well as premium plans. - -## How To Establish Integration With GoodDay From Your TestMu AI Account? - -**Step 1:** Log in to your TestMu AI account. You would need to have Admin or User level access to see and install integrations. - -**Step 2:** Select ‘Integrations’ from the left navigation menu bar. This will guide you to a screen where you will find a list of 3rd party applications, available to integrate with your TestMu AI account. - -**Step 3:** Click on ‘ADD’ under the block that says ‘GoodDay’. - -**Step 4:** Once you click on Add icon, you will be redirected to the below screen. You will be asked to provide your GoodDay API Token credentials in the highlighted fields to establish integration with TestMu AI. - -> **What is an API token?** -What is an API token? When two different applications interact with each other, a server-to-server communication is established. API token is needed for authenticating user identity from one server to another, by fetching user-owned resources available on the server from where the communication is initiated. API token has been a necessity for maintaining security and data integrity of any application. They are just as vital to an end user as they are to an application. API tokens are relevant if an end user logs in an application through a 2-step verification process. - -**Step 5:** You can find your API token from your GoodDay account. If you have not generated one before, you can generate an API token by going to the settings. You will find the "Settings" icon on the top right corner of your dashboard. - -**Step 6:** As you click on "Settings", you will see a section named "Integrations" on the left side navigation bar. Upon clicking "Integrations", you need to select "API" from the drop-down. - -**Step 7:** Now click on the button "Generate" in order to generate your API token. - -> You can keep a note of this API token with you for future authentication. Although, in case you forget to do so, or you end up neglecting it due to some reason. Then there is no need to panic! You can create a new token by defining a new label. You can also delete a previously declared API Key, in case you feel that your API Key has been placed in some wrong hands. You can delete an API Key by hovering over the label and clicking on the bin icon. - -**Step 8:** Copy the newly generated API token. - -**Step 9:** Paste the API token into the field provided at TestMu AI & hit the ‘Install’ button. You are all set to experience one-click bug logging to share your issues directly from your TestMu AI account to your project on GoodDay. - -That’s it! Go to Integrations again and you will be able to notice a green tick indicating that GoodDay is successfully installed. - -## How To Log Your First Bug Through GoodDay Integration? - -**Step 1:** Go for any of the test from the left navigation menu. For demo, we will be taking "Real Time Test" option. - -**Step 2:** Present a URL of the web-app you need to test in the dialog box. After that, select any configuration for browser and operating system of your choice & hit ‘Start‘. - -**Step 3:** After the VM is launched and operable. You can perform testing on your web-app for finding bugs. If a bug gets revealed, then you need to click on the Bug icon from the left panel for capturing a screenshot of the same. - -**Step 4:** After a screenshot is captured, you can annotate any issue or a task with an in-built image editor. Once you are done highlighting the bug, click on the button that says "Mark as Bug". - -**Step 5:** After clicking on "Mark as Bug" button a GoodDay specific form would open up. Fill the fields as per your requirement. Click on "Create Issue" & observe it being successfully marked through a single click effort. You can select the following: - -* You can select which **Project** should the ticket go under. -* You can set the particular **Story Type**. -* You can assign the bug to a colleague by populating the **Assignee** field. -* You can mention the **Summary** as well to easily organize your bugs. -* You also get to post a **Description** to help relate the cause of the issue or the task. - -**Step 6:** Visit your GoodDay dashboard. Under "Pinned Projects", you will find the recently raised issue being highlighted. - -**Step 8:** As you click on the recently raised issue under the "Not Started" tab, you will notice that all the data which you provided through TestMu AI would already be presented in it. TestMu AI automatically includes test environment details and related screenshots in the ticket as attachments. - -## How To Uninstall GoodDay Integration? - -> You can work with one integration at a time. So if you would want to integrate to a similar 3rd party application, then you would have to `uninstall` your current integration. Here is how you can do that. - -**Step 1:** Login to your TestMu AI account. - -**Step 2:** Select ‘Integration’ from the left navigation menu bar. This will guide you to a screen where you will find 3rd party applications, available to integrate with your TestMu AI account. - -**Step 3:** Click on ‘REMOVE’. You can find the remove button right under the ‘GoodDay’ block which would be highlighted with a green tick. - -> That was all you need to know for TestMu AI + GoodDay Integration. Increase your productivity with our integrations. If you still have any questions for us, please feel free to let us know. Our experts are always available on **chat** to help you out with any roadblock regarding our product. Happy testing! diff --git a/static/docs/google-chat-integration.md b/static/docs/google-chat-integration.md deleted file mode 100644 index ca9ad461e..000000000 --- a/static/docs/google-chat-integration.md +++ /dev/null @@ -1,57 +0,0 @@ -# Google Chat Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Google Chat enables business teams to streamline their communications within Google Workspace. With Google Chat, you can chat using text, create collaborative chat rooms, share files, and hold virtual conferences. - -TestMu AI and Google Chat can be used to create test cards in Google Chats Spaces for logged bugs in TestMu AI. This will notify you every time a new bug is pushed while performing [cross browser testing](https://www.testmuai.com/) of websites. - -## Integrating Google Chat With TestMu AI - -**Step 1:** Sign in to your Google account and head to **Google Chat**. - -**Step 2:** Create or select a **Google Space** from the left panel. If you don’t have a Google Space, you can create one by clicking the **"+"** icon. - -**Step 3:** Click on **Create a new space**. - -**Step 4:** Enter your preferred **Space Name** and click **Create**. - -**Step 5:** Now you have a new **Google Space** created. - -**Step 6:** Head to the **Integrations** in your TestMu AI Dashboard. Add **Google Chat** from under the Communication section. - -**Step 7:** Click on the **Sign in with Google** button. - -**Step 8:** Select the **Google account** that you have used to create your Google Space. - -**Step 9:** Click on **Allow** to let TestMu AI access your Google Account to complete the integration. - -**Step 10:** Now select your Google Space from the dropdown and click **Install**. - -**Step 11:** You can now see in your **Google Chat Dashboard** that we have successfully integrated TestMu AI with Google Chat. - -## One-Click Bug Logging With Google Chat - -To demonstrate, we take an example of Real-time testing. - -**Step 1:** Select Real-Time Testing from the left menu. - -**Step 2:** Provide your test URL, and select the configurations: Browser, Browser versions, OS, and Resolution. Click on **Start**. - -**Step 3:** A virtual machine will launch where you can perform real-time testing of websites and web apps. If you identify a bug, click on the **'Mark as Bug'** icon from the left bar to capture the screenshot. - -**Step 4:** Once the screenshot is captured, annotate issues or tasks you have encountered using in-built image editor tools. After annotating the bugs, click on the **bug** icon in the right-side panel. - -**Step 5:** A Google Chat ticket will open up. Fill in the details like **Assignee, Issue Type, Priority, Summary, and Description**. After populating the required fields, check the **"Send to Google Chat"** option and click **'Mark as Bug'**. - -**Step 6:** To view your logged bugs, head to the **Issue Tracker** under the **Manage** section on the left navigation menu. You'll notice the marked bugs along with the Test details. - -**Step 7:** You can find the test log details directly in your selected Google Chat Space as shown below. - -## Uninstalling The Google Chat Integration - -**Step 1:** Sign in to your TestMu AI account and visit Integrations from the left sidebar menu. - -**Step 2:** From the My Integrations section, click the remove option under the Google Chat block. - -> That was all you need to know for TestMu AI + Google Chat Integration. Increase your productivity with our integrations. If you still have any questions for us, please feel free to let us know. Our experts are always available on **chat** to help you out with any roadblock regarding our product. Happy testing! diff --git a/static/docs/google-cloud-ci-integration.md b/static/docs/google-cloud-ci-integration.md deleted file mode 100644 index 38602ac65..000000000 --- a/static/docs/google-cloud-ci-integration.md +++ /dev/null @@ -1,118 +0,0 @@ -# Run Selenium Tests With Google Cloud CI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Google Cloud Platform is one of the market’s leading public cloud providers and offers a variety of management services. Google Cloud Build and Google Cloud Run are two Google Cloud services that use pipelines to automate builds and deployments. - -Integrate your Selenium test suites with Google Cloud CI and perform automated cross-browser testing on [online Selenium Grid](https://www.testmuai.com/selenium-automation) over 3000+ real browser environments. With this integration, you can also perform local testing of locally or privately hosted projects and Parallel testing to cut short test execution cycles and achieve faster go-to-market delivery. - -## Prerequisites - -Before integrating Google Cloud CI with TestMu AI, make sure you have the following things: - -1. A GitHub or a BitBucket repository. All the code samples in this documentation can be found at [TestMu AI Google Cloud CI](https://github.com/LambdaTest/Java-TestNG-Selenium) repository on GitHub . - -2. To integrate test suites with Google Cloud CI, you would need to set your TestMu AI username and access key in environment variables. You can get them from [Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/) by clicking on the Key button from the top-right. - -* **For Windows:** -``` -set LT_USERNAME="YOUR_LAMBDATEST_USERNAME" -set LT_ACCESS_KEY="YOUR_LAMBDATEST_ACCESS_KEY" -``` - -* **For Mac/Linux:** -``` -export LT_USERNAME="YOUR_LAMBDATEST_USERNAME" -export LT_ACCESS_KEY="YOUR_LAMBDATEST_ACCESS_KEY" -``` - -## Integrating A GitHub Repository With Google Cloud CI - -To integrate Google Cloud CI with TestMu AI, you will need to fork the above TestMu AI Google Cloud CI repository. - -1. Go to the Google Cloud CI console and select **'Cloud Build'**. - -2. Once you're in the Cloud Build, click on **'Triggers'**. - -3. To use the trigger functionality, you will need to connect your GitHub repository to Google Cloud CI instance. Click on **'Connect Repository'**. - -Once you click on Connect Repository, you will need to complete the steps in order to connect your repository. - -## Running Single Test Via Google Cloud CI - -1. After connecting to the repository, click on **'Create Trigger'**. - -* Provide a Trigger name. - -* Select the repository's source and then select the branch. - -* Under the **'Configuration'**, select _**Cloud Build configuration file (yaml or json)**_. - -* In the **'Advanced'** category, click on **'Add Variable'**. - -Enter your TestMu AI Username and Access Key in the provided field and then click on **'Create'**. - -2. Your trigger project will be created on the selected Google Cloud CI GitHub repository. To run the single test through Google Cloud CI, click on **'Run'**. - -A new build can be seen in the Google Cloud CI console. The current status of this build will be ‘Running’ or ‘Successful’. - -You can also view the detailed insights of your test execution on [TestMu AI Automation dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/timeline/). - -## Running Parallel Test Via Google Cloud CI - -To run parallel tests on TestMu AI cloud, you will need to make minor tweaks in `cloudbuild.yml` file. - -``` -steps: -- name: gcr.io/cloud-builders/docker -args: ['run' , '-d', '--name=lt', '--network=cloudbuild', 'lambdatest/tunnel', '--user', '${_LT_USERNAME}', '--key', '${_LT_ACCESS_KEY}', '--tunnelName', 'GCloud', '--infoAPIPort', '15000','--load-balanced'] -- name: curlimages/curl -args: ['-s', '--retry-connrefused', '--connect-timeout', '5', '--max-time', '5', '--retry', '30', '--retry-delay', '2', '--retry-max-time', '60', 'http://lt:15000/api/v1.0/info'] -- name: 'ubuntu' -args: ['sleep', '20'] -- name: 'bash' -args: ['ls'] -- name: 'maven' -entrypoint: 'mvn' -args: ['test', '-P', 'parallel'] -env: -- 'LT_USERNAME=${_LT_USERNAME}' -- 'LT_ACCESS_KEY=${_LT_ACCESS_KEY}' -``` - -In the above YAML code, you need to add the following command to **args**. - -`mvn test -P parallel` - -Below is the screenshot of the above-executed parallel tests. - -## Running Local Test Via Google Cloud CI - -To run local tests on TestMu AI cloud, you will need to make minor tweaks in `cloudbuild.yml` file. - -``` -steps: -- name: gcr.io/cloud-builders/docker -args: ['run' , '-d', '--name=lt', '--network=cloudbuild', 'lambdatest/tunnel', '--user', '${_LT_USERNAME}', '--key', '${_LT_ACCESS_KEY}', '--tunnelName', 'GCloud', '--infoAPIPort', '15000','--load-balanced'] -- name: curlimages/curl -args: ['-s', '--retry-connrefused', '--connect-timeout', '5', '--max-time', '5', '--retry', '30', '--retry-delay', '2', '--retry-max-time', '60', 'http://lt:15000/api/v1.0/info'] -- name: 'ubuntu' -args: ['sleep', '20'] -- name: 'bash' -args: ['ls'] -- name: 'maven' -entrypoint: 'mvn' -args: ['test', '-P', 'local'] -env: -- 'LT_USERNAME=${_LT_USERNAME}' -- 'LT_ACCESS_KEY=${_LT_ACCESS_KEY}' -``` - -Add the below command to **args**. - -`mvn test -P local` - -Below is the screenshot of the above-executed local tests. - -> -That's all! Ship your quality product faster using TestMu AI and Google Cloud CI integration. In case you have any queries for us, feel free to drop them at our 24*7 Customer Chat Support or you can also mail us at [support@testmuai.com](mailto:support@testmuai.com). Happy Testing! diff --git a/static/docs/gradle-integration-with-testmu.md b/static/docs/gradle-integration-with-testmu.md deleted file mode 100644 index 0f5fd7312..000000000 --- a/static/docs/gradle-integration-with-testmu.md +++ /dev/null @@ -1,86 +0,0 @@ -# Gradle Plugin Integration With TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Gradle is a build automation tool for multi-language software development. It controls the development process across various tasks, including compilation, packaging, testing, deployment, and publishing. Supported languages include Java, Kotlin, Groovy, Scala, C/C++, and JavaScript. - -## Steps to Setup the Gradle Plugin - -### Step 1: Add TestMu AI Gradle Plugin to the `build.gradle` file - -```java title="build.gradle" -plugins { -id 'io.github.lambdatest.gradle' version '1.0.7' -} - -repositories { -maven { -url "https://plugins.gradle.org/m2/" -} -} -``` - -### Step 2: Configure TestMu AI Parameters in `build.gradle` file - -```java title="build.gradle" -runLambdaTest { -username = 'YOUR_LAMBDATEST_USERNAME' -accessKey = 'YOUR_LAMBDATEST_ACCESS_KEY>' -appFilePath = 'PATH_TO_YOUR_APP_FILE' -testSuiteFilePath = 'PATH_TO_YOUR_TEST_SUITE' -device = 'YOUR_DESIRED_DEVICE_FOR_TESTING' -isFlutter = true //if you are running flutter dart tests -appId = "lt://1234343" //provide this only if you have already uploaded the app -testSuiteId = "lt://1223444" //provide this only if you have already uploaded the app -} -``` - -### Step 3: Configure Upload APK Parameters -With the release of version **`1.0.7`** of the `lambdatest-gradle-plugin`, users now have the option to upload their APK files directly to TestMu AI. Add the following configuration to build.gradle to enable APK upload: - -```java title="build.gradle" -uploadApkToLambdaTest { -username = 'YOUR_LAMBDATEST_USERNAME' -accessKey = 'YOUR_LAMBDATEST_ACCESS_KEY>' -appFilePath = 'PATH_TO_YOUR_APP_FILE' -testSuiteFilePath = 'PATH_TO_YOUR_TEST_SUITE' -} -``` -### Step 4: Execute the Plugin -To run the plugin configured in build.gradle, execute the following command: - -```bash -./gradlew runLambdaTest -``` - -If you only want to upload your APK files to TestMu AI, use the following command: - -```bash -./gradlew uploadApkToLambdaTest -``` - -## Post Upload Configuration -Once the upload process is completed, TestMu AI will return unique IDs for the uploaded files. These IDs can be used in the runTestMu AI configuration to execute your tests seamlessly. - -## Virtual Device Support -The TestMu AI Gradle plugin now supports virtual devices. To execute tests on a virtual device, set **`isVirtualDevice: true`** in both **`runLambdaTest`** and **`uploadApkToLambdaTest`** configurations. If this flag is not provided, tests will default to running on a real device. - -## Supported Capabilities - -The TestMu AI Gradle Plugin supports the following capabilities: - -| Capability | Description | Example | -|------------|-------------|---------| -| `appId` | Enter the app id generated while uploading the app. | `appId= lt://APP123456789123456789` | -| `testSuiteID` | Enter the test suite id generated while uploading the test suite. | `testSuiteID= lt://APP123456789123456789` | -| `device` | Enter the name and OS version of the device in "DeviceName-OSVersion" format. | `device= 'Pixel 3 XL-9'` or `device= 'Galaxy S21 Ultra 5G-11'` | -| `video` | Generate a video for all the tests that have run. | `true` or `false` | -| `queueTimeout`| Enter the time in seconds after which you want your build to timeout from queue. | `queueTimeout= 300` | -| `idleTimeout` | Enter the time in seconds for maximum running time on a test in the build. | `idleTimeout= 120`| -| `deviceLog` | Boolean value to generate device logs. | `deviceLog= true` or `false` | -| `build` | Set the name of the Espresso test build. | `My Espresso Build` | -| `geoLocation` | Set the geolocation country code if you want to enable the same in your test. | `FR` | -| `tunnel`, `tunnelName`| Set `tunnel` as `true` and provide the `tunnelName` as needed if you are running a tunnel. | `tunnel= true` `tunnelName= NewTunnel`| -| `disableAnimation` | Set `disableAnimation` to `true` if you want to disable animations for espresso tests. Default value is `false`. | `disableAnimation= false`| -| `clearPackageData` | Set `clearPackageData` to `true` if you want to clear the app data after each test has completed running. Default value is `false`. | `clearPackageData= false` | -| `singleRunnerInvocation` | Set `singleRunnerInvocation` to `true` if you want to execute test cases together with a single test runner invocation. This can help significantly speed up the test execution time. By default, each test case is invoked separately. Default value is `false`.| `singleRunnerInvocation= true` | diff --git a/static/docs/group-and-filter-your-test-builds-using-build-tags.md b/static/docs/group-and-filter-your-test-builds-using-build-tags.md deleted file mode 100644 index b1092030f..000000000 --- a/static/docs/group-and-filter-your-test-builds-using-build-tags.md +++ /dev/null @@ -1,67 +0,0 @@ -# Group and Filter Your Test Builds Using Build Tags - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -With TestMu AI, you can group your test builds with Build tags. This document shows how to create Build tags and use them to group or filter your test builds on the TestMu AI Automation Dashboard. - -## How to Create Build Tags - -Add a buildTags capability with a String array to your desired capabilities. - -While writing your automation tests, add another capability when you create the [Desired Capabilities](/docs/selenium-automation-capabilities/) via code. Follow the below steps: - -* Create an array of Strings that contains your Build tags, each separated by a comma. - -```java -// For example, when you have only 1 tag -String[] buildTagList = {"Build Tag"}; - -// For example, when you have multiple tags -String[] buildTagList = {"Tag 1", "Tag 2", "Tag 3", ...}; -``` - -* Now add this Build tag array in your Desired Capabilities instance: **For example:** While creating a sample Desired Capabilities instance in Java, the code will be: - -```java -// Creating the Build Tags -String[] buildTagList = {"Tag1", "Tag2", "Tag3", "BuildTagRishabh"}; - -DesiredCapabilities caps = new DesiredCapabilities(); -caps.setCapability("browser", "Safari"); -caps.setCapability("version", "13"); -caps.setCapability("platform", "macos Catalina"); -caps.setCapability("build", "Build Tags Demo"); -caps.setCapability("name", "Sample Test"); - -// To create custom tags -caps.setCapability("buildTags", buildTagList); - -System.out.println("Desired Caps: " + caps); -driver = new RemoteWebDriver(new URL("https://" + username + ":" + authkey + hub), caps); -``` - -You have successfully created the Build tags. Run your test and navigate to [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/timeline/) to see the tags in that build: - -## Guidelines for Creating Build Tags - -Follow these limits when creating build tags to avoid unexpected behavior. - -While creating Build Tags, follow the below guidelines: - -* Add a maximum of 5 custom tags to a build. -* You can update the existing build by specifying different tags. -* If you change a tag name or number of tags, no new build is created. The existing build is updated with the new tag to avoid unnecessary build creation. - -## How to Filter Tests Using Build Tags - -Select one or more build tags in the filter toolbar to narrow down your test builds. - -To group or filter the tests on your automation dashboard with Build tags: - -Navigate to [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/timeline/), and you can see a filter by name **Build Tags** on the right corner of the filter toolbar. - -Select the Build tag by which you want to filter your test. For example, if we choose _Tag1_, only the test builds with the tag "Tag1" appear: - -You can choose multiple Build tags to filter your tests, as shown in the image below: - -> You have successfully created a Build tag and grouped and filtered your tests based on these Build tags. If you have questions, share them with us through our **24/7 chat support** or by mailing us at [support@testmuai.com](mailto:support@testmuai.com). diff --git a/static/docs/group-folder-redirect-on-real-device.md b/static/docs/group-folder-redirect-on-real-device.md deleted file mode 100644 index e10fbf337..000000000 --- a/static/docs/group-folder-redirect-on-real-device.md +++ /dev/null @@ -1,24 +0,0 @@ -# Group Folder Redirect On Real Devices - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI supports **Group Folder Redirect** for iOS apps on Real Devices. -This feature ensures your app uses its **private container directory** instead of the **shared app group container**, which becomes inaccessible after **app resigning** on Real Devices. - -## Use Cases - -- Ensure your app maintains **file system access** after being re-signed on Real Devices. -- Prevent issues when your app relies on the **shared App Group container**, which becomes inaccessible after resigning. -- Guarantee consistent **storage and retrieval of files** by using the app’s private container. - -## Using Group Folder Redirect in Manual Testing - -**Step 1**: Click on the Real Devices > App Testing - -**Step 2**: Upload your application. Open the **App Settings** to enable the **Group Folder Redirect** toggle. - -**Step 4**: Select your device and start your Real Device testing session. - -- An **instrumented version** of your app with Group Folder Redirect support will launch. -- The app will store and retrieve files from its **private container directory** instead of the shared App Group container. -- File system features will function consistently even after app resigning. diff --git a/static/docs/group-folder-redirects.md b/static/docs/group-folder-redirects.md deleted file mode 100644 index cf2719efb..000000000 --- a/static/docs/group-folder-redirects.md +++ /dev/null @@ -1,44 +0,0 @@ -# Group Folder Redirect for iOS Apps - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI now supports **Group Folder Redirect** for iOS apps during automation testing with Appium. -This feature ensures your app uses its **private container directory** instead of the **shared app group container**, which becomes inaccessible after **app resigning** on Real Devices. - -**Availability** -This feature is being rolled out gradually. Please contact your TestMu AI support representative to get it enabled or you could reach out at our **[24X7 Chat Support]** or mail us directly at support@testmuai.com. - - ## Use Cases - -- Ensure your app maintains **file system access** after being re-signed on Real Devices. -- Prevent issues when your app relies on the **shared App Group container**, which becomes inaccessible after resigning. -- Guarantee consistent **storage and retrieval of files** by using the app’s private container. - -## Steps to Enable Group Folder Redirect - -### 1. Upload Your Application -- Upload your iOS app to TestMu AI following standard procedures. -- Refer to the [**Upload your Application**](/support/docs/application-setup-via-api/#upload-your-application) documentation for detailed instructions. -- Note the **App ID** returned after uploading, you will use this in your automation scripts. - -### 2. Configure Desired Capabilities -Include the `groupFolderRedirectEnabled` capability in your Appium desired capabilities and set it to `True`. - -{`desired_caps = { - "deviceName":"iPhone 16", - "platformName":"iOS", - "platformVersion":"18", - "isRealMobile": True, - "app":"YOUR_APP_URL", - "build":"Sample Build", - "name":"Sample Test", - #highlight-next-line - "groupFolderRedirectEnabled": True -}`} - -### 3. Run Your Automation Test -Launch your Appium test, the app will automatically use its private container for all file operations without any code changes. - -- TestMu AI launches an **instrumented version** of your app with **Group Folder Redirect** support. -- All file system operations that previously relied on the App Group container now use the **private container**. -- Your tests remain compatible and functional, even after app resigning. diff --git a/static/docs/group-management.md b/static/docs/group-management.md deleted file mode 100644 index 7c7fe7c16..000000000 --- a/static/docs/group-management.md +++ /dev/null @@ -1,37 +0,0 @@ -# Group Based Concurrency Distribution - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The TestMu AI Group Management will help you in creating the different Groups and allot concurrency to those groups. In this way, you can perform the test automation by dividing the concurrency among the groups. - -## How to Create a New Group? - -**Step 1:** Login to your [LambdaTest](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard) account and navigate to Profile icon on the top-right. Click on ***Organization Settings***. - -**Step 2:** Select the 'Groups' tab and click on '+ Create'. - -**Step 3:** Fill in the group name of your choice, allot the maximum concurrency using the sliders. After providing the details, click on 'Next'. - -**Step 4:** Select users you want to add to the group, click on 'Next'. - -**Step 5:** Review the list of users you have selected to add to the group, then click on 'Finish'. Your new group will be created successfully. - -## How to View and Edit a Group? - -**Step 1:** Login to your TestMu AI account and navigate to Profile icon on the top-right. Click on ***Organization Settings***. - -**Step 2:** Click on the 'Groups' tab, select the desired group you want to view and edit. Click on the 3 dots and then select 'Edit' button. - -**Step 3:** You can view and edit the Group Name and Concurrency settings. After making changes, click on 'Next' to proceed. - -**Step 4:** You can add or remove users from the group. Select or deselect users as needed, then click on 'Add Users' or 'Save Changes' to save your modifications. - -## How to Delete a Group? - -**Step 1:** Login to your TestMu AI account and navigate to Profile icon on the top-right. Click on ***Organization Settings***. - -**Step 2:** Click on the 'Groups' tab, select the desired group you want to delete. Click on the 3 dots and then select 'Delete' button. - -**Step 3:** A confirmation dialog will appear. Click on the 'Delete' button to confirm. Your selected group will be deleted. - -> In case you have any queries regarding TestMu AI Group Management or need any additional information, please reach out at our 24*7 Chat Portal or drop us a mail to support@testmuai.com. diff --git a/static/docs/group-tests-using-custom-tags.md b/static/docs/group-tests-using-custom-tags.md deleted file mode 100644 index d5cd74ab1..000000000 --- a/static/docs/group-tests-using-custom-tags.md +++ /dev/null @@ -1,63 +0,0 @@ -# Group Tests Using Custom Tags - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI allows you to group your automation tests using custom tags. This document shows how to create custom tags and use them. For demonstration purposes, we use a [sample TestNG script](https://github.com/LambdaTest/Java-TestNG-Selenium) to run on the TestMu AI platform. - -## How to Create Custom Tags on the Selenium Grid - -Add a tags capability with a String array to your desired capabilities. - -You can create a custom tag while writing your Selenium automation tests. When you create the [Desired Capabilities](/docs/selenium-automation-capabilities/) via code, add the below lines of code. - -* Create a String array that contains the names of your custom tags, separated by a comma. -```javascript -// In case for just 1 tag, just add 1 element in the array -String[] customTags = {"Custom Tag"}; - -// In case for multiple tags, add them in the array separated by comma -String[] customTags = {"Tag 1", "Tag 2", "Tag 3", ...}; -``` - -* Now add this custom tag in your Desired Capabilities instance: -```javascript -DesiredCapabilities caps = new DesiredCapabilities(); -. -. - -// To create custom tags -caps.setCapability("tags", customTags); -``` -*For example:* - -You have successfully created the custom tags. Let us now see how to view and group tests based on custom tags. - -## How to View Custom Tags on the Platform - -View your custom tags in the Timeline or Automation Logs views on the dashboard. - -There are multiple ways to view custom tags in the automation dashboard, such as Timeline view and Automation logs view. - -### How to View Custom Tags on Your Timeline - -Navigate to [Timeline](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/timeline/) view of your automation dashboard, and toggle the **Build View** to **Test View**: - -You can see the applied custom tags below the tests in this Test View. - -### How to View Custom Tags on Your Automation Logs - -Navigate to [Automation Logs](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/logs) of your automation dashboard, and you can see the applied custom tags below the tests in the left panel. - -## How to Filter Tests Using Custom Tags on the Platform - -Select one or more custom tags in the filter toolbar to narrow down your test results. - -You can filter tests on your automation dashboard with these custom tags. - -Navigate to [Automation Logs](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/logs) of your automation dashboard, and you can see a filter by name **Tags** in the filter toolbar. - -Click on it and select the Tag by which you want to filter the tests on your dashboard. For example, we have filtered the test using _Tag2_ custom tag: - -You can filter tests by selecting multiple custom tags at once from the filter toolbar: - -> You have successfully created a custom tag and grouped your tests based on the custom tag. If you have questions, share them with us through our **24/7 chat support** or by mailing us at [support@testmuai.com](mailto:support@testmuai.com). diff --git a/static/docs/har-log-viewer-with-waterfall.md b/static/docs/har-log-viewer-with-waterfall.md deleted file mode 100644 index f2a7a0f4e..000000000 --- a/static/docs/har-log-viewer-with-waterfall.md +++ /dev/null @@ -1,25 +0,0 @@ -# HAR Log Viewer With Waterfall - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The HAR (HTTP Archive) Log Viewer with Waterfall displays how resources on a web page load and interact over time. Use it to identify bottlenecks and optimize page performance in your Selenium tests. - -## How to Use the HAR Log Viewer - -Enable HAR logging by passing `network.full.har = true` in your test capabilities. - -Once the test runs, open the Web Automation Dashboard and navigate to the **Network** tab to see the waterfall chart. - -Each resource (HTML, CSS, JavaScript, images) appears as a horizontal bar. The bar length shows load time, and the sequence reflects the order the browser requested them. - -## Use Cases - -Apply the HAR Log Viewer to diagnose and improve page performance. - -- **Performance optimization** - Pinpoint slow-loading resources and compress images, minify CSS/JS, or improve caching. -- **Load order diagnosis** - Check if critical CSS and JavaScript load before non-essential resources. -- **Network issue identification** - Determine if delays are server-side (slow time to first byte) or network-related (slow download). -- **Performance monitoring** - Compare HAR logs over time to detect degradation after deployments. -- **Third-party resource impact** - Measure how ads, analytics scripts, and other external resources affect load time. -- **Before/after comparison** - Compare HAR files before and after optimization to measure improvements. -- **Cross-browser analysis** - Analyze HAR files from different browsers to understand loading differences. diff --git a/static/docs/he-integration-with-testmu-products.md b/static/docs/he-integration-with-testmu-products.md deleted file mode 100644 index f06eda10a..000000000 --- a/static/docs/he-integration-with-testmu-products.md +++ /dev/null @@ -1,19 +0,0 @@ -# HyperExecute Integrations with Products - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -You can now integrate HyperExecute with other TestMu AI products like Smart UI testing. You can go through the links below to see the integrations we offer at the moment. - - Smart UI - Hooks - Execute Smart UI Tests to ensure the visual consistency of web applications using HyperExecute. - - Smart UI - SDK - Execute Smart UI Tests using SDK to ensure the visual consistency of web applications using HyperExecute. - - Virtual Devices - Run app automated tests on Emulators and Simulators using HyperExecute. - - Real Devices - Run app automated tests on Real Mobile Devices using HyperExecute.. - ->If your favourite products are not mentioned in the list above, then just give us a shout and we will have them ready for you. diff --git a/static/docs/hive-integration.md b/static/docs/hive-integration.md deleted file mode 100644 index 1002de4c3..000000000 --- a/static/docs/hive-integration.md +++ /dev/null @@ -1,70 +0,0 @@ -# Hive Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Hive is an online project management application that aims at maximizing your workplace productivity. With Hive you can plan project flexible and organize them in a Gantt chart, kanban board, table, or calendar. You can also keep a check on multiple projects simultaneously using their views and they also offer summary views to help you combine several projects in a single space, so you can visualize where all the teams are headed. TestMu AI integration with Hive helps to manage your bugs in Hive by moving them into a project through a single click effort. All the details you provide in TestMu AI like task list, assignee, title and description would automatically be presented in the project on Hive. - -The TestMu AI Hive integration helps you in filing issues to your project in Hive directly from TestMu AI platform. With one-click integration, you can push annotated issues to the project of your choice, assign it to the required teammate, and attach necessary screenshots. You can do all that while in the middle of a test session in TestMu AI platform. The fields populated by you when marking as a bug through TestMu AI are displayed as information on the Hive for a testing instance. - -> Hive Integration with TestMu AI, like all of the integrations to 3rd party applications, is available for freemium as well as premium plan. - -## How To Establish Integration With Hive From Your TestMu AI Account? - -**Step 1:** Login to your TestMu AI account. You would need to have Admin or User level access to see and install integrations. - -**Step 2:** Select ‘Integrations’ from the left navigation menu bar. This will guide you to a screen where you will find a list of 3rd party applications, available to integrate with your TestMu AI account. - -**Step 3:** Click on ‘ADD’ under the block that says **‘Hive’**. - -**Step 4:** Once you click on install, you will be redirected to the below screen. You will be asked to provide your Hive User ID and your Hive API Key credentials in the highlighted fields to establish integration with TestMu AI. - -> **What is an API key?** -When two different applications interact with each other, a server-to-server communication is established. API key is needed for authenticating user identity from one server to another, by fetching user-owned resources available on the server from where the communication is initiated. API key has been a necessity for maintaining security and data integrity of any application. They are just as vital to an end user as they are to an application. API keys are relevant if an end user logs in an application through a 2-step verification process. - -**Step 5:** You can find your Hive User ID & API key from your Hive workspace. Click the display picture on the top right and go to **Edit profile**. - -**Step 6:** Click **API info** from the left menu. - -**Step 7**: Copy your **API Key** and **User ID**. - -> You can keep a note of this API key with you for future authentication. Although, in case you forget to do so, or you end up neglecting it due to some reason. Then there is no need to panic! You can always regenerate the key. - -**Step 8:** Paste the copied strings on the field provided at TestMu AI & hit the ‘Next’ button and you are all set to experience one-click bug logging to share your issues directly from your TestMu AI account to your project on Hive. - -That’s it! Go to Integrations again and you will be able to notice a green tick indicating that Hive is successfully installed. - -## How To Log Your First Bug Through Hive Integration? - -**Step 1:** Go for any of the test from the left navigation menu. For demo, we will be taking "Real Time Test" option. - -**Step 2:** Present a URL of the web-app you need to test in the dialog box. After that, select any configuration for browser and operating system of your choice & hit ‘Start‘. - -**Step 3:** After the VM is launched and operable. You can perform testing on your web-app for finding bugs. If a bug gets revealed, then you need to click on the Bug icon from the left panel for [capturing a screenshot](https://www.testmuai.com/full-page-screen-capture) of the same. We have highlighted that controller button in the below image. - -**Step 4:** After a screenshot is captured, you can annotate any issue or a task with an in-built image editor. Once you are done highlighting the bug, click on the button that says "Mark as Bug". - -**Step 5:** After clicking on "Mark as Bug" button a Hive specific form would open up. Fill the fields as per your requirement. - -* You can set the relevant Workspaces for your UI observation. -* You can select which project should the task go under. -* You can assign the bug to a colleague by populating the field ‘Assignee‘. -* You can mention the Title as well to easily organize your bugs. -* You also get to post a description to help relate the cause of the issue or the task. - -**Step 6:** Click on "Create Bug" & observe it being successfully marked through a single click effort. Once you hit the button, you will get prompt messages on top of your Virtual Machine indicating the progress of bug logging. You will get a prompt message in few seconds mentioning **"Bug successfully marked"** indicating that the issue has been successfully pushed to the respective workspace. - -**Step 7:** Visit your Hive dashboard. You will be able to notice the logged issue right away by going to the particular project and task list you assigned it to! - -As you click on the task you will notice that all the data you provided through TestMu AI would already be presented in it. TestMu AI automatically includes test environment details and related screenshots in the task as attachments. - -## How To Uninstall Hive Integration? - -> You can work with one integration at a time. So if you would want to integrate to a similar 3rd party application, then you would have to `uninstall` your current integration. Here is how you can do that. - -**Step 1:** Login to your TestMu AI account. - -**Step 2:** Select ‘Integration’ from the left navigation menu bar. This will guide you to a screen where you will find 3rd party applications, available to integrate with your TestMu AI account. - -**Step 3:** Click on `‘Remove’`. You can find the remove button right under the ‘Hive’ block which would be highlighted with a green tick. - -> That was all you need to know for TestMu AI + Hive Integration. Increase your productivity with our integrations. If you still have any questions for us, please feel free to let us know. Our experts are always available on **chat** to help you out with any roadblock regarding our product. Happy testing! diff --git a/static/docs/how-to-set-the-open-file-limit-while-using-tunnel.md b/static/docs/how-to-set-the-open-file-limit-while-using-tunnel.md deleted file mode 100644 index b9d3acbf9..000000000 --- a/static/docs/how-to-set-the-open-file-limit-while-using-tunnel.md +++ /dev/null @@ -1,45 +0,0 @@ -# How To Set The Open File Limit While Using TestMu AI Tunnel? - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -If you are using a tunnel service with TestMu AI for cross browser testing then you may get the following warning. - -> -*File descriptor limit is too low! Increase your limit to at least 2000.* - -This warning is sent across to help you avoid a lack of available file handles. If this warning is ignored for a long time then you might have issues connecting your tunnel with TestMu AI. This would also be reflected in the TestMu AI logs if you fail to connect your tunnel with TestMu AI. The logs would be reflecting the below error message. - -> -*accept tcp 127.0.0.1:9090: accept: too many open files.* - -We recommend that you expand the open file limit to 2000 for a seamless browser testing experience. - -## How To Expand The Open File Limit Using macOS/Linux? - -You can increase the open file limit either temporarily or permanently for your profile. In case of temporary modification, your profile settings will be restored once the terminal is shut down. - -### Increasing The Open File Limit Temporarily - -**Step 1**: Gain Sudo rights and run the below command to set the open file limit to 2000. - -`ulimit -SHn 2000` - -> Note: If your system is capable of handling a higher limit then please feel free to set the limit above 2000. However, 2000 is the minimum recommendation. - -**Step 2**: Next, we need to verify if the setting are modified by running the below command. - -`ulimit -n` - -**Step 3**: Restart the tunnel and you’re ready to go! - -### Increasing The Open File Limit Permanently - -**Step 1**: You would have to edit the profile settings and set the file limit in the your `.profile` or `.bashrc` file. Add the below command in your `.profile` or `.bashrc` file. - -`ulimit -SHn 2000` - -**Step 2**: Execute `source ~/.bash_profile` or `.bashrc` or the profile you're using. - -**Step 3**: Run the command `ulimit -n` to verify if the setting are modified. - -**Step 4**: Restart the tunnel and you're ready to go! diff --git a/static/docs/how-to-use-in-session-testing-tools.md b/static/docs/how-to-use-in-session-testing-tools.md deleted file mode 100644 index 47fb1a36c..000000000 --- a/static/docs/how-to-use-in-session-testing-tools.md +++ /dev/null @@ -1,103 +0,0 @@ -# How to Use In-Session Testing Tools ? - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -With TestMu AI�s Real Device browser testing, you get access to a plethora of features that helps QA ease their real device browser testing process, without the need of leaving the instance window. From features such as mark as bug and changing IP geolocation, to in-depth Chrome DevTools and Safari Web Inspector, TestMu AI's Real Device browser testing offers it all. - -Let us get an overview of all the features in our real device instance: - -## Comprehensive Sidebar - -### Home - -The **Home** feature on the left-sidebar allows you to go to the home screen of your selected real device instantly. - -### Screenshot - -The **Screenshot** feature allows you to take a screenshot in an instant of the current screen of the device that you are currently testing on. You can access the screenshots and even download them in the gallery section. - -### Record Session - -The **Record Session** feature allows you to screen record the current test session inside the real device. As you click on the button a timer would start, indicating the recording time. You can click on the stop button to stop the video recording. - -Just as your screenshots, you can find your recorded video in the gallery. You can also download the recording by clicking on the download button. - -### Mark As Bug - -The **Mark as Bug** feature allows you to push your UI related issues/edited screenshot to your integrated workspace instantly. - -### Gallery - -All screenshots and video recordings from test sessions are stored in the Gallery. To access, select **Gallery** from the left toolbar and you'll find the saved screenshots and videos.To download the screenshots or videos to your local machine, click on the Download icon. - -### IP Geolocation - -The **IP Geolocation** feature allows you to emulate website and mobile experiences from more than 45 countries using local IP addresses. This enables you to test localized elements like pricing, languages, and product offerings. With IP Geolocation activated, you can verify that your app's traffic is coming from a particular country, facilitating thorough global testing. - -Check out our detailed support documentation to know more about the [IP Geolocation](/support/docs/list-of-supported-locales/) feature. - -### GPS Location - -**GPS Geolocation** testing allows you to simulate specific device locations using exact GPS coordinates to evaluate location-based app scenarios. You can enter your preferred location by entering the name of the location. - -However, if your preferred location is not showing, you can specify the exact location by entering the lattitude and longitude of the location. - -### Network Throttling - -With TestMu AI�s **Network Throttling**, you can test mobile applications across a range of network conditions, including offline, as well as specific network profiles like 3G, 3G+, 4G LTE, 4G Advanced and even custom network profile with its extensive real mobile device cloud. - -You can simulate these network environments using either pre-defined profiles or by creating custom network profiles to suit your needs. - -Check out our detailed support documentation to know more about the [Network Throttling](/support/docs/app-auto-network-throttling/) feature. - -### iOS Settings - -With the **iOS settings**, you can enable the Accessibility Testing. It focuses on ensuring that applications and web content are usable by individuals with different abilities. Learn in detailed way about [Accessibility in iOS devices.](/support/docs/accessibility-settings-ios) - -### Device Controls - -With the **Device Controls** feature, you can easily manage various settings on your device, such as adjusting the volume and changing its orientation. - -Under Device Control, you'll find multiple options to customize your device's settings: - -1. **Volume**: Adjust the device's volume to your preferred level. -2. **Rotate**: Change the screen orientation between landscape and portrait modes. -3. **Recent Apps** : Available on `iOS 15` and above, this feature allows you to view recent apps. Note that it is not supported on devices that rely on a physical button for accessing recent apps instead of the swipe-up gesture. -4. **Bluetooth**: Used to enable the bluetooth functionality during the session. - -### Settings - -The **settings** section offer the following features: - -- **Idle Timeout:** This feature allows you to set the idle timeout for your test session by selecting from the dropdown menu. The default Idle Timeout is 5 minutes and the maximum limit is 45 minutes. -- **Time Zone:** With this feature, you can select the time zone of your preference from the dropdown menu. The time zone are set to GMT by default and can be changed as per preference. -- **Language:** With this feature, you can change the language in your test session for a more localized testing experience. Language locale doc. -- **Projects:** The projects feature allows you to track your existing test session in an existing project. You can create and access your projects under **More Tools**. - -### Switch - -This feature allows you to **switch** between devices, browser and their OS versions without exiting your current test session. - -### End Session - -By clicking on the end session button, you can close your current test session. - -## Comprehensive Top Bar - -### Zoom In-Out - -With the zoom in-out slider, you can zoom in and out in your real device as per your preference. - -### Chrome DevTools - -TestMu AI�s real device browser testing offers in-built developer tools to help QA debug their websites and web applications. - -### Safari Web Inspector - -TestMu AI�s real device browser testing offers in-built **Safari Web Inspector**, an open-source web development tool to simplify optimizing, prototyping, and debugging web applications for iOS and OS X. - -### Test Manager - -Unified **Test Manager** simplifies test authoring, management, triggering, and reporting, boosting efficiency throughout all testing phases. - -> **Notes:** The max duration for running manual tests on TestMu AI Real Device Cloud is 2 hours. In order to extend the duration, you can opt for our dedicated real device cloud. diff --git a/static/docs/how-to-use-testing-tools-in-session.md b/static/docs/how-to-use-testing-tools-in-session.md deleted file mode 100644 index 3569da771..000000000 --- a/static/docs/how-to-use-testing-tools-in-session.md +++ /dev/null @@ -1,135 +0,0 @@ -# How to Use Testing Tools In-Session? - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -With TestMu AI Real Device App Testing, you get access to a plethora of features that helps QA ease their app testing process, without the need of leaving the instance window. From mark as bug and changing IP geolocation, to in-depth device logs and network logs, TestMu AI's real device app testing offers it all. - -Let us get an overview of all the features in our real device instance: - -## Comprehensive Sidebar - -### Home - -The **Home** feature on the left-sidebar allows you to go to the home screen of your selected real device instantly. - -### App Controls - -The **App Control** feature offers you the ability to perform actions with your installed application within your current test session. This feature offers a dropdown with options such as: - -- **Install New App:** By selecting this option you can quickly install multiple applications in your already running test session, without having to quit and start a new session. Without any installation limit. -- **Kill App:** By selecting this option, you can easily kill all your installed applications. -- **Uninstall App:** By selecting this option, you can easily uninstall all the installed applications. - -### Screenshot - -The **Screenshot** feature allows you to take a screenshot of the current screen of the device that you are currently testing on. You can access the screenshots and even download them in the gallery section. - -### Record Session - -The **Record Session** feature allows you to screen record the current test session inside the real device. As you click on the button a timer would start running to indicate the recording time. You can click on the stop button to stop the video recording. - -Just as your screenshots, you can find your recorded video in the gallery. You can also download the recording by clicking on the download button. - -### Mark As Bug - -The **Mark as Bug** feature allows you to push your UI related issues/edited screenshot to your integrated workspace instantly. - -### Gallery - -All screenshots and video recordings from test sessions are stored in the Gallery. To access, select **Gallery** from the left toolbar and you'll find the saved screenshots and videos.To download the screenshots or videos to your local machine, click on the Download icon. - -### IP Geolocation - -The **IP Geolocation** feature allows you to emulate website and mobile experiences from more than 45 countries using local IP addresses. This enables you to test localized elements like pricing, languages, and product offerings. With IP Geolocation activated, you can verify that your app's traffic is coming from a particular country, facilitating thorough global testing. - -Check out our detailed support documentation to know more about the [IP Geolocation](/support/docs/list-of-supported-locales/) feature. - -### GPS Location - -**GPS Geolocation** testing allows you to simulate specific device locations using exact GPS coordinates to evaluate location-based app scenarios. You can enter your preferred location by entering the name of the location. - -However, if your preferred location is not showing, you can specify the exact location by entering the lattitude and longitude of the location. - -### Network Throttling - -With TestMu AI’s **Network Throttling**, you can test mobile applications across a range of network conditions, including offline, as well as specific network profiles like 3G, 3G+, 4G LTE, 4G Advanced and even custom network profile with its extensive real mobile device cloud. - -You can simulate these network environments using either pre-defined profiles or by creating custom network profiles to suit your needs. - -Check out our detailed support documentation to know more about the [Network Throttling](/support/docs/app-auto-network-throttling/) feature. - -### Files Upload - -Real device app testing on TestMu AI comes with preloaded media files on remote devices for testing scenarios that need file access. Additionally, you have the option to upload your own media or non-media files to cover your testing scenarios. You can upload files with formats: JPEG, PNG, MP4, 3GP, PDF, CSV, DOC and other major formats. - -Check out our detailed support documentation to know more about the [Files Upload](/support/docs/upload-media/) feature. - -### iOS Settings - -With the **iOS settings**, you can access the crash logs as well as the app settings. For crash logs, If the app crashes at any point, the crash log gets downloaded automatically. Application settings allows you to access the default application settings in your iOS devices. - -- **Crash Logs :** You can download your crashed logs of the application. -- **App Settings :** This will open the settings of the application in your mobile device. - - The **App Settings** option is shown only for apps that include a Settings Bundle. If the app under test does not have a Settings Bundle, the **App Settings** option is hidden. Legacy apps where the bundle flag is not set continue to show it. - -- **Accessibility :** Accessibility testing on iOS devices focuses on ensuring that applications and web content are usable by individuals with different abilities. Learn in detailed way about [Accessibility in iOS devices.](/support/docs/accessibility-settings-ios) - -- **Select Date and Time :** Opens a modal where you can: - - Toggle **Set Automatically** to sync with network time. - - Manually select a **Date** (up to 7 days ahead). - - Enter a **Time** in **HH:MM:SS** format. - - Choose between **12-hour** or **24-hour** time formats. - - Once configured, click **Update** to apply the changes to the device. - -### Device Controls - -With the **Device Controls** feature, you can easily manage various settings on your device, such as adjusting the volume and changing its orientation. - -Under Device Control, you'll find multiple options to customize your device's settings: - -1. **Volume**: Adjust the device's volume to your preferred level. -2. **Rotate**: Change the screen orientation between landscape and portrait modes. -3. **Recent Apps** : Available on `iOS 15` and above, this feature allows you to view recent apps. Note that it is not supported on devices that rely on a physical button for accessing recent apps instead of the swipe-up gesture. -4. **Bluetooth**: Used to enable the bluetooth functionality during the session. - -### Settings - -The settings section offer the following features: - -- **Idle Timeout:** This feature allows you to set the idle timeout for your test session by selecting from the dropdown menu. The default Idle Timeout is 5 minutes and the maximum limit is 45 minutes. -- **Time Zone:** With this feature, you can select the time zone of your preference from the dropdown menu. The time zone are set to GMT by default and can be changed as per preference. -- **Language:** With this feature, you can change the language in your test session for a more localized testing experience. Language locale doc. -- **Projects:** The projects feature allows you to track your existing test session in an existing project. You can create and access your projects under **More Tools**. - -### Switch - -This feature allows you to **switch** between devices, browser and their OS versions without exiting your current test session. - -### End Session - -By clicking on the end session button, you can close your current test session. - -## Comprehensive Top Bar - -### Zoom In-Out - -With the **zoom in-out** slider, you can zoom in and out in your real device as per your preference. - -### Network Logs - -**Network Logs** record performance metrics including network traffic, latency, and HTTP requests and responses in the HAR (HTTP Archive) format. These logs provide a detailed overview of all network requests, such as REST API calls, from your app and their response times, aiding in the identification of performance bottlenecks and troubleshooting of failed REST API responses. - -### Device Logs - -**Device logs** are application-specific system logs generated by iOS or Android that provide access to your application's logs. They are particularly useful for analyzing issues when the application crashes during testing. - -### UI Inspector - -The Intelligent **UI Inspector** for real device testing streamlines mobile app testing by enabling direct interaction with devices, allowing testers to efficiently inspect app elements without the need of repeatedly opening and closing the inspector. The Intelligent UI Inspector offers the capability to crеatе custom XPaths еffortlеssly for automation test scripts. - -### Test Manager - -Unified **Test Manager** simplifies test authoring, management, triggering, and reporting, boosting efficiency throughout all testing phases. - -> **Notes:** The max duration for running manual tests on TestMu AI Real Device Cloud is 2 hours. In order to extend the duration, you can opt for our dedicated real device cloud. diff --git a/static/docs/html-dom-smartui-options.md b/static/docs/html-dom-smartui-options.md deleted file mode 100644 index 1af3b65d7..000000000 --- a/static/docs/html-dom-smartui-options.md +++ /dev/null @@ -1,210 +0,0 @@ -# Handling Dynamic Data with DOM Configuration and Options for SmartUI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Using the TestMu AI platform, perform regression testing in just one click and explore various possible solutions for grouping your screenshots into different builds and map as per your testing suite needs. - -In case if you have any dynamic elements that are not in the same position across test runs, you can ignore / select a specific area to be removed from the comparison. - -## Configuration for Selenium - -The following are the different options which are currently supported: - -| Key | Description | -| ------------------------- | ------------------------------------------------------------------------------------------------------------------------- | -| `screenshotName` (string) | Specify a name for the screenshot in your tests to match the same screenshot with the name from your baseline | -| `fullPage` (boolean) | Specify `true` if you want to take a Full Page Screenshot and `false` for viewport screenshots; `fullPage: true` is currently only supported for `Chrome` | -| `ignoreDOM` (object) **** | Specify one or a combination of selectors based on the `HTML DOM ID, CSS class, CSS selector or Xpath` used by your webpage that should be excluded from the comparison | -| `selectDOM` (object) | Specify one or a combination of selectors based on the `HTML DOM ID, CSS class, CSS selector or XPath` used by your webpage that should be included in the comparison | - -```js title="This is a sample for your webhook configuration for JavaScript to ignore by ID" -let config = { -screenshotName: "Ignore-ID", -fullPage: false, //You can make this property as true in case of Chrome browser -ignoreDOM: { -id: ["ID-1", "ID-2"], // Ignoring elements by ID, you can ignore multiple at once -}, -}; -await driver.executeScript("smartui.takeScreenshot", config); -``` - -```js title="This is a sample for your webhook configuration for JavaScript to ignore by Class" -let config = { -screenshotName: "Ignore-Class", -fullPage: false, //You can make this property as true in case of Chrome browser -ignoreDOM: { -class: ["Class-1", "Class-2"], // Ignoring elements by class, you can ignore multiple at once -}, -}; -await driver.executeScript("smartui.takeScreenshot", config); -``` - -```js title="This is a sample for your webhook configuration for JavaScript to ignore by XPath" -let config = { -screenshotName: "Ignore-XPath", -fullPage: false, //You can make this property as true in case of Chrome browser -ignoreDOM: { -xpath: ["Xpath-1", "Xpath-2"], // Ignoring elements by XPath, you can ignore multiple at once -}, -}; -await driver.executeScript("smartui.takeScreenshot", config); -``` - -```js title="This is a sample for your webhook configuration for JavaScript to ignore by CSS Selector" -let config = { -screenshotName: "Ignore-cssSelector", -fullPage: false, //You can make this property as true in case of Chrome browser -ignoreDOM: { -cssSelector: ["CSS-Selector-1", "CSS-Selector-2"], // Ignoring elements by CSS selector, you can ignore multiple at once -}, -}; -await driver.executeScript("smartui.takeScreenshot", config); -``` - -```js title="This is a sample for your webhook configuration for JavaScript to select by ID." -let config = { -screenshotName: "Select-ID", -fullPage: false, //You can make this property as true in case of Chrome browser -selectDOM: { -id: ["ID-1", "ID-2"], // Selecting elements by ID, you can select multiple at once -}, -}; -await driver.executeScript("smartui.takeScreenshot", config); -``` - -```js title="This is a sample for your webhook configuration for JavaScript to select by Class" -let config = { -screenshotName: "Select-Class", -fullPage: false, //You can make this property as true in case of Chrome browser -selectDOM: { -class: ["Class-1", "Class-2"], // Selecting elements by class, you can select multiple at once -}, -}; -await driver.executeScript("smartui.takeScreenshot", config); -``` - -```js title="This is a sample for your webhook configuration for JavaScript to select by XPath" -let config = { -screenshotName: "Select-XPath", -fullPage: false, //You can make this property as true in case of Chrome browser -selectDOM: { -xpath: ["Xpath-1", "Xpath-2"], // Selecting elements by XPath, you can select multiple at once -}, -}; -await driver.executeScript("smartui.takeScreenshot", config); -``` - -```js title="This is a sample for your webhook configuration for JavaScript to select by CSS Selector" -let config = { -screenshotName: "Select-cssSelector", -fullPage: false, //You can make this property as true in case of Chrome browser -selectDOM: { -cssSelector: ["CSS-Selector-1", "CSS-Selector-2"], // Selecting elements by CSS selector, you can select multiple at once -}, -}; -await driver.executeScript("smartui.takeScreenshot", config); -``` - -## Configuration for Puppeteer - -The following are the different options which are currently supported: - -| Key | Description | -| ------------------------- | --------------------------------------- | -| `screenshotName` (string) | Specify a name for the screenshot in your tests to match the same screenshot with the name from your baseline | -| `ignoreDOM` (object) | Specify a selector based on the HTML DOM ID, CSS class or CSS selector used by your webpage that should be excluded from the comparison | -| `selectDOM` (object) | Specify a selector based on the HTML DOM ID, CSS class or CSS selector used by your webpage that should be included in the comparison | -| `ignoreXPath` (object) | Specify a selector based on theXPath of the element that should be excluded from the comparison | -| `selectXPath` (object) | Specify a selector based on theXPath of the element that should be included in the comparison | - -```js title="This is a sample for your webhook configuration for Puppeteer" -await page.evaluate((_) => {}, -`lambdatest_action: ${JSON.stringify({ action: 'smartui.takeScreenshot', arguments: { fullPage: false, screenshotName: 'dom-screenshot', ignoreDOM : {id:["HTML DOM Id"]} }})}`) -``` - -```js title="This is a sample for your webhook configuration for Puppeteer" -await page.evaluate((_) => {}, -`lambdatest_action: ${JSON.stringify({ action: 'smartui.takeScreenshot', arguments: { fullPage: false, screenshotName: 'dom-screenshot', ignoreDOM : {class:[""]} }})}`) -``` - -```js title="This is a sample for your webhook configuration for Puppeteer" -await page.evaluate((_) => {}, -`lambdatest_action: ${JSON.stringify({ action: 'smartui.takeScreenshot', arguments: { fullPage: false, screenshotName: 'dom-screenshot', ignoreXPath: ['Xpath-1', 'Xpath-2']} })}`) -``` - -```js title="This is a sample for your webhook configuration for Puppeteer" -await page.evaluate((_) => {}, -`lambdatest_action: ${JSON.stringify({ action: 'smartui.takeScreenshot', arguments: { fullPage: false, screenshotName: 'dom-screenshot', ignoreDOM : {cssSelector:[""]} }})}`) -``` - -```js title="This is a sample for your webhook configuration for Puppeteer" -await page.evaluate((_) => {}, -`lambdatest_action: ${JSON.stringify({ action: 'smartui.takeScreenshot', arguments: { fullPage: false, screenshotName: 'dom-screenshot', selectDOM : {id:["HTML DOM Id"]} }})}`) -``` - -```js title="This is a sample for your webhook configuration for Puppeteer" -await page.evaluate((_) => {}, -`lambdatest_action: ${JSON.stringify({ action: 'smartui.takeScreenshot', arguments: { fullPage: false, screenshotName: 'dom-screenshot', selectDOM : {class:[""]} }})}`) -``` - -```js title="This is a sample for your webhook configuration for Puppeteer" -await page.evaluate((_) => {}, -`lambdatest_action: ${JSON.stringify({ action: 'smartui.takeScreenshot', arguments: { fullPage: false, screenshotName: 'dom-screenshot', selectXPath: ['Xpath-1', 'Xpath-2']} })}`) -``` - -```js title="This is a sample for your webhook configuration for Puppeteer" -await page.evaluate((_) => {}, -`lambdatest_action: ${JSON.stringify({ action: 'smartui.takeScreenshot', arguments: { fullPage: false, screenshotName: 'dom-screenshot', selectDOM : {cssSelector:[""]} }})}`) -``` - -## Configuration for Playwright - -The following are the different options which are currently supported: - -| Key | Description | -| ------------------------- | ------------------------------------------------------------------------------------------------------------------------- | -| `screenshotName` (string) | Specify a name for the screenshot in your tests to match the same screenshot with the name from your baseline | -| `ignoreDOM` (object) | Specify a selector based on the HTML DOM ID, CSS class or CSS selector used by your webpage that should be excluded from the comparison | -| `selectDOM` (object) | Specify a selector based on the HTML DOM ID, CSS class or CSS selector used by your webpage that should be included in the comparison | -| `ignoreXPath` (object) | Specify a selector based on theXPath of the element that should be excluded from the comparison | -| `selectXPath` (object) | Specify a selector based on theXPath of the element that should be included in the comparison | - -```js title="This is a sample for your webhook configuration for Playwright" -await page.evaluate((_) => {}, -`lambdatest_action: ${JSON.stringify({ action: 'smartui.takeScreenshot', arguments: { fullPage: false, screenshotName: 'dom-screenshot', ignoreDOM : {id:["HTML DOM Id"]} }})}`) -``` - -```js title="This is a sample for your webhook configuration for Playwright" -await page.evaluate((_) => {}, -`lambdatest_action: ${JSON.stringify({ action: 'smartui.takeScreenshot', arguments: { fullPage: false, screenshotName: 'dom-screenshot', ignoreDOM : {class:[""]} }})}`) -``` - -```js title="This is a sample for your webhook configuration for Playwright" -await page.evaluate((_) => {}, -`lambdatest_action: ${JSON.stringify({ action: 'smartui.takeScreenshot', arguments: { fullPage: false, screenshotName: 'dom-screenshot', ignoreXPath: ['Xpath-1', 'Xpath-2']} })}`) -``` - -```js title="This is a sample for your webhook configuration for Playwright" -await page.evaluate((_) => {}, -`lambdatest_action: ${JSON.stringify({ action: 'smartui.takeScreenshot', arguments: { fullPage: false, screenshotName: 'dom-screenshot', ignoreDOM : {cssSelector:[""]} }})}`) -``` - -```js title="This is a sample for your webhook configuration for Playwright" -await page.evaluate((_) => {}, -`lambdatest_action: ${JSON.stringify({ action: 'smartui.takeScreenshot', arguments: { fullPage: false, screenshotName: 'dom-screenshot', selectDOM : {id:["HTML DOM Id"]} }})}`) -``` - -```js title="This is a sample for your webhook configuration for Playwright" -await page.evaluate((_) => {}, -`lambdatest_action: ${JSON.stringify({ action: 'smartui.takeScreenshot', arguments: { fullPage: false, screenshotName: 'dom-screenshot', selectDOM : {class:[""]} }})}`) -``` - -```js title="This is a sample for your webhook configuration for Playwright" -await page.evaluate((_) => {}, -`lambdatest_action: ${JSON.stringify({ action: 'smartui.takeScreenshot', arguments: { fullPage: false, screenshotName: 'dom-screenshot', selectXPath: ['Xpath-1', 'Xpath-2']} })}`) -``` - -```js title="This is a sample for your webhook configuration for Playwright" -await page.evaluate((_) => {}, -`lambdatest_action: ${JSON.stringify({ action: 'smartui.takeScreenshot', arguments: { fullPage: false, screenshotName: 'dom-screenshot', selectDOM : {cssSelector:[""]} }})}`) -``` diff --git a/static/docs/http-basic-authentication.md b/static/docs/http-basic-authentication.md deleted file mode 100644 index 884ca227c..000000000 --- a/static/docs/http-basic-authentication.md +++ /dev/null @@ -1,19 +0,0 @@ -# HTTP Basic Authentication - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -HTTP basic authentication represents a challenge and response mechanism to help the server make a request authentication from a client, passing the User ID and password in the URL as the standard HTTP "Authorization" header. - -For example: `https://username:password@www.example.com/example-page.php` - -## Browser Support For HTTP Basic Authentication - -HTTP authentication is incompatible among the majority of the web browser. It is supported by only Google Chrome and Mozilla Firefox. - -| BROWSER | SUPPORT FOR BASIC HTTP AUTHENTICATION | -| ------------ | --------------------------------------------------------------------------------------------------------------------------- | -| Google Chrome | Supported | -| Mozilla FireFox | Although supported, Mozilla Firefox may throw a prompt confirmation. Mozilla Firefox performs a check to evaluate whether basic HTTP authentication is mandatory to access the website. In cases, where it isn’t necessary, Firefox will throw a similar warning: "You are about to log in to the site "www.example.com" with the username "username", but the website does not require authentication. This may be an attempt to trick you." For more information, refer to the [MDN document](https://developer.mozilla.org/en-US/docs/Web/HTTP/Authentication). | -| Safari | Unsupported | -| Edge | Unsupported | -| Internet Explorer | Supported | diff --git a/static/docs/http2-support.md b/static/docs/http2-support.md deleted file mode 100644 index 95ac51bc6..000000000 --- a/static/docs/http2-support.md +++ /dev/null @@ -1,46 +0,0 @@ ---- -id: http2-support -title: HTTP/2 Support in TestMu AI Tunnel -hide_title: false -sidebar_label: HTTP/2 Support -description: Learn how TestMu AI Tunnel supports HTTP/2 for modern, high-performance web application testing. -keywords: - - http2 - - testmu ai tunnel - - performance testing - - web protocol - - automatic proxy -url: https://www.testmuai.com/support/docs/http2-support/ -site_name: TestMu AI -slug: http2-support/ -canonical: https://www.testmuai.com/support/docs/http2-support/ - -# HTTP/2 Support in TestMu AI Tunnel - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Overview - -TestMu AI Tunnel provides out-of-the-box support for HTTP/2, enabling users to test their web applications using the latest web protocol without any additional configuration. HTTP/2 support is essential for performance testing, as it includes improvements such as multiplexing, server push, and header compression. This document provides an overview of HTTP/2 support within TestMu AI Tunnel and its benefits. - -## Key Features - -- **Automatic HTTP/2 Proxying:** TestMu AI Tunnel automatically proxies both HTTP/1 and HTTP/2 traffic, simplifying the testing process for applications that use the latest web protocols. -- **Improved Performance Testing:** With HTTP/2 support, users can test their applications' performance characteristics, such as load times and response behavior, under conditions that mirror modern browser-server communication. -- **Seamless Integration:** No additional flags or configurations are required to enable HTTP/2 support, ensuring a smooth integration into existing testing workflows. - -## Usage - -Using HTTP/2 with TestMu AI Tunnel does not require any special configuration or flags. The tunnel automatically detects and proxies HTTP/2 traffic alongside HTTP/1, ensuring that your tests accurately reflect the behavior of web applications under real-world conditions. - -To start using TestMu AI Tunnel with HTTP/2 support, simply initiate the tunnel as you normally would: - -```sh -./LambdaTestTunnel --user YourLambdaTestUsername --key YourLambdaTestAccessKey -``` - -With the tunnel running, any HTTP/2 traffic between your local development environment and the TestMu AI cloud platform will be automatically proxied, allowing you to conduct thorough performance and functionality testing on your web applications. - -## Conclusion - -The inherent support for HTTP/2 in TestMu AI Tunnel is a testament to TestMu AI's commitment to providing developers and QA professionals with cutting-edge tools for web application testing. By automating the proxying of HTTP/2 traffic, TestMu AI Tunnel ensures that users can effortlessly test their applications in environments that utilize the latest web protocols, leading to faster, more reliable web applications. diff --git a/static/docs/hyperexecute-accelq-integration.md b/static/docs/hyperexecute-accelq-integration.md deleted file mode 100644 index cb4ef4355..000000000 --- a/static/docs/hyperexecute-accelq-integration.md +++ /dev/null @@ -1,228 +0,0 @@ -# Accelq Integration with HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -ACCELQ is a cloud-based, continuous testing platform for functional and API testing needs. It is a codeless test automation tool that allows users to create and execute tests without writing any code. ACCELQ uses a natural language abstraction to generate Java code, which makes it easy to learn and use. - -## Integrate Accleq with HyperExecute - -Accelq is the most powerful software testing tool to achieve Codeless test automation and accelerate continuous testing. - -### Prerequisites - -- Sample scripts in Accelq -- Ensure that you have integrated [Accelq with TestMu AI](/support/docs/accelq-integration/) Cloud Selenium Grid -- Parameters- accelq url, userId, apiKey, tenant code, Job Id -- Agent.properties file for you configured agent with TestMu AI - -### Steps to Integrate - -**Step 1:** Copy your local `agent.properties` file from `` to root folder of your HyperExecute project. - -**Step 2:** Update your YAML file with your account credentials. - -**Step 3:** Execute HyperExcute job using the command: - -``` -./hyperexecute --config hyperdemo.yaml -u -k -``` - -## Additional Details -### Where to Find the Parameters? - -#### AccelQ - URL, User-ID, API key, Tenant Code - -#### How to create a Job ID? - -**Step 1:** Create an LT profile with with the below configuration and save it - -**Step 2:** Create a [CI job](https://support.accelq.com/hc/en-us/articles/360016077871-How-do-I-create-a-job-that-I-can-invoke-from-CI-tool#:~:text=Step%201%3A%20On%20the%20ACCELQ,Click%20on%20%22Create%20Job%22) with the same profile and agent which has TestMu AI configured with it. - -**Step 3:** Copy the Job ID - -### Agent.properties file -Agent.property file is a type of file that gets generated when you configures the agents from the Accelq to the TestMu AI. - -- You can configure your [TestMu AI from Accelq Agent Dashboard](/support/docs/accelq-integration/#integrating-lambdatest-from-accelq-agent-dashboard) -- You can also find the same at the location `(ACCELQAgent/AgentInstances//agent.properties)` in your local system. - -#### Sample File -```bash -# name of the agent, which will appear in your Run modal -# must be a unique name across all projects in the tenant -# only accepts alphabets, numeric and underscore (_) -# Maximum length 32 characters. -agent_name = - -# Port on which the Agent should run. Leave it blank to use an available port dynamically -# agent_port=5000 - -# ACCELQ server url along with port number (if applicable) -accelq_server_url = - -# ACCELQ user id you use for login -user_id = - -# API key (available in the profile menu under the auth properties once you login to ACCELQ) -api_key = - -# Scope of the agent, which defines visibility (User, Project or Tenant) -# "User" Agent is accessible only for the User starting this Agent -# "Project" scope allows sharing the Agent for all Project users -# "Tenant" scope allows all users on the Tenant to use this Agent -agent_type = User - -# Project codes where this Agent should be visible, separated with a comma -# Applicable only when scope of the Agent is set to "Project" -# Project Code can be found in the profile menu under the auth properties once you login to ACCELQ -# Example: project_code=proj1,proj2,proj3 -# project_code= - -# Web Provider Types - LOCALHOST, SAUCE_LABS, BROWSER_STACK, PERFECTO, LAMBDATEST, HEADSPIN, DIGITAL_AI -# Refer to documentation for applicable properties for different providers -web_provider_type = LAMBDATEST -# web_provider_type=LOCALHOST -web_provider_username = -web_provider_password = -# web_provider_cloud_name= -# web_provider_security_token= -# web_provider_serverUrl= -# web_provider_hostname= -# web_provider_port= -# web_provider_data_center_endpoint= - -# Mobile Provider Types - LOCALHOST, SAUCE_LABS, BROWSER_STACK, PERFECTO, LAMBDATEST, HEADSPIN, DIGITAL_AI -# Refer to documentation for applicable properties for different providers -mobile_provider_type = LOCALHOST -# mobile_provider_username= -# mobile_provider_password= -# mobile_provider_cloud_name= -# mobile_provider_security_token= -# mobile_provider_serverUrl= -# mobile_provider_hostname= -# mobile_provider_port= -# mobile_provider_data_center_endpoint= - -# Browserstack Local Settings -# auto_start_bs_local=true -# browserstack_local_key= - -# Number of concurrent jobs to allow -# Any invalid number will be considered as one concurrent job -num_concurrent_jobs = 2 - -# Number of parallel test cases to execute in a given job -# Any invalid number will be considered as single thread -num_parallel_tc_per_job = 4 - -# Proxy settings for HTTP - uncomment following lines if applicable -# http_proxy_host= -# http_proxy_port= -# http_proxy_user= -# http_proxy_password= -# http_proxy_nonproxyhosts= - -# Proxy settings for HTTPS - uncomment following lines if applicable -# https_proxy_host= -# https_proxy_port= -# https_proxy_user= -# https_proxy_password= -# https_proxy_nonproxyhosts= - -# SSL Certificate Verification -# ssl_cert_verification=true - -# Appuim URL -appium_url = http://localhost:4723 - -# TE Settings -# te_start_server=false -# te_server_port=9095 - -# Indicates if the web driver should be auto-updated based on the browser version on the agent machine (local, server_side, off) -# local : Fetch driver jars directly from the Agent host -# server_side : Fetch driver jars from ACCELQ server connection -# off : Do not auto-update the driver jars -# web_driver_auto_update=local - -# Port numbers to use for virtualization servers -# Leave this blank to use ports dynamically. If not, give the range with comma separation (e.g. 9000, 10000) -# Number of available ports should be at least twice the number of concurrent threads running at any point -# virtualization_ports_range= - -# Experimental Flags (Internal use only) -agent_host = LT-MBP-194 -tenant_code = poc -# work_root = file path of agent -# Available Sharing Types: USER (0) or Shared (1) -sharing_type = 0 -# cleanup_run_folders=true -# virtualization_debug_log=true -# run_re_trigger_timeout=30 -# tc_stmt_line_max_count=100000 -# sync_runtime=true -# force_sync_runtime_at_startup=false -# Available Image Qualities: high, medium or low -# image_quality=high - -# Extra Flags -provider_type = LAMBDATEST -provider_password = -provider_username = -``` - -### Sample YAML Files - -#### Single YAML File - -```bash -version: 0.1 -user: -key: -globalTimeout: 150 -testSuiteTimeout: 150 -testSuiteStep: 150 - -autosplit: true -runson: win -concurrency: 1 - -pre: -- move agent.properties C:\Users\ltuser.ghtestVM\Downloads\acc_installer_win\ACCELQAgent\AgentInstances\agent\agent.properties #move your agent file to hyperex machines agent folder -- acc start # start accelq agent on hyperexecute machines - -testDiscovery: -type: raw -mode: remote -command: echo test - -testRunnerCommand: node $env:ACCELQ --url "https://poc.accelq.io" --userID "" --apiKey "" --tenantCode "poc" --jobID ""; C:\Users\ltuser.ghtestVM\Downloads\acc_installer_win\ACCELQAgent\Dashboard\AQAgentControllerShutdown.exe; -``` - -#### Parallel YAML File - -```bash -version: 0.1 -globalTimeout: 150 -testSuiteTimeout: 150 -testSuiteStep: 150 - -runson: win - -matrix: -jobID: ["",""] -agent: [hyper1\agent.properties,hyper2\agent.properties] - -exclusionMatrix: -- jobID: [''] -agent: [hyper2\agent.properties] -- jobID: [''] -agent: [hyper1\agent.properties] - -pre: -- move agent.properties C:\Users\ltuser.ghtestVM\Downloads\acc_installer_win\ACCELQAgent\AgentInstances\agent\agent.properties #move your agent file to hyperex machines agent folder -- acc start # start accelq agent on hyperexecute machines - -testSuites: -- node $env:ACCELQ --url "" --userID "" --apiKey "" --tenantCode "poc" --jobID $jobID; -``` diff --git a/static/docs/hyperexecute-algoqa-integration.md b/static/docs/hyperexecute-algoqa-integration.md deleted file mode 100644 index 4cc5a2edb..000000000 --- a/static/docs/hyperexecute-algoqa-integration.md +++ /dev/null @@ -1,127 +0,0 @@ -# Integrate algoQA with HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -algoQA is an AI Native software testing platform that automates end-to-end tests across web, mobile, desktop, and API applications. It generates optimal test cases, datasets, and executable scripts, reducing manual effort and improving test coverage. - -This document will guide you how to integrate your algoQA project with HyperExecute - -## Prerequisites - -To run the Tests on HyperExecute from your Local System, you are required: - -- Your TestMu AI [Username and Access key](/support/docs/hyperexecute-how-to-get-my-username-and-access-key/) -- [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) in order to initiate a test execution Job . -- Setup the [Environment Variable](/support/docs/hyperexecute-environment-variable-setup/) -- [HyperExecute YAML](/support/docs/hyperexecute-yaml-version0.2/) file which contains all the necessary instructions. - -## Step 1: Setup Your Test Suite - -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** -Download or Clone the code sample for the algoQA from the TestMu AI GitHub repository to run the tests on the HyperExecute. - - View on GitHub - -## Step 2: Setup the CLI in your Test Suite - -After cloning / downloading the sample repo, you need to setup the CLI and the environment variables. - -### Download the HyperExecute CLI - -The CLI is used for triggering the tests on HyperExecute. It is recommend to download the CLI binary on the host system and keep it in the root directory of the suite to perform the tests on HyperExecute. - -You can download the CLI for your desired platform from the below mentioned links: - -| Platform | HyperExecute CLI | -| ---------| ---------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| MacOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -### Setup Environment Variable - -Now, you need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in the [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). - -Run the below mentioned commands in your terminal to setup the CLI and the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -## Step 3: Configure YAML in your Test Suite - -```yaml ---- -version: 0.1 -globalTimeout: 150 -testSuiteTimeout: 150 -testSuiteStep: 150 - -runson: win - -autosplit: true - -retryOnFailure: true -maxRetries: 1 - -concurrency: 2 - -cacheKey: '{{ checksum "pom.xml" }}' -cacheDirectories: -- .m2 - -pre: -- mvn -Dmaven.repo.local=./.m2 dependency:resolve - -report: true -partialReports: -type: json -location: target/ -frameworkName: extent-native - -uploadArtifacts: -- name: FinalReport -path: -- ExtentReports/** - -testDiscovery: -type: automatic -mode: static -args: -featureFilePaths: src/test/java/features/ -frameWork: java - -testRunnerCommand: mvn test `-Dplatname=win `-Dmaven.repo.local=./.m2 `-Dcucumber.features="$test" - -jobLabel: ['HYP', 'algoQA'] -``` - -## Step 4: Amend in your ApplicationSettings.xml file - -In your `ApplicationSettings.xml` file, update the following capabilities: - -- **BrowserType :** It should be Lambda. -- **LambdaTestBuild :** Provide your build name. -- **LambdaTestURL :** Provide your HUB URL. For e.g. `https://username:access-key@hub.lambdatest.com/wd/hub` - -## Step 5: Execute your Test Suite - -> **NOTE :** In case of MacOS, if you get a permission denied warning while executing CLI, simply run **`chmod u+x ./hyperexecute`** to allow permission. In case you get a security popup, allow it from your **System Preferences** → **Security & Privacy** → **General tab**. - -Run the below command in your terminal at the root folder of the project: - -```bash -./hyperexecute --config RELATIVE_PATH_OF_YOUR_YAML_FILE -``` - -OR use this command if you have not exported your username and access key in the step 2. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE `} - -## Step 6: Monitor the Test Execution - -Visit the [HyperExecute Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) and check your Job status. diff --git a/static/docs/hyperexecute-api-testing.md b/static/docs/hyperexecute-api-testing.md deleted file mode 100644 index cafbe4b49..000000000 --- a/static/docs/hyperexecute-api-testing.md +++ /dev/null @@ -1,118 +0,0 @@ -# Boost Your API Testing Efficiency with HyperExeucte - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -An API acts as a middleman between different software applications, allowing them to communicate and exchange data. Just like any other part of a software system, APIs need to be thoroughly tested to ensure they function as intended. - -## Why is API Testing Important? - -API testing is crucial for several reasons: - -- **Ensures application functionality:** It verifies that the API delivers the correct data and behaves as expected under various conditions. -- **Catches bugs early:** By testing APIs early in the development process, you can identify and fix issues before they impact the overall application. -- **Improves application reliability:** Robust API testing leads to a more stable and reliable application for end users. - -## How to Automate API Testing with Rest Assured? - -REST-Assured, a robust Java library, simplifies interacting with RESTful web services for testing purposes. It provides intuitive syntax for constructing requests and validating responses, streamlining API testing within Java environments. - -It is majorly used for: -- Writing automated API tests -- Validating REST API endpoints -- Ensuring API reliability, performance, and security - -REST-Assured seamlessly integrates with popular testing frameworks like JUnit and TestNG, enabling organized and efficient test execution with its core capabilities: - -- Sending various HTTP requests (GET, POST, PUT, DELETE, etc.) -- Parsing various response formats (JSON, XML, etc.) -- Asserting response status codes and body content -- Handling authentication mechanisms -- Supporting data-driven testing with external data sources - -For e.g. REST-Assured provides function to write get method such as `given().when().get(url).then().log().all();` This will help you to call a get request to fetch all the data from the API. Once the Call is complete you can either verify using the response of that call or by checking the status of the request. - -```bash -given().when().get(url).then().assertThat().statusCode(200); -``` - -- **Setting Base URI:** `RestAssured.baseURI` specifies the common part of API endpoints, avoiding repetition. - -- **Constructing Request:** - - **given()** initiates test case setup. - - **when()** sends the HTTP request (GET in this case). -- **Capturing Response:** - - **then()** handles response handling and assertions. - - **log().all()** logs request and response details for debugging. - - **extract().response().asString()** extracts the response body as a string. - -## How to Test API on HyperExecute? - -### Step 1: Setup Your Test Suite - -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** - -Download or Clone the code sample for the Maestro framework from the TestMu AI GitHub repository to run the tests on the HyperExecute. - - View on GitHub - -### Step 2: Setup the CLI in your Test Suite - -After cloning / downloading the sample repo, you need to setup the CLI and the environment variables. - -#### Download the HyperExecute CLI - -The CLI is used for triggering the tests on HyperExecute. It is recommend to download the CLI binary on the host system and keep it in the root directory of the suite to perform the tests on HyperExecute. - -You can download the CLI for your desired platform from the below mentioned links: - -| Platform | HyperExecute CLI | -| ---------| ---------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| MacOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -#### Setup Environment Variable - -Now, you need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in the [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). - -Run the below mentioned commands in your terminal to setup the CLI and the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -### Step 3: Configure YAML in your Test Suite - -```yaml reference title="hyperexecute.yaml" -https://github.com/LambdaTest/hyp-api-testing/blob/master/hyperexecute.yaml -``` - -### Step 4: Execute your Test Suite - -> **NOTE :** In case of MacOS, if you get a permission denied warning while executing CLI, simply run **`chmod u+x ./hyperexecute`** to allow permission. In case you get a security popup, allow it from your **System Preferences** → **Security & Privacy** → **General tab**. - -Run the below command in your terminal at the root folder of the project: - -```bash -./hyperexecute --config RELATIVE_PATH_OF_YOUR_YAML_FILE -``` - -OR use this command if you have not exported your username and access key in the step 2. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE `} - -### Step 5: Monitor and Validate the Test Execution - -Visit the [HyperExecute Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) and check your Job status. - -You can validate the tested API via the pre, post, and scenrios logs in the dashboard. - -#### pre - -#### scenario - -#### post diff --git a/static/docs/hyperexecute-appium-testing.md b/static/docs/hyperexecute-appium-testing.md deleted file mode 100644 index dce83aa8c..000000000 --- a/static/docs/hyperexecute-appium-testing.md +++ /dev/null @@ -1,201 +0,0 @@ -# Appium Testing - Real Device On HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This page outlines how to execute your Appium tests on HyperExecute using TestNG with YAML 0.2 -> HyperExecute uses [YAML 0.2](/support/docs/hyperexecute-yaml-version0.2/) to perform the tests using Appium. - -## Prerequisites - -To run the Tests on HyperExecute from your Local System, you are required: - -- [HyperExecute YAML](/support/docs/hyperexecute-yaml-version0.2/) file which contains all the necessary instructions. -- [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) in order to initiate a test execution Job . -- Your TestMu AI [Username and Access key](/support/docs/hyperexecute-how-to-get-my-username-and-access-key/) -- Setup the Environmental Variable -- Ensure you have Appium’s [Java client library](https://github.com/appium/java-client) installed. -- Access to an **Android** app (.apk or .aab file) or an **iOS** app (.ipa file). - -**Sample repo** -All the code samples in this documentation can be found on **TestMu AI's Github Repository**. You can either download or clone the repository to quickly run your tests. View on GitHub - -> If you do not have any **.apk** or **.ipa** file, you can run your sample tests on TestMu AI by using our sample :link: [Android app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk) or sample :link: [iOS app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_ios.ipa). - -### Download HyperExecute CLI - -The *HyperExecute CLI* is used for triggering tests on HyperExecute. It is recommend to download the HyperExecute CLI binary on the host system to perform the tests on HyperExecute. The CLI download site for various platforms is displayed below: - -| Platform | HyperExecute CLI download location | -| ---------| --------------------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| macOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -### Setup Environment Variable -Export the environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in the [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). -Run the below mentioned commands in the terminal to setup the CLI and the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -## Steps to Run Your Test - -### Step 1: Upload your Application -Upload your _iOS_ application (.ipa file) or _android_ application (.apk file) to the TestMu AI servers using our REST API. You need to provide your Username and AccessKey in the format `Username:AccessKey` in the cURL command for authentication. Make sure to add the path of the appFile in the cURL request. Here is an example cURL request to upload your app using our REST API: - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" --location --request POST 'https://manual-api.lambdatest.com/app/upload/realDevice' --form 'name="Android_App"' --form 'appFile=@""' -`} - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "appFile=@""" -`} - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" \\ ---location --request POST 'https://manual-api.lambdatest.com/app/upload/realDevice' \\ ---form 'name="Android_App"' \\ ---form 'url="https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk"'`} - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" --location --request POST "https://manual-api.lambdatest.com/app/upload/realDevice" --header "Content-Type: application/x-www-form-urlencoded" --data-urlencode "url=:https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk" --data-urlencode "name=Proverbial_App"`} - -> Response of above cURL will be a **JSON** object containing the `App URL` of the format - ``lt://APP123456789123456789`` and will be used in the next step. - -### Step 2: Configure your Test Script -Write Your Automation Script in the client language of your choice from the ones supported by Appium. An automation script for the sample applications have been provided below. - -Here is a sample automation script in Java for the sample app downloaded above. Ensure to update the `app_url`, `username` and `accesskey` in the below code. - -```java reference title="AndroidApp.java" -https://github.com/LambdaTest/hyperexecute-real-device-appium-testng/blob/main/src/main/java/AndroidApp.java -``` - -```java reference title="iOSApp.java" -https://github.com/LambdaTest/hyperexecute-real-device-appium-testng/blob/main/src/main/java/iOSApp.java -``` - -### Step 3: Update your XML file -Create `.XML` file in order to run your test and define device capabilities. Please find sample code below for the same. - -```xml reference title="android-parallel.xml" -https://github.com/LambdaTest/hyperexecute-real-device-appium-testng/blob/main/src/test/java/android-parallel.xml -``` - -```xml reference title="ios-parallel.xml" -https://github.com/LambdaTest/hyperexecute-real-device-appium-testng/blob/main/src/test/java/ios-parallel.xml -``` - -### Step 4: Configure YAML and Execute your Script - -```yaml reference title="android-parallel.yaml" -https://github.com/LambdaTest/hyperexecute-real-device-appium-testng/blob/main/yaml/android/hyp-rd-android-multiple.yaml -``` - -```yaml reference title="ios-parallel.yaml" -https://github.com/LambdaTest/hyperexecute-real-device-appium-testng/blob/main/yaml/ios/hyp-rd-ios-multiple.yaml -``` - -> The ```region``` parameter specifies the region or location where the Appium tests will be executed. Our platform supports the following three regions: -- ap (Asia-Pacific) -- us (United States) -- eu (European Union) - -### Step 5: Execute your Test Suite -> **NOTE :** In case of MacOS, if you get a permission denied warning while executing CLI, simply run **`chmod u+x ./hyperexecute`** to allow permission. In case you get a security popup, allow it from your **System Preferences** → **Security & Privacy** → **General tab**. - -Run the below command in your terminal at the root folder of the project: - -```bash -./hyperexecute --config RELATIVE_PATH_OF_YOUR_YAML_FILE -``` - -OR use this command if you have not exported your username and access key in the step 2. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE `} - -## Configure Smart UI - -To configure SmartUI with HyperExecute, you need to simply add a capability in your test file: - -```bash java -"smartUI.project": "" -``` - -> Refer to the [Automation Capabilities Generator](https://www.testmuai.com/capabilities-generator/) to understand how to write the capability in different languages. - -## More About Desired Capabilities -Sample Capabilities for both android and iOS are mentioned below - - -```java -{ -"deviceName": "Galaxy Tab S4", -"platformName": "android", -"platformVersion": "10", -"app": "App_url", -"visual": True, -"console": True, -"deviceOrientation": "PORTRAIT", -"build": "new-12", -"isRealMobile": True, -} -``` - -```java -{ -"deviceName": "iPhone 12 Mini", -"platformName": "ios", -"platformVersion": "14", -"app": "App_url", -"isRealMobile": True, -"visual": True, -"console": True, -"build": "lt-web-4", -"network": True, -} -``` - -> For more details, please refer to our guide on [Desired Capabilities in Appium](/support/docs/desired-capabilities-in-appium/). - -## Navigation in Automation Dashboard - -Every test run on the HyperExecute has a unique *jobId* associated with it. Each *jobId* can in turn constitute single (or multiple) *groupId*(s). You can visit [HyperExecute automation dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/hyperexecute/) for checking the status of the test execution. - -The snapshot below shows how to navigate to the respective *testID* for viewing the Selenium logs: - -## Using the Appium Agent Skill with TestMu AI - -The [appium-skill](https://github.com/LambdaTest/agent-skills/tree/main/appium-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The appium-skill package includes: - -``` -appium-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing Appium Agent Skill - -Install a Appium Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/appium-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/appium-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only appium-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). diff --git a/static/docs/hyperexecute-appium-virtual-device.md b/static/docs/hyperexecute-appium-virtual-device.md deleted file mode 100644 index 62d3f599b..000000000 --- a/static/docs/hyperexecute-appium-virtual-device.md +++ /dev/null @@ -1,96 +0,0 @@ -# Appium Testing - Virtual Device On HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This page outlines how to execute your Appium tests on HyperExecute using TestNG with YAML 0.2 -> HyperExecute uses [YAML 0.2](/support/docs/hyperexecute-yaml-version0.2/) to perform the tests using Appium. - -## Prerequisites - -To run the Tests on HyperExecute from your Local System, you are required: - -- [HyperExecute YAML](/support/docs/hyperexecute-yaml-version0.2/) file which contains all the necessary instructions. -- [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) in order to initiate a test execution Job . -- Your lambdatest [Username and Access key](/support/docs/hyperexecute-how-to-get-my-username-and-access-key/) -- Setup the Environmental Variable -- Ensure you have Appium’s [Java client library](https://github.com/appium/java-client) installed. -- Access to an **Android** app (.apk or .aab file) - -**Sample repo** -All the code samples in this documentation can be found on **TestMu AI's Github Repository**. You can either download or clone the repository to quickly run your tests. View on GitHub - -> If you do not have any **.apk** file, you can run your sample tests on TestMu AI by using our sample :link: [Android app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk) - -### Download HyperExecute CLI - -The *HyperExecute CLI* is used for triggering tests on HyperExecute. It is recommend to download the HyperExecute CLI binary on the host system to perform the tests on HyperExecute. The CLI download site for various platforms is displayed below: - -| Platform | HyperExecute CLI download location | -| ---------| --------------------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| macOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -### Setup Environment Variable -Export the environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in the [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). -Run the below mentioned commands in the terminal to setup the CLI and the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -## Steps to Run Your Test - -### Step 1: Upload your Application -Upload your _android_ application (.apk file) to the TestMu AI servers using our REST API. You need to provide your Username and AccessKey in the format `Username:AccessKey` in the cURL command for authentication. Make sure to add the path of the appFile in the cURL request. Here is an example cURL request to upload your app using our REST API: - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" --location --request POST 'https://manual-api.lambdatest.com/app/upload/virtualDevice' --form 'name="Android_App"' --form 'appFile=@""' -`} - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/virtualDevice" -F "appFile=@""" -`} - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" \\ ---location --request POST 'https://manual-api.lambdatest.com/app/upload/virtualDevice' \\ ---form 'name="Android_App"' \\ ---form 'url="https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk"'`} - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" --location --request POST "https://manual-api.lambdatest.com/app/upload/virtualDevice" --header "Content-Type: application/x-www-form-urlencoded" --data-urlencode "url=:https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk" --data-urlencode "name=Proverbial_App"`} - -> Response of above cURL will be a **JSON** object containing the `App URL` of the format - ``lt://APP123456789123456789`` and will be used in the next step. - -### Step 2: Configure your Test Script -Write Your Automation Script in the client language of your choice from the ones supported by Appium. An automation script for the sample applications have been provided below. - -Here is a sample automation script in Java for the sample app downloaded above. Ensure to update the `app_url`, `username` and `accesskey` in the below code. - -```java reference title="AndroidEmulator.java" -https://github.com/LambdaTest/hyperexecute-appium-testng/blob/android-emulator/src/main/java/AndroidEmulator.java -``` - -### Step 3: Update your XML file -Create `.XML` file in order to run your test and define device capabilities. Please find sample code below for the same. - -```xml reference title="android-emulator.xml" -https://github.com/LambdaTest/hyperexecute-appium-testng/blob/android-emulator/src/test/java/android-emulator.xml -``` - -### Step 4: Configure YAML and Execute your Script -```yaml reference title="android-emulator.yaml" -https://github.com/LambdaTest/hyperexecute-appium-testng/blob/android-emulator/yaml/hyp-android-emulator.yaml -``` - -### Step 5: Execute your Test Suite -> **NOTE :** In case of MacOS, if you get a permission denied warning while executing CLI, simply run **`chmod u+x ./hyperexecute`** to allow permission. In case you get a security popup, allow it from your **System Preferences** → **Security & Privacy** → **General tab**. - -Run the below command in your terminal at the root folder of the project: - -```bash -./hyperexecute --config RELATIVE_PATH_OF_YOUR_YAML_FILE -``` - -OR use this command if you have not exported your username and access key in the step 2. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE `} diff --git a/static/docs/hyperexecute-artifacts-url.md b/static/docs/hyperexecute-artifacts-url.md deleted file mode 100644 index 4a4754e2e..000000000 --- a/static/docs/hyperexecute-artifacts-url.md +++ /dev/null @@ -1,59 +0,0 @@ -# Test Artifacts Management - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -HyperExecute now allows you to access your _Artifacts_ directly from the HyperExecute UI and create URLs for them. - -## Navigating Artifacts - -To view or download your Artifacts, click the **Artifacts** button. - -### For a Single file - -If you only have one file as an _Artifact_, then a preview of the file will be opened on your browser. - -### For Multiple files - -If you have multiple files as an _Artifact_, then you will get an option to download all the files directly. - -## Managing Test Artifacts URL - -To make the most of this feature, follow the steps below. - -> **Prioritize**: To access your _Artiacts_ for a particular _Job ID_, they must have a name. - -### 1. Constructing the URL - -The URL to view an _Artifact_ in a standard job is: - -`https://hyperexecute.lambdatest.com/artifact/view/?artifactName=` - -- Insert the **_Job ID_** of your test and the name of your _Artifact_ in place of the `` and `` placeholder values. - -- This URL will take you to a page where you can access the _Artifact_ and its contents. - -For example, if you want to access the _Artifacts_ for a job with _Job ID_: `2d835a95-e7ea-4ea7-83e8-608c2f71d95c`, and the name of your _Artifact_ is `ExecutionSnapshots`, then your URL should be: - -`https://hyperexecute.lambdatest.com/artifact/view/`**2d835a95-e7ea-4ea7-83e8-608c2f71d95c**`?artifactName=`**ExecutionSnapshots** - -### 2. Additional Filters - -You can also use this feature to cycle through the files in your _Artifacts_. - -**1. `filter=*.log`:** You can filter through all of your files by adding this at the end of your URL. However, you will have to insert the entire path of your _Artifacts_ folder or the names of the folder you want to use the filter in. - -- For example, if you want to filter through the `.html` files of a folder called `site` with the path `/target/site/`, for the same job that was used above, then your URL will be: - -`https://hyperexecute.lambdatest.com/artifact/view/2d835a95-e7ea-4ea7-83e8-608c2f71d95c?artifactName=ExecutionSnapshots&`**filter=target/site/*.html** - -**2. `view=flat`:** If you add this at the end of the Artifact URL, then you will be able to view your artifacts in a list instead of a tree structure. - -- For example, if you want to view the results of the filter used above in a flat manner, then your URL will be: - -`https://hyperexecute.lambdatest.com/artifact/view/2d835a95-e7ea-4ea7-83e8-608c2f71d95c?artifactName=ExecutionSnapshots&filter=target/site/*.html&`**view=flat** - -**3. `&taskId=`:** If you want to access the Artifacts for a particular task, then add this in your URL. - -- For example, if you want to view the _Artifacts_ for a task with _Task ID_ `df8604cf-7af2-48b6-903d-786028078640` for the same job that was used above, then your URL will be: - -`https://hyperexecute.lambdatest.com/artifact/view/2d835a95-e7ea-4ea7-83e8-608c2f71d95c?artifactName=ExecutionSnapshots&`**taskId=2731df9f-bc9f-4fbc-85c4-fe675aeffeb7** diff --git a/static/docs/hyperexecute-artifacts.md b/static/docs/hyperexecute-artifacts.md deleted file mode 100644 index 50a0c10fe..000000000 --- a/static/docs/hyperexecute-artifacts.md +++ /dev/null @@ -1,48 +0,0 @@ -# Job Artifacts - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Artifacts are files generated during test execution, including reports and logs. You can refer to these files for obtaining consolidated information about the executed test cases, and can seamlessly interact with artifacts through the HyperExecute UI. - -Following the completion of a Job, these artifacts are automatically stored in the **artifacts folder**, distinguishing previous files into **old-artifacts** on subsequent runs while retaining the recent files in artifacts. - -## Steps to Generate and Download Artifacts -Follow the below mentioned steps to generate the Artifacts for your desired framework: - -**Step 1:** Go to the HyperExecute YAML file - -**Step 2:** Add the below mentioned parameters in the YAML file to generate the Artifacts - -```yaml -uploadArtifacts: -- name: Executed-Job-Artifacts #user defined name of the artifact -path: -- src/test/index.html #path of the generated artifact -- name: My-Job-Artifacts -path: -- src/collect/info.html -``` - -- If you are generating multiple artifacts, then it is recommended to use this parameter, [`mergeArtifacts`](/support/docs/deep-dive-into-hyperexecute-yaml/#mergeartifacts) in your YAML, as it will zip all your Artifacts files while downloading from the dashboard. - -- You can replace the Artifacts folder name value with any of the Matrix parameters value. - -```yaml -matrix: -os: [win] -browser: ["chrome-latest-1", "firefox-latest-2"] - -mergeArtifacts: true -uploadArtefacts: -- name: $browser -path: -- allure-results/ -``` - -As shown in the above code, the artifacts folder generated will be named **chrome-latest-1** and **firefox-latest-2** - -**Step 3:** Now trigger your job and go to the [HyperExecute](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute/jobs) dashboard. - -**Step 4:** Click on any of the artifacts generated. You can also download the artifacts from the dashboard - -Here is the sample artifact generated. diff --git a/static/docs/hyperexecute-auto-healing.md b/static/docs/hyperexecute-auto-healing.md deleted file mode 100644 index 97dcc5ec8..000000000 --- a/static/docs/hyperexecute-auto-healing.md +++ /dev/null @@ -1,70 +0,0 @@ -# Auto Healing for HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -HyperExecute's Auto Healing feature automatically recovers from locator failures during test execution, reducing flaky tests and improving the reliability of your test suites. - -## Enable Auto Healing - -**Step 1:** To enable the Auto Healing feature, you need to pass the **`autoHeal: true`** as a capability in your WebDriver configuration. - -```javascript -const capability = { -"browserName": "Chrome", -"LT:Options": { -"platformName": "Windows 10", -# highlight-next-line -"autoHeal": true -} -} -``` -**Step 2:** Now, trigger your test on HyperExecute using the mentioned command. Please replace the placeholder values with your details. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE `} - -**Step 3:** Now visit the HyperExecute Dashboard to check your triggered test. Click on the **View Test Detail** button. - -**Step 4:** Click on the **band-aid** icon as shown below to check what part of your test case is healed. - -## How Auto Healing Works - -Auto Healing employs a dynamic locator strategy that adapts to DOM (Document Object Model) changes in real-time: - -1. **Record:** When an element is initially located, the system records its DOM path and associated attributes. -2. **Detect:** If subsequent attempts to find the element fail due to application updates, the Auto Healing mechanism activates. -3. **Heal:** It analyzes the current DOM structure, compares it against the previously recorded state, and generates new locators by combining element attributes and hierarchical information. -4. **Continue:** The test continues execution seamlessly despite locator changes. - -## Using AutoHeal with Hooks - -In addition to enabling AutoHeal globally through capabilities, HyperExecute also supports **AutoHeal Hooks** for fine-grained control during test execution. - -These hooks allow you to **start or stop the AutoHeal mechanism dynamically** at any point within your Selenium test scripts. This is especially useful when only certain test steps involve dynamic elements or frequently changing locators. - -By using these hooks, you can: -- Start AutoHeal before performing actions on elements likely to change. -- Stop AutoHeal immediately after those interactions are completed. -- Ensure the rest of your test executes with standard Selenium behavior. - -This provides precise control over when element healing should be applied, improving reliability and reducing flakiness without impacting unrelated steps. - -> To learn how to use AutoHeal hooks in your Selenium scripts, refer to the detailed guide: [How to Use AutoHeal using Hooks](/support/docs/autoheal-with-hooks/) - -## Benefits of Auto Healing - -- **Reduces Flaky Tests:** Automatically recovers from intermittent locator failures caused by DOM changes. -- **Improves Test Reliability:** Tests continue execution despite minor application updates, increasing confidence in results. -- **Saves Time and Effort:** Reduces manual effort spent debugging and fixing broken locators. - -## Limitations - -- **Non-recoverable Errors:** Auto Healing cannot recover from WebDriver initialization errors or system-level failures. -- **Test Accuracy:** May mask real issues in your application or test scripts. Review logs to understand why a test needed healing. -- **Performance Impact:** Minimal, but enabling Auto Healing can slightly increase test execution time. - -## Related Docs - -- [Selenium Auto Healing](/support/docs/auto-healing/):Auto-heal for Selenium web automation tests -- [Playwright Auto Healing](/support/docs/playwright-auto-healing/):Auto-heal for Playwright web automation tests -- [Smart Heal for Appium](/support/docs/smart-heal-appium/):AI-powered self-healing for mobile app automation on real devices -- [Auto-Heal in KaneAI](/support/docs/kaneai-auto-heal/):AI-native auto-heal with natural language understanding for KaneAI-authored tests diff --git a/static/docs/hyperexecute-auto-split-strategy.md b/static/docs/hyperexecute-auto-split-strategy.md deleted file mode 100644 index 4eb02c8b8..000000000 --- a/static/docs/hyperexecute-auto-split-strategy.md +++ /dev/null @@ -1,80 +0,0 @@ -# AutoSplit Strategy - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The Smart **Auto Split Strategy** enables you to distribute tests across multiple virtual machines efficiently. This strategy ensures optimal utilization of concurrency by allowing you to split tests based on various levels, such as files, modules, and scenarios. - -## Key Benefits of Auto Split Strategy -- Efficient distribution of tests across nodes. -- Fine-grained control over test distribution. -- Maximization of concurrency for faster results. -- [Automatic reordering of test cases](/support/docs/hyperexecute-auto-split-strategy/#automatic-reordering-of-test-cases), prioritizing failed ones for faster feedback in subsequent executions. - -> HyperExecute provides flexibility to distribute tests at file level, scenario level, module level, and practically anything, provided you have a way to list down all the possible values of the entity that has to be distributed. - -## Enabling Auto-Test Splitting - -For enabling auto-test splitting, set the `autosplit` key to *true* in the HyperExecute YAML file. - -``` yaml -autosplit: true -``` - -Along with that, you need to provide [`testDiscovery`](/support/docs/deep-dive-into-hyperexecute-yaml/#testdiscovery) and [`testRunnerCommand`](/support/docs/deep-dive-into-hyperexecute-yaml/#testrunnercommand) flags in your YAML file as well. - -### Configuration Parameters - -#### 1. `concurrency` -The `concurrency` key indicates the total number of concurrent sessions that can run in parallel. With both of these commands (`testDiscovery` and `testRunnerCommand`), HyperExecute intelligently distributes tests over multiple virtual machines as specified by the `concurrency` directive in the YAML file. - -For example, assume that the total number of test scenarios [discovered via `testDiscovery`] are 27 and `concurrency` is set to 7. In this case, HyperExecute would allocate 7 nodes for running the 27 tests in parallel. - -``` yaml -concurrency: 7 -``` - -#### 2. `testDiscovery` -- It is used to list down all the values to be distributed. HyperExecute will execute the **`testDiscovery`** command verbatim and expects distinct values to be on separate lines. It can be as simple as a list of files on different lines. -- This can be used later to split tests over files, modules, or any level supported by your [language and framework](/support/docs/hyperexecute-supported-languages-and-frameworks/). -- It is used for locating/discovering the tests that are a part of the project. You can use the combination of *grep*, **`awk`**, **`sed`**, or any valid command to locate the tests that you intend to run. For NodeJS, usually at file level, for Python at class level, in Java at class and module level. For Cucumber based frameworks at scenario level or feature level or at tag level. - -A few samples are given below: - -```yaml -# The following command (or value) when assigned to testDiscoverer key searches for the scenarios by matching the string Scenario [or Scenario Outline] in the .feature files located in the *src* directory of the project. -testDiscovery: -type: raw -mode: remote -command: grep -nri 'Scenario:\^|Scenario Outline:' src -ir --include=\*.feature | awk '{print $1}' | sed 's/\.\///g' | sed 's/\(.*\):/\1 /' -``` - -```yaml -testDiscovery: -type: raw -mode: remote -command: grep 'class name' testng.xml | awk '{print$2}' | sed 's/name=//g' | sed 's/\x3e//g' -``` - -#### 3. `testRunnerCommand` -- It is used to run a single item from the **`testDiscovery`**. You need to add **`$test`** in the `testRunnerCommand` for interpolating one of the values from the `testDiscovery`. -- It tells the system how to run a single test entity in isolation. This entity could be a file, module, feature or scenario. Test runner command will run over each of the values extracted from the `testDiscovery` command. - -## Advanced Concepts - -### Automatic Reordering of Test Cases - -Suppose you are performing regression testing in your large test suite project via github actions, but some test cases are failing unexpectedly for no reason. These flaky tests have become a time-consuming nightmare for QA and developers. After fixing the issues, you trigger the testing pipeline again, but you have to wait to check the status of that particular test. - -It would be much better if those flaky tests ran first in the sequence, so you don't have to wait longer to see the status of your previously failed tests. - -That's exactly the problem HyperExecute's autosplit with automatic re-ordering aims to solve! It's like having a testing assistant who remembers which tests were tricky last time. - -#### How does it work? -When you run tests with HyperExecute's autosplit feature, your tests are divided up and run simultaneously on multiple machines, executing them in parallel. This is great for speeding things up, but during this run, HyperExecute keeps track of which tests fail. - -After the initial run is complete, HyperExecute takes note of the failing tests. It may use this information to automatically reorder your tests for the next time you run them. - -On subsequent runs, HyperExecute might prioritize the failing tests from the previous run for faster feedback. - -**info** -Automatic reordering is a behind-the-scenes feature that optimizes test execution. You don't need to specifically configure it; it might happen automatically as part of the autosplit process. It's like a hidden perk that can significantly improve your testing experience. diff --git a/static/docs/hyperexecute-azure-associate-test-case.md b/static/docs/hyperexecute-azure-associate-test-case.md deleted file mode 100644 index ea632a495..000000000 --- a/static/docs/hyperexecute-azure-associate-test-case.md +++ /dev/null @@ -1,35 +0,0 @@ -# Step 2: How to Associate the Test Cases - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -> Test Plan works in Association with specific [**Visual Studio subscriptions**](https://learn.microsoft.com/en-us/azure/devops/test/create-a-test-plan?view=azure-devops#:~:text=Basic%20%2B%20Test%20Plans%20access%20level%20to%20add%20test%20plans%20and%20test%20suites%2C%20delete%20test%20artifacts%2C%20and%20define%20test%20configurations.%20Or%2C%20have%20one%20of%20the%20following%20Visual%20Studio%20subscriptions). - -After setting up a Test Plan, now you need to associate your test cases as well with the Test Plan. To perform the same, we must have a Visual Studio IDE installed in our local machine. Login with same credentials for Visual Studio as in Test Plan. - -Follow the steps below to compelete the test case association: - -**Step 1:** In your Visual Studio Code IDE, click on a **sln** file in a **C# project**. - -> **NOTE :** Login with the same credentials for Visual Studio as in the Test Plan. - -**Step 2:** Navigate to the Team Explorer view. - -**Step 3:** Click the **Home** icon and pull down the list to the right of Home icon. - -- Select the **Projects and My Teams** >> **Manage Connections**. - -- Click on **Manage Connections** >> **Connect to a Project**. - -**Step 4:** Select your repository, click the right button of your mouse, and then click the **Connect** button. - -> **NOTE :** The connection might ask you to connect with GitHub; please provide access to GitHub as well. - -**Step 5:** Once connected, click on the **Test** menu, then **Test Explorer**. - -**Step 6:** Inside the Test Explorer, select any test, right click and select **Associate to Test Case**. - -**Step 7:** Enter the **Test Case ID**, click on **Add Association**, and then click the **Save** button. This will associate the test with the Test Case ID in the Test Plan. - -**Step 8:** You can check the associated test in test plan by hovering on three dots beside test case name and click on Open Test Case. - -Associated test cases will look like this: diff --git a/static/docs/hyperexecute-azure-setup-test-plan.md b/static/docs/hyperexecute-azure-setup-test-plan.md deleted file mode 100644 index 53cda5ffa..000000000 --- a/static/docs/hyperexecute-azure-setup-test-plan.md +++ /dev/null @@ -1,19 +0,0 @@ -# Step 1: Setup the Azure Test Plan - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -> You need to first [**setup the Azure Test Plan**](https://learn.microsoft.com/en-us/azure/devops/test/create-a-test-plan?view=azure-devops#prerequisites) Project before integrating HyperExecute with it. - -**Step 1:** Login to your [Azure DevOps](https://dev.azure.com/) account. - -**Step 2:** Click on the New Project button. Enter the basic details like **Project Name**, **Visibility**. Click on **Create Project** button. - -**Step 3:** Click on Test Plans option in the left pane. Then, click on New Test Plan to start creating new Test Plan. - -**Step 4:** Enter the **Name** of the Test Plan, choose the **Area Path**, and select the **iteration**. Click on Create button. - -**Step 5:** Now, hover over the test plan name and click on the three-dots. Click on **New Suite** >> **Static Suite**. Set a name for the Suite. - -**Step 6:** Your test suite is now created. Click on **New Test Case** button and create a new test case. Once created, click on **Save and Close**. - -Once created, you will see a screen something similar to below diff --git a/static/docs/hyperexecute-azure-test-plan.md b/static/docs/hyperexecute-azure-test-plan.md deleted file mode 100644 index ac4dd98d4..000000000 --- a/static/docs/hyperexecute-azure-test-plan.md +++ /dev/null @@ -1,32 +0,0 @@ -# Azure Test Plan Integration with HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -> It is a **Premium** feature of Azure. You will have to buy access to [Azure Test Plans](https://learn.microsoft.com/en-us/azure/devops/organizations/billing/buy-access-tfs-test-hub?view=azure-devops-2022#buy-monthly-access-to-azure-test-plans) - -In this fast-paced world of software development, ensuring fast and quality testing cycle in not an option anymore, but a necessity. Imagine a scenario where your team is diligently working on a project, but managing test plans, suites, spreadsheets, and cases becomes a maze of complexity. This is where Azure Test Plan comes in as your bestfriend. - -Azure Test Plan is a Microsoft Azure cloud-based testing service that allows you to manage test plans, test suites, and test cases for your web and desktop applications. Using test plans, you can Azure Test Plans also provides a browser extension for exploratory testing and gathering feedback from stakeholders. - -It includes other features as well to assist you in improving the quality of your software, including: - -- **Test Case Management:** You can create, edit, and organize your test cases in a central location. -- **Test Execution:** Run your test cases manually or using automated tools. -- **Defect Tracking:** You can track and manage bugs and other issues found during testing. -- **Reporting:** Generate reports to track your testing progress and identify areas for improvement. - -## Steps to Integrate Test Plan with HyperExecute - -Integrate the Azure Test Plan with HyperExecute in just 3 simple steps 🚀 - -### Step 1: Setup the Azure Test Plan - -You will have to [setup your test plan](/support/docs/hyperexecute-azure-setup-test-plan/) in the Azure Test Plan Dashboard. - -### Step 2: Associate the Test Cases with Azure Test Plan - -Now, [associate your test cases](/support/docs/hyperexecute-azure-associate-test-case/) in order to integrate it. - -### Step 3: Integrate the Test Plan with HyperExecute - -Now, [integrate your associated test plan](/support/docs/hyperexecute-integrate-azure-test-plan) with HyperExecute diff --git a/static/docs/hyperexecute-background-services.md b/static/docs/hyperexecute-background-services.md deleted file mode 100644 index 230912c96..000000000 --- a/static/docs/hyperexecute-background-services.md +++ /dev/null @@ -1,59 +0,0 @@ -# Background Services - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Background Services is a feature of HyperExecute which is generally used to run the long running **Application Servers**. You can trigger the background services by using either the `background` or `backgroundDirectives` property in the [HyperExecute YAML](/support/docs/deep-dive-into-hyperexecute-yaml/#background) file. - -This feature is useful for the following cases: - -- Web Applications or Servers. -- Running databases. -- Providing file sharing or streaming services. -- Running messaging or chat applications. -- Providing APIs for other applications to use. - -## Enable Background Service -There are two methods to enable the **Background Service** from the HyperExecute YAML file: - -### 1. Using `backgroundDirectives` - -```yaml -backgroundDirectives: -shell: bash -commands: -- name: YOUR_SERVICE_NAME -command: npx static-server -- name: Database -command: mysql-server -``` -- `shell`: defines the terminal it should use to run the background service. -- `command`: specifies the command to be executed in the background. -- `name`: specifies the name of your Background Service. - -### 2. Using `background` - -```yaml -background: -- npx static-server -- mysql-server -``` - -## Background Service Logs - -To check the background service logs, follow the below mentioned steps: - -**Step 1:** After executing your job, go to the [HyperExecute Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute/jobs) to check the status of the job. - -**Step 2:** Click on the **Background Service** button. - -**Step 3:** Click on the desired **background service** you have triggered to check its logs. - -## Frequently Asked Questions - -Is the service intended to operate within the same Virtual Machine or externally? Yes, the service will run within the same Virtual Machine. - -What is the service's commencement and cessation schedule? It initiates the execution of all background commands concurrently with the pre-stage and continues until the completion of post-run command execution. - -Will the Database, for example, run on a single VM and be accessible for testing on other VMs? No, the background service can only be accessed from within the same VM. - -Will the identical set of background services be initiated within every VM? Yes, if the same set of background services, such as **`npm run server`**, are establishing HTTP servers on the same port, it is likely that one of the commands will encounter a binding issue and fail. diff --git a/static/docs/hyperexecute-bidi-testing.md b/static/docs/hyperexecute-bidi-testing.md deleted file mode 100644 index 63352701c..000000000 --- a/static/docs/hyperexecute-bidi-testing.md +++ /dev/null @@ -1,85 +0,0 @@ -# Execute BiDi Framework Tests on HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -**BiDi (Bi-Directional)** refers to the **WebDriver BiDi protocol**, a new standard from the W3C (World Wide Web Consortium). Unlike the traditional WebDriver protocol (which works in a request → response model), **WebDriver BiDi enables two-way communication between your test script and the browser in real time**. - -That means instead of only sending commands (like “click this element” or “navigate to this URL”), your test can also **subscribe to events happening inside the browser** as they occur, such as console logs, network requests, page lifecycle events, or JavaScript exceptions. In simple terms: -- **Old WebDriver** = One-way communication (test script sends a command, browser replies). -- **BiDi WebDriver** = Two-way communication (test script can also “listen” to the browser while interacting with it). - -BiDi brings modern automation capabilities that were missing in Selenium until now. Some key reasons to use it: -- Real-time event monitoring -- Network interception and mocking -- Enhanced debugging -- DOM and page lifecycle tracking -- Ensures your tests are aligned with the latest W3C standards. - -[HyperExecute](/support/docs/getting-started-with-hyperexecute/) is an AI Native Test Orchestration Cloud Platform that empowers you to run **end-to-end** tests **quickly** and **efficiently**. It provides Just-in-Time (JIT) testing infrastructure with fast execution **speeds**, **smart orchestration**, and **detailed logs**. - -This guide details how to execute your **BiDi** tests on **HyperExecute** using [YAML 0.1](/support/docs/hyperexecute-yaml-parameters/) - -## Run a Sample Project -### Step 1: Clone Sample Repository - -**BiDi repo** -Download or Clone the BiDi sample from TestMu AI’s GitHub Repository - - View on GitHub - -### Step 2: Download the CLI in your Project -Download the HyperExecute CLI and copy it into the root folder of the downloaded sample repository. - -| Platform | HyperExecute CLI | -| ---------| ---------------- | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| macOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | - -### Step 3: Add the required Capabilities - -> The `webSocketUrl` capability is set to `true` to enable BiDi support. - -```javascript title="wdio.lambdatest.config.js" - -capabilities: [{ -"LT:Options": { -browserName: "chrome", -// highlight-next-line -webSocketUrl: true, -version: "latest", -platform: process.env.HYPEREXECUTE_PLATFORM || 'windows 10', -name: "HyperExecute BiDi Test", -build: "HyperExecute BiDi Test", -console: true, -} -}], - -``` - -### Step 4: Configure YAML in your Test Suite - -```yaml reference title="hyperexecute.yaml" -https://github.com/LambdaTest/hyperexecute-bidi-sample/blob/main/yaml/hyperexecute-linux.yaml -``` - -### Step 5: Execute your Test Suite -From the project root directory, execute the below CLI command in your terminal: - -> **NOTE :** In case of MacOS, if you get a permission denied warning while executing CLI, simply run **`chmod u+x ./hyperexecute`** to allow permission. In case you get a security popup, allow it from your **System Preferences** → **Security & Privacy** → **General tab**. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config yaml/hyperexecute-linux.yaml`} - -### Step 6: Monitor the Test Execution - -Visit the [HyperExecute Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) and check your Job status. - -## Advanced Parameters -Optimize your test pipeline using the following advanced features: - -- ✅ [Smart Test Splitting](/support/docs/hyperexecute-test-splitting-and-multiplexing/) – Automatically distribute tests across parallel runners -- ⚡ [Fail Fast](/support/docs/hyperexecute-failfast/) – Stop test runs on the first failure -- 📊 [Detailed Reports](/support/docs/hyperexecute-reports/) – Real-time terminal logs & rich test reports -- 🔄 [Auto-Healing](/support/docs/hyperexecute-auto-healing/) – Automatically recover from certain failures - -> For any query or doubt, please feel free to contact us via **24×7 chat support** or you can also drop a mail to **support@testmuai.com**. diff --git a/static/docs/hyperexecute-browser-list-api.md b/static/docs/hyperexecute-browser-list-api.md deleted file mode 100644 index 436d3b4bf..000000000 --- a/static/docs/hyperexecute-browser-list-api.md +++ /dev/null @@ -1,164 +0,0 @@ -# How to GET Supported Browsers in HyperExecute via API - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The Browser List API is a service provided by HyperExecute. It allows users to programmatically retrieve a list of all supported browsers and their versions on the HyperExecute platform. This information can be useful for compatibility checks and ensuring your web applications work across various browsers. - -It will provide you the details like: - - **platform name** - - **browsers name** and **versions** - - **resolutions** - -## API Endpoint and Authorization - -The Endpoint for the Browser List API is: - -```bash -https://api.hyperexecute.cloud/v2.0/browsers -``` - -``` -Request Type : GET -Authorization: Basic -``` - -## Request -The Browser List API supports a simple HTTP GET request to fetch the list of supported browsers - -```bash -https://api.hyperexecute.cloud/v2.0/browsers -``` - -## Sample Response - -The response will be in JSON format, containing an array of objects representing each supported browser along with its versions. - -```bash -{ -"message": "", -"platforms": { -"desktop": [ -{ -"platform": "ubuntu 20", -"browsers": [ -{ -"name": "Chrome", -"version": "99.0" -}, - -{ -"name": "Firefox", -"version": "99.0" -}, - -{ -"name": "MicrosoftEdge", -"version": "canary" -}, - -{ -"name": "Opera", -"version": "85.0" -}, -{ -"name": "Chrome", -"version": "latest" -}, -{ -"name": "Firefox", -"version": "latest" -}, -{ -"name": "MicrosoftEdge", -"version": "latest" -}, -{ -"name": "Opera", -"version": "latest" -} -], -"resolutions": [ -"1024x768", -"1280x800", -"1280x1024", -"1366x768", -"1440x900", -"1680x1050", -"1600x1200", -"1920x1200", -"1920x1080", -"2048x1536", -"2560x1440", -"1280x960" -] -}, -] -}, -"status": 200 -} -``` - -## Params - -```bash -{ -"platform": "windows 10" #optional -"stable": true #if passed we will return only stable version of browsers -} -``` - -### Sample Response - -The response will be in JSON format, containing an array of objects representing each supported browser in **windows 10** only along with its versions. - -```bash -{ -"message": "", -"platforms": { -"desktop": [ -{ -"platform": "Windows 10", -"browsers": [ -{ -"name": "Firefox", -"version": "99.0" -}, -{ -"name": "Firefox", -"version": "100.0" -}, -{ -"name": "Firefox", -"version": "latest" -} -], -"resolutions": [ -"1024x768", -"1280x800", -"1280x1024", -"1366x768", -"1440x900", -"1680x1050", -"1600x1200", -"1920x1200", -"1920x1080", -"2048x1536", -"2560x1440" -] -} -] -}, -"status": 200 -} -``` - -## cURL Command -You can also use the `curl` command to trigger the following api: - -``` -curl --location --request GET 'https://api-hyperexecute.lambdatest.com/sentinel/v2.0/browsers' -``` - -## Conclusion - -The Browser List API simplifies the process of obtaining a comprehensive list of supported browsers and their versions on the HyperExecute platform. By using this information, you can ensure that your web applications are compatible across a wide range of browsers, leading to a better user experience for your audience. diff --git a/static/docs/hyperexecute-build-mobile-apps-using-tools.md b/static/docs/hyperexecute-build-mobile-apps-using-tools.md deleted file mode 100644 index 141762edd..000000000 --- a/static/docs/hyperexecute-build-mobile-apps-using-tools.md +++ /dev/null @@ -1,52 +0,0 @@ -# Building Mobile Applications Using Tools - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Suppose you're working on an Android app using Gradle, and all of a sudden you realize that this project requires an older Java version and a specific Android SDK. Now, setting up a whole new environment in not a very convenient option. - -HyperExecute facilitate the development of mobile applications using various tools such as **Gradle**, **Maven**, etc. It provides a language and framework agnostic environment, supporting a wide range of tools and version combinations crucial for building **Android APKs** efficiently. - -## Building Apps with `runtime` Flag - -HyperExecute provides a [`runtime`](/support/docs/deep-dive-into-hyperexecute-yaml/#runtime) flag feature that dynamically downloads and installs required language and framework versions based on your needs. This removes the need for pre-installed environments on the execution machines. - -```bash -runtime: -language: java -version: ${STATIC_DATA_1_JAVA_VERSION} -addons: -- name: "gradle" -version: "${STATIC_DATA_1_GRADLE_VERSION}" -- name: "android-sdk" -version: ${STATIC_DATA_1_ANDROID_SDK_VERSION} -``` - -## Leveraging DataJsonPaths for Dependency Management - -[`DataJsonPaths`](/support/docs/deep-dive-into-hyperexecute-yaml/#datajsonpath) helps to distribute data/configs over the VMs. In this you can create a json files and put configurations/data required for your suite as json array inside the file. - -This is useful when you have the project and you have to build it across different java, gradle and android-sdk versions. - -To provide multiple versions for each of these dependencies, the DataJson can be leveraged provided by HyperExecute. - -```bash -[ -{ -"JAVA_VERSION": "11", -"ANDROID_SDK_VERSION": "24", -"GRADLE_VERSION": "7.5" -}, -{ -"JAVA_VERSION": "178", -"ANDROID_SDK_VERSION": "25", -"GRADLE_VERSION": "8" -}, -{ -"JAVA_VERSION": "20", -"ANDROID_SDK_VERSION": "32", -"GRADLE_VERSION": "8.4" -} -] -``` - -HyperExecute empowers developers to build mobile applications effectively by providing a flexible and adaptable development environment. With features like dynamic dependency management, YAML parameter configuration, support for multiple dependency versions, and distributed configuration capabilities, HyperExecute streamlines the development process and enhances productivity. diff --git a/static/docs/hyperexecute-cli-error.md b/static/docs/hyperexecute-cli-error.md deleted file mode 100644 index d2d964993..000000000 --- a/static/docs/hyperexecute-cli-error.md +++ /dev/null @@ -1,32 +0,0 @@ -# How to Debug your CLI Errors - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This document will help you debug your failed job if you are facing any CLI issues. - -If you are encountering any error which is properly not detailed, it is recommend to use [**`verbose`**](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/#--verbose) mode in HyperExecute CLI command. - -## Symbolic Link Error - -This error will mark your Job as a **Lambda Error**. - -You can encounter this error when you try to upload the **node_modules** directory along with the payload to trigger your Job in HyperExecute. This error generally occurs with your Node.js projects. You must avoid doing this, as it will not only save storage space but also avoid potential issues with version control. The node_modules folder holds all the project dependencies, and adding it to the repository can cause unnecessary file duplication and increase the repository size. - -To resolve this issue, either you remove that directory or you simply add node_modules to the **.gitignore** file. This ensures you have a smooth development process without unnecessary hiccups. - -## Snooper Failed Error - -It's crucial to understand a specific characteristic of Gherkin – it exclusively supports the use of **Examples** with the **Scenario Outline**. If you come across an error, it likely indicates that the **Scenario** is being employed with **Examples** instead of the intended **Scenario Outline**. To resolve this, ensure that you use the appropriate structure, employing the Scenario Outline when incorporating Examples. - -## Authentication Error - -There may be cases where, when trying to connect to HyperExecute services, the CLI throws the following error: - -Now, to validate the connectivity and to make sure the **auth.hyperexecute.cloud** is reachable, you have to run a `scan` command in your CLI, as shown below. If the network scan fails, then you are required to whitelist the IP address. - -#### IPs to be Whitelisted - -| FQDN | Protocol:Port | Reason | -|------|---------------|--------| -|*.hyperexecute.cloud | Https:443 |HyperExecute’s registered wildcard domain to allow access to all HyperExecute’s services| -|*.lambdatest.com | Https:443 | TestMu AI’s registered wildcard domain to allow access to all TestMu AI’s services (like auth, plan-validation) | diff --git a/static/docs/hyperexecute-cli-faqs.md b/static/docs/hyperexecute-cli-faqs.md deleted file mode 100644 index 20d455a7d..000000000 --- a/static/docs/hyperexecute-cli-faqs.md +++ /dev/null @@ -1,24 +0,0 @@ -# HyperExecute CLI FAQs - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -#### 1. **Can HyperExecute CLI download test artifacts generated during execution?** - - Yes, HyperExecute can download the artifacts generated during the test execution. In addition, HyperExecute has APIs to download artifacts for a particular job. - -#### 2. **I am not able to open HyperExecute on MacOS because the developer is not verified. What do I do?** - - The binary for Mac is unsigned. You might get a permission denied warning while executing HyperExecute on macOS and linux machines. To allow permissions you can use the following command: `chmod u+x ./hyperexecute`. - Thus, you might see a security popup on whether to allow the execution of the tool or not. You can allow it from your **System Preferences** → **Security & Privacy** → **General** tab. - -#### 3. **Is there a way where I can follow all of the steps in my test run on HyperExecute?** - - Every time you run a test via the [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/), a `hyperexecute-cli.log` file is generated. It is present in the parent directory of your test and contains the details of your entire test run. - - This `hyperexecute-cli.log` file will also help you in debugging any issues with your tests. You can view the file and figure out exactly where your test run failed. - - If you want more help with debugging your issues, you can always contact [support@testmuai.com](mailto:support@testmuai.com) and provide them with this file along with your Job ID. - -For more information on the HyperExecute CLI, visit this [page](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/). - ->If you have more questions, then just give us a shout and we will answer them for you. diff --git a/static/docs/hyperexecute-cli-gui.md b/static/docs/hyperexecute-cli-gui.md deleted file mode 100644 index 2acdb7374..000000000 --- a/static/docs/hyperexecute-cli-gui.md +++ /dev/null @@ -1,93 +0,0 @@ -# HyperExecute GUI App - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -**HyperExecute Graphical User Interface (GUI)** streamlines your test automation workflow and brings HyperExecute's power of test orchestration with the most intuitive way to interact. HyperExecute GUI makes testing simple, fast, and powerful. It abstracts all technical complexity to make **high-performance test execution available to everyone**. - -## Why HyperExecute GUI? - -**[HyperExecute](https://www.testmuai.com/hyperexecute)** has been a **cornerstone for automation testing**, offering unparalleled flexibility with its YAML configurations and command-line interface. Now, with HyperExecute GUI, you can experience AI-powered test detection, set up test environments with simple clicks and run your tests without coding. - -## Prerequisites - -**BETA** - -HyperExecute GUI is currently in **BETA** and supports the **Java TestNG Framework (with Java Version 11 or higher)**. We're actively working on expanding support for additional frameworks. If you have specific framework requests, please reach out to our support team. - -**1. HyperExecute Access** : You need a TestMu AI account to use the HyperExecute GUI. If you don’t have one, sign up now. Ensure your account includes HyperExecute access (free or paid plan) to get started. - -**2. UnderPass App Download** : Before getting started, download the TestMu AI UnderPass installer from your dashboard and install it on your system. Run the installer by following the on-screen instructions and launch the application. - -Currently HyperExecute GUI is a part of the **TestMu AI UnderPass Application**. If you already have UnderPass app on your system, please update it to the latest version. - -**Download the GUI 🔗** -You can download the app directly from your **HyperExecute Dashboard** or simply click the links below based on your operating system: - - **[Download for MacOS](https://downloads.lambdatest.com/underpass/master/UnderPass.dmg)** - - **[Download for Windows](https://downloads.lambdatest.com/underpass/master/UnderPass.exe)** - - **[Download for Linux](https://downloads.lambdatest.com/underpass/master/UnderPass.AppImage)** - -## Getting Started - -### Step 1: Select and Analyze Your Project -Launch the application, log in with your TestMu AI credentials, and select your test project folder. The GUI analyzes your project i.e. the framework and language, network connectivity etc. Once the analysis completes, click **"Proceed"** to continue to the next step. - -**Sample Project 🔗** -If you do not have a project handy, please feel free to use this **[Sample Project](https://github.com/LambdaTest/testng-selenium-hyperexecute-sample)** for trying out the GUI. - -**Common Issues & Solutions** -The GUI will check the system and your project to ensure everything is ready. If any issues are found, you'll see clear instructions to resolve them. Some common issues faced by the users are: -- **Missing programming language or package manager**: If your project uses a language or package manager that isn't installed on your computer, the GUI will guide you through installing it. -- **Network connectivity issues**: If the GUI can't connect to TestMu AI servers, it will ask you to set up the necessary network settings through proxy. - -### Step 2: Manage Configurations -**A "Configuration"** is a saved combination of selected tests and execution settings that helps you create different run-profiles for easy management and operation. - -- **For new users:** A default configuration will be created to help you get started quickly. You can select the tests and run config settings as per your requirements. -- **For existing users:** If you have existing configurations (version 0.2 YAML files) in the project, they'll be automatically imported, preserving all your settings. This ensures a smooth transition for existing HyperExecute users while providing a more intuitive interface. - -#### Quick Edit and Run -At the configuration listing page you can find basic details and tags for each configuration. You can also **make quick changes and run the configuration** without going inside the configuration. There are a few other operations which you can access from this page like **Duplicate, Rename, Delete or Download the configuration as YAML.** - - **What is a Configuration?** A configuration is essentially a **complete test execution profile** that includes which specific tests to run from your test suite, all execution parameters (OS, concurrency, etc.), environment variables and other settings. Configurations eliminate the need to repeatedly select tests and adjust settings for common testing scenarios. For example, you might create separate configurations for smoke tests and full regression. - -### Step 3: Discover and Select Tests -Once you navigate to any Configuration, the app will automatically initiate Test Discovery. **Test Discovery is an automated process** that scans your project to identify and organize all test cases of your project. You can use either of the following methods for Tests Selection - -#### Manual Selection -Allows you to browse and select tests individually or in groups. Tests can be organized by **Suite Tests, Files, Classes or Methods** for better visibility and easier navigation. You can select or deselect tests manually using checkboxes. This method works well for smaller test suites or when you need precise control over specific test cases. - -#### Query Selection -For larger test suites, the query selection provides powerful filtering capabilities to **automatically select tests based on specified criteria**. -For detailed instructions on creating and using advanced queries, refer to our [Query Selection Guide](/support/docs/hyperexecute-gui-query). - - - **Orchestration Level:** When using query selection, you can define the orchestration level to determine how tests are **distributed and executed** across multiple parallel instances. This feature helps optimize execution time by intelligently grouping and distributing tests. - -### Step 4: Specify Run Config -The **Run Config tab** in HyperExecute GUI offers an intuitive visual interface to configure your test environment and execution settings. -You’ll find the basic preset configurations displayed just below your project name. If you need to modify or customize these settings, simply navigate to the Run Config tab and adjust your execution environment effortlessly, _no YAML editing required_. - -To learn about any configuration in detail, please search for the keyword in the **[HyperExecute YAML Documentation](/support/docs/deep-dive-into-hyperexecute-yaml)**. Please note that the GUI follows version 0.2 YAML parameters instead of discovery and runner commands as in version 0.1 YAML which you can find [here](/support/docs/hyperexecute-yaml-version0.2). - -### Step 5: Run Tests and View Job -Once you've set up your configuration by selecting the appropriate tests and reviewing your execution settings, you're ready to run your tests. The **"Run Tests"** button is available both from the configuration details screen and directly from the configurations listing page for quick access. - -A sidebar panel with job status and creation logs will appear with option to navigate to HyperExecute Dashboard to view more details. The HyperExecute-GUI displays the job logs in real-time as your tests run. Once complete, you can view the detailed test results in the HyperExecute Dashboard. - -## Already Using HyperExecute via YAML? - -If you’re a HyperExecute version 0.2 YAML user, you can easily import your existing version 0.2 YAML configurations into the GUI. Here’s how: - -Once your project is set up and you want to import a configuration using an existing YAML file : - - Go to the Configuration Listing page. - - Click on **"Import Config.”** - - Choose a YAML file from your project, and its settings will be imported to a new configuration. - -## FAQs - - 1. Are Appium tests supported on GUI as of now? No, Appium test support will be coming soon. Currently, the GUI is in beta and primarily supports TestNG framework with Selenium only. - - 2. If my tests are written using Chrome Driver or any other local driver, can I use that project? Yes, you can use the project, but the associated test IDs will not be created. You would need to enable the "Screen Recording For Scenarios" key to record the entire scenario execution, with the video accessible from the HyperExecute dashboard. (This is applicable for non-Selenium based tests) - - 3. If I have a scenario where the project has two folders - one which creates the test dependencies and another which uses those dependencies to run the tests, how can I use that? You would not be able to use such a project. Please ensure that each project is created from one folder only. We will be supporting this feature in the future. - - 4. My test discovery failed on the HyperExecute GUI. What may be the most possible cause, and how do I fix it? Test discovery failures often stem from misconfigured environment variables or discovery flags when you may have private dependencies in your project. Here’s how to troubleshoot. You can try adding environment variables or discovery flags in the GUI itself which may be required to resolve the dependencies. You will find two types of Environment Variables - Local and Remote. Local environment variables are used for local discovery and it is advisable to add these without any secret values since these are stored on local environment only. diff --git a/static/docs/hyperexecute-cli-release-notes-0-2-249.md b/static/docs/hyperexecute-cli-release-notes-0-2-249.md deleted file mode 100644 index 0b698ba2d..000000000 --- a/static/docs/hyperexecute-cli-release-notes-0-2-249.md +++ /dev/null @@ -1,31 +0,0 @@ -# Version 0.2.249 - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Update CLI to the Latest Version - 0.2.249 -We’re excited to announce several key updates and enhancements in the latest version of our [CLI - 0.2.249](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/#download-the-hyperexecute-cli). You can update to the latest version to take advantage of the following improvements: - -### New Features and Enhancements -- **Support for ProjectLink Payloads:** Added compatibility for projectLink-type sources. -- **Maestro Framework Support:** Enhanced framework detection with default region handling. -- **Capability Flags:** Introduced support for capability values in the [runson flag](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/#--runson). -- **Cypress Integration:** Improved support for Cypress operations, including build configuration fixes. -- **XCUI Enhancements:** Enabled XCUI test reports and increased dispatcher timeouts for XCUI jobs. - -### Bug Fixes -- Fixed [`--use-zip`](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/#-z---use-zip) flag functionality for seamless execution. -- Resolved GUI execution issues for C# projects. -- Addressed issues with [inherited YAML configurations](/support/docs/hyperexecute-inherit-config/) and environment variable replacements. -- Fixed nil map errors and streamlined email array replacement using variables. -- Corrected XcTestPlanId configurations for improved iOS testing. - -### Performance Improvements -- **Binary Updates:** Auto-update feature for binaries now functions without terminating the CLI process, with added support for Windows systems. -- **Timeout Adjustments:** Increased app upload timeout to 5 minutes for smoother operations. - -### Miscellaneous Updates -- Added support for Slack channel customization for alerting. -- Improved discovery results with additional suite types for better test identification. -- Moved essential files like `executeEngine.jar` to the project root for better accessibility. - -> We strongly recommend updating to the [latest CLI version](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/#download-the-hyperexecute-cli) to leverage these enhancements, ensuring a smoother and more efficient experience. diff --git a/static/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid.md b/static/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid.md deleted file mode 100644 index 5393dfaf9..000000000 --- a/static/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid.md +++ /dev/null @@ -1,364 +0,0 @@ -# HyperExecute CLI - Run test on HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -`HyperExecute CLI` is a command line interface that is required to run tests on HyperExecute. This page outlines all you need to know about the CLI and also explains about the different flags & functionalities supported by the HyperExecute CLI. - -> HyperExecute CLI credentials (access keys, API tokens, etc.) are private and confidential. A system compromise may result in their unauthorized disclosure; therefore, you must secure the machine running the TestMu AI-HyperExecute CLI. Do not share or expose credentials in public code, repositories, or environments. In the event of a compromise, you are solely responsible for immediately revoking and regenerating the credentials. TestMu AI assumes no liability for any losses arising from failure to comply. - -## Download the HyperExecute CLI - -You can download the `HyperExecute CLI` for your OS from the links given below : - -| Platform | Download Link | -| ---------| --------------------------- | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| macOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | - -## Verifying the Authenticity of the CLI -To ensure the binary you’ve downloaded is genuine and has not been tampered with, you can verify its digital signature or code signing certificate. TestMu AI signs all official HyperExecute CLI releases for security and integrity assurance. - -You can verify the code signature of the binary using the `openssh` command. Download the [Signature](https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute.sig) and [Public Key](https://downloads.lambdatest.com/hyperexecute/linux/public_key.pem), keep it with your CLI, and paste the location in the below command: - -```bash -openssl dgst -sha256 -verify -signature -``` - -You can verify the code signature of the binary using the `codesign` command. In your terminal run the following command: - -```bash -codesign -dvvv -``` - -**Step 1:** Locate the downloaded CLI binary (e.g., hyperexecute.exe). -**Step 2:** Right-click on the file and select Properties. -**Step 3:** Go to the Digital Signatures tab. -**Step 4:** Select the listed signature and click Details. -**Step 5:** Click View Certificate → Details to inspect the certificate information. - -This certificate should display TestMu AI, Inc. as the verified publisher. - -## HyperExecute CLI Supported Flags - -HyperExecute CLI provides different command-line flags that let you control the HyperExecute platform as per your customised requirements. You can run `hyperexecute.exe --help` on your terminal to explore the different flags. - -The various flags that are supported are listed below. - -| Flags | Type | Description| -|-------|------|------------| -| [analyze](#analyze) | NA | Analyzes the language and environment supported in your project.| -| [completion](#completion) | NA | Generates the autocompletion script for HyperExecute for the specified shell. | -| [serve](#serve) | NA | Starts a server for HyperExecute info APIs | -| [update](#update) | NA | Update the HyperExecute Binary Version | -| [--add-testids](#--add-testids) | NA | Add test ids in job summary. | -| [--auto-proxy](#--auto-proxy) | NA | Use detected system proxy.| -| [--concurrency](#--concurrency) | Numerical | Indicates the number of concurrent sessions on HyperExecute. | -| [--config](#--config) | string | Custom location for hyperexecute.yaml. | -| [--disable-updates](#--disable-updates) | NA | Disable auto update | -| [--download-artifacts](#--download-artifacts) | NA | Download the artifacts for a job. | -| [--download-artifacts-path](#--download-artifacts-path) | string | Path of the directory to download artifacts for a job | -| [--download-artifacts-zip](#--download-artifacts-zip) | NA | Download the artifacts for a job as a zip. | -| [--download-logs](#--download-logs) | NA | Downloads console logs of your every tests of a job. | -| [--download-report](#--download-report) | NA | Whether reports have to be downloaded or not. | -| [--force-clean-artifacts](#--force-clean-artifacts) | NA | Download the updated artifacts of your job. | -| [-h, --help](#-h---help) | NA | Help for hyperexecute. | -| [--job-secret-file](#--job-secret-file) | string | Enter your secrets file path | -| [-k, --key](#-k---key) | string | TestMu AI Access Key. | -| [--labels](#--labels) | string | Incorporates the labels in your job. | -| [--no-track](#--no-track) | NA | Stops updating the progress of the uplodaed jobs. | ---onprem-host -| [--preserve-payload](#--preserve-payload) | string | Preserves the job payload after its successful completion. | -| [--runson](#--runson) | string | (comma-separated values of) OS on which the tests should run. | -| [-s --scan](#-s---scan) | NA | Runs the network logs | -| [--server-port](#--server-port) | string | Port number for the server (default "9191") | ---static-data-filter -| [-t, --target-directory](#-t---target-directory) | string | Directory where the test script has to be uploaded. | -| [--target-path](#--target-path) | string | Upload the files and folders as part of the suite payload. | -| [--tests-per-tunnel](#--tests-per-tunnel) | INT | Run the number of tests via tunnel. | -| [-z, --use-zip](#-z---use-zip) | string | Path of the zip file that needs to be uploaded. | -| [-u, --user](#-u---user) | string | TestMu AI username. | -| [--vars](#--vars) | string | Method used to name keys. | -| [--validate](#--validate) | NA | Validate HyperExecute YAML file. | -| [--verbose](#--verbose) | NA | Logging of every proxy request to stdout. | -| [--version](#--version) | NA | Version of the HyperExecute CLI. | - -### `analyze` -This flag runs **HyperExecute Analyze**, which is a language and environment detection tool used to render every language and framework detail the user has in his project, which includes finding private dependencies. - -```bash -hyperexecute analyze -``` - -### `completion` -This flag generates the autocompletion script for HyperExecute for the specified shell. - -```bash -hyperexecute completion -``` - -### `serve` -This flag starts a server for HyperExecute info APIs. - -```bash -hyperexecute --user USER_NAME --key ACCESS_KEY serve -``` - -### `update` -This flag updates the hyperexecute binary version. - -```bash -hyperexecute update -``` - -### `--add-testids` -This flag will add the test ids of your job in the result.json file that gets generated after your test execution. - -```bash -hyperexecute --add-testids -``` - -### `--auto-proxy` -This flag use detected system proxy. It identifies and sets proxies including http_proxy and https_proxy variables. - -```bash -hyperexecute --auto-proxy -``` - -### `--concurrency` -This flag allows you to define the number of concurrent sessions running on HyperExecute. For more information on concurrency, go to [this page](/support/docs/hyperexecute-auto-split-strategy/). -```bash ---concurrency 2 -``` - -### `--config` - -This flag allows you to enter a custom location where you can download the HyperExecute YAML file. To learn more about the HyperExecute YAML, visit [this page](/support/docs/deep-dive-into-hyperexecute-yaml/). -```bash ---config "/home/users/work/yaml/" -``` - -### `--disable-updates` -This flag allows you to disable the auto updation of your HyperExecute binary. - -### `--download-artifacts` -This flag allows you to download all the test related results that are generated by your testing framework. You can use this key if you want to download the artifacts. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE --download-artifacts`} - -- For Espresso with private devices HyperExecute can download user generated artefacts. - -### `--download-artifacts-path` -In case you want the artifacts to be downloaded to a specific directory, please provide the exact path followed by the flag as shown in the example below. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE --download-artifacts-path "PATH_OF_YOUR_SPECIFIED_DIRECTORY"`} - -### `--download-artifacts-zip` -If you want to download the zip file of the artifacts for a job. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE --download-artifacts-zip`} - -### `--download-logs` -This flag allows you to download the detailed console logs of your entire job including your individual tests and your pre and discovery phase. - -### `--download-report` -This flag allows you to download all the test related results that are generated by your testing framework. You can use this key if you want to download the reports. - -### `--force-clean-artifacts` -This flag will download the new artifacts of your job and will store it into the `artifacts` folder. If you have previously downloaded artifacts, it will be renamed as `artifacts-old`. - -### `-h, --help` -You can run this command if you need any help in understanding the definition of the flags on the CLI itself. However, if you need any further assistance we're just a **ping** away. -```bash ---help -``` - -### `--job-secret-file` - -- This flag allows you to specify the Secrets file containing the necessary secrets (credentials, API keys, ) for a particular job. - -- The contents of this file will be accessible only within the scope of that specific job execution, enhancing security and streamlining secret management. - -- This eliminates the need to store secrets on the service account and prevents their accidental exposure through the `--vars` flag. - -```bash ---job-secret-file -``` - -#### Sample Secret File - -```bash title="secretFile.txt" -secret1=value1 -secret2=value2 -``` - -The secrets file contains highly sensitive information and must be strictly excluded from public exposure. Here's how to ensure its security: - -**Store Outside the Repository (Recommended)** - -- Store the secrets file in a location outside your project repository altogether. This method is more secure as it automatically gets excluded from your code scripts zip file. - -**Exclude from Version Control** - -- Adding the relative path of the file (in which you have stored your secret data) to your [`.gitignore` or `.hyperexecuteignore`](/support/docs/hyperexecute-gitignore/) file. This prevents accidental inclusion of the file in your code repository. - -### `-k, --key` -The access key of your TestMu AI account. To find out how to retrieve your access key, visit [this page](/support/docs/hyperexecute-guided-walkthrough/#hyperexecute-dashboard). - - {`--key ${ YOUR_LAMBDATEST_ACCESS_KEY()}`} - -### `--labels` -This flag allows you to pass the labels of your Job. -```bash ---labels "testing" -``` - -### `--no-track` -You can track the progress of your uploaded jobs with HyperExecute. However, this flag allows you to opt out of receiving those updates. - -```bash ---no-track -``` - -### `--preserve-payload` -This flag allows you to preserve the job payload after the job is completed successfully. -```bash ---preserve-payload -``` - -### `--runson` -This flag overrides the defined OS value in your YAML file's [`runson`](/support/docs/deep-dive-into-hyperexecute-yaml/#runson) flag. You can also define multiple values for OS (comma separated) in the case of [Matrix](/support/docs/hyperexecute-matrix-multiplexing-strategy/) and [Hybrid](/support/docs/hyperexecute-hybrid-strategy/) mode execution. - -In case of [AutoSplit](/support/docs/hyperexecute-auto-split-strategy/) mode: - -```bash ---runson linux -``` - -In case of [Matrix](/support/docs/hyperexecute-matrix-multiplexing-strategy/) or [Hybrid](/support/docs/hyperexecute-hybrid-strategy/) mode: - -```bash ---runson linux,win,win11,mac,mac13 -``` - -### `-s, --scan` -This flag runs the network logs in your local machine console . - -```bash ---scan -``` - -### `--server-port` -The `--server-port` flag is used to specify the port number that the HyperExecute server will use for communication. This can be useful if you need to run the server on a specific port due to network configurations or to avoid port conflicts with other applications. - -```bash ---server-port=8080 -``` - -### `-t, --target-directory` - -The `targetDirectory` option in the HyperExecute CLI specifies the directory where the output of the executed command will be stored. If this option is not specified, the output will be stored in the current working directory. - -```bash ---target-directory "/home/users/work/yaml/scripts/" -``` - -### `--target-path` - -The `target-path` flag allows you to selectively upload specific files and folders as part of the suite payload. This can be used for streamlining the uploading process and ensuring that only relevant files are included in the payload. -It can also be useful if a user wants to run tests in a monorepo for multiple services. - -```bash ---target-path "src/test-suite/**" -``` - -### `--tests-per-tunnel` - -This flag allows you to run a certain number of tests via tunnel. The default maximum number of tests is 10. - -```bash ---tests-per-tunnel 5 -``` - -### `-z, --use-zip` -This flag allows you to indicate the path of the zip file that needs to be uploaded for your tests to run. -Let's say you already have a zipped payload that you want to upload, so you can provide the path of that payload and upload it. - -Whenever you zip your test files, it creates a folder, let's say **tests.zip**. Now, when this zip folder is unarchived, HyperExecute searches for a file named **`hyperexecute.yaml`**. - -In another scenario, suppose you first created a folder. Inside that folder, you put all your required test files and then zip that folder (say **folder.zip**). When this zip folder is unarchived, HyperExecute will again search for the **`hyperexecute.yaml`** file, which in this case won't be available as it will be inside the folder, which will throw an error. - -```bash ---use-zip "/home/users/work/yaml/zip/" -``` - -### `-u, --user` -The username of your TestMu AI account. - - {`--user ${ YOUR_LAMBDATEST_USERNAME()}`} - -### `--vars` -This method allows you to name your variables. It helps you optimize the content of your YAML file by allowing for more flexibility. - -You can specify the email address for sharing [reports](/support/docs/hyperexecute-reports/) or [artifacts](/support/docs/hyperexecute-artifacts/) in a YAML configuration file using a variable. This allows you to dynamically set the email address and pass it via CLI, providing more flexibility than hardcoding the email address in the YAML file. Learn about this in [detail](/support/docs/hyperexecute-email-reports/#how-to-dynamically-set-your-email-address). -```bash ---vars "org=Lambdatest" --vars "product=HyperExecute" --vars "email=xyz@abc.com" --vars "email1=abc@xyz.com" -``` - -### `--validate` -The new `--validate` flag in the HyperExecute CLI allows you to quickly check the syntax and structure of your YAML file without running any tests. Simply pass the `--validate` flag with your YAML file, and the CLI will validate its structure and syntax. It will then return a success or error message, helping you identify if the file is correctly formatted or if there are any issues that need attention. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE --validate`} - -### `--verbose` -This flag allows you to log all of your proxy requests to the output section (stdout). -```bash ---verbose -``` - -### `--version` -This flag indicates the version of HyperExecute CLI binary that you are running on your system. -```bash ---version -``` - -## Trigger your Test from HyperExecute CLI - -In order to trigger your job on HyperExecute, execute the following command in your terminal. You will need to insert the name of your YAML file in place of the `RELATIVE_PATH_OF_YOUR_YAML_FILE` before running this command. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE `} - -In case of macOS, if you get a permission denied warning while executing CLI, simply run `chmod u+x ./hyperexecute` to allow permission. In case you get a security popup, allow it from your **System Preferences** → **Security & Privacy** → **General** tab. - -If you're running on a minimal alpine image, please make sure to install essential linux dependencies for HyperExecute CLI to work. -(An example with our sample repo has been given below): -``` bash -apk add libc6-compat git bash -git clone https://github.com/LambdaTest/behave-selenium-hyperexecute-sample.git -cd behave-selenium-hyperexecute-sample/ -wget https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute -chmod +x hyperexecute -./hyperexecute --user --key --config yaml/win/behave_hyperexecute_autosplit_sample.yaml -``` - -## Running HyperExecute CLI in Quiet Mode - -Each time a job is run in Hyperexecute, a large number of logs are produced. These logs are sometimes beneficial for thorough debugging and analysis, it occasionally overwhelms Continuous Integration (CI) systems with an excessive amount of logs, especially when handling a large number of test executions. - -Hyperexecute provides a **"Quiet mode"** as a solution to this issue and to give users more control over the logging output. By only displaying significant updates during the test execution, this enables you to run tests with less logs. - -To enable the **Quiet Mode**, you need to paas these commands in your terminal and then trigger the command to execute your job. - -```yaml -env | grep CI # Used to check if the CI variable is set to true or not -export CI=true # If it's not set to true, then use this command to set as true -``` - -```yaml -env | grep CI # Used to check if the CI variable is set to true or not -set CI=true # If it's not set to true, then use this command to set as true -``` - -#### Output - -> 📕 Use the HyperExecute CLI's [Frequently Asked Questions](/support/docs/hyperexecute-cli-faqs/) to gain more knowledge about it. diff --git a/static/docs/hyperexecute-csharp-use-cases.md b/static/docs/hyperexecute-csharp-use-cases.md deleted file mode 100644 index 2db7b3563..000000000 --- a/static/docs/hyperexecute-csharp-use-cases.md +++ /dev/null @@ -1,413 +0,0 @@ -# HyperExecute CSharp Use Cases - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This guide provides a complete reference for integrating C# NUnit SpecFlow tests with HyperExecute, covering edge cases, solutions, YAML setup, artifact management, remote test discovery, and reporting. - -## 1. Autosplit -### Edge Case -Large test suites with hundreds or thousands of tests can leave some nodes idle while others are overloaded, resulting in wasted time and uneven execution. - -### Solution -AutoSplit divides tests intelligently across nodes at file, module, or scenario level. Previously failed tests are prioritized for faster feedback. This ensures maximum concurrency usage and reduces test flakiness, enabling faster feedback loops for large BDD suites. - -```yaml title="hyperexecute.yaml" -autosplit: true -``` - -### What Happens If We Don’t Use It - -- Inefficient node usage -- Slower test execution -- Flaky tests delay feedback -- No control over test splitting - -## 2. Matrix Setup -### Edge Case -Tests behave differently on various OS platforms. Running only on one OS may hide platform-specific bugs. - -### Solution -The matrix runs tests concurrently on Mac, Windows, and Linux, detecting OS-specific issues like file paths, encoding, or browser behavior. - -```yaml title="hyperexecute.yaml" -matrix: -os: [mac, win, linux] - -runson: ${matrix.os} -``` - -### What Happens If We Don’t Use It -- Tests may pass on one OS but fail on others -- Hidden bugs reach end users -- Manual cross-OS testing required - -## 3. Environment Variables (NuGet Paths) -### Edge Case -Parallel jobs sharing default NuGet caches may cause corrupted packages, race conditions, or build failures, especially in ephemeral environments or low disk space. - -### Solution -Predefining NuGet paths ensures controlled caching, reliable builds across OSes, and efficient reuse of restored packages. - -```yaml title="hyperexecute.yaml" -env: -NUGET_PACKAGES: '/home/ltuser/.nuget/packages/' -NUGET_HTTP_CACHE_PATH: '/home/ltuser/.local/share/NuGet/v3-cache' -NUGET_PLUGINS_CACHE_PATH: '/home/ltuser/.local/share/NuGet/plugins-cache' -``` - -### What Happens If We Don’t Use It -- Random failures in parallel jobs -- Slower builds -- High network usage -- Disk space issues -- Inconsistent behavior across OSes - -## 4. Pre Steps -### Edge Case -Dependencies may not be restored or projects may not build correctly if pre-steps are missing, causing test failures downstream. - -### Solution - -```yaml title="hyperexecute.yaml" -pre: -- dotnet restore -- dotnet list OnlySpecTest.csproj package > packages.txt -# - chmod 777 /tmp/NuGetScratch -# - nuget locals all -clear -- dotnet build -c Release -``` - -- **dotnet restore** -> Restores NuGet packages -- **dotnet list ... > packages.txt:** -> Saves package list for caching -- **dotnet build -c Release:** -> Builds the project in Release mode -- Optional commands handle permissions or cache clearing - -### What Happens If We Don’t Use It -- Builds fail or produce outdated DLLs -- Tests fail due to missing dependencies -- Manual fixes required - -## 5. Artifact Upload -### Edge Case -Without artifacts, failed tests provide no screenshots, logs, or reports for debugging, and outputs can be scattered in parallel executions. - -### Solution -Artifacts centralize outputs like HTML reports and screenshots, enabling debug, traceability, and audit. - -```yaml title="hyperexecute.yaml" -uploadArtefacts: -- name: Execution_Report -path: -- Report/** -- name: Execution_Screenshots -path: -- Screenshots/**/** -``` - -### What Happens If We Don’t Use It -- No visual proof or logs -- Debugging intermittent failures is hard -- Manual reporting is required - -## 6. Report Generation -### Edge Case -Partial or misconfigured reports can be unreadable, incomplete, or fail to merge in parallel executions. - -### Solution -Generates human-readable HTML reports showing pass/fail, steps, screenshots, and aggregated results for stakeholders. - -```yaml title="hyperexecute.yaml" -report: true -partialReports: -location: Report/ -type: html -frameworkName: specflow -``` - -### What Happens If We Don’t Use It -- No centralized reporting -- Harder to analyze test performance trends -- Manual effort needed to share results - -## 7. Test Discovery -### Edge Case -Static or incorrect discovery may skip new tests, misdetect test tags, or fail in parallel executions. - -### Solution -Remote discovery automatically detects new @tags or features, enabling parallelization and reducing manual maintenance. - -```yaml title="hyperexecute.yaml" -testDiscovery: -type: raw -mode: remote -command: grep -rni 'Features' -e '@' --include=*.feature | sed 's/.*@//' -``` - -### What Happens If We Don’t Use It -- Manual selection of tests -- New tests may never run -- Reduced test coverage - -## 8. testRunnerCommand -### Edge Case -Running all tests every time is inefficient, and category typos or missing attributes can prevent execution. - -### Solution -Executes only tests with the specified Category. Supports parallelization and selective reruns. - -```yaml title="hyperexecute.yaml" -testRunnerCommand: dotnet test --filter "(Category=$test)" -``` - -### What Happens If We Don’t Use It -- All tests run every time -- Slower pipelines -- Increased build cost - -## 9. Multiple Tag-Based Test Discovery -### Edge Case -Projects may have diverse tagging strategies, making standard discovery unreliable or incomplete. - -### Solution - -```yaml title="hyperexecute.yaml" -testDiscovery: -type: raw -mode: remote -shell: bash -command: grep -E -w -R -A10 --include='*.feature.cs' ${tagname} AllInOneProject/features | grep 'public void' | cut -d' ' -f2- | awk '{print $3}' | sed 's/()$//' | sed 's/TestTearDown//g' -``` - -- Dynamically extracts test names based on tags -- Ensures new tests are automatically included -- Supports parallel execution and CI/CD pipelines - -### What Happens If We Don’t Use It -- Missing tests due to inconsistent tagging -- Manual updates required -- Increased risk of regressions - -## 10. Framework Version Compatibility Handling -### Edge Case -Different .NET versions require different build steps; incorrect handling causes build failures or runner crashes. - -### Solution -Builds projects correctly based on target framework, ensuring compatibility with HyperExecute runners. - -**`.NET 6`+**: Use dotnet build in pre-steps -**`.NET <6`**: Build locally and upload DLLs - -```yaml title="hyperexecute.yaml" -C:\PROGRA~2\Micros~1\2019\Community\MSBuild\Current\Bin\MSBuild.exe /p:Configuration=Release /p:TargetFramework=net472 /t:restore -``` - -### What Happens If We Don’t Use It -- Build failures -- Outdated or incompatible DLLs -- Test runner crashes - -## 11. Private Dependencies (config.nuget) -### Edge Case -Projects depending on private NuGet feeds fail restore if credentials or feeds are not configured. - -### Solution -Configures private feeds securely to ensure reliable dependency resolution across environments. - -```xml - - - - - - - - - - - - - -``` - -### What Happens If We Don’t Use It -- Restore failures (401 Unauthorized) -- CI/CD errors -- Manual debugging and inconsistent builds - -## 12. Custom Test Runner Alternatives -### Edge Case -Some projects cannot run with dotnet test due to configuration or .NET version constraints. - -### Solution -Alternative runner allows execution of targeted tests, respecting project-specific configurations. - -```yaml -vstest.console.exe "GlobalPolaris\bin\Debug\net472\GlobalPolaris.dll" /Settings:GlobalPolaris\RunSettings.runsettings /TestCaseFilter:"FullyQualifiedName~$test" -``` - -### What Happens If We Don’t Use It -- Tests may fail to execute -- Build verification and pipelines are blocked - -## 13. NUnit Console for Feature Tag Discovery -### Edge Case -Standard discovery may fail with complex NUnit tag setups. - -### Solution -Uses NUnit console to explore feature tests by tags. -```yaml -& "C:\PROGRA~2\NUnit.org\nunit-console\nunit3-console.exe" "" --where="cat==$tag" --explore -``` - -### What Happens If We Don’t Use It -- Tag-based tests are skipped -- Partial coverage -- Manual intervention required - -## 14. Differential Upload Configuration -### Edge Case -Large test suites take long to upload; repeated full uploads waste time and bandwidth. - -### Solution -Uploads only changed files, reducing upload time and network usage. - -```yaml title="hyperexecute.yaml" -differentialUpload: -enabled: true -ttlHours: 300 -``` - -### What Happens If We Don’t Use It -- Slow uploads -- Wasted resources -- Longer pipeline durations - -## 15. Certificate Management for Browser Authentication -### Edge Case -Browser tests requiring client certificates fail without proper installation. - -### Solution -Installs certificates in browser environment for authenticated sessions. - -```yaml title="hyperexecute.yaml" -%HYPEREXECUTE_WORKING_DIR%//Hyperexecute//cert_manager.exe --chrome -i atest089.pfx -``` - -### What Happens If We Don’t Use It -- Browser authentication tests fail -- Manual certificate setup required - -## 16. Custom Status Reporting via CleanUp Function -### Edge Case -Test statuses may not be reported to HyperExecute properly, leading to incomplete dashboards. - -### Solution -Custom function reports statuses for tracking and dashboards. - -```java title="test.java" -public void CleanUp(string[] tagName, ScenarioExecutionStatus scenarioStatus, string scenarioName) -{ -string STATUS = scenarioStatus.ToString() switch -{ -"OK" => "passed", -"UndefinedStep" => "skipped", -"TestError" => "failed", -_ => "others" -}; -((IJavaScriptExecutor)Properties.driver).ExecuteScript($"lambda-status={STATUS}"); -} -``` - -### What Happens If We Don’t Use It -- Missing or inaccurate test reporting -- Dashboard metrics incomplete - -## 17. Custom SpecFlow Reporting Configuration -### Edge Case -Standard reports may not provide enough detail for complex SpecFlow tests. - -### Solution -Generates custom reports with metadata and email notifications. - -```yaml title="hyperexecute.yaml" -report: true -partialReports: -location: GlobalPolaris/Reports -type: html -frameworkName: specflow-custom -email: -to: ["example@lambdatest.com"] -metaInfo: ["project-name:Global Polaris Regression","project-env:QA"] -``` - -### What Happens If We Don’t Use It -- Reports lack detail -- Stakeholders have limited visibility -- Manual reporting required - -## 18. File Level Discovery (DLL) -### Edge Case -Running tests on the wrong or outdated DLL causes failures or misses new tests. - -### Solution -Ensures correct DLL is built and executed on the proper runner. - -```yaml title="hyperexecute.yaml" -runson: win - -pre: -commands: -- dotnet restore -- dotnet build YourSolution.sln -c Release - -test: -commands: -- nunit3-console.exe "bin/Release/net6.0/YourProject.dll" -``` - -### What Happens If We Don’t Use It -- Tests fail -- New features not executed -- Manual DLL verification required - -## 19. Feature File Level Discovery (Matrix) -### Edge Case -All tests running together reduce parallelism and rerun flexibility. - -### Solution -Run each feature individually to enable parallel execution and selective reruns. - -```yaml title="hyperexecute.yaml" -matrix: -featurefile: -- "Features/Login.feature" -- "Features/Search.feature" -- "Features/Checkout.feature" - -test: -commands: -- nunit3-console.exe "bin/Release/net6.0/YourProject.dll" --where "cat == '$featurefile'" -``` - -### What Happens If We Don’t Use It -- Parallelism lost -- Slower execution -- Rerunning failed features is difficult - -## 20. Passing Arguments in HyperExecute -### Edge Case -Incorrect YAML quoting or missing result file paths cause unintended test runs or missing artifacts. - -### Solution -Properly passing arguments ensures selective execution and saves results for artifact uploads. - -```yaml title="hyperexecute.yaml" -test: -commands: -- nunit3-console.exe "bin/Release/net6.0/YourProject.dll" --where "cat == 'Regression'" --result="regression-result.xml" -``` - -### What Happens If We Don’t Use It -- All tests may run accidentally -- Slow pipelines -- Missing artifacts for debugging diff --git a/static/docs/hyperexecute-detox-testing.md b/static/docs/hyperexecute-detox-testing.md deleted file mode 100644 index 877dfa16d..000000000 --- a/static/docs/hyperexecute-detox-testing.md +++ /dev/null @@ -1,129 +0,0 @@ -# Detox Testing on HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Detox is a popular end-to-end testing framework for React Native applications. This documentation will help you run your Detox tests on TestMu AI HyperExecute using your local system setup. - -Detox testing on TestMu AI is supported via HyperExecute on **Android real devices** and **iOS simulators**. - -## Prerequisites - -Before you begin, ensure you have the following: - -- A TestMu AI account. You can get your **Username** and **Access Key** from your [Account Settings](https://accounts.lambdatest.com/detail/profile). -- [Node.js](https://nodejs.org/en/download) and **npm** installed on your local system. -- A React Native application with Detox tests configured, or use the [sample repository](https://github.com/LambdaTest/LT-detox) referenced in this guide. - -## Run a Sample Project -### Step 1: Clone Sample Repository - -**Detox repo** -Download or Clone the Detox sample from TestMu AI’s GitHub Repository - - View on GitHub - -### Step 2: Download the CLI in your Project -Download the HyperExecute CLI and copy it into the root folder of the downloaded sample repository. - -| Platform | HyperExecute CLI | -| ---------| ---------------- | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| macOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | - -### Step 3: Execute your Test Suite -From the project root directory, execute the below CLI command in your terminal: - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE `} - -> **NOTE :** In case of macOS, if you get a permission denied warning while executing CLI, simply run **`chmod u+x ./hyperexecute`** to allow permission. In case you get a security popup, allow it from your **System Preferences** → **Security & Privacy** → **General tab**. - -You've successfully triggered the Detox tests on HyperExecute. Visit the [HyperExecute Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) and check your Job status. - -## Configure your own Project - -### Step 1: Download the CLI in your Project -Download the HyperExecute CLI and copy it into the root folder. - -| Platform | HyperExecute CLI | -| ---------| ---------------- | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| macOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | - -### Step 2: Configure YAML in your Test Suite -Use the sample YAML below to configure your execution: - -```yaml reference title="hyperexecute.yaml" -https://github.com/LambdaTest/LT-detox/blob/main/yaml/hyperexecute.yaml -``` - -> 📘 To explore more configuration options and tailor settings to your project needs, check out the [YAML documentation](/support/docs/hyperexecute-yaml-parameters/). - -### Step 3: Monitor the Test Execution - -Visit the [HyperExecute Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) and check your Job status. - -You can click on **View Test** from the **HyperExecute logs**, which will take you to the [App Automation dashboard](https://www.testmuai.com/login/?redirectTo=https://appautomation.lambdatest.com/build). There, you can access detailed **device logs**, **crash logs**, and **Metadata** for your test. You’ll also be able to watch a **video recording** of the Test sessions for better **debugging** and **analysis**. - -## Advanced Configuration - -HyperExecute supports the following capabilities for your Detox test runs. Add the relevant keys to your `hyperexecute.yaml` file. - -### Tunnel - -Test applications hosted on your local or private network by routing traffic through a secure TestMu AI Tunnel. Enable it by setting the top-level `tunnel` flag in your YAML: - -```yaml -tunnel: true -``` - -You can further customize the tunnel behavior (arguments, scope, and more) using `tunnelOpts`. Learn more in the [Tunnel Configuration](/support/docs/hyperexecute-how-to-configure-tunnel/) guide. - -### Dedicated Proxy - -Route your test traffic through a dedicated proxy server with a fixed IP that your network administrators can whitelist, giving controlled access to network-restricted resources. This feature is enabled for your account by the TestMu AI support team. Learn more in the [Dedicated Proxy](/support/docs/dedicated-proxy/) guide. - -### IP Geolocation - -Run your tests as if from a specific geographic location by passing a supported two-letter country code, letting you validate region-based content and geo-restricted features. Learn more in the [IP Geolocation on Real Devices](/support/docs/ip-geolocation-on-real-devices-app/) guide, and refer to the [supported country codes](/support/docs/appium-ip-geolocation/). - -> For any query or doubt, please feel free to contact us via **24×7 chat support** or you can also drop a mail to **support@testmuai.com**. - -## Using the Detox Agent Skill with TestMu AI - -The [detox-skill](https://github.com/LambdaTest/agent-skills/tree/main/detox-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The detox-skill package includes: - -``` -detox-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing Detox Agent Skill - -Install a Detox Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/detox-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/detox-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only detox-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). diff --git a/static/docs/hyperexecute-email-reports.md b/static/docs/hyperexecute-email-reports.md deleted file mode 100644 index fa3427b5b..000000000 --- a/static/docs/hyperexecute-email-reports.md +++ /dev/null @@ -1,162 +0,0 @@ -# Receive Job Reports and Artifacts via Email - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Downloading Job Reports and Artifacts manually from the HyperExecute UI can be a time-consuming and repetitive task. To address this pain point, HyperExecute now provides the convenience of receiving Job Reports and Artifacts directly to your specified email addresses. This eliminates the need for manual downloads, allowing you to access critical job information instantly and effortlessly. Embrace the efficiency of automated delivery and spend less time navigating the UI and more time focusing on your development tasks. - -## YAML Configuration - -To receive the Artifacts and Reports via mail, you will have to add the `email` flag with a `to` tag to select the email IDs where the report or artifacts should be sent. The example added below shows you how to add your email IDs: - -```yaml -email: -to: -- -- -``` - -You can also use the `currentUser` tag to send the reports to the email ID of your choice. - -- **currentUser**: This tag will allow you to send the report to the email ID associated with your TestMu AI account. - -```yaml -email: -to: -- currentUser -``` - -## How to receive your Artifacts via Email? - -Follow the below mentioned steps to receive your Artifacts via email: - -**Step 1:** You need to mention the `email` flag along with the `to` tag under the `uploadArtifacts` flag in the YAML file configuration. - -> **NOTE:** You can also add a separate email ID for each report generated, as mentioned below in the YAML code. - -```yaml -uploadArtifacts: -- name: Reports 1 -path: -- ProtractorTestReport.html -- xmlresults.xml -email: -to: -- -- - -- name: Reports 2 -path: -- ProtractorTestReport.html -- xmlresults.xml -email: -to: -- currentUser -``` - -## How to receive your Job Report via Email? - -Follow the below mentioned steps to receive your Job Reports via email: - -**Step 1:** Set the `report` flag to `true` in the HyperExecute YAML. - -**Step 2:** Make sure to check the `location`, `type` and `frameworkName` fields in the `partialReports` flag are configured correctly. - -**Step 3:** Add the `email` flag with `to` tag in the YAML file configuration: - -```yaml -report: true -partialReports: -frameworkName: testng -location: target/surefire-reports/html -type: html -email: -to: -- -- -``` - -## Correct format of entering the Email IDs - -The Email IDs that you enter must be valid. You can enter your email IDs in the formats mentioned below. - -```yaml -- John Doe -- John -- johndoe@example.com -``` - -However, if your email IDs are added in an unsupported format, the feature will not work. A few examples of unsupported email ID formats are added below. - -```yaml -- John Doe johndoe@example.com> -- John < -- John johndoe@example.com -- John Doe -``` - -Now that you have added your email IDs successfully, you can access your job reports. Download the report from the email, and get all the information that you need. Alternatively, you can also open the clickable link in the email and view the report on your browser. - -## How to dynamically set your email address? -In your YAML configuration file instead of hardcoding the email address to which you want to share the report or artifacts, you can use a variable that can be set dynamically when you pass the execution command. - -In this example, the `${email}` and `${email1}` variables are used to specify the email address. You can pass the value of this variable using the [`vars`](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/#--vars) flag as an argument when executing your test via CLI. - -```yaml title="hyperexecute.yaml" -report: true -partialReports: -location: target/surefire-reports/html -type: html -frameworkName: extent -# highlight-start -email: -to: -- "${email}" -- "${email1}" -# highlight-end - -uploadArtifacts: -- name: Reports 1 -path: -- ProtractorTestReport.html -# highlight-start -email: -to: -- "${email}" -- "${email1}" -# highlight-end -``` - -This is how you can pass the value of your email address via CLI by running the command - -```bash -./hyperexecute --config RELATIVE_PATH_OF_YOUR_YAML_FILE --vars "email=xyz@abc.com" --vars "email1=abc@xyz.com" -``` - -If you have a pipeline that requires multiple email values, consider consolidating them into a single variable separated by commas. This approach eliminates the need for multiple variables or manual pipeline edits whenever the email list changes. - -```yaml title="hyperexecute.yaml" -report: true -partialReports: -location: target/surefire-reports/html -type: html -frameworkName: extent -# highlight-start -email: -to: -- "${email}" -# highlight-end - -uploadArtifacts: -- name: Reports 1 -path: -- ProtractorTestReport.html -# highlight-start -email: -to: -- "${email}" -# highlight-end -``` - -```bash -./hyperexecute --config RELATIVE_PATH_OF_YOUR_YAML_FILE --vars "email=xyz@abc.com,abc@xyz.com,def@wxy.com" -``` diff --git a/static/docs/hyperexecute-emu-simu-devices-list.md b/static/docs/hyperexecute-emu-simu-devices-list.md deleted file mode 100644 index a6351af02..000000000 --- a/static/docs/hyperexecute-emu-simu-devices-list.md +++ /dev/null @@ -1,157 +0,0 @@ -# HyperExecute Emulator and Simulator Devices - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -HyperExecute supports a wide range of emulator and simulator devices for testing your applications. Below is the categorized list of available devices. - -## Emulators - -| Device Name | Category | -|--------------------|----------| -| Galaxy Note10 | Phone | -| Galaxy Note10+ | Phone | -| Galaxy S10 | Phone | -| Galaxy S10 Lite | Phone | -| Galaxy S10+ | Phone | -| Galaxy S21 5G | Phone | -| Galaxy Tab S5e | Tablet | -| Galaxy Tab S6 | Tablet | -| Galaxy Tab S7 | Tablet | -| Galaxy Tab S7+ | Tablet | -| LG Q7 | Phone | -| Nokia G10 | Phone | -| OnePlus 6 | Phone | -| OnePlus 7 | Phone | -| OnePlus 8 | Phone | -| OnePlus 9 | Phone | -| Pixel 3 | Phone | -| Pixel 3 XL | Phone | -| Pixel 3a | Phone | -| Pixel 3a XL | Phone | -| Pixel 4 | Phone | -| Pixel 4 XL | Phone | -| Pixel 4a | Phone | - -| Device Name | Category | -|--------------------|----------| -| Galaxy Note20 | Phone | -| Galaxy Note20 Ultra| Phone | -| Galaxy S20 | Phone | -| Galaxy S20 FE | Phone | -| Galaxy S20 Ultra | Phone | -| Galaxy S20+ | Phone | -| Galaxy S21 | Phone | -| Galaxy S22 5G | Phone | -| Galaxy Tab A 10.1 | Tablet | -| Pixel 2 | Phone | -| Pixel 2 XL | Phone | -| Pixel 3 XL | Phone | -| Pixel 3a | Phone | -| Pixel 3a XL | Phone | -| Pixel 4 | Phone | -| Pixel 4 XL | Phone | -| Pixel 4a | Phone | -| Pixel 4a 5G | Phone | -| Pixel 5 | Phone | -| Pixel 5a 5G | Phone | - -| Device Name | Category | -|------------------------|----------| -| ASUS ZenFone 8 | Phone | -| Galaxy A33 5G | Phone | -| Galaxy A51 / A51 5G | Phone | -| Galaxy A53 5G | Phone | -| Galaxy A71 / A71 5G | Phone | -| Galaxy A73 / A73 5G | Phone | -| Galaxy A90 5G | Phone | -| Galaxy Fold / Fold 5G | Foldable | -| Galaxy Note10 / Note10 Lite / Note10+ | Phone | -| Galaxy Note20 / Note20 Ultra | Phone | -| Galaxy S10 / S10 Lite / S10+ / S10e | Phone | -| Galaxy S20 | Phone | -| Galaxy S21 / S21 FE / S21+ / S21 Ultra | Phone | -| Galaxy S22 / S22+ / S22 Ultra | Phone | -| Galaxy Tab S6 / S6 Lite / S6 5G | Tablet | -| Galaxy Tab S7 / S7+ 5G | Tablet | -| Galaxy Tab S8 / S8+ / S8 Ultra | Tablet | -| Galaxy Z Flip / Z Flip 5G / Z Flip3 5G | Foldable | -| Galaxy Z Fold2 / Z Fold3 5G | Foldable | -| LG Velvet (5G / LTE) | Phone | -| LG Wing 5G | Phone | -| OnePlus 10 Pro | Phone | -| OPPO Find X5 Pro | Phone | -| Pixel 4 / 4 XL / 4a / 4a 5G | Phone | -| Pixel 5 / 5a 5G | Phone | -| Pixel 6 / 6 Pro | Phone | -| Realme GT2 Pro | Phone | -| Vivo X80 Pro | Phone | -| Xiaomi 12 / 12 Pro | Phone | - -| Device Name | Category | -|-------------------|----------| -| ASUS ZenFone 8 | Phone | -| Galaxy A33 5G | Phone | -| Galaxy A51 / A51 5G | Phone | -| Galaxy A53 5G | Phone | -| Galaxy A71 / A71 5G | Phone | -| Galaxy A73 / A73 5G | Phone | -| Galaxy A90 5G | Phone | -| Galaxy Fold / Fold 5G | Foldable | -| Galaxy Note10 / Note10 Lite / Note10+ | Phone | -| Galaxy Note20 / Note20 Ultra | Phone | -| Galaxy S10 / S10 Lite / S10+ / S10e | Phone | -| Galaxy S20 | Phone | -| Galaxy S21 / S21 FE / S21+ / S21 Ultra | Phone | -| Galaxy S22 / S22+ / S22 Ultra | Phone | -| Galaxy Tab S6 / S6 Lite / S6 5G | Tablet | -| Galaxy Tab S7 / S7+ 5G | Tablet | -| Galaxy Tab S8 / S8+ / S8 Ultra | Tablet | -| Galaxy Z Flip / Z Flip 5G / Z Flip3 5G | Foldable | -| Galaxy Z Fold2 / Z Fold3 5G | Foldable | -| LG Velvet (5G / LTE) | Phone | -| LG Wing 5G | Phone | -| OnePlus 10 Pro | Phone | -| OPPO Find X5 Pro | Phone | -| Pixel 4 / 4 XL / 4a / 4a 5G | Phone | -| Pixel 5 / 5a 5G | Phone | -| Pixel 6 / 6 Pro | Phone | -| Realme GT2 Pro | Phone | -| Vivo X80 Pro | Phone | -| Xiaomi 12 / 12 Pro | Phone | - -| Device Name | Category | -|-------------------|----------| -| Pixel 6a | Phone | -| Pixel 7 | Phone | -| Pixel 7 Pro | Phone | -| Pixel 4a | Phone | -| Pixel 5 | Phone | -| Pixel 5a 5G | Phone | - -## Simulators - -| Device Name | Category | -|--------------------------------------|----------------| -| iPhone SE (3rd generation) | Phone | -| iPhone 14 | Phone | -| iPhone 14 Plus | Phone | -| iPhone 14 Pro | Phone | -| iPhone 14 Pro Max | Phone | -| iPad (9th generation) | Tablet | -| iPad Pro (11-inch) (3rd generation) | Tablet | -| iPad Pro (12.9-inch) (5th generation)| Tablet | -| iPad Air (5th generation) | Tablet | -| iPad mini (6th generation) | Tablet | - -| Device Name | Category | -|--------------------------------------|----------| -| iPhone SE (3rd generation) | Phone | -| iPhone 15 | Phone | -| iPhone 15 Plus | Phone | -| iPhone 15 Pro | Phone | -| iPhone 15 Pro Max | Phone | -| iPad Air (5th generation) | Tablet | -| iPad (10th generation) | Tablet | -| iPad mini (6th generation) | Tablet | -| iPad Pro (11-inch) (4th generation) | Tablet | -| iPad Pro (12.9-inch) (6th generation)| Tablet | diff --git a/static/docs/hyperexecute-enterprise-faqs.md b/static/docs/hyperexecute-enterprise-faqs.md deleted file mode 100644 index 12b3ad120..000000000 --- a/static/docs/hyperexecute-enterprise-faqs.md +++ /dev/null @@ -1,13 +0,0 @@ -# HyperExecute Enterprise FAQs - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -#### 1. **Does HyperExecute provide enterprise level features?** - - HyperExecute provides you with the best in class enterprise features including SSO, dedicated chat support, on-prem and dedicated cloud setups, custom retention period, usage analytics, advance access control, workflow management, and much more. - -#### 2. **Can I use my own vault for testing with HyperExecute?** - - HyperExecute enterprise version allows organizations to integrate their custom vaults with the HyperExecute testing infrastructure for greater flexibility. - ->If you have more questions, then just give us a shout and we will answer them for you. diff --git a/static/docs/hyperexecute-environment-variable-setup.md b/static/docs/hyperexecute-environment-variable-setup.md deleted file mode 100644 index a285fb3ad..000000000 --- a/static/docs/hyperexecute-environment-variable-setup.md +++ /dev/null @@ -1,50 +0,0 @@ -# HyperExecute Environment Variable Setup - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -To provide you with a secure [cross-browser testing](https://www.testmuai.com) environment on-cloud, we recommend integrating TestMu AI Authentication Credentials in your test scripts. This would allow our cloud servers to establish a secure connection between your local system and [HyperExecute](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/). This document will help you understand how to set up HyperExecute environment variable for your respective operating system. This set up would also help authenticate your colleague's test cases when executed against an individual account. - -## Where to find TestMu AI Authentication Credentials? - -TestMu AI Authentication Credentials consists of two fields: - -1. TestMu AI Username -2. TestMu AI Access Key - ->You can find these authentication credentials by either visiting the [HyperExecute Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/) or your [TestMu AI Profile](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). - -#**Step 1:** Go to the right side Profile dropdown and select **Account Settings**. - -**Step 2:** Under Account Settings, go to **Password & Security** - -**Step 3:** You will find your LT Username and Access Key here, this would be required in order to trigger a Job using the [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) - -You can directly copy your Username and Access Key using shortcut key button. - -## Configuring Environment Variables on your Operating System - -Before the tests are run, please set the environment variables LT_USERNAME & LT_ACCESS_KEY from the terminal. The account details are available on your [TestMu AI Profile](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile) page. - -### Download the HyperExecute CLI - -The CLI is used for triggering the tests on HyperExecute. It is recommend to download the CLI binary on the host system and keep it in the root directory of the suite to perform the tests on HyperExecute. - -You can download the CLI for your desired platform from the below mentioned links: - -| Platform | HyperExecute CLI | -| ---------| ---------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| MacOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -### Setup Environment Variable - -Now, you need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in the [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). - -Run the below mentioned commands in your terminal to setup the CLI and the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} diff --git a/static/docs/hyperexecute-espresso-testing.md b/static/docs/hyperexecute-espresso-testing.md deleted file mode 100644 index 03fa50917..000000000 --- a/static/docs/hyperexecute-espresso-testing.md +++ /dev/null @@ -1,286 +0,0 @@ -# Espresso Testing On HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This page outlines how to execute your Espresso tests on HyperExecute with [YAML 0.2](/support/docs/hyperexecute-yaml-version0.2/) - -## Prerequisites - -To run the Tests on HyperExecute from your Local System, you are required: - -- Your TestMu AI [Username and Access key](/support/docs/hyperexecute-how-to-get-my-username-and-access-key/) -- [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) in order to initiate a test execution Job . -- Setup the [Environmental Variable](/support/docs/hyperexecute-environment-variable-setup/) -- [HyperExecute YAML](/support/docs/hyperexecute-yaml-version0.2/) file which contains all the necessary instructions. -- An Android app (.apk or .aab file). - -## Step 1: Setup Your Test Suite - -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** - -Download or Clone the code sample for the Espresso framework from the TestMu AI GitHub repository to run the tests on the HyperExecute. - - View on GitHub - -## Step 2: Setup the CLI in your Test Suite - -After cloning / downloading the sample repo, you need to setup the CLI and the environment variables. - -### Download the HyperExecute CLI - -The CLI is used for triggering the tests on HyperExecute. It is recommend to download the CLI binary on the host system and keep it in the root directory of the suite to perform the tests on HyperExecute. - -You can download the CLI for your desired platform from the below mentioned links: - -| Platform | HyperExecute CLI | -| ---------| ---------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| MacOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -### Setup Environment Variable - -Now, you need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in the [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). - -Run the below mentioned commands in your terminal to setup the CLI and the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -## Step 3: Upload your Application - -Upload your _android_ application (.apk file) to the TestMu AI servers using our REST API. You need to provide your Username and AccessKey in the format `Username:AccessKey` in the cURL command for authentication. - -Enter your local path of your android app instead of `` in the below cURL command. - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "appFile=@""" -F "name="sampleApp"" -`} - -> Response of above cURL will be a **JSON** object containing the `App ID` of the format - `` and will be used in the next step. - -## Step 4: Upload your Test Suite - -Upload your **test suite (.apk file)** to the TestMu AI servers using our REST API by providing your **Username** and **AccessKey** in the format `Username:AccessKey` in the cURL command for authentication. - -Here is an example cURL request to upload your app using our REST API: - -Enter your local path of your test suite app instead of `` in the below cURL command. - -## Step 5: Configure YAML in your Test Suite - -Enter your `` and `` in the YAML file that you have fetched in the above step. - -```yaml ---- -version: "0.2" -globalTimeout: 150 -testSuiteTimeout: 150 -testSuiteStep: 150 - -concurrency: 2 - -runson: android - -autosplit: true - -report: true #to generate reports - -framework: -name: "android/espresso" -args: -buildName: "Espresso" -video: true -deviceLog: true -artifacts: true #to generate artifacts - -# You can use either the appId (lt://APP1234567) or provide the path of the application using appPath. Both examples are given below. -appPath: proverbial_android.apk -testSuitePath: proverbial_android_expressotest.apk -# We have used the appPath and testSuitePath here. - -# highlight-next-line -appId: lt:// -# highlight-next-line -testSuiteAppId: lt:// -# We have used the appId and testSuiteAppID here. - -deviceSelectionStrategy: all -devices: ["Galaxy.*", "Pixel.*"] - -jobLabel: ['HYP', 'Espresso', 'Auto Sharding'] -``` - -You can use either the `appId` and `testSuiteAppId` or provide the path of the application using `appPath` and `testSuitePath`. Both examples are given below. - -```yaml ---- -version: "0.2" -globalTimeout: 150 -testSuiteTimeout: 150 -testSuiteStep: 150 - -concurrency: 2 - -runson: android - -autosplit: false - -report: true #to generate reports - -framework: -name: android/espresso -args: -buildName: Espresso -video: true -deviceLog: true -artifacts: true #to generate artifacts - -# highlight-next-line -appPath: proverbial_android.apk -# highlight-next-line -testSuitePath: proverbial_android_expressotest.apk # We have used the appPath and testSuitePath here. -# highlight-next-line -appId: lt://APP123456789 -# highlight-next-line -testSuiteAppId: lt://APP123456789 # We have used the appId and testSuiteAppID here. -deviceSelectionStrategy: all -devices: -- Galaxy.* -- Pixel.* -shards: -mappings: -- name: shard1 -strategy: class -values: -- com.lambdatest.proverbial.BrowserTest -# The strategy for this shard is based on "class". -# This shard will run all tests from the class com.lambdatest.proverbial.BrowserTest. -- name: shard2 -strategy: package -values: -- com.lambdatest.proverbial -# The strategy for this shard is based on "package". -# This shard will run all tests that belong to the package com.lambdatest.proverbial.l. -- name: shard3 -strategy: skipClass -values: -- com.lambdatest.proverbial.BrowserTest -# The strategy for this shard is to skip a specific class. -# This shard will avoid running tests from the class com.lambdatest.proverbial.BrowserTest. - -jobLabel: ['HYP', 'Espresso', 'Manual Sharding'] -``` - -**info** -- The **uploadArtefact** flag is currently not available for Espresso framework tests. -- To run **adb commands** in the **pre** and **post-steps**, include the following two flags in your YAML file: To execute it, you must have purchased an equivalent number of parallels for both HyperExecute and Real Device. - -```yaml -privateCloud: true -preservedDevice: true -``` - -## Step 6: Generate Reports and Artifacts -To generate artifacts for your Espresso tests, add the `artifacts: true`, and `report: true` flag in your YAML file: - -```yaml -report: true - -framework: -name: "android/espresso" -args: -artifacts: true -...// -``` - -To download these artifacts in your local machine, you can pass the `--download-artifacts` and `--download-report` flag with the CLI command to execute the tests as shown in the next step. - -## Step 7: Execute your Test Suite - -> **NOTE :** In case of MacOS, if you get a permission denied warning while executing CLI, simply run **`chmod u+x ./hyperexecute`** to allow permission. In case you get a security popup, allow it from your **System Preferences** → **Security & Privacy** → **General tab**. - -Run the below command in your terminal at the root folder of the project: - -```bash -./hyperexecute --config RELATIVE_PATH_OF_YOUR_YAML_FILE --download-artifacts --download-report -``` - -OR use this command if you have not exported your username and access key in the step 2. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE --download-artifacts --download-report`} - -> Visit the [HyperExecute Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) and check your Job status. - -**Troubleshoot Guide** - -If you are stumbling upon errors during Espresso test execution, then refer to the [**Espresso Troubleshoot**](/support/docs/troubleshoot-espresso-tests/#sharded-espresso-errors) guide for detailed explanations to your common errors. - -## Additional Details - -### Sharding - -**Sharding** is a mobile test automation technique that distributes test cases across multiple real and virtual devices via parallel testing. It entails breaking down test suites into smaller subsets known as shards and running each shard concurrently on separate test execution environments. - -There are two types of Sharding: - -### Manual Sharding -This feature enables users to manually specify shards along with their definitions, offering multiple strategies such as class, package, and more. By doing so, users can have greater flexibility and control over the sharding process. - -### Auto Sharding -The system intelligently determines the distribution of tests across devices, employing specific criteria to optimize the testing process. - -- **When Shards are added :** - - If you are using the `deviceSelectionStrategy: all`,then in that case all the specified shards will be executed on every device available. - - - If you are using the `deviceSelectionStrategy: any`, then in that case all the specified shards will be executed on just one device from the provided list. - -- **When Shards are not added :** - - If you are using the `deviceSelectionStrategy: all`, then in that case the tests will be executed on all mentioned devices in `.yaml` based on the concurrency. - - - If you are using the `deviceSelectionStrategy: any`, then in that case all the specified tests will be executed on each device from the provided list, considering the concurrency setting. - -## Conclusion -By following the instructions in this documentation, you can seamlessly execute the Espresso tests on HyperExecute, leveraging its secure cloud infrastructure, advanced features, and optimized test execution workflow. - -## Using the Espresso Agent Skill with TestMu AI - -The [espresso-skill](https://github.com/LambdaTest/agent-skills/tree/main/espresso-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The espresso-skill package includes: - -``` -espresso-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing Espresso Agent Skill - -Install a Espresso Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/espresso-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/espresso-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only espresso-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). diff --git a/static/docs/hyperexecute-failfast.md b/static/docs/hyperexecute-failfast.md deleted file mode 100644 index 7c1565051..000000000 --- a/static/docs/hyperexecute-failfast.md +++ /dev/null @@ -1,36 +0,0 @@ -# FailFast - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -HyperExecute's FailFast feature lets you to optimize your testing regimen by automatically aborting jobs that surpass a specified number of consecutive failures. This intelligent feature ensures that you receive prompt feedback and maintain a well-orchestrated testing process. - -## `maxNumberOfTests` - -The `maxNumberOfTests` flag serves as a parameter that determines the number of consecutive tests that need to be failed before the job is aborted automatically. - -For instance, if you set the `maxNumberOfTests` flag to 4, then your job will be aborted after four consecutive test failures. However, if three of your tests fail and the fourth one is passed, then the FailFast feature will reset and look for the next four consecutive tests that are failing. - -This allows you to speed up your test execution process and provide you with faster feedback. - -## Implementing FailFast - -To leverage the **FailFast** feature, simply incorporate the following command in your HyperExecute YAML file: - -```yaml -failFast: -maxNumberOfTests: 2 -``` - -To apply `failFast` at the **scenario** level instead of the default test level, set the `level: scenario` in the YAML configuration. - -```yaml -failFast: -maxNumberOfTests: 2 -level: scenario -``` - -> **NOTE :** You can specify `failFast` at either the test level or scenario level, depending on your requirements. Both configurations will work, but can lead to unexpected results, only one should be used at a time. - -For a deeper understanding of the HyperExecute YAML and its versatile features, we invite you to explore our dedicated [HyperExecute YAML documentation](/support/docs/deep-dive-into-hyperexecute-yaml/) page. This resource-rich section provides valuable insights to empower your testing endeavors. - ->If you have more questions, then just give us a shout and we will answer them for you. diff --git a/static/docs/hyperexecute-faqs.md b/static/docs/hyperexecute-faqs.md deleted file mode 100644 index 60e48c8f1..000000000 --- a/static/docs/hyperexecute-faqs.md +++ /dev/null @@ -1,21 +0,0 @@ -# HyperExecute FAQs !! - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -### General - -### YAML - -### CLI - -### Features - -### Platform - -### Pricing - -### Security - -### Enterprise - -> If you have more questions, then just give us a shout and we will answer them for you. diff --git a/static/docs/hyperexecute-feature-faqs.md b/static/docs/hyperexecute-feature-faqs.md deleted file mode 100644 index 7129304a1..000000000 --- a/static/docs/hyperexecute-feature-faqs.md +++ /dev/null @@ -1,35 +0,0 @@ -# HyperExecute Feature FAQs - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -#### 1. **How can I integrate HyperExecute with other CI/CD platforms?** - - Since HyperExecute is operated using a universal CLI, it essentially takes two lines of terminal commands to integrate HyperExecute with any CI/CD platform. You need to download the CLI and trigger it to do this. - -#### 2. **How is HyperExecute different from other testing clouds?** - - While other testing platforms throw infrastructure at the users to run their tests, HyperExecute is a smart orchestration cloud where the platform decides the best execution plan to finish the jobs in the least amount of time. HyperExecute accomplishes this by distributing tests smartly on available resources, providing other features, such as, retrying failed scenarios, automatic reordering and more. HyperExecute acts as a co-pilot for developers to run and triage their tests as fast as possible. - -#### 3. **What smart CI features does HyperExecute provide?** - - HyperExecute provides you with the option to run pre and post steps that allow users to run any command before and after running tests on every VM, define custom environment variables, deep dependency caching, pick a specific version of language runtimes, artifacts management, and more. HyperExecute allows you to move your sophisticated pipelines for end to end testing completely. Triggering and execution of tests happens entirely on HyperExecute infrastructure and frees your CI or local system to do other meaningful tasks. - -#### 4. **Can I test my private web applications on HyperExecute?** - - Absolutely. HyperExecute allows multiple provisions to test private applications by the following directives: automatically running tunnels using the CLI, and by creating dedicated NAT for organizations that can be whitelisted by their firewalls. We have users who are able to run VPN clients in the pre steps before starting their tests which allows them to securely connect to their private workloads. - -#### 5. **Can I do geolocation testing on HyperExecute?** - - Yes, HyperExecute provides geolocation testing by providing capabilities in selenium, playwright, and cypress tests. All the geolocation testing features available on the standard platform are available on HyperExecute. - -#### 6. **Can HyperExecute run my tests on virtual machines with custom specifications?** - - Yes, with the enterprise version, HyperExecute provides directives to use custom virtual machines based on the HyperExecute base virtual machine image. This allows for greater flexibility to run customer frameworks. - -#### 7. **Can I use HyperExecute to run test cases on Microsoft Excel?** - - Yes, you can configure HyperExecute to run your Microsoft Excel test cases. However, this feature is only available on request at this moment. Visit [our documentation](/support/docs/hyperexecute-winapp-integration/) for this feature or contact support@testmuai.com to know more. - -For more information on HyperExecute features, visit this [page](/support/docs/key-features-of-hyperexecute/). - ->If you have more questions, then just give us a shout and we will answer them for you. diff --git a/static/docs/hyperexecute-gattling-testing.md b/static/docs/hyperexecute-gattling-testing.md deleted file mode 100644 index 40d675722..000000000 --- a/static/docs/hyperexecute-gattling-testing.md +++ /dev/null @@ -1,303 +0,0 @@ -# Gatling Automation on HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Gatling is an open-source load and performance testing framework designed for testing web applications. It uses a powerful Scala-based DSL to define test scenarios and is capable of simulating thousands of concurrent users, making it ideal for stress and performance testing. - -## 🚀 Why Choose HyperExecute for Gatling Performance Testing? -While Gatling is powerful on its own, running large-scale performance tests using traditional setups often introduces significant bottlenecks: - -### ⚠️ Common Challenges with Traditional Runners -- **Infrastructure Limitations:** Tests run sequentially or on underpowered machines, leading to long execution times. -- **Manual Setup & Maintenance:** Developers must handle environment configuration, dependency resolution, and runtime orchestration manually. -- **Limited Scalability:** Scaling tests across multiple machines requires complex setup and resource coordination. -- **Debugging Overhead:** Analyzing failures or performance issues often involves sifting through scattered logs and inconsistent outputs. - -### ✅ How HyperExecute Solves These Challenges -TestMu AI’s HyperExecute platform is purpose-built to streamline and supercharge your Gatling test execution: - -- **⚡ Smart Test Distribution:** Automatically splits and runs tests across multiple machines for faster execution and optimal resource utilization. -- **🧪 Isolated, Reproducible Environments:** Each test job runs in its own isolated environment with managed dependencies for consistent results. -- **📊 Built-in Reporting & Artifact Management:** Uploads Gatling reports and logs as artifacts that can be easily accessed and analyzed post-run. -- **🔁 Built-in Retry & Stability Handling:** Supports automatic retries for failed scenarios to minimize false negatives and flakiness. -- **🔍 Deep Debugging Insights:** Rich logs, structured output, and centralized monitoring help you trace performance bottlenecks with ease. - -### 🚀 Result: Scalable, Reliable, and Faster Gatling Test Cycles -By integrating Gatling with HyperExecute, you can move beyond traditional limitations and run large-scale load tests with speed, precision, and confidence. This guide provides a structured approach to integrating the Gatling performance testing framework with the TestMu AI HyperExecute platform. - -## Prerequisites -To run the Tests on HyperExecute from your Local System, you are required: - -- Your TestMu AI [Username and Access key](/support/docs/hyperexecute-how-to-get-my-username-and-access-key/) -- [HyperExecute YAML](/support/docs/hyperexecute-yaml-version0.2/) file which contains all the necessary instructions. -- [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) in order to initiate a test execution Job . -- Setup the [Environmental Variable](/support/docs/hyperexecute-environment-variable-setup/) - -## Run a Sample Project -### Step 1: Download Project -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** -Download or Clone the code sample for the Gatling Performance Testing from the TestMu AI GitHub repository to run the tests on the HyperExecute. - - View on GitHub - -### Step 2: Download the CLI in your Project -Download the HyperExecute CLI and copy it into the root folder of the downloaded sample repository. - -| Platform | HyperExecute CLI | -| ---------| ---------------- | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| macOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | - -### Step 3: Configure your hyperexecute.yml file -The core of HyperExecute configuration lies in the `hyperexecute.yaml` file. Let’s understand how it is constructed, step by step: - -#### 1. Define Test Environment and Execution Strategy -The first step is to define the environment your tests will run on using the runson parameter. You can also configure intelligent parallelization and test exit conditions. - -```yaml -version: 0.1 -runson: linux # OS to run the tests (e.g., linux, win) -autosplit: true -concurrency: 1 # Defines the number of test sessions to run concurrently -scenarioCommandStatusOnly: true -``` - -#### 2. Dependency Resolution with Maven -Before running the actual performance test, ensure all project dependencies are resolved locally for a reproducible build. This step pulls all required Maven dependencies to a local directory (.m2), ensuring environment consistency. - -```yaml -pre: -- mvn -Dmaven.repo.local=./.m2 dependency:resolve -``` - -#### 3. Configure Test Discovery (Optional) -Test discovery determines what files or test suites should be run. You can customize this to dynamically scan simulation classes. - -```yaml -testDiscovery: -type: raw -mode: static -command: echo "Test" -``` -The `command` is a placeholder here. In advanced setups, you can provide a script to programmatically discover test files. - -#### 4. Configure the Test Execution Command -Specify the command that runs your Gatling tests via Maven: - -```yaml -testRunnerCommand: mvn gatling:test -``` - -#### 5. Add Background Services (Optional) -If your test depends on a backend server or application under test, you can spin it up in the background: - -```yaml -background: -- mvn spring-boot:run -Dspring-boot.run.main-class=dev.simonverhoeven.gatlingdemo.GatlingDemoApplication || true -``` - -When `autosplit` is enabled, this command can be intelligently distributed across runners. - -#### 6. Upload Test Reports -Once the test completes, use uploadArtefacts to store Gatling reports: - -```yaml -uploadArtefacts: -- name: TestReport -path: -- target/gatling/** -``` - -These will be visible in the HyperExecute logs UI after the run. - -#### 7. Add Retry Logic (Optional) -You can automatically retry failed tests to avoid flakiness. - -```yaml -retryOnFailure: true -maxRetries: 1 -``` - -Here is a complete working YAML configuration that runs Gatling performance tests on linux runners via HyperExecute: - -```yaml reference title="hyperexecute.yaml" -https://github.com/prateekLambda/Gatling-performance-hyperexecute/blob/main/HYE.yaml -``` - -> 📘 For a deeper understanding and project-specific configurations, check out the [YAML documentation](/support/docs/hyperexecute-yaml-parameters/). - -### Step 4: Execute your Test Suite -From the project root directory, execute the below CLI command in your terminal: - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config hyperexecute.yaml`} - -> **NOTE :** In case of macOS, if you get a permission denied warning while executing CLI, simply run **`chmod u+x ./hyperexecute`** to allow permission. In case you get a security popup, allow it from your **System Preferences** → **Security & Privacy** → **General tab**. - -## Run Gatling Tests via HyperExecute UI - -In addition to CLI-based execution, HyperExecute provides a UI-based approach to configure and run Gatling performance tests directly from the dashboard. - -### Step 1: Setup your Gatling Project - -1. Go to the [HyperExecute Project's Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute/projects) and click on the **New Project** button. -2. Select **Gatling** as your performance testing framework. -3. Browse and upload your Gatling simulation files (`.java`) to create a new performance test project. -4. Click on the **Save** button. - -### Step 2: Configure Test Load Criteria - -Select your simulation file and click on the **Run** button. A configuration modal will appear with the following options: - -#### Test Types - -HyperExecute supports three types of performance tests: - -| Test Type | Description | Use Case | -|-----------|-------------|----------| -| **Capacity Test** | Learn how your application scales and monitor when performance starts to decrease | Determine maximum load capacity | -| **Stress Test** | Simulate a load peak to find out if your application crashes and recovers | Test system stability under extreme conditions | -| **Soak Test** | Simulate regular production use and monitor how your application behaves over time | Identify memory leaks and performance degradation | - -#### Configuration Parameters - -Depending on the selected test type, configure the following parameters: - -**Capacity Test:** -- **Duration (min):** Specify the desired duration for the test -- **Initial Users:** Starting user arrival rate per second -- **Final Users:** Target user arrival rate per second - -**Stress Test:** -- **Duration (min):** Specify the desired duration for the test -- **Total Injected Users:** Total number of virtual users to inject - -**Soak Test:** -- **Duration (min):** Specify the desired duration for the test -- **Constant User Arrival Rate:** Steady rate of users per second throughout the test - -Click **Continue** to proceed to load distribution configuration. - -### Step 3: Configure Test Load Distribution - -Configure how the load should be distributed across regions and machines: - -- **Region:** Select the geographic region(s) for test execution - - East US (Richmond, Virginia) - *default* - - West US 2 (Moses Lake, Washington) - - Central India (Pune, Maharashtra) - - Southeast Asia (Singapore) - - Brazil South (São Paulo State, Brazil) - - Mexico Central (Querétaro State, Mexico) - -- **% of Traffic:** Distribute load percentage across selected regions - -- **Job Labels:** Add meaningful labels or tags for better identification and traceability - -- **Max Users per Engine:** Specify the maximum number of virtual users per engine (default: 2000) - -- **Global Timeout:** Set the maximum duration for the HyperExecute job (default: 90 minutes) - -Click **Run Test** to start the execution. - -### Step 4: Analyze Your Test Results - -After the test execution completes, navigate to the **Jobs** section to analyze results: - -- **Job Summary:** View job duration, test duration, execution time, and status -- **Scenarios Tab:** Monitor individual scenario execution and validate if tests ran for the required duration -- **Logs:** View detailed execution logs including Maven build output and Gatling simulation logs -- **Artifacts:** Access test artifacts including Gatling HTML reports -- **Report:** Download the consolidated HTML report for detailed analysis - -**Download Reports** -Click the **Report** button in the job details page to download the consolidated Gatling HTML report containing response time distributions, throughput graphs, and error analysis. - -### Sample Simulation Code - -To leverage the UI-based configuration, your Gatling simulation should read parameters passed via system properties. Add the following load profile methods to your simulation class: - -```java -/** -* Open workload model - users arrive at a specified rate -*/ -public static OpenInjectionStep openLoadProfile() { -String injectType = System.getProperty("injectType", "constantUsersPerSec"); -int users = Integer.getInteger("users", 10); -int duration = Integer.getInteger("duration", 30); -int rampDuration = Integer.getInteger("rampDuration", 60); -int usersStart = Integer.getInteger("usersStart", users / 2); -int usersEnd = Integer.getInteger("usersEnd", users); - -switch (injectType.toLowerCase()) { -case "soaktest": -// Gradual ramp up followed by sustained load - ideal for soak tests -return rampUsers(users).during(Duration.ofSeconds(rampDuration)); - -case "capacitytest": -// Gradual increase to find capacity limits -return rampUsersPerSec(1).to(users).during(Duration.ofSeconds(duration)); - -case "stresspeakusers": -return stressPeakUsers(users).during(Duration.ofSeconds(duration)); - -case "rampuserspersec": -return rampUsersPerSec(usersStart).to(usersEnd).during(Duration.ofSeconds(duration)); - -case "constantusers": -return rampUsers(users).during(Duration.ofSeconds(rampDuration)); - -default: -return constantUsersPerSec(users).during(Duration.ofSeconds(duration)); -} -} - -/** -* Closed workload model - maintains constant concurrent users -*/ -public static ClosedInjectionStep closedLoadProfile() { -int users = Integer.getInteger("users", 10); -int duration = Integer.getInteger("duration", 30); -String injectType = System.getProperty("injectType", "constantUsersPerSec"); - -switch (injectType.toLowerCase()) { -case "soaktest": -case "capacitytest": -// For soak/capacity tests, ramp up to target concurrent users then maintain -return rampConcurrentUsers(1).to(users).during(Duration.ofSeconds(duration)); -default: -return constantConcurrentUsers(users).during(Duration.ofSeconds(duration)); -} -} - -/** -* Determine which workload model to use based on test type -*/ -private PopulationBuilder getPopulationBuilder() { -String workloadModel = System.getProperty("workloadModel", "open"); -String injectType = System.getProperty("injectType", "constantUsersPerSec"); - -// Use closed model for soak and capacity tests by default -if ("closed".equalsIgnoreCase(workloadModel) || -"soaktest".equalsIgnoreCase(injectType) || -"capacitytest".equalsIgnoreCase(injectType)) { -return scenario.injectClosed(closedLoadProfile()); -} else { -return scenario.injectOpen(openLoadProfile()); -} -} -``` - -**Sample repo** -Download or Clone the code sample for the Gatling Performance Testing from the GitHub repository to run the tests on the HyperExecute. - - View on GitHub - -## Advanced Parameters -Optimize your test pipeline using the following advanced features: - -- ✅ [Smart Test Splitting](/support/docs/hyperexecute-test-splitting-and-multiplexing/) – Automatically distribute tests across parallel runners -- ⚡ [Fail Fast](/support/docs/hyperexecute-failfast/) – Stop test runs on the first failure -- 📊 [Detailed Reports](/support/docs/hyperexecute-reports/) – Real-time terminal logs & rich test reports diff --git a/static/docs/hyperexecute-general-faqs.md b/static/docs/hyperexecute-general-faqs.md deleted file mode 100644 index 609a5652c..000000000 --- a/static/docs/hyperexecute-general-faqs.md +++ /dev/null @@ -1,80 +0,0 @@ -# HyperExecute General FAQs - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -#### 1. **Which testing frameworks does HyperExecute support?** - - Though HyperExecute has a [very deep support](/support/docs/hyperexecute-supported-languages-and-frameworks/) for Selenium, Cypress, Playwright, CDP, Taiko and other web browser testing frameworks, nothing stops it from running any other tests over the HyperExecute platform using YAML. Core orchestration features and real time logs are native to the platform and will be available over any kind of testing. - -#### 2. **Can I run a specific command before running tests on HyperExecute?** - - HyperExecute provides you with smart CI features where you can run any command before and after running tests using pre and post steps. Common use cases include, building/ compiling binaries, downloading dependencies, starting services in the background or any other arbitrary command. - -#### 3. **Do I need to change my code to run my tests on HyperExecute?** - - No code or logic changes have to be done in order to run your end to end tests on HyperExecute. HyperExecute requires a YAML configuration file to determine which tests to run and configure other settings. This YAML file can be created using an online YAML generator, hosted on the HyperExecute onboarding page. - -#### 4. **Where will HyperExecute run my tests?** - - HyperExecute takes your test scripts and places them in virtual machines having all the components required to run your tests and collect logs and metrics. These virtual machines are hosted by HyperExecute in its secure cloud environment. HyperExecute is also available on dedicated and private clouds where virtual machines can run in your personal cloud accounts. - -#### 5. **How to use HyperExecute?** - - Tests are triggered on HyperExecute using [HyperExecute CLI](/support/docs/getting-started-with-hyperexecute/) using a YAML file to configure the executing settings including target OS, concurrency and other CI level information. This YAML file can be generated easily using a web based YAML generator that supports multiple [languages and frameworks](/support/docs/hyperexecute-supported-languages-and-frameworks/). As a user you need to focus on writing tests and running the CLI with YAML, everything else is taken care of by the orchestration platform. - -#### 6. **Why is HyperExecute faster than other end to end testing platforms?** - - Traditional end to end testing platforms add a lot of latency in test execution due to the fragmented components by adding multiple hops. Each and every request has to travel through multiple hops before it can be served by the application under test. This doesn’t just slow tests down, but adds a lot of platform-induced flakiness. HyperExecute puts all the components and the test script in a single isolated environment to run the tests in the fastest and the most efficient way. - -#### 7. **How does HyperExecute help me debug my tests faster?** - - HyperExecute collects every possible log, including terminal logs, selenium and more (for every single test), and stores them separately from each other. These isolated logs ensure that the end user is not spending time in filtering useful logs and focuses on getting the insights from the logs as fast as possible. HyperExecute also streams these test logs in real time to drastically reduce time wasted in debugging failed tests. - -#### 8. **Does HyperExecute provide APIs to consume logs?** - - Yes, HyperExecute provides APIs to consume all the logs generated during test execution for offline usage. - -#### 9. **Does HyperExecute provide the ability to store test artifacts?** - - Yes, HyperExecute provides convenient artifact management where you can upload test artifacts or any files or folders generated during test execution. These artifacts are stored on HyperExecute. HyperExecute exposes APIs to access these as well. - -#### 10. **Does HyperExecute record video recording of the test session?** - - Yes, HyperExecute generates video recording of every single end to end test triggered over the platform. All the features available in the standard automation platform including video, screenshot, network logs and a lot more, are available in HyperExecute. - -#### 11. **Can I run parallel tests using HyperExecute?** - - Absolutely, HyperExecute’s core feature is to provide a massively parallel testing infrastructure where the platform intelligently distributes tests across multiple nodes. - -#### 12. **What are the configurations for HyperExecute virtual machines?** - - HyperExecute provides a generous configuration for virtual machines to run your tests. Windows comes with 7 GB of memory and 4 cores of processors, Linux comes with 4 GB of memory and 2 cores of processors, Mac comes with 7 GB of memory and 4 cores of processors. HyperExecute also provides the ability to customize the execution environment for enterprise accounts. - -#### 13. **Can I group my tests to run in sequence instead of randomly distributing them?** - - Yes, HyperExecute provides directives to group dependent tests together which need to run in sequence. Using remote test discovery, users can write unique test discovery commands for every node, where dependent tests will run in sequence. HyperExecute allows various directives to construct sophisticated testing pipelines to fulfill your use cases. - -#### 14. **Can I run performance tests with HyperExecute?** - - Though HyperExecute currently doesn’t have a deep support for performance testing, nothing stops the users from running performance tests. HyperExecute will still orchestrate and run performance tests in parallel and provide the terminal logs like all other tests. - -#### 15. **In which regions does HyperExecute provide its testing infrastructure?** - - HyperExecute is available in more than 40 cloud availability regions across the globe. HyperExecute is supported by multiple hyper clouds which makes it available in most of the availability zones across the globe. - -#### 16. **How can I open my Microsoft Excel files with HyperExecute?** - - You can access your Microsoft Excel files with HyperExecute by modifying the files that contain your tests. In order to automate the opening of your Excel file, set the `ms:waitForAppLaunch` and `appArguments` capabilities to: - -```bash -cap.setCapability("ms:waitForAppLaunch", 15); -cap.setCapability("appArguments", " /e "); -``` - - This will ensure that your Excel file is opened after 15 seconds, and that it is opened in edit mode. If you want your file to open after a certain period of time, just enter that time in the `ms:waitForAppLaunch` field. - -#### 17. **How can I access my TestMu AI Hub URL?** - - Your TestMu AI Hub URL can be accessed from our [automation page](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build), along with your username and access key. Click on the **Access Key** button on the right side of your screen to retrieve your Hub URL. The image below indicates exactly where you can find your Hub URL. - ->If you have more questions, then just give us a shout and we will answer them for you. diff --git a/static/docs/hyperexecute-generate-sample-yaml.md b/static/docs/hyperexecute-generate-sample-yaml.md deleted file mode 100644 index d25e1c274..000000000 --- a/static/docs/hyperexecute-generate-sample-yaml.md +++ /dev/null @@ -1,42 +0,0 @@ -# Generate your HyperExecute YAML file - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -You can also generate the HyperExecute YAML directly from the HyperExecute Portal. To accomplish this, follow the below mentioned steps: - -**Step 1:** Go to [HyperExecute dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute), click on the **Help Center** and select **Generate YAML** from the dropdown. - -**Step 2:** Select your preferred test automation framework. - -**Step 3:** You will be redirected to the configuring section, where you can configure the HyperExecute YAML parameters. Click on **Generate YAML** button. - -**Step 4:** Once done, you can download the YAML file using the small icon on top. Alternatively, you can copy the contents of the file and modify them further according to your needs on your favorite IDE. - -> HyperExecute supports multiple languages and testing frameworks. See the comprehensive list & samples [here](/support/docs/hyperexecute-supported-languages-and-frameworks/). - -## Sample HyperExecute YAML -A sample HyperExecute YAML file looks like this: - -```yaml ---- -version: 0.1 -runson: linux - -autosplit: true -concurrency: 2 - -pre: -- mvn dependency:resolve - -testDiscovery: -type: raw -mode: static -command: grep 'test name' xml/testng_linux.xml | awk '{print$2}' | sed 's/name=//g' | sed 's/>//g' - -testRunnerCommand: mvn test -Dplatname=linux -Dmaven.repo.local=./.m2 dependency:resolve -DselectedTests=$test - -retryOnFailure: true -maxRetries: 1 - -jobLabel: [selenium-testng, linux, v1, autosplit] -``` diff --git a/static/docs/hyperexecute-github-app-integration.md b/static/docs/hyperexecute-github-app-integration.md deleted file mode 100644 index 7fe1217a4..000000000 --- a/static/docs/hyperexecute-github-app-integration.md +++ /dev/null @@ -1,167 +0,0 @@ -# GitHub App Integration Support for HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -> This is currently in the **Beta** version. - -HyperExecute supports integration with GitHub App, enabling secure, fine-grained, and scalable access to your GitHub repositories without relying on long-lived Personal Access Tokens (PATs). - -With this integration, HyperExecute automatically generates short-lived installation access tokens via GitHub App for all Git-related operations in Projects and Workflows, enhancing security, compliance, and maintainability. - -This integration supports three onboarding paths: -- **TestMu AI's Marketplace GitHub App** (Public Cloud) -- **Your Existing GitHub App** (On-Prem / Private Cloud) -- **Private GitHub App Dedicated to Your Organization** (On-Prem / Air-Gapped) - -## How It Works -1. Install a GitHub App using one of the three supported approaches. -2. Complete the post-installation registration within the TestMu AI platform. -3. HyperExecute backend (Logistics) sends data to Sentinel to persist details in the `github_app_integration` table. -4. For customer-managed GitHub App, create an org-level secret using the Logistics Secrets API: `github_app_private_key_{{git_tenant}}`. - -Once setup is complete, all HyperExecute Project and Workflow Git operations use short-lived installation tokens instead of PAT tokens. The PAT-based flow is used only as a fallback when no GitHub App is configured. - -## Installation Approaches -HyperExecute supports three installation models, depending on your cloud type and organizational setup. - -### Approach 1: TestMu AI's Marketplace GitHub App (Public Cloud) - -**Prerequisites :** - -- Admin permissions in your target GitHub organization - -**Installation Steps** - -1. Navigate to [GitHub Marketplace](https://github.com/marketplace) -2. Search for "TestMu AI" or "HyperExecute" -3. Open the TestMu AI GitHub App listing -4. Click **Install** -5. Select your target GitHub Organization - - For GitHub Enterprise, installation must be completed per organization -6. Configure repository access: - - **All repositories** (recommended) - - **Select repositories** -7. Review the required permissions: - - **Contents** (read/write) - - **Metadata** (read) -8. Click **Install** to complete the installation - -**Post-Installation Configuration** - -After installation, GitHub automatically redirects you to: `https://{{accounts_base_url}}/org-settings/hyperexecute/git-integration` - -The redirect URL includes an `installation_id` parameter required for configuration. - -> **Note:** Only Org Admins can complete the registration. Organization users can view mappings but cannot modify them. - -### Approach 2: Your Existing GitHub App (On-Prem) - -**Prerequisites** - -- A GitHub App already exists in your GitHub organization -- Admin access to the GitHub App -- Required permissions are configured: - - **Contents** (read) - - **Metadata** (read) - -**Installation Steps** - -1. Obtain your app's installation ID from: - - Navigate to **GitHub App → Installations → Installation URL** - - Example URL format: `https://github.com/settings/installations/{installation_id}` - -2. Navigate to the TestMu AI setup page: -``` -https://{{accounts_base_url}}/org-settings/hyperexecute/git-integration -``` - -3. Manually enter: - - `installation_id` - - Additional app details as required - -> **Note:** For GitHub Enterprise setups with multiple organizations, repeat this setup for each organization. - -**Additional Required Configuration** - -Create an org-level secret via the Logistics Secret API: -``` -github_app_private_key_{{git_tenant}} -``` - -This private key is used to generate short-lived tokens internally. - -### Approach 3: Private GitHub App (On-Prem Dedicated App) - -**Prerequisites** - -- Admin access in your target GitHub organization -- Private GitHub App installation URL shared by the TestMu AI/Workflow team -- Setup URL inside the GitHub App updated to the TestMu AI private setup URL - -**Installation Steps** - -1. Navigate to the private app installation URL: -``` -https://github.com/apps/{private-app-name}/installations/new -``` - -2. Select your GitHub organization - - Enterprise customers must repeat this process for each organization - -3. Configure repository access permissions - -4. Review and confirm the installation - -**Post-Installation Configuration** - -GitHub will redirect you to the private setup URL configured for your environment: -``` -https://{{private_accounts_base_url}}/org-settings/hyperexecute/git-integration -``` - -The redirect contains the `installation_id` parameter, which is required for configuration. - -> **Note:** Only Org Admins can complete this configuration step. - -## Multi-Tenant GitHub Support - -HyperExecute supports connecting your organization to **multiple GitHub instances** at the same time. This is useful when your team works across both GitHub.com and a private GitHub Enterprise Server (e.g., `ghes.yourcompany.com`). - -Each GitHub instance is called a **tenant**, identified by the hostname of the Git provider. - -### How It Works - -When you create a project, HyperExecute automatically detects the tenant from your repository URL: - -| Repository URL | Detected Tenant | -|---------------|----------------| -| `https://github.com/my-org/my-repo` | `github.com` | -| `https://ghes.yourcompany.com/my-org/my-repo` | `ghes.yourcompany.com` | - -Each tenant maintains its own: -- GitHub App configuration (App ID, Installation ID) -- RSA private key for token generation -- Short-lived installation access tokens - -Credentials are **fully isolated** between tenants, tokens and keys from one GitHub instance are never used for another. - -### What You Can Do - -- Connect to **multiple GitHub instances** (e.g., GitHub.com + GitHub Enterprise Server) -- Have **multiple GitHub Apps** on the same tenant for different GitHub organizations -- Each TestMu AI org can have **one GitHub App per (tenant + GitHub org name)** combination - -### Setup for Non-Default Tenants - -For GitHub Enterprise Server or any tenant other than `github.com`: - -1. Follow [Approach 2](#approach-2-your-existing-github-app-on-prem) or [Approach 3](#approach-3-private-github-app-on-prem-dedicated-app) to install the GitHub App on your enterprise instance. - -2. Create an **org-level secret** for the tenant's private key: - `github_app_private_key_` - For example, if your GitHub Enterprise Server is `ghes.yourcompany.com`: - `github_app_private_key_ghes.yourcompany.com` - -3. Complete the registration on the TestMu AI setup page. The tenant is recorded automatically based on the GitHub App's origin. - -For the default tenant (`github.com`), the platform's built-in Marketplace GitHub App handles key management automatically, no manual secret creation is needed. diff --git a/static/docs/hyperexecute-gitignore.md b/static/docs/hyperexecute-gitignore.md deleted file mode 100644 index 85e8b0c86..000000000 --- a/static/docs/hyperexecute-gitignore.md +++ /dev/null @@ -1,36 +0,0 @@ -# Ignore Unnecessary Files in your Payload - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -When you trigger your job, [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) creates a zip of your test scripts before sending it for execution on the HyperExecute Platform. - -If you want to ignore any un-necessary files during this process, you can use the `.hyperexecuteignore` file. The HyperExecuteIgnore file serves as a configuration file to specify files that should be excluded when HyperExecute prepares a zipped archive of your test scripts. This functionality is similar to the `.gitignore` file used in Git version control. - -## Benefits -Excluding unnecessary files from the archive offers several advantages: - -- **Improved Security and Privacy :** HyperExecuteIgnore allows you to exclude sensitive information like log files, hidden files, or any other private data from the test archive. This helps maintain security and protects confidential information during test execution. -- **Reduced Archive Size :** By omitting irrelevant files, the zipped archive containing your test scripts becomes smaller. This is beneficial for storage and transmission purposes. -- **Faster Testing Process :** A smaller archive size can potentially lead to a faster test execution process, as less -data needs to be transferred and uploaded to the testing environment. - -## How to write `.hyperexecuteignore` file? - -The HyperExecuteIgnore file follows a format similar to the .gitignore file used with Git. It allows the use of wildcards to exclude patterns of files based on specific naming conventions. - -```bash -examplename -hyperexecute.exe -*.type -examplename/ -``` -The above file would make sure that following are excluded while the zip is created: - -1. **examplename**: All files and folder with the name `examplename`. -2. **examplename.type**: All files with the name `examplename` and format `type`. -3. ***.type**: All files of the format `type`. -4. **examplename/**: All files and folders under the parent folder called `examplename`. - -- By leveraging `.hyperexecuteignore` file, you can achieve similar functionality to `.gitignore` and streamline the test execution process by keeping the archive size minimal. - -- If both files are present in the directory then the files listed in the `.gitignore` only are ignored. The items listed in `.gitignore` file take **precedence** over the items listed under `.hyperexecuteignore`. diff --git a/static/docs/hyperexecute-global-post-artifacts.md b/static/docs/hyperexecute-global-post-artifacts.md deleted file mode 100644 index 0228e51f1..000000000 --- a/static/docs/hyperexecute-global-post-artifacts.md +++ /dev/null @@ -1,173 +0,0 @@ -# Download and Process Job Artifacts in Global Post - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The **`globalPost`** step can download every task's artifacts onto a VM after your HyperExecute job finishes, run the shell commands you specify against them, and place the processed output back into the job's **Artifacts** section on the dashboard. It can also email the final result to a list of recipients. This lets you turn the raw artifacts a job produced into a custom report or summary, entirely within HyperExecute and without any changes to your test framework code. - -> 📘 This page covers the artifact download and processing behavior of `globalPost`. For the base `globalPost` step (running cleanup commands after a job), see [`globalPost`](/support/docs/deep-dive-into-hyperexecute-yaml/#globalpost) in the YAML deep dive. - -## Why Process Artifacts in Global Post? - -Consider a long HyperExecute pipeline that produces reports, logs, and screenshots across dozens of tasks. Traditionally, to merge those into a single client-specific report, you would wait for the whole job to finish and then run a separate post-processing stage in an external CI/CD pipeline. That means your outer pipeline has to stay alive and keep tracking the job the whole time, just to do a bit of work at the end. - -With artifact processing in `globalPost`, that final step moves inside HyperExecute. After all tasks complete, HyperExecute gathers the artifacts, runs your commands against them, and delivers the result, so your outer pipeline no longer has to babysit the job. You can terminate it early, for example with the CLI's `--no-track` flag, and still get your report generated and emailed. - -**Run your tests → HyperExecute collects every task's artifacts → your commands process them → results appear in the dashboard and inbox** - -### Benefits - -- **No external post-processing pipeline:** Merge reports, build summaries, or transform data right after the job, without a separate CI/CD stage. -- **Terminate your pipeline early:** Because processing runs inside HyperExecute, your outer pipeline can exit as soon as tests are queued (for example, with `--no-track`) and still receive the final report. -- **No test framework changes:** The processing is defined entirely in YAML, so you don't touch your test code to produce a custom report. -- **Custom, client-specific output:** Run any shell command against the artifacts to shape the exact report or artifact bundle your team or customer needs. - -## Before You Begin - -For the artifact-download step to run, all of the following must be true. If any is missing, the download step is silently skipped and `globalPost` just runs its commands as before, without downloading anything. - -- [`uploadArtifacts`](/support/docs/hyperexecute-artifacts/) is configured in your YAML so that your tasks actually produce artifacts to download. -- `commands` under `globalPost` is not empty. -- `downloadArtifacts: true` is explicitly set under `globalPost`. - -## Configuration - -Add a `globalPost` section to your HyperExecute YAML and set `downloadArtifacts: true`: - -```yaml -globalPost: -mode: remote -commands: -- echo "Downloaded artifacts are at: ${ARTIFACTS_DIR}" -- echo "Place upload files here: ${UPLOAD_DIR}" -- ls ${ARTIFACTS_DIR} -- mkdir -p ${UPLOAD_DIR}/processed-reports -- # ... your processing commands here ... -runson: linux -downloadArtifacts: true -email: -to: -- alice@yourcompany.com -- bob@yourcompany.com -templatePath: mailtemplates/template.html -``` - -### Fields - -| Field | Required | Description | -|---|---|---| -| `mode` | Yes | Must be `remote` for artifact downloads to work. | -| `commands` | Yes | The commands to run after artifacts are downloaded. They run in order. | -| `runson` | Yes | The OS to run commands on: `linux`, `win`, or `mac`. | -| `downloadArtifacts` | Yes | Set to `true` to enable artifact downloading. | -| `email` | No | Configure email recipients and template for the final output. | -| `email.to` | No | List of email addresses to receive the results. | -| `email.templatePath` | No | Path to a custom HTML email template (advanced use). | -| `disableEmail` | No | Set to `true` to skip sending emails entirely. | - -## Working With Downloaded Artifacts - -Once your tasks finish, HyperExecute downloads their artifacts to a VM and runs your `commands` in order. Your commands read the downloaded artifacts from one directory and write anything they want uploaded to another. Both locations are exposed as environment variables you can reference in your commands. - -### `ARTIFACTS_DIR` — where the downloaded artifacts live - -Artifacts from all tasks land under `ARTIFACTS_DIR`, with one top-level folder per task ID. Inside each task folder, the artifacts keep their original upload structure (a folder per artifact name, then the original hierarchy). - -```text -$ARTIFACTS_DIR/ -├── task-id-1/ -│ ├── artifact-name-1/ -│ │ ├── homepage.png -│ │ └── checkout.png -│ └── artifact-name-2/ -│ └── output.log -├── task-id-2/ -│ ├── artifact-name-1/ -│ │ └── login.png -│ └── artifact-name-2/ -│ └── output.log -``` - -### `UPLOAD_DIR` — where to place your output - -Write anything you want returned to the job into `UPLOAD_DIR`. Everything placed here becomes visible in the job's **Artifacts** section on the HyperExecute dashboard. The folder structure you create is preserved exactly, so even a deep, multi-level layout is uploaded as-is. - -```text -$UPLOAD_DIR/ -├── merged-report.html ← created by user commands -├── summary.json -└── processed/ -└── final-results.csv -``` - -If `downloadArtifacts` is enabled but your commands don't write anything to `${UPLOAD_DIR}`, `globalPost` still completes successfully. There is simply nothing new in the **Artifacts** section — an empty upload is not treated as a failure. - -## Email Delivery - -When `email.to` is configured, HyperExecute sends an email after your `globalPost` commands finish and the results are uploaded. The email includes the job number and dashboard link, the size of the uploaded artifact zip, and a direct download link for the artifacts. - -### Reusing your `uploadArtifacts` email config - -If you do not specify `email` under `globalPost` but your YAML's [`uploadArtifacts`](/support/docs/hyperexecute-artifacts/) section already has its own email configuration, that configuration is reused automatically for the `globalPost` results. You do not need to repeat it. - -### Disabling email - -To suppress email entirely, set `disableEmail: true` under `globalPost`. This is useful when you want the processing and upload but not a notification, and it overrides any email configured under `uploadArtifacts`. - -```yaml -globalPost: -mode: remote -runson: linux -downloadArtifacts: true -commands: -- ls ${ARTIFACTS_DIR} -disableEmail: true -``` - -### Custom email templates - -`email.templatePath` points to a custom HTML email template. This is an advanced option — the default template covers most needs. The template is a [Go template](https://pkg.go.dev/text/template) and supports the following variables: - -| Variable | Description | -|---|---| -| `{{.JobNumber}}` | The HyperExecute job number. | -| `{{.DownloadLargeFileURL}}` | Signed download link for the artifact zip. | -| `{{.DashboardURL}}` | Link to the job in the HyperExecute dashboard. | -| `{{.FileSize}}` | Human-readable zip size (for example, `15.2 MB`). | -| `{{.Timestamp}}` | When the artifacts were generated. | - -## Size and Time Limits - -| What | Limit | -|---|---| -| Maximum artifact download size | 10 GB | -| Maximum upload size | 10 GB | -| Download timeout | 30 minutes | -| Upload timeout | 30 minutes | - -If the total size of the artifacts to download exceeds **10 GB**, the download step fails with an error. If your jobs produce more than this, scope `uploadArtifacts` to only the files you actually need to process. - -## Frequently Asked Questions - -Why aren't my artifacts downloading? - -The download step runs only when every prerequisite is met. Confirm that `downloadArtifacts: true` is set, that [`uploadArtifacts`](/support/docs/hyperexecute-artifacts/) is configured, and that `commands` is not empty. If any of these is missing, `globalPost` runs your commands without downloading anything. - -Why is ARTIFACTS_DIR empty? - -The directory is populated from the artifacts your tasks upload. If it's empty, your tasks likely didn't produce any artifacts — verify that they actually upload them via [`uploadArtifacts`](/support/docs/hyperexecute-artifacts/). - -Why didn't I get an email? - -Check that `email.to` contains valid addresses and that `disableEmail` is not set to `true`. Remember that `disableEmail: true` suppresses email even when `uploadArtifacts` has its own email configuration. - -Why is the upload directory empty in the dashboard? - -Only files your commands write into `${UPLOAD_DIR}` are uploaded. Make sure your commands actually create output there. If they don't, the job still completes successfully — there's just nothing new to show in the **Artifacts** section. - -Why is my download failing on a large job? - -The total artifacts to download cannot exceed **10 GB**; above that, the download step fails with an error. Narrow `uploadArtifacts` to just the files you need to process so you stay under the limit. - -Why are my commands failing? - -Commands run on the OS set in `runson` (`linux`, `win`, or `mac`). Make sure the tools and scripts you call are installed and compatible with that operating system. diff --git a/static/docs/hyperexecute-gui-query.md b/static/docs/hyperexecute-gui-query.md deleted file mode 100644 index 627e034af..000000000 --- a/static/docs/hyperexecute-gui-query.md +++ /dev/null @@ -1,84 +0,0 @@ -# Search Query in GUI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## What is a Query? - -Think of a query as a smart search filter that automatically finds and selects the exact tests you want to run. Instead of manually clicking through hundreds of test checkboxes, you simply describe what you're looking for using simple English-like instructions. - -**Key Benefits** - -- **Dynamic updates:** As new tests matching your criteria are added, they **automatically appear** in the results. This ensures you’re always working with the most up-to-date set. -- **Precise test selection:** Combine multiple criteria like annotations, file locations and names to pinpoint exactly the tests you need for any given configuration. -- **Cleaner workspace:** Only the relevant tests show up, keeping your test view streamlined and easy to navigate. Also, it helps decrease the manual errors. - -## Understand the Query Structure -Every query follows this simple pattern: `. ""` - -For example: `suiteTest.name = "LoginTest"` where, -- `suiteTest` : what to look at (test within a suite) -- `name` : what property (the name of the test) -- `=` : how to compare (exactly equals) -- `"LoginTest"` : what value (the specific name) - -## Write Your First Query - -1. Navigate to your configuration details screen -2. Click on the **"Test List"** tab -3. Toggle to **"Query Selection"** mode -4. You'll see a query input box with helpful dropdowns - -Let's build a query to find all the Suite (XML) Tests which have **login** in their name. - -### Step 1: Choose What to Look At -Start typing in the query box. You'll immediately see a dropdown with options: - -- `suite` : XML test suites -- `suiteTest` : Tests within suites -- `suiteClass` : Test classes within suites -- `suiteMethod` : Individual test methods within suites -- `fileClass` : Independent test classes (which are not part of any suite) -- `fileMethod` : Methods in independent classes - -Let's choose `suiteTest` because we want to find tests within suites - -### Step 2: Select a Property to Check -After selecting `suiteTest`, another dropdown appears: - -- `name` : Name of the suite, class, or method -- `path` : File path of the test in your project -- `annotation_name` : Annotations like @Smoke, @Regression - -Select `name` or type it because we want to check test names. Your query now looks like `suiteTest.name`. - -### Step 3: Pick a Comparison Operator -Once you’ve typed or selected the property, a dropdown shows comparison options: - -- `=` : Equals (matches exactly) -- `~` : Contains (matches part of the text in value) -- `!~`: Not Contains (shouldn't contain part of the text in value) -- `!=` : Not Equals (excludes exact matches) - -Choose `~` because we want names that contain "login". Your query now looks like: `suiteTest.name ~` - -### Step 4: Enter the Value - -Type the value you’re looking for in quotes, like `"login"`. The app suggests values based on your project. With this, our query is complete and now looks like: `suiteTest.name ~ "login"` - -Press Enter and all the Suite Tests which contain the word "login" in their name will be listed down in a tree-like format. **This tree-like format** shows you the complete structure with the lowest entity showing the runnable test (`methods`, in case of TestNG). - -### Add Multiple Conditions - -Want to be more specific? Add logical operators to combine conditions. After your first condition, you'll see a dropdown: - -- `AND` : Both conditions must be true -- `OR` : Either condition can be true -- `NOT` : Exclude what follows - -**Example:** Find login tests that are also smoke tests - -`suiteTest.name ~ "login" AND suiteTest.annotation_name ~ "Smoke"` - -## Save the Query - -You can save your query in order to dynamically select tests whenever you come back to that specific Configuration. Queries are saved at **Configuration level** so that you can keep unique Configurations for different types of testing scenarios. For example, A configuration for smoke tests may have a saved query like `suiteTest.name ~ "Smoke"`. diff --git a/static/docs/hyperexecute-guided-walkthrough.md b/static/docs/hyperexecute-guided-walkthrough.md deleted file mode 100644 index 483b37069..000000000 --- a/static/docs/hyperexecute-guided-walkthrough.md +++ /dev/null @@ -1,106 +0,0 @@ -# Guided Walkthrough of HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This page gives you a guided walkthrough of all the things on the HyperExecute Dashboard and how you can get the most out of the platform once you have successfully [run your first sample Job](/support/docs/hyperexecute-running-your-first-job/). The dashboard is built to be **WCAG-compliant**, ensuring an accessible experience for all users. - -## Jobs Page - -All the tests on HyperExecute get executed as a Job. All the Jobs can be found on our [Jobs page](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute). Each [Job](/support/docs/hyperexecute-status/#1-job-level-status) has a [`Status`](/support/docs/hyperexecute-status/#3-stage-level-status), a unique `Job Number`, a set of [`Labels`](/support/docs/deep-dive-into-hyperexecute-yaml/#joblabel), info on number of [`Tasks`](/support/docs/hyperexecute-status/#2-task-level-status) executed as part of that Job, and a `Summary` section followed by the details of the `Job Duration` and who initiated the Job. - -> Learn more about [Secrets Management](/support/docs/hyperexecute-how-to-save-and-manage-secrets/), getting [User Name and Access Key](/support/docs/hyperexecute-how-to-get-my-username-and-access-key/) from our [Knowledge Base](/support/docs/hyperexecute-knowledge-base/) - -### Top Panel - -The top panel contains multiple items: - -- **Parallel**: It reflects the number of tasks running parallelly -- **Queued**: It shows number of the tasks queued and are not yet started. -- **Configure Tunnel**: Tunnel is used to ensure the security of your data for private websites. -- **Announcements**: The latest announcements and updates can be found here. -- **Settings**: Manage all of your account and organisation details in this section. -- **Upgrade**: You can upgrade your TestMu AI subscription from here. - -### Filters - -The Jobs page provides a filter bar to narrow down the job list based on the following criteria: - -- **Status**: Filter jobs by their current status (Running, Completed, Failed, Aborted, etc.) -- **Label**: Filter by job labels defined in your YAML configuration. -- **Users**: Filter jobs by the user who triggered them. This is useful when multiple team members share the same organization and you want to view only your jobs. -- **Type**: Filter by job type (Selenium, Playwright, Cypress, etc.) -- **Date**: Filter jobs by date range. -- **View**: Toggle between different view options (e.g., archived jobs). - -To filter jobs for a specific user, click on the **Users** filter and select the desired user from the dropdown list. - -## Job Details Page - -You can click on a particular Job to navigate to the Job Details Page. - -- On the top you can see all the Job related details like Job Number, Summary etc. - - **Job Number** - The Number of your Job. - - **Summary** - A brief breakdown of all of your tests on the basis of their status. - - **Status** - The status of your Job. - - **Created At** - The date of the creation of the Job. - - **User** - Your user details who triggered the Job. - - **Job Duration**: The time it took to run your entire Job. - - **Test Duration**: The time it took to run your tests. - - **Type**: The job type (Selenium, Playwright, etc) along with the mode of the job (AutoSplit or Matrix). - - **Git Commit ID** - For jobs triggered via [Workflows](/support/docs/hyperexecute-workflows/) or [sourcePayload](/support/docs/hyperexecute-how-to-configure-sourcePayload/), the git commit hash is displayed in the job metadata, providing traceability to the specific code changes tested. - - **[Artifacts](/support/docs/hyperexecute-artifacts/)**: Can be used to configure and generate custom reports and artifacts. - - **[Report](/support/docs/hyperexecute-reports/)**: A detailed report for your test execution for the Job. -- The left side shows all the Tasks (Just-in-time testing environments) that were a part of the Job. -- Each Task has 3 types of Stages : [Pre stages, Scenario stages, Post stages](/support/docs/hyperexecute-status/#3-stage-level-status). - -## Test Details - -You can access a much more detailed section for your test cases on the Automation dashboard by clicking on the **Test** button. - -It provides you with all the necessary data points that you need for your test. - -### Basic Summary -You can see the details of your test on the panel on the top of the screen. It contains several details like the browser, the test status, the test name, and more. - -- **Test Status**: The status of your test. If the test has passed, the status will be passed. -- **Test Name**: The name of your test. You can rename the test, copy the name and see all the tests in the build. -- **Test ID**: The test ID of your test. You can copy the test ID by clicking on the **Copy** button. -- **Browser**: The browser on which your tests are run. -- **OS**: The Operating System on which your tests are run. -- **Resolution**: The resolution of the screen. -- **Duration**: The amount of time it took for your test to run. -- **Test Meta Data**: You can view the test meta data by clicking on the **info** icon. -- **Tunnel Information**: Tells whether you have used a tunnel to run your tests or not. -- **Share**: You can share the details of this test by clicking on the **Share** icon. -- **Delete**: You can delete the test entirely by clicking on the **Delete** icon. -### Videos and Screenshots - -HyperExecute provides you with tools like videos and screenshots of your tests to give you a clear picture of what goes on in the background. - -### Commands and Logs - -You can access the commands involved in the test, and the logs on the right hand side on this Dashboard. - -- **All Commands**: Each command that is run as a part of your test case is shown here. Each command also contains its own parameters, that can be accessed by hovering on the command and clicking on **View Parameters**. -- **Network**: Access the network logs while your tests were getting executed on the **Network** tab. -- **Logs**: View all kinds of logs at one place. You can toggle between the **Console Logs**, **Terminal Logs** and the **Test Framework Logs**. - ->**Note**: You can download the logs by clicking on the Download button on the top right-hand corner of the logs tab. - -## Test Summary Hyperlink - -This introduces clickable links within your **test summary**. When you click on these hyperlinks, it dynamically redirects you to the automation dashboard, providing immediate access to detailed test information corresponding to the specific test status clicked. - -Additionally, upon redirection, the automation dashboard opens the relevant tab that aligns with the selected test status, streamlining your experience. - -### How to Activate Test Summary Hyperlinks - -- #### Update your Org Configuration : - -- #### Frontend Configuration : - - > For frontend activation, please contact us via our **24×7 chat support** or you can also drop a mail to **support@testmuai.com**. - -- #### Backend Configuration : - - Update your designated HyperExecute YAML file with [`buildConfig`](/support/docs/deep-dive-into-hyperexecute-yaml/#buildconfig) configuration to activate the "**Test Summary Hyperlink**" feature. This configuration allows you to define specific parameters that govern the behavior of the hyperlinks based on the test summary status. diff --git a/static/docs/hyperexecute-how-smart-caching-boosts-tests-speed.md b/static/docs/hyperexecute-how-smart-caching-boosts-tests-speed.md deleted file mode 100644 index d7d829b6f..000000000 --- a/static/docs/hyperexecute-how-smart-caching-boosts-tests-speed.md +++ /dev/null @@ -1,44 +0,0 @@ -# How Smart Caching Boosts Your Tests Execution Speed - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Smart Caching significantly reduces test execution times by optimizing resource utilization and minimizing redundant processes. This document provides a comprehensive technical overview of Smart Caching, detailing its architecture, functionalities, and benefits for software development teams. - -## Architecture - -Smart Caching operates on a distributed caching architecture, leveraging a network of caching nodes across TestMu AI's global infrastructure. This ensures low latency and high availability for test execution, regardless of user location. - -The core functionalities of Smart Caching include: - -### Dependency Doping - -Caches frequently used libraries, frameworks, and test assets at caching nodes, eliminating the need for repeated downloads and reducing test execution times. - -### Environment Enhancers - -Pre-configures and stores ready-to-use testing environments based on test configurations, eliminating time-consuming environment setup for each test case. - -## Functionalities -Key functionalities offered by Smart Caching: - -- **Global Caching Network:** Leverage a geographically distributed network of caching nodes for optimal performance and low latency. - -- **Intelligent Caching Strategy:** Smart algorithms identify and cache the most frequently used dependencies and environments for maximum efficiency. - -- **Seamless Scalability:** The caching network automatically scales to accommodate growing test suites and concurrent test runs. - -## Benefits - -Implementing Smart Caching offers numerous benefits: - -- **50%-80% Faster Test Execution:** Dramatically reduce test execution times, freeing up valuable developer resources. - -- **Improved Resource Utilization:** Minimize resource consumption, leading to cost savings and infrastructure optimization. - -- **Enhanced Developer Productivity:** Developers spend less time waiting for tests, focusing more on coding, innovation, and problem-solving. - -- **Increased Scalability:** Handle large and complex test suites with ease due to the inherent scalability of the caching network. - -## Additional Resources - -- [Deliver Faster: How Smart Caching Turbocharges Your Testing Workflow](https://www.testmuai.com/blog/how-smart-caching-turbocharges-your-testing-workflow/) diff --git a/static/docs/hyperexecute-how-to-configure-os-and-browser.md b/static/docs/hyperexecute-how-to-configure-os-and-browser.md deleted file mode 100644 index 7cd04f97b..000000000 --- a/static/docs/hyperexecute-how-to-configure-os-and-browser.md +++ /dev/null @@ -1,50 +0,0 @@ -# How to configure Operating System and Browser in your Test Suite - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This documentation will guide you through configuring the Operating System and Browser for your test suite on HyperExecute. We will cover the step-by-step method to implement the configuration for various test frameworks supported by HyperExecute. Each section provides practical examples to help you integrate HyperExecute with your existing automation suite. - -For demonstration purposes, we use the TestNG sample repository. However, you can apply the same concepts to your preferred framework. You can access the sample repositories for your required frameworks: - -## Step 1: Update TestMu AI Capabilities in your test suite -Configuring the capabilities is the first step to define the environment for your tests. These capabilities allow you to specify parameters such as the Operating System, Browser, and Browser Version on which your tests will run. TestMu AI capabilities serve as a bridge between your test framework and the HyperExecute cloud infrastructure. - -### Key Attributes to Configure -- **`platform` :** Specifies the operating system (e.g., Windows, macOS, or Linux). -- **`browserName` :** Indicates the browser to be used (e.g., Chrome, Firefox, or Edge). -- **`version` :** Defines the specific version or sets it as latest to always use the most recent version. - -Below is a Java example using the TestNG framework: -```java title="Test.java" -DesiredCapabilities caps = new DesiredCapabilities(); -caps.setCapability("platform", "Windows"); -caps.setCapability("browserName", "Chrome"); -caps.setCapability("version", "latest"); //latest, dev, beta, latest-1, latest-2, ... -``` - -The above configuration ensures that your tests run on Windows OS using the latest version of Google Chrome. - -## Step 2: Configure your HyperExecute YAML file -The HyperExecute YAML configuration file is the blueprint for running your tests in the HyperExecute cloud environment. It defines the testing environment, execution strategy, and resource allocation. Properly configuring this file ensures that your tests are executed efficiently across the desired platforms and browsers. - - [AutoSplit mode](/support/docs/deep-dive-into-hyperexecute-yaml/#autosplit) automatically splits your scenarios among the concurrent number of tasks and executes them parallelly. This ensures optimal utilization of resources and significantly reduces the overall execution time of your test suite. - - In this method, you simply set the [`runson`](/support/docs/deep-dive-into-hyperexecute-yaml/#runson) flag to specify the Operating System for running your tests. - -```yaml title="hyperexecute_autosplit.yaml" -runson: linux #linux, win, win11, mac, mac13 -``` - > 📕 Learn more about [AutoSplit Test Method](/support/docs/hyperexecute-auto-split-strategy/) - - [Matrix mode](/support/docs/deep-dive-into-hyperexecute-yaml/#matrix) allows you to run the same set of tests across multiple combinations of environments, configurations, or parameters. This is particularly useful for ensuring that your software works correctly under different conditions, such as various operating systems, browser versions, or dependency files. - - - **`runson: ${matrix.os}` :** Dynamically selects the operating system from the matrix.os list (Windows, macOS, Linux) for each test job. - - **`os` :** Lists the operating systems for testing: `win`, `win11`, `mac`, `mac13`, and `linux`. - -```yaml title="hyperexecute_matrix.yaml" -runson: ${matrix.os} - -matrix: -os: ["win", "mac", "linux"] #linux, win, win11, mac, mac13 -``` - > 📕 Learn more about [Matrix Test Method](/support/docs/hyperexecute-matrix-multiplexing-strategy/) diff --git a/static/docs/hyperexecute-how-to-configure-sourcePayload.md b/static/docs/hyperexecute-how-to-configure-sourcePayload.md deleted file mode 100644 index f77d83999..000000000 --- a/static/docs/hyperexecute-how-to-configure-sourcePayload.md +++ /dev/null @@ -1,142 +0,0 @@ -# How to configure sourcePayload - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -When you run your tests on HyperExecute, the test scripts are zipped, encrypted and uploaded to our secure servers temporarily for execution through the [HyperExecute CLI binary](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/). However, to make this process seamlessly fit in your pipeline you can use the `sourcePayload` Yaml parameter. - -Once you set `sourcePayload`, your test scripts are directly sourced from your Git provider with the help of secure access tokens and only your HyperExecute YAML file is encrypted and sent through the HyperExecute CLI. Learn more detailed documentation below. - -## Setting up - -You can start by configuring the `sourcePayload` key in the [HyperExecute YAML](/support/docs/deep-dive-into-hyperexecute-yaml/) file. The format is explained below. - -```bash -sourcePayload: -platform: git -link: https://-------- -ref: -accessToken: -commit: -``` - -1. **platform**: The platform on which your repository exists. It should be set to `git` if your repository is deployed on GitHub or GitLab. - -2. **link**: The link to your repository that contains the code to be tested. - -3. **ref**: The name of the branch that you want to test. - -4. **accessToken**: Your personal access token (PAT), generated on your Git platform. If you want to know how to generate a PAT for GitHub, go through [this](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens) section. - -> **Note**: You don’t need to use a Personal Access Token if your repository is public. - -5. **commit**: If you want to run a specific commit file other than the **HEAD** commit, then you need to define here the specific commit ID. - -> As a best practice, you should not use your PAT directly in the YAML file. Instead of that, you can make use of our [vault](/support/docs/hyperexecute-how-to-save-and-manage-secrets/) feature and add the PAT to your secrets. To use the vault feature in the HyperExecute YAML file, you can refer the example given below. - -```yaml title="hyperexecute.yaml" -sourcePayload: -platform: git -link: https://-------- -ref: -accessToken: ${{ .secrets. }} -commit: -``` -> You can assign a name to your PAT via the HyperExecute vault feature and replace it in place of the `GIT_PAT` tag in the example given above. - -### Sample YAML file - -A sample HyperExecute YAML file is added below for your reference. - -```yaml title="hyperexecute.yaml" ---- -version: "0.1" -globalTimeout: 90 -testSuiteTimeout: 90 -testSuiteStep: 90 -runson: win -autosplit: true -retryOnFailure: false -testType: "playwright" -maxRetries: 1 -concurrency: 1 -pre: -- npm install -- npx playwright install -cacheKey: '{{ checksum "package-lock.json" }}' -cacheDirectories: -- node_modules -testDiscovery: -type: raw -mode: remote -command: grep -lr 'describe' ltblogsearch.test.js -testRunnerCommand: npm test -- $test -sourcePayload: -platform: git -link: -ref: -accessToken: -``` - -### Generating a Personal Access Token - -Generate Personal Access Token on GitHub - You can generate a **PAT** on **GitHub** by following the steps below. - -1. Log in to your [GitHub](https://github.com/ "https://github.com/") account. You will be redirected to your homepage on GitHub. - -2. Click on your profile on the top right-hand side corner of your page to access your **Settings**. - -3. Go to the bottom of the **Settings** page and click on **Developer settings**. - -4. Click on **Tokens (classic)** to generate your own Personal Access Token. - -5. Click on **Generate new token** and select the **classic** or **fine-grained** version as per your choice. - -6. Proceed to configuring the PAT and click on **Generate token** to create your Personal Access Token. - - > **Note**: While configuring your PAT, ensure that the access is given for all keys under `repo`. Moreover, you will also have to give all the accesses for `read` keys. - -Generate Personal Access Token on GitLab - -1. Navigate to [GitLab Settings](https://gitlab.com/-/profile/personal_access_tokens). -2. Enter **Token name** and add **Expiration date**. -3. **Select the scopes** for the token. Add `read_repository` access to the scope. -4. Click on **Create Personal Access Token**. -5. **Copy and Save** your token securely. - -Generate Project Access Token on GitLab - -1. Navigate to the project for which you want to create a token. -2. Select **Settings** from the top menu. Choose **Access Tokens** from the left sidebar. -3. Click on **Add new token**. Enter a name for the token. -4. Select the scopes for the token. -5. Select the role for the token, which determines the level of access it has. -6. Click on **Create project access token**. - -Generate Repository Access Token on Bitbucket - -We need to follow the 2 steps mentioned below: -1. Need to create the access token for bitbucket repo -2. Need to use bitbucket URL and access token to run the job on HyperExecute - -- Create access token for the repository - -Please refer to the following link for generating access tokens if your/your client’s repository is private. This access token will allow HyperExecute to directly fetch the test scripts/Provar tests from bitbucket whenever a Job is triggered. -https://support.atlassian.com/bitbucket-cloud/docs/create-a-repository-access-token/ - -- Add the token in Job trigger API payload - -In order for HyperExecute to be able to fetch tests from **bitbucket**, **bitbucket-server**, or **github**, the token has to be added in the Job Trigger API. Example given below: - -```yaml ---- -sourcePayload: -platform: git -link: https://bitbucket.org//junit-selenium-hyperexecute-sample.git, -ref: main -accessToken: ${{ .secrets.BITBUCKET_SERVER_REPO_CLONE_SECRET_KEY }} -verifySSL: false -gitProvider: bitbucket -``` - -> Note: Please make sure yo enter your username while adding the repository URL. diff --git a/static/docs/hyperexecute-how-to-configure-tunnel.md b/static/docs/hyperexecute-how-to-configure-tunnel.md deleted file mode 100644 index 4a2fc3bd1..000000000 --- a/static/docs/hyperexecute-how-to-configure-tunnel.md +++ /dev/null @@ -1,98 +0,0 @@ -# Tunnel Configuration in HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -You can configure a secure tunnel in HyperExecute to test applications that are: - -- Hosted locally on your system, or -- Behind a private network/firewall. - -## Prerequisites -- HyperExecute YAML file -> HyperExecute supports multiple ways of setting up tunnels directly from your YAML configuration. You can use any version of [YAML](/support/docs/hyperexecute-yaml-parameters/) to pass the Tunnel parameters. -- [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) to trigger your tests. - -## Ways to Use Tunnel in HyperExecute -There are 3 main ways to use a tunnel in HyperExecute: - -### 1. Simple `tunnel` Connection -Enable tunnel directly by setting the `tunnel` flag to `true` in your HyperExecute YAML file. The HyperExecute CLI will launch a tunnel as sub process for accessing your applications which are locally hosted or behind a firewall. - -```yaml title="hyperexecute.yaml" -tunnel: true -``` -> Recommended if you want to quickly enable tunnel access for locally hosted apps. - -### 2. Using Existing `tunnel` -If you already have a running tunnel, you can reuse it by specifying its name to access your applications which are locally hosted or behind a firewall. - -```yaml title="hyperexecute.yaml" -tunnelNames: ["lambdatest_tunnel"] -``` - -> Useful when running multiple jobs and you don’t want to spawn new tunnels each time. - -### 3. Advanced `tunnel` Configurations -These are the parameters you can use to configure your [tunnel](/support/docs/deep-dive-into-hyperexecute-yaml/#tunnel) settings when running it. - -| Options | Type | Description | -|---------|------|-------------| -|`args`| List of strings | The arguments to pass to your tunnel. For a list of arguments checkout our [tunnel flags](/support/docs/lambda-tunnel-modifiers/) | -|`preOnly`| Boolean | Should the tunnel be enabled for only the pre step. Default false. | -|`postOnly`| Boolean | Should the tunnel be enabled for only the post step. Default false. | -|`global`| Boolean | Should the tunnel be enabled for all the steps? Default true. | -|`systemProxy`| Boolean | Should the tunnel be OS system wide? Default false. | -|`checkTunnelOnFailure`| Boolean | Check tunnel on failure adds a check on our system to check the tunnel connection if a test fails and the tunnel is set to true. This option will retry the test 2 times if tunnel connection is flaky. | - -> HyperExecute now supports tunnel capabilities for [Maestro](/support/docs/hyperexecute-maestro-testing/) tests running on both virtual devices and real devices using the Raw Framework configuration. - -```yaml title="hyperexecute.yaml" -tunnel: true - -tunnelOpts: -global: true -``` - -## Clone a Repository via Tunnel - -Cloning a repository using a tunnel can help secure the transfer of data between the repository and your local machine. - -Private repositories are hosted on a private network or behind a firewall, and you need to access them from outside the network. By using a tunnel, you can create a secure connection to the private network and clone the repository as if you were within the network. - -You can now clone all your **public** and **private** repositories from your Git provider via **tunnel**. - -After specifying the details of your repository, you need to set the `useInCodeSetUp` flag to `true` as shown below. - -```bash -sourcePayload: -platform: git -link: https://-------- -ref: -accessToken: -commit: - -# highlight-next-line -tunnelOpts: -# highlight-next-line -useInCodeSetUp: true -## this flag is mandatory to pass to enable cloning via tunnel -``` - -## Troubleshooting - -If you are unable to access a particular website via the Actions environment even after setting the `tunnel: true` in the YAML file then please execute the following curl command: - -```bash -response=$(curl -o /dev/null -s -w "%{http_code}" https://www.testmuai.com) - -## You need to enter your website URL in place of LambdaTest URL -``` - -Now you need to check the response of the website by passing the command: - -```bash -echo $response -``` - -The ideal expected response in **200**. - ->If you have more questions, feel free to Chat with us. diff --git a/static/docs/hyperexecute-how-to-debug-job.md b/static/docs/hyperexecute-how-to-debug-job.md deleted file mode 100644 index 29c5ef65a..000000000 --- a/static/docs/hyperexecute-how-to-debug-job.md +++ /dev/null @@ -1,57 +0,0 @@ -# How to Debug a Failed Job - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -As a developer or tester, you know that running tests is an essential part of the software development process. However, there are times when your tests may fail, and you may not be sure how to debug the issue, and you find yourself scratching your head, wondering what went wrong. - -The HyperExecute Dashboard provides a central view of all your test runs, including the test results, error logs, and performance metrics. By understanding how to find and interpret error logs, you can quickly identify the root cause of a failed test and take steps to fix the issue. This document will help you debug failed tests on HyperExecute by finding error logs in different stages using the HyperExecute Dashboard. - -## Prerequisites - -- HyperExecute Dashboard Access. -- A Job is triggered on HyperExecute, and it has a `Failed` status. -- Identify which Task has failed. Click on that task and check which of the Stage categories is throwing an error. - -## Pre Steps -You can see in the screenshot mentioned below that the Pre Step of the Task has failed. To understand the issue causing the step to fail, scroll down, go through the logs once, and you can find the **type of error** and its **location** in the code as well. - -> **NOTE :** During the execution of pre-commands, you might come across different issues. To troubleshoot and resolve these issues, please refer to the relevant solutions provided on [this](/support/docs/hyperexecute-pre-steps-error/) page. - -## Scenarios -To check the issues caused in the Scenario Step, there are multiple ways to find the issue. Let's explore each way: - -### Checking the Logs -To check the logs, you need to expand the test case that has failed in the scenario. Go through the logs once until the end, and you will encounter the reason for the test failure. - -In the given screenshot, we are encountering an **Exception Error** that **No Such Element Found** for the Test Case @BingSearch and it can be due to the wrong path entered in the code file or that particular Feature Case is not present in the code directory. - -### Video Recordings and Command Logs - -If you were unable to find the error and want a deep analysis of it, then you can click on the **Test Associated** button or the **View Test** button. - -#### Video Recording - -Here you can check which part of your test case is causing an error in the video format, along with the required logs to check at which timestamp you are encountering this error and what the reason behind it is. - -#### Command Logs - -- You can view all kinds of logs as well like [**Network Logs**](/support/docs/debugging-options/#network-logs). You can toggle between the [**Console Logs**](/support/docs/debugging-options/#console-logs), [**Terminal Logs**](/support/docs/debugging-options/#terminal-logs) and the **Test Framework Logs**. - -> To enable these logs, you need to explicitly mention the required flag in your [Test Capability Configuration](/support/docs/selenium-automation-capabilities/). - -> **NOTE :** During the execution of scenario, you might come across different issues. To troubleshoot and resolve these issues, please refer to the relevant solutions provided on [this](/support/docs/hyperexecute-scenario-steps-error/) page. - -## Root Cause Analysis Report -You can also check out our AI Native [RCA Report](/support/docs/ai-powered-test-failure-analysis/), which will help you to quickly identify various error categories and also offer a comprehensive overview of the underlying causes. - -To generate the RCA, simply click on the **Errors** icon, where you can also see the **type of error**, and then click on the **See Details** button to go through the detailed logs. - -Here you can check the detailed log for the cause of failure as well as the **Line number** for the particular **Code Snippet** along with its **Stack Trace**. - -If you are still facing an issue, then scroll down and click on the [**Generate RCA**](/support/docs/ai-powered-test-failure-analysis/#generate-the-rca-of-your-failed-jobs) button, which will offer a structured approach to addressing each failure type. You can efficiently get to the precise corrective measures or Remedies recommended, along with Additional suggestions for the particular error. - -## Post Steps - -In the below screenshot, we can clearly see that there is some issue in fetching the path directory of the artifacts file and folder. And for the analysis of it, the path is also given in the logs so you can easily debug this. - -> You can also check the complete [Detailed Job Report](/support/docs/hyperexecute-reports/) which can help you to analyze the outcome of your tests based on your framework. diff --git a/static/docs/hyperexecute-how-to-find-correct-concurrency.md b/static/docs/hyperexecute-how-to-find-correct-concurrency.md deleted file mode 100644 index 64e16835d..000000000 --- a/static/docs/hyperexecute-how-to-find-correct-concurrency.md +++ /dev/null @@ -1,140 +0,0 @@ -# How to Find the Correct Concurrency for my Job - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This page explains how you can find what concurrency would be best suited for your test execution job. - -The `concurrency` is one of the parameters in the HyperExecute Yaml. It indicates the number of concurrent tasks to run for processing all your test scenarios/test-cases. A HyperExecute job, thus triggered, creates as many threads(also known as tasks) as per the value provided in the `concurrency` key. - -```bash -concurrency: 10 -``` - -This is a **required** parameter in case you are using [autosplit](/support/docs/hyperexecute-auto-split-strategy/). However if you are using [matrix](/support/docs/hyperexecute-matrix-multiplexing-strategy/) the number of concurrent threads is automatically defined by the possible permutations and combinations of parameters give. - -The platform automatically analyzes your usage and test cases and will guide you to increase the concurrency to obtain substantial time savings. You can find this information on the left side banner. - -You can also see the overall concurrency trends using our analytics widgets. [Learn more](/support/docs/analytics-modules-resource-utilization/#concurrency-trends) - -## Shared Cloud Daily Usage Limit - -To ensure fair usage, maintain system stability, and promote optimal test practices across all our users, TestMu AI enforces a daily usage limit on test execution time at the organization level. Each organization is allocated a maximum allowable test duration per day, calculated based on the number of concurrent (parallel) sessions provisioned to the account. This safeguard helps prevent overutilization and ensures equitable access to resources across the platform. - -## Why This Matters -Running long-duration test sessions, especially those exceeding 120 minutes, can cause memory and CPU spikes that impact system performance and reliability. To prevent such disruptions and improve the efficiency of test execution, we encourage users to: - -- Split long-running tests into smaller, modular suites. -- Implement test retries, timeouts, and teardown routines to handle edge cases and resource leaks. -- Use parallelization effectively to optimize test execution time without extending the runtime of individual sessions. - -By adhering to these guidelines, your team not only aligns with industry best practices but also ensures smoother and more reliable test runs on TestMu AI infrastructure. - -## Daily Test Time Limits by Parallel Sessions -The daily usage cap scales linearly based on the number of parallel sessions available to your organization. Below is a reference table: - -| PARALLEL SESSIONS | COMPLETED RUN TIME (HRS) | -| ---------- | ----------------- | -| 1 | 6 | -| 2 | 12 | -| 3 | 18 | -| 4 | 24 | -| 5 | 30 | -| 6 | 36 | -| 7 | 42 | -| 8 | 48 | -| 9 | 54 | -| 10 | 60 | -| 11 | 66 | -| 12 | 72 | -| 13 | 78 | -| 14 | 84 | -| 15 | 90 | -| 16 | 96 | -| 17 | 102 | -| 18 | 108 | -| 19 | 114 | -| 20 | 120 | -| 21 | 126 | -| 22 | 132 | -| 23 | 138 | -| 24 | 144 | -| 25 | 150 | -| 26 | 156 | -| 27 | 162 | -| 28 | 168 | -| 29 | 174 | -| 30 | 180 | -| 31 | 186 | -| 32 | 192 | -| 33 | 198 | -| 34 | 204 | -| 35 | 210 | -| 36 | 216 | -| 37 | 222 | -| 38 | 228 | -| 39 | 234 | -| 40 | 240 | -| 41 | 246 | -| 42 | 252 | -| 43 | 258 | -| 44 | 264 | -| 45 | 270 | -| 46 | 276 | -| 47 | 282 | -| 48 | 288 | -| 49 | 294 | -| 50 | 300 | -| 51 | 306 | -| 52 | 312 | -| 53 | 318 | -| 54 | 324 | -| 55 | 330 | -| 56 | 336 | -| 57 | 342 | -| 58 | 348 | -| 59 | 354 | -| 60 | 360 | -| 61 | 366 | -| 62 | 372 | -| 63 | 378 | -| 64 | 384 | -| 65 | 390 | -| 66 | 396 | -| 67 | 402 | -| 68 | 408 | -| 69 | 414 | -| 70 | 420 | -| 71 | 426 | -| 72 | 432 | -| 73 | 438 | -| 74 | 444 | -| 75 | 450 | -| 76 | 456 | -| 77 | 462 | -| 78 | 468 | -| 79 | 474 | -| 80 | 480 | -| 81 | 486 | -| 82 | 492 | -| 83 | 498 | -| 84 | 504 | -| 85 | 510 | -| 86 | 516 | -| 87 | 522 | -| 88 | 528 | -| 89 | 534 | -| 90 | 540 | -| 91 | 546 | -| 92 | 552 | -| 93 | 558 | -| 94 | 564 | -| 95 | 570 | -| 96 | 576 | -| 97 | 582 | -| 98 | 588 | -| 99 | 594 | -| 100 | 600 | - -> **Note :** If your test executions consistently exceed the recommended runtime or you face test instability, please consider reviewing your test strategy or reaching out to our support team for optimization guidance. - ->If you have more questions, feel free to Chat with us. diff --git a/static/docs/hyperexecute-how-to-get-my-username-and-access-key.md b/static/docs/hyperexecute-how-to-get-my-username-and-access-key.md deleted file mode 100644 index 87d479360..000000000 --- a/static/docs/hyperexecute-how-to-get-my-username-and-access-key.md +++ /dev/null @@ -1,15 +0,0 @@ -# How to Get my Username and Access key - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This page explains where you can find your LT Username and Access Key. - -**Step 1:** Go to the right side Profile dropdown and select **Account Settings**. - -**Step 2:** Under Account Settings, go to **Password & Security** - -**Step 3:** You will find your LT Username and Access Key here, this would be required in order to trigger a Job using the [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) - -You can directly copy your Username and Access Key using shortcut key button. - ->If you have more questions, then just give us a shout and we will answer them for you. diff --git a/static/docs/hyperexecute-how-to-guide.md b/static/docs/hyperexecute-how-to-guide.md deleted file mode 100644 index 1f1fdb262..000000000 --- a/static/docs/hyperexecute-how-to-guide.md +++ /dev/null @@ -1,27 +0,0 @@ -# HyperExecute how to guide - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Username and Access Key - Understand the step by step process of how to get your username and access key. - - Environment Variables - Learn how to setup HyperExecute environment variables. - - Manage Secrets - Learn how to save and manage your secrets in HyperExecute Vault. - - Configure Tunnel - Learn how to configure tunnel for testing locally hosted websites using HyperExecute. - - Job Concurrency - Learn how to find correct concurrency for my Job - - Configure your Source Payload - Learn about how you can configure your sourcePayload. - - Supported Browsers - How to get the list of all the supported browsers in HyperExecute via API call. - - Local Selenium Webdriver - Learn how to run your selenium tests via local driver without setting remote driver in HyperExecute. diff --git a/static/docs/hyperexecute-how-to-manage-project-level-secrets.md b/static/docs/hyperexecute-how-to-manage-project-level-secrets.md deleted file mode 100644 index 38d192330..000000000 --- a/static/docs/hyperexecute-how-to-manage-project-level-secrets.md +++ /dev/null @@ -1,68 +0,0 @@ -# Manage Project Level Secrets in HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Secrets in TestMu AI HyperExecute are encrypted environment variables used to store sensitive data securely like **Access tokens**, **API Keys**, **Passwords**. Secrets are injected into your test environment at runtime, so you never expose them directly in your codebase or logs. - -Managing sensitive information like API tokens, credentials, and access keys is crucial when running tests in TestMu AI HyperExecute. To address this, HyperExecute offers a Secrets Management system, and one of its powerful features is Project Level Secrets. This feature lets you define secrets scoped to a specific project, making secret handling more secure, easier, and collaborative for your teams. - -## What Are Project Level Secrets? -Project-level secrets are bound to a specific HyperExecute project instead of a user or account. When a job runs using that project ( referenced by id or both id and name in the YAML configuration, which are sub-parameters of project parameter), the project-level secrets are automatically available to the test environment. - -### Key Points -- Define secrets once per project. -- Use them across all jobs that reference the project. -- Simplifies managing secrets when multiple users or CI/CD pipelines access the same project. - -## How to Add Project Level Secrets? -### Step 1: Create a Project - -To begin, create a new project on the TestMu AI platform. Follow the instructions to [Create a Project](/support/docs/hyperexecute-projects/#setup-your-project) - -### Step 2: Add Secrets to the Project -Once your project is created: -- Navigate to the Secrets tab within the project. -- Click Add Secret. -- Enter a Key (e.g., LT_SECRET_KEY) and its corresponding Value (e.g., secureP@ss123). -- Click Add Secret button. - -Your secret will be encrypted and securely stored. - -### Step 3: Use Secrets in Your HyperExecute YAML -To reference the secrets in your HyperExecute configuration file (`hyperexecute.yaml`): - -```yaml title="hyperexecute.yaml" -version: 0.1 -runson: linux - -autosplit: true -concurrency: 2 - -project: -name: 'demo-project' -id: '1abc2345-56de-7890-f1cg-h2ij34k56789' - -# highlight-start -env: -LT_SECRET_KEY: ${{.secrets.token}} -# highlight-end -``` - -## User-Level Secrets vs. Project-Level Secrets - -| Features | User Level Secrets | Project Level Secrets | -|----------|--------------------|-----------------------| -| **Scope** | Linked to individual user account | Scoped to a specific project | -| **Reusability** | Not reusable across users | Reusable by all users on project | -| **Team Collaboration** | Limited (not ideal for shared projects) | Designed for team collaboration | -| **Management Location** | Managed in user’s Secret Manager | Managed inside the project UI | -| **Access Control** | Controlled by user account permissions | Controlled by project permissions | - -## Why Use Project-Level Secrets? -- **Centralized Management :** Keep all secrets related to a test framework or application in one place: the project. -- **Team Collaboration :** Multiple team members can access the same secrets via shared project access. -- **Simplified CI/CD Setup :** Reference your project in the YAML, and secrets are injected automatically without extra configuration. -- **Security :** Secrets remain encrypted and are never logged or exposed in your code. -- **Easy Maintenance :** Update secrets in one place when values change, and all users/jobs referencing the project get the updated secrets immediately. - -By defining and managing secrets at the project level in TestMu AI HyperExecute, teams can ensure safer handling of sensitive data, reduce duplication efforts, and make test runs more consistent and secure. diff --git a/static/docs/hyperexecute-how-to-perform-dependent-test-based-discovery.md b/static/docs/hyperexecute-how-to-perform-dependent-test-based-discovery.md deleted file mode 100644 index 2847c270b..000000000 --- a/static/docs/hyperexecute-how-to-perform-dependent-test-based-discovery.md +++ /dev/null @@ -1,116 +0,0 @@ -# How to Perform Dependent Test Case Discovery - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Dependent tests signify that one test relies on the outcome of another. To achieve this, TestNG offers the '**dependsOnMethods**' attribute within @Test annotations. - -For instance, consider the code snippet in which '**SignIn()**' depends on '**OpenBrowser()**,' and '**LogOut()**' depends on 'SignIn().' - -```java -import org.testng.annotations.Test; -public class DependsOnTest { -@Test -public void OpenBrowser() { -System.out.println("The browser is opened"); -} - -@Test (dependsOnMethods = { "OpenBrowser" }) -public void SignIn() { -System.out.println("User has signed in successfully"); -} - -@Test (dependsOnMethods = { "SignIn" }) -public void LogOut() { -System.out.println("The user logged out successfully"); -} -} -``` - -To discover and manage dependent tests using the Test Discovery command, you can use the following syntax: - -```yaml -mvn test -Dmode=discover -Dplatname=win -Dframework=testng -Ddiscovery=dependent -``` - -This command will provide a Test Discovery Result that lists the tests and their dependencies, ensuring that dependent tests are executed in the correct order, such as ["Test1#SignIn,Test1#LogOut,Test1#OpenBrowser"]. - -## Group-Based Test Discovery in TestNG - -TestNG provides group-based test discovery functionality. You can specify groups for your tests and execute or exclude specific groups during test runs. - -To perform group-based test discovery, you can use the ***@BeforeGroups*** and ***@AfterGroups*** annotations in your TestNG tests. These annotations allow you to specify setup and cleanup methods that run before and after specific groups of tests. - -For example: - -```java -public class GroupIntegrationTest { - -@BeforeGroups("database") -public void setupDB() { -System.out.println("setupDB()"); -} - -@AfterGroups("database") -public void cleanDB() { -System.out.println("cleanDB()"); -} - -@Test(groups = "selenium-test") -public void runSelenium() { -System.out.println("runSelenium()"); -} - -@Test(groups = "selenium-test") -public void runSelenium1() { -System.out.println("runSelenium()1"); -} - -@Test(groups = "database") -public void testConnectOracle() { -System.out.println("testConnectOracle()"); -} - -@Test(groups = "database") -public void testConnectMsSQL() { -System.out.println("testConnectMsSQL"); -} - -} -``` - -In the Hyperexecute YAML Version 0.2 configuration, you can use the ```discoveryFlags``` parameter to specify the groups to discover during test discovery. - -```yaml -framework: -name: "maven/testng" -discoveryFlags: ["-Dgroups=database"] -``` - -Here it will only discover tests belonging to the group database. Use comma-separated values if you want to specify multiple groups. - -Similarly, you can use the ```excludedGroups``` parameter that can be used to run all test groups except for the defined set of groups. - -```yaml -framework: -name: "maven/testng" -discoveryFlags: ["-DexcludedGroups=database"] -``` - -This discovers all test of groups except database. - -Alternatively, you can also specify the groups or excluded groups directly in the pom.xml file using the Maven Surefire Plugin - -```yaml - -[...] - -org.apache.maven.plugins -maven-surefire-plugin -2.22.1 - -database,selenium-test - - -[...] - -``` diff --git a/static/docs/hyperexecute-how-to-perform-group-based-test-discovery-in-testng.md b/static/docs/hyperexecute-how-to-perform-group-based-test-discovery-in-testng.md deleted file mode 100644 index 8ed8f1c59..000000000 --- a/static/docs/hyperexecute-how-to-perform-group-based-test-discovery-in-testng.md +++ /dev/null @@ -1,82 +0,0 @@ -# How to Perform Group-Based Test Discovery in TestNG - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestNG provides group-based test discovery functionality. You can specify groups for your tests and execute or exclude specific groups during test runs. - -To perform group-based test discovery, you can use the ***@BeforeGroups*** and ***@AfterGroups*** annotations in your TestNG tests. These annotations allow you to specify setup and cleanup methods that run before and after specific groups of tests. - -For example: - -```java -public class GroupIntegrationTest { - -@BeforeGroups("database") -public void setupDB() { -System.out.println("setupDB()"); -} - -@AfterGroups("database") -public void cleanDB() { -System.out.println("cleanDB()"); -} - -@Test(groups = "selenium-test") -public void runSelenium() { -System.out.println("runSelenium()"); -} - -@Test(groups = "selenium-test") -public void runSelenium1() { -System.out.println("runSelenium()1"); -} - -@Test(groups = "database") -public void testConnectOracle() { -System.out.println("testConnectOracle()"); -} - -@Test(groups = "database") -public void testConnectMsSQL() { -System.out.println("testConnectMsSQL"); -} - -} -``` - -In the Hyperexecute YAML Version 0.2 configuration, you can use the ```discoveryFlags``` parameter to specify the groups to discover during test discovery. - -```yaml -framework: -name: "maven/testng" -discoveryFlags: ["-Dgroups=database"] -``` - -Here it will only discover tests belonging to the group database. Use comma-separated values if you want to specify multiple groups. - -Similarly, you can use the ```excludedGroups``` parameter that can be used to run all test groups except for the defined set of groups. - -```yaml -framework: -name: "maven/testng" -discoveryFlags: ["-DexcludedGroups=database"] -``` - -This discovers all test of groups except database. - -Alternatively, you can also specify the groups or excluded groups directly in the pom.xml file using the Maven Surefire Plugin - -```yaml - -[...] - -org.apache.maven.plugins -maven-surefire-plugin -2.22.1 - -database,selenium-test - - -[...] - -``` diff --git a/static/docs/hyperexecute-how-to-resolve-version-conflict.md b/static/docs/hyperexecute-how-to-resolve-version-conflict.md deleted file mode 100644 index c77f3bc80..000000000 --- a/static/docs/hyperexecute-how-to-resolve-version-conflict.md +++ /dev/null @@ -1,56 +0,0 @@ -# How to Resolve Version Conflict Issues - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Maintaining consistent versions across testing frameworks in your project setup is crucial to avoid runtime errors, compatibility issues, and unexpected behaviors. This guide focuses on specifying framework versions in YAML configuration files and aligning them with the version details in project-specific files like `package.json`, `pom.xml`, `.csproj`, and system environment variables for frameworks like Selenium, Playwright, Puppeteer, Cypress, Maestro, and Appium. - -## Why Version Consistency Matters -When versions defined in YAML configuration files do not match those in project files, it can lead to: - -- **Dependency Conflicts :** Incompatibility between framework components, leading to failures. -- **Runtime Errors :** Unexpected crashes or malfunctions during test execution. -- **Inconsistent Test Results :** Version mismatches can produce different outcomes or misbehave in automated test cases. -To prevent these issues, it is recommended to ensure that framework versions match between the YAML pre-steps and project files. - -## Framework-Specific Guidelines -This section provides detailed instructions for version consistency across different frameworks. - -### Selenium - Java -For Java projects using Selenium, TestNG and Cucumber dependencies should match between the YAML pre-steps and the pom.xml file. - -- **TestNG :** Ensure the TestNG version in your `pom.xml` matches the version specified in the YAML configuration under [`pre-steps`](/support/docs/deep-dive-into-hyperexecute-yaml/#pre): -```xml - -org.testng -testng -7.5 -test - -``` - -> Always verify compatibility between Java and TestNG versions as specified in TestNG's documentation to avoid compatibility conflicts. - -### Selenium/Playwright/Puppeteer/Cypress - JavaScript -For JavaScript frameworks like Selenium, Playwright, Puppeteer, and Cypress, the framework versions specified in the YAML configuration should align with `package.json`: - -```json -{ -"dependencies": { -"selenium-webdriver": "4.1.0", -"cypress": "9.5.0", -"playwright": "1.15.0" -} -} -``` - -Consistency ensures dependencies are installed with the expected versions, avoiding issues like dependency mismatch and incompatible libraries. - -### Selenium/Playwright - C# (.NET) -For .NET projects, Selenium and Playwright versions should match between the YAML pre-steps and the `.csproj` file, which specifies NuGet packages: - -```xml - -net6.0 -Library - -``` diff --git a/static/docs/hyperexecute-how-to-run-tests-using-local-selenium-drivers.md b/static/docs/hyperexecute-how-to-run-tests-using-local-selenium-drivers.md deleted file mode 100644 index 247bd797c..000000000 --- a/static/docs/hyperexecute-how-to-run-tests-using-local-selenium-drivers.md +++ /dev/null @@ -1,154 +0,0 @@ -# Trigger Your Job via Local Selenium Web Driver - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -As a tester, whenever you switch from local web driver to remote web driver can be a tedious and time-consuming process. You will have to manually modify your tests which includes adding Desired Capabilities, and adjusting code just to get them running on a cloud testing platform. - -HyperExecute's support for **Local Selenium Driver** allows you to seamlessly run your tests on our platform with very minimal changes. It will help you in - -- **Reduce Onboarding Time :** You can skip the manual code changes and configuration associated with remote web drivers, and accelerate your integration with HyperExecute. -- **Minimal Code Refactoring :** You can now maintain your existing test code, without worrying about the need for significant modifications. -- **Seamless Platform Transitions :** You can run both web and mobile tests (using Selenium and Appium) without platform-specific changes. - -## How to Trigger Your Tests? - -To utilize the native Selenium driver support in HyperExecute, you need to incorporate the following steps into your testing workflow: - -### Step 1: Configure Your Test Suite - -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** -Download or Clone the code sample for the TestNG from the TestMu AI GitHub repository to run the tests on the HyperExecute. - - View on GitHub - -### Step 2: Update YAML Configuration: - -- Add the `platformConfig` flag to your YAML file, specifying the platforms and configurations for your tests. -- Use the `platform` flag to define individual platform configurations (OS, browser/device, capabilities). -- **(Optional)** Utilize the `config` flag to set global capabilities applicable to all platforms. - -```yaml ---- -version: 0.1 -globalTimeout: 150 -testSuiteTimeout: 150 -testSuiteStep: 150 - -runson: win - -autosplit: true - -retryOnFailure: true -maxRetries: 1 - -parallelism: 1 -concurrency: 1 - -env: -CACHE_DIR: m2_cache_dir - -cacheKey: '{{ checksum "pom.xml" }}' -cacheDirectories: -- m2_cache - -pre: -- mvn dependency:resolve - -mergeArtifacts: true -uploadArtefacts: -- name: ExecutionSnapshots -path: -- target/surefire-reports/html/** - -report: true -partialReports: -location: target/surefire-reports/html -type: html -frameworkName: extent - -testDiscovery: -type: raw -mode: remote -command: grep 'test name' xml/testng.xml | awk '{print$2}' | sed 's/name=//g' | sed 's/\x3e//g' - -testRunnerCommand: mvn test dependency:resolve -winTestRunnerCommand: mvn test dependency:resolve -macTestRunnerCommand: mvn test dependency:resolve -linuxTestRunnerCommand: mvn test dependency:resolve - -#highlight-start -platformConfig: -platform: -- os: win10 -browserName: chrome -browserVersion: latest -build: win10 -network: false -- os: win11 -browserName: chrome -goog:chromeOptions: -args: [ "--start-maximized", "--disable-gpu" ] -- os: mac -browserName: firefox -browserVersion: 119 -- os: linux -browserName: firefox -browserVersion: 115 -- os: android -deviceName: .* -browserName: chrome -- os: ios -deviceName: .* - -config: -build: latest -network: true -console: true -#highlight-end -``` - -- For operating system **win, mac,** and **linux**, only the **os** flag is mandatory, rest are optional. -- For operating system **android** and **ios**, the **os** and **deviceName** flags are mandatory, rest are optional. - -In the above sample YAML file, we have passed **parallelism = 1** and **total number of platforms = 6** - -The number of tasks created = **(total number of platforms) * (parallelism)** = 6 tasks - -- 1 windows 10 with Chrome Browser -- 1 windows 11 with Chrome browser -- 1 mac with Firefox browser -- 1 android with any device (we have not specified device name) -- 1 ios with any device (we have not specified device name) - -> You can also use **dev** and **beta** browser versions. To generate capabilities for your test requirements, you can use our inbuilt 🔗 Capabilities Generator. - -### Step 3: Run Your Tests: - -- Execute your tests as usual through your test framework (e.g., JUnit, TestNG). -- HyperExecute automatically intercepts Selenium commands and routes them to its platform. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE `} - -## Priority Mapping of Capability Selection - -HyperExecute follows a specific priority order when selecting capabilities. The capabilities added in the `config` flag are **global** and applicable to all the platforms. - -So in the above example, `network:true` capability is applied to all the platform configurations but in the case of **platform win10-chrome** network capabilities will be set as `false`. - -| Script Config | YAML `config` | YAML `platform` | Final Status | -|---------------|-------------|-------------|--------------| -|`"console": info` | `"console": debug` | `"console": warning` | `"console": warning` | -|`"console": info` | `"console": debug` | NA | `"console": debug` | -|`"console": debug` | NA | `"console": debug` | `"console": debug` | -| NA | `"console": debug` | `"console": warning` | `"console": warning` | - -## Considerations and Constraints - -While HyperExecute's native Selenium driver support offers significant advantages, it's essential to note the following constraints: - -- If you are using **IEDriver** in your scripts, the tests won’t be run on any platform beside **Windows 10** -- Currently **SafariDriver** is not compatible with the following feature. -- Scripts with hardcoded paths for Selenium drivers may require minor adjustments. -- Support for **AndroidDriver** is not yet available; however, **Chromedriver** can be used for Appium testing. diff --git a/static/docs/hyperexecute-how-to-save-and-manage-secrets.md b/static/docs/hyperexecute-how-to-save-and-manage-secrets.md deleted file mode 100644 index e71c84e9b..000000000 --- a/static/docs/hyperexecute-how-to-save-and-manage-secrets.md +++ /dev/null @@ -1,159 +0,0 @@ -# Manage User-Level Secrets in HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Testing your webapp or applications often involves using sensitive data like passwords, API keys, and other credentials. Storing these secrets directly in your code can be risky and exposing them can lead to potential security breaches. - -HyperExecute provides a secure **Secret Management Vault** to keep your sensitive information safe. This doc covers **user-level secrets** that are tied to your individual account. - -## Secret Scopes - -HyperExecute supports secrets at multiple levels: - -| Scope | Access | Use Case | -|-------|--------|----------| -| **User-Level** (this doc) | Tied to your individual account, available to all your jobs | Personal credentials, user-specific API keys | -| **[Org-Level](/support/docs/hyperexecute-how-to-save-and-manage-secrets/#org-level-secrets)** *(Enterprise only)* | Shared across all users and projects in the organization, managed by admins | Org-wide API keys, shared access tokens | -| **[Project-Level](/support/docs/hyperexecute-how-to-manage-project-level-secrets/)** | Scoped to a specific project, shared across all users on the project | Project-specific credentials, team-shared keys | - -Secrets are stored as `key:value` pairs where **values** are encrypted and not visible to anyone. They can only be [updated](/support/docs/hyperexecute-how-to-save-and-manage-secrets/#update-your-existing-secrets) with a new value. - -## Create a New Secret - -Follow the steps mentioned to seamlessly add new secrets to HyperExecute and ensure their safe storage: - -**Step 1:** Navigate to the **Secrets Section** in the top right corner of the HyperExecute dashboard. - -**Step 2:** Click on the **Add a Secret** button to create a New Secret. - -**Step 3:** Provide a descriptive **Key Name** and then enter your secret **Value**. This could be a password, API key, or any other credential. - -**Step 4:** Click on **Add Secret** button and your secret will be stored safely within HyperExecute's system. - -## Update Your Existing Secrets - -Follow the steps mentioned to update your secrets values to HyperExecute and ensure their safe storage: - -**Step 1:** Navigate to the **Secrets Section** in the top right corner of the HyperExecute dashboard. - -**Step 2:** Click on the **...** and then **Edit** button of the secret you want to update. - -**Step 3:** Enter updated value of the secret. - -**Step 4:** Click on Edit Secret Value to complete the editing. - -## Delete the Secrets - -Follow the steps mentioned to delete your secrets values from the HyperExecute: - -**Step 1:** Navigate to the **Secrets Section** in the top right corner of the HyperExecute dashboard. - -**Step 2:** Click on the **...** and then **Delete** button of the secret you want to delete. - -**Step 3:** Click on **Confirm Deletion** button to delete your secret. - -## Using Secrets in YAML - -During HyperExecute test execution, you might need to access sensitive information like environment variables or access tokens. This section guides you on how to securely integrate these secrets into your YAML file. - -Consider these secret key names: `secretKey1` and `secretKey2`. - -```yaml -version: 0.1 -env: -MyKey1: ${{.secrets.token}} - -runson: win -testDiscovery: -type: raw -mode: remote -args: -featureFilePath: ${Path} - -testRunnerCommand: npm run --username abc --password ${{.secrets.SecretKey2}} -``` - -- The `env` and `vars` sections within your YAML file are used to define variables. -- The syntax `{{.secrets.secretKey1}}` allows you to access the corresponding secret value stored securely in HyperExecute. - -In the above sample YAML file: - -- **MyKey1** in the env section uses `secretKey1`. -- **MyKey2** in the vars section uses `secretKey2`. -- **testRunnerCommand** utilizes `secretKey2` for the password. - -- By using secrets within your YAML file, you avoid embedding sensitive information directly in your code, enhancing security. -- HyperExecute securely retrieves these secrets from its storage i.e. **HashiCorp Vault** at runtime, ensuring they are not exposed during code sharing or storage. - -## Org-Level Secrets - -Org-level secrets are organization-wide secrets that are shared across all projects and users within the organization. They are managed by organization admins and are useful for storing shared credentials like API keys, access tokens, or service account credentials that the entire team needs access to. - -Org-level secrets are an **enterprise-only** feature. Contact your account team or **24×7 chat support** to enable this for your organization. - -## Managing Project-Level Secrets - -Secrets in TestMu AI HyperExecute are encrypted environment variables used to store sensitive data securely like **Access tokens**, **API Keys**, **Passwords**. Secrets are injected into your test environment at runtime, so you never expose them directly in your codebase or logs. - -Managing sensitive information like API tokens, credentials, and access keys is crucial when running tests in TestMu AI HyperExecute. To address this, HyperExecute offers a Secrets Management system, and one of its powerful features is Project Level Secrets. This feature lets you define secrets scoped to a specific project, making secret handling more secure, easier, and collaborative for your teams. - -## What Are Project Level Secrets? -Project-level secrets are bound to a specific HyperExecute project instead of a user or account. When a job runs using that project ( referenced by id or both id and name in the YAML configuration, which are sub-parameters of project parameter), the project-level secrets are automatically available to the test environment. - -### Key Points -- Define secrets once per project. -- Use them across all jobs that reference the project. -- Simplifies managing secrets when multiple users or CI/CD pipelines access the same project. - -## How to Add Project Level Secrets? -### Step 1: Create a Project - -To begin, create a new project on the TestMu AI platform. Follow the instructions to [Create a Project](/support/docs/hyperexecute-projects/#setup-your-project) - -### Step 2: Add Secrets to the Project -Once your project is created: -- Navigate to the Secrets tab within the project. -- Click Add Secret. -- Enter a Key (e.g., LT_SECRET_KEY) and its corresponding Value (e.g., secureP@ss123). -- Click Add Secret button. - -Your secret will be encrypted and securely stored. - -### Step 3: Use Secrets in Your HyperExecute YAML -To reference the secrets in your HyperExecute configuration file (`hyperexecute.yaml`): - -```yaml title="hyperexecute.yaml" -version: 0.1 -runson: linux - -autosplit: true -concurrency: 2 - -project: -name: 'demo-project' -id: '1abc2345-56de-7890-f1cg-h2ij34k56789' - -# highlight-start -env: -LT_SECRET_KEY: ${{.secrets.token}} -# highlight-end -``` - -## User-Level Secrets vs. Project-Level Secrets - -| Features | User Level Secrets | Project Level Secrets | -|----------|--------------------|-----------------------| -| **Scope** | Linked to individual user account | Scoped to a specific project | -| **Reusability** | Not reusable across users | Reusable by all users on project | -| **Team Collaboration** | Limited (not ideal for shared projects) | Designed for team collaboration | -| **Management Location** | Managed in user’s Secret Manager | Managed inside the project UI | -| **Access Control** | Controlled by user account permissions | Controlled by project permissions | - -## Why Use Project-Level Secrets? -- **Centralized Management :** Keep all secrets related to a test framework or application in one place: the project. -- **Team Collaboration :** Multiple team members can access the same secrets via shared project access. -- **Simplified CI/CD Setup :** Reference your project in the YAML, and secrets are injected automatically without extra configuration. -- **Security :** Secrets remain encrypted and are never logged or exposed in your code. -- **Easy Maintenance :** Update secrets in one place when values change, and all users/jobs referencing the project get the updated secrets immediately. - -By defining and managing secrets at the project level in TestMu AI HyperExecute, teams can ensure safer handling of sensitive data, reduce duplication efforts, and make test runs more consistent and secure. diff --git a/static/docs/hyperexecute-hybrid-strategy.md b/static/docs/hyperexecute-hybrid-strategy.md deleted file mode 100644 index a6081f273..000000000 --- a/static/docs/hyperexecute-hybrid-strategy.md +++ /dev/null @@ -1,152 +0,0 @@ -# Hybrid Strategy in HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -HyperExecute’s Hybrid Strategy is a combination of AutoSplit and Matrix Strategy. You can run all of the combinations of [Matrix Strategy](/support/docs/hyperexecute-matrix-multiplexing-strategy/), and use the [AutoSplit Strategy](/support/docs/hyperexecute-auto-split-strategy/) to run them in parallel on our virtual machines. - -## Why do you need the Hybrid Strategy? - -- **Addressing Long-Running Matrix Jobs :** -For Matrix jobs involving numerous combinations, execution time might be longer. Hybrid Strategy mitigates this by distributing tests across specified machines. - -- **Time-Saving Potential of Hybrid Approach :** -The Hybrid Strategy is a time-saving solution for matrix-intensive jobs. By harnessing parallelism, you can expedite testing without compromising coverage. - -Based on the above instructions passed as parameters in your [YAML](#sample-yaml-file) file, HyperExecute will now generate Virtual Machines. -- 1 VM for Win + Chrome -- 1 VM for Win + Firefox -- 1 VM for Linux + Chrome -- 1 VM for Linux + Firefox - -Now the given `parallelism` is 2, hence it will create the same set of the above combination, and since Hybrid Mode is the combination of Autosplit and Matrix method, HyperExecute will intelligently distribute tests over these Virtual Machines. - -## Using Hybrid Strategy with HyperExecute - -### Prerequisites: - -- The version of the YAML file that you are using is `0.1`. -- The test discovery mode has to be `remote`. - -> You can run your tests in Hybrid Strategy by configuring the following flags in your [YAML file](/support/docs/deep-dive-into-hyperexecute-yaml/). - -- **autosplit**: To use HyperExecute’s AutoSplit Strategy, you need to set this flag to `true`. Similarly, this flag should be set to `true` in Hybrid Strategy too. - -```yaml -autosplit: true -``` - -- **parallelism**: This key indicates the number of tests that can run in parallel. - -```yaml -parallelism: 5 -``` - -- You can define the `parallelism` for each operating system. If you have not defined the `parallelism` for any particular OS, then the value for that OS will be set to the global `parallelism` value. - -```yaml -Parallelism: 4 -winParallelism: 2 -macParallelism: 3 -``` - -- In the example given above, the `parallelism` for Linux tests is not defined and if any tests are run on Linux OS, then they will be run in 4 different virtual machines because the global value of `parallelism` is set to 4. - -- **matrix**: This flag is used to define the combination of tests you want to run in your job. You can use the matrix flag to define combinations of browsers, operating systems, and even custom parameters like files, folders, tags, scenarios, and more. - -```yaml -matrix: -os: [mac, win, linux] -var: [1,2,3] -runson: ${matrix.os} -``` - -- If you want to run a combination of different operating systems in Hybrid Strategy, then you can configure the `runson` key as `${matrix.os}`. This will only work if you have added some input for the `os` flag under `matrix` in your YAML file. - -- **testDiscovery**: The `testDiscovery` command is used to list down all the values that have to be distributed. It can be used to split tests over files, modules, or any level supported by your language and framework. - -```yaml -testDiscovery: -type: raw -mode: remote -command: snooper --featureFilePaths=src/main//Features/ --frameWork=java -``` - -- It can also be defined selectively for every platform. If you have not defined the `testDiscovery` command for a specific OS, then it takes the global value of the command. - -```yaml -testDiscovery: -type: raw -mode: remote -command: snooper --featureFilePaths=src/main//Features/ --frameWork=java -winCommand: snooper --featureFilePaths=src/main//Features/ --frameWork=java -macCommand: snooper --featureFilePaths=src/main/**/Features/ --frameWork=java -``` - -- In the example given above, the `testDiscovery` command for Linux will be: `snooper --featureFilePaths=src/main//Features/ --frameWork=java` - -- **testRunnerCommand**: The `testRunnerCommand` tells the system how to run a single test entity in isolation. This entity could be a file, module, feature or scenario. It will run over each of the values extracted from the `testDiscovery` command. - -```yaml -testRunnerCommand: mvn test -Dcucumber.options="$test" -Dscenario="$test" -DOs="win 10" -winTestRunnerCommand: mvn test -Dcucumber.options="$test" -Dscenario="$test" -DOs="win 10" -macTestRunnerCommand: mvn test -Dcucumber.options="$test" -Dscenario="$test" -DOs="Mac" -linuxTestRunnerCommand: mvn test -Dcucumber.options="$test" -Dscenario="$test" -DOs="linux" -``` - -- If the `testRunnerCommand` is not defined for a particular OS, then it takes the global value of the parameter that is defined. In the example given above, if the `testRunnerCommand` was not defined for macOS, then it would have taken the value: `mvn test -Dcucumber.options="$test" -Dscenario="$test" -DOs="win 10"` - -### Sample YAML file - -A sample YAML file used to run a job on Hybrid Strategy for your reference below: - -```yaml ---- -version: 0.1 -runson: ${matrix.os} - -autosplit: true -concurrency: 3 - -matrix: -os: [mac, win, linux] -parallelism: 2 - -env: -CACHE_DIR: m2_cache_dir - -cacheKey: '{{ checksum "pom.xml" }}' -cacheDirectories: -- ${CACHE_DIR} - -pre: -- mvn -Dmaven.repo.local=${CACHE_DIR} -Dmaven.test.skip=true clean install - -testDiscovery: -type: raw -mode: remote -macCommand: | -grep 'test name' xml/testng_mac.xml | awk '{print$2}' | sed 's/name=//g' | sed 's/\x3e//g' -winCommand: | -grep 'test name' xml/testng_win.xml | awk '{print$2}' | sed 's/name=//g' | sed 's/\x3e//g' -linuxCommand: | -grep 'test name' xml/testng_linux.xml | awk '{print$2}' | sed 's/name=//g' | sed 's/\x3e//g' ${param} - -linuxTestRunnerCommand: mvn test -Dplatname=linux -Dmaven.repo.local=./.m2 dependency:resolve -DselectedTests=$tests -winTestRunnerCommand: mvn test `-Dplatname=win `-Dmaven.repo.local=.m2 dependency:resolve `-DselectedTests=$tests -macTestRunnerCommand: mvn test -Dplatname=mac -Dmaven.repo.local=./.m2 dependency:resolve -DselectedTests=$tests - -retryOnFailure: false -maxRetries: 1 - -post: -- cat yaml/${matrix.os}/testng_hyperexecute_matrix_sample.yaml - -mergeArtifacts: true - -uploadArtefacts: -- name: XmlReports -path: -- target/surefire-reports/html/** - -jobLabel: ['hybrid-mode'] -``` diff --git a/static/docs/hyperexecute-inherit-config.md b/static/docs/hyperexecute-inherit-config.md deleted file mode 100644 index 910fe936f..000000000 --- a/static/docs/hyperexecute-inherit-config.md +++ /dev/null @@ -1,128 +0,0 @@ -# Inherit Your YAML Configurations - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -YAML inheritance in HyperExecute allows you to reuse common configuration settings across multiple [YAML](/support/docs/deep-dive-into-hyperexecute-yaml) files, improving efficiency, consistency, and flexibility. - -For example, you can create a base YAML file with common settings like **browsers**, **OS versions**, and **devices**, and then inherit from it in other YAML files for different environments or test types. This **reduces boilerplate code and errors** while making it easy to switch between environments and manage complex test execution configurations. - -## Prerequisites -- You must inherit a YAML file of the same version as your original file. -- The mode of execution, i.e. Matrix, AutoSplit, or Hybrid, must remain the same in both of the YAML files. -- The files that you want to inherit cannot be cyclically dependent. If you want to inherit **A.yaml** in **B.yaml**, then **A.yaml** cannot inherit **B.yaml**. - -## How do you use the Inheritance feature? - -You can use the inheritance feature by entering the following flag in your YAML file: - -```yaml -base: -yamls: -- ./ -- ./ -``` -- Insert the path of the YAML files you want to inherit in place of the placeholder value ``. The path of the YAML file that you want to inherit is relative to the main YAML file. - -- If the configuration YAML file is passed via the [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/), then it will be treated as the main YAML. Hence, all values of the base YAML will be overwritten by the values from the parent YAML file. However, if the configuration YAML file has a flag that contains the Boolean value False or numeric value '0', then the values from the parent YAML are used (if they are present). - -> **Note**: If you want to inherit two YAML files, then the second YAML file takes precedence. If a key is not defined in the first file or the main YAML file, but it is defined in the second, then that key will take the value of the second file. Similarly, if a key is defined in the two YAML files that are inherited but not in the main file, then the value defined in the second file is used. - -Go through the example attached below to understand how the resultant YAML will function if you inherit a base YAML in your parent YAML file. - -### Parent YAML - -```yaml ---- -version: 0.1 -# highlight-next-line -runson: linux - -autosplit: true -# highlight-next-line -concurrency: 4 - -base: -yamls: -- ./base.yaml - -pre: -- mvn dependency:resolve - -# highlight-start -testDiscovery: -mode: remote -command: grep 'test name' xml/testng_linux.xml | awk '{print$2}' | sed 's/name=//g' | sed 's/>//g' -# highlight-end - -testRunnerCommand: mvn test -Dplatname=linux -Dmaven.repo.local=./.m2 dependency:resolve -DselectedTests=$test - -retryOnFailure: true -maxRetries: 1 - -# highlight-next-line -jobLabel: [selenium-testng, linux, v1, autosplit] -``` - -### Base YAML -```yaml ---- -version: 0.1 -# highlight-next-line -runson: win - -autosplit: true -# highlight-next-line -concurrency: 2 - -pre: -- mvn dependency:resolve - -# highlight-start -testDiscovery: -type: raw -mode: static -command: grep 'test name' xml/testng_linux.xml | awk '{print$2}' | sed 's/name=//g' | sed 's/>//g' -# highlight-end - -testRunnerCommand: mvn test -Dplatname=linux -Dmaven.repo.local=./.m2 dependency:resolve -DselectedTests=$test - -retryOnFailure: true -maxRetries: 1 - -# highlight-next-line -jobLabel: [selenium-testng, autosplit] -``` - -### Resultant YAML -```yaml -version: 0.1 -# highlight-next-line -runson: linux - -autosplit: true -# highlight-next-line -concurrency: 4 - -base: -yamls: -- ./base.yaml - -pre: -- mvn dependency:resolve - -# highlight-start -testDiscovery: -command: grep 'test name' xml/testng_linux.xml | awk '{print$2}' | sed 's/name=//g' | sed 's/>//g' -mode: remote -# highlight-end - -testRunnerCommand: mvn test -Dplatname=linux -Dmaven.repo.local=./.m2 dependency:resolve -DselectedTests=$test - -maxRetries: 1 -retryOnFailure: true - -# highlight-next-line -jobLabel: [selenium-testng, linux, v1, autosplit] -``` - -To learn more about the HyperExecute YAML file, go through [this page](/support/docs/deep-dive-into-hyperexecute-yaml). diff --git a/static/docs/hyperexecute-integrate-azure-test-plan.md b/static/docs/hyperexecute-integrate-azure-test-plan.md deleted file mode 100644 index 49dca72e2..000000000 --- a/static/docs/hyperexecute-integrate-azure-test-plan.md +++ /dev/null @@ -1,68 +0,0 @@ -# Step 3: Integrate the Test Plan with HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -> Test Distribution of the suite is recommended to be at **test level** such that each HyperExecute stage has one test. - -To execute the Test Plan with HyperExecute, you will have to follow the below mentioned steps: - -**Step 1:** Download these exeutable files and place it in the root directory of your test suite. - -| Binary | Download Link | -|--------|---------------| -| Testplan Discovery | https://downloads.lambdatest.com/hyperexecute/testplan-discovery-win.exe | -| Testplan Status | https://downloads.lambdatest.com/hyperexecute/testplan-status-update-win.exe | -| Test Link Attachment | https://downloads.lambdatest.com/hyperexecute/test-link-attach-win.exe | - -**Step 2:** Add the following command in your YAML file - -```yaml -testDiscovery: -command: testplan-discovery-win.exe -mode: static -type: raw -``` - -**Step 3:** To keep the test case distribution perfectly, make sure you pass the `dynamicAllocation` flag as `true`. - -```yaml -dynamicAllocation: true -``` - -**Step 4:** Now, you need to pass the following command in your YAML file. - -```yaml -testRunnerCommand: dotnet test path\of\dll --settings path\of\runsettings --filter '"Name=$test"' ; ./testplan-status-update-win.exe ; ./test-link-attach-win.exe -``` - -**Step 5:** Trigger your test plan on HyperExecute. - -- Download the Hyperexecute CLI Binary based on the OS you use. - -| Platform | Download Link | -|-----------|--------------| -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| macOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -- Run the below mentioned commands in your terminal to setup the CLI and the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -> In case of MAC, if you get a permission denied warning while executing CLI, simply run `chmod u+x ./hyperexecute` to allow permission. In case you get a security popup, allow it from your **System Preferences → Security & Privacy → General tab**. - -- Run the below command in your terminal at the root folder of the project - -```bash -./hyperexecute --config RELATIVE_PATH_OF_YOUR_YAML_FILE -``` - -OR use this command if you have not exported your username and access key. - -```bash -./hyperexecute --user ${YOUR_LAMBDATEST_USERNAME()} --key ${YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE -``` diff --git a/static/docs/hyperexecute-integration-with-ci-cd-tools.md b/static/docs/hyperexecute-integration-with-ci-cd-tools.md deleted file mode 100644 index 922212d7f..000000000 --- a/static/docs/hyperexecute-integration-with-ci-cd-tools.md +++ /dev/null @@ -1,111 +0,0 @@ -# HyperExecute Integration With CI/CD Tools - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -HyperExecute offers integrations with numerous CI/CD tools to assist you to: -- Expend less manual effort -- Deliver faster -- Reduce risk - -To integrate any CI pipeline with HyperExecute, follow these steps: - -1. **Download the HyperExecute CLI Binary:** - -To integrate HyperExecute with your preferred CI pipeline, you need to make use of the Hyperexecute CLI. HyperExecute CLI is a command line client used to interact and run tests on HyperExecute and provide a host of other useful features that accelerate test execution. - -For triggering tests using HyperExecute CLI, you have to download the HyperExecute CLI binary corresponding to the platform (or OS) from where the tests are triggered: Below are the HyperExecute CLI download links for different operating systems: - -| OS | HyperExecute CLI download link | -| ---------| --------------------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| macOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -- If the Pipeline VM Image agent has the OS flavor of Linux or Mac, then the HyperExecute CLI binary would need additional permission. Run the command below to get the required permission: - -``` -chmod u+x hyperexecute -``` - -2. **Execute the HyperExecute CLI Binary** - -To execute the HyperExecute CLI binary, run the following command: - -- **For Windows** -``` -hyperexecute.exe –user LT_USERNAME –key LT_ACCESS_KEY –config path of the Hyperexecute YAML file -``` - -- **For Mac/Linux** - -``` -./hyperexecute –user LT_USERNAME –key LT_ACCESS_KEY –config path of the Hyperexecute YAML file -``` - -Below are detailed instructions for CI/CD tool integrations to help you get the most out of HyperExecute. - -AWS CodePipeline Integration - -Azure DevOps Pipeline Integration - -GitHub Actions Pipeline Integration - -GitLab Pipeline Integration - -CircleCI Pipeline Integration - -Jenkins Integration - -Bitbucket Pipeline Integration - -Travis CI Integration - -TeamCity Integration - -GoCD Integration - -Bamboo CI Integration - -Semaphore Integration - -Want to know more about Hyperexecute? Here’s a [link](/support/docs/getting-started-with-hyperexecute/) to our Documentation. - ->If your favourite CI/CD tool is not in the list mentioned above, then just give us a shout and we will have it ready shortly for you. - -## Using the CI/CD Pipeline Agent Skill with TestMu AI - -The [cicd-pipeline-skill](https://github.com/LambdaTest/agent-skills/tree/main/cicd-pipeline-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The cicd-pipeline-skill package includes: - -``` -cicd-pipeline-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing CI/CD Pipeline Agent Skill - -Install a CI/CD Pipeline Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/cicd-pipeline-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/cicd-pipeline-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only cicd-pipeline-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). diff --git a/static/docs/hyperexecute-integration-with-products.md b/static/docs/hyperexecute-integration-with-products.md deleted file mode 100644 index 65284ee3b..000000000 --- a/static/docs/hyperexecute-integration-with-products.md +++ /dev/null @@ -1,46 +0,0 @@ -# HyperExecute Integrations with Products - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI now offers you the chance to integrate HyperExecute with products like Tosca, Katalon and more. You can go through the links below to see the integrations we offer at the moment. - - Accelq - Perform codeless automated functional and API testing using Acceelq. - - algoQA - Perform software testing through automated test case generation and execution for various applications. - - Azure Test Plan - Manage test plans, test suites, and test cases for your web and desktop applications. - - Katalon - Perform Cross Platform Testing ranging from web, API, mobile, and desktop applications. - - Provar - Enable efficient creation and execution of UI and API tests. - - qTest - Perform Cross Platform Testing, and get a centralized test management tool. - - Sikuli - Automate the visual testing of any software by integrating Sikuli with HyperExecute. - - Slack - Receive real-time notifications and crucial job details directly within your Slack workspace. - - Testim - Automate your software testing by creating stable automated tests. - - Testsigma - Perform codeless test automation to easily create and maintain tests. - - Tosca - Optimize and accelerate end-to-end testing for your entire digital landscape. - - WinAppDriver - Perform the testing for the tests that are based on Microsoft Office Suite for TestNG Framework. - - Zephyr - Zephyr is a test case management tool allows you to centrally manage your test suite cases. - ->If your favourite products are not mentioned in the list above, then just give us a shout and we will have them ready for you. diff --git a/static/docs/hyperexecute-integration-with-virtual-devices.md b/static/docs/hyperexecute-integration-with-virtual-devices.md deleted file mode 100644 index 23d88d291..000000000 --- a/static/docs/hyperexecute-integration-with-virtual-devices.md +++ /dev/null @@ -1,224 +0,0 @@ -# Virtual Device Integration with HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This page outlines how to execute your Virtual Device tests on HyperExecute with [YAML 0.2](/support/docs/hyperexecute-yaml-version0.2/) - -**Note** - -Currently, App Automation is available for Emulators and Simulators in the Beta phase, with support for the Appium framework. To use this feature, [Contact Sales](https://www.testmuai.com/contact-us). - -## Prerequisites - -To run the Tests on HyperExecute from your Local System, you are required: - -- Install [Appium Java Client](https://github.com/appium/java-client). -- Your TestMu AI [Username and Access key](/support/docs/hyperexecute-how-to-get-my-username-and-access-key/) -- [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) in order to initiate a test execution Job . -- Setup the [Environmental Variable](/support/docs/hyperexecute-environment-variable-setup/) -- [HyperExecute YAML](/support/docs/hyperexecute-yaml-version0.2/) file which contains all the necessary instructions. -- You have an access to an **Android** app (*.apk* or *.aab* file) or an **iOS** *zip* file (containing *.app* file). - -Run your samplе tеsts on HyperExecute using our samplе Android and iOS applications: -* [Samplе Android App](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk) -* [Samplе iOS App](https://prod-mobile-artefacts.lambdatest.com/assets/docs/firefox.zip) - -## Step 1: Setup Your Test Suite - -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -## Step 2: Setup the CLI in your Test Suite - -After cloning / downloading the sample repo, you need to setup the CLI and the environment variables. - -### Download the HyperExecute CLI - -The CLI is used for triggering the tests on HyperExecute. It is recommend to download the CLI binary on the host system and keep it in the root directory of the suite to perform the tests on HyperExecute. - -You can download the CLI for your desired platform from the below mentioned links: - -| Platform | HyperExecute CLI | -| ---------| ---------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| MacOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -### Setup Environment Variable - -Now, you need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in the [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). - -Run the below mentioned commands in your terminal to setup the CLI and the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -## Step 3: Upload your Application - -Upload your **iOS** application (*.app* file) or **Android** application (*.apk* file) to the TestMu AI servers using our **REST API**. You need to provide your **Username** and **AccessKey** in the format `Username:AccessKey` in the **cURL** command for authentication. Make sure to add the path of the **appFile** in the cURL request. Here is an example cURL request to upload your app using our REST API: - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" --location --request POST 'https://manual-api.lambdatest.com/app/upload/virtualDevice' --form 'name="Android_App"' --form 'appFile=@"/Users/macuser/Downloads/proverbial_android.apk"' -`} - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" --location --request POST 'https://manual-api.lambdatest.com/app/upload/virtualDevice' --form 'name="Android_App"' --form 'appFile=@"/Users/macuser/Downloads/proverbial_android.apk"' -`} - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" --location --request POST 'https://manual-api.lambdatest.com/app/upload/virtualDevice' --form 'name="Android_App"' --form 'url="https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk"'`} - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" --location --request POST "https://manual-api.lambdatest.com/app/upload/virtualDevice" --header "Content-Type: application/x-www-form-urlencoded" --data-urlencode "url=:https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk" --data-urlencode "name=Proverbial_App"`} - -> Response of above cURL will be a **JSON** object containing the `App URL` of the format `lt://APP123456789123456789` - -## Step 4: Update Your Automation Script - -Here is a sample automation script in Java for the sample app downloaded above. In the below test script, ensure to update the `app_url`, or `app_id` - -**Set the Capability** -To run the test on Emulator/Simulator, set the `isRealMobile` capability to `false`. - -```java -@Test -@org.testng.annotations.Parameters(value = {"device", "version", "platform"}) -public void AndroidApp1(String device, String version, String platform) { -version = System.getProperty("platformVersion"); -try { -DesiredCapabilities capabilities = new DesiredCapabilities(); -capabilities.setCapability("build","Java TestNG Android"); -capabilities.setCapability("name",platform+" "+device+" "+version); -capabilities.setCapability("deviceName", device); -capabilities.setCapability("platformVersion",version); -capabilities.setCapability("platformName", platform); -// highlight-next-line -capabilities.setCapability("isRealMobile", false); - -// highlight-next-line -capabilities.setCapability("app", "lt://APP1123456789"); //Enter your app url -capabilities.setCapability("deviceOrientation", "PORTRAIT"); -capabilities.setCapability("console", true); -capabilities.setCapability("network", false); -// capabilities.setCapability("visual", true); -capabilities.setCapability("devicelog", true); -//capabilities.setCapability("geoLocation", "HK"); -``` - -```java -@Test -@org.testng.annotations.Parameters(value = {"device", "version", "platform"}) -public void iOSApp1(String device, String version, String platform) { -try { -DesiredCapabilities capabilities = new DesiredCapabilities(); -capabilities.setCapability("build","Java TestNG iOS"); -capabilities.setCapability("name",platform+" "+device+" "+version); -capabilities.setCapability("deviceName", device); -capabilities.setCapability("platformVersion",version); -capabilities.setCapability("platformName", platform); -// highlight-next-line -capabilities.setCapability("isRealMobile", false); - -// highlight-next-line -capabilities.setCapability("app", "lt://APP123456789"); //Enter your app url -capabilities.setCapability("deviceOrientation", "PORTRAIT"); -capabilities.setCapability("console", true); -capabilities.setCapability("network", false); -// capabilities.setCapability("visual", true); -capabilities.setCapability("devicelog", true); -//capabilities.setCapability("geoLocation", "HK"); -``` - -2. Create `.XML` file in order to run your test and define device capabilities. Please find sample code below for the same. - -```xml - - - - - - - - - - - - - - - - - - - - - - -``` - -```xml - - - - - - - - - - - - - - - - - - - - - - -``` - -## Step 5: Configure YAML in your Test Suite - -```yaml ---- -version: 0.2 -globalTimeout: 150 -testSuiteTimeout: 150 -testSuiteStep: 150 - -runson: android - -concurrency: 2 - -autosplit: true - -retryOnFailure: false -maxRetries: 1 - -appium: true -framework: -name: maven/testng -defaultReports: false -discoveryType: xmltest -flags: ["-Pios-single"] - -jobLabel: ['HYP', 'Virtual Device', 'iOS', 'Single Device'] -``` - -## Step 6: Execute your Test Suite - -> **NOTE :** In case of MacOS, if you get a permission denied warning while executing CLI, simply run **`chmod u+x ./hyperexecute`** to allow permission. In case you get a security popup, allow it from your **System Preferences** → **Security & Privacy** → **General tab**. - -Run the below command in your terminal at the root folder of the project: - -```bash -./hyperexecute --config RELATIVE_PATH_OF_YOUR_YAML_FILE -``` - -OR use this command if you have not exported your username and access key in the step 2. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE `} - -Visit the [HyperExecute Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) and check your Job status 🚀 diff --git a/static/docs/hyperexecute-job-reports.md b/static/docs/hyperexecute-job-reports.md deleted file mode 100644 index 3ac6aa344..000000000 --- a/static/docs/hyperexecute-job-reports.md +++ /dev/null @@ -1,80 +0,0 @@ -# HyperExecute Job Reports - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -HyperExecute Job Reports are designed to offer deep insights into the performance of your tests. By utilizing these reports, you can identify issues, track progress, and enhance your software testing workflow. Whether you're a developer or QA engineer, understanding and leveraging the various report types is crucial to delivering high-quality software efficiently. - -If you want to learn how to **generate** and **download** detailed reports, including **multiple reports**, please refer to the [HyperExecute Reports documentation](/support/docs/hyperexecute-reports/) page. - -## Extent Report -The [Extent Report](/support/docs/extent-report/) is a robust reporting framework commonly used with Java TestNG, and Selenium tests. It provides a feature-rich environment to explore test results, offering: - -- Comprehensive test case summaries. -- Visuals like screenshots and videos of test executions. -- Detailed execution logs. -- Graphical data, including charts and graphs to simplify test result analysis. - -## Extent Native -[Extent Native](/support/docs/native-extent-report/) is a more modern reporting solution from the Extent Reports family. Extent Native offers a native reporting experience that is highly interactive and responsive. It supports Java, TestNG, and Selenium tests and includes: - -- Drill-down analysis to examine results in depth. -- Custom filtering options for tailored views. -- Real-time updates, accessible on any device for a seamless experience. - -## Native TestNG Reports -The [Native TestNG Reports](/support/docs/hyperexecute-native-testng-report/) provide a standardized summary extracted from raw Extent reports for each Virtual Machine (VM) at the conclusion of a HyperExecute job. These reports deliver a consolidated HTML view, generated by the TestNG framework, ensuring easy access to comprehensive testing results - -## Allure Report -[Allure](/support/docs/allure-reports/) is a cross-platform reporting tool compatible with various testing frameworks and languages. It offers a rich feature set for comprehensive analysis, including: - -- Interactive and responsive reporting dashboards. -- Drill-down analysis to explore test outcomes. -- Custom filters and real-time updates. -- CI/CD tool integration for streamlined workflows. - -## Cucumber Report -[Cucumber Report](/support/docs/cucumber-report/) focuses on Cucumber test executions. It provides: - -- Test step summaries with detailed logs. -- Screenshots and videos for visual verification of results. -- Charts and graphs for analyzing test trends. - -## Playwright HTML -[Playwright HTML](/support/docs/playwright-html-report/) report is specifically designed for Playwright tests and offers detailed execution summaries, including: - -- Test suite, case, and step summaries. -- Visual assets such as screenshots and test execution videos. -- Execution logs along with graphical data for analyzing results. - -## Specflow -[Specflow](/support/docs/specflow-report/) is a popular testing framework for Behavior-Driven Development (BDD), supporting languages like C#, Java, and Ruby. Its reports include: - -- Feature summaries and scenario breakdowns. -- Step-by-step execution details. -- Screenshots, videos, and execution logs for comprehensive result visualization. - -## Cypress Mochawesome Report -[Cypress Mochawesome](/support/docs/cypress-mochaawesome-report/) is designed for Cypress and Mocha tests, this report offers: - -- Test suite and case summaries. -- Test step breakdowns with logs. -- Screenshots and videos to validate test executions. - -## Error Categorization Report -The [Error Categorization Report](/support/docs/error-categorization-report/) report is crafted to help users classify and understand test failures systematically. Key benefits include: - -- Simplified error classification by grouping failures with shared characteristics. -- Detailed navigation through the problematic test stages. -- Downloadable data for thorough analysis and resolution. - -## JUnit XML Report -The [JUnit XML Report](/support/docs/junit-xml-report/) provides a summary of test results conducted on [Date and Time]. The tests are organized into multiple test suites, with detailed information on execution time, status, and individual test cases. - -## Karate Report -[Karate Reports](/support/docs/karate-report/) provide a comprehensive overview of your test execution results, offering detailed insights into test performance, coverage, and failures. With visually appealing and interactive HTML reports, you can quickly analyze test outcomes, debug issues, and share results across teams. These reports are seamlessly integrated into Karate's powerful testing framework, ensuring clarity and efficiency in your test automation journey. - -## Robot Report -[Robot Reports](/support/docs/robot-report/) provides a detailed summary of automated test execution using Robot Framework, including test case results, execution time, logs, and captured output. It helps analyze pass/fail trends, identify failures, and validate system functionality across environments. - -## Katalon Report -[Katalon Reports](/support/docs/hyperexecute-katalon-report/) summarizes the results of automated test executions in Katalon Studio, including test status, execution time, environment details, and error logs. It provides clear insights into test coverage, performance, and overall build quality. diff --git a/static/docs/hyperexecute-job-widgets.md b/static/docs/hyperexecute-job-widgets.md deleted file mode 100644 index 21f02d4c9..000000000 --- a/static/docs/hyperexecute-job-widgets.md +++ /dev/null @@ -1,27 +0,0 @@ -# HyperExecute Job Widgets - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Job Trends - -The `Job Trends` Widget empowers you to analyze the trends of executed Jobs on the TestMu AI platform, categorized as **Failed**, **Aborted**, **Lambda Error**, **Timeout**, and **Completed**. - -You can leverage the widget to understand the distribution of Job statuses, enabling a comprehensive assessment of your team's application performance. By utilizing legends for seamless filtering, you can quickly identify specific failing Job numbers and troubleshoo - -## Job Summary - -The `Job Summary` widget enables you to track the total Jobs being run on the platform grouped by their status including **Completed**, **Partially Completed**, **Failed**, **Aborted**, etc. - -If you wants to get a total number of the Jobs being executed on the platform for a specific environment or status. With the Job Summary widget you can see the total count of the desired use cases. - -## Job Queue Time Trends - -The **Job Queue Time Trends** Widget provides you a comprehensive view of the total queue time for all jobs in a visual format. The widget displays a stacked line or bar chart, with the total queue time for each job on the y-axis and the job number in descending order from right on the x-axis. - -This visualization enables you to quickly identify trends and patterns in the queue time across multiple jobs, helping to optimize the testing processes and resource allocation with other use cases like: - -- Identifying trends in queue time -- Optimizing resource allocation -- Monitoring the impact of changes -- Comparing performance over time -- Sharing insights with team members diff --git a/static/docs/hyperexecute-jobs-archiving.md b/static/docs/hyperexecute-jobs-archiving.md deleted file mode 100644 index 2356a132a..000000000 --- a/static/docs/hyperexecute-jobs-archiving.md +++ /dev/null @@ -1,19 +0,0 @@ -# Jobs Archiving - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -HyperExecute's job archiving feature lets you seamlessly organize your workspace by tucking away irrelevant jobs, enhancing your team's efficiency. By archiving jobs, you ensure that only the most relevant and significant jobs occupy your workspace, enabling your team to stay focused and productive. - -> Archived jobs are not deleted, they are just removed from the default view on this page. - -## Archive the Jobs - -In order to archive any job, just click on the eclipses (three dots) that you see on the right side on job list page, it would provide you a small dropdown to archive that job. - -## View the Archived Jobs - -In order to view the archived jobs, you can simply go to filters and select view archived jobs. All the jobs that you archive, will be available here. - -## Unarchive the Jobs - -You can also unarchive your jobs similarly the way you archived your jobs. diff --git a/static/docs/hyperexecute-k6-testing.md b/static/docs/hyperexecute-k6-testing.md deleted file mode 100644 index 0ae2047a5..000000000 --- a/static/docs/hyperexecute-k6-testing.md +++ /dev/null @@ -1,99 +0,0 @@ -# Execute k6 Tests on HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -[k6](https://grafana.com/docs/k6/latest/) is an open-source tool designed to make load testing easy and accessible for developers and QA engineers. It's specifically built for testing the performance of APIs, microservices, and websites. - -[HyperExecute](/support/docs/getting-started-with-hyperexecute/) is an AI Native Test Orchestration Cloud Platform that empowers you to run **end-to-end** tests **quickly** and **efficiently**. It provides Just-in-Time (JIT) testing infrastructure with fast execution **speeds**, **smart orchestration**, and **detailed logs**. - -This guide details how to execute your **k6** tests on **HyperExecute** using [YAML 0.1](/support/docs/hyperexecute-yaml-parameters/) - -### Prerequisites - -To run the Tests on HyperExecute from your Local System, you are required: - -- Your TestMu AI [Username and Access key](/support/docs/hyperexecute-how-to-get-my-username-and-access-key/) -- [HyperExecute YAML](/support/docs/hyperexecute-yaml-version0.2/) file which contains all the necessary instructions. -- [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) in order to initiate a test execution Job . -- Setup the [Environmental Variable](/support/docs/hyperexecute-environment-variable-setup/) - -### Step 1: Configure Your Test Suite - -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** -Download or Clone the code sample for the k6 from the TestMu AI GitHub repository to run the tests on the HyperExecute. - - View on GitHub - -### Step 2: Setup the CLI in your Test Suite - -After cloning / downloading the sample repo, you need to setup the CLI and the environment variables. - -#### Download the HyperExecute CLI - -The CLI is used for triggering the tests on HyperExecute. It is recommend to download the CLI binary on the host system and keep it in the root directory of the suite to perform the tests on HyperExecute. - -You can download the CLI for your desired platform from the below mentioned links: - -| Platform | HyperExecute CLI | -| ---------| ---------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| MacOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -#### Setup Environment Variable - -Now, you need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in the [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). - -Run the below mentioned commands in your terminal to setup the CLI and the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -### Step 3: Configure YAML in your Test Suite - -You will have to add these flags mandatorily in your YAML file to execute the k6 tests: - -- [runtime flag](/support/docs/deep-dive-into-hyperexecute-yaml/#runtime) to download and install the k6 of that respective version. - -```yaml -runtime: -addons: -- name: k6 -version: "v0.52.0" -``` - -- [env flag](/support/docs/deep-dive-into-hyperexecute-yaml/#env) is used to install certain dependencies to run your test cases - -```yaml -env: -K6_BROWSER_ENABLED: true #Indicates whether browser emulation is enabled for the k6 test. -K6_BROWSER_HEADLESS: false #Determines if the browser should run in headless mode -HE_CONTEXT_K6_SETUP_DEFAULT_BROWSER_PATH: true -``` - -```yaml reference title="hyperexecute.yaml" -https://github.com/LambdaTest/HYP-K6-browser-sample/blob/main/hyperexecute.yaml -``` - -### Step 4: Execute your Test Suite - -> **NOTE :** In case of MacOS, if you get a permission denied warning while executing CLI, simply run **`chmod u+x ./hyperexecute`** to allow permission. In case you get a security popup, allow it from your **System Preferences** → **Security & Privacy** → **General tab**. - -Run the below command in your terminal at the root folder of the project: - -```bash -./hyperexecute --config RELATIVE_PATH_OF_YOUR_YAML_FILE -``` - -OR use this command if you have not exported your username and access key in the step 2. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE `} - -### Step 5: Monitor the Test Execution - -Visit the [HyperExecute Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) and check your Job status. diff --git a/static/docs/hyperexecute-karate-testing.md b/static/docs/hyperexecute-karate-testing.md deleted file mode 100644 index 307f8e152..000000000 --- a/static/docs/hyperexecute-karate-testing.md +++ /dev/null @@ -1,219 +0,0 @@ -# Karate Automation on HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Karate is a popular open-source test automation framework that combines API test-automation, mocks, performance testing, and even UI testing into a single framework. It allows writing expressive tests in a simple Gherkin syntax, while leveraging Java for extensibility. - -## Why Choose Karate for API Testing? -Karate is a unified test automation framework that combines API test automation, mocks, performance testing, and UI automation into a single tool. It’s designed for simplicity and power: - -### Key Benefits of Karate: - -- **Readable BDD syntax:** Write feature files in plain English. -- **Built-in HTTP client:** No need for external tools. -- **Supports parallel execution:** Speed up test cycles. -- **All-in-one testing:** Covers API, UI, mocks, and performance. -- **CI-friendly:** Works well with Jenkins, GitHub Actions, GitLab, and more. - -## The Challenge: Scaling Karate Tests in CI/CD -As your project grows, so do your tests. And with scale come challenges: - -- **Long test execution times** (e.g., 30+ minutes for 100+ feature files) -- **Inefficient retries**, often requiring full suite re-runs -- **Hard-to-debug flakiness** and poor visibility into trends -- **Scattered logs** and reports across environments - -## Why HyperExecute Is Built for Karate Teams -| Challenge | Karate Alone | Karate + HyperExecute | -| --------------------- | ----------------- | ----------------------------- | -| Slow suite execution | 30+ mins | < 5 mins with autosplitting | -| Flaky test tracking | Manual logs | Built-in flakiness insights | -| Parallel test scaling | Custom threads | Seamless, node-based scaling | -| Debug logs & reports | Scattered | Unified dashboard & rich logs | -| Retry support | Full suite reruns | Scenario-level retry logic | - -# Service Virtualization - -TestMu AI supports service virtualization at two layers, without dedicated mock infrastructure to maintain. - -- **Framework-level request stubbing** - intercept requests and return controlled responses at the network layer using your framework's native tooling: Playwright `route`, Cypress `intercept`, WebdriverIO `mock`, and Selenium CDP/BiDi. Stubs run on our CDP/BiDi-capable grids and live in your test code, versioned in Git. -- **Karate mock servers** - stand up Karate mock servers as part of a HyperExecute run to deliver controlled responses to the system under test, defined in the same Gherkin feature files as your tests; scale them across parallel nodes with autosplit. - -In agentic workflows, a coding agent using Kane CLI or Agent Skills can generate stub definitions from an API spec and execute the flow with those responses injected. Note: TestMu AI does not host a standalone, spec-driven virtual-service generator - virtualization is delivered through framework stubbing and in-run Karate mocks. - -## Example Use Case: Scaling Karate API Tests with HyperExecute -A QA team executes 100+ Karate feature files nightly. Over time, they face: - -- 30+ minute run times -- Flaky test behavior hard to trace -- Manual re-runs of the entire test suite on failure - -### HyperExecute Solution: -#### 1. Blazing Fast Execution via Autosplit - -```yaml -autosplit: true -concurrency: 10 -``` -HyperExecute distributes feature files intelligently across defined parallel nodes, bringing execution time down to minutes from hours. - -#### 2. Retry Only What Fails -Automatically re-run failed scenarios without restarting the entire suite. - -#### 3. Real-Time Logs & Reports** -Debug faster with per-test logs, reports, and console outputs, available instantly in the HyperExecute dashboard. - -#### 4. Flakiness & Stability Insights** -Track unstable tests using built-in analytics that detect patterns of failure across builds. - -## Getting Started with Karate on HyperExecute - -### Prerequisites -To run the Tests on HyperExecute from your Local System, you are required: - -- Your TestMu AI [Username and Access key](/support/docs/hyperexecute-how-to-get-my-username-and-access-key/) -- [HyperExecute YAML](/support/docs/hyperexecute-yaml-version0.2/) file which contains all the necessary instructions. -- [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) in order to initiate a test execution Job . -- Setup the [Environmental Variable](/support/docs/hyperexecute-environment-variable-setup/) - -### Step 1: Download Project -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** -Download or Clone the code sample for the Karate from the TestMu AI GitHub repository to run the tests on the HyperExecute. - - View on GitHub - -### Step 2: Download the CLI in your Project -Download the HyperExecute CLI and copy it into the root folder of the downloaded sample repository. - -| Platform | HyperExecute CLI | -| ---------| ---------------- | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| macOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | - -### Step 3: Configyure `karate-config.js` file -This file defines runtime behaviors and integrates Karate with TestMu AI status reporting. - -```javascript title="karate-config.js" -function fn() { -var lambdaHooks = function() { -if (karate.info.errorMessage) { -script('lambda-status=failed'); -} else { -script('lambda-status=passed'); -} -} - -var env = karate.env || 'dev'; -karate.log('karate.env:', env); - -var config = { -env: env, -hub: karate.properties['hub'] -}; - -karate.configure('afterScenario', lambdaHooks); - -return config; -} -``` - -- `lambdaHooks` sets test status based on execution outcome. -- `hub` is dynamically picked to support Selenium Grid for UI tests. -- Supports environment-based configuration using `karate.env`. - -### Step 4: Create your hyperexecute.yml file -The core of HyperExecute configuration lies in the `hyperexecute.yaml` file. Let’s break down the structure and understand each section: - -#### 1. Environment & Runtime Setup -This section specifies the OS, runtime language, concurrency, and autosplit features: - -```yaml -version: 0.1 -runson: linux # OS to run the tests (e.g., linux, win) -autosplit: true -concurrency: 2 # Defines the number of test sessions to run concurrently - -runtime: -language: java -version: 11 -``` - -#### 2. Dependency Resolution with Maven -Before running the actual performance test, ensure all project dependencies are resolved locally for a reproducible build. This step pulls all required Maven dependencies to a local directory (.m2), ensuring environment consistency. - -```yaml -pre: -- mvn -Dmaven.repo.local=./.m2 dependency:resolve -``` - -#### 3. Configure the Test Execution Command -The `testRunnerCommand` defines how each test is executed on the HyperExecute infrastructure. With `autosplit: true`, HyperExecute dynamically distributes each test to a separate runner, enabling parallel execution. - -```yaml -testRunnerCommand: mvn test -Dtest=MyApiRunner -DFeaturePath="$test" -Dhub=https://LT_USERNAME:LT_ACCESS_KEY@hub.lambdatest.com/wd/hub -Dmaven.repo.local=./.m2 -``` - -- **-Dtest=MyApiRunner:** Specifies the Java class that acts as the Karate runner. It must extend KarateRunner. -- **-DFeaturePath="$test":** $test is dynamically provided by the discovery command. Each value here is a specific .feature file path. -- **-Dhub=...:** Optional. Used when your Karate test needs to connect to Selenium Grid for UI tests. -- **-Dmaven.repo.local=./.m2:** Uses a local Maven repo to avoid repeated dependency downloads on each VM. - -This command runs one feature file per runner, as decided by autosplit. However, you can change the granularity and run: - -- All scenarios tagged with @smoke -- A specific step definition class -- A subset of feature files based on folder/module - -> mvn test -Dkarate.options="--tags @smoke" - -#### 4. Test Discovery Configuration -Test discovery determines which test files or test cases to run, and provides those to the test runner. In HyperExecute, the `testDiscovery` block parses and lists all .feature files, which are then split and executed. - -```yaml -testDiscovery: -type: raw -mode: static -command: snooper --targetOs=win --featureFilePaths=src/test/java/app --frameWork=java | sed 's/:.*//' | uniq -``` - -- **snooper:** A built-in utility provided by TestMu AI to list relevant test files. -- **--targetOs=win:** Targets Windows OS runners. -- **--featureFilePaths=src/test/java/app:** Points to where Karate .feature files are located. -- **--frameWork=java:** Indicates framework type for parsing. -- **sed 's/:.*//' | uniq:** Cleans up duplicate or unnecessary output from the snooper tool. - -#### How It Works -Discovery command runs first and lists paths to all .feature files. These paths are saved as individual test cases. HyperExecute passes one path at a time to each test runner via the $test placeholder. - -The runner command executes the test against that feature. Each of these will be executed in parallel across the nodes defined under concurrency. - -``` -src/test/java/app/login.feature -src/test/java/app/signup.feature -src/test/java/app/payments.feature -``` - -Here is a complete working YAML configuration that runs Gatling performance tests on linux runners via HyperExecute: - -```yaml reference title="hyperexecute.yaml" -https://github.com/LambdaTest/hyperexecute-karate-sample/blob/main/HyperExecute.yaml -``` - -> 📘 For a deeper understanding and project-specific configurations, check out the [YAML documentation](/support/docs/hyperexecute-yaml-parameters/). - -### Step 5: Execute your Test Suite -From the project root directory, execute the below CLI command in your terminal: - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config hyperexecute.yaml`} - -> **NOTE :** In case of macOS, if you get a permission denied warning while executing CLI, simply run **`chmod u+x ./hyperexecute`** to allow permission. In case you get a security popup, allow it from your **System Preferences** → **Security & Privacy** → **General tab**. - -## Advanced Parameters -Optimize your test pipeline using the following advanced features: - -- ✅ [Smart Test Splitting](/support/docs/hyperexecute-test-splitting-and-multiplexing/) – Automatically distribute tests across parallel runners -- ⚡ [Fail Fast](/support/docs/hyperexecute-failfast/) – Stop test runs on the first failure -- 📊 [Detailed Reports](/support/docs/hyperexecute-reports/) – Real-time terminal logs & rich test reports diff --git a/static/docs/hyperexecute-katalon-report.md b/static/docs/hyperexecute-katalon-report.md deleted file mode 100644 index 31b9b95f2..000000000 --- a/static/docs/hyperexecute-katalon-report.md +++ /dev/null @@ -1,32 +0,0 @@ -# Katalon Reports on HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -HyperExecute provides seamless integration with Katalon Studio, enabling you to generate and view detailed execution reports directly from your test runs. These reports include details such as: - -- Test case execution status (Pass/Fail/Skipped) -- Execution time and duration -- Environment and browser details -- Stack traces for failed tests - -> Ensure your project is configured with the [Katalon framework]( https://www.testmuai.com/support/docs/katalon-integration-with-hyperexecute/) and all test scripts are in place. - -## Steps to Generate Katalon Reports on HyperExecute - -**Step 1:** Configure all the necessary dependencies and configurations to your project. - -**Step 2:** Configure the report parameters in the HyperExecute YAML file - -Add the following configurations in your YAML file: - -```yaml -report: true -partialReports: -type: html -location: Reports/ -frameworkName: katalon -``` - -**Step 3:** Now execute your job by triggering the HyperExecute CLI. You can visit the HyperExecute dashboard to download the report after job completion. - -> This image shows consolidated report of katalon 10+ versions diff --git a/static/docs/hyperexecute-knowledge-base.md b/static/docs/hyperexecute-knowledge-base.md deleted file mode 100644 index f7ee5d5d9..000000000 --- a/static/docs/hyperexecute-knowledge-base.md +++ /dev/null @@ -1,23 +0,0 @@ -# Hyperexecute Knowledge Base - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -HyperExecute Status - Learn all the status of the HyperExecute at different levels - Job level, Task level, Stage level and Test level. - - How To Guide - Understand how to perform various activities in the HyperExecute Dashboard. - - HyperExecute Use Cases - Learn about the real world implementations of the HyperExecute. - - HyperExecute Widgets - Learn the insights of your Tasks based on the defined parameters in the Tests. - - HyperExecute Job Reports - Learn about different types of Job Reports based on various frameworks. - - Migration Guide - Learn how to migrate your tests from the Saucelabs to HyperExecute. - ->If you have more questions, then just give us a shout and we will answer them for you. diff --git a/static/docs/hyperexecute-maestro-testing.md b/static/docs/hyperexecute-maestro-testing.md deleted file mode 100644 index 50b85da37..000000000 --- a/static/docs/hyperexecute-maestro-testing.md +++ /dev/null @@ -1,269 +0,0 @@ -# Execute Maestro Framework Tests on HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This page outlines how to execute your Maestro tests on HyperExecute with [YAML 0.2](/support/docs/hyperexecute-yaml-version0.2/) - -## Prerequisites -To run the Tests on HyperExecute from your Local System, you are required: - -- Your TestMu AI [Username and Access key](/support/docs/hyperexecute-how-to-get-my-username-and-access-key/) -- [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) in order to initiate a test execution Job . -- Setup the [Environmental Variable](/support/docs/hyperexecute-environment-variable-setup/) -- [HyperExecute YAML](/support/docs/hyperexecute-yaml-version0.2/) file which contains all the necessary instructions. - -## Step 1: Setup Your Test Suite -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** -Download or Clone the code sample for the Maestro framework from the TestMu AI GitHub repository to run the tests on the HyperExecute. - View on GitHub - -## Step 2: Setup the CLI in your Test Suite -After cloning / downloading the sample repo, you need to setup the CLI and the environment variables. - -### Download the HyperExecute CLI -The CLI is used for triggering the tests on HyperExecute. It is recommend to download the CLI binary on the host system and keep it in the root directory of the suite to perform the tests on HyperExecute. - -You can download the CLI for your desired platform from the below mentioned links: - -| Platform | HyperExecute CLI | -| ---------| ---------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| MacOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -## Step 3: Upload your Application -Upload your _android_ application (.apk file) or iOS application (.ipa file) to the TestMu AI servers using our REST API. You need to provide your Username and AccessKey in the format `Username:AccessKey` in the cURL command for authentication. - -Enter your local path of the code repository instead of `` in the below cURL command. - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "appFile=@""" -F "name="sampleApp"" -`} - -> Response of above cURL will be a **JSON** object containing the `App ID` of the format - `` and will be used in the next step. - -## Step 4: Configure YAML in your Test Suite -Enter your `APP_ID` in the YAML file that you have fetched in the above step. - -> To enable this for your organizaton, connect with us through our **24/7 chat support** or drop us an email to [support@testmuai.com](mailto:support@testmuai.com). - -```yaml reference title="hyperexecute.yaml" -https://github.com/LambdaTest/hyperexecute-maestro-sample-test/blob/main/yaml/android/android-emulator.yaml -``` - -```yaml reference title="hyperexecute.yaml" -https://github.com/LambdaTest/hyperexecute-maestro-sample-test/blob/main/yaml/android/android-realdevice.yaml -``` - -> To enable this for your organizaton, connect with us through our **24/7 chat support** or drop us an email to [support@testmuai.com](mailto:support@testmuai.com). - -```yaml reference title="hyperexecute.yaml" -https://github.com/LambdaTest/hyperexecute-maestro-sample-test/blob/main/yaml/ios/ios-simulator.yaml -``` - -> HyperExecute now supports [tunnel capabilities](/support/docs/hyperexecute-how-to-configure-tunnel/) for Maestro tests running on both virtual devices and real devices using the Raw Framework configuration. - -### Run Tests on iOS Virtual Devices -To run tests on iOS Virtual Devices, make the following changes in your `hyperexecute.yaml` file: - -- Change the `runson` key to `ios26`. -- Set the `devices` array to `["iPhone 17"]`. - -Here is the complete `hyperexecute.yaml` for running Maestro tests on iOS Virtual Devices: - -```yaml title="hyperexecute.yaml" -# Define the version of the configuration file -version: "0.2" - -# Enable autosplit for test execution -autosplit: true - -# Set the concurrency level for test execution (2 devices in parallel) -concurrency: 2 - -# Specify the target platform for test execution (iOS in this case) -# runson: ios -runson: ios26 - -# Enable dynamic allocation of resources -dynamicAllocation: true - -# Test framework configuration -framework: -# Name of the test framework (raw in this case) -name: raw -args: -# List of devices to run tests on (iPhone 17 on iOS 26.0 in this case) -# devices: [".*-.*", ".*-.*", ".*-.*"] -devices: ["iPhone 17"] -# devices: [".*-26.0"] -# Enable or disable video recording support -video: true -# Enable or disable device log support -deviceLog: true -# App ID to be installed (mandatory field, using ) -# x86 build -# appId: lt://APP10160362031781245339521143 #Need to upload .zip file -# ARM Build for iOS 26.0 & above -appId: lt://APP123456789012345678901234567 -# Build name for identification on the automation dashboard -buildName: maestro-t1 -# Timeout for device queue -queueTimeout: 600 -# Configuration fields specific to running raw tests -# region: ap -disableReleaseDevice: true -reservation: false -isRealMobile: false -network: true -platformName: ios - -env: -MAESTRO: true -MAESTRO_LOGS_DIR: MaestroLogs - -# Pre-install required dependencies using pip -# will need java and maestro inside the container -pre: -- chmod +x maestro-test/setup-script-iOS.sh -- chmod +x ./maestro-test/runTest_ios.sh -- ./maestro-test/setup-script-iOS.sh - -# Test discovery configuration -testDiscovery: -# Command to discover tests from the test.txt file -command: cat ./maestro-test/discover-iOS.txt -# Test discovery mode can be static/dynamic -mode: static -# Test type is raw (custom test implementation) -type: raw - -# Command to run the tests using the testRunnerCommand -testRunnerCommand: ./maestro-test/runTest_ios.sh $test - -# Only report the status of the test framework -frameworkStatusOnly: true - -report: true -partialReports: -- location: . -type: xml -frameworkName: junit - -jobLabel: ['HYP', 'Maestro', 'iOS', Simulator] -``` - -Ensure that the app is built for ARM or Universal (Dual-Architecture) and not as an x86-only binary. As shown in the `appId` field above, use the ARM build for iOS 26.0 and above. - -## Step 5: Generate JUnit XML Report -1. Update the `runTest.sh` file to include the `--format junit` flag in the maestro test command: - -```yaml -/home/ltuser/.maestro/bin/maestro test $1 --debug-output ./MaestroLogs --format junit -``` - -The above command will generate a `report.xml` file in the root directory after each test execution. Here is the complete reference of the `runTest.sh` file: - -```yaml reference -https://github.com/LambdaTest/hyperexecute-maestro-sample-test/blob/main/maestro-test/runTest.sh -``` -When running on iOS real devices, you need to use a dedicated script since the execution flow differs slightly from iOS simulators and Android. - -```yaml reference -https://github.com/LambdaTest/hyperexecute-maestro-sample-test/blob/main/maestro-test/runTest_ios_realdevice.sh -``` - -2. Update your HyperExecute YAML file to enable the native reporting in HyperExecute using the generated JUnit XML files. - -```yaml title="hyperexecute.yaml" -report: true -partialReports: -- location: . -type: xml -frameworkName: junit -``` - -### 📘 Use Cases -#### Use Case 1: One Test per Task -If you're executing one test per task, a single `report.xml` will be generated per job. These individual reports can then be merged later for a consolidated result. - -#### Use Case 2: Multiple Tests on the same Task -In this case, the `report.xml` file gets overwritten after each test execution. This results in only the last test's results being preserved. To prevent overwriting, update your `testRunnerCommand` in the `hyperexecute.yaml` file to rename the report after each test: - -```yaml title="hyperexecute.yaml" -testRunnerCommand: ./maestro-test/runTest.sh $test && mv report.xml $test.xml -``` - -This ensures that each test result is saved with a unique name like test1.xml, test2.xml, etc. - -## Step 6: Execute your Test Suite -> **NOTE :** In case of MacOS, if you get a permission denied warning while executing CLI, simply run **`chmod u+x ./hyperexecute`** to allow permission. In case you get a security popup, allow it from your **System Preferences** → **Security & Privacy** → **General tab**. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE `} - -## Step 7: Monitor the Test Execution -Visit the [HyperExecute Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) and check your Job status. - -## Additional Information: Launching Pre-Installed Apps with Maestro -In some cases, you may want to test against a pre-installed application on the device (instead of uploading and installing a new APK/IPA). Maestro supports this by allowing you to specify the app’s package identifier (Android) or bundle identifier (iOS) in your test configuration. - -### Step 1: Identify the App ID (Package Name / Bundle ID) -#### For Android: - - Visit the app’s page on the Google Play Store. - - The id parameter in the URL is the package name. - - Example: For the Wikipedia app → `org.wikipedia`. - -#### For iOS: - - Identify the bundle identifier (e.g., com.apple.Preferences for Settings). - -### Step 2: Update Your HyperExecute Configuration -You can configure your YAML files to launch the pre-installed app instead of uploading a new one. - -```yaml title="hyperexecute.yaml" -...// -framework: -name: raw -args: -#highlight-next-line -appId: stock -``` - -and the launcher yaml file to tells maestro to use the pre-installed Wikipedia app. - -```yaml reference title="android-launch.yaml" -https://github.com/LambdaTest/hyperexecute-maestro-sample-test/blob/main/yaml/android/android-launch.yaml -``` - -### Step 3: Execute your Test Suite -> **NOTE :** In case of MacOS, if you get a permission denied warning while executing CLI, simply run **`chmod u+x ./hyperexecute`** to allow permission. In case you get a security popup, allow it from your **System Preferences** → **Security & Privacy** → **General tab**. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE `} - -The Wikipedia app will open directly on the device, and your Maestro test steps will execute against it. - -**Example: Wikipedia Search Flow** - -```yaml title="android-launch.yaml" -appId: org.wikipedia ----- -launchApp - -tapOn: "Search Wikipedia" -inputText: "Maestro framework" -pressKey: Enter -assertVisible: "Mobile UI testing" -``` - -**Explanation:** - -- **launchApp:** Opens the Wikipedia app. -- **tapOn:** "Search Wikipedia" → Focuses the search bar. -- **inputText:** "Maestro framework" → Enters the text. -- **pressKey:** Enter → Submits the search. -- **assertVisible:** "Mobile UI testing" → Validates results. - -### Best Practices -- Make sure the app is already installed on the device; otherwise, Maestro cannot launch it. -- The same approach works for iOS using the bundle identifier. -- You can also switch between multiple apps in a single flow by providing different appId values in separate steps. diff --git a/static/docs/hyperexecute-matrix-multiplexing-strategy.md b/static/docs/hyperexecute-matrix-multiplexing-strategy.md deleted file mode 100644 index a13a531c9..000000000 --- a/static/docs/hyperexecute-matrix-multiplexing-strategy.md +++ /dev/null @@ -1,88 +0,0 @@ -# Matrix Multiplexing Strategy - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -HyperExecute's Matrix Multiplexing Strategy allows you to execute test cases across a variety of combinations efficiently. This strategy is especially useful when dealing with browser and OS combinations or other parameters like files, tags, and more. - -By utilizing the Matrix Multiplexing Strategy, you can parallelize your testing process and achieve faster results. HyperExecute generates permutations and combinations using key-value pairs, launching virtual machines for each combination and running tests in parallel. - -## Configuration and Execution - -### Setting Up Matrix Multiplexing - -To initiate Matrix Multiplexing, define `key-value` pairs in the HyperExecute YAML file. Each pair represents a combination of parameters under consideration, such as browser versions, OS, files, folders, tags, features, scenarios, input values, etc. - -### Generating and Running Virtual Machines - -HyperExecute will generate and manage virtual machines for each key-value combination, injecting them as environment variables or command-line variables. The specified commands will then run for each combination. - -Consider a scenario where the matrix calculation results in 10 different possible combinations. In such a case, 10 different Test Execution nodes, known as `Tasks` will be spawned in parallel and all the tests will be run in parallel on the respective node. - -You can use a key as [browser], [files] or any other unique key name. - -``` yaml -matrix: -files: ["Test1","Test2","Test3"] -testSuites: -- mvn test -Dtest=$files -``` - -In the YAML file shown above, **files** is a user-generated key. **Files** and its associated value(s) are opaque to HyperExecute. You can also name it as **Files1**, **FileNames**, etc., as long as the Key is unique in that YAML file. - -## When to opt for Matrix Strategy - -Matrix multiplexing strategy is ideally used when you have to run specific test scenarios across different input (or value) combinations. For example, this approach would be useful if you have to run "scenario1" on "N" version(s) of Chrome and Firefox browsers (that are installed on the Windows 10 platform). - -Here, ‘N’ could be browser version number(s) that are common between the two web browsers e.g. 68.0, 69.0, 89.0, etc. You can also leverage matrix multiplexing strategy to run scenarios based on different tags, files, features, and more. You can specify required key:value pairs in the HyperExecute YAML file or you can specify the same in the source code. - -In matrix multiplexing strategy, *`key`:`value`* pairs mentioned in the source code override the same `key`:`value` pairs mentioned in the HyperExecute YAML file. - -Let’s modify the above YAML file to run the scenarios on "latest" version of the Firefox browser: - -``` yaml -#runson defines the OS of your test execution node. -runson: ${matrix.os} - -matrix: -os: ["win", "linux"] -version: ["latest", "beta", "dev"] -browser: ["Chrome", "Firefox"] -files: ["@File1","@File2"] - -``` - -A combination of test scenarios is generated using the following formula: - -> **Total Test Scenarios** = *[Entries in os List] x [Entries in browser version List] x [Entries in browser List] x [Entries in Files List]* - -Let’s take a case where the test scenarios are implemented in Java. Files File1.java and File2. java contains one test method each. In such a scenario, the total number of test combinations would be (1 x 1 x 2 x 2) → 4. Here is the list of the total test scenarios that would be run on HyperExecute (based on the YAML file shown earlier): - -| Test Number | FileName | OS | Browser | Browser Version | -|---|---|---|---|---| -| 1 | @File1 (Test Method_1) | Win (Windows 11) | Chrome | latest | -| 2 | @File1 (Test Method_1) | Win (Windows 11) | Firefox | latest | -| 3 | @File2 (Test Method_2) | Win (Windows 11) | Chrome | latest | -| 4 | @File2 (Test Method_2) | Win (Windows 11) | Firefox | latest | - -## Exclusion in Matrix Strategy -Assuming that you created a matrix but there is one specific combination that you don't want the system to consider. This can happen for instance if you have a combination of Safari and Windows which won't be a valid combination, in such cases you can exclude such a combination using the `exclusionMatrix` parameter as shown below. - -```yaml -#runson defines the OS of your test execution node. -runson: ${matrix.os} - -matrix: -# Browser version(s) separated by Comma -version: ["latest"] -# OS name sepataed by Coma -os: ["win", "mac", "linux"] -# Browser name separated by Comma -browser: ["Chrome", "Firefox", "Safari"] -# Test Files separated by Comma -files: ["@File1","@File2","@File3","@File4"] - -exclusionMatrix: -- os: [win] -browser: ["Safari"] -files: ["@File2","@File3"] -``` diff --git a/static/docs/hyperexecute-mcp-server-release-notes-1-0-0.md b/static/docs/hyperexecute-mcp-server-release-notes-1-0-0.md deleted file mode 100644 index 0909bae83..000000000 --- a/static/docs/hyperexecute-mcp-server-release-notes-1-0-0.md +++ /dev/null @@ -1,124 +0,0 @@ -# MCP TestMu AI Release Notes - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Version 1.0.0 - Major Release - -### 🎉 What's New - -#### Rebranding & Package Updates - -- **MCP Hyperexecute** is now **MCP TestMu AI** to better reflect our expanded capabilities -- Updated package name to `mcp-lambdatest` on NPM -- Stable 1.0.0 release with comprehensive feature set - -#### Transport Support - -- **STDIO Transport**: Available via NPM package for local installations -- **StreamableHTTP Transport**: Replaces deprecated SSE transport for remote connections -- Enhanced connection reliability and performance - -#### Enhanced Migration Tools - -- Expanded migration capabilities for easier transition to TestMu AI platform -- Improved app upload functionality for mobile testing -- Resolved template loading and asset management for migration tools - -#### Security & Reliability Improvements - -- Enhanced authentication mechanisms -- Improved credential handling and validation -- Better security practices for remote connections -- Fixed MCP resource loading issues for capabilities mapping and hooks executor -- Updated webpack configuration to properly copy JSON assets during build process - -### 🔧 Connection Configuration - -#### ⚠️ Breaking Changes - -The previous SSE-based connection method has been **removed**: - -``` -❌ OLD (No longer supported): -http://mcp.lambdatest.com/hyperexecute?&username=[your-lt-username]&accessKey=[your-lt-access-key] -``` - -#### ✅ Connection Methods - -**Option 1: Remote MCP Server (StreamableHTTP)** - -```json -{ -"mcpServers": { -"mcp-lambdatest-remote": { -"disabled": false, -"timeout": 60, -"command": "npx", -"args": [ -"mcp-remote@latest", -"https://mcp.lambdatest.com/mcp", -"--header", -"username:", -"--header", -"accessKey:" -], -"transportType": "streamableHTTP" -} -} -} -``` - -**Option 2: Local STDIO Server (NPM Package)** - -```json -{ -"mcpServers": { -"mcp-lambdatest-stdio": { -"disabled": false, -"timeout": 100, -"command": "npx", -"args": ["-y", "mcp-lambdatest", "--transport=stdio"], -"env": { -"LT_USERNAME": "", -"LT_ACCESS_KEY": "" -}, -"transportType": "stdio" -} -} -} -``` - -### 🚀 Getting Started - -1. **Choose your connection method** (Remote or Local STDIO) -2. **Update your configuration** with the new format above -3. **Replace credentials** with your TestMu AI username and access key -4. **Verify connection** by asking your AI assistant to list available MCP tools -5. **Start using** TestMu AI tools directly within your AI assistant - -### 📋 Available Tools - -#### HyperExecute Tools - -- **generateHyperExecuteYAML**: Generate YAML configurations for your test projects -- **answerHyperExecuteQuery**: Get answers from HyperExecute documentation -- **getHyperExecuteJobInfo**: Fetch detailed job information -- **getHyperExecuteJobSessions**: Retrieve job session details - -#### Migration Tools - -- **upload_app**: Upload mobile applications for testing - -### 🔗 Compatibility - -This release is compatible with: - -- Cline (Claude Dev) -- Any MCP-compatible AI assistant -- Both local and remote deployment scenarios - -### 📞 Support - -For questions or issues with the connection methods, please refer to our documentation or contact TestMu AI support. - -**Note**: Make sure to update your configurations to use the new connection methods as the old SSE-based approach is no longer supported. diff --git a/static/docs/hyperexecute-mcp-server.md b/static/docs/hyperexecute-mcp-server.md deleted file mode 100644 index 396046624..000000000 --- a/static/docs/hyperexecute-mcp-server.md +++ /dev/null @@ -1,39 +0,0 @@ -# Getting Started with HyperExecute MCP Tool - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -HyperExecute MCP Tool is part of the [TestMu AI MCP Server](/support/docs/testmu-mcp-server/). It lets you set up HyperExecute test jobs, generate YAML configurations, and monitor runs directly from your IDE using natural language prompts. - -## Key Benefits - -Here are the key benefits of using the HyperExecute tool: - -- **Automated Setup:** Reduce configuration time from hours to seconds -- **Smart Analysis:** Automatically identify project types, frameworks, and test structures -- **Dynamic Commands:** Generate precise test commands tailored to your project -- **Auto-Configuration:** Create accurate YAML configuration files without manual intervention -- **Real-Time Monitoring:** Track test executions with immediate updates -- **Efficient Debugging:** Inspect test results with comprehensive recordings -- **Instant Documentation:** Access relevant documentation through integrated AI agents - -## Capabilities - -The HyperExecute MCP Tool provides five capabilities: - -1. **Test Runner Command Generation:** Analyzes your project structure and generates appropriate test commands. -2. **HyperExecute YAML Generation:** Creates a HyperExecute YAML configuration file based on your project analysis. -3. **HyperExecute Documentation Lookup:** Answers your queries by fetching information from the HyperExecute documentation. -4. **HyperExecute Job Info:** Retrieves information about a specific test job. -5. **HyperExecute Sessions:** Fetches session details for a given test run. - -## How to Invoke - -Once your MCP client is connected, use natural language to interact with the HyperExecute tool. Example prompts: - -- "Analyze this project and create a HyperExecute YAML file." -- "What does `failFast` mean in HyperExecute?" -- "Get the status of HyperExecute job ``." -- "Show me session details for the last failed run." -- "Generate the test runner command for this Selenium project." - -Your AI client will route the request to the appropriate capability and return the results inline. diff --git a/static/docs/hyperexecute-native-testng-report.md b/static/docs/hyperexecute-native-testng-report.md deleted file mode 100644 index c23b50215..000000000 --- a/static/docs/hyperexecute-native-testng-report.md +++ /dev/null @@ -1,36 +0,0 @@ -# Native TestNG Reports on HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The Native TestNG Reports offer a standardized and easily accessible summary of information extracted from raw Extent reports per Virtual Machine (VM) at the end of a HyperExecute job. - -## Implementation Steps - -Follow these steps to enable Extent Native Reports for your HyperExecute job: - -### 1. Generate JSON Reports - -You need to ensure that the `emailable-report.html` file should be generated within your test-suite - -### 2. Update HyperExecute YAML Configuration - -In the HyperExecute YAML configuration, add the following section to instruct the HyperExecute systems to generate Extent Native Reports: - -```yaml -report: true -partialReports: -type: json -location: -frameworkName: testng-native -``` - -- Set `report: true` to enable report generation. -- Specify the `type` as `json` in the `partialReports` section. -- Define the `location` where the reports will be stored. -- Provide a meaningful `frameworkName` for the Native TestNG Reports. - -x - -## Conclusion - -By following these steps, your HyperExecute job will generate Native TestNG Reports, providing a consolidated HTML report generated by TestNG framework. This enhancement allows customers to access comprehensive and standardized reports conveniently at the conclusion of their HyperExecute jobs. diff --git a/static/docs/hyperexecute-org-product-preferences.md b/static/docs/hyperexecute-org-product-preferences.md deleted file mode 100644 index 6d420b506..000000000 --- a/static/docs/hyperexecute-org-product-preferences.md +++ /dev/null @@ -1,20 +0,0 @@ -# How to Access and Change Org Product Preferences in HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -In this detailed documentation, learn how you can access and change organization product preferences as per your requirements. - -Now, to access your organization product preferences, follow the below mentioned steps: - -**Step 1:** Go to the [TestMu AI Dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard). - -**Step 2:** Click on the **Settings** button. - -**Step 3:** From the **left sidebar**, select **Settings**. - -**Step 4:** Go to the **Org Product Preferences** section. - -**Step 5:** Select HyperExecute and you will be presented with two options - **Auto-mute tests** and **Test Viewing Preferences**. - -**Session Expiry** -To configure automatic session expiry for your organization, navigate to **Organization Settings → Security → Session Expiry**. You can set the duration for session expiration from predefined options (default: 30 days). Admins can also allow team members to customize their own session expiry settings. Learn more in [Settings and Security](/support/docs/settings-and-security/). diff --git a/static/docs/hyperexecute-performance-testing.md b/static/docs/hyperexecute-performance-testing.md deleted file mode 100644 index d400d0303..000000000 --- a/static/docs/hyperexecute-performance-testing.md +++ /dev/null @@ -1,25 +0,0 @@ -# Performance Testing with HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -**Imagine launching your dream app...only to see it crash under the weight of excited users !!** - -Your team devoted months to meticulously crafting an app or a website that is finally ready for the world. Your users are hyped, anticipation is boiling, and you hit the **launch** button. 🚀 - -But instead of smooth sailing, the app stumbles under the surge of eager users. And now they are left with only frustrating delays, error messages, and app crash notifications. It might sound familiar and scary at the same time, but unfortunately, it's a common scenario for apps without the armor of **performance testing**. - -When you build an application, before launching it on the market, you need to evaluate how it responds under pressure, ensuring it stays smooth and responsive even when bombarded with users. This crucial step simulates real-world user traffic, exposing weaknesses before they unleash chaos on your launch day. - -Performance Testing is a crucial aspect of software development aimed at evaluating the speed, responsiveness, and stability of applications under various load conditions. - -Think of it like putting your app through a rigorous training regime. We throw virtual users at it, mimicking a **Black Friday** rush or a product launch state. By monitoring its response, you can identify potential bottlenecks and weak spots before it can face real-world problems. - -## Benefits of Using HyperExecute for Performance Testing - -HyperExecute offers a comprehensive platform for performance testing, providing users with the following benefits: - -- **Real-Time Monitoring and Reporting:** HyperExecute provides real-time monitoring and reporting capabilities, enabling users to track performance metrics, identify bottlenecks, and analyze results as tests are running. This empowers users to make informed decisions and take immediate action to optimize performance. - -- **Customizable Load Distribution:** HyperExecute enables users to configure load distribution, specifying different data center names and percentages of load to be generated from them. This flexibility ensures efficient utilization of resources and accurate simulation of real-world scenarios. - -- **Robust Security and Compliance:** HyperExecute prioritizes security, implementing robust measures to protect user data and test plans. Furthermore, it ensures compliance with relevant data protection regulations, such as GDPR, safeguarding user privacy and data integrity. diff --git a/static/docs/hyperexecute-platform-faqs.md b/static/docs/hyperexecute-platform-faqs.md deleted file mode 100644 index afab1157e..000000000 --- a/static/docs/hyperexecute-platform-faqs.md +++ /dev/null @@ -1,66 +0,0 @@ -# HyperExecute Platform FAQs - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -#### 1. **What OS does HyperExecute provide to execute tests?** - - HyperExecute provides the latest Windows, MacOS and Linux based virtual machines to run your tests. HyperExecute supports multiple versions of Windows, Linux and MacOS. - -#### 2. **What browsers are available on HyperExecute to run cross browsers tests?** - - HyperExecute provides all the versions of Chrome, Firefox, Edge, Opera, and Safari (Mac only) available to run on the latest windows, MacOS and linux versions. - -#### 3. What is the current configuration of the Virtual Machines? - -Our Virtual Machines are currently configured with **4vCPU** and **16GB RAM**. - -This will result in improved availability and a decrease in preemptions. Furthermore, faster IOPs are observed on these machines after conducting benchmarking tests. - -#### 4. **Is HyperExecute available as an on-prem tool?** - - Yes, HyperExecute is available as a dedicated cloud and on-prem mode option on Azure, AWS and GCP. - -#### 5. **What analytics does HyperExecute provide for test execution?** - - HyperExecute provides fine-grained analytics over test steps, jobs, browser trends, consistently failing tests and error categorizations, and a lot more in the analytics dashboard. In addition to this, it provides a detailed downloadable _html_ report for every job triggered. This report can be downloaded by the CLI after the job is completed or via the reports API. - -#### 6. **Which logs does HyperExecute provide for Selenium tests?** - - Video recording, screenshots, command logs, network logs, selenium node logs, browser logs and a lot more are available for every single end-to-end test run over the platform. - -#### 7. **Can I access the file system while running my tests?** - - Yes, with HyperExecute you get access to all the underlying virtual machines to completely customize the testing environment. Generating or reading files for your test case is very convenient with HyperExecute. - -#### 8. **Do I have access to the underlying OS while running tests?** - - Yes, with HyperExecute, you get access to all the underlying virtual machines to completely customize the testing environment. Some good use cases of this are changing registry settings, downloading files, starting processes and more. - -#### 9. **Can I customize the testing environment?** - - Yes, with HyperExecute, you can completely customize the test environment using pre and post steps in the YAML. For enterprise accounts, HyperExecute provides the ability to define custom virtual machine images for all the OS for deeper level customizations to the testing environments. Enterprise customers can create their own customized virtual machine images over HyperExecute base images as per their use case. - -#### 10. **Can I create custom virtual machine images for my testing?** - - Yes. For enterprise accounts, HyperExecute provides the ability to define custom virtual machine images for every OS to allow deeper level customizations to the testing environments. Enterprise customers can create their own customized virtual machine images over HyperExecute base images as per their use cases. - -#### 11. **Can I integrate my reporting tool with HyperExecute?** - - Yes. It’s very easy to integrate HyperExecute with your reporting tool by hitting their APIs in the post-steps using the YAML. HyperExecute is also bringing more integrations with popular reporting tools. You can request new integrations by emailing us at hyperexecute@lambdatest.com. - -#### 12. **Do HyperExecute VMs have basic command line tools installed?** - - HyperExecute virtual machines come preinstalled with a variety of open source utilities and language runtimes to ensure that they are ready to use the testing environment. - - - For MacOS: - https://github.com/actions/runner-images/blob/main/images/macos/macos-15-Readme.md - - - For Windows: - https://github.com/actions/runner-images/blob/main/images/windows/Windows2025-Readme.md - - - For Linux: - https://github.com/actions/runner-images/blob/main/images/ubuntu/Ubuntu2404-Readme.md - -We use Github’s open source images for Github actions and pre-installed softwares. Visit the links to view the list of pre-installed softwares on our VMs. - ->If you have more questions, then just give us a shout and we will answer them for you. diff --git a/static/docs/hyperexecute-pre-steps-error.md b/static/docs/hyperexecute-pre-steps-error.md deleted file mode 100644 index 9496bdc31..000000000 --- a/static/docs/hyperexecute-pre-steps-error.md +++ /dev/null @@ -1,105 +0,0 @@ -# How to Debug your Pre Steps Errors - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This document will help you debug your failed job if you are facing any errors in the pre-command stage. - -## Lambda Error - -While executing a playwright project in HyperExecute, you may encounter a Lambda Error. - -To resolve this issue, you need to pass the below mentioned command in the pre step in your YAML file - -```bash -pre: -- npx install playwright -``` - -## .Net Build Issue in C# - -If you encounter a similar error while running the command `dotnet build` in pre steps, as mentioned below, it means that you are not using a **core dotnet** version but instead a **framework** version. - -```bash -error : MSB4803: The task "ResolveComReference" is not supported on the .NET Core version of MSBuild -``` - -We need to build it by invoking **MSBuild.exe** from the Visual Studio directory inside our VMs. Please target MSBuild.exe of Visual Studio 19. Replace dotnet build with the below command - -`C:\PROGRA~2\Micros~1\2019\Community\MSBuild\Current\Bin\MSBuild.exe /p:Configuration=Release /p:TargetFramework=net481 /t:restore` - -The target Framework can be replaced with dotnet version targeted in the user’s repository. **``/p:TargetFramework=net70302``** - -## SSL Certificate Error - -This error is generated while installing the dependencies in your project suite. - -#### Error in Python - -You can encounter a similar error while installing dependencies - -```bash -There was a problem confirming the ssl certificate: -HTTPSConnectionPool(host='http://pypi.org ', port=443): -Max retries exceeded with url: /simple/selenium/ -(Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] -certificate verify failed: self signed certificate in certificate chain -(_ssl.c:1131)'))) - skipping -``` - -To resolve the above issue, you are required to pass the following command - -```bash -pip install -r requirements.txt --cache-dir -pip_cache --trusted-host http://pypi.org --trusted-host http://pypi.python.org --trusted-host files.pythonhosted.org -pip setuptools -``` - -#### Error in Maven - -You can encounter a similar error while running Maven Project as well, Now to resolve this issue for **maven**, pass the following command - -```bash --Dmaven.wagon.http.ssl.insecure=true -Dmaven.wagon.http.ssl.allowall=true -Dmaven.wagon.http.ssl.ignore.validity.dates=true -``` - -## Registry Error - -If you incorporate any npm registry related error in the **Pre** stage, then pass the following command in your YAML file in the pre command before npm install: - -```bash -pre: -- npm config set registry -``` - -## Incompatible Node Version - -If you encounter an error as mentioned below, it implies that you are using node version **below 15**, so you need to upgrade to **>=15.0** using runtime command. - -```bash -SyntaxError: Unexpected token '||=' -``` - -To resolve this, simply pass the below mentioned command in your YAML file - -```bash -runtime: -language: node -version: "15" -``` - -## Java version Conflict - -This error incorporates that the version of Java that you are using in your local machine or in pom.xml file is not matching with the one used in the HyperExecute. - -```bash -[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.7.0:compile (default-compile) on project My-Project: Fatal error compiling: invalid target release: 1.10 -> [Help 1] -org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.7.0:compile (default-compile) on project My-Project: Fatal error compiling -``` - -To rectify this, you need to simply pass the **runtime** flag with the correct java version in your YAML file. - -```bash -runtime: -language: java -version: "11" -``` diff --git a/static/docs/hyperexecute-pricing-faqs.md b/static/docs/hyperexecute-pricing-faqs.md deleted file mode 100644 index 32e023d0b..000000000 --- a/static/docs/hyperexecute-pricing-faqs.md +++ /dev/null @@ -1,21 +0,0 @@ -# HyperExecute Pricing FAQs - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -#### 1. **Is there any limit on parallel testing on HyperExecute?** - - No, HyperExecute can provide virtually unlimited nodes to you for massive parallel testing as it is backed and spanned across multiple clouds. We have users distributing tests across thousands of nodes. - -#### 2. **How does the licensing for HyperExecute work?** - - Each parallel test execution consumes a single HyperExecute license. For example, if you want to run 50 tests in parallel, you need 50 HyperExecute licenses. If you have 10 HyperExecute licenses and trigger 50 tests, only 10 will run in parallel and the rest 40 will be queued by the platform. Once every test finishes the execution, it will be dequeued and moved to the running state. - -#### 3. **How many parallel tests can I run on a single HyperExecute license?** - - Each parallel test execution consumes a single HyperExecute license. HyperExecute puts no capping on the number of parallel tests that can run simultaneously. We have users running thousands of tests in parallel. - -#### 4. **Can I use the standard TestMu AI automation platform with the HyperExecute license?** - - Each HyperExecute license provides access to standard automation platforms and features. This makes transitioning to HyperExecute easier and makes sure that the best cross-browser platform features are still available to users purchasing a HyperExecute license. - ->If you have more questions, then just give us a shout and we will answer them for you. diff --git a/static/docs/hyperexecute-prioritize-tests.md b/static/docs/hyperexecute-prioritize-tests.md deleted file mode 100644 index 4cc56b20e..000000000 --- a/static/docs/hyperexecute-prioritize-tests.md +++ /dev/null @@ -1,26 +0,0 @@ -# Prioritize Your Tests - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -HyperExecute's job prioritization feature empowers you to ensure that your most crucial tests are executed with precision and timeliness. This feature not only enhances the order of execution but also enables effortless searchability within your testing repository. - -## `jobLabel` - -The `jobLabel` YAML key is used to add tags or labels to jobs. In addition to prioritizing your jobs, you can also use the job labels to search for a job using the relevant label or tag that is assigned to it. To learn more about how to use this feature with the HyperExecute YAML, go through [this section](/support/docs/deep-dive-into-hyperexecute-yaml/#23-joblabel). - -## Prioritize your Jobs using `jobLabel` - -You can prioritize your jobs by assigning the job labels **low**, **medium** or **high** to your jobs. The jobs with the priority **high** will be executed first, followed by **medium** priority jobs with **low** priority jobs at the end. - -```yaml -jobLabel: ['high', 'medium', 'low'] -``` - -- By default, all of your jobs are assigned **medium** priority. -- The priority keys are not case sensitive. -- You can use the priority keys along with your regular job labels too. - -## Visualizing Priority Levels -As you can see in the diagram below, we have executed 6 jobs; one job is designated as **high** priority, three are assigned as **medium** priority, while the remaining two hold **lower** priority labels. HyperExecute is configured to adhere to this prioritization framework during execution. As a result, the high-priority job takes precedence and is processed first, followed sequentially by the medium-priority job and subsequently the low-priority job, even though the higher-priority job is executed later. This prioritized execution sequence ensures efficient task management aligned with the specified priority levels. - ->If you have more questions, then just give us a shout and we will answer them for you. diff --git a/static/docs/hyperexecute-private-cloud-benefits.md b/static/docs/hyperexecute-private-cloud-benefits.md deleted file mode 100644 index 3cd671375..000000000 --- a/static/docs/hyperexecute-private-cloud-benefits.md +++ /dev/null @@ -1,36 +0,0 @@ -# Benefits of HyperExecute Private Cloud - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -HyperExecute Private Cloud offers you with a host of features to optimize your testing process, just like our Standard Cloud. However, HyperExecute Private Cloud goes one step further to provide you with the best testing experience. - -It is designed to meet the needs and wants of an enterprise. The flexibility that you get with HyperExecute Private Cloud is its biggest strength. HyperExecute Private Cloud can be designed to fit your enterprise's testing process, and help you automate it with any changes that you want. - -You can go through the table below to understand exactly why HyperExecute Private Cloud is a must-have for your organization. - -| Feature | HyperExecute Standard Cloud | HyperExecute Private Cloud | -| --------| --------------------------- | -------------------------- | -| Unlimited end to end automation testing over multiple OS | Linux, Windows, and macOS | Linux and Windows | -| Smart test orchestration features including [Auto-splitting of tests](/support/docs/hyperexecute-auto-split-strategy/), automatic reordering of tests for [Fail Fast](/support/docs/hyperexecute-failfast/), and more. | ✅ | ✅ | -| [YAML driven](/support/docs/deep-dive-into-hyperexecute-yaml/) Test orchestration driven. | ✅ | ✅ | -| [Powerful CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) to trigger tests. | ✅ | ✅ | -| Real time streaming of Terminal Logs | ✅ | ✅ | -| Comprehensive Test Artifacts management | ✅ | ✅ | -| Automated report generation for all testing frameworks. | ✅ | ✅ | -| User Workflow Management | ✅ | ✅ | -| Intelligent Test Analytics | ✅ | ❌ | -| Integration with major [CI/CD platforms and products](/support/docs/integration-with-hyperexecute/) | ✅ | ✅ | -| Includes all features from TestMu AI's Web + Mobile Automation Plan, along with exclusive HyperExecute benefits | ✅ | ❌ | -| 24/7 dedicated customer support for uninterrupted assistance. | ✅ | ✅ | -| Support for custom testing frameworks | ❌ | ✅ | -| Advanced data retention rules | ❌ | ✅ | -| Role-Based Access Control (RBAC) support for granular user permissions and security management. | ✅ | ✅ | -| SSO Support | ✅ | ✅ | -| Custom test execution environments that are completely managed by TestMu AI | ❌ | ✅ | -| Access to standard HyperExecute cloud for additional redundancy | NA | ✅ | -| Enhanced Performance and Boosted Security | ❌ | ✅ | -| Data isolation and region-specific deployment capabilities for enhanced security and compliance. | ❌ | ✅ | -| Increased Flexibility | ❌ | ✅ | -| Whitelabled Dashboards | ❌ | ✅ | - -> If you need more information about the HyperExecute Private Cloud, reach out to us at [hyperexecute-privatecloudsupport@testmuai.com](mailto:hyperexecute-privatecloudsupport@testmuai.com). Happy testing! diff --git a/static/docs/hyperexecute-private-cloud-on-jumphost.md b/static/docs/hyperexecute-private-cloud-on-jumphost.md deleted file mode 100644 index fc645f3bc..000000000 --- a/static/docs/hyperexecute-private-cloud-on-jumphost.md +++ /dev/null @@ -1,21 +0,0 @@ -# HyperExecute Private Cloud on JumpHost - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Congratulations on getting the HyperExecute Private setup deployed on your Azure Cloud! You can now trigger your test cases and view your dashboard from the JumpHost virtual machine. To understand how to do that, go through the page below. - -## Running HyperExecute via a JumpHost VM - -**Step 1:** Log in to your Microsoft Azure account and go to your [homepage](https://portal.azure.com/#home). - -**Step 2:** Go to the **search bar** on the top of your screen and search for your managed applications. - -**Step 3:** You will be redirected to your managed applications where you can see your installed HyperExecute Private Cloud application. Select the HyperExecute application that you have installed recently. - -**Step 4:** You will be redirected to a page where you can see all of the resources that are a part of the selected **Managed Resource Group**. Search for a Virtual Machine called `hyex-jump` in this list and open it. - -**Step 5:** You can now see the VM details and can use the shown public IP to connect to the virtual machine using the shared credentials over mail. - -To ensure that you have complete control over your HyperExecute application and your browsers, the virtual machine is completely customizable. You can make the modifications that you require and test all of your applications. Happy testing! - -> If you have more questions regarding HyperExecute Private Cloud, reach out to us at [hyperexecute-privatecloudsupport@testmuai.com](mailto:hyperexecute-privatecloudsupport@testmuai.com). diff --git a/static/docs/hyperexecute-private-cloud-setup-with-aws.md b/static/docs/hyperexecute-private-cloud-setup-with-aws.md deleted file mode 100644 index 1f5ce6396..000000000 --- a/static/docs/hyperexecute-private-cloud-setup-with-aws.md +++ /dev/null @@ -1,141 +0,0 @@ -# How to setup HyperExecute Private Cloud with AWS - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Prerequisites - -Before deploying the CloudFormation template for HyperExecute through the AWS Marketplace, ensure that the following prerequisites are met: - -1. **Region with min 3 availability zones**: Make sure you are running the CloudFormation in a region that supports a minimum of 3 availability zones. The currently supported regions are `us-east-1`, `us-east-2`, `us-west-2`, `eu-central-1`, `ap-south-1`, and `ap-southeast-2`. - -2. **Elastic IPs (EIPs)**: HyperExecute requires 3 Elastic IPs for each availability zone (AZ). Ensure that you have sufficient quota for EIPs in your AWS account. You can refer to the [EIP Limits](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/elastic-ip-addresses-eip.html#using-instance-addressing-limit) to check your quota. - -3. **EC2 quota limits**: The CloudFormation template will create EC2 instances as part of a Managed NodeGroup, with a minimum requirement of 5 nodes, and an Auto Scaling Group with a maximum size based on your concurrency license. Make sure to increase your [EC2 quota](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-resource-limits.html) limits accordingly. - -## Steps to Deploy - -To deploy the CloudFormation template on AWS, follow these steps: - -**Step 1:** Download the CloudFormation template named **hyperexecute_deployment.yaml** from the AWS Marketplace. - -**Step 2:** Navigate to the AWS CloudFormation console and choose **Create Stack** . - -**Step 3:** On the **Create Stack** page, select the option **Template is ready** and click on the **Upload a template file" option**. - -**Step 4:** Browse for the main template file **hyperexecute_deployment.yaml** and click **Next**. - -**Step 5:** On the **Specify Stack details** page, provide a name for the stack (deployment) and provide the necessary configuration details. Refer to the configuration parameters details sections for guidance. - -**Step 6:** On the next page, under the **Configure stack** option, change the **Stack failure options** to preserve the successfully provisioned resources. This helps in preserving the created resources and troubleshooting issues in case of a stack failure. - -**Step 7:** Click **Next** to review all the details, and then initiate the launch to deploy the HyperExecute CloudFormation stack. - -By following these steps, you can deploy the HyperExecute CloudFormation template from the AWS Marketplace and set up the required infrastructure for your testing needs. - -## Configuration Parameters - -Please provide the following configuration details based on the type of installation you want to perform: - -### User Details - -In the User Details section, provide the following details as required: - -| Parameter | Description | -|--------------|-------------------------------------------------------------------------------------------------------| -| EmailID | Your valid email ID to set up a TestMu AI account. If you already have a TestMu AI Account, you can specify that email ID. | -| ManagedAppName | Provide a unique ManagedAppName to be used for the application environment in the cluster. It must be between 5 to 20 characters long and can only contain lowercase letters. | - -### VPC Configuration - -In the VPC Configuration, provide the following details as required: - -| Parameter | Description | -|------------------|--------------------------------------------------------------------------------------------------------------------------| -| VpcCidr | The VPC CIDR block. Must be a valid IP CIDR range of the form x.x.x.x/x. Default Value: 10.10.0.0/16 | -| AvailabilityZones | List of Availability Zones to use for the subnets in the VPC. You must select three Availability Zones in the specified order. | - -### EKS Configuration - -In the EKS Configuration, provide the following details as required: - -| Parameter | Description | -|----------------------------|---------------------------------------------------------------------------------------------------------------| -| ManagedNodeGroupInstanceType | Specify the instance types for a node group. Default Value: m5.xlarge | -| SpotNodes | Enable Spot Nodes for the EKS cluster. Default Value: No | -| IngressLBType | Type of Ingress Load Balancer to create for the EKS Cluster. Private Load Balancer will only be accessible within the VPC. | - -### VMS Autoscaling Group - -In the VMS Autoscaling Group, provide the following details as required: - -| Parameter | Description | -|---------------|------------------------------------------------------------------------------------------------------------| -| VMInstanceType | Specify the instance types for an EC2 Auto Scaling Group. | -| SpotVMs | Enable Spot EC2 instances for the Auto Scaling Group. Default Value: No | -| KeyPairName | Name of an existing EC2 key pair for SSH access to the instances in the Auto Scaling Group. | - -### Other Configuration - -In the Other Configuration, provide the following details as required: - -| Parameter | Description | -|--------------------|---------------------------------------------------------------------------------------------------------------| -| RedisClusterNodeType | The ElasticCache node type. Default value: cache.t3.small | -| DebugMode | Enable debug mode for your environment to run preview images in your EKS cluster. Default Value: No | -| WindowsJumpHost | Enable a Windows jump host for accessing your Windows machines in your Auto Scaling Group. Default Value: No | - -## Accessing HyperExecute after deployment - -Once the CloudFormation template is successfully deployed, you can access HyperExecute and perform the following steps: - -**Step 1:** Login into the EC2 instance named "**HyperExecute-Admin-Bastion**" using either SSM (Session Manager) or the SSH key provided in the Configuration Parameters. Note that SSH access is allowed by default within the VPC. - -**Step 2:** Log in to the AWS environment and access the Kubernetes cluster by running the following command on AWS CLI: - -```bash -aws eks --region update-kubeconfig --name -``` - - Replace `` with the appropriate region and `` with the name of your cluster. - -**Step 3:** Execute the command: - -```bash -kubectl get namespaces -``` - - This command will provide a list of namespaces created. - -**Step 4:** Execute the command: - -```bash -kubectl get po -n hyperexecute -``` - - This command will provide a list of pods created under the `hyperexecute` namespace. - -**Step 5:** Access the Private Cloud deployment using the provided [link](https://dashboard.hyperexecute.cloud/). - -**Step 6:** If the emailID you used to setup this deployment is not an existing user on TestMu AI then check your Email, there should be a email titled `Create Password - LambdaTest`, generate a new password using the link provided in the email. - -**Step 7:** Login using your email and password and run jobs using the HyperExecute CLI. For further details about HyperExecute, refer to the [documentation](/support/docs/getting-started-with-hyperexecute/). - -## Cleanup the Environment - -If you wish to clean up the resources created during the deployment, follow these steps: - -**Step 1 :** Ensure that the objects inside the S3 bucket created by the CloudFormation template with the name `${AWS::StackName}-${AWS::AccountId}-hyperexecute-bucket` are deleted before the bucket can be deleted. - -**Step 2 :** Delete the images in the ECR (Elastic Container Registry) repository with the name `${ManagedAppName}-hyperexecute-private-cloud` before deleting the repository. - -**Step 3 :** Delete the root stack from CloudFormation. This will delete all the nested stacks and their resources. - -**Step 4 :** CloudWatch log groups are not deleted by default. Manually navigate to CloudWatch and delete the log groups starting with the following regex patterns: - - `/aws/lambda/*-ClusterCreatorFunction-*` - - `/aws/lambda/*-ClusterOIDCURLFunction-*` - - `/aws/lambda/*-HyePrivateCloudFunction-*` - - `/aws/containerinsights/${ManagedAppName}-hyperexecute-eks-cluster/*` - -**Step 5 :** If you have used the secrets feature in HyperExecute, the secrets with the regex pattern `hyperexecute-secrets-*` will not be deleted. Manually navigate to AWS Secret Manager and delete the secrets. - -Make sure to follow these cleanup steps to remove the resources associated with the HyperExecute deployment. diff --git a/static/docs/hyperexecute-private-cloud-setup-with-azure.md b/static/docs/hyperexecute-private-cloud-setup-with-azure.md deleted file mode 100644 index e2373bfba..000000000 --- a/static/docs/hyperexecute-private-cloud-setup-with-azure.md +++ /dev/null @@ -1,62 +0,0 @@ -# HyperExecute Private Cloud Setup with Azure - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Follow the steps below to understand how to setup **HyperExecute Private Cloud** via the [Azure Marketplace](https://azuremarketplace.microsoft.com/en-us/marketplace/apps/lambdatestinc1584019832435.hyperexeonprem?tab=Overview). - -## Prerequisites - -1. **Resource Provider :** The following resource providers must be registered to install HyperExecute successfully. If you do not have them registered, follow the steps on this [page](https://learn.microsoft.com/en-us/azure/azure-resource-manager/management/resource-providers-and-types); `Microsoft.Compute`, `Microsoft.OperationsManagement` - -2. **CPU Quotas :** In order to successfully deploy HyperExecute, we need to increase VM-family vCPU quotas. If you want to increase the quotas of your VM-family vCPU, follow the steps on this [page](https://learn.microsoft.com/en-us/azure/quotas/per-vm-quota-requests). - - - Standard DSv3 Family vCPUs. vCPU core count - 80 - - Standard FSv2 Family vCPUs. vCPU core count - 8 * number of parallel licenses. - -3. **Dedicated Resource Group.** - -4. A user with Admin Access who would be running the ARM script as certain resources like resource group, UAMI and related operations would require elevated access for creation. - -## IP Ranges -By default the setup creates its own network with IP range `10.26.0.0/16`. if in case the customer needs customised environment than minimum 2 subnets are required to accommodate the resources. - -- /24 subnet for test execution machines -- /22 subnet for Kubernetes - -> **NOTE:** These requirements vary if the licences are increased more than 100. - -## Steps to Setup HyperExecute Private Cloud - -**Step 1:** Sign in to your Azure Marketplace account. Alternatively, you can create a new account if you don’t already have one. - -**Step 2:** You can access the HyperExecute Private Cloud in two ways, by searching for it on Azure Marketplace, or by directly accessing the [HyperExecute’s Azure Marketplace](https://azuremarketplace.microsoft.com/en-us/marketplace/apps/lambdatestinc1584019832435.hyperexeonprem?tab=Overview) listing and click on **Get It Now** button. - -- Fill your details in the pop-up, and select a plan between **HyperExecute Private Cloud - Monthly** or **HyperExecute Private Cloud - Annual**. Click on **Continue** to proceed. - -**Step 3:** You will be redirected to HyperExecute Private Cloud listing page. Click on **Create** to proceed. - -**Step 4:** You will be redirected to the **Basics** page where you have to configure your plan for **HyperExecute Private Cloud.** Configure the fields as per your requirements, and click on **Review + create** to proceed further. - -- **Subscription**: The Azure subscription where the resource will be billed. -- **Resource Group**: Choose a unique name for this new [Resource group](https://learn.microsoft.com/en-us/azure/azure-resource-manager/management/manage-resource-groups-portal#what-is-a-resource-group), or allot it to an older resource group of your choice. -- **Select Location**: Select the location where you want to deploy HyperExecute Private Cloud. -- **App Insight Retention Period**: Select the retention period for [application insights](https://learn.microsoft.com/en-us/azure/azure-monitor/app/app-insights-overview?tabs=net) (in this case HyperExecute Private Cloud) in days. -- **Email ID**: The Email ID associated with your Azure account. -- **Select Parallel Tests**: The number of parallel test executions that you need. You can also check HyperExecute’s pricing plans by clicking on the link below the field. -- **Managed Application Name**: Choose a unique name for your application. -- **Ingress Type**: Choose between **Private** or **Public** to determine if you want a private endpoint or a public endpoint. -- **Managed Resource Group**: This field will be automatically filled, and you do not need to change it. - -**Step 5:** You can review all of your entries and then select the checkbox under **Co-Admin Access Permission**, indicating that you agree to the **Terms and Conditions** listed above. Click on **Create** and you will have successfully set up the HyperExecute Private Cloud. - -## Running HyperExecute Private Cloud - -Now that you have set up your HyperExecute Private Cloud application, you can access it by visiting `https://dashboard.hyperexecute.cloud`. However, before you do that, you need to ensure that you are connected to the same provisioned network where your HyperExecute application is deployed. - -To do this, you can either perform VNet peering or VNet linking, or run HyperExecute Private Cloud on a JumpHost virtual machine. You can go through the pages below to understand how to do that. - -### [VNet Peering and VNet Links](/support/docs/hyperexecute-vnet-peering-for-private-cloud/) - -### [Running HyperExecute on a JumpHost VM](/support/docs/hyperexecute-private-cloud-on-jumphost/) - -> If you have more questions regarding HyperExecute Private Cloud, reach out to us at [hyperexecute-privatecloudsupport@testmuai.com](mailto:hyperexecute-privatecloudsupport@testmuai.com). diff --git a/static/docs/hyperexecute-private-cloud-setup.md b/static/docs/hyperexecute-private-cloud-setup.md deleted file mode 100644 index 34e1813d0..000000000 --- a/static/docs/hyperexecute-private-cloud-setup.md +++ /dev/null @@ -1,25 +0,0 @@ -# HyperExecute Private Cloud - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -HyperExecute offers you the possibility to complete your test execution process at the highest speeds possible, even on your own private cloud. - -This allows you to have complete control over your testing infrastructure. With this added flexibility, you can run your tests on your own terms, while being completely comfortable with the security of your data. It is an enterprise-ready solution that can be modified to support the unique wants of your organization. - -With [smart features](/support/docs/key-features-of-hyperexecute/) such as auto-grouping, auto-retry, and fail-fast mechanisms, HyperExecute takes away all the pain to run tests leaving the developers or testers to focus on other important tasks. - -You can learn more about the HyperExecute Private Cloud by going through our documentation below. - - Benefits of HyperExecute Private Cloud - Learn why HyperExecute Private Cloud is a must-have for your organization. - - Setup with Azure - Learn how to distribute concurrency to different sub-organizations . - - Setup with AWS - Learn how to take access to your TestMu AI account and log in on your behalf from the backend. - - Guided Walkthrough of the HyperExecute Portal - Learn how to take access to your TestMu AI account and log in on your behalf from the backend. - -> If you have more questions regarding HyperExecute Private Cloud, reach out to us at [hyperexecute-privatecloudsupport@testmuai.com](mailto:hyperexecute-privatecloudsupport@testmuai.com). diff --git a/static/docs/hyperexecute-projects.md b/static/docs/hyperexecute-projects.md deleted file mode 100644 index 333756df4..000000000 --- a/static/docs/hyperexecute-projects.md +++ /dev/null @@ -1,74 +0,0 @@ -# HyperExecute Projects - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Projects serve as a centralized platform for organizing and managing test executions. By grouping similar tests, controlling access, and tracking progress, Projects streamline the testing process. You can configure [custom project](/support/docs/hyperexecute-projects/#setup-your-project) and [workflow setups](/support/docs/hyperexecute-projects/#schedule-your-workflows), as well as [integrate Tosca, Tosca DEX, and JMeter tests](/support/docs/hyperexecute-projects/#setting-up-specialized-projects) for unified management. This enables efficient test script creation, automated execution scheduling, and valuable insights into test results. - -## Link Jobs to Projects Using YAML -When you run jobs on HyperExecute, connecting them to projects helps you track performance trends, compare results over time, and get valuable insights from your test history. You can link jobs to projects in two ways using your YAML configuration file. - -### Method 1: Use an Existing Project -If you already have a project in HyperExecute, follow these steps to connect your jobs to it: - -1. Go to [Projects List](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute/projects) tab on HyperExecute Dashboard. -2. Select and open the project you want to link your jobs to. -3. Click the copy icon next to the project name. This copies the project configuration you'll need for your YAML file. -4. Paste the copied details into your YAML file before running your job. -5. All jobs using this YAML will automatically appear in your selected project. - -The copied project detail which you need to add to the YAML will look like this: -```yaml -project: -name: 'demo-project' #project name -id: '1abc2345-56de-7890-f1cg-h2ij34k56789' #project id -``` - -### Method 2: Create a New Project Automatically - -Want to start fresh or organize jobs under a new project? You can create one directly from your YAML file. To create a new project for your job run, follow the steps given below. - -1. Think of a unique name that clearly identifies what you're testing. -2. Include the project name in your YAML file in the following format. - -```yaml -project: -name: 'unique_project_name' -``` - -3. All jobs using this YAML will automatically appear in your new project. - -If a project with your chosen name already exists, HyperExecute will link your jobs to the existing project instead of creating new. - -## Setup Custom Project -**Step 1:** Go to the **Projects** section in HyperExecute. Click on the **New Project** to create your projects on HyperExecute. - -**Step 2:** Select **Custom Project** as the **Type of Project**. Provide a unique **Project Name** (required) and select your desired framework and language used in your project. Click on the **Continue**. - -**Step 3:** Provide your **GitHub URL** and the **[Access Token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens)** / **Bitbucket URL** and the **[Repository Access Tokens](https://support.atlassian.com/bitbucket-cloud/docs/repository-access-tokens/)** / **Azure Repo URL** and the **[Access Token](https://learn.microsoft.com/en-us/azure/devops/organizations/accounts/use-personal-access-tokens-to-authenticate?view=azure-devops&tabs=Windows)** of your repository. You can also add your team members as **Co-Owners** to allow them to edit or delete the project alongside you. - -**important** -Currently, only **GitHub**, **Bitbucket** and **Azure** repositories are supported. - -Click on **Save** button to finish the setup of your project. - -Once the project is created, the project header displays key details including the framework, language, owner, creation time, and a **GitHub/Bitbucket/Azure repository link**. Click the repository icon to navigate directly to the linked source code repository. - -## Schedule your Workflows -Within each HyperExecute Project, you can set up one or more workflows. These workflows allow for scheduled execution of your tests, providing you with the flexibility to automate your testing process according to your development and release cycles. - -> To learn more about Workflows in HyperExecute, refer to our [detailed documentation](/support/docs/hyperexecute-workflows) - -## Setting Up Specialized Projects -In addition to custom projects, HyperExecute supports integration with specialized testing tools such as Tosca, Tosca DEX, and Performance testing with JMeter. For detailed instructions on setting up these specific projects, please refer to their dedicated documentation pages: - - Tosca using Commander - Optimize and accelerate end-to-end testing for your entire digital landscape using Tosca. - - Tosca using DEX - Perform end-to-end testing using Tosca DEX with HyperExecute. - - Tosca for SAP - Use Tosca for SAP to integrate with HyperExecute. - - JMeter - Experience the JMeter Performance Testing with HyperExecute. diff --git a/static/docs/hyperexecute-provar-integration.md b/static/docs/hyperexecute-provar-integration.md deleted file mode 100644 index 0656d3622..000000000 --- a/static/docs/hyperexecute-provar-integration.md +++ /dev/null @@ -1,142 +0,0 @@ -# Integrate Provar with HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Provar is a low-code/no-code testing tool specifically designed for Salesforce applications, enabling efficient creation and execution of UI and API tests. - -This documentation guides you on how to integrate Provar seamlessly with HyperExecute to leverage distributed testing capabilities for comprehensive Salesforce test automation. - -## Prerequisties - -- Download and install the latest version of [Provar Automation Installer](https://documentation.provar.com/documentation/general-information/installing-provar/). -- You must have an active License key to run Provar. - -## Step 1: Configure the Browser Provider Information - -- Update the Desktop Settings to **HyperExecuteGrid: ANY Chrome**. - -- Click on **Test Settings** >> **Browser Providers** section. Click on the **+** symbol to add a new Browser Provider. - -- You will receive a pop-up to enter your configurations: - - - Enter the **Name** for your new Browser Provider. - - Select the **Type** as **TestMu AI** - - Enter your [TestMu AI **User Name** and **Access Key**](/support/docs/hyperexecute-how-to-get-my-username-and-access-key/) - - Update all the **browser** options that you want to run your tests on. - - Update all the **capabilities** that you want to pass through the test. - -- Click on **Test Connection** >> **OK** - -## Step 2: Generate and Update the build.xml file - -You have to generate the `build.xml` file using the Provar GUI. You can follow the steps mentioned in the [documentation](https://documentation.provar.com/documentation/devops/apache-ant/apache-ant-generating-a-build-file/) to generate your build file. - -- Update the `prover.home` property with the installation path of Provar in HyperExceute as shown below: - -```bash - -``` - -- Add the following tags inside your `build.xml` file. These tags should be present inside the `` tag as mentioned below: - -```bash - - -``` - -- Under the `Run-Test-Case` tag, replace the `` tag with the following code: - -```bash - -``` - -## Step 3: Set the Provar License as Secret - -You will have to [create a new secret](/support/docs/hyperexecute-how-to-save-and-manage-secrets/#create-a-new-secret) in HyperExecute. - -- Set **Key Name** as `provarLicense` -- Set the **Value** as the contents of the license file. - -## Step 4: Setup the CLI in your Test Suite - -After cloning / downloading the sample repo, you need to setup the CLI and the environment variables. - -### Download the HyperExecute CLI - -The CLI is used for triggering the tests on HyperExecute. It is recommend to download the CLI binary on the host system and keep it in the root directory of the suite to perform the tests on HyperExecute. - -You can download the CLI for your desired platform from the below mentioned links: - -| Platform | HyperExecute CLI | -| ---------| ---------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| MacOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -### Setup Environment Variable - -Now, you need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in the [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). - -Run the below mentioned commands in your terminal to setup the CLI and the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -## Step 5: Configure YAML in your Test Suite - -Replace the `pre` command with your own command that discovers the desired **.testcase** files. - -```yaml ---- -version: 0.2 -globalTimeout: 150 - -runson: win - -runtime: -language: java -version: "11" - -autosplit: false - -concurrency: 4 - -retryOnFailure: true -maxRetries: 2 - -env: -ProvarSecretsPassword: - -pre: -- ls - -post: -- cat ./ANT/.lambdatestBuild.xml - -framework: -name: provar -discoveryType: test -additional_args: -discoveryLevel: file # required -``` - -## Step 6: Execute your Test Suite - -> **NOTE :** In case of MacOS, if you get a permission denied warning while executing CLI, simply run **`chmod u+x ./hyperexecute`** to allow permission. In case you get a security popup, allow it from your **System Preferences** → **Security & Privacy** → **General tab**. - -Run the below command in your terminal at the root folder of the project: - -```bash -./hyperexecute --config RELATIVE_PATH_OF_YOUR_YAML_FILE -``` - -OR use this command if you have not exported your username and access key in the step 2. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE `} - -## Step 5: Monitor Your Tests - -- Visit the [HyperExecute Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) and check your Job status. diff --git a/static/docs/hyperexecute-python-use-cases.md b/static/docs/hyperexecute-python-use-cases.md deleted file mode 100644 index f0ef3aa3b..000000000 --- a/static/docs/hyperexecute-python-use-cases.md +++ /dev/null @@ -1,413 +0,0 @@ -# HyperExecute Python Use Cases - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This guide covers the most common real-world scenarios and troubleshooting cases when running **Python**, **Pytest**, and **Behave** test frameworks on **TestMu AI HyperExecute**. Each section describes a use case, its root cause, and the precise YAML translation required to configure or fix the issue within your `hyperexecute.yaml`. - -## 1. Handling Python Version Compatibility Issues -Sometimes, test scripts fail due to Python version incompatibility. For example: - -```bash title="Terminal" -$ python test_script.py -Traceback (most recent call last): -File "test_script.py", line 3, in -from collections import MutableMapping -ImportError: cannot import name 'MutableMapping' from 'collections' -``` -In Python versions 3.3 to 3.9, MutableMapping was part of the collections module. From Python 3.10 onward, it was moved to collections.abc, causing import errors in older code. - -**Solution :** To ensure compatibility, match the Python runtime version with your local setup. Specify the version explicitly in your YAML configuration. - -**YAML Translation :** -```yaml title="hyperexecute.yaml" -runtime: -- language: python -version: "3.9" -``` - -## 2. Ensuring PIP Version Compatibility -Certain packages require a minimum version of PIP to install successfully. An outdated PIP version may cause the following error: -> **ERROR :** You must install pip 21.0 or later to install this package. - -**Solution :** Upgrade PIP before installing dependencies. - -```bash -python -m pip install --upgrade pip -``` - -If you are using Python 3: - -```bash -python3 -m pip install --upgrade pip -``` - -**YAML Translation :** -```yaml title="hyperexecute.yaml" -pre: -- python3 -m pip install --upgrade pip -``` - -## 3. Fixing Missing requirements.txt Errors -If your pipeline fails with: -> **ERROR:** Could not open requirements file: [Errno 2] No such file or directory: 'requirements.txt' - -It means that the `requirements.txt` file might be missing or ignored in .gitignore. - -**Solution :** Ensure the file exists in the project root. Remove or comment out any requirements.txt entry from .gitignore. For this issue, ther are no YAML change required. This is a file management fix. However, verify that the command below correctly references the existing file name. - -```bash -pip install -r requirements.txt --cache-dir CacheDir -``` - -## 4. Restoring Missing .env Files -When environment variables are not set, tests may fail due to missing credentials or configuration values because `.env` files are commonly excluded in `.gitignore` for security. If ignored, they aren’t uploaded to HyperExecute, and environment variables remain unset. - -**Solution :** Ensure that required `.env` files are not ignored in `.gitignore`. If sensitive credentials exist, use HyperExecute Secrets or Environment Variables instead of exposing them in `.env` files. - -## 5. Optimizing Package Caching for Faster Jobs -If your pre step installs dependencies every time, your caching configuration might be incorrect. - -**Common Causes :** -- Invalid `cacheKey` syntax. -- Incorrect filename in cache reference. -- Missing `--cache-dir` argument. - -**Solution :** Use proper syntax, ensure file naming consistency, and define a cache directory. - -YAML Translation: - -```yaml title="hyperexecute.yaml" -cacheKey: '{{ checksum "requirements.txt" }}' -cacheDirectories: -- CacheDir -pre: -- pip install -r requirements.txt --cache-dir CacheDir -``` - -## 6. Building Projects with Public and Private Registries -### Case 1: Public Registry Installation -If all dependencies are public, install them directly: - -```bash -pip3 install -r requirements.txt --cache-dir CacheDir -``` - -**YAML Translation :** - -```yaml title="hyperexecute.yaml" -pre: -- pip3 install -r requirements.txt --cache-dir CacheDir -``` - -### Case 2: Private Registry Installation -When using a private registry, authentication is required. Use proxy variables and authorization tokens to install packages securely. - -```bash -pip3 install --proxy http://$LT_PROXY_HOST:$LT_PROXY_PORT \ ---header "Authorization: Bearer $PRIVATE_REGISTRY_TOKEN" \ --r requirements.txt -``` - -**YAML Translation :** - -```yaml title="hyperexecute.yaml" -pre: -- pip3 install --proxy http://$LT_PROXY_HOST:$LT_PROXY_PORT \ ---header "Authorization: Bearer $PRIVATE_REGISTRY_TOKEN" \ --r requirements.txt -``` - -**Sample `requirements.txt` file :** - -```text ---extra-index-url https://your-private-registry.com/simple ---trusted-host your-private-registry.com -your-private-package==1.0.0 -another-package==2.0.0 -``` - -**Error Troubleshooting :** -- Verify `$LT_PROXY_HOST` and `$LT_PROXY_PORT `values. -- Confirm `$PRIVATE_REGISTRY_TOKEN` is valid. - -## 7. Configuring Database Connections -When connecting to a database during test runs, you may encounter: - -> **ConnectionRefusedError:** connect ECONNREFUSED 127.0.0.1:3306 - -**Solution :** Expose the database service using the `--expose` flag in your YAML configuration. - -**YAML Translation :** - -```yaml title="hyperexecute.yaml" -args: -- "--expose mysql1:localhost:3309" -- "--expose mysql2:localhost:3307" -``` - -**Sample Connection Code :** - -```python title="test.py -import os -import mysql.connector - -db_host = os.getenv("MYSQL1_PROXY_HOST", "localhost") -db_port = os.getenv("MYSQL1_PROXY_PORT", "3309") - -connection = mysql.connector.connect( -host=db_host, -port=db_port, -user="your_user", -password="your_password", -database="your_database" -) -``` - -## 8. Running Tests in Behave and Pytest -To execute tests based on substrings or tags: - -**Example Commands :** - -```bash -pytest -k "TestAdvanceBoy" -behave -t @smoke -``` - -**YAML Translation :** - -```yaml title="hyperexecute.yaml" -testRunnerCommand: pytest -k "TestAdvanceBoy" -# or -testRunnerCommand: behave -t @smoke -``` - -To execute an entire feature file: - -```bash -behave -f feature_path/sample.feature -``` - -**YAML Translation :** - -```yaml title="hyperexecute.yaml" -testRunnerCommand: behave -f feature_path/sample.feature -``` - -## 9. Discovering Tests Dynamically - -- List all feature files containing a specific tag (e.g., @smoke): - -```bash -grep -rl "@smoke" features/*.feature -``` - -**YAML Translation :** - -```yaml title="hyperexecute.yaml" -testDiscovery: -type: raw -command: grep -rl "@smoke" features/*.feature -``` - -- List all scenarios under that tag : - -```bash -grep -rH -A 3 "@smoke" features/*.feature | grep -i "Scenario" | cut -d ":" -f 2- -``` - -**YAML Translation :** - -```yaml title="hyperexecute.yaml" -testDiscovery: -type: raw -command: grep -rH -A 3 "@smoke" features/*.feature | grep -i "Scenario" | cut -d ":" -f 2- -``` - -## 10. Generating JSON Reports with Behave -To generate JSON reports for your test execution: - -```bash -behave -f json.pretty -o reports/test_report.json -``` - -**YAML Translation :** - -```yaml title="hyperexecute.yaml" -post: -- behave -f json.pretty -o reports/test_report.json -``` - -## 11. Dynamically Assigning Artifact Names -To dynamically name artifacts using environment variables, define them in your YAML configuration. - -**YAML Translation :** - -```yaml title="hyperexecute.yaml" -uploadArtefacts: -- name: "${abcd}" -path: -- reports/** -``` - -Pass variable values through CLI during job execution: - -```bash -./hyperexecute -u -k -i --vars "abcd=Folder" -``` -If abcd=Folder, the uploaded artifact will appear as Folder on the HyperExecute dashboard. - -## 12. Using a Cache Directory for Package Installation -To enable caching for faster dependency installation: - -```bash -pip3 install -r requirements.txt --cache-dir CacheDir -``` - -This ensures packages are stored and reused in subsequent runs, reducing job setup time. - -## 13. Monitoring Tunnel Health and Auto-Retry -When running automated tests in HyperExecute, tunnel connections may drop or become unstable, leading to test failures. - -**Scenario/Use Case:** -- Tunnel may drop due to network instability, firewall restrictions, or internal server issues. -- Goal: Continuously monitor tunnel health and automatically retry without manual intervention. - -**Solution** -Use the `checkTunnelOnFailure` parameter in your YAML configuration. This ensures HyperExecute automatically retries the tunnel if it fails during test execution. - -**YAML Translation:** - -```yaml title="hyperexecute.yaml" -checkTunnelOnFailure: true -``` - -## 14. Different Ways to Execute Tests in Pytest via CLI -### Case 1: Run Tests in a Specific Directory or File - -```bash -pytest path/to/test_directory/ -pytest path/to/test_file.py -``` - -### Case 2: Run a Specific Test Function Within a File -```bash -pytest path/to/test_file.py::test_function_name -``` - -### Case 3: Run Tests Matching a Substring or Keyword -```bash -pytest -k "test_keyword" -``` - -**Example :** -```bash -pytest -k "login" -``` - -## 15. Debugging in Pytest with Detailed Logs - -**Solution :** Use the `--log-cli-level=DEBUG` flag to capture debug-level logs. - -**Example Python Test :** - -```python title="test.py -import logging - -logging.basicConfig(level=logging.DEBUG) -logger = logging.getLogger() - -def test_debug_logging(): -value = 42 -if value == 42: -logger.debug("Value is 42, running debug checks") -logger.debug("This is a debug message") -logger.info("Test info message") -logger.warning("Test warning message") -logger.error("Test error message") -if value != 42: -logger.critical("Critical issue: Value should be 42!") -assert value == 42 -``` - -**Terminal Output Example :** - -```bash -DEBUG:root:Value is 42, running debug checks -DEBUG:root:This is a debug message -INFO:root:Test info message -WARNING:root:Test warning message -ERROR:root:Test error message -``` - -## 16. Automating Screenshot Download and PDF Generation from TestMu AI -During native app automation, screenshots captured per session ID can be retrieved using: - -```bash -curl -X GET "https://mobile-api.lambdatest.com/mobile-automation/api/v1/sessions/${session_id}/screenshots" \ --H "accept: application/json" \ --H "Authorization: Basic " -``` - -**Problem :** Manually downloading, extracting, and arranging screenshots into a PDF is time-consuming. - -**Solution :** Use the TestMu AI Screenshot Downloader & PDF Generator CLI. - -- **Installation :** -```bash -npm install -g lambdatest-screenshot-pdf -``` - -- **Usage :** -```bash -AUTH_HEADER="Basic your_encoded_auth_string" lambdatest-screenshot-pdf --output -``` - -- **Example :** -```bash -AUTH_HEADER="Basic cmF0aG9yZXYXYXYXYXXgybFlXTm5LclZPeVRRQ01RRGdyNTZjVg==" \ -lambdatest-screenshot-pdf 1a80510a-289a-46b7-9f60-da01d108de10 --output ~/Desktop/screenshots -``` - -**What Happens When You Run This Command:** -- Fetches the ZIP file containing screenshots for the session ID. -- Downloads the ZIP file. -- Extracts screenshots in the original execution order. -- Generates a screenshots.pdf containing the ordered screenshots. -- Cleans up temporary files automatically. -- Ensure your AUTH_HEADER is a valid Base64-encoded API key from TestMu AI. - -## 17. Base YAML Configuration for Pytest on Windows with Autosplit - -Example hyperexecute.yaml: - -```yaml title="hyperexecute.yaml" ---- -version: 0.1 -globalTimeout: 90 -testSuiteTimeout: 90 -testSuiteStep: 90 - -runson: win -autosplit: true -concurrency: 2 -runtime: -language: python -version: "3" - -cacheKey: '{{ checksum "requirements.txt" }}' -cacheDirectories: -- CacheDir -pre: -- pip3 install -r requirements.txt --cache-dir CacheDir -post: -- cat yaml/win/pytest_hyperexecute_autosplit_sample.yaml - -testDiscovery: -type: raw -mode: remote -command: grep -nri 'class' tests -ir --include=\*.py | sed 's/:.*//' - -testRunnerCommand: pytest -s --verbose $test - -jobLabel: [selenium-pytest, win, autosplit] -``` diff --git a/static/docs/hyperexecute-qtest-integration.md b/static/docs/hyperexecute-qtest-integration.md deleted file mode 100644 index 1b2ff7e3f..000000000 --- a/static/docs/hyperexecute-qtest-integration.md +++ /dev/null @@ -1,106 +0,0 @@ -# Integrate qTest with HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -qTest is a comprehensive test management tool designed for Project Management, Bug Tracking, and Test Management. It embraces a centralized test management approach, facilitating seamless communication and accelerating task development across QA teams and stakeholders. - -This document details the seamless integration between HyperExecute and qTest, enabling you to run your automated tests on a variety of devices and real devices provided by TestMu AI. - -## Prerequisites - -- You must have [Python 3.6](https://www.python.org/downloads/), [Apache Maven](https://maven.apache.org), and [Git](https://git-scm.com/downloads) installed on your local machine. -- [Download](https://support-hub.tricentis.com/open?sys_kb_id=194a54eedb4f5c181ea7bb13f3961950&id=kb_article_view&number=KB0015571) and extract the **agentctl-[version]-mac-x64-full.tgz** file to a desired installation directory. -- Login to qTest Manager as an administrator and access the Resources page. -- Expand the **APIs and SDK** sections. - -- Navigate to the **agentctl** directory and start the host by running the following command: - -```bash -./agentctl start -``` - -- Visit ``http://localhost:6789``, add your details and click on the **Register** button. - - - **Host Name :** You have to enter any name that will be assigned as the host name. - - **qTest URL :** Enter the URL of your qTest Manager page. For e.g. `https://aman.qtestnet.com` - - **qTest Token :** Enter the Token you have copied in the above step. - - **Full Name :** Mention your name here. - -## Step 1: Setup your Shell Agent - -- This repository is used to pull tests from GitHub, run an Apache Maven build of JUnit tests, parse the results, and automatically upload the test results to qTest Manager. - -**Automate JUnit** -Download or Clone the code for the Shell Script Automation Host Feature from the TestMu AI GitHub repository to run the tests on the HyperExecute. - - View on GitHub - -- You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** - -Download or Clone the code sample for the JUnit framework from the TestMu AI GitHub repository to run the tests on the HyperExecute. - - View on GitHub - -### Download the HyperExecute CLI - -The CLI is used for triggering the tests on HyperExecute. It is recommend to download the CLI binary on the host system and keep it in the root directory of the suite to perform the tests on HyperExecute. - -You can download the CLI for your desired platform from the below mentioned links: - -| Platform | HyperExecute CLI | -| ---------| ---------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| MacOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -> **NOTE :** In case of macOS, if you get a permission denied warning while executing CLI, simply run **`chmod u+x ./hyperexecute`** to allow permission. In case you get a security popup, allow it from your **System Preferences** → **Security & Privacy** → **General tab**. - -## Step 2: Update the Configuration Files - -- Go to the `conf.json` file and make the following changes: - - - **git_url :** The shell script uses the url to clone a repository and send pull requests every time it runs if -g input is used. - - **local_repository :** The folder containing the test cases. The shell script will use this to know where to run the maven build. Make sure to place this folder in the same directory as the the python and shell scripts. - - **qtest_api_token :** The token used to authorize the connection to qTest Manager. - - **qtest_url :** The personal url that is used to access QASymphony API. - -```bash -{ -"git_url": "", -"local_repository": "", -"qtest_api_token": "", -"qtest_url": "" -} -``` - -- If you are macOS user, go to the `run.sh` file or if you are a windows user go to the `run.bat` file and instead of `mvn` command, you need to pass the below mentioned command: - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE `} - -> Refer the documentation to understand how to get your [Username and Access key](/support/docs/hyperexecute-how-to-get-my-username-and-access-key/). - -## Step 3: Setup the Automation - -- Navigate to your Automation Host ``http://localhost:6789/home`` and click on the **Add** button to add a new agent. - -- A popup will appear to create an agent, fillup the following details: - - - **Agent Name :** You can give any name to your agent. - - **qTest Manager Project :** Choose your project inside which you want to create this build. - - **Agent Type :** Choose Shell Agent as your agent type. - - **Directory :** Path of the directory containing your scripts and shell agent. - - **Allocated Execution Time :** Amount of time you expect the script to take to execute in minutes - - **Kick-off scripts :** The file path to your shell script. This shell scripts takes in two inputs, one for using git and the second for updating your current test cycle. - -## Step 4: Execute the Shell Script - -- Click on the **Actions** button > **Kick-off shell scripts now** button to start the shell script. This will upload all of the tests cases to qTest. - -## Step 5: Monitor Your Tests - -- Login into qTest Manager, go to the Test Execution tab, and there should be a test cycle under your project called **Junit Automated Tests** -- Click on the test cycle and it should show all of the tests that were run through the maven build and their statuses. - -- Visit the [HyperExecute Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) and check your Job status. diff --git a/static/docs/hyperexecute-release-notes-2023.md b/static/docs/hyperexecute-release-notes-2023.md deleted file mode 100644 index 31c2fc7ce..000000000 --- a/static/docs/hyperexecute-release-notes-2023.md +++ /dev/null @@ -1,193 +0,0 @@ -# HyperExecute Release Notes 2023 - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Version 2.1.4 - -### Support for Multiple Runtime Environments - -You can now download and install **multiple languages** and and frameworks of a **specified versions** in your machine that is needed to execute your tests. - -```bash -runtime: -- language: java -version: '17' -addons: -- name: gauge -version: '1.5.6' -- name: gradle -version: '7.6' -- language: python -version: '3.10' -- language: node -version: '16' -``` - -> Go through the documentation to understand the [runtime](/support/docs/deep-dive-into-hyperexecute-yaml/#runtime) flag. - -## Version 2.1.0 - -### Clone your Private Repositories via Tunnel - -You can now clone the private repositories from your Git provider that are hosted on a private network or behind a firewall via tunnel. - -After specifying the details of your repository, you need to set the useInCodeSetUp flag to true as shown below. - -```bash -tunnelOpts: -useInCodeSetUp: true -### this flag is mandatory to pass to enable cloning via tunnel -``` - -> Refer to the detailed documentation on [how to clone your private repositories via tunnel](/support/docs/hyperexecute-how-to-configure-tunnel/#clone-a-repository-via-tunnel) - -## Version 1.140.0 - -### Test Muting - -Test Muting is a new feature designed to enhance test efficiency and effectiveness by allowing users to temporarily disable specific test cases. This feature proves particularly useful for skipping consistently failing or irrelevant tests, streamlining test runs and enabling a sharper focus on crucial results. - -**Key Benefits** - -- **Improved Test Efficiency and Effectiveness:** By eliminating the execution of unnecessary or problematic tests, Test Muting streamlines test runs, conserving valuable time and resources. - -- **Reduced Noise in Test Results:** By filtering out irrelevant or consistently failing tests, Test Muting provides a clearer picture of the overall test suite's performance, facilitating more informed decision-making. - -- **Faster Feedback on HyperExecute Jobs:** By streamlining test runs, Test Muting accelerates the feedback cycle on HyperExecute jobs, enabling testers to identify and address issues promptly. - -- **Customizable Threshold Number to Mute Tests:** Users can tailor Test Muting to their specific needs by setting a custom threshold for automatically muting tests based on consecutive failures. - -- **Manual Mute/Unmute of Specific Test Cases:** Users retain complete control over Test Muting, with the ability to manually mute or unmute specific test cases as needed. - -> For detailed instructions on utilizing Test Muting, including automatic and manual muting and unmuting of test cases, please refer to the comprehensive [Test Muting documentation](/support/docs/hyperexecute-test-muting/). - -## Version 1.139.0 - -### Enhanced User Experience with Refined UI/UX - -HyperExecute is committed to continuously improving the user experience by implementing thoughtful UI/UX enhancements. This release brings a host of refinements that streamline navigation, enhance accessibility, and elevate the overall user experience. - -**Key UI/UX Improvements** - -- **Simplified Navigation:** Revised navigation structure for intuitive access to key functionalities and resources. - -- **Enhanced Accessibility:** Improved accessibility features to ensure an inclusive and accessible experience for all users. - -- **Streamlined Workflows:** Optimized workflows to minimize manual steps and enhance task completion efficiency. - -- **Intuitive Interface:** Refreshed design elements and intuitive interface interactions for a seamless user experience. - -- **Visual Enhancements:** Refined visual aesthetics and consistent design principles for a cohesive and appealing user interface. - -## Version 1.138.0 - -### Infrastructure Improvement for Autoscaling - -HyperExecute continuously strives to enhance its autoscaling infrastructure to ensure optimal resource allocation and seamless test execution. This release introduces significant infrastructure improvements that elevate autoscaling efficiency, cost-effectiveness, and reliability. - -**Key Infrastructure Enhancements** - -- **Optimized Instance Selection:** HyperExecute now employs intelligent algorithms to select the most suitable instance types based on real-time workload demands, maximizing resource utilization and cost-efficiency. - -- **Automated Instance Termination:** Idle instances are automatically identified and terminated, preventing unnecessary resource consumption and optimizing infrastructure costs. - -- **Enhanced Load Balancing:** Load balancing strategies are refined to distribute traffic across multiple instances effectively, preventing bottlenecks and ensuring optimal resource utilization. - -## Version 1.137.0 - -### Performance Enhancements for a Smoother Experience - -HyperExecute is committed to continuously refining its performance, ensuring that testers can execute their tests efficiently and seamlessly. This release introduces subtle yet impactful performance enhancements that collectively elevate the user experience. - -**Key Performance Enhancements** - -**Marginal Reduction in Test Execution Time:** Experience slightly faster test execution speeds, contributing to a smoother and more efficient testing workflow. - -**Improved Job Startup Responsiveness:** Witness a slight decrease in job startup times, allowing testers to initiate their test runs promptly and without delay. - -- **Enhanced Load Handling Capacity:** HyperExecute now handles high-volume workloads with marginally greater efficiency, ensuring seamless test execution even during peak usage periods. - -- **Streamlined Resource Allocation:** HyperExecute optimizes resource utilization, ensuring that infrastructure resources are allocated effectively and efficiently. - -- **Reduced Test Result Processing Latency:** Experience slightly faster test result processing times, enabling testers to access their results promptly and make informed decisions. - -## Version 1.136.0 - -### Security Improvements - -HyperExecute is committed to prioritizing security and safeguarding our users' data and infrastructure. This release introduces a comprehensive set of security enhancements that reinforce our commitment to providing a secure and reliable testing platform. - -## Version 1.135.0 - -### Expanded Framework Support - -HyperExecute now extends its support to encompass four widely used testing frameworks: **Maven/JUnit4, Maven/JUnit5, WDIO/Mocha, and WDIO/Jasmine**. This expansion caters to a broader range of testing scenarios and preferences, enabling seamless integration with existing testing workflows. - -**Key Benefits** - -- **Enhanced Flexibility:** The addition of these frameworks empowers testers to leverage their preferred tools and methodologies within HyperExecute's comprehensive execution environment. - -- **Streamlined Integration:** Integration with these frameworks ensures a smooth and efficient testing process, eliminating the need for manual configuration or compatibility concerns. - -- **Broadened Testing Scope:** By supporting these frameworks, HyperExecute expands its reach to accommodate a wider range of testing projects and applications. - -**Supported Frameworks** - -- **Maven/JUnit4:** A popular Java testing framework that leverages the JUnit4 framework for test case organization and execution. - -- **Maven/JUnit5:** The latest iteration of the JUnit framework, offering enhanced features and capabilities for Java testing. - -- **WDIO/Mocha:** A JavaScript testing framework that utilizes Mocha as its test runner, providing asynchronous test execution and assertion capabilities. - -- **WDIO/Jasmine:** Another JavaScript testing framework that employs Jasmine as its test runner, offering behavior-driven development (BDD) features and a concise syntax. - -## Version 1.134.0 - -### Semaphore CI/CD Integration - -Semaphore CI/CD is a popular cloud-based platform that helps engineering teams of all sizes build, test, and deploy their software faster and more reliably. HyperExecute is a powerful test execution platform that offers unparalleled speed, efficiency, scalability, and compatibility. - -The new integration between Semaphore CI/CD and HyperExecute will allow teams to execute their tests on HyperExecute's cloud infrastructure seamlessly. This can help teams significantly shorten their test cycles and deliver software faster. - -> Refer to the detailed documentation on how to integrate [Semaphore CI/CD](/support/docs/semaphore-integration-with-hyperexecute/) with HyperExecute. - -### Smart UI Tests for Cypress Framework - -Smart UI Testing is a powerful tool that enables you to execute regression testing effortlessly and identify Visual UI Regression bugs with ease. - -HyperExecute now supports the Cypress framework, allowing teams to run their Cypress UI tests on our cloud infrastructure, thus helping teams improve the quality of their UI tests and reduce the time to test their applications. - -> To understand the step-by-step guide, follow the documentation to run your [Cypress Tests on Smart UI](/support/docs/hyperexecute-smart-ui-testing-using-cypress/) using HyperExecute. - -## Version 1.133.0 - -### Bamboo CI/CD Integration - -Bamboo is a continuous integration (CI) and continuous delivery (CD) server from Atlassian. It helps teams automate the software development and delivery process, from code commit to production deployment. Bamboo can be used to build, test, and deploy software on a variety of platforms, including cloud, on-premises, and hybrid environments. - -The new integration between Bamboo CI and HyperExecute will allow teams to execute their tests on HyperExecute's cloud infrastructure seamlessly. This can help teams significantly shorten their test cycles and deliver software faster. - -> Refer to the detailed documentation on how to integrate [Bamboo CI](/support/docs/bamboo-integration-with-hyperexecute/) with HyperExecute. - -### GoCD Integration - -GoCD, short for "Go Continuous Delivery," is an open-source continuous integration and continuous delivery (CI/CD) server that helps automate and streamline the software development and release process. It is designed to facilitate the efficient and reliable delivery of software from development to production. - -The integration between GoCD and HyperExecute will allow teams to execute their tests on HyperExecute's cloud infrastructure seamlessly and shorten the test cycles and deliver software faster. - -> Refer to the detailed documentation on how to integrate [GoCD](/support/docs/gocd-integration-with-hyperexecute/) with HyperExecute. - -## Version 1.131.0 - -### Snooper Now on HyperExecute! - -Boost your testing capabilities with Snooper's advanced features for in-depth analysis and debugging. -About Snooper: A snooper command helps you discover the list of feature file scenarios based on the arguments provided. - -> For more details, check the documentation on [Snooper Command](/support/docs/hyperexecute-snooper/). - -### Android-SDK Runtime Support! - -Simplify your validation process, enhance adaptability, and elevate your testing with Android-SDK runtime support. - -> For more details, Check the [documentation](/support/docs/deep-dive-into-hyperexecute-yaml/#runtime). diff --git a/static/docs/hyperexecute-release-notes-2024.md b/static/docs/hyperexecute-release-notes-2024.md deleted file mode 100644 index 8804cdda6..000000000 --- a/static/docs/hyperexecute-release-notes-2024.md +++ /dev/null @@ -1,627 +0,0 @@ -# HyperExecute Release Notes 2024 - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Version 2.6.5 - -### New Regions Added for HyperExecute Performance Testing -We are excited to announce the addition of four new regions for performance testing in HyperExecute. These regions enable better coverage, reduced latency, and enhanced testing capabilities for global users. - -The new regions are: -- **Central India** - Pune, Maharashtra -- **Malaysia** - Southeast Asia (Singapore) -- **Argentina** - South America (Brazil South) -- **Mexico** - Mexico Central - -> 📕 Visit our [Performance Testing Documentation](/support/docs/hyperexecute-run-jmeter-tests/) for step-by-step instructions on configuring your tests and making the most of HyperExecute's enhanced regional coverage. - -### Download Reports for XCUI Framework -Now, you can download the [JUnit XML report](/support/docs/junit-xml-report/) for your test runs on the XCUI framework. - -To generate reports for your XCUI tests, add the `reports: true` flag in your [HyperExecute YAML](/support/docs/hyperexecute-yaml-parameters/) file and pass the `--download-report` flag with the CLI command. - -> 📕 Refer to our detailed documentation to [Generate XCUI Reports](/support/docs/hyperexecute-xcui-testing/#step-5-generate-reports-and-artifacts) - -## Version 2.6.2 - -### Support for Appium and Maestro Android Emulator -HyperExecute now supports the Appium and the Maestro Android Emulator, providing enhanced flexibility for mobile application testing on Android emulators. - -**Key Features:** -- **Streamlined Emulator Integration**: Perform testing on virtual devices without requiring physical hardware, reducing setup complexity. -- **Parallel Execution :** Run multiple tests simultaneously on virtual devices using HyperExecute's parallelism to speed up test cycles and reduce execution time for functional and UI tests. -- **Customizable Capabilities:** Define emulator configurations such as device name, platform version, and orientation directly within your test files. - -To execute tests using the Android Emulator, add the following configurations to your `hyperexecute.yaml`: -```yaml -framework: -args: -isRealMobile: false -``` - -> Refer to our detailed documentation for [Appium](/support/docs/hyperexecute-appium-virtual-device/) and [Maestro](/support/docs/hyperexecute-maestro-testing/) on configuring and running tests with Android emulators. - -### New Version Release for Beta and Dev Edge Browser -HyperExecute expands its support for the latest versions of dev and beta browsers for the Windows platform. With this expanded compatibility, you can streamline your testing process and deliver exceptional user experiences with confidence! - -- **Dev version:** 132.0.2957.11 -- **Beta version:** 132.0.2957.11 - -## Version 2.6.0 - -### Label-Based Filtering for Test History -This feature addresses the challenge of managing and tracking test runs as development teams scale and test suites grow more complex. By allowing you to group and filter tests based on custom labels, it simplifies test result tracking, reporting, and team collaboration. - -Some of the specific benefits of the Test History Labels feature include: - -- Quickly identifying and grouping similar test runs (e.g., all regression tests, all tests for a particular environment). -- Simplifying the process of retrieving test runs for a specific sprint, feature, or environment. -- Enhancing overall test management and organization for development teams. - -## Version 2.5.9 - -### Consolidated JUnit XML Report Generation -The consolidated JUnit XML report includes detailed information on all executed tests, i.e., which tests passed, failed, or were skipped, along with any error messages and stack traces for failed tests. It is widely supported across CI/CD tools, which makes it easy to integrate with existing workflows and with other tools or systems. - -> 📕 Refer to our documentation for detailed instructions on setting up your project and using the new [JUnit XML report](/support/docs/junit-xml-report/) - -### Generate Artifacts for Espresso & XCUI Frameworks -Now, you can download artifacts for your test runs on both **Espresso** and **XCUI** frameworks and detailed reports for your **Espresso** test executions. - -- To generate artifacts and reports for your Espresso tests, add the `artifacts: true` and `reports: true` flag in your YAML file and pass the `--download-artifacts` and `--download-report` flag with the CLI command - - > 📕 Refer to our detailed documentation to generate [Espresso Artifacts and Reports](/support/docs/hyperexecute-espresso-testing/#step-6-generate-reports-and-artifacts) - -- To generate artifacts for your XCUI tests, add the `artifacts: true` flag in your YAML file and pass the `--download-artifacts` flag with the CLI command - - > 📕 Refer to our detailed documentation to generate [XCUI Artifacts](/support/docs/hyperexecute-xcui-testing/#step-5-generate-artifacts) - -### Browser and Selenium Updates - -- **Firefox** Browser Update to **Version 132.0** - - HyperExecute now supports Firefox version 132.0. This latest Firefox release includes performance and security improvements to ensure a smoother and safer testing experience. - > **Note :** Firefox 132.0 is currently not compatible with **Cypress**. Users running Cypress tests should continue to use a compatible Firefox version until an update is released. - -- **Selenium JAR** Update to **Version 4.26.0** - - HyperExecute has upgraded to Selenium JAR version 4.26.0. This update includes stability enhancements and bug fixes for better compatibility and reliability across various browsers. - -## Version 2.5.8 - -### Added Bitbucket Support in Projects Feature ✨ -HyperExecute Projects now supports Bitbucket repositories for workflow scheduling. You can seamlessly connect and manage workflows using either GitHub or Bitbucket repositories. - -**Key Functionalities:** -- Connect Bitbucket repositories using Repository Access Tokens -- Schedule and automate workflows from Bitbucket repositories -- Maintain existing GitHub integration with unchanged functionality -> 📕 Refer to the [Projects Documentation](/support/docs/hyperexecute-projects/) - -## Version 2.5.7 - -### Remote Test Discovery Mode -HyperExecute now includes a Remote Test Discovery Mode to enhance the test discovery process. This mode centralizes test discovery tasks on dedicated Virtual Machines, improving efficiency and resource utilization, especially for complex test setups and matrix configurations. - -**Key Features:** -- **Centralized Discovery :** Remote discovery on Virtual Machines, optimizing the process by focusing only on identifying tests. -- **Orchestration Support :** Efficient distribution of discovered tests across VMs to reduce idle time and improve resource usage. -- **Optimized Caching :** Reduces redundant cache operations by sharing caches across execution tasks. - -> For more details, go through the [Remote Discovery Documentation](/support/docs/deep-dive-into-hyperexecute-yaml/#testdiscovery) - -## Version 2.5.6 - -### Browser Version Updates: -- **Firefox :** Added support for Firefox-130 and Firefox-131, enabling testing on the latest versions. -- **Edge :** Downgraded to Edge-128 for compatibility and stability enhancements. -- **Chrome :** Upgraded to Chrome-129 to include the newest features and security updates. - -These updates ensure broader coverage for cross-browser testing and improve test stability across the latest browser versions. - -## Version 2.5.5 - -### Extended Report Formats for JUnit Framework -In addition to the existing JUnit XML report, support for generating reports in multiple formats (e.g., HTML, XML, JSON) has been added. This enhancement enables more flexible report generation to meet different user needs. - -> To avail this feature, connect with our Support Team. - -This can be configured with the following parameters: -- **frameworkName:** junit -- **type:** xml -- **location:** Specify the directory where the reports will be generated. - -```yaml -report: true -partialReports: -location: target/surefire-reports/html -type: xml -frameworkName: junit -``` - -## Version 2.5.4 - -### Browser and Selenium Version Updates - -- **Edge Browser :** Now updated to `Edge version-129`, ensuring compatibility with the latest browser features and optimizations. -- **Selenium JAR :** Upgraded to `Selenium 4.25.0`, offering enhanced automation capabilities and support for the newest web standards. - -**Browser Versions** -- **Edge (Latest) :** Now set to `Edge-latest-129` -- **Chrome (Latest) :** Updated to `Chrome-latest-128` -- **Firefox (Latest) :** Now at `Firefox-latest-129` - -These updates enhance test reliability and support for modern browser functionalities, ensuring seamless cross-browser testing. - -## Version 2.5.3 - -### New Feature: Commit Information Display - -The top panel now displays detailed information about the latest commit, including the commit ID and the contributor's name. This enhancement provides better visibility into the latest code changes and helps developers track contributions more efficiently. - -## Version 2.5.2 - -### 1. Cypress Detailed Command Logs for Enhanced Debugging -**Detailed Cypress Command Logs** in HyperExecute generate an extensive, human-readable record of all Cypress commands and their corresponding results, both in the console and as a file. It helps narrow down test logs, making debugging and troubleshooting Cypress tests easier. - -> 📕 Learn how to enable [Detailed Command Logs](/support/docs/cypress-detailed-command-logs/) for your Cypress tests. - -### 2. Parameterized Report Email Handling in YAML -HyperExecute now supports an enhanced approach for managing multiple email addresses within the YAML configuration file. Previously, you needed to define multiple variables for email addresses (`${email1}`, `${email2}`, etc.). With this update, a single variable can now hold multiple email addresses, separated by commas or underscores, simplifying report sharing. This enhancement eliminates the need for multiple variables, making maintaining and modifying email configurations easier. - -> 📕 Check the [Reports documentation](/support/docs/hyperexecute-email-reports/#how-to-dynamically-set-your-email-address) to learn more about it. - -### 3. Browser and Selenium Updates for Linux, Windows, and macOS -Updated the browser versions across multiple platforms and upgraded the Selenium jars. This ensures compatibility with the latest features and security updates, improving test stability across different environments. - -The following updates are now live: - -- **Firefox:** version 129.0 -- **Chrome:** version 128.0 -- **Selenium Jars:** version 4.24 - -## Version 2.5.1 - -### HyperExecute Projects - General Availability (GA) - -HyperExecute Projects is now generally available (GA), moving beyond the Beta phase! Projects offer a centralized platform for organizing and managing your test executions to streamline your testing process more efficiently than ever before. - -### Key Updates: - -- **Centralized Test Management :** Group similar tests, control access, and track progress within custom projects. -- **Workflow Automation :** Set up automated workflows with scheduled test executions and optional workflow linking. -- **GitHub Integration :** Support for GitHub repositories with easy setup for collaborative testing. -- **Framework Support :** Integration with Tosca, Tosca DEX, and JMeter for unified test management. - -> 📕 Read the [documentation](/support/docs/hyperexecute-projects/) to learn more about it. - -### New Version Release for Beta and Dev Edge Browser -HyperExecute expands its support for the latest versions of dev and beta browsers for the Windows platform. With this expanded compatibility, you can streamline your testing process and deliver exceptional user experiences with confidence! - -- **Dev version:** 130.0 -- **Beta version:** 128.0.2739.42 - -## Version 2.5.0 - -### HyperExecute - New Version Release for Beta and Dev Edge Browser -HyperExecute expands its support for the latest versions of dev and beta browsers for the Windows platform. With this expanded compatibility, you can streamline your testing process and deliver exceptional user experiences with confidence! - -- **Dev version:** 129.0 -- **Beta version:** 128.0.2739.22 - -## Version 2.4.8 - -### HyperExecute: Set Dynamic Email Address for Report Sharing -HyperExecute CLI added an enhancement in the [`--vars`](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/#--vars) flag providing greater flexibility in specifying email addresses for report and artifact sharing. You can now use a variable to dynamically set the email address used to share reports or artifacts. This gives you more flexibility than hardcoding the email address in the YAML configuration file. - -```yaml title="hyperexecute.yaml" -partialReports: -location: target/surefire-reports/html -type: html -frameworkName: extent -# highlight-start -email: -to: -- "${email}" -- "${email1}" -# highlight-end -``` - -Pass the value of your email address via CLI by running the command - -```bash -./hyperexecute --vars "email=xyz@abc.com" --vars "email1=abc@xyz.com" -``` - -> 📕 Read the documentation - [How to dynamically set your email address?](/support/docs/hyperexecute-email-reports/#how-to-dynamically-set-your-email-address) to learn more about it. - -## Version 2.4.7 - -### HyperExecute: Accessibility Testing Support for Selenium and Cypress - -HyperExecute now supports Accessibility Testing for both Selenium and Cypress! This update enables you to ensure that your web applications comply with accessibility standards such as **WCAG 2.2 AA**. - -### Benefits of Accessibility Testing in HyperExecute -With this feature, you can seamlessly validate a range of accessibility issues to create more inclusive and user-friendly web applications. Here's how it benefits you: - -- **Automated Issue Detection:** Identify common accessibility problems such as missing alt attributes, incorrect ARIA roles, and insufficient color contrast, early in the development process. -- **Detailed Accessibility Reports:** Receive comprehensive reports that pinpoint accessibility issues, with actionable recommendations for resolving them. -- **Improved Compliance:** Ensure your applications adhere to critical accessibility guidelines like **WCAG 2.2 AA** or previous versions: WCAG 2.0 A, WCAG 2.0 AA, WCAG 2.1 A, WCAG 2.1 AA, WCAG 2.1 AAA, and WCAG 2.2 A, helping you avoid legal risks and create a better experience for all users. - -### How to Use: -- **For Selenium Users:** -To start accessibility testing with Selenium, you have to pass `"accessibility", true` capability in your test files. -> 📕 Check out our detailed guide on [HyperExecute integration: Selenium accessibility automation](/support/docs/selenium-hyperexecute-accessibility-tests/). - -- **For Cypress Users:** -To start accessibility testing with Cypress, you have to pass `accessibility: true` capability in the [cypressOps](/support/docs/deep-dive-into-hyperexecute-yaml/#cypressops) flag of your YAML file as well as update other necessary configurations in your project. - -> 📕 Follow our step-by-step documentation to set up and run accessibility checks in [Cypress v10](/support/docs/cypress-v10-hyprerexecute-accessibility-tests/) or [Cypress v9](/support/docs/cypress-v9-hyprerexecute-accessibility-tests/) on HyperExecute. - -## Version 2.4.6 - -### HyperExecute CLI `--runson` flag - -The HyperExecute CLI now supports a [`--runson`](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/#--runson) flag, allowing you to override the [`runson`](/support/docs/deep-dive-into-hyperexecute-yaml/#runson) value specified in the YAML configuration file. - -**Key Values** -- **OS Override**: Overrides the OS value defined in the YAML file's `runson` flag -- **Multi-Mode Support**: Compatible with all execution modes: [AutoSplit](/support/docs/hyperexecute-auto-split-strategy/), [Matrix](/support/docs/hyperexecute-matrix-multiplexing-strategy/), and [Hybrid](/support/docs/hyperexecute-hybrid-strategy/). -- **Multi-OS Specification**: Enables defining multiple operating systems for Matrix and Hybrid modes. - -**Usage** -Add the `--runson` flag to your HyperExecute CLI command, followed by the desired OS: - -```bash ---runson YOUR_OS_NAME -``` - -> 📕 Read the documentation - [HyperExecute CLI `runson`](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/#--runson) to learn more about it. - -## Version 2.4.5 - -### Selenium Version Support Expanded -HyperExecute now supports **Selenium versions 4.16 to 4.21**. This update ensures broader options for test automation frameworks and provides access to the latest Selenium features. - -### Firefox Browser Updated -To maintain browser compatibility and access the newest features, we've incorporated support for **Firefox version 128**. This update guarantees reliable testing on the latest Firefox releases. - -## Version 2.4.4 - -### HyperExecute now integrates with k6 - -HyperExecute now supports k6, a powerful open-source performance testing tool. HyperExecute supports all versions of k6 (till 0.52). To use any particular version, all you have to do is mention that specific version in the [runson flag](/support/docs/deep-dive-into-hyperexecute-yaml/#runson) in your [HyperExecute YAML](/support/docs/deep-dive-into-hyperexecute-yaml/) file. - -```yaml -runtime: -addons: -- name: k6 -version: "v0.52.0" -``` - -Also, add these environment variables in your YAML file to install the necessary dependencies for your tests - -```yaml -env: -K6_BROWSER_ENABLED: true -K6_BROWSER_HEADLESS: false -HE_CONTEXT_K6_SETUP_DEFAULT_BROWSER_PATH: true -``` - -📕 Check the [k6 integration documentation](/support/docs/hyperexecute-k6-testing/) to learn more about it. - -## Version 2.4.0 - -### Debug your Cypress Tests with Network HAR Logs - -Now gain deeper insights into your application's network activity during Cypress testing with Network HAR Logs. - -This feature is applicable on all versions of the Cypress - -Benefits of Network HAR Logs: - -- **Streamlined Debugging:** Isolate network-related issues with greater ease by analyzing request behavior and response details. -- **Performance Optimization:** Identify performance bottlenecks by pinpointing slow requests and optimizing network interactions. -- **Improved Load Order Diagnosis:** Gain insights into load order issues that might affect your application's functionality. - -📕 Check the [Cypress documentation](/support/docs/cypressv10-on-hyperexecute/#har-logs) to learn more about it. - -## Version 2.3.8 - -### Location Support for TimeZone Capability 🚀 - -HyperExecute has released an update for the location-based timezone capability. - -With this exciting update, you can now use location names (e.g., "Kolkata") alongside UTC offsets. This empowers you to set precise timezones based on a user's location, ensuring a more seamless user experience. - -**note** -This capability will work with Selenium, Playwright, Puppeteer, Cypress, and Appium, except for Espresso and XCUI frameworks. - -Previously, you might have needed to use: -```yaml -capabilities.setCapability("timezone", "UTC+05:30"); -``` - -Now, you can simply set: -```bash -capabilities.setCapability("timezone", "Kolkata"); -``` - -📕 Learn how to use [timezone capability](/support/docs/selenium-automation-capabilities/#lambdatest-advanced-selenium-capabilities:~:text=Name%20not%20exists-,timezone,-Example%3A%20UTC%2B07) - -## Version 2.3.7 - -### Improved Data Handling for Distributed Tests - -[`dataJsonBuilder`](/support/docs/deep-dive-into-hyperexecute-yaml/#datajsonbuilder) flag provides you a more efficient way to manage JSON data for distributed testing scenarios. - -It empowers you to: - -- **Read JSON data:** Specify the path to your JSON file using the `path` configuration. -- **Filter data (optional):** Employ the filters configuration to target specific data based on user-defined criteria. - - Leverage queries for complex filtering logic. - - Utilize attributes for simpler filtering with higher precedence over queries. -- **Index JSON objects**: Enhance distribution efficiency by indexing objects using a chosen key with the index configuration. - -## Version 2.3.6 - -### HyperExecute Expands Browser Support for Faster, More Comprehensive Testing! 🚀 - -HyperExecute expands its support for the latest versions of browsers for all operating systems: **Linux, Windows**, and **macOS**. Now you can ensure flawless functionality across a wider range of environments with support for the latest versions of these popular browsers: - -- Chrome: 125 -- Edge: 125 -- Firefox: 126 - -With this expanded compatibility, you can streamline your testing process and deliver exceptional user experiences with confidence! - -## Version 2.3.3 - -### Enhanced Reporting with Multi-Framework Support - -You can now create multiple reports from different frameworks for your job. This allows you to: - -- **Gain Framework-Specific Insights:** Instantly identify performance metrics and potential issues unique to each framework. - -- **Simplify Analysis:** Focus on the results that matter most without sifting through a combined report. Explore framework-specific details and pinpoint areas for improvement with ease. - -- **Enhanced Organization:** Maintain a clear and organized overview of your entire test suite. Separate reports provide a structured view, allowing you to efficiently track the performance of each framework. - -```bash -partialReports: -- location: reports/json -type: json -frameworkName: extent-native -email: -to: -- johndoe@example.com -- location: target/surefire-reports -type: html -frameworkName: testng -email: -to: -- johndoe@example.com -``` - -> 📕 Read the documentation - [Test Reports](/support/docs/hyperexecute-reports/) to learn more about it. - -## Version 2.3.1 - -### Mask Your Sensitive Data - -Test recordings often capture all user interactions, including potentially sensitive information like credentials, geo-location, storage-state etc. This can be a security risk if the recordings are accidentally exposed. Hence to overcome this limitation, HyperExecute releases a new feature **Lambda Masking**. - -### Benefits of Masking - -- **Enhanced Security:** Masking allows you to automatically hide sensitive data in your recorded test commands, mitigating the risk of exposure. - -- **Improved Compliance:** This feature can help you adhere to stricter data security regulations and best practices. - -📕 Check the documentation - [Playwright Capabilities](/support/docs/capabilities-for-playwright/) to learn more about it. - -## Version 2.3.0 - -### Test Summary Hyperlinks for Enhanced Debugging Efficiency - -[Test Summary Hyperlinks](/support/docs/hyperexecute-guided-walkthrough/#test-summary-hyperlink) provide a seamless redirection from your test summary to detailed test information within the automation dashboard. This eliminates the need for time-consuming navigation, allowing you to focus on what matters most - analyzing results and optimizing your tests. - -### Benefits: - -- **Accelerated Debugging:** Effortlessly transition from high-level summaries to granular test data, streamlining your troubleshooting process. -- **Improved Workflow Efficiency:** Minimize time spent navigating between screens and maximize focus on actionable insights. -- **Enhanced User Experience:** Experience an intuitive workflow that prioritizes clear visibility into your test results. -Start Testing 🚀 - -## Version 2.2.9 - -HyperExecute now supports multiple versions of Katalon. You can switch to any version of Katalon for your desired use case. - -Add a runtime flag to your YAML file and define your required version. - -```bash -runtime: -- language: katalon -version: 8.6.5 -``` - -📕 Refer to our [runtime YAML flag](/support/docs/deep-dive-into-hyperexecute-yaml/#runtime/) to know more! - -## Version 2.2.8 - -With the all-new Team Management feature, enhance team collaboration across your organization's automation testing efforts. By enabling isolated testing environments, this functionality streamlines workflow and improves productivity within teams. - -Here's why Team Management can be a game changer for your automation testing efforts: - -- **Organized Test Management:** Easily group users and their corresponding tests based on specific projects, teams, or departments to keep your testing environment structured and efficient. - -- **Improved Visibility and Control:** Administrators can now manage user access to specific test data, ensuring that sensitive or confidential tests are protected from unauthorized access. - -- **Enhanced Team Collaboration:** This feature fosters better communication and knowledge sharing within teams, enabling a more integrated and informed testing process. - -📕 Refer to our comprehensive [Team Management documentation](/support/docs/team-management/) to know more! - -## Version 2.2.7 - -### Effortless Visual Testing with HyperExecute - -HyperExecute integrates seamlessly with SmartUI SDK to deliver effortless visual testing for your web and mobile apps. Ensure pixel-perfect rendering across browsers, devices, and platforms, saving you valuable time and resources. - -You can seamlessly integrate SmartUI with your favorite frameworks - Selenium, Playwright, Puppeteer, Cypress, or Appium. - -📕 Refer to our comprehensive documentation - [HyperExecute with SmartUI SDK](/support/docs/hyperexecute-smart-ui-sdk-selenium-javascript/) - -### Support for Chrome and Edge Beta Versions 🧑‍💻 - -HyperExecute now supports the beta versions of Chrome and Edge, allowing you to run your Cypress tests seamlessly on our blazing-fast platform. - -## Version 2.2.6 - -### Support for Native TestNG Test Reports - -HyperExecute now supports the Native TestNG Reports that offer a standardized and easily accessible summary of information extracted from raw Extent reports per Virtual Machine (VM) at the end of a HyperExecute job. - -> Read the [Native TestNG documentation](/support/docs/hyperexecute-native-testng-report/) to understand how to generate it. - -## Version 2.2.5 - -### Improved Secrets Management 🔐 -You can now specify a dedicated secrets file containing sensitive information like credentials and API keys. - -These secrets within the designated file are accessible only within the scope of that particular job execution. This significantly reduces the risk of accidental exposure and improves overall security. - -📗 Refer to our comprehensive [documentation](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/#--job-secret-file) for a better understanding. - -### Microsoft Edge Canary Support for macOS 🚀 -HyperExecute users running macOS can now leverage Microsoft Edge Canary for testing. - -## Version 2.2.3 - -### Local Selenium Driver Support 🧑‍💻 - -HyperExecute now natively supports local Selenium drivers. This means you can run your tests without having to set up and use a remote Webdriver. This simplifies the setup process and can improve test execution performance. - -> 📕 Refer to the detailed [documentation](/support/docs/hyperexecute-how-to-run-tests-using-local-selenium-drivers/). - -### Playwright Real Device Support 📲 -HyperExecute now supports running Playwright tests on real devices. This allows you to test your web applications on actual devices, ensuring they function correctly in real-world scenarios. - -> 📘 Read the [documentation](/support/docs/playwright-real-device-on-hyperexecute/) to understand the step-by-step guidance. - -### Smart UI Testing in HyperExecute using Playwright - -HyperExecute now supports integration with SmartUI for your Playwright tests. This allows you to use SmartUI's visual verification capabilities to ensure that your web applications are visually consistent and functional. - -> :green_book: Check the [documentation](/support/docs/hyperexecute-smart-ui-test-using-playwright/) to trigger your tests. - -## Version 2.2.2 - -### Testsigma Integration - -Testsigma is an AI Native, cloud-based test automation platform designed to help testers create, run, and maintain automated tests for various applications. It caters to web, mobile, and API testing, aiming to simplify the automation process for users of all experience levels. - -> Refer to the detailed documentation on how to integrate [Testsigma](/support/docs/hyperexecute-testsigma-integration/) with HyperExecute. - -## Version 2.2.1 - -### Configure the behavior of Error Categorization of Your Job!! - -The `errorCategorizedOnFailureOnly` flag lets you control when error categorization happens after a job runs. Normally, errors are categorized for every stage, even if the stage succeeded. Enabling this flag ensures that errors are only categorized for stages that failed (not green). - -> Refer to the documentation for more details on the [errorCategorizedOnFailureOnly](/support/docs/deep-dive-into-hyperexecute-yaml/#errorcategorizedonfailureonly) - -## Version 2.2.0 - -### Speed Up Your Development Workflow ⚡ - -Supercharge your development workflow by significantly reducing codebase upload times, especially when you're making frequent, incremental changes. - -**Key benefits of `differentialUpload` Flag:** - -- **Lightning-fast Uploads :** Experience upload times faster for incremental changes. ⚡️ - -- **Boost your Productivity :** Spend less time waiting for uploads and more time developing. 🧑‍💻 - -- **Efficient Resource Utilization :** Reduce strain on your network and platform resources. 👨🏻‍💻🧑‍💻 - -> 📕 Check the documentation of [differentialUpload](/support/docs/deep-dive-into-hyperexecute-yaml/#differentialupload) flag - -### Enhance your UI Testing Experience - -You can selectively exclude specific elements from visual comparisons during your automated tests. - -**Advantages of `ignoreDOM` flag** - -- **Handling Dynamic Content :** Exclude elements that frequently change, like timestamps or user-generated content, to avoid false positives. - -- **Focusing on core functionality :** Zero in on critical elements by ignoring UI components like headers, footers, or navigation menus. - -- **Testing Specific Sections :** Target a particular area of your webpage for more granular testing. - -> 📕 Learn more about the [ignoreDOM](/support/docs/hyperexecute-smart-ui-testing-using-selenium/#smart-ui-test-keywords) flag - -## Version 2.1.6 - -### Improved test execution flow with `alwaysRunPostSteps` flag - -The `alwaysRunPostSteps` flag provides finer control over post-step execution. When enabled, this flag ensures that all post-steps within a stage always execute, regardless of whether any preceding test scenarios within that stage fail. This guarantees the completion of critical post-execution actions, regardless of the test execution outcome. - -This enhancement streamlines your testing workflow by ensuring essential post-steps are consistently executed, even in the presence of failing test scenarios. - -```bash -alwaysRunPostSteps: true -``` - -> 📕 Check the documentation to understand more about the [alwaysRunPostSteps](/support/docs/deep-dive-into-hyperexecute-yaml/#alwaysrunpoststeps) flag - -## Version 2.1.5 - -### Optimize Maven Runtime Integration for Enhanced Project Development 🧑‍💻 - -Optimize your development workflow with HyperExecute's streamlined Maven runtime integration. Benefit from seamless integration and performance enhancements designed specifically for Maven-driven projects, refining your execution processes to increase efficiency. - -> Check the [documentation](/support/docs/deep-dive-into-hyperexecute-yaml/#runtime) for detailed understanding. - -### Refined Test Control Through Regex Retry Option ⚙️ - -Regular expressions offer precise control, allowing you to fine-tune when test retries are triggered. This allows you to precisely define scenarios where retests are triggered, optimizing resource utilization and ensuring targeted test execution. - -> For more details, Check the documentation on [retryOptions](/support/docs/deep-dive-into-hyperexecute-yaml/#retryoptions) flag. - -### Simplified Artifact Naming with Matrix Parameter Values 📝 - -Artifact management is simplified by allowing you to replace artifact names with browser parameter values. This enhancement ensures consistency and clarity in your project, making it easier to manage and track artifacts associated with different parameters. - -> Go through the documentation to learn [how to change the artifacts folder name value](/support/docs/hyperexecute-artifacts/#:~:text=You%20can%20replace%20the%20Artifacts%20folder%20name%20value%20with%20any%20of%20the%20Matrix%20parameters%20value.) - -## Version 2.1.3 - -### Beta Browser Support in HyperExecute 🚀 - -HyperExecute now supports beta browsers for the **Windows Platform!** 🎉 - -You can now test your web applications on the latest beta versions of popular browsers for the Windows platform, ensuring compatibility and catching potential issues early on. - -**Benefits Offered:** - -- **Beta Browser Deployment:** Easily deploy beta browsers on HyperExecute, just as you would stable browsers. - -- **Expanded Testing Coverage:** Expand your testing coverage to include beta browser versions, identifying potential compatibility issues before they impact your users. - -## Version 2.1.1 - -### Email Delivery of HyperExecute Job Artifacts - -You can now receive [Job Artifacts](/support/docs/hyperexecute-artifacts/) directly via email, streamlining the process of accessing and managing your execution results. - -To receive the Artifacts via mail, you simply need to pass a `email` flag in your YAML file. - -```bash -uploadArtifacts: -... -email: -to: -- -- -``` - -> Refer to the detailed documentation on [how to receive artifacts via email](/support/docs/hyperexecute-email-reports/#how-to-receive-your-artifacts-via-email) diff --git a/static/docs/hyperexecute-release-notes-2025.md b/static/docs/hyperexecute-release-notes-2025.md deleted file mode 100644 index 9ae09d1eb..000000000 --- a/static/docs/hyperexecute-release-notes-2025.md +++ /dev/null @@ -1,632 +0,0 @@ -# HyperExecute Release Notes 2025 - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Version 3.1.7 -### Lighthouse Report Support with Playwright on HyperExecute -HyperExecute now supports generating Lighthouse reports using Playwright, enabling teams to measure and track web performance, accessibility, SEO, and best practices directly within their Playwright test workflows. - -> Learn more about [Lighthuouse PlayWright Reports](/support/docs/playwright-lighthouse-library/) - -## Version 3.1.6 -### Chrome for Testing Browser Support -HyperExecute now supports Chrome for Testing, enabling more reliable and version-controlled browser testing across automation workflows. - -**Why This Matters** -Traditional Chrome auto-updates can introduce test instability due to unexpected version changes. Chrome for Testing provides: -- Predictable browser versions -- Better alignment with corresponding ChromeDriver versions -- More stable and debuggable test runs in CI/CD pipelines - -**How It Helps You** -- Run tests against specific Chrome versions without surprises -- Reduce flaky failures caused by browser mismatches -- Maintain consistency across local, staging, and CI executions - -## Version 3.1.5 -### Workflows for JMeter Projects -HyperExecute now supports Workflows for JMeter projects, enabling teams to orchestrate, automate, and manage performance test execution more efficiently within CI/CD pipelines. - -**Why This Matters** -Performance testing often needs more than a single test run. Workflows allow you to: - -- Improve repeatability and reliability of load and performance testing -- Align JMeter tests with real-world CI/CD practices - -**How It Helps You** - -- Reduces manual intervention in performance testing -- Improves visibility and control over JMeter executions in HyperExecute - -> Learn more about Workflows from the [documentation](/support/docs/hyperexecute-workflows/). - -## Version 3.1.4 -### Enhanced Support for Azure Repositories in HyperExecute Projects -We have introduced an enhancement to the HyperExecute [Custom Project feature](/support/docs/hyperexecute-projects/#setup-custom-project) to support Azure Repos. This enables you with code repositories hosted in Azure DevOps to execute tests directly on HyperExecute, without relying on Jenkins or any external CI/CD tool. - -With this update, you can now configure Azure Repos in the same intuitive manner as GitHub, or Bitbucket. Simply provide the repository details and Personal Access Token (PAT), and HyperExecute will fetch the code and trigger workflows seamlessly. - -> 📘 **Read More:** Learn more about configuring Azure Repos, project setup, workflows, and job triggers in the [HyperExecute documentation](/support/docs/hyperexecute-projects/). - -## Version 3.1.2 -### Rerun Failed Tests on HyperExecute -Quickly validate fixes without rerunning your entire test suite. With the new Rerun Failed Tests feature, you can relaunch only the failed scenarios or the full suite directly from the dashboard, drastically reducing execution time and improving feedback cycles. Reruns use the same YAML configuration for consistent, reproducible results, with options to pick the same or latest commit and adjust concurrency. - -> **Learn more:** Refer to the detailed [Rerun Failed Tests documentation.](/support/docs/hyperexecute-rerun-failed-tests/) - -### Snooper Compatibility Update -Snooper is not compatible with Alpine-based Docker images because Alpine uses musl libc instead of glibc, which Snooper requires. To avoid runtime errors or unexpected failures, ensure your images are built on a non-Alpine base that includes glibc. This helps maintain consistent behavior when updating or troubleshooting Docker environments. - -> **Learn more:** Refer to the [Snooper documentation](/support/docs/hyperexecute-snooper/). - -### View & Verify Code-Signed HyperExecute CLI Binaries -You can now easily verify the authenticity of downloaded HyperExecute CLI binaries using platform-specific code-signing checks. Each release is digitally signed to ensure integrity and trust, with clear instructions for Linux, macOS, and Windows users to validate the signature or certificate before use. - -> **Learn more:** Refer to the [Code Signing Verification Guide](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/#verifying-the-authenticity-of-the-cli) - -## Version 3.1.0 -HyperExecute now supports **Katalon Reports** generation, including all new report formats introduced in Katalon Studio 10 and above with the following support: -- **Seamless Katalon Report Integration:** You can now generate Katalon HTML reports directly within your HyperExecute workflow. -- **Support for Latest Report Structure:** HyperExecute is fully compatible with the new report schema and format updates introduced in Katalon Studio 10+. - -> 📘 Refer to our [Katalon Report Generation Guide](/support/docs/hyperexecute-katalon-report/) for a detailed setup walkthrough and examples. - -## Version 3.0.8 -### Job Insights View in HyperExecute -With the Job Insights View, you can now access a comprehensive overview of all your test results across different configurations, all in one unified interface, Even when tests are split across multiple environments or parallel jobs, HyperExecute intelligently aggregates the data, giving you a complete, consolidated view of your job performance. - -This feature empowers you to: - -- **Compare results across configurations** effortlessly. -- **Review job summaries and outcomes** in a single glance. -- **Analyze historical job runs** within the same project to identify patterns, regressions, or performance drifts quickly. - -The **Job Insights View** is especially valuable for teams running the same tests across multiple combinations (such as different browsers, OS versions, or device types). It helps you pinpoint inconsistencies and regressions instantly, enabling faster decision-making and smoother test management. - -## Version 3.0.6 -### Feature: HyperExecute Logs Highlights -Debugging large test executions often generates extensive logs, making it difficult to identify key issues. The new **Logs Highlights** feature enhances log readability by automatically highlighting and grouping critical entries. - -HyperExecute categorizes logs into Errors and Warnings using intelligent pattern detection based on regular expressions (regex). This will help you quickly locate and review the most relevant log lines during test analysis. - -## Version 3.0.5 -### Feature: syncStart -HyperExecute now supports the `syncStart` directive in the job YAML. When enabled, all VMs in `autosplit` or `matrix` mode start their tasks simultaneously after allocation, ideal for benchmarking or comparative test runs. - -If some VMs are not ready within 15 minutes, the job starts with the available ones. - -> **NOTE :** For detailed usage and examples, refer to the detailed [HyperExecute documentation](/support/docs/deep-dive-into-hyperexecute-yaml/#syncstart). - -## Version 3.0.4 -### New Capability: `bypassProxyDomains` -You can now use the `bypassProxyDomains` capability to bypass Dedicated Proxy for specific domains. This ensures that local services (e.g., localhost, 127.0.0.1, or internal endpoints) remain accessible during test runs. - -Note: `dedicatedProxy: true` must be enabled for this capability to take effect. - -> Visit the [bypassProxyDomains documentation](/support/docs/bypass-proxy-domains/) page to understand this capability in detail. - -### Live Command Logs -We've improved the way command logs are displayed in the HyperExecute dashboard. Earlier, logs appeared in chunks, which delayed visibility and created a less interactive experience. With the new **Live Command Logs**, you'll now see logs in real-time as they are generated, giving you instant feedback during test execution. - -## Version 3.0.3 -### MITM Support in Emulators -We've added Man-in-the-Middle (MITM) proxy support in HyperExecute Emulators. - -- You can now capture network logs directly from emulator sessions using MITM. -- This feature is available for all emulators when configured with a flag that you can add in your `hyperexecute.yaml` file: - -```yaml title="hyperexecute.yaml" -framework: -name: raw -args: -mitmProxy: true -``` - -This enhancement enables deeper debugging of API calls, request/response payloads, and overall network traffic during test execution. - -### Enhancement: Retry Visibility in HyperExecute Reports -We've enhanced the HyperExecute Report to give better visibility into retries at both the summary and test level. - -- **Total Counts Refined :** The "No. of Scenarios" and "No. of Tests" metrics will now display values excluding retries. -- **Retry Indicators :** A retry icon and the number of retried tests/scenarios are displayed alongside the counts, similar to the Task Summary section. -- **Test Case Details :** In the "Test Cases" view, retried tests are now clearly marked with a retry icon. - -This update ensures accurate reporting and makes it easier to differentiate between unique executions and retries. - -### Muted Test Count & Bulk Unmute Option -We've enhanced the Test Muting functionality in HyperExecute with improved visibility and control: - -- **Total Muted Test :** The dashboard now shows the number of muted test cases in your suite. -- **Unmute All :** You can now unmute all tests at once instead of manually unmuting them one by one. - -This update streamlines test management and makes it easier to re-enable muted tests when they are ready to be executed again. - -## Version 3.0.1 -### Workflow Variables on HyperExecute -With Workflow Variables, you can now customize your workflow execution. This makes it easier to reuse the same workflow across multiple scenarios without editing YAML or creating duplicate workflows. - -- **Workflow-level Variables :** Define reusable variables (e.g., ENVIRONMENT, BROWSER) while creating or editing workflows. -- **Default & Multiple Values :** Assign defaults or provide multiple values per variable. -- **Runtime Overrides :** Update variable values at run time without changing saved defaults. - -> To learn more about Workflow Variables on HyperExecute, refer to our [detailed documentation](/support/docs/hyperexecute-workflows/#workflow-variables) - -## Version 3.0.0 -### Language Support for Android Real Device Maestro Framework - -We have introduced language support for Maestro tests on Android devices in HyperExecute. Now, you can configure the language parameter directly in your HyperExecute YAML under the [framework](/support/docs/hyperexecute-yaml-version0.2/#framework) flag. - -This enhancement is particularly useful when your testing applications in non-English environments. For example, you can now run tests in Spanish by setting: - -```yaml title="hyperexecute.yaml" -framework: -name: raw -args: -language: es -locale: ES -``` - -### BiDi Testing Support with Selenium WebDriver -HyperExecute now supports BiDi (Bi-Directional) Testing with Selenium WebDriver. BiDi provides a two-way communication channel between the client and the browser, allowing you to go beyond traditional WebDriver commands. With BiDi enabled, you can interact with real-time browser events such as console logs, network traffic, JavaScript exceptions, and more, all directly from your test scripts. - -> To learn more about BiDi testing with Selenium WebDriver on HyperExecute, refer to our [detailed documentation](/support/docs/hyperexecute-bidi-testing/) - -## Version 2.9.9 -### Emulator Support with Chrome for Browser Automation -We are excited to announce emulator support for Chrome browser automation on HyperExecute. This initial release introduces emulator functionality exclusively for Chrome, providing enhanced flexibility for testing mobile browser-based workflows. - -**Key Features Included** - -- **Command Logs** – Track and debug automation commands executed during test runs. -- **Video Recording** – Access full session recordings for detailed test analysis. -- **Screenshots** – Capture screenshots at various stages of execution for validation. -- **Comprehensive Logs** – Access Appium, Console, Network, and Device logs for improved troubleshooting. - -### Supported Environments -Currently available on **Android 11** and **Android 12** emulators. More versions and browsers will be supported in future releases. - -👉 You can check detailed device support and capabilities at:[ TestMu AI Capabilities Generator](https://www.testmuai.com/capabilities-generator/) - -## Version 2.9.8 -### SmartWait Support on HyperExecute -We are pleased to announce that the SmartWait functionality is now supported on HyperExecute. - -SmartWait intelligently manages wait times by performing actionability checks before executing actions on webpage elements. This ensures that actions are only carried out when elements are ready, improving both accuracy and efficiency in Selenium automation. - -With SmartWait on HyperExecute, you can: - -- Reduce reliance on explicit and implicit waits. -- Optimize test scripts for readability and maintainability. -- Execute tests with improved reliability in parallel and distributed environments. - -> 📌 Learn more about configuring SmartWait: [SmartWait Documentation](/support/docs/smart-wait/) - -## Version 2.9.7 -### HyperExecute Security Update - -We have implemented important security enhancements across HyperExecute to strengthen the overall protection of your test execution environment. This update includes: - -- Application of the latest security patches to address identified vulnerabilities. -- Improved handling of session authentication and access controls. -- Strengthened security controls to ensure a safer execution environment. - -These updates ensure that your test executions run in a more secure and reliable environment, with improved safeguards against potential risks. - -## Version 2.9.6 -### Regex Support in `partialReports` Parameter -You can now define dynamic or variable folder structures for report generation using **wildcard (`**`)** based regex patterns in the `location` parameter under `partialReports`. This allows HyperExecute to automatically discover and upload test reports from non-static, dynamically generated directories, eliminating the need to manually move reports to a fixed path. - -### Problem It Solves -Previously, HyperExecute required a static path to locate and upload test reports for frameworks like **Cucumber**. Customers generating test data in nested or date-based folders (e.g. reports/2025-07-29/cucumber/) encountered "path not found" errors, breaking the report generation process. - -### New Behavior -HyperExecute now supports regex-style directory matching (using **) to recursively search for report files in nested folders. This flexibility ensures that test reports are picked up correctly even if the folder structure changes between runs. - -### Supported Syntax Example - -```yaml title="hyperexecute.yaml" -partialReports: -location: reports/**/cucumber/ -frameworkName: cucumber -type: json -``` - -This will successfully match paths like: - -```yaml -reports/2025-07-29/cucumber/ -reports/staging/run_001/cucumber/ -reports/builds/test_42/cucumber/ -``` - -> Your Testrunner.java file should have the complete path from where partial reports will be added. Example: -> ```javascript title="Testrunner.java" -> plugin = { -> "json:target/abc/mno/pqr/xyz/cucumber-reports/CucumberTestReport.json", -> "json:target/abc/its/pqr/abc/cucumber-reports/CucumberTestReport.json" -> } -> ``` -> You can configure your hyperexecute.yaml as: -> ```yaml title="hyperexecute.yaml" -> partialReports: -> location: target/abc/**/pqr/**/cucumber-reports -> frameworkName: cucumber -> type: json -> ``` - -## Version 2.9.5 -### Playwright Real Device iOS Support on HyperExecute -You can now run Playwright tests on real iOS devices using HyperExecute. This unlocks the power of native device testing with the speed and flexibility of Playwright on our real device cloud. - -**To enable this mode:** -- Set `runson: ios` in your YAML. -- Use the following under `framework` flag: - -```yaml title="hyperexecute.yaml" -runson: ios - -framework: -name: appium -args: -playwrightRD: true -mobileDC: true -``` - -> **Note:** Currently supported on Safari for iOS real devices. - -## Version 2.9.4 -### YAML-Based Capability Overrides for Selenium Tests -You can now override Selenium capabilities directly from your `hyperexecute.yaml` file using the new `ltOptions` flag. This enhancement allows you to define key-value pairs, like browser type, version, logs, video, tunnel, and more without changing your test scripts. - -**Why it matters?** -- Avoid code changes for environment-specific needs. -- Reuse the same script across different setups. -- Keep test code clean and capability-agnostic. - -```yaml hyperexecute.yaml -ltOptions: -browserName: firefox -build: 'testng_build_yaml_ltOptions' -console: true -name: 'testng_test_yaml_ltOptions' -network: true -selenium_version: 4.20.0 -tunnel: false -version: latest -video: false -``` - -> 📘 Refer to our detailed documentation for [`ltOptions`](/support/docs/hyperexecute-yaml-capability-overrides/) flag. - -### Network Logs & Tunnel Support for Maestro Framework -HyperExecute now supports `network logs` and `tunnel` capabilities for Maestro tests running on both virtual devices and real devices using the Raw Framework configuration. - -**Feature Highlights** -- **Network Logs:** Capture detailed request/response data during test execution for enhanced debugging and visibility. -- **Tunnel Support:** Access internal or staging environments securely using TestMu AI Tunnel. -- Supported across both virtual and real mobile devices. - -> 📘 Refer to our detailed documentation for [`Maestro`](/support/docs/hyperexecute-maestro-testing/). - -## Version 2.9.1 -### Support for `customHeaders` in HyperExecute -You can now inject custom HTTP headers into your test sessions using the `customHeaders` capability. This gives you more control over outbound requests made during test execution, especially when working around network security controls like firewalls or header-based access restrictions. - -```java -caps.setCapability("customHeaders", new HashMap() {{ -put("x-px-access-token", "testuil"); -}}); -``` - -These headers are attached to your outbound HTTP requests and can help: -- Authenticate against secured endpoints -- Pass custom metadata -- Work around network/firewall restrictions - -> 📘 Refer to our detailed documentation, [Custom Headers](/support/docs/custom-headers/) for detailed explanation and examples. - -## Version 2.9.0 -### Git Token Support & Git Validation Improvements in HyperExecute Projects -We've enhanced Projects in HyperExecute to streamline Git integration. You can now also provide a `git_token` instead of PAT Token during project creation for GitHub authentication. If a token isn't provided at request time, HyperExecute will resolve it in the following order: -- Request-level token -- User-level secret -- Organization-level secret - -You can now integrate Enterprise GitHub accounts seamlessly into Projects. - -> 📘 Refer to our detailed documentation for [`Projects`](/support/docs/hyperexecute-projects/). - -## Version 2.8.9 -### Windows 11 Infrastructure Optimizations -Enhanced speed and stability for tests running on Windows 11 environments. - -### Smart UI Improvements for Playwright -Optimized performance and accuracy of Smart UI validations in Playwright-based tests. - -### GPU Acceleration for Emulators -Introduced GPU-based acceleration to significantly boost emulator performance during test runs. - -## Version 2.8.8 -### Platform-Specific Error Codes -Introduced clear and consistent error codes to help you identify and resolve issues faster. - -### Chrome Driver Bug Fixes: -Addressed key issues to improve test reliability and compatibility with the latest Chrome versions. - -### Cypress Fixes for macOS 14 -Resolved execution issues specific to Cypress tests running on macOS 14. - -### Scheduled Workflow Optimizations -Improved the reliability and performance of scheduled test executions. - -## Version 2.8.7 -### Folder Upload Support for JMeter Projects -You can now upload entire folders for JMeter performance tests, making it easier to manage complex test setups. - -### Mobile Test Execution Optimizations -Improved execution speed and stability for mobile tests running on HyperExecute. - -### Bug Fixes in Extent Native Reports -Resolved formatting and rendering issues to ensure more accurate and consistent reporting. - -## Version 2.8.6 -### Slack Integration Enhancements -Improved message formatting, richer test summaries, and better control for smoother collaboration. - -### Video Generation Optimizations -Faster and more reliable test case video generation with smaller file sizes and improved playback quality. - -## Version 2.8.5 -### 🔧 Bug Fixes -- Resolved various stability and performance issues reported by users to ensure a smoother and more reliable experience across the platform. -- Addressed inconsistencies in data rendering and export functionality within the reporting modules. -- Fixed minor UI glitches and alignment issues to enhance usability. - -### 🚀 Platform Enhancements -**Improved Reporting Capabilities:** -- Enhanced data visualization for better insights and clarity. -- Faster report generation and improved data accuracy. -- New filters and customization options for more flexible reporting workflows. -- Backend optimizations were made to boost performance and scalability, particularly under high-load conditions. -- Improved compatibility with third-party integrations to streamline data sharing and analysis. - -> We continue to listen to your feedback and are committed to delivering features and fixes that improve your experience. Stay tuned for more! - -## Version 2.8.4 -### 🚀 New Feature: Enhanced Left-Side Test List View for Cypress -We're excited to announce the launch of a brand-new Left-Side Test List View in the HyperExecute Automation Dashboard, tailored specifically for Cypress test executions. This enhancement is designed to provide users with a more intuitive, informative, and actionable interface for navigating and debugging their Cypress automation builds. - -**🔍 Overview** -The new left-side panel now displays a vertical list of executed Cypress spec files within each build, making it easier than ever to: - -- Understand the status of individual test files at a glance. -- Navigate between test files quickly. -- Debug failed or skipped tests efficiently. -- Monitor execution time per file. -This feature significantly improves user experience by streamlining test result analysis and accelerating troubleshooting. - -**✅ Key Features Implemented** -**1. Test File Listing** -- Each Cypress spec file (e.g., actions.spec.js, tunnel.spec.js) is displayed in a clean vertical list. -- Files are grouped under their corresponding build for better context. -- Sorted by execution start time to reflect chronological order. - -**2. Status Indicators** -- Green checkmark (✔️): Test passed successfully. -- Red cross (✖️): Test failed. -- Yellow clock (⏳): Test is currently running (real-time updates). -- Grey icon (–): Test was skipped or not executed. - -✅ Statuses are clearly visible and consistent across both the Build Summary page and individual Test Run pages. - -**3. Execution Time Display** -- Execution duration shown next to each test file. -- Time format adapts based on length: - - 25s (seconds) - - 2m 59s (minutes and seconds) - -✅ Times like 2m 59s, 35s, and 44s are accurately rendered in UI as expected. - -**4. Search and Filter** -- A dedicated "Search Tests" bar is available at the top of the test list. -- Users can filter test files dynamically by filename for faster navigation. - -**🧪 Use Cases & Benefits** -| Use Case | Benefits | -|----------|----------| -| Quickly identifying failed tests | Streamlined debugging process | -| Monitoring execution time per file | Better performance tracking | -| Searching through long test lists | Improved productivity and usability | -| Viewing real-time test statuses | Enables proactive monitoring during CI/CD | - -**📌 Accessing the Feature** -This feature is enabled by default for all Cypress-based test executions on HyperExecute. Simply navigate to your Automation Dashboard , select a Cypress build, and view the enhanced left-side test list in the Build Summary or individual Test Run views. - -## Version 2.8.2 -### Validate your HyperExecute YAML with ease !! -The [`--validate`](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/#--validate) flag checks the syntax and structure of your YAML file without executing any tests, allowing you to quickly verify if the file is properly formatted. - -### How It Works -When you pass the `--validate` flag along with your YAML file, the CLI will: -- Validate the structure and syntax of the YAML file. -- Return a success or error message indicating whether the file is correct or if there are issues that need attention. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE --validate`} - -## Version 2.8.1 -### Enhanced `failFast` at the Scenario Level -We have made am enhancement to our test configuration: the ability to use [`failFast`](https://www.testmuai.com/support/docs/deep-dive-into-hyperexecute-yaml/#failfast) at the scenario level! - -Previously, `failFast` could only be applied at the test level. Now, with this update, you can configure `failFast` to stop further scenario execution after a specified number of failed scenarios, providing more granular control over your testing process. - -**Key Features:** -- **Granular Control:** Apply failFast to individual scenarios instead of the entire test suite. -- **Easy Configuration:** Simply set the level to scenario in the YAML configuration. -- **Max Test Failures:** Control the maximum number of allowed test failures at the scenario level. - -Example YAML Configuration: - -```yaml -failFast: -maxNumberOfTests: 2 -level: scenario -``` - -> **Note:** You can use `failFast` either at the test level or scenario level, but not both simultaneously. - -This enhancement brings more flexibility to your testing setup, helping you better manage the execution flow and quickly pinpoint issues. - -## Version 2.7.8 -### Default Reporter Configuration in CypressOps -You can now explicitly specify a custom reporter setup using the new `reporterConfigFile` option in [`cypressOps`](/support/docs/deep-dive-into-hyperexecute-yaml/#cypressops). If your Cypress project does not have a custom reporter configured, `cypressOps` will automatically enable command logs using the mochawesome reporter by default. This ensures that test results are always captured in a readable format, without requiring manual setup. - -```yaml -cypressOps: -reporterConfigFile: "reporter_config.json" -``` - -Value of this JSON: - -```json -{ -"reporterEnabled": "mochawesome,mocha-junit-reporter", -"mochawesomeReporterOptions": { -"reportDir": "cypress/results", -"overwrite": true, -"html": false, -"json": true -}, -"mochaJunitReporterReporterOptions": { -"mochaFile": "cypress/results/my-test-output.xml", -"toConsole": true -} -} -``` - -### Setup Runtime Updates -HyperExecute expands its support for the dotnet by introducing the versions `4.7` and `4.8` in the [runtime](/support/docs/deep-dive-into-hyperexecute-yaml/#runtime). - -```yaml -runtime: -- language: dotnet -version: "4.7" -``` - -### Support for Project Capability in CypressOps -You can now define your [Project](/support/docs/hyperexecute-projects/) name and autocreate your project via `ProjectName` capability defined in [`cypressOps`](/support/docs/deep-dive-into-hyperexecute-yaml/#cypressops). - -```yaml -cypressOps: -ProjectName: "Cypress_Project" -``` - -### Consolidated Native Robot Report Generation -The consolidated Native Robot report includes detailed information on all executed tests, i.e., which tests passed, failed, or were skipped, along with any error messages and stack traces for failed tests. It is widely supported across CI/CD tools, which makes it easy to integrate with existing workflows and with other tools or systems. - -Add the following configurations in your YAML file: - -```yaml -testRunnerCommand: YOUR_RUNNER_COMMAND --outputdir Reports - -report: true -partialReports: -type: json -location: Reports -frameworkName: robot -``` - -## Version 2.7.6 -### Parameterising `appium` flag -To support advanced mobile test scenarios, particularly for iOS simulation using desktop browsers, we've added support for parameterising the `appium` flag. You can now configure your test matrix to include macOS + Appium combinations for mobile browser testing. This supports more flexible testing strategies for iOS use cases using desktop browsers on mac environments. - -```yaml -version: 0.1 - -runson: ${matrix.os} - -matrix: -os: [mac, linux] -appium: [true,false] - -exclusionMatrix: -- os: ["mac"] -appium: [true] -- os: ["linux"] -appium: [false] - -appium: ${matrix.appium} -``` - -## Version 2.7.3 -### `globalPre` and `globalPost` flag in HyperExecute YAML -The [`globalPre`](/support/docs/deep-dive-into-hyperexecute-yaml/#globalpre) and [`globalPost`](/support/docs/deep-dive-into-hyperexecute-yaml/#globalpost) flags in the HyperExecute YAML configuration allow for global setup and teardown operations for test execution. These flags enable users to define commands that run before any tests begin (globalPre) and after all tests have completed (globalPost), providing greater control over test executions. - -```yaml title="hyperexecute.yaml" -# globalPre -globalPre: -mode: remote #local or remote -commands: -- "echo 'Setting up environment'" -- "apt-get update && apt-get install -y curl" -- "curl -X POST https://api.example.com/init" -runson: linux - -# globalPost -globalPost: -mode: remote #local or remote -commands: -- "echo 'Cleaning up test environment'" -- "rm -rf /tmp/test-results" -- "curl -X POST https://api.example.com/cleanup" -runson: linux -``` - -> 📘 Refer to our detailed documentation for [`globalPre`](/support/docs/deep-dive-into-hyperexecute-yaml/#globalpre) and [`globalPost`](/support/docs/deep-dive-into-hyperexecute-yaml/#globalpost) flags. - -## Version 2.7.1 -### Artifact Extraction on Job Abortion -- Artifacts will now be extracted even if a job is aborted in the middle of a test. -- This ensures that partial test results and logs are still available for review, improving debugging and post-test analysis. - -### `afterEachScenario` Flag for Test Cleanup -- Introduced the `afterEachScenario` flag to automate post-test scenario tasks. This flag is useful for cleanup activities such as clearing cookies, resetting databases, or capturing logs. -- Commands or scripts listed under this flag will execute sequentially after each test scenario, ensuring test isolation and consistency. - -```yaml -afterEachScenario: -- echo "Running a script" -- pwsh run.test -``` - -> 📘 Refer to our detailed documentation for [`afterEachScenario`](/support/docs/deep-dive-into-hyperexecute-yaml/#aftereachscenario) flag - -### Role-Based Access Control (RBAC) for Projects - -- Implemented RBAC to enhance project security and access management. -- Users can now be assigned specific roles with defined permissions. - -This feature improves security by ensuring that only authorized users can access or modify project settings and resources. - -### UX Enhancements for Performance Testing Jobs -- Improved UI/UX for managing and analyzing performance testing jobs. -- Enhanced navigation and usability for a smoother experience. -- Group-Wise Error View - - Introduced an error grouping feature to help identify and troubleshoot issues efficiently. - - Errors are now categorized for better readability and quicker resolution. -- Toggle Metrics: ms :left_right_arrow: s - - Added the ability to toggle performance metrics between milliseconds (ms) and seconds (s). -- Provides flexibility for analyzing test execution times in preferred units. - -### Browser Version Updates - -- **Edge Beta** : Released 133.3065.31 2.7.1 and 133.3065.39 2.7.2 for beta testing - -## Version 2.7.0 -### Browser Version Updates: - -- **Firefox :** Added support for Firefox-133, and Firefox-134, enabling testing on the latest versions -- **Edge :** Updated to Edge-132 for compatibility and stability enhancements. -- **Edge Beta** : Released 133.0.3065.19 for beta testing -- **Chrome :** Upgraded to Chrome-132 to include the newest features and security updates. - -These updates ensure broader coverage for cross-browser testing and improve test stability across the latest browser versions diff --git a/static/docs/hyperexecute-release-notes.md b/static/docs/hyperexecute-release-notes.md deleted file mode 100644 index e201dd104..000000000 --- a/static/docs/hyperexecute-release-notes.md +++ /dev/null @@ -1,206 +0,0 @@ -# HyperExecute Release Notes - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -HyperExecute is an AI Native Test Orchestration Cloud Platform, enabling test automation teams worldwide to achieve an accelerated time to market by providing JIT (Just-in-time) testing infrastructure that offers enhanced test execution speeds, smart test orchestration, detailed execution logs and much more. - -## HyperExecute Releases - -### MCP releases -#### May, 2025 -| Release Number | Release Date | -|----------------|--------------| -| [Version 1.0.0](/support/docs/hyperexecute-mcp-server-release-notes-1-0-0/) | May 28, 2025| - -### CLI releases -#### January, 2025 -| Release Number | Release Date | -|----------------|--------------| -| [Version 0.2.249](/support/docs/hyperexecute-cli-release-notes-0-2-249) | January 13, 2025| - -### 2025 releases -#### December, 2025 -| Release Number | Release Date | -|----------------|--------------| -| [Version 3.1.7](/support/docs/hyperexecute-release-notes-2025/#version-317) | Dec 15, 2025| -| [Version 3.1.6](/support/docs/hyperexecute-release-notes-2025/#version-316) | Dec 08, 2025| -| [Version 3.1.5](/support/docs/hyperexecute-release-notes-2025/#version-315) | Dec 01, 2025| - -#### November, 2025 -| Release Number | Release Date | -|----------------|--------------| -| [Version 3.1.4](/support/docs/hyperexecute-release-notes-2025/#version-314) | Nov 24, 2025| -| [Version 3.1.2](/support/docs/hyperexecute-release-notes-2025/#version-312) | Nov 10, 2025| - -#### October, 2025 -| Release Number | Release Date | -|----------------|--------------| -| [Version 3.1.0](/support/docs/hyperexecute-release-notes-2025/#version-310) | Oct 13, 2025| -| [Version 3.0.8](/support/docs/hyperexecute-release-notes-2025/#version-308) | Oct 13, 2025| - -#### September, 2025 -| Release Number | Release Date | -|----------------|--------------| -| [Version 3.0.6](/support/docs/hyperexecute-release-notes-2025/#version-306) | Sept 29, 2025| -| [Version 3.0.4](/support/docs/hyperexecute-release-notes-2025/#version-304) | Sept 15, 2025| -| [Version 3.0.3](/support/docs/hyperexecute-release-notes-2025/#version-303) | Sept 08, 2025| - -#### August, 2025 -| Release Number | Release Date | -|----------------|--------------| -| [Version 3.0.0](/support/docs/hyperexecute-release-notes-2025/#version-300) | August 18, 2025| -| [Version 2.9.9](/support/docs/hyperexecute-release-notes-2025/#version-299) | August 11, 2025| -| [Version 2.9.8](/support/docs/hyperexecute-release-notes-2025/#version-298) | August 04, 2025| - -#### July, 2025 -| Release Number | Release Date | -|----------------|--------------| -| [Version 2.9.7](/support/docs/hyperexecute-release-notes-2025/#version-297) | July 28, 2025| -| [Version 2.9.6](/support/docs/hyperexecute-release-notes-2025/#version-296) | July 21, 2025| -| [Version 2.9.5](/support/docs/hyperexecute-release-notes-2025/#version-295) | July 14, 2025| -| [Version 2.9.4](/support/docs/hyperexecute-release-notes-2025/#version-294) | July 07, 2025| - -#### June, 2025 -| Release Number | Release Date | -|----------------|--------------| -| [Version 2.9.1](/support/docs/hyperexecute-release-notes-2025/#version-291) | June 16, 2025| -| [Version 2.9.0](/support/docs/hyperexecute-release-notes-2025/#version-290) | June 09, 2025| -| [Version 2.8.9](/support/docs/hyperexecute-release-notes-2025/#version-289) | June 02, 2025| - -#### May, 2025 -| Release Number | Release Date | -|----------------|--------------| -| [Version 2.8.8](/support/docs/hyperexecute-release-notes-2025/#version-288) | May 26, 2025| -| [Version 2.8.7](/support/docs/hyperexecute-release-notes-2025/#version-287) | May 19, 2025| -| [Version 2.8.6](/support/docs/hyperexecute-release-notes-2025/#version-286) | May 12, 2025| -| [Version 2.8.5](/support/docs/hyperexecute-release-notes-2025/#version-285) | May 05, 2025| - -#### April, 2025 -| Release Number | Release Date | -|----------------|--------------| -| [Version 2.8.4](/support/docs/hyperexecute-release-notes-2025/#version-284) | April 28, 2025| -| [Version 2.8.2](/support/docs/hyperexecute-release-notes-2025/#version-282) | April 21, 2025| -| [Version 2.8.1](/support/docs/hyperexecute-release-notes-2025/#version-281) | April 14, 2025| - -#### March, 2025 -| Release Number | Release Date | -|----------------|--------------| -| [Version 2.7.8](/support/docs/hyperexecute-release-notes-2025/#version-278) | March 24, 2025| -| [Version 2.7.6](/support/docs/hyperexecute-release-notes-2025/#version-276) | March 24, 2025| - -#### February, 2025 -| Release Number | Release Date | -|----------------|--------------| -| [Version 2.7.3](/support/docs/hyperexecute-release-notes-2025/#version-273) | February 17, 2025| -| [Version 2.7.1](/support/docs/hyperexecute-release-notes-2025/#version-271) | February 03, 2025| - -#### January, 2025 -| Release Number | Release Date | -|----------------|--------------| -| [Version 2.7.0](/support/docs/hyperexecute-release-notes-2025/#version-270) | January 27, 2025| - -### 2024 releases -#### December, 2024 -| Release Number | Release Date | -|----------------|--------------| -| [Version 2.6.5](/support/docs/hyperexecute-release-notes-2024/#version-265) | December 23, 2024| -| [Version 2.6.2](/support/docs/hyperexecute-release-notes-2024/#version-262) | December 02, 2024| - -#### November, 2024 -| Release Number | Release Date | -|----------------|--------------| -| [Version 2.6.0](/support/docs/hyperexecute-release-notes-2024/#version-260) | November 18, 2024| -| [Version 2.5.9](/support/docs/hyperexecute-release-notes-2024/#version-259) | November 11, 2024| -| [Version 2.5.8](/support/docs/hyperexecute-release-notes-2024/#version-258) | November 04, 2024| - -#### October, 2024 -| Release Number | Release Date | -|----------------|--------------| -| [Version 2.5.7](/support/docs/hyperexecute-release-notes-2024/#version-257) | October 28, 2024| -| [Version 2.5.6](/support/docs/hyperexecute-release-notes-2024/#version-256) | October 21, 2024| -| [Version 2.5.5](/support/docs/hyperexecute-release-notes-2024/#version-255) | October 14, 2024| -| [Version 2.5.4](/support/docs/hyperexecute-release-notes-2024/#version-254) | October 07, 2024| - -#### September, 2024 -| Release Number | Release Date | -|----------------|--------------| -| [Version 2.5.3](/support/docs/hyperexecute-release-notes-2024/#version-253) | September 30, 2024| -| [Version 2.5.2](/support/docs/hyperexecute-release-notes-2024/#version-252) | September 23, 2024| -| [Version 2.5.1](/support/docs/hyperexecute-release-notes-2024/#version-251) | September 16, 2024| -| [Version 2.5.0](/support/docs/hyperexecute-release-notes-2024/#version-250) | September 10, 2024| - -#### August, 2024 -| Release Number | Release Date | -|----------------|--------------| -| [Version 2.4.8](/support/docs/hyperexecute-release-notes-2024/#version-248) | August 27, 2024| -| [Version 2.4.7](/support/docs/hyperexecute-release-notes-2024/#version-247) | August 20, 2024| -| [Version 2.4.6](/support/docs/hyperexecute-release-notes-2024/#version-246) | August 12, 2024| - -#### July, 2024 -| Release Number | Release Date | -|----------------|--------------| -| [Version 2.4.5](/support/docs/hyperexecute-release-notes-2024/#version-245) | July 29, 2024| -| [Version 2.4.4](/support/docs/hyperexecute-release-notes-2024/#version-244) | July 22, 2024| -| [Version 2.4.0](/support/docs/hyperexecute-release-notes-2024/#version-240) | July 01, 2024| - -#### June, 2024 -| Release Number | Release Date | -|----------------|--------------| -| [Version 2.3.8](/support/docs/hyperexecute-release-notes-2024/#version-238) | June 18, 2024| -| [Version 2.3.7](/support/docs/hyperexecute-release-notes-2024/#version-237) | June 10, 2024| -| [Version 2.3.6](/support/docs/hyperexecute-release-notes-2024/#version-236) | June 03, 2024| -#### May, 2024 -| Release Number | Release Date | -|----------------|--------------| -| [Version 2.3.3](/support/docs/hyperexecute-release-notes-2024/#version-233) | May 13, 2024| -#### April, 2024 -| Release Number | Release Date | -|----------------|--------------| -| [Version 2.3.1](/support/docs/hyperexecute-release-notes-2024/#version-231) | April 29, 2024| -| [Version 2.3.0](/support/docs/hyperexecute-release-notes-2024/#version-230) | April 22, 2024| -| [Version 2.2.9](/support/docs/hyperexecute-release-notes-2024/#version-229) | April 15, 2024| -| [Version 2.2.8](/support/docs/hyperexecute-release-notes-2024/#version-228) | April 8, 2024| -| [Version 2.2.7](/support/docs/hyperexecute-release-notes-2024/#version-227) | April 01, 2024| -#### March, 2024 -| Release Number | Release Date | -|----------------|--------------| -| [Version 2.2.6](/support/docs/hyperexecute-release-notes-2024/#version-226) | March 28, 2024| -| [Version 2.2.5](/support/docs/hyperexecute-release-notes-2024/#version-225) | March 18, 2024| -| [Version 2.2.3](/support/docs/hyperexecute-release-notes-2024/#version-223) | March 03, 2024| -#### February, 2024 -| Release Number | Release Date | -|----------------|--------------| -| [Version 2.2.2](/support/docs/hyperexecute-release-notes-2024/#version-222) | February 25, 2024| -| [Version 2.2.1](/support/docs/hyperexecute-release-notes-2024/#version-221) | February 18, 2024| -| [Version 2.2.0](/support/docs/hyperexecute-release-notes-2024/#version-220) | February 11, 2024| -#### January, 2024 -| Release Number | Release Date | -|----------------|--------------| -| [Version 2.1.6](/support/docs/hyperexecute-release-notes-2024/#version-216) | January 14, 2024| -| [Version 2.1.5](/support/docs/hyperexecute-release-notes-2024/#version-215) | January 07, 2024| - -### 2023 releases -#### December, 2023 -| Release Number | Release Date | -|----------------|--------------| -| [Version 2.1.4](/support/docs/hyperexecute-release-notes-2023/#version-214) | December 31, 2023| -| [Version 2.1.3](/support/docs/hyperexecute-release-notes-2024/#version-213) | December 10, 2023| -| [Version 2.1.1](/support/docs/hyperexecute-release-notes-2024/#version-211) | December 17, 2023| -| [Version 2.1.0](/support/docs/hyperexecute-release-notes-2023/#version-210) | December 10, 2023| -| [Version 1.139.0](/support/docs/hyperexecute-release-notes-2023/#version-11390) | December 3, 2023| -#### November, 2023 -| Release Number | Release Date | -|----------------|--------------| -| [Version 1.138.0](/support/docs/hyperexecute-release-notes-2023/#version-11380) | November 12, 2023| -| [Version 1.137.0](/support/docs/hyperexecute-release-notes-2023/#version-11370) | November 5, 2023| -#### October, 2023 -| Release Number | Release Date | -|----------------|--------------| -| [Version 1.136.0](/support/docs/hyperexecute-release-notes-2023/#version-11360) | October 29, 2023| -| [Version 1.135.0](/support/docs/hyperexecute-release-notes-2023/#version-11350) | October 23, 2023| -| [Version 1.134.0](/support/docs/hyperexecute-release-notes-2023/#version-11340) | October 15, 2023| -| [Version 1.133.0](/support/docs/hyperexecute-release-notes-2023/#version-11330) | October 8, 2023| -#### September, 2023 -| Release Number | Release Date | -|----------------|--------------| -| [Version 1.131.0](/support/docs/hyperexecute-release-notes-2023/#version-11310) | Sept 24, 2023| diff --git a/static/docs/hyperexecute-reports.md b/static/docs/hyperexecute-reports.md deleted file mode 100644 index 27921cb60..000000000 --- a/static/docs/hyperexecute-reports.md +++ /dev/null @@ -1,69 +0,0 @@ -# Job Reports - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Reports provide summarized job information, including status counts and links to tests. You can generate and download detailed reports at the end of job execution that helps you analyze the outcome of your tests based on your framework. - -**note** -Refer to this detailed [Job Reports](/support/docs/hyperexecute-job-reports/) page to learn more about the types of reports being offered and how to configure them. - -## Steps to Generate and Download Reports - -Follow the below mentioned steps to generate the Reports for your desired framework: - -**Step 1:** Go to the HyperExecute YAML file - -**Step 2:** Add the below mentioned parameters in the YAML file to generate the Reports - -- You must set the `report` flag to **true**. -- You will have to specify the **location**, **type** and **framework** associated with your report in the `partialReports` field to generate it. - -```yaml -report: true -partialReports: -frameworkName: testng -location: target/surefire-reports/html -type: html -``` -> **Note**: If you do not specify a framework but you do mention the type of the report, then a report of the default framework is created. - -**Step 3:** Now trigger your job and go to the [HyperExecute](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute/jobs) dashboard. - -**Step 4:** Click on the **Open** button of the **Reports**. - -Here is the sample report generated. - -## Generate Multiple Reports -If you want to generate multiple reports of different types [supported by HyperExecute](/support/docs/hyperexecute-job-reports/), simply add the required specifications in the `partialReports` field as shown below: - -```yaml -partialReports: -- location: reports/json -type: json -frameworkName: extent-native - -- location: target/surefire-reports -type: html -frameworkName: testng -``` - -## Regex Support for Dynamic Report Paths -You can now use **glob-style wildcards `(**)`** in the location field of `partialReports` to dynamically match nested folders and custom report paths. This is especially useful when your test reports are generated inside directories that include date stamps, environment names, or other runtime variables. - -Example: -```yaml title="hyperexecute.yaml" -partialReports: -location: reports/**/cucumber/ -# location: target/abc/**/pqr/**/cucumber-reports -frameworkName: cucumber -type: json -``` - -Ensure that your Testrunner.java file should have the complete path from where partial reports will be added. Example: - -```java title="Testrunner.java" -plugin = { -"json:target/abc/mno/pqr/xyz/cucumber-reports/CucumberTestReport.json", -"json:target/abc/its/pqr/abc/cucumber-reports/CucumberTestReport.json" -} -``` diff --git a/static/docs/hyperexecute-rerun-failed-tests.md b/static/docs/hyperexecute-rerun-failed-tests.md deleted file mode 100644 index 4d72091c0..000000000 --- a/static/docs/hyperexecute-rerun-failed-tests.md +++ /dev/null @@ -1,50 +0,0 @@ -# Rerun Failed Tests on HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -**BETA** -The feature is currently in **BETA**. - -Debugging test failures often involves **identifying the root cause, fixing the issue, and rerunning tests to validate the fix**. However, in large test suites with hundreds or thousands of tests, rerunning the entire job consumes significant time and resources. - -To address this, HyperExecute supports Rerun Job functionality, a feature that allows you to rerun only the failed scenarios or all the scenarios directly from the dashboard, drastically reducing your feedback loop and helping you validate the fixes faster. - -## Prerequisites -To rerun a job, the following requirements must be fulfilled. - -- Your test suite should be available on **Git** (Github, Azure Repos and Bitbucket are supported) and linked to a [Project](/support/docs/hyperexecute-projects). This is required because each time a job is rerun, HyperExecute needs access to your test suite. -- Make sure you are using **YAML version 0.1** with auto-split mode and discovery mode as local only. - -> Support for YAML version 0.2 and other modes will be added soon. - -You can use the rerun feature for jobs started from your **local machine as long as Git info is available** (i.e., you’re running tests from a cloned Git repo). - - Make sure the project details are set in your job’s YAML, and the project is linked to the same Git repo you’re running from. - - If your local repo has uncommitted changes (a “dirty” working tree), rerun will not work. Commit or stash your changes before running the job. - -## Why Use Rerun Feature? -Consider a scenario where you’ve executed 1000 tests in a single HyperExecute job, and 50 tests failed due to a bug, either in your test code, infrastructure, or application under test. Previously, you were bound to run the complete job again which would have taken a lot more time (as you would have run the passed tests too) to check if the previously failed tests are passing or not. With **Rerun Job Feature**, your whole process is streamlined as given below: - -**View failed scenarios -> Identify the fix via AI RCA -> Apply your fix and deploy -> Rerun only failed scenarios of the job** - -### Benefits -- **Faster feedback loop:** Rerun only what matters (failed scenarios), so you confirm fixes quickly instead of re-running the full suite. - - **Reproducible results:** Reruns always use the same YAML as the original job, even when you choose the latest commit, ensuring consistent environment settings. - - **Better flaky test diagnosis:** By rerunning only the failed scenarios on the same configuration, it’s much easier to distinguish genuine regressions from flaky tests. -## Rerun Configuration Options -HyperExecute provides you flexibility to select different parameters while rerunning the job. -- **Failed or All Scenarios:** You can rerun only the failed scenarios or all scenarios of that job. -- **Using Same or Latest Commit:** You can reuse the same commit as the original job or the latest commit from your repository. This lets you rerun after fixing your test suite or application under test. -- **Adjusting Concurrency:** The **YAML remains the same** for the rerun and the original job to maintain the same environment, but you can change the concurrency on which the rerun job will run. - -> If there are multiple test cases under a failed scenario, all those test cases will be rerun if the scenario is marked as failed. - -## How to Rerun a Job -**Step 1 :** Go to the **Jobs** section in the HyperExecute dashboard. -**Step 2 :** Open the job you want to rerun and click the **Rerun** button in the top bar. - -**Step 3 :** In the Rerun dialog: -- Select the **commit** you want to use (same commit as the original job or the latest commit from the repo). -- Optionally adjust the **concurrency** for the rerun. -Then click **Rerun** to start the new job. - -> If you do not satisfy the mentioned conditions but still want to use the rerun job feature, please reach out to us at [support@testmuai.com](mailto:support@testmuai.com). We will be constantly evolving this feature and are looking forward to your feedback to make it better. diff --git a/static/docs/hyperexecute-resource-widgets.md b/static/docs/hyperexecute-resource-widgets.md deleted file mode 100644 index 378f88bc9..000000000 --- a/static/docs/hyperexecute-resource-widgets.md +++ /dev/null @@ -1,7 +0,0 @@ -# HyperExecute Resource Utilization Widgets - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Concurrency Trends - -The Concurrency Trends widget allows you to know the tests which are kept in the queue and the tests that are put in a running state. The information can be filtered by clicking on the relevant legends at the bottom of the graph. diff --git a/static/docs/hyperexecute-run-jmeter-tests.md b/static/docs/hyperexecute-run-jmeter-tests.md deleted file mode 100644 index a398e0d41..000000000 --- a/static/docs/hyperexecute-run-jmeter-tests.md +++ /dev/null @@ -1,177 +0,0 @@ -# Unleash the Power of JMeter Performance Testing with HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Prerequisites - -- Setup the [JMeter Test Plan](https://jmeter.apache.org/usermanual/build-web-test-plan.html) using Apache JMeter GUI. After setting up a Test Plan, now you need to associate your test cases and execute it as well via HyperExecute. - -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -> By default, HyperExecute supports **standard Thread Group** but you can use other custom thread groups as well. - -**Sample repo** -Download or Clone the code sample for the JMeter Performance Testing from the TestMu AI GitHub repository to run the tests on the HyperExecute. - - View on GitHub - -## Step 1: Setup your JMeter Project - -- Go to the [HyperExecute Project's Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute/projects) and click on the **New Project** button. -- Browse and Upload your JMeter test files to create a new performance test project. -- Click on the **Save** button - -## Step 2: Execute your Test Plans -- Select your desired **.jmx** file and click on the **Run** button. -- Configure the following parameters to perform the testing and click on the **Continue** button - - **Total Users :** This indicates the number of users you intend to test for. - - **Duration (min) :** This specifies the desired duration for the test. - - **Ramp-up Time (min) :** This specifies the amount of time it should take to reach the peak test load. - - **Total Load Distribution :** Configure the **regions** in which you want to distribute the load testing as per the % of the users. - - **Split CSV :** Enables splitting the input data from a CSV file among different threads or regions. This ensures organized data utilization across parallel executions. - - **Job Labels :** Add meaningful labels or tags to test executions for better identification and traceability in logs and reports. - - **Machine count :** Specify the number of machines (or nodes) to use for parallel test execution. This parameter is essential for scaling test workloads or reducing execution time. - - **Java Version :** Choose the Java runtime to execute the tests with. Java 11 is the default; Java 25 is also available. - -**Supported Regions :** HyperExecute supports the following regions for performance testing -- West US 2 (Moses Lake, Washington) -- East US (Richmond, Virginia) (**default**) -- Central India (Pune, Maharashtra) -- Southeast Asia (Singapore) -- Brazil South (São Paulo State, Brazil) -- Mexico Central (Querétaro State, Mexico) - -### User Properties - -Override selected JMeter properties for this run. The following [user properties](https://jmeter.apache.org/usermanual/test_plan.html#properties) are currently configurable: - -- **`jmeter.save.saveservice.subresults`**: Whether sub-results (for example, sub-samples generated within a transaction or by HTTP redirects) are written to the results file. Set to `false` to reduce result file size. -- **`httpclient.socket.https.cps`**: Characters per second cap on HTTPS downloads for the HttpClient sampler. Use it to simulate low-bandwidth conditions; `0` means unlimited. - -- Click on the **Run Test** button. - -## Step 3: Analyze Your Project Performance - -- Toggle to the **Jobs** section. - -### Summary Report - -- Here you can check the Summary of your Job executed that comprises of the details: - - - **Total number of Virtual Users** which basically simulates an instance script of real-world visitors to your app or site. - - **Ramp-up** and **Average Response** time. - - Average **Throuput** and **Bandwidth** time. - - **Load** and **Response Time** graph. - - Additional details like **regions** and test duration. - -### Timeline Report - -### Request Stats - -You can analyze the number of requests sent on that particular time. - -### Errors -- Here you can check the Error Code type, Error Count, percentage of that error, and the percent of error in all the samples. - -### Logs -- You can check the logs of your Jobs executed. - -## Handling Special Scenarios: Overriding and Default Values - - Scenario 1: Overriding Values via Projects portal - - When executing performance tests using HyperExecute, you have the option to override default parameters directly in the Projects portal. Let’s explore a sample scenario to understand how HyperExecute handles these overrides. - - You configure the following parameters in the HyperExecute UI: - - **Virtual Users :** (The total number of simulated users for the test): 500 - - **Ramp-Up Time :** 1 minute - - **Total Duration :** 2 minutes - - **Regions :** 2 (50% distribution each) - - **Maximum Users per Machine :** 100 - - #### Resulting Test Distribution: - - **Load Distribution Across Regions :** - - The total 500 virtual users are divided equally between the two regions. - - Each region is allocated 250 users (500 users ÷ 2 regions). - - - **Machine Allocation :** Since a single machine can handle a maximum of 100 users, each region requires multiple machines to support its load: - - **Region 1 :** 250 users → 3 machines (lets say it distributed as 84, 83, 83 users per machine). - - **Region 2 :** 250 users → 3 machines (lets say it distributed as 84, 83, 83 users per machine). - - #### Key Takeaways: - - **Flexible Overrides :** HyperExecute dynamically adjusts the load distribution and machine allocation based on your specified parameters. - - **Optimal Resource Utilization :** It ensures that no machine exceeds the maximum user threshold, maintaining a balanced and efficient test execution. - - **Region-Specific Allocation :** Users are distributed proportionally based on the defined load percentages for each region. - - This approach ensures smooth execution of performance tests and provides precise control over resource utilization, enabling you to simulate real-world scenarios effectively. - - Scenario 2: Default Parameters - - In this scenario, you proceed without overriding the default values in the HyperExecute Projects portal. The configuration parameters from your JMeter file and project setup are applied as-is. - - #### Scenario Details: - - **Total Users :** 250 (Specified in the JMeter .jmx file.) - - **Regions :** 2 - - **Machines :** 3 (Each region is allocated three machines.) - - #### Resulting Test Distribution: - - **Load Distribution Across Regions :** The total of 250 virtual users is not divided across the regions because no overrides were applied. Instead, each region receives the full 250 users. - - - **Machine Allocation :** Each region is allocated three machines. The total user load for the region is replicated across all three machines in the region: - - **Region 1 :** 3 machines → 250 users per machine. - - **Region 2 :** 3 machines → 250 users per machine. - - - **User Allocation per Machine :** Each machine in both regions processes **250 users**, resulting in a total of 750 users (250 users × 3 machines) per region, and overall 1500 users (250 users x 3 machines * 2 regions). - - #### Key Takeaways: - - **Default Behavior :** Without overrides, the user load from the .jmx file is replicated across all configured machines in each region. - - **Resource Replication :** Instead of splitting the total users among machines, HyperExecute applies the same load to each machine in a region. - - **Importance of Overrides :** To distribute users proportionally across machines and regions, use the UI to override parameters such as total users, maximum users per machine, or load percentages. - - This behavior highlights how HyperExecute ensures flexibility in test execution while allowing for configuration control based on project requirements. - -## Frequently Asked Questions - -### Why is my JMeter job failing before reaching 2000 users? - -"Up to 2000 users" describes the ceiling HyperExecute's infrastructure can support under favorable conditions: lightweight requests, sensible timeouts, and load spread across enough machines and regions. It isn't a guarantee for every test plan. The actual achievable concurrency depends on what each virtual user does and how much CPU, memory, and network bandwidth that consumes on the load generator machine. When a job fails before reaching your configured user count, it's almost always because the load generator resources were exhausted before ramp-up completed, not because of an arbitrary platform limit. - -### What factors impact the maximum achievable load in JMeter? - -Four configuration patterns most commonly reduce the number of users a JMeter test can sustain: - -- **UI/browser-based testing through JMeter:** JMeter is built for protocol-level (HTTP/API) load generation, not browser rendering. Driving an actual browser (for example through a WebDriver Sampler) makes each virtual user consume CPU and memory comparable to a full browser session, which can reduce the number of concurrent users a single machine can sustain by an order of magnitude. -- **Test scripts without proper timeouts:** if HTTP Request timeouts are unset or too high, slow requests cause threads to pile up instead of failing fast and releasing resources. Set explicit Connect Timeout and Response Timeout values on your HTTP Request Defaults, and use a Duration Assertion or thread interrupt settings so hung threads are released rather than accumulating. -- **APIs with large request/response payloads:** larger payloads require more memory per in-flight thread and more network throughput per request, so large-payload APIs support fewer concurrent users per machine than lightweight requests do at the same request rate. -- **End-to-end or chained API workflows:** multi-step scenarios (such as login → search → add-to-cart → checkout) require more CPU and per-thread memory each iteration, so the more work JMeter does per virtual user, the fewer users a single machine can sustain within the same resource limits. - -### How can I identify why my JMeter test is failing? - -Match the symptom to its likely cause and where to investigate: - -| Symptom | Likely Cause | Where to Look | -|---|---|---| -| Job errors out well before the target user count is reached | Load generator resource exhaustion, driven by test design | Task Metrics → Scenario Stage CPU/memory graph | -| High error rate but resource usage looks normal | The application under test is the bottleneck (server-side errors, timeouts, or capacity limits) | HTTP response codes / error breakdown in the execution report | -| Response times/thread counts increase gradually and never recover | Thread or connection leak from missing timeouts in the script | Response Time graph trend and thread count over time | -| Failures concentrated during ramp-up spikes | Ramp-up too aggressive for the target system or machine allocation | Load and Response Time graph, correlated with ramp-up schedule | -| Failures on one region only | Load distribution misconfigured, or a network/latency issue to that region | Total Load Distribution settings and per-region error breakdown | - -### How do I know if the load generator has reached its resource limits? - -HyperExecute's Task Metrics feature shows CPU and memory consumption at the task level (for each individual task) and the stage level (broken into pre-stage, scenario stage, and post-stage). For load-ceiling investigations, focus on the scenario stage graph: a curve that climbs steadily and flattens near 100% CPU or memory shortly before failures begin is a clear sign the load generator itself, not your application, is the limiting factor. Specific signs a machine has hit its ceiling: - -- CPU utilization sustained near 90-100% while the error rate rises -- Memory utilization climbing continuously without stabilizing -- Throughput plateaus or drops even as the configured user count keeps ramping up -- Job logs show OutOfMemoryError, thread-creation failures, or GC-related stalls - -If you see these signs, reduce per-machine load (fewer users per machine, more machines/regions in your Total Load Distribution) or optimize your JMX. - -### Can I load test private or internal endpoints with JMeter? - -Yes, but the recommended way to reach them differs from functional automation. For automation testing it's common to use a tunnel, but for load testing a tunnel isn't feasible: it adds an extra network hop with its own resource and latency overhead, and since load testing specifically measures response time and how much load an API can handle, that overhead gets mixed into your results. Instead, reach out to the TestMu AI support team and ask for HyperExecute's static outbound IPs. Whitelist those IPs in your firewall/security group, and HyperExecute's load generators will reach your private endpoints directly, without a tunnel, so your results stay accurate. - -### Can JMeter be used for UI testing? - -JMeter is not recommended for UI/browser-based testing. It has no native rendering engine, so it can't validate what actually renders in a browser, execute JavaScript, or verify visual behavior. Any UI testing bolted onto JMeter through browser-driving samplers reintroduces full browser overhead per virtual user, defeating the purpose of lightweight load generation and significantly lowering achievable concurrency. Use JMeter for backend/API load and throughput testing, and a browser-based automation tool (such as Selenium or Playwright) for validating UI behavior. The two are complementary, not interchangeable. diff --git a/static/docs/hyperexecute-running-your-first-job.md b/static/docs/hyperexecute-running-your-first-job.md deleted file mode 100644 index 2a28402bb..000000000 --- a/static/docs/hyperexecute-running-your-first-job.md +++ /dev/null @@ -1,71 +0,0 @@ -# Running Your First Job on HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This page outlines how you can run your first test execution job on HyperExecute. - -## Before you start - -We recommend running your first test execution job using a sample project on HyperExecute. This can be done in three different ways, as mentioned below: - -1. [**Using Quick Run**](/support/docs/hyperexecute-running-your-first-job/#1-using-quick-run): Execute tests directly from the HyperExecute Portal. ([Recommended](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/quickstart) for new users) -2. [**Using Local System**](/support/docs/hyperexecute-running-your-first-job/#2-using-local-system): Execute tests from your Local System. (Requires [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/)) -3. [**Using the HyperExecute Agent Skill**](/support/docs/getting-started-with-hyperexecute/#using-the-hyperexecute-agent-skill-with-testmu-ai): Generate and run tests using an AI coding assistant with the HyperExecute Agent Skill. - -## 1. Using Quick Run - -As you are running tests directly from the HyperExecute GUI, you don't need to do anything. -The platform automatically gets the instructions from the HyperExecute YAML file and starts execution using the HyperExecute CLI. - -**Step 1:** Go to the [Quickstart](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/quickstart) page. - -**Step 2:** Select your preferred test automation framework and hit **Get Started**. - -**Step 3:** Configure your [HyperExecute YAML](/support/docs/deep-dive-into-hyperexecute-yaml) (optional), otherwise just click on **Trigger test** to run your first job. - -**Step 4:** Your test execution is initiated using the [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) in the background. - -**Step 5:** Click on **View Test Result** to navigate to [HyperExecute Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute/jobs) where you can see the job getting executed and the test results. - -## 2. Using Local System - -To run your Test Scripts on HyperExecute from you Local system, you'll need the following: -- [HyperExecute YAML](/support/docs/deep-dive-into-hyperexecute-yaml/) file which contains all the necessary instructions. -- [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) in order to initiate a test execution Job . -- Your TestMu AI [Username and Access key](/support/docs/hyperexecute-how-to-get-my-username-and-access-key/) - -We will be taking care of everything one by one, just follow the steps given below: - -**Step 1:** Go to the [Quickstart](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/quickstart) page. - -**Step 2:** Choose your test automation framework, Select **Run on Local System** and click on **Get Started**. - -**Step 3:** Download our sample project by clicking on **Download Sample**. (Skip this step if you already have a project you want to test) - -**Step 4:** Download the [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) as per the OS that you are on. - -**Step 5:** Select the OS on which you want to run your tests and then download the [HyperExecute YAML](/support/docs/deep-dive-into-hyperexecute-yaml/) file shown on screen. This contains all the configurations required for running tests on HyperExecute. - - > **Note**: Your Yaml file should be in the root directory of your project. If you are renaming it, also change the file name in the execution script as well. You can learn more about the advanced configurations [here](/support/docs/deep-dive-into-hyperexecute-yaml/), however for the given sample projects you can simply download the file using the small icon on top right. - -**Step 6:** In order to run your tests, copy the command from screen and run in your terminal. - -**Step 7:** When you run this command, the HyperExecute CLI will send your test scripts to the HyperExecute Platform for execution across multiple parallel Test Execution Nodes. - -**Step 8:** Once your Job is running, you can click on **View Test Results** to navigate to HyperExecute Dashboard where you can see the job getting executed along with the test results. - -The generic command required to initiate test execution is given below. You will need to insert the name of your YAML file in place of the `RELATIVE_PATH_OF_YOUR_YAML_FILE` before running this command. (Skip if you already used the command from Step 6. ) - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE `} - -In case of MAC, if you get a permission denied warning while executing CLI, simply run `chmod u+x ./hyperexecute` to allow permission. In case you get a security popup, allow it from your **System Preferences** → **Security & Privacy** → **General** tab. - -> If you are facing any issues in running your first job? Watch our [Webinar Video](/support/docs/hyperexecute-running-your-first-job/#hyperexecute-webinar-video) - -## 3. Using the HyperExecute Agent Skill - -You can generate production-ready HyperExecute tests and run them directly from your AI coding assistant (such as Claude or Cursor) using the HyperExecute Agent Skill — no manual YAML or CLI setup required. - -For installation and usage, refer to [Using the HyperExecute Agent Skill with TestMu AI](/support/docs/getting-started-with-hyperexecute/#using-the-hyperexecute-agent-skill-with-testmu-ai). - -## HyperExecute Webinar Video diff --git a/static/docs/hyperexecute-scenario-steps-error.md b/static/docs/hyperexecute-scenario-steps-error.md deleted file mode 100644 index 498c5262a..000000000 --- a/static/docs/hyperexecute-scenario-steps-error.md +++ /dev/null @@ -1,196 +0,0 @@ -# How to Debug your Scenario Steps Errors - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This document will help you debug your failed job if you are facing any errors in the Scenario stage. - -## Invalid Credentials - -If you're encountering an error message as shown below, it means you have entered an invalid **Username** and **Access Key** in your test file capabilities. - -```bash -OpenQA.Selenium.WebDriverException : The newSession command returned an unexpected error. -401 Unauthorized !! Please include `user` & `accessKey` in desired capabilities. -``` - -To resolve this issue, you have to visit our [Automation Capability Generator](https://www.testmuai.com/capabilities-generator/) site and configure your project suite capabilities generated as per your desired language and framework. - -## Unable to create Session - -When you initiate a test execution and encounter the issue that your session is not getting started, - -This is possible due to the following reasons: - -- If your credentials are not correct. -- If you have created local driver instead of remote webdriver connection -- If your HUB URL is incorrect. - -To address this challenge, our recommended solution is to visit the [Automation Capability Generator](https://www.testmuai.com/capabilities-generator/) website. Here, you can seamlessly generate accurate and valid sets of credentials and capabilities, thereby rectifying issues associated with test failures. - -## Invalid HUB Endpoint - -If you put the wrong HUB Endpoint in your test file, you will receive the following error: - -To rectify this issue, you need to cross-check your HUB URL. You can also visit [this](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) website and click on **Access Key** button to get your correct HUB URL. - -## Tunnel Error - -#### When Tunnel is Disconnected or Not Running - -- If you have mentioned the Tunnel URL in both capabilities and the YAML file, then it is likely that your Tunnel may get disconnected or will not run. To resolve this, you need to simply remove the Tunnel URL from either of the files. - -- If you are running Tunnel using Binary, then you need to ensure that your credentials for binary and capability are the same. - -#### When you try to access a Private Endpoint - -If you are trying to access an internal website or any other private endpoint which is behind any firewall using tunnel and you are getting the below mentioned error: - -```bash -System.AggregateException: One or more errors occurred. ---> -OpenQA.Selenium.WebDriverException: unknown error: net::ERR_NAME_NOT_RESOLVED -``` - -To resolve this, you have to pass this flag in your YAML file: - -```bash -tunnel: true -``` - -## Stage is Stuck - -When you encounter a situation where the stage appears to be stuck or unresponsive (machines are taking lot of time to allocate), then follow these measures to diagnose and address the issue. - -As you can see below, stage is taking a lot of time to allocate Virtual Machines to the tests. - -- **Identify the Task:** Begin by identifying the specific task or workflow where the machine appears to be stuck. This involves determining the action or process that triggered the unresponsive stage. - -- **Locate the Stuck Stage:** Once the task is identified, pinpoint the exact stage that is causing the delay. This can be done by reviewing the platform's progress indicators or monitoring tools. - -- **Examine Stage Logs:** Check the stage logs associated with the stuck stage. These logs provide detailed information about the stage's execution and may contain error messages or warnings that indicate the root cause of the issue. - -- **Analyze Martian Logs:** If the stage logs suggest an API call is stuck, refer to the Martian logs available on the dashboard. These logs reveal whether the API call originated from the machine and capture the response received from the endpoint. - -- **Review Tunnel Logs:** For users employing Tunnel with MITM mode, consult the Tunnel logs to gain deeper insights into network calls. These logs provide granular details about network interactions, potentially revealing the cause of the stuck stage. - -After identifying the issue, follow these steps to resolve the situation: - -**Step 1:** Go to the specific test case method or test class where the problematic REST Assured API is being called. - -**Step 2:** Ensure that you have set an appropriate connection timeout and socket timeout. This ensures that the system doesn't wait indefinitely for a response. - -**Step 3:** Consider printing the responses during debugging to verify whether the API is providing the expected responses. - -## Test is Stuck - -When a particular test is taking longer than expected to complete, follow these measures to diagnose the issue: - -- **Identify the Stuck Test:** Begin by identifying the specific test that is hanging or exceeding the average execution time. This can be done by reviewing test execution reports or monitoring tools. - -- **Examine Test Code:** Locate the corresponding test case in the codebase. Analyze the test code to identify any API calls that might be causing delays in response times. - -- **Check for Infinite Loops:** Inspect the test code for potential infinite loops. Infinite loops can occur when commands are repeatedly executed without proper termination conditions, leading to excessive test execution time. - -- **Local Machine Testing:** Replicate the test execution on the user's local machine to determine if the issue is specific to their environment or if it's a general problem with the test itself. - -- **Selenium Command Analysis:** Analyze the Selenium commands within the test case to understand their intended behavior and identify any potential irregularities or inefficiencies that could be contributing to the prolonged test execution. - -After identifying the issue, follow these steps to resolve the situation: - -1. **Optimize API Calls:** Refine the API calls within the test case to improve response times. This may involve optimizing query parameters, using caching mechanisms, or adopting alternative API endpoints. - -2. **Eliminate Infinite Loops:** Modify the test code to eliminate any identified infinite loops. Implement proper loop termination conditions to ensure that commands are executed within the expected timeframe. - -3. **Restructure Test Code:** Refactor the test code to improve its overall structure and efficiency. Consider using conditional statements, data-driven testing techniques, or parallel execution strategies to streamline test execution. - -## Test Idle Timeout - -When the test encounters an idle timeout, it indicates that commands are not being executed for an extended period of time. Additionally, the quit() command fails to execute, causing the test to get stuck. - -> -You can see the test has been being executed for a very long time, which is not ideal behavior. - -And when you click on the View Test button, you can see that the status of your test is Idle_Timeout. - -Measures to Take: - -- **Identify Task and Stage:** Determine the specific task and stage associated with the idle timeout. This information can be obtained from test execution reports or monitoring tools. - -- **Verify Command Execution Count:** Confirm the number of commands executed before the idle timeout occurs. - -- **Check quit() Command Execution:** Ensure that the quit() command is properly included in the test code and that there are no errors preventing its execution. - -- **Local Machine Testing:** Replicate the test execution on the user's local machine to rule out environment-specific factors. - -- **Analyze Codebase:** Examine the codebase for potential issues between the last successfully executed command and the very last command. This may involve syntax errors, logic flaws, or unexpected interactions with external systems. - -Remedy: - -1. **Debug Codebase:** Insert logging statements in the test code after the last successfully executed command. This will provide insights into the execution flow and identify any unexpected behavior or errors that could be causing the idle timeout. - -2. **Review Code Logic:** Carefully review the code logic between the failed commands to ensure that commands are executed in the intended sequence and that there are no infinite loops or unintended pauses. - -3. **Check External Interactions:** Verify that any interactions with external systems, such as APIs or third-party services, are functioning properly and not causing delays or disruptions in test execution. - -4. **Optimize Resource Usage:** Ensure that the test environment has sufficient resources, such as CPU, memory, and network bandwidth, to handle the test workload. Resource constraints can lead to performance bottlenecks and idle timeouts. - -## API Timeout - -When the test encounters an API timeout, indicating that the API call did not receive a response within the specified timeout period, follow these measures to diagnose and address the issue. - -- **Identify Task and Test:** Determine the specific task and test associated with the API timeout. This information can be obtained from test execution reports or monitoring tools. - -- **Review Martian Logs:** Analyze the Martian logs for relevant entries related to the API call in question. Martian logs provide timestamps, request details, and response bodies, if available. - -- **Verify Response Presence:** Check if the Martian logs contain a response body for the API call. If no response is present, the API is likely timed out on the server side. - -- **Local Machine Testing:** Replicate the test execution on the user's local machine to rule out environment-specific factors. - -- **Server-Side Logging:** If the API consistently fails to respond within the specified timeout, both locally and in the production environment, investigate server-side logging to identify potential issues on the server. - -After identifying the issue, follow these steps to resolve the situation: - -**Step 1:** Analyze Server-Side Logs for error messages, warnings, or exceptions related to the API call. This can pinpoint the root cause of the timeout and guide further troubleshooting. - -**Step 2:** If the test code does not explicitly define **connection and socket timeouts** for the API call, incorporate these timeouts to ensure that the API call doesn't hang indefinitely. - -**Step 3:** Evaluate the appropriateness of the timeout values specified in the test code and on the server-side. Adjust the timeouts if necessary to accommodate the expected response time of the API. - -**Step 4:** Consider optimizing the API call itself to improve its performance and reduce the likelihood of **timeouts**. This may involve caching mechanisms, load balancing, or alternative API endpoints. - -Here's an example of how you can set connection and **socket timeouts**: - -```java -public static void main(String[] args) { - -// Set connection and socket timeouts -int connectionTimeout = 5000; // in milliseconds -int socketTimeout = 5000; // in milliseconds - -RestAssured.config = RestAssuredConfig.config().socketConfig( -SocketConfig.socketConfig().setConnectTimeout(connectionTimeout) -.setReadTimeout(socketTimeout)); - -// Your Rest Assured request code here -// For example: -RestAssured.given() -.baseUri("https://api.example.com") -.basePath("/some/path") -.get() -.then() -.statusCode(200); - -} -``` - -Make sure to include the necessary dependencies in your project. If you are using Maven, you can add the following dependencies to your pom.xml file: - -```bash - - -io.rest-assured -rest-assured -4.5.0 -test - - -``` diff --git a/static/docs/hyperexecute-seamless-integration-with-tools.md b/static/docs/hyperexecute-seamless-integration-with-tools.md deleted file mode 100644 index 87f86f496..000000000 --- a/static/docs/hyperexecute-seamless-integration-with-tools.md +++ /dev/null @@ -1,35 +0,0 @@ -# Seamless Integration for Enhanced Test Orchestration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -HyperExecute, a powerful test automation orchestration platform, boasts seamless compatibility and effortless integration with a wide range of industry-leading testing tools. This document delves into HyperExecute's integration capabilities, highlighting its compatibility with Katalon, AccelQ, Provar, Tosca, Azure Test Plans, Excel, and other TestMu AI products. - -## Integrate HyperExecute Effortlessly - -- [**Katalon Studio**](/docs/katalon-integration-with-hyperexecute/): HyperExecute directly integrates with Katalon Studio, enabling you to leverage Katalon's robust object repository and keyword-driven testing within your orchestrated test suites. Execute Katalon tests seamlessly alongside other tools for comprehensive test coverage. - -- [**AccelQ**](/docs/hyperexecute-accelq-integration/): HyperExecute seamlessly integrates with AccelQ's data-driven testing capabilities. Leverage AccelQ's data sheets and dynamic test case generation within HyperExecute's orchestration workflows for efficient test execution. - -- [**MS Office Suite**](/docs/hyperexecute-winapp-integration/): HyperExecute seamlessly integrates with MS Office tools. Leverage Excel, Notepad and other tools for data-driven testing within HyperExecute's orchestration workflows, using familiar functionalities for efficient test data management. - -- [**Tosca**](/docs/tosca-integration-with-hyperexecute/): HyperExecute integrates with Tosca's model-based testing capabilities. Execute Tosca tests orchestrated within HyperExecute's workflows, combining the power of model-based testing with flexible orchestration. - -- [**Azure Test Plans**](/docs/hyperexecute-azure-test-plan/): HyperExecute integrates with Azure Test Plans, enabling you to execute tests defined within Azure Pipelines directly from HyperExecute. Manage and orchestrate your Azure test executions efficiently. - -- [**TestMu AI Products**](/docs/he-integration-with-testmu-products/): HyperExecute integrates with various TestMu AI products, including Smart UI and Real Devices. Execute tests across diverse browsers, devices, and operating systems seamlessly within HyperExecute's orchestration framework. - -## Benefits of Integration - -There are enormous advantages of integrating HyperExecute with the above mentioned tools, some of them are: - -- **Unified Test Orchestration:** Manage and execute tests from various tools within a single platform, streamlining your testing process. - -- **Improved Efficiency:** Automate repetitive tasks and orchestrate complex test flows across diverse tools, saving time and resources. - -- **Enhanced Coverage:** Combine the strengths of different tools to achieve comprehensive test coverage across various testing methodologies. - -- **Simplified Management:** Gain centralized control over your testing ecosystem, reducing complexity and improving visibility. - -## Conclusion - -HyperExecute's seamless compatibility and effortless integration with various testing tools empower you to create robust, efficient, and scalable test automation strategies. Embrace the power of unified test orchestration and experience the value of HyperExecute's integrations firsthand. diff --git a/static/docs/hyperexecute-security-faqs.md b/static/docs/hyperexecute-security-faqs.md deleted file mode 100644 index fd223773b..000000000 --- a/static/docs/hyperexecute-security-faqs.md +++ /dev/null @@ -1,51 +0,0 @@ -# HyperExecute Security FAQs - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -#### 1. **How secure is HyperExecute?** - - HyperExecute is built with security at its core and provides industry standard security on the entire infrastructure. HyperExecute provides fresh virtual machines every time a new job is triggered and deletes the infrastructure after the test execution of the job is completed. Dozens of microservices ensure that test scripts are deleted after the retention period and uniquely encrypted during their retention period. HyperExecute uses a powerful vault to store client side secrets for the users. Enterprise version enables users to add their own security policies over the HyperExecute infrastructure. - -#### 2. **Does HyperExecute provide the ability to define secrets?** - - Yes, HyperExecute provides a vault service to securely store client side secrets instead of hardcoding them in the yaml or test script files. In enterprise versions, organizations can pick the vault of their choice. - -#### 3. **What’s the retention period of logs on HyperExecute?** - - By default, HyperExecute has a retention period of 60 days where it keeps records and test execution logs. After the retention period, the logs and records are deleted permanently. Enterprise version allows a custom retention period to be defined. - -#### 4. **How do I download private dependencies on HyperExecute?** - - HyperExecute provides multiple methods to connect to private artifactories and private package modules using automatic tunnel, dedicated NAT and using pre-steps to use VPN. Users usually add their private package management keys in the HyperExecute vault to fetch private repositories. - -#### 5. Is the VM allocated secure? -We are SOC2 compliant, all the VMs are highly secure and its compliant with the CIS benchmark - -#### 6. What happens to VM after the Test? -All the VMs get deleted, and every time you run the test, a new machine is allocated. We completely delete it. - -#### 6. Can we do SSH to machines? -No Machines are private in nature and cannot be accessed by the Internet directly. - -#### 7. What happens to secrets present in test code? -All the logs are encrypted and stored in a secure manner. But we always prefer customers to use Vault to pass the credentials. - -#### 8. Is the product GDPR compliant? -Yes, we follow GDPR guidelines. - -#### 9. What is the security level? -We follow strict security levels at each step. We are SOC 2 compliant. - -#### 10. What is the data retention time period? Can we ask to delete it anytime? -We store the data for a max of 60 days. Yes, you can ask the TestMu AI team fore the deletion of your data via email. - -#### 11. How to update the HyperExecute Binary? -HyperExecute Binary is a secure binary, this can be updated via a link, Notification regarding the same will be shared. - -#### 12. Can this be set up completely on the Premises? -Yes, HyperExecute can be set up completely on the Premises. - -#### 13. What about the SAS token? -SAS token is generated only for 60 min in use and is different every time. This token is time-based authenticated and can only be used once. - ->If you have more questions, then just give us a shout and we will answer them for you. diff --git a/static/docs/hyperexecute-shared-cloud-usage-limit.md b/static/docs/hyperexecute-shared-cloud-usage-limit.md deleted file mode 100644 index a7236f011..000000000 --- a/static/docs/hyperexecute-shared-cloud-usage-limit.md +++ /dev/null @@ -1,122 +0,0 @@ -# Daily Usage Limit Per Organization | HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -To ensure fair usage, maintain system stability, and promote optimal test practices across all our users, TestMu AI enforces a daily usage limit on test execution time at the organization level. Each organization is allocated a maximum allowable test duration per day, calculated based on the number of concurrent (parallel) sessions provisioned to the account. This safeguard helps prevent overutilization and ensures equitable access to resources across the platform. - -## Why This Matters -Running long-duration test sessions, especially those exceeding 120 minutes, can cause memory and CPU spikes that impact system performance and reliability. To prevent such disruptions and improve the efficiency of test execution, we encourage users to: - -- Split long-running tests into smaller, modular suites. -- Implement test retries, timeouts, and teardown routines to handle edge cases and resource leaks. -- Use parallelization effectively to optimize test execution time without extending the runtime of individual sessions. - -By adhering to these guidelines, your team not only aligns with industry best practices but also ensures smoother and more reliable test runs on TestMu AI infrastructure. - -## Daily Test Time Limits by Parallel Sessions -The daily usage cap scales linearly based on the number of parallel sessions available to your organization. Below is a reference table: - -| PARALLEL SESSIONS | COMPLETED RUN TIME (HRS) | -| ---------- | ----------------- | -| 1 | 6 | -| 2 | 12 | -| 3 | 18 | -| 4 | 24 | -| 5 | 30 | -| 6 | 36 | -| 7 | 42 | -| 8 | 48 | -| 9 | 54 | -| 10 | 60 | -| 11 | 66 | -| 12 | 72 | -| 13 | 78 | -| 14 | 84 | -| 15 | 90 | -| 16 | 96 | -| 17 | 102 | -| 18 | 108 | -| 19 | 114 | -| 20 | 120 | -| 21 | 126 | -| 22 | 132 | -| 23 | 138 | -| 24 | 144 | -| 25 | 150 | -| 26 | 156 | -| 27 | 162 | -| 28 | 168 | -| 29 | 174 | -| 30 | 180 | -| 31 | 186 | -| 32 | 192 | -| 33 | 198 | -| 34 | 204 | -| 35 | 210 | -| 36 | 216 | -| 37 | 222 | -| 38 | 228 | -| 39 | 234 | -| 40 | 240 | -| 41 | 246 | -| 42 | 252 | -| 43 | 258 | -| 44 | 264 | -| 45 | 270 | -| 46 | 276 | -| 47 | 282 | -| 48 | 288 | -| 49 | 294 | -| 50 | 300 | -| 51 | 306 | -| 52 | 312 | -| 53 | 318 | -| 54 | 324 | -| 55 | 330 | -| 56 | 336 | -| 57 | 342 | -| 58 | 348 | -| 59 | 354 | -| 60 | 360 | -| 61 | 366 | -| 62 | 372 | -| 63 | 378 | -| 64 | 384 | -| 65 | 390 | -| 66 | 396 | -| 67 | 402 | -| 68 | 408 | -| 69 | 414 | -| 70 | 420 | -| 71 | 426 | -| 72 | 432 | -| 73 | 438 | -| 74 | 444 | -| 75 | 450 | -| 76 | 456 | -| 77 | 462 | -| 78 | 468 | -| 79 | 474 | -| 80 | 480 | -| 81 | 486 | -| 82 | 492 | -| 83 | 498 | -| 84 | 504 | -| 85 | 510 | -| 86 | 516 | -| 87 | 522 | -| 88 | 528 | -| 89 | 534 | -| 90 | 540 | -| 91 | 546 | -| 92 | 552 | -| 93 | 558 | -| 94 | 564 | -| 95 | 570 | -| 96 | 576 | -| 97 | 582 | -| 98 | 588 | -| 99 | 594 | -| 100 | 600 | - -> **Note :** If your test executions consistently exceed the recommended runtime or you face test instability, please consider reviewing your test strategy or reaching out to our support team for optimization guidance. diff --git a/static/docs/hyperexecute-sikuli-integration.md b/static/docs/hyperexecute-sikuli-integration.md deleted file mode 100644 index d81957eef..000000000 --- a/static/docs/hyperexecute-sikuli-integration.md +++ /dev/null @@ -1,101 +0,0 @@ -# Integrate Sikuli with HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Sikuli is an open-source tool that empowers you to automate tasks on your computer screen. It utilizes a unique approach called "visual image match" to interact with graphical user interfaces (GUIs). - -This documentation guides you on how to integrate Sikuli seamlessly with HyperExecute to leverage the power of image recognition. - -## Prerequisites -- An active TestMu AI account with Admin or User-level access. -- Refer to the [Sikuli](https://www.softwaretestinghelp.com/sikuli-tutorial-part-1/) documentation to understand how to write a sikuli test. - -## Step 1: Build your Visual Test with SikuliX - -Write your project code and all the other necessary dependencies in your code-repository. - -## Step 2: Configure your HyperExecute YAML - -Here is a sample YAML file, you can configure it with different [YAML flags](/support/docs/hyperexecute-yaml-parameters/) as per your requirements. - -```yaml ---- -version: 0.1 -runson: win -testSuiteTimeout: 90 - -autosplit: true - -retryOnFailure: true -maxRetries: 1 - -concurrency: 1 - -env: -CACHE_DIR: m2_cache_dir - -cacheKey: '{{ checksum "pom.xml" }}' -cacheDirectories: -- $CACHE_DIR - -runtime: -language: java -version: 17 - -pre: -# Create the Cache directory -- mvn -Dmaven.repo.local=./.m2 -Dmaven.test.skip=true clean install - -testDiscovery: -type: raw -mode: remote -shell: bash -command: grep 'Test_1' testng.xml | awk '{print$4}' | sed 's/name=//g' | sed 's/\x3e//g' | sed 's/"//' | sed 's/"//' - -testRunnerCommand: mvn test `-Dtests=$test -``` - -## Step 3: Setup your Authentication / Environment Variables - -After configuring your project and HyperExecute YAML file, you need to setup the CLI and the environment variables. - -### Download the HyperExecute CLI - -The CLI is used for triggering the tests on HyperExecute. It is recommend to download the CLI binary on the host system and keep it in the root directory of the suite to perform the tests on HyperExecute. - -You can download the CLI for your desired platform from the below mentioned links: - -| Platform | HyperExecute CLI | -| ---------| ---------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| MacOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -### Setup Environment Variable - -Now, you need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in the [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). - -Run the below mentioned commands in your terminal to setup the CLI and the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -## Step 4: Execute and Monitor your Project - -**note** -In case of MacOS, if you get a permission denied warning while executing CLI, simply run **`chmod u+x ./hyperexecute`** to allow permission. In case you get a security popup, allow it from your **System Preferences** → **Security & Privacy** → **General tab**. - -Run the below command in your terminal at the root folder of the project: - -```bash -./hyperexecute --config RELATIVE_PATH_OF_YOUR_YAML_FILE -``` - -OR use this command if you have not exported your username and access key in the step 2. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE `} - -> Visit the [HyperExecute Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) and check your Job status. diff --git a/static/docs/hyperexecute-slack-integration.md b/static/docs/hyperexecute-slack-integration.md deleted file mode 100644 index bdf8d9c22..000000000 --- a/static/docs/hyperexecute-slack-integration.md +++ /dev/null @@ -1,48 +0,0 @@ -# Receive Instant Notification on your Jobs Directly in your Slack - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This document details the seamless integration between HyperExecute and Slack, enabling you to streamline your workflow and stay informed about your automated tasks. Through this integration, you can receive real-time notifications and crucial job details directly within your Slack workspace. - -## Prerequisite - -- An active TestMu AI account with Admin or User-level access. -- Set up a dedicated Slack channel where you want to receive notifications from HyperExecute. - -## Step 1: Navigate to the Integration Page - -- Login to your TestMu AI Account. -- Navigate to the **Settings** > **Integration** page. -- Select the **Communication** tab and search for Slack. - -## Step 2: Integrate the Slack with your TestMu AI Account - -- Click on the **Connect** button and then **Install** for the Slack integration. - -> **NOTE :** If you are already logged into Slack, you'll be redirected to a page where you have to post to a channel to confirm your identity or else you will be asked to provide Slack URL of your workspace. - -- Select the channel you want to post on. Click on **Allow** button. - -A notification would be shared on to all the members belonging to that channel, informing about your integration. - -## Step 3: Update the Notification Settings and Trigger the Job - -Once you integrate Slack, you need to configure notification settings to get test automation notifications on your integrated Slack channel. - -- Click on the **Settings** button. -- Choose your **Notification Preferences** like Screenshot test completion messages, Build completion messages, etc. -- Update the **Notification Time** as well. -- Now run the test and visit the Slack channel to view the build notification containing Job Number, Job Status, Executed By, Started At, Job Duration, Test Duration etc. - -## Step 4: Report a bug for a Failed Test - -- Click on the failed test. It will navigate you to the automation page. -- Click on the bug icon. -- A pop-up menu will appear, fill up the details and create a issue for the same. -- The issue will be notified in the slack channel. - -## Remove the Slack Integration - -- Go to the **Settings** > **Integration** page. -- Select the **Communication** tab and search for Slack. -- Click on the **Remove** button. diff --git a/static/docs/hyperexecute-smart-ui-capture-onboarding.md b/static/docs/hyperexecute-smart-ui-capture-onboarding.md deleted file mode 100644 index 422518769..000000000 --- a/static/docs/hyperexecute-smart-ui-capture-onboarding.md +++ /dev/null @@ -1,1065 +0,0 @@ -# SmartUI Capture Onboarding with HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This comprehensive guide will walk you through setting up and running SmartUI Capture tests on HyperExecute. SmartUI Capture allows you to perform visual regression testing by capturing screenshots of static URLs across multiple browsers and devices, all orchestrated through HyperExecute's powerful test execution platform. - -## Prerequisites - -Before you begin, ensure you have the following: - -- **Node.js v20.3 or higher** (required for SmartUI CLI v4.x.x) -- **npm** or **yarn** package manager -- **TestMu AI account** with active subscription -- **HyperExecute CLI** installed ([Download Guide](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/)) -- Basic understanding of: - - Command Line Interface (CLI) - - YAML configuration files - - JSON file structure - - Visual regression testing concepts - -**Node.js Version Requirement** -If you face any problems executing tests with SmartUI-CLI `versions >= v4.x.x`, upgrade your Node.js version to `v20.3` or above. - -## Overview - -SmartUI Capture on HyperExecute combines the power of: -- **SmartUI CLI Capture**: Captures screenshots of static URLs across browsers and devices -- **HyperExecute**: Orchestrates and executes tests at scale with parallel execution -- **Visual Regression Testing**: Compares screenshots to detect UI changes - -This integration is ideal for: -- Testing production and staging environments -- Monitoring website visual consistency -- Detecting UI regressions across multiple pages -- Running large-scale visual tests in parallel - -## Step 1: Create a SmartUI Project - -The first step is to create a SmartUI project in the dashboard. This project will contain all your builds and visual comparisons. - -1. Go to [SmartUI Projects page](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) -2. Click on the **New Project** button -3. Select the platform as **CLI** for executing your capture tests -4. Add the following details: - - **Project Name**: A descriptive name (e.g., "Example Company Visual Tests") - - **Approvers**: Team members who can approve visual changes - - **Tags**: Optional tags for filtering and organization -5. Click on **Submit** or **Continue** - -**Project Token** -After creating the project, you will receive a `PROJECT_TOKEN`. Save this token securely as you'll need it in the next steps. The token format looks like: `123456#1234abcd-****-****-****-************` - -## Step 2: Set Up Your Project Structure - -Create a new directory for your SmartUI Capture project or use an existing one. Here's the recommended structure: - -``` -smartui-capture-project/ -├── config.json # SmartUI configuration -├── urlTest.json # Test environment URLs -├── urlProd.json # Production environment URLs -├── test/ # Optional: Split test URLs -│ ├── urls_test_1.json -│ ├── urls_test_2.json -│ └── ... -├── prod/ # Optional: Split production URLs -│ ├── urls_prod_1.json -│ ├── urls_prod_2.json -│ └── ... -├── hyperexecute.yaml # HyperExecute configuration -└── package.json # Node.js dependencies -``` - -### Initialize Your Project - -If you're starting fresh, initialize a new Node.js project: - -```bash -mkdir smartui-capture-project -cd smartui-capture-project -npm init -y -``` - -## Step 3: Install Dependencies - -Install the required SmartUI CLI package. You can install it globally or locally in your project. - -### Global Installation (Recommended) - -```bash -npm install -g @lambdatest/smartui-cli@4.1.54-beta.0 -``` - -### Local Installation - -```bash -npm install @lambdatest/smartui-cli@4.1.54-beta.0 -``` - -**Sample Repository** -You can also clone the sample repository to get started quickly: - -```bash -git clone https://github.com/LambdaTest/smartui-playwright-sample -cd smartui-playwright-sample -``` - - View on GitHub - -## Step 4: Configure Environment Variables - -Set up your environment variables for authentication and configuration. - -### Set Project Token - -The `PROJECT_TOKEN` is required for SmartUI CLI capture commands. - -```bash -export PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -``` - -```bash -set PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -``` - -```powershell -$env:PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -``` - -### Set HyperExecute Credentials - -For HyperExecute, you'll need your LambdaTest username and access key. - -```bash -export LT_USERNAME="${YOUR_LAMBDATEST_USERNAME}" -export LT_ACCESS_KEY="${YOUR_LAMBDATEST_ACCESS_KEY}" -``` - -```bash -set LT_USERNAME="${YOUR_LAMBDATEST_USERNAME}" -set LT_ACCESS_KEY="${YOUR_LAMBDATEST_ACCESS_KEY}" -``` - -```powershell -$env:LT_USERNAME="${YOUR_LAMBDATEST_USERNAME}" -$env:LT_ACCESS_KEY="${YOUR_LAMBDATEST_ACCESS_KEY}" -``` - -**Getting Your Credentials** -- **Username and Access Key**: Available in your [LambdaTest Profile](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile) -- **Project Token**: Available in your SmartUI project settings after creation - -## Step 5: Create URL Configuration Files - -Create JSON files containing the URLs you want to test. Each URL entry should have a name, URL, and optional wait timeout. - -### Create URL File Using CLI - -You can generate a sample URL file using the SmartUI CLI: - -```bash -npx smartui config:create-web-static urls.json -``` - -### Manual URL File Creation - -Alternatively, create your URL file manually. Here's the structure: - -```json title="urlTest.json" -[ -{ -"name": "homepage", -"url": "https://test.example.com/", -"waitForTimeout": 5000 -}, -{ -"name": "about_page", -"url": "https://test.example.com/about/", -"waitForTimeout": 5000 -}, -{ -"name": "contact_page", -"url": "https://test.example.com/contact/", -"waitForTimeout": 5000 -} -] -``` - -### URL File Structure - -Each URL object supports the following properties: - -| Property | Type | Required | Description | -|----------|------|----------|-------------| -| `name` | string | Yes | Unique identifier for the screenshot (used in SmartUI dashboard) | -| `url` | string | Yes | The URL to capture | -| `waitForTimeout` | number | No | Wait time in milliseconds before capturing (useful for lazy-loaded content) | - -**Wait Timeout Best Practices** -- Use `waitForTimeout` only when necessary (e.g., for lazy-loaded components or async content) -- Higher timeout values increase test execution time -- Start with lower values (1000-3000ms) and increase if needed -- Consider using `execute` options for more complex interactions - -### Splitting URLs for Parallel Execution - -For large test suites, you can split URLs into multiple files for better parallel execution: - -```json title="test/urls_test_1.json" -[ -{ -"name": "product_category_feature_a", -"url": "https://test.example.com/product-category/feature-a/", -"waitForTimeout": 5000 -}, -{ -"name": "product_category_feature_b", -"url": "https://test.example.com/product-category/feature-b/", -"waitForTimeout": 5000 -} -] -``` - -## Step 6: Create SmartUI Configuration - -Create a `config.json` file to configure browsers, viewports, and other SmartUI settings. - -### Generate Configuration File - -```bash -npx smartui config:create config.json -``` - -### Manual Configuration - -Create `config.json` with your desired settings: - -```json title="config.json" -{ -"web": { -"browsers": [ -"safari", -"chrome" -], -"viewports": [ -[1367] -] -}, -"mobile": { -"devices": [ -"iPhone 14", -"iPad 10.2 (2021)", -"Pixel 8" -], -"orientation": "portrait" -}, -"cliEnableJavaScript": true, -"lazyLoadConfiguration": { -"enabled": true, -"jumpBackToTop": true, -"scrollDelay": 250, -"scrollStep": 250 -}, -"waitForTimeout": 5000 -} -``` - -### Configuration Options Explained - -#### Web Configuration - -- **browsers**: Array of browser names (`"chrome"`, `"firefox"`, `"safari"`, `"edge"`) -- **viewports**: Array of viewport sizes - - `[1367]` - Full page screenshot at 1367px width - - `[1920, 1080]` - Viewport screenshot at 1920x1080 - -#### Mobile Configuration - -- **devices**: Array of device names (e.g., `"iPhone 14"`, `"Galaxy S24"`, `"Pixel 8"`) -- **orientation**: `"portrait"` or `"landscape"` -- **fullPage**: `true` by default for mobile - -#### Lazy Loading Configuration - -- **enabled**: Enable lazy loading detection -- **jumpBackToTop**: Scroll back to top after capturing -- **scrollDelay**: Delay between scroll steps (milliseconds) -- **scrollStep**: Pixels to scroll per step - -**Advanced Configuration** -For more configuration options, refer to the [SmartUI SDK Config Options](/support/docs/smartui-sdk-config-options/) documentation. - -## Step 7: Create HyperExecute YAML Configuration - -Create a `hyperexecute.yaml` file to configure HyperExecute execution settings. - -### Basic HyperExecute YAML - -```yaml title="hyperexecute.yaml" ---- -version: 0.1 -globalTimeout: 150 -testSuiteTimeout: 150 -testSuiteStep: 150 - -runson: win - -retryOnFailure: true -maxRetries: 1 - -concurrency: 1 - -env: -CACHE_DIR: node_modules_cache -PROJECT_TOKEN: ${PROJECT_TOKEN} - -# Dependency caching -cacheKey: '{{ checksum "package.json" }}' -cacheDirectories: -- ${CACHE_DIR} - -pre: -# Install SmartUI CLI and dependencies -- npm install @lambdatest/smartui-cli@4.1.54-beta.0 -- npm install playwright@1.57.0 -- npx playwright install - -testSuites: -- npx smartui capture urlTest.json --config config.json --buildName "Test-Release-v1.0" - -jobLabel: ['HYP', 'SmartUI', 'Capture'] -``` - -### Advanced HyperExecute YAML with Multiple Test Suites - -For running multiple URL files in parallel: - -```yaml title="hyperexecute.yaml" ---- -version: 0.1 -globalTimeout: 300 -testSuiteTimeout: 300 -testSuiteStep: 150 - -runson: win - -retryOnFailure: true -maxRetries: 1 - -concurrency: 3 - -env: -CACHE_DIR: node_modules_cache -PROJECT_TOKEN: ${PROJECT_TOKEN} - -cacheKey: '{{ checksum "package.json" }}' -cacheDirectories: -- ${CACHE_DIR} - -pre: -- npm install @lambdatest/smartui-cli@4.1.54-beta.0 -- npm install playwright@1.57.0 -- npx playwright install - -matrix: -urlFile: ["urlTest.json", "urlProd.json"] - -testSuites: -- npx smartui capture ${urlFile} --config config.json --buildName "Build-${urlFile}" - -jobLabel: ['HYP', 'SmartUI', 'Capture', 'Parallel'] -``` - -### HyperExecute YAML Parameters - -| Parameter | Description | Example | -|-----------|-------------|---------| -| `version` | YAML schema version | `0.1` | -| `globalTimeout` | Maximum time for entire job (minutes) | `150` | -| `testSuiteTimeout` | Maximum time per test suite (minutes) | `150` | -| `runson` | Operating system (`win`, `mac`, `linux`) | `win` | -| `concurrency` | Number of parallel test executions | `3` | -| `retryOnFailure` | Retry failed tests | `true` | -| `maxRetries` | Maximum retry attempts | `1` | -| `pre` | Commands to run before test execution | Installation commands | -| `testSuites` | Test commands to execute | SmartUI capture commands | -| `matrix` | Matrix for parallel execution | Multiple URL files | - -**HyperExecute YAML Documentation** -For detailed YAML configuration options, refer to the [HyperExecute YAML Documentation](/support/docs/hyperexecute-yaml-version0.2/). - -## Step 8: Execute Tests on HyperExecute - -### Download HyperExecute CLI - -Download the HyperExecute CLI for your operating system: - -| Platform | Download Link | -|----------|--------------| -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| MacOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -### Set Execute Permissions (MacOS/Linux) - -```bash -chmod u+x ./hyperexecute -``` - -### Run Tests - -Execute your tests using the HyperExecute CLI: - -```bash -./hyperexecute --config hyperexecute.yaml -``` - -Or with explicit credentials: - -```bash -./hyperexecute --user ${YOUR_LAMBDATEST_USERNAME} --key ${YOUR_LAMBDATEST_ACCESS_KEY} --config hyperexecute.yaml -``` - -### Capture Command Options - -You can enhance your capture commands with additional options: - -```bash -# With custom build name -npx smartui capture urlTest.json --config config.json --buildName "Release-v1.0" - -# With results export -npx smartui capture urlTest.json --config config.json --fetch-results results.json - -# With parallel execution -npx smartui capture urlTest.json --config config.json --parallel 3 - -# Combined options -npx smartui capture urlTest.json --config config.json --buildName "Release-v1.0" --fetch-results results.json --parallel 3 -``` - -## Step 9: Monitor Test Execution - -### HyperExecute Dashboard - -1. Visit the [HyperExecute Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) -2. Find your job in the job list -3. Click on the job to view detailed execution logs -4. Monitor test progress in real-time - -### SmartUI Dashboard - -1. After test execution completes, navigate to [SmartUI Dashboard](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) -2. Select your project -3. View the build with your specified build name -4. Review captured screenshots -5. Compare with baseline images (if available) - -### Understanding Results - -- **Baseline Build**: First build in a project (reference images) -- **Comparison Build**: Subsequent builds compared against baseline -- **Mismatches**: Visual differences detected between builds -- **Approved**: Manually approved visual changes -- **Rejected**: Visual changes that need fixing - -### Using SmartUI Reporter Tool - -The [SmartUI Reporter](https://smartui-reporter.netlify.app/) is a web-based tool that provides a comprehensive tabular view of your test results with statistics and export capabilities. - -#### Step 1: Export Results JSON - -**From CLI Capture Command:** -```bash -npx smartui capture urlTest.json --config config.json --fetch-results results.json -``` - -**From CLI Exec Command:** -```bash -npx smartui --config config.json exec --fetch-results results.json -- -``` - -**From SmartUI API:** -You can also fetch results using the [Fetch Build Screenshots API endpoint](https://swagger-api-support.lambdatest.com/smartui/index.html#/Fetch%20Build%20Screenshots): -``` -GET /build/screenshots?project_id=YOUR_PROJECT_ID&build_id=YOUR_BUILD_ID -``` - -#### Step 2: Upload to Reporter - -1. Visit [SmartUI Reporter](https://smartui-reporter.netlify.app/) -2. Upload your `results.json` file (drag & drop or click to upload) -3. View results in a tabular format with: - - Screenshot statistics - - Browser and viewport information - - Mismatch percentages - - Status indicators - - Direct links to baseline, captured, and diff images - -#### Step 3: Export Options - -- **Export to PDF**: Generate a comprehensive PDF report for sharing -- **Export to CSV**: Export data for analysis in spreadsheet applications - -For detailed information, see the [Fetch Results Documentation](/support/docs/smartui-sdk-fetch-results/#using-smartui-reporter-tool). - -## Best Practices - -### 1. URL Organization - -**Separate Test and Production URLs** - -Keep test and production URLs in separate files: - -- `urlTest.json` - Staging/test environment -- `urlProd.json` - Production environment - -This allows you to test both environments independently. - -**Split Large URL Lists** - -For projects with many URLs, split them into multiple files: - -``` -test/ -├── urls_test_1.json (10 URLs) -├── urls_test_2.json (10 URLs) -└── urls_test_3.json (10 URLs) -``` - -This enables better parallel execution and easier management. - -**Use Descriptive Names** - -Use clear, descriptive names for your URLs: - -```json -{ -"name": "product_category_feature_a", -"url": "https://example.com/product-category/feature-a/" -} -``` - -Avoid generic names like `"page1"` or `"test"`. - -### 1.1. URL Grouping Strategies for Large Test Suites - -When working with large numbers of URLs (e.g., 1000+ URLs) that can be logically divided into sections or groups (e.g., Section 1: Product Category A, Section 2: Product Category B, Section 3: Product Category C, etc.), you have two primary strategies for organizing your SmartUI projects: - -**Single Project with Branching Strategy** - -Use a single SmartUI project and leverage SmartUI's branching capabilities to organize different URL groups within the same project. - -#### How It Works - -- **Single Project**: All URL groups/sections are managed in one SmartUI project -- **Branching Strategy**: Use SmartUI's [Smart Git feature](/support/docs/smartui-smart-git-strategy/) to create separate branches for each section/group -- **Unified Tracking**: All URLs and automation are tracked in a single project dashboard -- **Independent Baselines**: Each branch maintains its own baseline and comparison history - -#### Implementation - -**Step 1: Create URL Groups** - -Organize your URLs into logical groups: - -``` -project/ -├── sections/ -│ ├── section1_category_a.json (200 URLs) -│ ├── section2_category_b.json (200 URLs) -│ ├── section3_category_c.json (200 URLs) -│ ├── section4_category_d.json (200 URLs) -│ └── section5_category_e.json (200 URLs) -├── config.json -└── hyperexecute.yaml -``` - -**Step 2: Configure HyperExecute with Matrix** - -Use HyperExecute matrix to run each section: - -```yaml title="hyperexecute.yaml" ---- -version: 0.1 -globalTimeout: 300 -testSuiteTimeout: 150 - -runson: win -concurrency: 5 - -env: -PROJECT_TOKEN: ${PROJECT_TOKEN} -SMART_GIT: true # Enable Smart Git for branch management - -pre: -- npm install @lambdatest/smartui-cli@4.1.54-beta.0 - -matrix: -section: ["section1_category_a", "section2_category_b", "section3_category_c", "section4_category_d", "section5_category_e"] - -testSuites: -- npx smartui capture sections/${section}.json --config config.json --buildName "${section}-Build" -``` - -**Step 3: Use Branching for Organization** - -Each section can run in its own branch or use build names to group: - -```bash -# Option A: Use Smart Git with branches -export SMART_GIT=true -git checkout -b section1-category-a -npx smartui capture sections/section1_category_a.json --config config.json --buildName "Section1-CategoryA" - -# Option B: Use build names for grouping (single branch) -npx smartui capture sections/section1_category_a.json --config config.json --buildName "Section1-CategoryA-Build" -``` - -#### Use Cases - -✅ **Best for:** -- Teams that want centralized project management -- Organizations needing unified reporting and analytics -- Projects where sections share common configuration -- Teams that want to track all URLs in one dashboard -- Scenarios requiring cross-section comparisons - -✅ **Benefits:** -- Single project dashboard for all sections -- Unified baseline management -- Easier cross-section analysis -- Simplified project administration -- Single project token to manage - -❌ **Not ideal for:** -- Teams requiring completely isolated section management -- Organizations with different approval workflows per section -- Projects where sections have vastly different configurations - -**Separate Projects Strategy** - -Create individual SmartUI projects for each URL group/section, providing complete isolation and independent management. - -#### How It Works - -- **Multiple Projects**: Each section/group has its own dedicated SmartUI project -- **Independent Management**: Each project operates independently with its own baseline, approvals, and settings -- **Minimal Projects**: Each project contains only one branch (typically `main` or `master`) -- **Clear Separation**: Complete bifurcation between different sections - -#### Implementation - -**Step 1: Create Separate Projects** - -Create a SmartUI project for each section: - -- Project 1: "Product Category A Visual Tests" → `PROJECT_TOKEN_1` -- Project 2: "Product Category B Visual Tests" → `PROJECT_TOKEN_2` -- Project 3: "Product Category C Visual Tests" → `PROJECT_TOKEN_3` -- Project 4: "Product Category D Visual Tests" → `PROJECT_TOKEN_4` -- Project 5: "Product Category E Visual Tests" → `PROJECT_TOKEN_5` - -**Step 2: Organize URL Files** - -``` -project/ -├── category_a/ -│ ├── urls_category_a.json -│ ├── config.json -│ └── hyperexecute_category_a.yaml -├── category_b/ -│ ├── urls_category_b.json -│ ├── config.json -│ └── hyperexecute_category_b.yaml -└── category_c/ -├── urls_category_c.json -├── config.json -└── hyperexecute_category_c.yaml -``` - -**Step 3: Configure Each Project Separately** - -Each project has its own configuration and HyperExecute YAML: - -```yaml title="category_a/hyperexecute_category_a.yaml" ---- -version: 0.1 -globalTimeout: 150 -testSuiteTimeout: 150 - -runson: win -concurrency: 2 - -env: -PROJECT_TOKEN: ${PROJECT_TOKEN_CATEGORY_A} # Unique token per project - -pre: -- npm install @lambdatest/smartui-cli@4.1.54-beta.0 - -testSuites: -- npx smartui capture urls_category_a.json --config config.json --buildName "CategoryA-Build" -``` - -**Step 4: Execute Projects Independently** - -Run each project separately or in parallel: - -```bash -# Execute category A project -cd category_a -export PROJECT_TOKEN="${PROJECT_TOKEN_CATEGORY_A}" -./hyperexecute --config hyperexecute_category_a.yaml - -# Execute category B project -cd ../category_b -export PROJECT_TOKEN="${PROJECT_TOKEN_CATEGORY_B}" -./hyperexecute --config hyperexecute_category_b.yaml -``` - -#### Use Cases - -✅ **Best for:** -- Teams requiring complete section isolation -- Organizations with different approval workflows per section -- Projects where sections have independent release cycles -- Teams needing separate access controls per section -- Scenarios where sections have different configurations - -✅ **Benefits:** -- Complete isolation between sections -- Independent baseline management -- Separate approval workflows -- Individual project settings per section -- Clear project boundaries - -❌ **Not ideal for:** -- Teams wanting unified reporting -- Organizations needing cross-section analysis -- Projects with shared configurations -- Teams preferring centralized management - -#### Decision Matrix - -Use this matrix to decide which strategy fits your needs: - -| Requirement | Single Project with Branching | Separate Projects | -|-------------|------------------------------|-------------------| -| **Unified Dashboard** | ✅ Yes | ❌ No | -| **Cross-Section Analysis** | ✅ Yes | ❌ No | -| **Complete Isolation** | ❌ No | ✅ Yes | -| **Independent Approvals** | ⚠️ Partial (via branches) | ✅ Yes | -| **Project Management Complexity** | ✅ Lower | ❌ Higher | -| **Token Management** | ✅ Single token | ❌ Multiple tokens | -| **Configuration Sharing** | ✅ Easy | ❌ Requires duplication | -| **Reporting & Analytics** | ✅ Unified | ❌ Separate per project | - -#### Hybrid Approach - -You can also combine both strategies: - -- **Major Sections**: Use separate projects (e.g., "Production URLs" vs "Staging URLs") -- **Sub-sections**: Use branching within each project (e.g., within "Production URLs", use branches for different product categories) - -```bash -# Production Project - Category A Section -export PROJECT_TOKEN="${PROJECT_TOKEN_PROD}" -export SMART_GIT=true -git checkout -b production-category-a -npx smartui capture sections/category_a_prod.json --config config.json - -# Staging Project - Category A Section -export PROJECT_TOKEN="${PROJECT_TOKEN_STAGING}" -export SMART_GIT=true -git checkout -b staging-category-a -npx smartui capture sections/category_a_staging.json --config config.json -``` - -#### Best Practices for Large URL Sets - -1. **Logical Grouping**: Group URLs by functionality, product category, or business domain -2. **Consistent Naming**: Use consistent naming conventions across groups -3. **Documentation**: Document which URLs belong to which section -4. **Regular Review**: Periodically review and reorganize groups as needed -5. **Parallel Execution**: Leverage HyperExecute's parallel execution for faster test runs - -### 2. Configuration Management - -**Select Relevant Browsers** - -Only test browsers that your users actually use: - -```json -{ -"web": { -"browsers": ["chrome", "safari", "firefox"] -} -} -``` - -Avoid testing unnecessary browsers to reduce execution time and costs. - -**Optimize Viewport Sizes** - -Test realistic viewport sizes: - -```json -{ -"viewports": [ -[1920], // Desktop -[1366], // Laptop -[768] // Tablet -] -} -``` - -Focus on viewports your users actually use. - -**Optimize Wait Timeouts** - -Start with minimal timeouts and increase only when needed: - -```json -{ -"waitForTimeout": 2000 // Start low, increase if needed -} -``` - -Higher timeouts increase execution time. - -### 3. HyperExecute Configuration - -**Optimize Concurrency** - -Set appropriate concurrency based on your test suite size: - -```yaml -concurrency: 3 # For 20-30 URLs -``` - -Too high concurrency may cause resource issues. - -**Set Realistic Timeouts** - -Configure timeouts based on your test duration: - -```yaml -globalTimeout: 300 # 5 hours for large suites -testSuiteTimeout: 150 # 2.5 hours per suite -``` - -Account for network latency and page load times. - -**Enable Dependency Caching** - -Cache node_modules to speed up subsequent runs: - -```yaml -cacheKey: '{{ checksum "package.json" }}' -cacheDirectories: -- node_modules_cache -``` - -This significantly reduces setup time. - -### 4. Build Naming - -Use descriptive build names that include: -- Environment (Test/Prod) -- Version or release number -- Date or timestamp - -```bash ---buildName "Test-Release-v1.0-2024-01-15" -``` - -### 5. Regular Baseline Updates - -- Update baselines when intentional UI changes are made -- Review and approve changes regularly -- Keep baseline builds organized with clear naming - -## Troubleshooting - -### Common Issues and Solutions - -**Issue**: `PROJECT_TOKEN is not set` or authentication failures - -**Solutions**: -1. Verify `PROJECT_TOKEN` is set correctly: -```bash -echo $PROJECT_TOKEN # MacOS/Linux -echo %PROJECT_TOKEN% # Windows CMD -``` - -2. Check token format (should include `#`): -``` -123456#1234abcd-****-****-****-************ -``` - -3. Verify token in SmartUI dashboard project settings - -4. For HyperExecute, also verify `LT_USERNAME` and `LT_ACCESS_KEY` - -**Issue**: Errors with SmartUI CLI v4.x.x - -**Solutions**: -1. Check Node.js version: -```bash -node --version # Should be v20.3 or higher -``` - -2. Upgrade Node.js if needed: -```bash -# Using nvm -nvm install 20.3 -nvm use 20.3 -``` - -3. Verify npm version: -```bash -npm --version -``` - -**Issue**: Tests timing out or incomplete captures - -**Solutions**: -1. Increase `waitForTimeout` in URL files: -```json -{ -"name": "slow-page", -"url": "https://example.com/", -"waitForTimeout": 10000 -} -``` - -2. Increase HyperExecute timeouts: -```yaml -globalTimeout: 300 -testSuiteTimeout: 300 -``` - -3. Check network connectivity and page load times - -**Issue**: Some URLs not captured or missing in dashboard - -**Solutions**: -1. Verify URLs are accessible and return 200 status -2. Check for JavaScript errors on pages -3. Ensure `waitForTimeout` is sufficient for page load -4. Review SmartUI dashboard for error messages -5. Check HyperExecute logs for execution errors - -**Issue**: Tests failing with parallel execution - -**Solutions**: -1. Reduce concurrency: -```yaml -concurrency: 1 # Start with 1, increase gradually -``` - -2. Split URLs into smaller files -3. Check system resources and network bandwidth -4. Verify HyperExecute account limits - -**Issue**: HyperExecute YAML parsing errors - -**Solutions**: -1. Validate YAML syntax using online validators -2. Check indentation (YAML is space-sensitive) -3. Verify all required fields are present -4. Review [HyperExecute YAML documentation](/support/docs/hyperexecute-yaml-version0.2/) - -### Debugging Tips - -1. **Test Locally First**: Run SmartUI capture locally before HyperExecute: -```bash -npx smartui capture urlTest.json --config config.json -``` - -2. **Check Logs**: Review HyperExecute job logs for detailed error messages - -3. **Verify Configuration**: Use `npx smartui --help` to verify CLI installation - -4. **Test Individual URLs**: Test problematic URLs individually to isolate issues - -5. **Monitor Dashboard**: Check both HyperExecute and SmartUI dashboards for errors - -## Advanced Usage - -### CI/CD Integration - -Integrate SmartUI Capture with your CI/CD pipeline: - -```yaml title=".github/workflows/smartui-capture.yml" -name: SmartUI Capture Tests - -on: -pull_request: -branches: [main] -workflow_dispatch: - -env: -LT_USERNAME: ${{ secrets.LT_USERNAME }} -LT_ACCESS_KEY: ${{ secrets.LT_ACCESS_KEY }} -PROJECT_TOKEN: ${{ secrets.PROJECT_TOKEN }} - -jobs: -smartui-capture: -name: Execute SmartUI Capture -runs-on: ubuntu-latest -steps: -- name: Checkout repo -uses: actions/checkout@v2 - -- name: Install Dependencies -run: | -npm install @lambdatest/smartui-cli@4.1.54-beta.0 -npm install playwright@1.57.0 -npx playwright install - -- name: Run SmartUI Capture -run: | -npx smartui capture urlTest.json --config config.json --buildName "PR-${{ github.event.pull_request.number }}" - -- name: Fetch Results -run: | -npx smartui capture urlTest.json --config config.json --fetch-results results.json -``` - -### Scheduled Test Runs - -Use the `--scheduled` flag for scheduled test executions: - -```bash -npx smartui capture urlProd.json --config config.json --scheduled "schedule-123" -``` - -### Custom Build Names with Variables - -Use environment variables in build names: - -```yaml -testSuites: -- npx smartui capture urlTest.json --config config.json --buildName "Build-${BUILD_NUMBER}" -``` - -## Next Steps - -Now that you've set up SmartUI Capture with HyperExecute, explore these resources: - -- **[SmartUI Dashboard Guide](/support/docs/smartui-guided-walkthrough/)**: Learn to manage tests and approve changes -- **[Baseline Management](/support/docs/smartui-baseline-management/)**: Understand baseline workflows -- **[Handling Dynamic Content](/support/docs/smartui-handle-dynamic-data/)**: Test pages with dynamic elements -- **[Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide/)**: Comprehensive troubleshooting solutions -- **[SmartUI CLI Reference](/support/docs/smartui-cli-complete-reference/)**: Complete CLI command reference -- **[HyperExecute Documentation](/support/docs/hyperexecute-running-your-first-job/)**: Learn more about HyperExecute - -## Additional Resources - -- [SmartUI CLI Documentation](/support/docs/smartui-cli/) - Complete CLI guide -- [SmartUI Configuration Options](/support/docs/smartui-sdk-config-options/) - All configuration options -- [HyperExecute YAML Reference](/support/docs/hyperexecute-yaml-version0.2/) - YAML configuration details -- [SmartUI Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide/) - Common issues and solutions -- [LambdaTest Support](https://www.testmuai.com/support) - Get help from our support team diff --git a/static/docs/hyperexecute-smart-ui-sdk-maestro.md b/static/docs/hyperexecute-smart-ui-sdk-maestro.md deleted file mode 100644 index 375d1801c..000000000 --- a/static/docs/hyperexecute-smart-ui-sdk-maestro.md +++ /dev/null @@ -1,128 +0,0 @@ -# UI Testing with Maestro using HyperExecute and SmartUI SDK - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This document guides you on how to perform visual regression testing for your Android applications by running your Maestro tests on the HyperExecute platform and capturing screenshots with the SmartUI SDK. - -## Prerequisites - -Before you begin, ensure you have the following prerequisites in place: - -* Your TestMu AI Username and Access Key. You can find these on your dashboard -> left side bar -> credentials. -* The [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) downloaded to your local machine. -* An existing Android application file (`.apk`). -* Node.js and npm installed on your system. - -## Step 1: Create a SmartUI Project - -First, you need to create a project in SmartUI. All your visual regression test builds for this project will be grouped here. - -1. Navigate to the [SmartUI Projects page](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/). -2. Click on **New Project**. -3. Select **CLI** as the platform. -4. Provide a **Project Name**, select approvers, and add any relevant tags. -5. Click **Submit**. - -Once the project is created, you will get a unique `PROJECT_TOKEN`. Copy this token as it is required in the HyperExecute YAML configuration file later. - -## Step 2: Clone The Sample Repository - -You can configure your own existing Maestro project or get started quickly by cloning our sample repository. - -```bash -git clone https://github.com/LambdaTest/smartui-maestro-sample -cd smartui-maestro-sample -``` - -**Sample Repo** -You can find the complete code for this example in the following TestMu AI GitHub repository. - - View on GitHub - -## Step 3: Upload Your Application - -To run your tests on TestMu AI real devices or emulators, you need to upload your Android app (`.apk`) and get an `appId`. - -Run the following `cURL` command in your terminal. Ensure you have set your `LT_USERNAME` and `LT_ACCESS_KEY` as environment variables. - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "appFile=@""" -F "name="sampleApp"" -`} - -From the JSON response, copy the value of `app_url`. It will be in the format `lt://...`. This is your `appId`. - -## Step 4: Configure Your Test Suite - -Now, let's set up the test suite to integrate with SmartUI. - -#### 4.1. Install SmartUI CLI - -The SmartUI CLI is required to capture and upload screenshots. Install it in your project directory: - -```bash -npm install @lambdatest/smartui-cli -``` - -If you face any problems executing tests with SmartUI-CLI `versions >= v4.x.x`, upgrade your Node.js version to `v20.3` or above. - -#### 4.2. Add SmartUI Screenshots to Your Maestro Flow - -In your Maestro test YAML files (e.g., `maestro-test/test.yaml`), add the `takeScreenshot` command wherever you want to capture a visual checkpoint. The `path` specified will be used to name the screenshot in the SmartUI dashboard. - -```yaml -- launchApp -- takeScreenshot: -path: ./screenshots/home-screen -- tapOn: "Login" -- takeScreenshot: -path: ./screenshots/login-page -``` - -#### 4.3. Set Up Environment Variables - -Now, you need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in the [TestMu AI Username and Access Key Page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/security/username-accesskey). - -Run the below mentioned commands in your terminal to setup the CLI and the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -## Step 5: Configure the HyperExecute YAML - -The HyperExecute YAML file is the blueprint for your test execution. It tells HyperExecute how to set up the environment and run your tests. - -You need to add your `PROJECT_TOKEN` (from Step 1) and your `appId` (from Step 3) to the YAML file. Below are example configurations for running Maestro tests on Emulators and Real Devices. - -```yaml reference title="maestro_Real_Devices.yaml" -https://github.com/LambdaTest/smartui-maestro-sample/blob/main/yaml/maestro_Real_Devices.yaml -``` - -> To enable this for your organizaton, connect with us through our **24/7 chat support** or drop us an email to [support@testmuai.com](mailto:support@testmuai.com). - -```yaml reference title="maestro_Emulator_HyperEx.yaml" -https://github.com/LambdaTest/smartui-maestro-sample/blob/main/yaml/maestro_Emulator_HyperEx.yaml -``` - -In the real device configuration, the `testDiscovery` and `testRunnerCommand` are handled internally by the `maestro` framework integration on HyperExecute. You just need to specify the `framework` block. - -## Step 6: Execute Your Test Suite - -Run the below command in your terminal at the root folder of the project: - -```bash -./hyperexecute --config RELATIVE_PATH_OF_YOUR_YAML_FILE -``` - -OR use this command if you have not exported your username and access key in the step 2. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE `} - -## Step 7: View Your Visual Test Results on SmartUI - -After the HyperExecute job is finished, all screenshots captured using the `takeScreenshot` command will be available in your SmartUI project for review. - -1. Navigate to the [SmartUI Dashboard](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/). -2. Select your project and the most recent build. -3. Here, you can compare screenshots against the baseline, highlight differences, and approve or reject changes. You can use features like Smart Ignore to mask dynamic areas for more stable comparisons. diff --git a/static/docs/hyperexecute-smart-ui-sdk-selenium-csharp.md b/static/docs/hyperexecute-smart-ui-sdk-selenium-csharp.md deleted file mode 100644 index 0cbe1ff9b..000000000 --- a/static/docs/hyperexecute-smart-ui-sdk-selenium-csharp.md +++ /dev/null @@ -1,184 +0,0 @@ -# UI Testing with C# using HyperExecute and SmartUI SDK - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This documentation will guide you step-by-step to execute the SmartUI SDK tests on the HyperExecute platform using **Selenium - C Sharp** - -## Prerequisites - -To run the Tests on HyperExecute from your Local System, you are required: - -- Your TestMu AI [Username and Access key](/support/docs/hyperexecute-how-to-get-my-username-and-access-key/) -- [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) in order to initiate a test execution Job . -- Setup the [Environmental Variable](/support/docs/hyperexecute-environment-variable-setup/) -- [HyperExecute YAML](/support/docs/hyperexecute-yaml-version0.2/) file which contains all the necessary instructions. -- Login to [TestMu AI SmartUI](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) with your credentials. - -## Step 1: Create a SmartUI Project - -The first step is to create a project with the application in which we will combine all your builds run on the project. To create a SmartUI Project, follow these steps: - -1. Go to [Projects page](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) -2. Click on the `new project` button -3. Select the platform as CLI or Web for executing your `SDK` tests. -4. Add name of the project, approvers for the changes found, tags for any filter or easy navigation. -5. Click on the **Submit**. - -After creating the project, you will get your `PROJECT_TOKEN`. You need to keep this project token safe as it will be used in the further steps below. - -## Step 2: Setup Your Test Suite - -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -### Update the Dependencies - -- Add the following dependencies in your `.csproj` file - -```cs - - - -``` - -You can check the latest version of [TestMu AI.Selenium.Driver]( https://www.nuget.org/packages/LambdaTest.Selenium.Driver) and update the latest version accordingly. - -### Configuring your Project -You can configure your project meta information from here, like build name, project name, defining the platform, browser, browser version, your credentials etc. - -```javascript title="LTCloudTest.cs" -var capabilities = new JObject { -["browserName"] = "Chrome", -["browserVersion"] = "latest", -["platformName"] = "Windows 10", -// highlight-next-line -["name"] = "", -// highlight-next-line -["build"] = "", -["user"] = username, -["accessKey"] = accessKey, -["network"] = true, -["video"] = true, -["console"] = true -}; -``` - -### Adding SmartUI function to take screenshot - -You can incorporate SmartUI into your custom `Selenium` automation test (any platform) script by adding the `smartuiSnapshot` function in the required segment of selenium script of which we would like to take the screenshot, as shown below: - -```java title="LTCloudTest.cs" -using System; -using System.Threading.Tasks; -using OpenQA.Selenium; -using OpenQA.Selenium.Chrome; -using LambdaTest.Selenium.Driver; - -namespace LambdaTest.Selenium.TestProject -{ -public static class LocalTest{ -public static async Task Run(){ -using IWebDriver driver = new ChromeDriver(); -try { -Console.WriteLine("Driver started"); -driver.Navigate().GoToUrl("Required URL"); -// highlight-next-line -await SmartUISnapshot.CaptureSnapshot(driver, "Screenshot Name"); //utilize this function to take the dom snapshot of your test -}catch (Exception ex) { -Console.WriteLine(ex); -}finally { -driver.Quit(); -} -} -} -} -``` - -## Step 3: Setup the CLI in your Test Suite - -After cloning / downloading the sample repo, you need to setup the CLI and the environment variables. - -### Download the HyperExecute CLI - -The CLI is used for triggering the tests on HyperExecute. It is recommend to download the CLI binary on the host system and keep it in the root directory of the suite to perform the tests on HyperExecute. - -You can download the CLI for your desired platform from the below mentioned links: - -| Platform | HyperExecute CLI | -| ---------| ---------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| MacOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -### Setup Environment Variable - -Now, you need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in the [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). - -Run the below mentioned commands in your terminal to setup the CLI and the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -## Step 4: Configure YAML in your Test Suite - -You need to edit the `PROJECT_TOKEN: "YOUR_PROJECT_TOKEN"` flag and enter your project token that show in the SmartUI app after, creating your project. - -```yaml title="hyperexecute.yaml" ---- -version: 0.1 -runson: linux - -autosplit: true -concurrency: 1 - -runtime: -language: dotnet -version: "8.0" - -env: -# PAT: ${{ .secrets.testKey }} -PROJECT_TOKEN: - -cacheKey: '{{ checksum "package-lock.json" }}' -cacheDirectories: -- node_modules - -pre: -- dotnet clean -- dotnet build -- npm install @lambdatest/smartui-cli -- dotnet restore -- npx smartui config:create .smartui.json - -testDiscovery: -type: raw -mode: remote -command: echo cloud - -testRunnerCommand: npx smartui --config .smartui.json exec -- dotnet run $test - -jobLabel: ["smart-ui-sdk", "hyperexecute", "selenium", "csharp"] -``` - -## Step 5: Execute your Test Suite - -> **NOTE :** In case of macOS, if you get a permission denied warning while executing CLI, simply run **`chmod u+x ./hyperexecute`** to allow permission. In case you get a security popup, allow it from your **System Preferences** → **Security & Privacy** → **General tab**. - -Run the below command in your terminal at the root folder of the project: - -```bash -./hyperexecute --config RELATIVE_PATH_OF_YOUR_YAML_FILE -``` - -OR use this command if you have not exported your username and access key in the step 3. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE `} - -## Step 6: Monitor the Test Execution - -Visit the [HyperExecute Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) and check your Job status. - -**discover more** -📕 Learn more about the other [supported arguments](/support/docs/smartui-selenium-csharp-sdk), how you can [handle the dynamic data](/support/docs/smartui-selenium-csharp-sdk), and how to [capture the screenshot of a specific element](/support/docs/smartui-selenium-csharp-sdk) diff --git a/static/docs/hyperexecute-smart-ui-sdk-selenium-javascript.md b/static/docs/hyperexecute-smart-ui-sdk-selenium-javascript.md deleted file mode 100644 index 9c975ea26..000000000 --- a/static/docs/hyperexecute-smart-ui-sdk-selenium-javascript.md +++ /dev/null @@ -1,149 +0,0 @@ -# UI Testing with Node.js using HyperExecute and SmartUI SDK - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This documentation will guide you step-by-step to execute the SmartUI SDK tests on the HyperExecute platform using **Selenium - JavaScript** - -## Prerequisites - -To run the Tests on HyperExecute from your Local System, you are required: - -- Your TestMu AI [Username and Access key](/support/docs/hyperexecute-how-to-get-my-username-and-access-key/) -- [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) in order to initiate a test execution Job . -- Setup the [Environmental Variable](/support/docs/hyperexecute-environment-variable-setup/) -- [HyperExecute YAML](/support/docs/hyperexecute-yaml-version0.2/) file which contains all the necessary instructions. -- Login to [TestMu AI SmartUI](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) with your credentials. - -## Step 1: Create a SmartUI Project - -The first step is to create a project with the application in which we will combine all your builds run on the project. To create a SmartUI Project, follow these steps: - -1. Go to [Projects page](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) -2. Click on the `new project` button -3. Select the platform as CLI or Web for executing your `SDK` tests. -4. Add name of the project, approvers for the changes found, tags for any filter or easy navigation. -5. Click on the **Submit**. - -After creating the project, you will get your `PROJECT_TOKEN`. You need to keep this project token safe as it will be used in the further steps below. - -## Step 2: Setup Your Test Suite - -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** -Download or Clone the code sample for the Selenium - JavaScript framework from the TestMu AI GitHub repository to run the tests on the HyperExecute. - - View on GitHub - -### Configure Your Test Suite - -In the `sdkCloud.js` file, update the following capabilities - -```java -let capabilities = { -"LT:Options": { -project: "PROJECT_NAME", // Enter your Project Name -name: "TEST_NAME", // Provide the name of your test -build: "BUILD_NAME", // Mention your desired build name -}, -}; -``` - -## Step 3: Setup the CLI in your Test Suite - -After cloning / downloading the sample repo, you need to setup the CLI and the environment variables. - -### Download the HyperExecute CLI - -The CLI is used for triggering the tests on HyperExecute. It is recommend to download the CLI binary on the host system and keep it in the root directory of the suite to perform the tests on HyperExecute. - -You can download the CLI for your desired platform from the below mentioned links: - -| Platform | HyperExecute CLI | -| ---------| ---------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| MacOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -### Setup Environment Variable - -Now, you need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in the [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). - -Run the below mentioned commands in your terminal to setup the CLI and the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -## Step 4: Configure YAML in your Test Suite - -You need to edit the `PROJECT_TOKEN: "YOUR_PROJECT_TOKEN"` flag and enter your project token that show in the SmartUI app after, creating your project. - -```yaml ---- -version: 0.1 -globalTimeout: 90 -testSuiteTimeout: 90 -testSuiteStep: 90 - -runson: win - -autosplit: true - -retryOnFailure: true -maxRetries: 1 - -concurrency: 1 - -env: -CACHE_DIR: m2_cache_dir -PROJECT_TOKEN: "YOUR_PROJECT_TOKEN" #Enter your project token here - -cacheKey: '{{ checksum "package.json" }}' -cacheDirectories: -- node_modules - -pre: -- npm install @lambdatest/smartui-cli @lambdatest/selenium-driver selenium-webdriver -- npx smartui config:create smartui-web.json - -post: -- cat hyp-smartui-sdk.yaml - -testDiscovery: -type: raw -mode: static -command: ls sdk/sdkCloud.js - -testRunnerCommand: npx smartui exec node sdk/sdkCloud.js --config smartui-web.json - -jobLabel: ["smart-ui-sdk", "hyperexecute", "selenium", "javascript"] -``` - -It is mandatory to mention these commands in the pre flag to download all the necessary dependencies - -```bash -pre: -- npm install @lambdatest/smartui-cli @lambdatest/selenium-driver selenium-webdriver -- npx smartui config:create smartui-web.json -``` - -## Step 5: Execute your Test Suite - -> **NOTE :** In case of macOS, if you get a permission denied warning while executing CLI, simply run **`chmod u+x ./hyperexecute`** to allow permission. In case you get a security popup, allow it from your **System Preferences** → **Security & Privacy** → **General tab**. - -Run the below command in your terminal at the root folder of the project: - -```bash -./hyperexecute --config RELATIVE_PATH_OF_YOUR_YAML_FILE -``` - -OR use this command if you have not exported your username and access key in the step 3. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE `} - -## Step 6: Monitor the Test Execution - -Visit the [HyperExecute Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) and check your Job status. diff --git a/static/docs/hyperexecute-smart-ui-sdk-using-cypress.md b/static/docs/hyperexecute-smart-ui-sdk-using-cypress.md deleted file mode 100644 index 0a666dd53..000000000 --- a/static/docs/hyperexecute-smart-ui-sdk-using-cypress.md +++ /dev/null @@ -1,159 +0,0 @@ -# UI Testing with Cypress using HyperExecute and SmartUI SDK - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This documentation will guide you step-by-step to execute the SmartUI tests on the HyperExecute platform using Selenium - -> **Note :** SmartUI SDK only supports Cypress versions >= 10.0.0 - -## Prerequisites - -To run the Tests on HyperExecute from your Local System, you are required: - -- Your TestMu AI [Username and Access key](/support/docs/hyperexecute-how-to-get-my-username-and-access-key/) -- [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) in order to initiate a test execution Job . -- Setup the [Environmental Variable](/support/docs/hyperexecute-environment-variable-setup/) -- [HyperExecute YAML](/support/docs/hyperexecute-yaml-version0.2/) file which contains all the necessary instructions. -- Login to [TestMu AI SmartUI](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) with your credentials. - -## Step 1: Create a SmartUI Project - -The first step is to create a project with the application in which we will combine all your builds run on the project. To create a SmartUI Project, follow these steps: - -1. Go to [Projects page](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) -2. Click on the `new project` button -3. Select the platform as CLI or Web for executing your `SDK` tests. -4. Add name of the project, approvers for the changes found, tags for any filter or easy navigation. -5. Click on the **Submit**. - -After creating the project, you will get your `PROJECT_TOKEN`. You need to keep this project token safe as it will be used in the further steps below. - -## Step 2: Setup Your Test Suite - -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** - -Download or Clone the code sample for the Maestro framework from the TestMu AI GitHub repository to run the tests on the HyperExecute. - - View on GitHub - -### Adding SmartUI function to take screenshot -You can incorporate SmartUI into your custom `Cypress` test script, as shown below: - -```javascript -/// -describe('Test Case name', () => { -beforeEach(() => { - -cy.visit('Required URL') -}) - -it('SmartUI Snapshot', () => { -cy.smartuiSnapshot('Screenshot Name'); -}) -}) -``` - -## Step 3: Setup the CLI in your Test Suite - -After cloning / downloading the sample repo, you need to setup the CLI and the environment variables. - -### Download the HyperExecute CLI - -The CLI is used for triggering the tests on HyperExecute. It is recommend to download the CLI binary on the host system and keep it in the root directory of the suite to perform the tests on HyperExecute. - -You can download the CLI for your desired platform from the below mentioned links: - -| Platform | HyperExecute CLI | -| ---------| ---------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| MacOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -### Setup Environment Variable - -Now, you need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in the [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). - -Run the below mentioned commands in your terminal to setup the CLI and the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -## Step 4: Configure YAML in your Test Suite - -You need to edit the `PROJECT_TOKEN: "YOUR_PROJECT_TOKEN"` flag and enter your project token that show in the SmartUI app after, creating your project. - -```yaml ---- -version: 0.1 -globalTimeout: 90 -testSuiteTimeout: 90 -testSuiteStep: 90 - -runson: linux - -autosplit: true - -# highlight-next-line -cypress: true - -retryOnFailure: true -maxRetries: 1 - -concurrency: 1 - -env: -CYPRESS_CACHE_FOLDER: cypressCache -PROJECT_TOKEN: "YOUR_PROJECT_TOKEN" - -cacheKey: '{{ checksum "package.json" }}' -cacheDirectories: -- node_modules -- cypressCache - -pre: -- npm install @lambdatest/smartui-cli @lambdatest/cypress-driver cypress@v13 -- npx smartui config:create smartui-web.json - -post: -- cat hyp-smartui-sdk-cypress.yaml - -testDiscovery: -type: raw -mode: static -command: ls cypress/e2e - -testRunnerCommand: npx smartui --config smartui-web.json exec -- npx cypress run --spec cypress/e2e/smartuiSDKLocal.cy.js --browser chrome --headed - -jobLabel: ["smart-ui-sdk", "hyperexecute", "cypress"] -``` - -It is mandatory to mention these commands in the pre flag to download all the necessary dependencies - -```bash -pre: -- npm install @lambdatest/smartui-cli @lambdatest/cypress-driver cypress@v13 -- npx smartui config:create smartui-web.json -``` - -## Step 5: Execute your Test Suite - -> **NOTE :** In case of MacOS, if you get a permission denied warning while executing CLI, simply run **`chmod u+x ./hyperexecute`** to allow permission. In case you get a security popup, allow it from your **System Preferences** → **Security & Privacy** → **General tab**. - -Run the below command in your terminal at the root folder of the project: - -```bash -./hyperexecute --config RELATIVE_PATH_OF_YOUR_YAML_FILE -``` - -OR use this command if you have not exported your username and access key in the step 3. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE `} - -## Step 6: Monitor the Test Execution - -Visit the [HyperExecute Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) and check your Job status. diff --git a/static/docs/hyperexecute-smart-ui-sdk-using-playwright.md b/static/docs/hyperexecute-smart-ui-sdk-using-playwright.md deleted file mode 100644 index a086c1abd..000000000 --- a/static/docs/hyperexecute-smart-ui-sdk-using-playwright.md +++ /dev/null @@ -1,144 +0,0 @@ -# UI Testing with Playwright using HyperExecute and SmartUI SDK - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This documentation will guide you step-by-step to execute the SmartUI tests on the HyperExecute platform using Selenium - -## Prerequisites - -To run the Tests on HyperExecute from your Local System, you are required: - -- Your TestMu AI [Username and Access key](/support/docs/hyperexecute-how-to-get-my-username-and-access-key/) -- [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) in order to initiate a test execution Job . -- Setup the [Environmental Variable](/support/docs/hyperexecute-environment-variable-setup/) -- [HyperExecute YAML](/support/docs/hyperexecute-yaml-version0.2/) file which contains all the necessary instructions. -- Login to [TestMu AI SmartUI](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) with your credentials. - -## Step 1: Create a SmartUI Project - -The first step is to create a project with the application in which we will combine all your builds run on the project. To create a SmartUI Project, follow these steps: - -1. Go to [Projects page](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) -2. Click on the `new project` button -3. Select the platform as CLI or Web for executing your `SDK` tests. -4. Add name of the project, approvers for the changes found, tags for any filter or easy navigation. -5. Click on the **Submit**. - -After creating the project, you will get your `PROJECT_TOKEN`. You need to keep this project token safe as it will be used in the further steps below. - -## Step 2: Setup Your Test Suite - -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** - -Download or Clone the code sample for the Maestro framework from the TestMu AI GitHub repository to run the tests on the HyperExecute. - - View on GitHub - -### Configure Your Test Suite - -In the `playwrightCloud.js` file, update the following capabilities - -```javascript -const capabilities = { -"LT:Options": { -build: "Playwright SmartUI Build", // Mention your desired build nameP -name: "Playwright SmartUI Test", // Provide the name of your test -}, -}; -``` - -## Step 3: Setup the CLI in your Test Suite - -After cloning / downloading the sample repo, you need to setup the CLI and the environment variables. - -### Download the HyperExecute CLI - -The CLI is used for triggering the tests on HyperExecute. It is recommend to download the CLI binary on the host system and keep it in the root directory of the suite to perform the tests on HyperExecute. - -You can download the CLI for your desired platform from the below mentioned links: - -| Platform | HyperExecute CLI | -| ---------| ---------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| MacOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -### Setup Environment Variable - -Now, you need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in the [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). - -Run the below mentioned commands in your terminal to setup the CLI and the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -## Step 4: Configure YAML in your Test Suite - -You need to edit the `PROJECT_TOKEN: "YOUR_PROJECT_TOKEN"` flag and enter your project token that show in the SmartUI app after, creating your project. - -```yaml ---- -version: 0.1 -globalTimeout: 90 -testSuiteTimeout: 90 -testSuiteStep: 90 - -runson: win - -autosplit: true - -retryOnFailure: true -maxRetries: 1 - -concurrency: 1 - -env: -PROJECT_TOKEN: "YOUR_PROJECT_TOKEN" #Enter your project token here - -pre: -- npm install @lambdatest/smartui-cli @lambdatest/playwright-driver playwright -- npx smartui config:create smartui-web.json - -post: -- cat hyp-smartui-sdk.yaml - -testDiscovery: -type: raw -mode: static -command: ls sdk/playwrightCloud.js - -testRunnerCommand: npx smartui exec node sdk/playwrightCloud.js --config smartui-web.json - -jobLabel: ["smart-ui-sdk", "hyperexecute", "playwright"] -``` - -It is mandatory to mention these commands in the pre flag to download all the necessary dependencies - -```bash -pre: -- npm install @lambdatest/smartui-cli @lambdatest/playwright-driver playwright -- npx smartui config:create smartui-web.json -``` - -## Step 5: Execute your Test Suite - -> **NOTE :** In case of MacOS, if you get a permission denied warning while executing CLI, simply run **`chmod u+x ./hyperexecute`** to allow permission. In case you get a security popup, allow it from your **System Preferences** → **Security & Privacy** → **General tab**. - -Run the below command in your terminal at the root folder of the project: - -```bash -./hyperexecute --config RELATIVE_PATH_OF_YOUR_YAML_FILE -``` - -OR use this command if you have not exported your username and access key in the step 3. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE `} - -## Step 6: Monitor the Test Execution - -Visit the [HyperExecute Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) and check your Job status. diff --git a/static/docs/hyperexecute-smart-ui-sdk-using-puppeteer.md b/static/docs/hyperexecute-smart-ui-sdk-using-puppeteer.md deleted file mode 100644 index 0b6e8f025..000000000 --- a/static/docs/hyperexecute-smart-ui-sdk-using-puppeteer.md +++ /dev/null @@ -1,143 +0,0 @@ -# UI Testing with Puppeteer using HyperExecute and SmartUI SDK - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This documentation will guide you step-by-step to execute the SmartUI tests on the HyperExecute platform using **Puppeteer** - -## Prerequisites - -You must do the following in order to run the tests on HyperExecute from your local system: - -- Get your TestMu AI [Username and Access key](/support/docs/hyperexecute-how-to-get-my-username-and-access-key/) from the dashboard -- Download [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) in order to initiate a test execution Job . -- Set your [Environmental Variables](/support/docs/hyperexecute-environment-variable-setup/) -- [HyperExecute YAML](/support/docs/hyperexecute-yaml-version0.2/) file which contains all the necessary instructions. -- Use your login credentials to access [TestMu AI SmartUI](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/). - -## Step 1: Create a SmartUI Project - -With the application, create a project in which we will merge all of your builds that have been applied to the project. To initiate a SmartUI Project, take the following actions: - -1. Access the [Projects page](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/). -2. Click the `new project` button now. -3. Choose the **CLI** or **Web** platform to run your `SDK` tests on. -4. Include the project name, the approvers of the changes that were discovered, and tags for any filters or simple navigation. -5. Select **Submit** by clicking. - -After creating the project, you will get your `PROJECT_TOKEN`. You need to keep this project token safe as it will be used in the further steps below. - -## Step 2: Setup Your Test Suite - -To configure and test it, you can use your own project. We are using the sample repository for demonstration purposes. - -**Sample repo** -Download or Clone the code sample for the Puppeteer framework from the TestMu AI GitHub repository to run the tests on the HyperExecute. - - View on GitHub - -### Configure Your Test Suite - -In the `sdkCloud.js` file, update the following capabilities - -```javascript -const capabilities = { -"LT:Options": { -"build": "Puppeteer SmartUI Build", // Mention your desired build name -"name": "Puppeteer SmartUI Test", // Provide the name of your test -}, -}; -``` - -## Step 3: Setup the CLI in your Test Suite - -Following the download or cloning of the sample repository, you must configure the environment variables and the CLI. - -### Download the HyperExecute CLI - -The tests on HyperExecute are started using the Command Line Interface (CLI). To run the tests on HyperExecute, it is advised to download the CLI binary to the host system and store it in the suite's root directory. - -You can download the CLI for your desired platform from the below mentioned links: - -| Platform | HyperExecute CLI | -| ---------| ---------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| MacOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -### Setup Environment Variable - -Now, you have to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in the [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). - -Run the below mentioned commands in your terminal to setup the CLI and the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -## Step 4: Configure YAML in your Test Suite - -Edit the `PROJECT_TOKEN: "YOUR_PROJECT_TOKEN"` flag and enter your project token that show in the SmartUI app after, creating your project. - -```yaml ---- -version: 0.1 -globalTimeout: 90 -testSuiteTimeout: 90 -testSuiteStep: 90 - -runson: win - -autosplit: true - -retryOnFailure: true -maxRetries: 1 - -concurrency: 1 - -env: -PROJECT_TOKEN: "YOUR_PROJECT_TOKEN" #Enter your project token here - -pre: -- npm install @lambdatest/smartui-cli @lambdatest/puppeteer-driver puppeteer -- npx smartui config:create smartui-web.json - -post: -- cat hyperexecute-smartui-sdk.yaml - -testDiscovery: -type: raw -mode: static -command: ls sdk/puppeteerCloud.js - -testRunnerCommand: npx smartui exec node sdk/puppeteerCloud.js --config smartui-web.json - -jobLabel: ["smart-ui-sdk", "hyperexecute", "puppeteer"] -``` - -It is mandatory to mention these commands in the pre flag to download all the necessary dependencies - -```bash -pre: -- npm install @lambdatest/smartui-cli @lambdatest/puppeteer-driver puppeteer -- npx smartui config:create smartui-web.json -``` - -## Step 5: Execute your Test Suite - -> **NOTE:** On macOS, you can grant permission by running **`chmod u+x./hyperexecute`** if you receive a permission denied warning when using the CLI. If a security pop-up appears, click the Allow button on the **System Preferences** → **.Security and Privacy** → **.General tab**. - -Run the below command in your terminal at the root folder of the project: - -```bash -./hyperexecute --config RELATIVE_PATH_OF_YOUR_YAML_FILE -``` - -OR use this command if you have not exported your username and access key in the step 3. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE `} - -## Step 6: Monitor the Test Execution - -Go to the [HyperExecute Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) to see the status of your job. diff --git a/static/docs/hyperexecute-smart-ui-test-using-playwright.md b/static/docs/hyperexecute-smart-ui-test-using-playwright.md deleted file mode 100644 index 7879af2ad..000000000 --- a/static/docs/hyperexecute-smart-ui-test-using-playwright.md +++ /dev/null @@ -1,160 +0,0 @@ -# SmartUI Testing in HyperExecute using Playwright - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This documentation will guide you step-by-step to execute the SmartUI tests on the HyperExecute platform using Playwright. - -## Prerequisites - -- Your TestMu AI [Username and Access key](/support/docs/hyperexecute-how-to-get-my-username-and-access-key/) -- [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) in order to initiate a test execution Job . -- Setup the [Environment Variable](/support/docs/hyperexecute-environment-variable-setup/) -- [HyperExecute YAML](/support/docs/hyperexecute-yaml-version0.2/) file which contains all the necessary instructions. -- To run SmartUI tests, you need to have a **Baseline Image** and a **Comparison Image**. A baseline image is the reference image with which you want to compare. You can either upload this image from your computer or take a screenshot from your browser. - -After creating the project, you will get your `PROJECT_TOKEN`. You need to keep this project token safe as it will be used in the further steps below. - -## Step 1: Configure Your Test Suite - -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** -Download or Clone the code sample for the Playwright JavaScript from the TestMu AI GitHub repository to run the tests on the HyperExecute. - - View on GitHub - -If you are using your own project, make sure you update the **CDP endpoint** in your tests file. - -By setting up the CDP endpoint, you establish the communication channel between your tests and the browser nodes, enabling effective test distribution and execution. - -```bash -wsEndpoint: `wss://cdp.lambdatest.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` -``` - -Configure the desired capabilities based on your test requirements. For example: - -```bash -const capabilities = { -'browserName': 'Chrome', // Browsers allowed: `Chrome`, `MicrosoftEdge`, `pw-chromium`, `pw-firefox` and `pw-webkit` -'browserVersion': 'latest', -'LT:Options': { -'platform': process.env.HYPEREXECUTE_PLATFORM, -'build': 'Playwright SmartUI Build', -'name': 'Playwright SmartUI Test', -'user': process.env.LT_USERNAME, -'accessKey': process.env.LT_ACCESS_KEY, -'network': true, -'video': true, -'console': true, -'smartUIProjectName': process.env.SMARTUI_PROJECT || 'SmartUI-PW', -'smartUIBaseline': true -} -} -``` - -> You can also use **dev** and **beta** browser versions. To generate capabilities for your test requirements, you can use our inbuilt 🔗 Capabilities Generator. - -## Step 2: Setup the CLI in your Test Suite - -After cloning / downloading the sample repo, you need to setup the CLI and the environment variables. - -### Download the HyperExecute CLI - -The CLI is used for triggering the tests on HyperExecute. It is recommend to download the CLI binary on the host system and keep it in the root directory of the suite to perform the tests on HyperExecute. - -You can download the CLI for your desired platform from the below mentioned links: - -| Platform | HyperExecute CLI | -| ---------| ---------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| MacOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -### Setup Environment Variable - -Now, you need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in the [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). - -Run the below mentioned commands in your terminal to setup the CLI and the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -## Step 3: Configure YAML in your Test Suite - -Configure your YAML file as per your use cases using **key value** pairs. - -In this sample YAML file, we have mentioned: - -- **version** of the YAML file -- **Timeouts** for executing your project -- **Pre** command -- and other necessary YAML Parameters - -```yaml ---- -version: 0.1 -globalTimeout: 150 -testSuiteTimeout: 150 -testSuiteStep: 150 - -runson: win - -retryOnFailure: true -maxRetries: 1 - -concurrency: 1 - -env: -CACHE_DIR: m2_cache_dir - -# Dependency caching for Windows -cacheKey: '{{ checksum "pom.xml" }}' -cacheDirectories: -- ${CACHE_DIR} - -matrix: -classname: ["pw-smartui.js"] - -pre: -# install playwright version to be used. -- npm install playwright@1.42.0 --save-exact - -testSuites: -- node hooks/pw-smartui.js - -jobLabel: ['HYP', 'Playwright' 'SmartUI'] - -``` - -## Step 4: Execute your Test Suite - -> **NOTE :** In case of MacOS, if you get a permission denied warning while executing CLI, simply run **`chmod u+x ./hyperexecute`** to allow permission. In case you get a security popup, allow it from your **System Preferences** → **Security & Privacy** → **General tab**. - -Run the below command in your terminal at the root folder of the project: - -```bash -./hyperexecute --config RELATIVE_PATH_OF_YOUR_YAML_FILE -``` - -OR use this command if you have not exported your username and access key in the step 3. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE `} - -## Step 5: Monitor the Test Execution - -- Visit the [HyperExecute Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) and check your Job status. Click on the **Approved** button. - -- This will redirect you the SmartUI dashboard where you can see all the images you have pushed for the testing. - -- Now you can make the changes in the UI as per your requirement and trigger the Job once again. - -- Again click on the **Apprved** button again. - -- Now, you will find two builds in your dashboard. One will be the **Baseline Build** and the other one will be the **Non Baseline Build** - -- Open any screenshot from the **Non Baseline Build**, and you can see the comparison between the two screenshots, i.e. the Baseline Image and the Non Baseline Image. - -- You can now approve or reject the changes directly from the dashboard as well. diff --git a/static/docs/hyperexecute-smart-ui-testing-using-cypress.md b/static/docs/hyperexecute-smart-ui-testing-using-cypress.md deleted file mode 100644 index 0dde8bab2..000000000 --- a/static/docs/hyperexecute-smart-ui-testing-using-cypress.md +++ /dev/null @@ -1,96 +0,0 @@ -# SmartUI Tests in HyperExecute using Cypress - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This documentation will guide you step-by-step to execute the SmartUI tests on the HyperExecute platform using Cypress - -## Prerequisites - -- Your lambdatest [Username and Access key](/support/docs/hyperexecute-how-to-get-my-username-and-access-key/) -- [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) in order to initiate a test execution Job . -- Setup the Environment Variable -- [HyperExecute YAML](/support/docs/hyperexecute-yaml-version0.2/) file which contains all the necessary instructions. -- To run SmartUI tests, you need to have a **Baseline Image** and a **Comparison** image. A baseline image is the reference image with which you want to compare. You can either upload this image from your computer or take a screenshot from your browser. - -## Step 1: Setup your Project - -### Download HyperExecute CLI - -The *HyperExecute CLI* is used for triggering tests on HyperExecute. It is recommend to download the HyperExecute CLI binary on the host system to perform the tests on HyperExecute. The CLI download site for various platforms is displayed below: - -| Platform | HyperExecute CLI download location | -| ---------| --------------------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| macOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -### Setup Environment Variable - -Export the environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in the [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). -Run the below mentioned commands in the terminal to setup the CLI and the environment variables. - -For macOS / Linux: - -```bash -export LT_USERNAME=YOUR_LT_USERNAME -export LT_ACCESS_KEY=YOUR_LT_ACCESS_KEY -``` - -For Windows: - -```bash -set LT_USERNAME=YOUR_LT_USERNAME -set LT_ACCESS_KEY=YOUR_LT_ACCESS_KEY -``` - -## Step 2: Configure your Cypress Project - -1. Go to your project **YAML** file. - -2. Your test files must have this code snippet in order for them to take screenshots. - -```java -cy.screenshot('my-image') -``` - -Here **'my-image'** is the name of the image which will be rendered on the Dashboard. -> Refer to the [Cypress Documentation](https://docs.cypress.io/api/commands/screenshot) to understand the different methods for taking screenshots. - -3. Paste the below mentioned code in your YAML file. - -```bash -cypressOps: -Build: "test_01" -Tags: ["smart-ui", "hyperexecute", "cypress"] -BuildTags: ["t1"] -smartUICaps: -# highlight-next-line -smartUIProjectName: -smartUIOptions: -output: -errorColor: -red: 0 -green: 200 -blue: 0 -errorType: flat, -transparency: 1, -largeImageThreshold: 100, -useCrossOrigin: false, -outputDiff: true -scaleToSameSize: true, -ignore: antialiasing -SmartUIBaseline: true -``` - -## Step 3: Execute your Project and Compare the Results - -To execute the project run the below command in your terminal at the root folder. - -```bash -./hyperexecute --config --download-artifacts --force-clean-artifacts -``` -- Now, visit to the [HyperExecute Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) to check the status of your project. - -- Click on the **SmartUI Tests** button. - -- Click on **Compare** button to compare the changes made. diff --git a/static/docs/hyperexecute-smart-ui-testing-using-selenium.md b/static/docs/hyperexecute-smart-ui-testing-using-selenium.md deleted file mode 100644 index b8ce0a7c7..000000000 --- a/static/docs/hyperexecute-smart-ui-testing-using-selenium.md +++ /dev/null @@ -1,257 +0,0 @@ -# SmartUI Tests in HyperExecute using Selenium - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This documentation will guide you step-by-step to execute the SmartUI tests on the HyperExecute platform using Selenium - -## Prerequisites - -- Your TestMu AI [Username and Access key](/support/docs/hyperexecute-how-to-get-my-username-and-access-key/) -- [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) in order to initiate a test execution Job . -- Setup the [Environment Variable](/support/docs/hyperexecute-environment-variable-setup/) -- [HyperExecute YAML](/support/docs/hyperexecute-yaml-version0.2/) file which contains all the necessary instructions. -- To run SmartUI tests, you need to have a **Baseline Image** and a **Comparison** image. A baseline image is the reference image with which you want to compare. You can either upload this image from your computer or take a screenshot from your browser. - -## Step 1: Configure Your Test Suite - -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** -Download or Clone the code sample for the TestNG from the TestMu AI GitHub repository to run the tests on the HyperExecute. - - View on GitHub - -If you are using your own project, make sure you update the **HUB endpoint** in your tests file. - -By setting up the HUB endpoint, you establish the communication channel between your tests and the browser nodes, enabling effective test distribution and execution. - -```bash -const GRID_HOST = "@hub.lambdatest.com/wd/hub"; -``` - -Configure the desired capabilities based on your test requirements. For example: - -```bash -DesiredCapabilities capability = new DesiredCapabilities(); -capability.setCapability(CapabilityType.BROWSER_NAME, browser); -capability.setCapability(CapabilityType.VERSION, version); -capability.setCapability(CapabilityType.PLATFORM, os); -capability.setCapability("screen_resolution", res); -capability.setCapability("build", "Cucu-Sel-TestNG-" + jobid); -capability.setCapability("name", scenario); -capability.setCapability("network", true); -capability.setCapability("video", true); -capability.setCapability("console", true); -capability.setCapability("visual", true); -capability.setCapability("smartUI.project", "HyperExecute-smartUI-project"); -``` -> You can also use **dev** and **beta** browser versions. To generate capabilities for your test requirements, you can use our inbuilt 🔗 Capabilities Generator. - -## Step 2: Setup the CLI in your Test Suite - -After cloning / downloading the sample repo, you need to setup the CLI and the environment variables. - -### Download the HyperExecute CLI - -The CLI is used for triggering the tests on HyperExecute. It is recommend to download the CLI binary on the host system and keep it in the root directory of the suite to perform the tests on HyperExecute. - -You can download the CLI for your desired platform from the below mentioned links: - -| Platform | HyperExecute CLI | -| ---------| ---------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| MacOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -### Setup Environment Variable - -Now, you need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in the [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). - -Run the below mentioned commands in your terminal to setup the CLI and the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -## Step 3: Configure YAML in your Test Suite - -Configure your YAML file as per your use cases using **key value** pairs. - -In this sample YAML file, we have mentioned: - -- **version** of the YAML file -- **Timeouts** for executing your project -- **Mode of execution** is [Autosplit](/support/docs/hyperexecute-auto-split-strategy/). You can also opt for [Matrix](/support/docs/hyperexecute-matrix-multiplexing-strategy/) or [Hybrid](/support/docs/hyperexecute-hybrid-strategy/) mode. -- **Pre and Post** commands -- **Reports and Artefacts** that will be generated after the completion of tests -- and other necessary YAML Parameters - -```yaml ---- -version: "0.1" -globalTimeout: 90 -testSuiteTimeout: 90 -testSuiteStep: 90 - -runson: win - -autosplit: true - -retryOnFailure: true -maxRetries: 2 - -concurrency: 1 - -debug: false - -dataJsonPath: ['src/datame.json', 'src/data1.json','me/data.json', 'data.json'] - -mergeArtifacts: true -stripParentDirectory: true -taskIdentifierInNonConflictingArtefacts: true - -pre: -- pwd -- echo ${PAT} -- echo ${qa} -- echo %PAT% -- echo %qa% -- sed -i -e 's/MyKey/%PAT%/g' key.txt -- cat key.txt -- mvn compile -- cat longlogs.txt - -post: -- cat hyperexecute.yaml -- cat longlogs.txt - -testRunnerCommand: mvn test `-Dcucumber.options="$test" `-Dscenario="$test" `-Dmaven.repo.local=m2_cache_dir - -testDiscovery: -type: raw -mode: static -command: snooper --featureFilePaths=src/main/**/ --frameWork=java --query="@Addition AND @me"| sed -n 1,'5p' - -jobLabel: [ 'HYP', 'Selenium', 'smartUI'] -``` - -## Step 4: Execute your Test Suite - -> **NOTE :** In case of MacOS, if you get a permission denied warning while executing CLI, simply run **`chmod u+x ./hyperexecute`** to allow permission. In case you get a security popup, allow it from your **System Preferences** → **Security & Privacy** → **General tab**. - -Run the below command in your terminal at the root folder of the project: - -```bash -./hyperexecute --config RELATIVE_PATH_OF_YOUR_YAML_FILE -``` - -OR use this command if you have not exported your username and access key in the step 2. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE `} - -## Step 5: Monitor the Test Execution - -Visit the [HyperExecute Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) and check your Job status. - -Here you can see the difference. - -## Handling Dynamic Data with DOM Configuration - -| Key | Description | -|-----|------------ | -| `screenshotName` (string) | Specify a name for the screenshot in your tests to match the same screenshot with the name from your baseline. | -| `fullPage` (boolean) | Specify `true` if you want to take a Full Page Screenshot and `false` for viewport screenshots; `fullPage: true` is currently only supported for `Chrome`. | -| `ignoreDOM` (object) | Specify one or a combination of selectors based on the `HTML DOM ID, CSS class, CSS selector or Xpath` used by your webpage that should be excluded from the comparison. | -| `selectDOM` (object) | Specify one or a combination of selectors based on the `HTML DOM ID, CSS class, CSS selector or XPath` used by your webpage that should be included in the comparison. | - -```js title="This is a sample for your webhook configuration for JavaScript to ignore by ID" -let config = { -screenshotName: "Ignore-ID", -fullPage: false, //You can make this property as true in case of Chrome browser -ignoreDOM: { -id: ["ID-1", "ID-2"], // Ignoring elements by ID, you can ignore multiple at once -}, -}; -await driver.executeScript("smartui.takeScreenshot", config); -``` - -```js title="This is a sample for your webhook configuration for JavaScript to ignore by Class" -let config = { -screenshotName: "Ignore-Class", -fullPage: false, //You can make this property as true in case of Chrome browser -ignoreDOM: { -class: ["Class-1", "Class-2"], // Ignoring elements by class, you can ignore multiple at once -}, -}; -await driver.executeScript("smartui.takeScreenshot", config); -``` - -```js title="This is a sample for your webhook configuration for JavaScript to ignore by XPath" -let config = { -screenshotName: "Ignore-XPath", -fullPage: false, //You can make this property as true in case of Chrome browser -ignoreDOM: { -xpath: ["Xpath-1", "Xpath-2"], // Ignoring elements by XPath, you can ignore multiple at once -}, -}; -await driver.executeScript("smartui.takeScreenshot", config); -``` - -```js title="This is a sample for your webhook configuration for JavaScript to ignore by CSS Selector" -let config = { -screenshotName: "Ignore-cssSelector", -fullPage: false, //You can make this property as true in case of Chrome browser -ignoreDOM: { -cssSelector: ["CSS-Selector-1", "CSS-Selector-2"], // Ignoring elements by CSS selector, you can ignore multiple at once -}, -}; -await driver.executeScript("smartui.takeScreenshot", config); -``` - -```js title="This is a sample for your webhook configuration for JavaScript to select by ID." -let config = { -screenshotName: "Select-ID", -fullPage: false, //You can make this property as true in case of Chrome browser -selectDOM: { -id: ["ID-1", "ID-2"], // Selecting elements by ID, you can select multiple at once -}, -}; -await driver.executeScript("smartui.takeScreenshot", config); -``` - -```js title="This is a sample for your webhook configuration for JavaScript to select by Class" -let config = { -screenshotName: "Select-Class", -fullPage: false, //You can make this property as true in case of Chrome browser -selectDOM: { -class: ["Class-1", "Class-2"], // Selecting elements by class, you can select multiple at once -}, -}; -await driver.executeScript("smartui.takeScreenshot", config); -``` - -```js title="This is a sample for your webhook configuration for JavaScript to select by XPath" -let config = { -screenshotName: "Select-XPath", -fullPage: false, //You can make this property as true in case of Chrome browser -selectDOM: { -xpath: ["Xpath-1", "Xpath-2"], // Selecting elements by XPath, you can select multiple at once -}, -}; -await driver.executeScript("smartui.takeScreenshot", config); -``` - -```js title="This is a sample for your webhook configuration for JavaScript to select by CSS Selector" -let config = { -screenshotName: "Select-cssSelector", -fullPage: false, //You can make this property as true in case of Chrome browser -selectDOM: { -cssSelector: ["CSS-Selector-1", "CSS-Selector-2"], // Selecting elements by CSS selector, you can select multiple at once -}, -}; -await driver.executeScript("smartui.takeScreenshot", config); -``` - -> **NOTE :** Please contact [Customer Support](mailto:support@testmuai.com) team to enable the **ignoreDOM** feature. diff --git a/static/docs/hyperexecute-snooper.md b/static/docs/hyperexecute-snooper.md deleted file mode 100644 index b1b754831..000000000 --- a/static/docs/hyperexecute-snooper.md +++ /dev/null @@ -1,143 +0,0 @@ -# Using Snooper Command on Hyperexecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -A `snooper` is a command that can be used to discover the list of Feature file scenarios that would be further executed using the value passed in the `testRunnerCommand`. The `snooper` command takes two arguments: - -- `featureFilePaths`: This argument specifies the path to the Feature files that you want to discover. -- `frameWork`: This argument specifies the framework that you are using for your tests. - -For example, the following command will discover the list of Feature file scenarios in the features folder that are using the Java framework: - -```yaml -snooper --featureFilePaths=features/ --frameWork=java -``` - -Snooper is **not compatible** with **Alpine-based** Docker images, as Alpine uses **musl libc** instead of **glibc**, which Snooper depends on. To ensure proper functionality, use a non-Alpine base image that includes glibc. - -## How to implement Snooper on HyperExecute - -> `snooper` command works only with HyperExecute YAML 0.1 - -To implement the `snooper` command on HyperExecute, we simply need to update our YAML file's `testDiscovery` and `testRunnerCommand` parameters. - -### `testDiscovery` -This command helps to discover all the test scenarios within the project. It offers two methods to implement it: - -#### Method 1: Discovering test cases on Hyperexecute machines: - -- `featureFilePaths`: Path where all feature files are located -- `frameWork`: Framework of Testing -- `specificTags`: (optional) Tags that need to be tested - -```yaml -testDiscovery: -type: automatic -mode: static -args: -featureFilePaths: src/test/java/Features/ -frameWork: java -specificTags: [""] -``` - -#### Method 2: Discovering test cases on Local machines - -- `command`: It is a mandatory parameter in the yaml file and would throw an error if not found by the compiler. - -```yaml -testDiscovery: -type: raw -mode: static -command: snooper --targetOs=win --featureFilePaths=src/test/java/Features/ --frameWork=java --specificTags=@tag1,@tag2 -``` - -### `testRunnerCommand` - -This command helps to execute the test cases and needs a command that will help to execute the single test. It is executed on the Hyperexecute VMs for each scenario at an individual level. - -There are two methods to execute the runner command for cucumber: - -#### Method 1: Using `options` flag - -```yaml -testRunnerCommand: mvn test -Dcucumber.options="$test" -``` - -- `-Dcucumber.options`: This parameter is used to specify the feature that will help execute on VMs. - -#### Method 2: Using `features` flag - -> **Note:** This flag is applicable for cucumber version 7 and above only. - -```yaml -testRunnerCommand: mvn test -Dcucumber.features="$test" -``` - -- `-Dcucumber.features`: This parameter is used to execute one single feature file. - -## Debug - -We can also debug `testDiscovery` using `snooper` to check whether all the desired test scenarios have been discovered or not. This can be implemented using the Snooper CLI binary present in the **.hyperexecute folder**. - -To debug the discovery of the snooper, configure the following command as per the desired parameters: - -```yaml -.hyperexecute/snooper --targetOs=win --ignoredTags=@ignore @skipDaily --featureFilePaths=src/test/java/Features --frameWork=java | sed 's/:.*//' | uniq -``` - -> **Note:** .hyperexecute and snooper binary are created only after first execution of the yaml file. - -#### How to effectively utilize tags incorporating various custom parameters for enhanced functionality? - -- Snooper discovery can be customized where you can use set of tags that you want to discover the tests. - -```yaml -specificTags: ["@tag1","@tag2"] -``` - -- Discovery can also be customized by passing logical query in the snooper using `query` tag: - -```yaml -query: "((@tag1 or @tag2) and not @tag3)" -``` - -- You can also skip the tests that have some specific tags using `ignoredTags`: - -```yaml -ignoredTags : ["@tag3","@tag2"] -``` - -## Sample YAML File -Here is a sample HyperExecute file with cucumber 7 and above to discover all the feature files without any tags. - -```yaml ---- -version: "0.1" -runson: win -autosplit: true - -concurrency: 1 - -retryOnFailure: true -maxRetries: 1 - -cacheKey: '{{ checksum "package-lock.json" }}' -cacheDirectories: -- node_modules - -pre: -- npm install - -testDiscovery: -type: automatic -mode: remote -args: -featureFilePaths: features/sample_website -frameWork: javascript -specificTags: [ "@test" ] -# command: .hyperexecute/snooper --targetOs=win --featureFilePaths=features/onepass_website --frameWork=javascript --specificTags=@test | sed 's/:.*//' | uniq - -testRunnerCommand: npm run execute-tests $test - -jobLabel: [snooper, autosplit] -``` diff --git a/static/docs/hyperexecute-stage-widgets.md b/static/docs/hyperexecute-stage-widgets.md deleted file mode 100644 index 3d25fc831..000000000 --- a/static/docs/hyperexecute-stage-widgets.md +++ /dev/null @@ -1,15 +0,0 @@ -# HyperExecute Stage Widgets - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Stage Trends - -The `Stage Trends` widget enables the user to track the total Stages being run on the platform grouped by their status including **Completed**, **Skipped**, **Failed**, **Aborted**, etc. - -You can use this widget when you want to get a total of the Stages being executed on the platform for a specific environment or status. With the Stage Summary widget you can see the total count of the desired use cases. - -## Stage Summary - -The `Stage Summary` widget enables the user to track the total Stages being run on the platform grouped by their status including **Completed**, **Skipped**, **Failed**, **Aborted**, etc. - -If you wants to get a total of the Stages being executed on the platform for a specific environment or status. With the Stage Summary widget you can see the total count of the desired use cases. diff --git a/static/docs/hyperexecute-status.md b/static/docs/hyperexecute-status.md deleted file mode 100644 index e10bfc6da..000000000 --- a/static/docs/hyperexecute-status.md +++ /dev/null @@ -1,118 +0,0 @@ -# HyperExecute Status - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This page provides a comprehensive overview of the various statuses available in HyperExecute, encompassing job, task, scenario, and test levels. These statuses will enable you to properly read test execution results as well as identify problems that may occur in automated testing processes. - -HyperExecute categorizes test execution outcomes into distinct levels, offering a granular view of the entire testing process. Here's a breakdown of these levels and their associated statuses: - -## 1. Job Level Status -All the tests on HyperExecute get executed as a Job. Each Job has a Status, a unique Job Number, a set of Labels, info on number of Tasks executed as part of that Job, and a Summary section followed by the details of the Job Duration and who initiated the Job. - -| Icon | Status | Status Description | -|------|--------|--------------------| -||Initiated| The payload is uploading and is waiting to be assigned to an available virtual machine as per the given OS request in the YAML file. | -||Running| The job is actively executing tests across different browser and OS combinations as defined in your configuration. | -||Completed| The job has finished executing all tests, irrespective of the overall outcome (passed/failed).| -||Partially Completed| This status indicates a non-standard scenario where all steps within the job's runner command execution finished, but no test session was created. This can occur primarily in two cases: 1. API/Desktop Tests 2. Local Web Test Execution. -||Failed| One or more test cases within the job encountered errors or failed assertions indicating an issue within the tests. | -| |Aborted| An unexpected error or issue caused the job to terminate prematurely before all tests could run. | -| |Lambda Error| An error originating from TestMu AI's side prevented the job from executing as planned. This could be a temporary glitch or require contacting TestMu AI support for further investigation. | -| | Time Out | The job exceeded the maximum allocated execution time limit. This might occur due to complex tests, slow environments, or resource limitations. | -||Ignored| It is a user-defined status, used when a test case is intentionally not executed, such as when it is marked for future development, or a feature is not yet implemented. -||Blocked| The job is currently stuck due to an external dependency or resource constraint. This could involve waiting for another job using the same resources to finish or limitations on available browsers/devices. | - -### Abort Job - -You can abort a running job directly from the HyperExecute portal. Once a job is in **Running** state, click on the job to open the Job Details page. You will see an **Abort** option that provides the following choices: - -- **Abort Current Job**: Stops only the selected running job. -- **Abort All Jobs**: Stops all currently running jobs. -- **Abort Selected Jobs**: Allows you to select specific jobs to abort. - -When a job is aborted, all active test sessions within that job are terminated immediately and no residual processes remain. The remaining tests that were not yet executed will be marked as **Cancelled**. - -## 2. Task Level Status - -A Task is a further subdivision within a job, often representing the execution of tests on a specific browser and operating system combination. - -| Icon | Status | Status Description | -|------|--------|--------------------| -||Initiated| The payload has been uploaded and is waiting to be assigned to an available virtual machine as per the given OS request in the YAML file. -||Running| The job is actively executing tests as defined in your YAML file configuration. -||Completed| The task has finished executing all its assigned test steps and all the test running in that tasks are passed. -||Skipped| It is a user-defined status indicates that the task was intentionally bypassed, possibly due to configuration settings, conditional execution logic, or irrelevance to the current test scenario. -||Failed| When one or more assertions within the task failed, indicating a problem with the tests themselves. -||Ignored| Similar to the job level, the task was entirely disregarded, likely due to configuration issues or specific conditions set to trigger execution. -| |Aborted| An unforeseen error or issue caused the task to terminate prematurely before all steps could run. -| | Time Out | The task exceeded the maximum allocated execution time limit. This might occur due to complex test steps, slow environments, or resource limitations -| |Lambda Error| An error originating from TestMu AI's side prevented the task from executing as planned. This could be a temporary glitch or require contacting TestMu AI support for further investigation. -||Blocked| The task is currently stuck due to an external dependency or resource constraint. This could involve waiting for another task using the same resources to finish or limitations on available browsers/devices. - -## 3. Stage Level Status -A Tasks can have multiple stages which are usually divided into 3 categories: -1. **Pre Steps** - The stages/actions that are performed before Test execution begins, like installation of dependencies etc. -2. **Scenarios** - Test execution stages. -3. **Post Steps** - The stages/actions that are performed after Test execution is completed, like creation of reports, artifacts etc. - -A Stage can attain different statuses based on the completion level. - -| Icon | Status | Status Description | -|------|--------|--------------------| -||Running| The stage is currently executing the defined actions or test steps. This indicates active progress within a specific stage of your test -||Completed| The stage has finished executing all its test steps successfully. -||Ignored| It is a user-defined status which indicates that the stage was entirely bypassed during execution, likely due to configuration settings or conditional logic within your test script. -||Skipped| It is a user-defined status indicates that the stage was intentionally bypassed, possibly due to configuration settings, conditional execution logic, or irrelevance to the current test scenario. -||Failed| One or more assertions within the stage failed, signifying an issue within the tests. -||Muted| This stage's results are being suppressed due to the HyperExecute [test muting](/support/docs/hyperexecute-test-muting/) functionality. -| |Aborted| An unexpected error or issue caused the stage to terminate prematurely before all actions could be completed. -| |Cancelled| If you have aborted your job in the middle of test execution, then all the left over tests will be marked as Cancelled. -| |Lambda Error| An error originating from TestMu AI's side prevented the stage from executing as planned. This could be a temporary glitch or require contacting TestMu AI support for further investigation. - -## 4. Tests Level Status - -The most granular level, representing an individual test case verifying a specific functionality of your application. It's like a single action you perform to test something. - -| Icon | Status | Status Description | -|------|--------|--------------------| -||Created| The test has been defined in your test script but hasn't begun execution yet. -||Running| The test is actively being executed as per the configurations. -||Completed| The test has finished execution and is successfully passed. -||Queue| The test is waiting for the virtual machine to be allocated. -||Queue Timeout| The wait time exceeded the maximum limit for the virtual machine to be allocated. -||Stopped| When you abort the stage, your tests are marked as Stopped. -||Idle Timeout| It indicates that your session was inactive for the defined time. -||Lambda Error| An error originating from TestMu AI's side prevented the stage from executing as planned. This could be a temporary glitch or require contacting TestMu AI support for further investigation. - -### User Defined Status -HyperExecute allows users to define the status of tests using **lambda hooks**. This can be helpful for monitoring test status and results. - -HyperExecute supports the following user-defined lambda hooks status: - -| Icon | Status | Status Description | -|------|--------|--------------------| -||Passed| A test case has passed when it has executed successfully, and all the assertions have been verified without any errors. `driver.executeScript("lambda-status=passed");`| -||Failed| A test case has failed when it has not executed as expected, and one or more assertions have not been verified or have failed. `driver.executeScript("lambda-status=failed");`| -||Skipped| A test case may be skipped if it is not relevant or cannot be executed due to some issues like environment setup, data, or configuration. This status can also be used for test cases that are marked for review or maintenance. `driver.executeScript("lambda-status=skipped");`| -||Ignored| The ignored status is used when a test case is intentionally not executed, such as when it is marked for future development, or a feature is not yet implemented. `driver.executeScript("lambda-status=ignored");`| -||Unknown| When a user wants to mark status of their tests as undecided. `driver.executeScript("lambda-status=unknown");`| -||Error| A test case can end in an error status when an unexpected exception is thrown during its execution. This error can be due to a coding error or a defect in the application under test. `driver.executeScript("lambda-status=error");`| - -### Priority Scenario Mapping - -In a **Scenario**, there can be numerous tests running, and each test has its own status, which means the final status of the scenario is derived from its test status depending on the status of the test with the **highest priority**. - -For example, if a scenario contains two tests, one with status `passed` and one with status `failed`, the scenario status will be `failed`. - -Consider the following Decreasing Order of Priority among Users with marked status: - -|Priority| User Marked Status| Scenario Status| -|------|------|------| -|P0|Failed|Failed| -|P1|Error|Failed| -|P2|Passed|Completed| -|P3|Ignored|Completed| -|P4|Skipped|Completed| -|P5|Unknown|Completed| - -Now that you have understood the concepts of HyperExecute, you can [get started](/support/docs/hyperexecute-running-your-first-job/#quick-run) with it quickly. Learn more about other [HyperExecute features](/support/docs/key-features-of-hyperexecute/). diff --git a/static/docs/hyperexecute-supported-languages-and-frameworks.md b/static/docs/hyperexecute-supported-languages-and-frameworks.md deleted file mode 100644 index 4348cec8a..000000000 --- a/static/docs/hyperexecute-supported-languages-and-frameworks.md +++ /dev/null @@ -1,8 +0,0 @@ -# Languages and Frameworks Supported On HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Out of the box, HyperExecute supports multiple languages and testing frameworks like Selenium, Cypress, Playwright, Puppeteer etc. Currently we provide sample code for the following languages and frameworks: - - In case your favorite framework or tool isn't listed here, just give us a Shout Out Here. - In case of any other problems, feel free to Contact Us. diff --git a/static/docs/hyperexecute-supported-languages-and-packages.md b/static/docs/hyperexecute-supported-languages-and-packages.md deleted file mode 100644 index 693060096..000000000 --- a/static/docs/hyperexecute-supported-languages-and-packages.md +++ /dev/null @@ -1,46 +0,0 @@ -# List of Supported Languages and Addons w.r.t. Operating System - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This document provides a comprehensive list of all the supported versions of programming languages across different operating systems on our platform. - -| Supported Languages | Version | -|---------------------|---------| -|Java| `1.8.0` **(default)**, `8`, `9`, `11`, `13`, `15`, `16`, `17`, `18`, `19`, `21`| -|Ruby|`2.0.0`, `2.1.9`, `2.3.0`, `2.3.1`, `2.3.3`, `2.4.1`, `2.4.2`, `2.4.3`, `2.4.4`, `2.4.5`, `2.4.6`,` 2.4.7`, `2.4.9`, `2.4.10`, `2.5.0`, `2.5.1`, `2.5.3`, `2.5.5`, `2.5.6`, `2.5.7`, `2.5.8`, `2.5.9`, `2.6.0`, `2.6.1`, `2.6.2`, `2.6.3`, `2.6.4`, `2.6.5`, `2.6.6`, `2.6.7`, `2.6.8`, `2.6.9`, `2.6.10`, `2.7.0` **(default)** , `2.7.1`, `2.7.2`, `2.7.3`, `2.7.4`, `2.7.5`, `2.7.5`, `2.7.6`, `3.0.0`, `3.0.1`, `3.0.2`, `3.0.3`, `3.0.4`, `3.1.0`, `3.1.1`, `3.1.2` | -|DotNet| `5`, `6.0.425` **(default)** , `7`, `8`| -|Python | `3.8.10` **(default)** , `3.9`, `3.10`, `3.11`| -|NodeJs | `12`, `13`, `14`, `15`, `16`, `17`, `18.20.0` **(default)**, `19`, `20` | - - | Supported Languages | Version | - |---------------------|---------| - |Java| `1.8.0` **(default)**, `8`, `9`, `11`, `13`, `15`, `16`, `17`, `18`, `19`, `21`| - |Ruby|`2.0.0`, `2.1.9`, `2.2.6`, `2.3.0`, `2.3.1`, `2.3.3`, `2.4.1`, `2.4.2`, `2.4.3`, `2.4.4`, `2.4.5`, `2.4.6`, `2.4.7`, `2.4.9`, `2.4.10`, `2.5.0`, `2.5.1`, `2.5.3`, `2.5.5`, `2.5.6`, `2.5.7`, `2.5.8`, `2.5.9`, `2.6.0`, `2.6.1`, `2.6.2`, `2.6.3`, `2.6.4`, 2.6.5, `2.6.6`, `2.6.7`, `2.6.8`, `2.6.9`, `2.6.10`, `2.7.0`, `2.7.1`, `2.7.2` **(default)** , `2.7.3`, `2.7.4`, `2.7.5`, `2.7.5`, `2.7.6`, `3.0.0`, `3.0.1`, `3.0.2`, `3.0.3`, `3.0.4`, `3.1.0`, `3.1.1`, `3.1.2`| - |DotNet| `4.6`, `4.7`, `5`, `6`, `7`, `8` **(default)**,| - |Python |`3.7.9` **(default)**, `3.8`, `3.9`, `3.10`, `3.11` | - |NodeJs | `12`, `13`, `14`, `15`, `16`, `17`, `18`, `18.17.1` **(default)**, `19`, `20` | - |Katalon| `9.0.0`, `9.1.0`, `9.2.0`, `9.3.1`| - - | Supported Languages | Version | - |---------------------|---------| - |Java| `1.8.0` **(default)**, `8`, `9`, `11`, `13`, `15`, `16`, `17`, `18`, `19`, `21`| - |Ruby| `2.0.0`, `2.1.9`, `2.2.6`, `2.3.0`, `2.3.1`, `2.3.3`, `2.4.1`, `2.4.2`, `2.4.3`, `2.4.4`, `2.4.5`, `2.4.6`, `2.4.7`, `2.4.9`, `2.4.10`, `2.5.0`, `2.5.1`, `2.5.3`, `2.5.5`, `2.5.6`, `2.5.7`, `2.5.8`, `2.5.9`, `2.6.0`, `2.6.1`, `2.6.2`, `2.6.3`, `2.6.4`, `2.6.5`, `2.6.6`, `2.6.7`, `2.6.8`, `2.6.9`, `2.6.10`, `2.7.0` **(default)**, `2.7.1`, `2.7.2`, `2.7.3`, `2.7.4`, `2.7.5`, `2.7.5`, `2.7.6`, `3.0.0`, `3.0.1`, `3.0.2`, `3.0.3`, `3.0.4`, `3.1.0`, `3.1.1`, `3.1.2`| - |DotNet|`5` **(default)**, `6`, `7`, `8` | - |Python |`3.8`, `3.9`, `3.10`, `3.11`, `3.11.1` **(default)** | - |NodeJs | `12`, `13`, `14`, `15`, `16`, `17`, `18`, `18.17.1` **(default)**, `19`, `20` | - - | Supported Languages | Version | - |---------------------|---------| - |Java| `8` **(default)**, `9`, `11`, `13`, `15`, `16`, `17`, `18`, `19`, `21`| - |Ruby| `2.0.0`, `2.1.9`, `2.2.6`, `2.3.0`, `2.3.1`, `2.3.3`, `2.4.1`, `2.4.2`, `2.4.3`, `2.4.4`, `2.4.5`, `2.4.6`, `2.4.7`, `2.4.9`, `2.4.10`, `2.5.0`, `2.5.1`, `2.5.3`, `2.5.5`, `2.5.6`, `2.5.7`, `2.5.8`, `2.5.9`, `2.6.0`, `2.6.1`, `2.6.2`, `2.6.3`, `2.6.4`, `2.6.5`, `2.6.6`, `2.6.7`, `2.6.8`, `2.6.9`, `2.6.10` **(default)**, `2.7.0`, `2.7.1`, `2.7.2`, `2.7.3`, `2.7.4`, `2.7.5`, `2.7.5`, `2.7.6`, `3.0.0`, `3.0.1`, `3.0.2`, `3.0.3`, `3.0.4`, `3.1.0`, `3.1.1`, `3.1.2`| - |DotNet| `5`, `5.0.407` **(default)**, `6`, `7`, `8`| - |Python |`3.8`, `3.9`, `3.10`, `3.10.11` **(default)**, `3.11` | - |NodeJs | `12`, `13`, `14`, `15`, `16`, `17`, `18`, `18.17.1` **(default)**, `19`, `20` | - - | Supported Languages | Version | - |---------------------|---------| - |Java| `8` **(default)**, `9`, `11`, `13`, `15`, `16`, `17`, `18`, `19`, `21`| - |Ruby| `2.0.0`, `2.1.9`, `2.2.6`, `2.3.0`, `2.3.1`, `2.3.3`, `2.4.1`, `2.4.2`, `2.4.3`, `2.4.4`, `2.4.5`, `2.4.6`, `2.4.7`, `2.4.9`, `2.4.10`, `2.5.0`, `2.5.1`, `2.5.3`, `2.5.5`, `2.5.6`, `2.5.7`, `2.5.8`, `2.5.9`, `2.6.0`, `2.6.1`, `2.6.2`, `2.6.3`, `2.6.4`, `2.6.5`, `2.6.6`, `2.6.7`, `2.6.8`, `2.6.9`, `2.6.10` **(default)**, `2.7.0`, `2.7.1`, `2.7.2`, `2.7.3`, `2.7.4`, `2.7.5`, `2.7.5`, `2.7.6`, `3.0.0`, `3.0.1`, `3.0.2`, `3.0.3`, `3.0.4`, `3.1.0`, `3.1.1`, `3.1.2`| - |DotNet| `5`, `5.0.407` **(default)**, `6`, `7`, `8`| - |Python |`3.8`, `3.9`, `3.10`, `3.10.11` **(default)**, `3.11` | - |NodeJs | `12`, `13`, `14`, `15`, `16`, `17`, `18`, `18.17.1` **(default)**, `19`, `20` | diff --git a/static/docs/hyperexecute-task-metrics.md b/static/docs/hyperexecute-task-metrics.md deleted file mode 100644 index 79216ea52..000000000 --- a/static/docs/hyperexecute-task-metrics.md +++ /dev/null @@ -1,25 +0,0 @@ -# HyperExecute- Monitoring CPU and Memory Resource Consumption - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The effectiveness of your product directly influences your user experience and how they will adopt it. As a result, it is essential to recognize and address any performance issues before the product is made available to the users. - -Test Metrics feature helps you to track whether your product is making inefficient use of resources or not, such as the CPU, and memory utilization by your product at an early stage of the development process. By doing so, you can ensure that your product functions optimally and provides a seamless user experience, increasing its chances of successful adoption by users. - -This document outlines how to monitor CPU and Memory consumption of virtual machines (VMs) at various levels: - -## Task Level View - -The task level view provides insights into the CPU and memory consumption at the individual task level. This means you can see specific metrics related to how much CPU and memory is utilized by each task. - -By analyzing the task level view, you can pinpoint any tasks that may be causing high CPU or memory usage issues, allowing you to optimize their performance for better overall results. - -## Stage Level View - -In the stage level view of HyperExecute, the analysis is segmented into three distinct phases: pre-stage, scenario stage, and post-stage. - -- **Pre-Stage :** The pre-stage view provides insights into resource utilization and actions taken before the main test scenario begins. This phase focuses on the setup, configuration, and preparatory tasks that may influence CPU and memory consumption. By examining the pre-stage metrics, you can ensure that the environment is properly configured for optimal test execution. - -- **Scenario Stage :** During the scenario stage, the focus shifts to CPU and memory consumption while the main test scenario is running. This stage captures performance metrics during the active execution of the test scenario, offering valuable data on resource usage patterns. Analyzing the scenario stage metrics helps identify any performance bottlenecks or inefficiencies affecting the test execution. - -- **Post-Stage :** The post-stage view concentrates on resource consumption and actions taken after completing the main test scenario. By reviewing the post-stage metrics, you can evaluate resource management efficiency and ensure system stability post-test scenario execution. diff --git a/static/docs/hyperexecute-task-widgets.md b/static/docs/hyperexecute-task-widgets.md deleted file mode 100644 index b387938a5..000000000 --- a/static/docs/hyperexecute-task-widgets.md +++ /dev/null @@ -1,19 +0,0 @@ -# HyperExecute Task Widgets - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Task Trends - -The `Task Trends` widget allows you to analyze the trends of the Task for each Job executed on the platform categorized into **Completed**, **Failed**, **Aborted**, **Lambda Error**, and **Cancelled**, which can be filtered by a specific date and time. - -If you want to know the status distribution of the Tasks in the Jobs executed by the team to understand the performance of their application. You can identify if any particular Task in the Job is failing and get the information about it. - -## Task Summary - -The `Task Summary` widget enables the user to track the total Tasks being run on the platform grouped by their status including **Completed**, **Skipped**, **Failed**, **Aborted**, etc. - -If you wants to get a total of the Tasks being executed on the platform for a specific environment or status. With the Task Summary widget you can see the total count of the desired use cases. - -## Task Status Ratio - -The `Task Status Ratio` widget provides an insightful overview of the quality of your current test cases, as marked by the status within your framework on the platform. diff --git a/static/docs/hyperexecute-test-chains.md b/static/docs/hyperexecute-test-chains.md deleted file mode 100644 index a78b2b86b..000000000 --- a/static/docs/hyperexecute-test-chains.md +++ /dev/null @@ -1,64 +0,0 @@ -# HyperExecute Test Chains - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Test Chains allow you to link multiple test workflows so that one workflow triggers another based on specific conditions, such as the success or failure of a previous workflow. This feature eliminates the need for manual intervention between different stages of testing, allowing for a seamless automated testing pipeline. - -In this guide, we will walk through the steps to create test chains using connected workflows in HyperExecute, providing real-world examples and step-by-step instructions on how to implement them. - -## Features of HyperExecute Connected Workflows -- **Automated Workflow Chains:** Define test chains where the success of one workflow triggers subsequent workflows. -- **Flexible Scheduling:** Schedule workflows to run based on your desired frequency and conditions. -- **Conditional Triggering:** Trigger workflows based on specific outcomes, such as the passing or failing of previous tests. -- **Centralized Configuration:** Set up and manage workflows easily within the HyperExecute platform. - -## Real-World Use Case Scenarios -### Example 1: Conditional Regression Testing -Let’s consider a development scenario where: -- **Workflow A** (smoke tests) runs first. This workflow contains a set of critical tests that must pass to ensure the build is functional. -- **Workflow B** (regression tests) runs next, but only if Workflow A passes. - -Our objective is to automatically trigger the regression tests after the smoke tests pass to ensure that the software does not break after critical functionalities are verified. - -**Workflow Flow:** -- **Workflow A - Smoke tests (runs daily):** Executes a subset of tests (e.g., 30 high-severity or critical tests). -- **Workflow B: Regression tests (triggered after successful execution of Workflow A):** Executes a larger set of tests, ensuring that the product remains functional after code changes. - -### Example 2: Severity-based Test Execution -In this scenario, you can define tests to run based on their severity levels. -- **Workflow A:** High-severity tests (e.g., critical functionality tests) are executed every day. -- **Workflow B:** General regression tests (run only after Workflow A passes). - -Our objective is to run essential tests first and trigger a broader set of tests only if the critical tests pass. - -**Workflow Flow:** -- **Workflow A:** High-severity tests run daily. -- **Workflow B:** General regression tests run if Workflow A passes. - -This setup ensures that resources are focused on high-priority tests while the more extensive tests are executed only when necessary. - -## Steps to Set Up Connected Workflows in HyperExecute -Follow these steps to configure connected workflows in HyperExecute: - -### Prerequisite -- Setup your [Project](/support/docs/hyperexecute-projects/#setup-your-project) before setting up the Workflows. -- You must have created your required [workflows](/support/docs/hyperexecute-projects/#schedule-your-workflows) that you want to trigger. - -### Step 1: Setup Workflow -- Click on the "**Setup Workflow**" button: -- Enter Workflow Details: - - **Workflow Name:** Give your workflow a descriptive name (e.g., "Smoke Tests" or "Regression Tests"). - - **Branch Name:** Specify the branch where your tests are stored. - - **YAML File Path:** Provide the path to your YAML configuration file that contains the test definitions for this workflow. -- Click "Next" to proceed to the scheduling configuration. - -### Step 2: Configure the Schedule -- Configure the schedule of your workflow. Select the **days** and **time** at which you want to trigger your tests. Click on **Next**. - -### Step 3: Workflow Linking -You can link this workflow to other existing workflows within the same project or across other projects you own. Once this workflow completes successfully, it can automatically trigger the linked workflows, enabling seamless execution across stages. To set this up, simply select the projects and workflows you wish to trigger from the available list. This feature allows you to build automated, end-to-end workflow chains, ideal for orchestrating complex testing and deployment pipelines. - -## Conclusion -HyperExecute’s Test Chain feature significantly enhances the automation of testing processes by creating logical dependencies between workflows. This feature allows for a more efficient, error-free testing pipeline that ensures quality software with minimal manual intervention. - -By setting up workflows that automatically trigger based on the outcomes of previous ones, teams can reduce the time spent managing tests, allowing them to focus on critical tasks. Whether you are running smoke tests first, followed by full regression tests, or organizing tests based on severity, HyperExecute provides the flexibility to streamline your testing processes. diff --git a/static/docs/hyperexecute-test-muting.md b/static/docs/hyperexecute-test-muting.md deleted file mode 100644 index ca2d37fa2..000000000 --- a/static/docs/hyperexecute-test-muting.md +++ /dev/null @@ -1,46 +0,0 @@ -# Test Muting - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Imagine you are a software tester, and you have a test suite with hundreds of tests. Some of these tests are known to fail consistently, but there is no way to disable them. This means that every time you run the test suite, these failing tests cause the suite to fail. This can be frustrating and time-consuming, especially if you are trying to identify and resolve real problems with the application under test. - -**Test Muting** is a feature that allows you to temporarily disable a test case from being executed. This can be useful for skipping failing or irrelevant tests, which can improve the efficiency and effectiveness of your test runs and help you focus on the most important results. - -## How do you Enable Test Muting for your Organization? - -As you can see, we have triggered a Job in HyperExecute, and we've encountered a recurring issue with a specific test consistently failing. To address this, we will implement Test Muting functionality, enabling automatic muting of a test that fails a predefined consecutive number of times. - -Now, to enable the Test Muting, follow the below mentioned steps: - -**Step 1:** Go to the [TestMu AI Dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard). - -**Step 2:** Click on the **Settings** button. - -**Step 3:** Click to the **Organization Settings** section. - -**Step 4:** Go to the **Org Product Preferences** section. - -**Step 5:** Click on the "**Automatically mute tests**" toggle button. - -- Select the threshold number to automatically mute your tests (default setting is 5 times). -- Customize your **Notification Settings** to mute specific tests and reduce distractions during test execution. -- You can also choose if you would like to receive updates on your mail id and click on **Save** button. - -**Step 6:** Now, if you trigger the same job again, you will notice that the particular test has muted. - -## How do you Manually Mute specific Tests? - -**Step 1:** Execute your job on HyperExecute. - -- Click on the **3 dots** of the desired test you want to **Mute**. -- Click on the **Mute Scenario** button. - -**Step 2:** Trigger the same job again, and you will notice that particular test won't be executed. - -## How do you UnMute Tests? - -**Step 1:** Click on the **3 dots** of the muted test. - -**Step 2:** Click on the **Unmute scenario** button. - -> The test will remain in-active until manually unmuted. diff --git a/static/docs/hyperexecute-test-splitting-and-multiplexing.md b/static/docs/hyperexecute-test-splitting-and-multiplexing.md deleted file mode 100644 index 348c89bf3..000000000 --- a/static/docs/hyperexecute-test-splitting-and-multiplexing.md +++ /dev/null @@ -1,19 +0,0 @@ -# Test Splitting and Multiplexing - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -HyperExecute has 3 different modes to select and execute tests: -- Auto Split Strategy -- Matrix Strategy -- Hybrid Strategy - -You can utilise these modes to split and execute tests as per your requirements. - - Auto Split Strategy - Allows you to distribute your tests over multiple nodes by intelligently distributing them over multiple nodes to reduce the test execution time. - - Matrix Multiplexing Strategy - Allows you to split the test cases across different combinations of browsers, OS and custom parameters like files, folders, features, scenarios, etc. - - Hybrid Strategy - Combination of both Auto Split Strategy and Matrix Multiplexing Strategy to run tests in parallel on our virtual machines. diff --git a/static/docs/hyperexecute-test-widgets.md b/static/docs/hyperexecute-test-widgets.md deleted file mode 100644 index 2a145dd4d..000000000 --- a/static/docs/hyperexecute-test-widgets.md +++ /dev/null @@ -1,54 +0,0 @@ -# HyperExecute Test Widgets - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Test Trends - -The Test Trends Widgets enable you to analyze the trends of the tests executed on the platforms over a period of time. You can compare the current trends with the previous trends on the platform by clicking on the legends at the bottom of the graph. - -Use Cases of Test Trends Widgets: - -- Analyze the performance of the tests that are running on the platform. -- Filter by any test capability and track the performance of the tests. -- Track trends in individual capability applied to the tests. - -## Test Summary - -The Test Summary Widget enables you to track the total tests that have run on the platform, grouped by their status, and also includes information about errors like test case errors, idle timeouts, queue timeouts, and Lambda errors. - -## Test Browser - -Test Browser widget displays the overall distribution of the tests on various browsers on the platform. - -You can drill-down more on clicking on the Browser name and get the list by the top 10 Browser versions by ranking order of the total number of tests executed by the browser versions. - -## Test OS - -Test OS widget displays the overall distribution of the tests on various Operating Systems. - -You can drill-down more on clicking on the Browser name and get the further details about that particular Operating System. - -## Test Status Ratio - -The Test Status Ratio widget seamlessly provides information about the quality of your current test cases on the basis of the status marked on the platform. The information can be filtered by clicking on the relevant legends at the bottom of the graph. - -Use Cases: - -- Review the Passed/Failed or other status distribution of the tests run on the platform. -- Track only required status by editing the widget in the settings. - -## Test Health Snapshot - -The Test Health Snapshot widget allows you to easily analyze the flakiness / inconsistency of the tests executed on the platform. You can effortlessly track the performance of the tests by viewing the count of tests that are consistently passing or failing. - -> **NOTE:** The test case health snapshot only displays the count of the unique test names executed on the platform. - -## Error Stats - -The Error Stats widget allows you to easily analyze the tests categorized by their Status on the HyperExecute platform including Test Case Errors, Idle Timeout, Queue Timeout, and Lambda Error. You can even filter the graph by clicking on the relevant legends at the bottom of the graph. - -## Build Health Snapshot - -The Build Health Snapshot widget enables you to track the performance of the builds created on the platform. You can easily analyze the flakiness / inconsistency of the builds created on the platform. It tracks the performance of the builds by viewing the count of builds that are consistently successful or error. - -> **NOTE:** The widget only displays the count of the unique build names created on the platform. diff --git a/static/docs/hyperexecute-testim-integration.md b/static/docs/hyperexecute-testim-integration.md deleted file mode 100644 index 207b54b97..000000000 --- a/static/docs/hyperexecute-testim-integration.md +++ /dev/null @@ -1,110 +0,0 @@ -# Integrate Testim with HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Testim is a AI Native test authoring platform designed to automate software testing, particularly web and mobile applications. It focuses on creating stable and reliable automated tests. It allows creating tests without writing code or by incorporating custom JavaScript for more intricate scenarios. You can manage and scale the test automation process efficiently, which is particularly valuable for Agile development teams. - -This document details the seamless integration between HyperExecute and Testim, enabling you to run your automated tests. - -## Prerequisite - -- You will need your Testim account. -- TestMu AI account. You can [sign up for free](https://www.testmuai.com/register/?redirectTo=https://accounts.lambdatest.com/dashboard). -- TestMu AI [Username and Access Key](/support/docs/hyperexecute-how-to-get-my-username-and-access-key/) - -## Step 1: Setup the Grid - -- Click on your profile icon >> **Grids** button. -- Click on **Add New Grid** button >> select **TestMu AI** as your Grid Type and click on **Next** button. -- Configure your Grid: - - Enter the name of your Grid. - - Update your Host and Port number. - - Enter your TestMu AI Username and Access Key >> click on **Add** button. - -## Step 2: Record your Tests - -- Click on the **New Test** button >> **Start Recording** button. -- Enter your app URL >> click on **Create Test**. -- It will start recording the tests. After your testing is completed, stop the recording and save your test. - -## Step 3: Configure your Test to execute from CLI - -- Go to the **Settings** >> select the **CLI** tab. -- You will find the sample command >> copy that and paste it in the `testRunnerCommand` in your YAML file. - -## Step 4: Configure your YAML - -Create an empty folder, inside which create your YAML file to trigger the test. - -- In the `pre` flag, enter the command to download the testim cli. -- In the `testRunnerCommand`, enter your runner command copied in the previous step. - -```yaml ---- -version: 0.1 -globalTimeout: 150 -testSuiteTimeout: 150 -testSuiteStep: 150 - -runson: mac - -pre: -- npm i -g @testim/testim-cli - -runtime: -language: node -version: "18.0.0" - -autosplit: true - -concurrency: 1 - -testDiscovery: -type: raw -mode: static -command: echo "HYP with Testim" - -testRunnerCommand: ./.hyperexecute/snooper --frameWork testim --testimProject YOUR_PROJECT_ID --testimToken YOUR_TESTIM_TOKEN --testimProjectBranch YOUR_BRANCH_NAME -``` - -## Step 5: Setup the CLI - -After configuring your YAML file, you need to setup the CLI and the environment variables. - -### Download the HyperExecute CLI - -The CLI is used for triggering the tests on HyperExecute. It is recommend to download the CLI binary on the host system and keep it in the root directory of the suite to perform the tests on HyperExecute. - -You can download the CLI for your desired platform from the below mentioned links: - -| Platform | HyperExecute CLI | -| ---------| ---------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| MacOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -### Setup Environment Variable - -Now, you need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in the [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). - -Run the below mentioned commands in your terminal to setup the CLI and the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -## Step 6: Execute your Test - -> **NOTE :** In case of MacOS, if you get a permission denied warning while executing CLI, simply run **`chmod u+x ./hyperexecute`** to allow permission. In case you get a security popup, allow it from your **System Preferences** → **Security & Privacy** → **General tab**. - -Run the below command in your terminal at the root folder of the project: - -```bash -./hyperexecute --config RELATIVE_PATH_OF_YOUR_YAML_FILE -``` - -OR use this command if you have not exported your username and access key in the step 2. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE `} diff --git a/static/docs/hyperexecute-testng-use-cases.md b/static/docs/hyperexecute-testng-use-cases.md deleted file mode 100644 index 6c68d3831..000000000 --- a/static/docs/hyperexecute-testng-use-cases.md +++ /dev/null @@ -1,1007 +0,0 @@ -# HyperExecute Language-Specific Use Cases - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This guide covers common language-specific use cases for running tests on HyperExecute, organized by framework: **Java + TestNG**, **Python**, and **C#**. - -## TestNG Use Cases - -This guide outlines common **Java + TestNG** scenarios for running tests on [HyperExecute](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute/jobs) - -### Q: How can I ensure my tests operate with the appropriate Java version on HyperExecute? - -By default, HyperExecute VMs are provisioned with **Java 8**. If your project requires another version (e.g., **11, 15, 18, 22**), you can use the [`runtime`](/support/docs/deep-dive-into-hyperexecute-yaml/#runtime) feature and specify the compatible version in the `hyperexecute.yaml`. - -**YAML Example for Java 11** - -```yaml title="hyperexecute.yaml" -runtime: -language: java -version: "11" -``` - -### Q: What is test discovery in HyperExecute, and how does it help? -[Test discovery](/support/docs/deep-dive-into-hyperexecute-yaml/#testdiscovery) is the process of pre-identifying the tests (classes, scenarios, or feature files) to be executed. - -##### Why Use Test Discovery? -- **Selective Execution →** Run only the tests you need. -- **Flexibility →** Filter by file paths, tags, or custom logic. -- **Pre-Execution Preview →** Know exactly which tests will run. - -##### Discovery Methods -| Type | Description | Use Case | -| ----------- | ---------------------------------------------------------- | --------------------------------------- | -| `raw` | Runs a shell command to list tests. | Simple, filename/class-based filtering. | -| `automatic` | Uses HyperExecute backend tools (`snooper`) for discovery. | Tag or scenario-based filtering. | - -##### Examples - -Automatic Discovery (Tag-based) - -```yaml title="hyperexecute.yaml" -testDiscovery: -type: automatic -mode: static -args: -featureFilePaths: web/src/test/resources/features -frameWork: java -specificTags: ["@AccountCombineSet"] -``` - -Raw Command Discovery - -```yaml title="hyperexecute.yaml" -testDiscovery: -type: raw -mode: local -command: grep 'public class' src/test/java/hyperexecute/*.java | awk '{print $3}' -``` - -##### Discovery Modes -- [local](/support/docs/deep-dive-into-hyperexecute-yaml/#testdiscovery:~:text=mode%3A%20remote-,mode%3A%20local,-Purpose%3A%20Test) → Runs discovery on your machine (useful for small/simple projects). -- [remote](/support/docs/deep-dive-into-hyperexecute-yaml/#testdiscovery:~:text=locally%2C%20limiting%20visibility.-,mode%3A%20remote,-NEW) → Runs discovery on HyperExecute VM (recommended for large projects). - -### Q: How do I include/exclude tests using tags? -You can pass logical tag expressions in testDiscovery or use the ignoredTags parameter. - -Example: Logical Tag Filtering - -```yaml title="hyperexecute.yaml" -testDiscovery: -command: .hyperexecute/snooper --targetOs=win \ ---featureFilePaths=web/src/test/resources/features \ ---frameWork=java \ ---query="@UAT2Miniregression and not @FLNAUAT2" \ -| awk '{gsub("web/", ""); print}' -mode: static -type: raw -``` - -Example: Ignored Tags - -```yaml -ignoredTags: ["@tag3", "@tag2"] -``` - -### Q: How do I configure the runner command for different Cucumber versions? - -| Cucumber Version | Runner Command Example | -| ---------------- | -------------------------------------- | -| v6 and below | `mvn test -Dcucumber.options="$test"` | -| v7 and above | `mvn test -Dcucumber.features="$test"` | - -### Q: What if my project has multiple Maven modules? -In projects with modules (web, api, mobile), discovered test paths may include the module prefix (e.g., web/), causing mismatches. - -**Solution :** Use awk to strip module prefixes from discovered test paths. - -```yaml title="hyperexecute.yaml" -testDiscovery: -command: .hyperexecute/snooper --targetOs=win \ ---featureFilePaths=web/src/test/resources/features \ ---frameWork=java \ ---specificTags=@AccountCombineSet \ -| awk '{gsub("web/", ""); print}' -mode: static -type: raw -``` - -### Q: How should I configure testng.xml when I have multiple runners? - -To avoid duplicate executions: - -- Use one runner class in `testng.xml`. -- Comment out tags in `@CucumberOptions`. -- Let HyperExecute discovery handle filtering. - -**Example: testng.xml** - -```xml title="testng.xml" - - - - - - - -``` - -**Example: @CucumberOptions** - -```javascript -@CucumberOptions( -features = "src/test/resources/features", -// tags = "@Regression and not @ignore", // Commented out -glue = "com/qt/sid/stepdefinitions", -plugin = { -"pretty", -"html:test-output/cucumber-reports/html-report.html", -"json:test-output/cucumber-reports/json-report.json", -"com.aventstack.extentreports.cucumber.adapter.ExtentCucumberAdapter:", -"json:target/cucumber.json" -}, -monochrome = true -) -``` - -Discovery YAML Example - -```yaml title="hyperexecute.yaml" -testDiscovery: -type: automatic -mode: static -args: -featureFilePaths: src/test/resources/features/SanitySuite -frameWork: java -specificTags: ["@Regression"] -``` - -### Q: What if my Allure reports/screenshots are too large to render on the dashboard? -Large reports may fail to render in HyperExecute dashboard. It is recommended to generate zipped Allure reports for local viewing. - -YAML Example - -```yaml title="hyperexecute.yaml" -report: true -partialReports: -location: allure-results/webapp -type: zip -frameworkName: allure-zip -``` - -To view Locally -- Install Allure - -```bash -brew install allure -``` - -- Open report - -```bash -allure open ./pathDirectory # Replace pathDirectory with the actual report folder. -``` - -## Python Use Cases - -This guide covers the most common real-world scenarios and troubleshooting cases when running **Python**, **Pytest**, and **Behave** test frameworks on **TestMu AI HyperExecute**. Each section describes a use case, its root cause, and the precise YAML translation required to configure or fix the issue within your `hyperexecute.yaml`. - -### 1. Handling Python Version Compatibility Issues -Sometimes, test scripts fail due to Python version incompatibility. For example: - -```bash title="Terminal" -$ python test_script.py -Traceback (most recent call last): -File "test_script.py", line 3, in -from collections import MutableMapping -ImportError: cannot import name 'MutableMapping' from 'collections' -``` -In Python versions 3.3 to 3.9, MutableMapping was part of the collections module. From Python 3.10 onward, it was moved to collections.abc, causing import errors in older code. - -**Solution :** To ensure compatibility, match the Python runtime version with your local setup. Specify the version explicitly in your YAML configuration. - -**YAML Translation :** -```yaml title="hyperexecute.yaml" -runtime: -- language: python -version: "3.9" -``` - -### 2. Ensuring PIP Version Compatibility -Certain packages require a minimum version of PIP to install successfully. An outdated PIP version may cause the following error: -> **ERROR :** You must install pip 21.0 or later to install this package. - -**Solution :** Upgrade PIP before installing dependencies. - -```bash -python -m pip install --upgrade pip -``` - -If you are using Python 3: - -```bash -python3 -m pip install --upgrade pip -``` - -**YAML Translation :** -```yaml title="hyperexecute.yaml" -pre: -- python3 -m pip install --upgrade pip -``` - -### 3. Fixing Missing requirements.txt Errors -If your pipeline fails with: -> **ERROR:** Could not open requirements file: [Errno 2] No such file or directory: 'requirements.txt' - -It means that the `requirements.txt` file might be missing or ignored in .gitignore. - -**Solution :** Ensure the file exists in the project root. Remove or comment out any requirements.txt entry from .gitignore. For this issue, ther are no YAML change required. This is a file management fix. However, verify that the command below correctly references the existing file name. - -```bash -pip install -r requirements.txt --cache-dir CacheDir -``` - -### 4. Restoring Missing .env Files -When environment variables are not set, tests may fail due to missing credentials or configuration values because `.env` files are commonly excluded in `.gitignore` for security. If ignored, they aren’t uploaded to HyperExecute, and environment variables remain unset. - -**Solution :** Ensure that required `.env` files are not ignored in `.gitignore`. If sensitive credentials exist, use HyperExecute Secrets or Environment Variables instead of exposing them in `.env` files. - -### 5. Optimizing Package Caching for Faster Jobs -If your pre step installs dependencies every time, your caching configuration might be incorrect. - -**Common Causes :** -- Invalid `cacheKey` syntax. -- Incorrect filename in cache reference. -- Missing `--cache-dir` argument. - -**Solution :** Use proper syntax, ensure file naming consistency, and define a cache directory. - -YAML Translation: - -```yaml title="hyperexecute.yaml" -cacheKey: '{{ checksum "requirements.txt" }}' -cacheDirectories: -- CacheDir -pre: -- pip install -r requirements.txt --cache-dir CacheDir -``` - -### 6. Building Projects with Public and Private Registries -#### Case 1: Public Registry Installation -If all dependencies are public, install them directly: - -```bash -pip3 install -r requirements.txt --cache-dir CacheDir -``` - -**YAML Translation :** - -```yaml title="hyperexecute.yaml" -pre: -- pip3 install -r requirements.txt --cache-dir CacheDir -``` - -#### Case 2: Private Registry Installation -When using a private registry, authentication is required. Use proxy variables and authorization tokens to install packages securely. - -```bash -pip3 install --proxy http://$LT_PROXY_HOST:$LT_PROXY_PORT \ ---header "Authorization: Bearer $PRIVATE_REGISTRY_TOKEN" \ --r requirements.txt -``` - -**YAML Translation :** - -```yaml title="hyperexecute.yaml" -pre: -- pip3 install --proxy http://$LT_PROXY_HOST:$LT_PROXY_PORT \ ---header "Authorization: Bearer $PRIVATE_REGISTRY_TOKEN" \ --r requirements.txt -``` - -**Sample `requirements.txt` file :** - -```text ---extra-index-url https://your-private-registry.com/simple ---trusted-host your-private-registry.com -your-private-package==1.0.0 -another-package==2.0.0 -``` - -**Error Troubleshooting :** -- Verify `$LT_PROXY_HOST` and `$LT_PROXY_PORT `values. -- Confirm `$PRIVATE_REGISTRY_TOKEN` is valid. - -### 7. Configuring Database Connections -When connecting to a database during test runs, you may encounter: - -> **ConnectionRefusedError:** connect ECONNREFUSED 127.0.0.1:3306 - -**Solution :** Expose the database service using the `--expose` flag in your YAML configuration. - -**YAML Translation :** - -```yaml title="hyperexecute.yaml" -args: -- "--expose mysql1:localhost:3309" -- "--expose mysql2:localhost:3307" -``` - -**Sample Connection Code :** - -```python title="test.py -import os -import mysql.connector - -db_host = os.getenv("MYSQL1_PROXY_HOST", "localhost") -db_port = os.getenv("MYSQL1_PROXY_PORT", "3309") - -connection = mysql.connector.connect( -host=db_host, -port=db_port, -user="your_user", -password="your_password", -database="your_database" -) -``` - -### 8. Running Tests in Behave and Pytest -To execute tests based on substrings or tags: - -**Example Commands :** - -```bash -pytest -k "TestAdvanceBoy" -behave -t @smoke -``` - -**YAML Translation :** - -```yaml title="hyperexecute.yaml" -testRunnerCommand: pytest -k "TestAdvanceBoy" -# or -testRunnerCommand: behave -t @smoke -``` - -To execute an entire feature file: - -```bash -behave -f feature_path/sample.feature -``` - -**YAML Translation :** - -```yaml title="hyperexecute.yaml" -testRunnerCommand: behave -f feature_path/sample.feature -``` - -### 9. Discovering Tests Dynamically - -- List all feature files containing a specific tag (e.g., @smoke): - -```bash -grep -rl "@smoke" features/*.feature -``` - -**YAML Translation :** - -```yaml title="hyperexecute.yaml" -testDiscovery: -type: raw -command: grep -rl "@smoke" features/*.feature -``` - -- List all scenarios under that tag : - -```bash -grep -rH -A 3 "@smoke" features/*.feature | grep -i "Scenario" | cut -d ":" -f 2- -``` - -**YAML Translation :** - -```yaml title="hyperexecute.yaml" -testDiscovery: -type: raw -command: grep -rH -A 3 "@smoke" features/*.feature | grep -i "Scenario" | cut -d ":" -f 2- -``` - -### 10. Generating JSON Reports with Behave -To generate JSON reports for your test execution: - -```bash -behave -f json.pretty -o reports/test_report.json -``` - -**YAML Translation :** - -```yaml title="hyperexecute.yaml" -post: -- behave -f json.pretty -o reports/test_report.json -``` - -### 11. Dynamically Assigning Artifact Names -To dynamically name artifacts using environment variables, define them in your YAML configuration. - -**YAML Translation :** - -```yaml title="hyperexecute.yaml" -uploadArtefacts: -- name: "${abcd}" -path: -- reports/** -``` - -Pass variable values through CLI during job execution: - -```bash -./hyperexecute -u -k -i --vars "abcd=Folder" -``` -If abcd=Folder, the uploaded artifact will appear as Folder on the HyperExecute dashboard. - -### 12. Using a Cache Directory for Package Installation -To enable caching for faster dependency installation: - -```bash -pip3 install -r requirements.txt --cache-dir CacheDir -``` - -This ensures packages are stored and reused in subsequent runs, reducing job setup time. - -### 13. Monitoring Tunnel Health and Auto-Retry -When running automated tests in HyperExecute, tunnel connections may drop or become unstable, leading to test failures. - -**Scenario/Use Case:** -- Tunnel may drop due to network instability, firewall restrictions, or internal server issues. -- Goal: Continuously monitor tunnel health and automatically retry without manual intervention. - -**Solution** -Use the `checkTunnelOnFailure` parameter in your YAML configuration. This ensures HyperExecute automatically retries the tunnel if it fails during test execution. - -**YAML Translation:** - -```yaml title="hyperexecute.yaml" -checkTunnelOnFailure: true -``` - -### 14. Different Ways to Execute Tests in Pytest via CLI -#### Case 1: Run Tests in a Specific Directory or File - -```bash -pytest path/to/test_directory/ -pytest path/to/test_file.py -``` - -#### Case 2: Run a Specific Test Function Within a File -```bash -pytest path/to/test_file.py::test_function_name -``` - -#### Case 3: Run Tests Matching a Substring or Keyword -```bash -pytest -k "test_keyword" -``` - -**Example :** -```bash -pytest -k "login" -``` - -### 15. Debugging in Pytest with Detailed Logs - -**Solution :** Use the `--log-cli-level=DEBUG` flag to capture debug-level logs. - -**Example Python Test :** - -```python title="test.py -import logging - -logging.basicConfig(level=logging.DEBUG) -logger = logging.getLogger() - -def test_debug_logging(): -value = 42 -if value == 42: -logger.debug("Value is 42, running debug checks") -logger.debug("This is a debug message") -logger.info("Test info message") -logger.warning("Test warning message") -logger.error("Test error message") -if value != 42: -logger.critical("Critical issue: Value should be 42!") -assert value == 42 -``` - -**Terminal Output Example :** - -```bash -DEBUG:root:Value is 42, running debug checks -DEBUG:root:This is a debug message -INFO:root:Test info message -WARNING:root:Test warning message -ERROR:root:Test error message -``` - -### 16. Automating Screenshot Download and PDF Generation from TestMu AI -During native app automation, screenshots captured per session ID can be retrieved using: - -```bash -curl -X GET "https://mobile-api.lambdatest.com/mobile-automation/api/v1/sessions/${session_id}/screenshots" \ --H "accept: application/json" \ --H "Authorization: Basic " -``` - -**Problem :** Manually downloading, extracting, and arranging screenshots into a PDF is time-consuming. - -**Solution :** Use the TestMu AI Screenshot Downloader & PDF Generator CLI. - -- **Installation :** -```bash -npm install -g lambdatest-screenshot-pdf -``` - -- **Usage :** -```bash -AUTH_HEADER="Basic your_encoded_auth_string" lambdatest-screenshot-pdf --output -``` - -- **Example :** -```bash -AUTH_HEADER="Basic cmF0aG9yZXYXYXYXYXXgybFlXTm5LclZPeVRRQ01RRGdyNTZjVg==" \ -lambdatest-screenshot-pdf 1a80510a-289a-46b7-9f60-da01d108de10 --output ~/Desktop/screenshots -``` - -**What Happens When You Run This Command:** -- Fetches the ZIP file containing screenshots for the session ID. -- Downloads the ZIP file. -- Extracts screenshots in the original execution order. -- Generates a screenshots.pdf containing the ordered screenshots. -- Cleans up temporary files automatically. -- Ensure your AUTH_HEADER is a valid Base64-encoded API key from TestMu AI. - -### 17. Base YAML Configuration for Pytest on Windows with Autosplit - -Example hyperexecute.yaml: - -```yaml title="hyperexecute.yaml" ---- -version: 0.1 -globalTimeout: 90 -testSuiteTimeout: 90 -testSuiteStep: 90 - -runson: win -autosplit: true -concurrency: 2 -runtime: -language: python -version: "3" - -cacheKey: '{{ checksum "requirements.txt" }}' -cacheDirectories: -- CacheDir -pre: -- pip3 install -r requirements.txt --cache-dir CacheDir -post: -- cat yaml/win/pytest_hyperexecute_autosplit_sample.yaml - -testDiscovery: -type: raw -mode: remote -command: grep -nri 'class' tests -ir --include=\*.py | sed 's/:.*//' - -testRunnerCommand: pytest -s --verbose $test - -jobLabel: [selenium-pytest, win, autosplit] -``` - -## C# Use Cases - -This guide provides a complete reference for integrating C# NUnit SpecFlow tests with HyperExecute, covering edge cases, solutions, YAML setup, artifact management, remote test discovery, and reporting. - -### 1. Autosplit -#### Edge Case -Large test suites with hundreds or thousands of tests can leave some nodes idle while others are overloaded, resulting in wasted time and uneven execution. - -#### Solution -AutoSplit divides tests intelligently across nodes at file, module, or scenario level. Previously failed tests are prioritized for faster feedback. This ensures maximum concurrency usage and reduces test flakiness, enabling faster feedback loops for large BDD suites. - -```yaml title="hyperexecute.yaml" -autosplit: true -``` - -#### What Happens If We Don’t Use It - -- Inefficient node usage -- Slower test execution -- Flaky tests delay feedback -- No control over test splitting - -### 2. Matrix Setup -#### Edge Case -Tests behave differently on various OS platforms. Running only on one OS may hide platform-specific bugs. - -#### Solution -The matrix runs tests concurrently on Mac, Windows, and Linux, detecting OS-specific issues like file paths, encoding, or browser behavior. - -```yaml title="hyperexecute.yaml" -matrix: -os: [mac, win, linux] - -runson: ${matrix.os} -``` - -#### What Happens If We Don’t Use It -- Tests may pass on one OS but fail on others -- Hidden bugs reach end users -- Manual cross-OS testing required - -### 3. Environment Variables (NuGet Paths) -#### Edge Case -Parallel jobs sharing default NuGet caches may cause corrupted packages, race conditions, or build failures, especially in ephemeral environments or low disk space. - -#### Solution -Predefining NuGet paths ensures controlled caching, reliable builds across OSes, and efficient reuse of restored packages. - -```yaml title="hyperexecute.yaml" -env: -NUGET_PACKAGES: '/home/ltuser/.nuget/packages/' -NUGET_HTTP_CACHE_PATH: '/home/ltuser/.local/share/NuGet/v3-cache' -NUGET_PLUGINS_CACHE_PATH: '/home/ltuser/.local/share/NuGet/plugins-cache' -``` - -#### What Happens If We Don’t Use It -- Random failures in parallel jobs -- Slower builds -- High network usage -- Disk space issues -- Inconsistent behavior across OSes - -### 4. Pre Steps -#### Edge Case -Dependencies may not be restored or projects may not build correctly if pre-steps are missing, causing test failures downstream. - -#### Solution - -```yaml title="hyperexecute.yaml" -pre: -- dotnet restore -- dotnet list OnlySpecTest.csproj package > packages.txt -# - chmod 777 /tmp/NuGetScratch -# - nuget locals all -clear -- dotnet build -c Release -``` - -- **dotnet restore** -> Restores NuGet packages -- **dotnet list ... > packages.txt:** -> Saves package list for caching -- **dotnet build -c Release:** -> Builds the project in Release mode -- Optional commands handle permissions or cache clearing - -#### What Happens If We Don’t Use It -- Builds fail or produce outdated DLLs -- Tests fail due to missing dependencies -- Manual fixes required - -### 5. Artifact Upload -#### Edge Case -Without artifacts, failed tests provide no screenshots, logs, or reports for debugging, and outputs can be scattered in parallel executions. - -#### Solution -Artifacts centralize outputs like HTML reports and screenshots, enabling debug, traceability, and audit. - -```yaml title="hyperexecute.yaml" -uploadArtefacts: -- name: Execution_Report -path: -- Report/** -- name: Execution_Screenshots -path: -- Screenshots/**/** -``` - -#### What Happens If We Don’t Use It -- No visual proof or logs -- Debugging intermittent failures is hard -- Manual reporting is required - -### 6. Report Generation -#### Edge Case -Partial or misconfigured reports can be unreadable, incomplete, or fail to merge in parallel executions. - -#### Solution -Generates human-readable HTML reports showing pass/fail, steps, screenshots, and aggregated results for stakeholders. - -```yaml title="hyperexecute.yaml" -report: true -partialReports: -location: Report/ -type: html -frameworkName: specflow -``` - -#### What Happens If We Don’t Use It -- No centralized reporting -- Harder to analyze test performance trends -- Manual effort needed to share results - -### 7. Test Discovery -#### Edge Case -Static or incorrect discovery may skip new tests, misdetect test tags, or fail in parallel executions. - -#### Solution -Remote discovery automatically detects new @tags or features, enabling parallelization and reducing manual maintenance. - -```yaml title="hyperexecute.yaml" -testDiscovery: -type: raw -mode: remote -command: grep -rni 'Features' -e '@' --include=*.feature | sed 's/.*@//' -``` - -#### What Happens If We Don’t Use It -- Manual selection of tests -- New tests may never run -- Reduced test coverage - -### 8. testRunnerCommand -#### Edge Case -Running all tests every time is inefficient, and category typos or missing attributes can prevent execution. - -#### Solution -Executes only tests with the specified Category. Supports parallelization and selective reruns. - -```yaml title="hyperexecute.yaml" -testRunnerCommand: dotnet test --filter "(Category=$test)" -``` - -#### What Happens If We Don’t Use It -- All tests run every time -- Slower pipelines -- Increased build cost - -### 9. Multiple Tag-Based Test Discovery -#### Edge Case -Projects may have diverse tagging strategies, making standard discovery unreliable or incomplete. - -#### Solution - -```yaml title="hyperexecute.yaml" -testDiscovery: -type: raw -mode: remote -shell: bash -command: grep -E -w -R -A10 --include='*.feature.cs' ${tagname} AllInOneProject/features | grep 'public void' | cut -d' ' -f2- | awk '{print $3}' | sed 's/()$//' | sed 's/TestTearDown//g' -``` - -- Dynamically extracts test names based on tags -- Ensures new tests are automatically included -- Supports parallel execution and CI/CD pipelines - -#### What Happens If We Don’t Use It -- Missing tests due to inconsistent tagging -- Manual updates required -- Increased risk of regressions - -### 10. Framework Version Compatibility Handling -#### Edge Case -Different .NET versions require different build steps; incorrect handling causes build failures or runner crashes. - -#### Solution -Builds projects correctly based on target framework, ensuring compatibility with HyperExecute runners. - -**`.NET 6`+**: Use dotnet build in pre-steps -**`.NET <6`**: Build locally and upload DLLs - -```yaml title="hyperexecute.yaml" -C:\PROGRA~2\Micros~1\2019\Community\MSBuild\Current\Bin\MSBuild.exe /p:Configuration=Release /p:TargetFramework=net472 /t:restore -``` - -#### What Happens If We Don’t Use It -- Build failures -- Outdated or incompatible DLLs -- Test runner crashes - -### 11. Private Dependencies (config.nuget) -#### Edge Case -Projects depending on private NuGet feeds fail restore if credentials or feeds are not configured. - -#### Solution -Configures private feeds securely to ensure reliable dependency resolution across environments. - -```xml - - - - - - - - - - - - - -``` - -#### What Happens If We Don’t Use It -- Restore failures (401 Unauthorized) -- CI/CD errors -- Manual debugging and inconsistent builds - -### 12. Custom Test Runner Alternatives -#### Edge Case -Some projects cannot run with dotnet test due to configuration or .NET version constraints. - -#### Solution -Alternative runner allows execution of targeted tests, respecting project-specific configurations. - -```yaml -vstest.console.exe "GlobalPolaris\bin\Debug\net472\GlobalPolaris.dll" /Settings:GlobalPolaris\RunSettings.runsettings /TestCaseFilter:"FullyQualifiedName~$test" -``` - -#### What Happens If We Don’t Use It -- Tests may fail to execute -- Build verification and pipelines are blocked - -### 13. NUnit Console for Feature Tag Discovery -#### Edge Case -Standard discovery may fail with complex NUnit tag setups. - -#### Solution -Uses NUnit console to explore feature tests by tags. -```yaml -& "C:\PROGRA~2\NUnit.org\nunit-console\nunit3-console.exe" "" --where="cat==$tag" --explore -``` - -#### What Happens If We Don’t Use It -- Tag-based tests are skipped -- Partial coverage -- Manual intervention required - -### 14. Differential Upload Configuration -#### Edge Case -Large test suites take long to upload; repeated full uploads waste time and bandwidth. - -#### Solution -Uploads only changed files, reducing upload time and network usage. - -```yaml title="hyperexecute.yaml" -differentialUpload: -enabled: true -ttlHours: 300 -``` - -#### What Happens If We Don’t Use It -- Slow uploads -- Wasted resources -- Longer pipeline durations - -### 15. Certificate Management for Browser Authentication -#### Edge Case -Browser tests requiring client certificates fail without proper installation. - -#### Solution -Installs certificates in browser environment for authenticated sessions. - -```yaml title="hyperexecute.yaml" -%HYPEREXECUTE_WORKING_DIR%//Hyperexecute//cert_manager.exe --chrome -i atest089.pfx -``` - -#### What Happens If We Don’t Use It -- Browser authentication tests fail -- Manual certificate setup required - -### 16. Custom Status Reporting via CleanUp Function -#### Edge Case -Test statuses may not be reported to HyperExecute properly, leading to incomplete dashboards. - -#### Solution -Custom function reports statuses for tracking and dashboards. - -```java title="test.java" -public void CleanUp(string[] tagName, ScenarioExecutionStatus scenarioStatus, string scenarioName) -{ -string STATUS = scenarioStatus.ToString() switch -{ -"OK" => "passed", -"UndefinedStep" => "skipped", -"TestError" => "failed", -_ => "others" -}; -((IJavaScriptExecutor)Properties.driver).ExecuteScript($"lambda-status={STATUS}"); -} -``` - -#### What Happens If We Don’t Use It -- Missing or inaccurate test reporting -- Dashboard metrics incomplete - -### 17. Custom SpecFlow Reporting Configuration -#### Edge Case -Standard reports may not provide enough detail for complex SpecFlow tests. - -#### Solution -Generates custom reports with metadata and email notifications. - -```yaml title="hyperexecute.yaml" -report: true -partialReports: -location: GlobalPolaris/Reports -type: html -frameworkName: specflow-custom -email: -to: ["example@lambdatest.com"] -metaInfo: ["project-name:Global Polaris Regression","project-env:QA"] -``` - -#### What Happens If We Don’t Use It -- Reports lack detail -- Stakeholders have limited visibility -- Manual reporting required - -### 18. File Level Discovery (DLL) -#### Edge Case -Running tests on the wrong or outdated DLL causes failures or misses new tests. - -#### Solution -Ensures correct DLL is built and executed on the proper runner. - -```yaml title="hyperexecute.yaml" -runson: win - -pre: -commands: -- dotnet restore -- dotnet build YourSolution.sln -c Release - -test: -commands: -- nunit3-console.exe "bin/Release/net6.0/YourProject.dll" -``` - -#### What Happens If We Don’t Use It -- Tests fail -- New features not executed -- Manual DLL verification required - -### 19. Feature File Level Discovery (Matrix) -#### Edge Case -All tests running together reduce parallelism and rerun flexibility. - -#### Solution -Run each feature individually to enable parallel execution and selective reruns. - -```yaml title="hyperexecute.yaml" -matrix: -featurefile: -- "Features/Login.feature" -- "Features/Search.feature" -- "Features/Checkout.feature" - -test: -commands: -- nunit3-console.exe "bin/Release/net6.0/YourProject.dll" --where "cat == '$featurefile'" -``` - -#### What Happens If We Don’t Use It -- Parallelism lost -- Slower execution -- Rerunning failed features is difficult - -### 20. Passing Arguments in HyperExecute -#### Edge Case -Incorrect YAML quoting or missing result file paths cause unintended test runs or missing artifacts. - -#### Solution -Properly passing arguments ensures selective execution and saves results for artifact uploads. - -```yaml title="hyperexecute.yaml" -test: -commands: -- nunit3-console.exe "bin/Release/net6.0/YourProject.dll" --where "cat == 'Regression'" --result="regression-result.xml" -``` - -#### What Happens If We Don’t Use It -- All tests may run accidentally -- Slow pipelines -- Missing artifacts for debugging diff --git a/static/docs/hyperexecute-testsigma-integration.md b/static/docs/hyperexecute-testsigma-integration.md deleted file mode 100644 index 211c38404..000000000 --- a/static/docs/hyperexecute-testsigma-integration.md +++ /dev/null @@ -1,201 +0,0 @@ -# Testsigma Integration With HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Testsigma is an AI Native, cloud-based test automation platform that helps testers create, run, and maintain automated tests for web, mobile, and API applications. - -Testsigma can help you with: - -- **Natural Language Support:** You can create test cases in simple English, which makes it easy for people without programming experience to get started with test automation. - -- **AI Native Test Generation:** Testsigma can automatically generate test cases for you based on your application's code or user interface. - -- **Cloud-Based Execution:** You can run your tests on the cloud without having to set up your own infrastructure. - -- **Cross-Browser Testing:** You can test your application on different browsers and devices. - -- **Continuous Integration:** Testsigma can be integrated with your CI pipeline so that you can run your tests automatically every time you make changes to your code. - -- **Reporting:** Testsigma provides detailed reports on the results of your tests. - -## Integrate Testsigma with HyperExecute - -### Step 1: Creation of Agent - -- Navigate to your Testsigma Dashboard to [Create an agent](https://testsigma.com/docs/agent/setup-on-windows-mac-linux/). - -> **Note:** -- This agent has an activation key. You must ensure that it is secure and accessible. -- This activation key will allow you to start your agent from any other machine. -- This key is associated with a single agent and can be used on only one machine at a time. - -### Step 2: Create your HyperExecute YAML file - -When creating the YAML, consider the following points: - -- **One-to-One Mapping:** Each test plan ID can only be linked to a single agent at a time. This means two separate agents will be required to execute two test plans parallelly. - -- **Agent Activation Wait:** Since the agents are activated before test execution, it's crucial to include a waiting period of 90-100 seconds. This ensures the agents are fully operational and ready to handle the upcoming workload. - -- **Dynamic Activation Key Handling:** The `$activationKeys` variable defined in the matrix section allows you to dynamically pass the appropriate activation key to each agent during execution. This simplifies configuration and avoids manual key assignment. - -Here is the sample YAML file for your reference: - -```yaml ---- -version: 0.1 -globalTimeout: 150 -testSuiteTimeout: 150 -testSuiteStep: 150 - -runson: win -concurrency: 2 - -env: -CACHE_DIR: m2_cache_dir - -cacheKey: '{{ checksum "pom.xml" }}' -cacheDirectories: -- ${CACHE_DIR} - -matrix: -tests: ["48","70"] -activationKeys: ["agent1_activationkey","agent2_activationkey"] -exclusionMatrix: -- tests: ["48"] -activationKeys: ["agent1_activationkey"] -- tests: ["70"] -activationKeys: ["agent2_activationkey"] - -pre: -- npm install -g npm@10.4.0 -- npm i axios - -mergeArtifacts: true - -uploadArtefacts: -- name: ExecutionSnapshots -path: -- target/surefire-reports/html/** - -report: true -partialReports: -location: target/surefire-reports/html -type: html -frameworkName: extent - -testSuites: -- Sleep 100 -- node script.js $tests; Sleep 45 - -post: -- stop-agent.bat - -captureScreenRecordingForScenarios: true - -background: -- start-agent.bat $activationKeys - -jobLabel: [testsigma] -``` - -### Step 3: Setup the Test Execution Files - -- Create a **start-agent.bat** file. This particular **.bat** file will start the agent in [background services](/support/docs/hyperexecute-background-services/). - -```bash -@echo off - -REM Step 1: Download TestsigmaAgent-Windows.zip -curl -O https://s3.amazonaws.com/hybrid-production.testsigma.com/agent/windows/4.7.0/TestsigmaAgent-Windows.zip - -REM Step 2: Extract the contents of TestsigmaAgent-Windows.zip -jar xf TestsigmaAgent-Windows.zip - -REM Step 3: Change the directory to TestsigmaAgent -cd TestsigmaAgent - -REM Step 4: Run start.bat with the provided activation key -start.bat "TS_ACTIVATION_KEY=%1" -``` - -- Create a **stop-agent.bat** file. This **.bat** file will stop the agent after the completion of execution. - -```bash -@echo off - -REM Step 1: Change the directory to TestsigmaAgent -cd TestsigmaAgent - -REM Step 2: Run start.bat with the provided activation key -stop.bat -``` - -- Now, map your tests (from the testsigma dashboard) via the agent to execute on HyperExecute platform. The main goal of the script file is to make the API call for test execution. - -- The script is created to start the agent, then execute the test plan and then stop the agent after the whole Job is executed. - -> **NOTE:** You can use any language to create your execution script file. We have used **javascript** for the reference below. - -```bash -const axios = require('axios'); - -const testPlanId = process.argv[2]; - -const numid1 = parseInt(testPlanId); -let data = JSON.stringify({ -"executionId": numid1 -}); - -let config = { -method: 'post', -maxBodyLength: Infinity, -url: 'https://app.testsigma.com/api/v1/execution_results', -headers: { -'Content-Type': 'application/json', -'Authorization': 'Bearer ' -}, -data: data -}; - -axios.request(config) -.then((response) => { -console.log(JSON.stringify(response.data)); -}) -.catch((error) => { -console.log(error); -}); - -``` - -### Step 4: Execute the Test - -Trigger your test plan on HyperExecute. - -- Download the Hyperexecute CLI Binary based on the OS you use. - -| Platform | Download Link | -|-----------|--------------| -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| macOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -- Run the below mentioned commands in your terminal to setup the CLI and the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -> In case of MAC, if you get a permission denied warning while executing CLI, simply run `chmod u+x ./hyperexecute` to allow permission. In case you get a security popup, allow it from your **System Preferences → Security & Privacy → General tab**. - -- Run the below command in your terminal at the root folder of the project - -```bash -./hyperexecute --config RELATIVE_PATH_OF_YOUR_YAML_FILE -``` - -OR use this command if you have not exported your username and access key. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE `} diff --git a/static/docs/hyperexecute-use-cases.md b/static/docs/hyperexecute-use-cases.md deleted file mode 100644 index 4b09d7c66..000000000 --- a/static/docs/hyperexecute-use-cases.md +++ /dev/null @@ -1,19 +0,0 @@ -# HyperExecute Use Cases - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestNG - - Python + Pytest + Behave - - C# + NUnit + SpecFlow - - YAML creation for Playwright tests - - How you can turbo boost your tests execution with our Smart Caching feature? - - How you can seamlessly integrate your tests testing tool with HyperExecute for enhanced Test Orchestration? - - How to build your mobile applications via HyperExecute? - - How to leverage custom libraries in SikuliX for advanced visual verification? diff --git a/static/docs/hyperexecute-utilizing-custom-libraries.md b/static/docs/hyperexecute-utilizing-custom-libraries.md deleted file mode 100644 index 5a79f7e84..000000000 --- a/static/docs/hyperexecute-utilizing-custom-libraries.md +++ /dev/null @@ -1,28 +0,0 @@ -# Leveraging Custom Libraries in SikuliX for Advanced Visual Verification - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Suppose you want to automate interactions with a desktop application, but traditional frameworks like Selenium fall short because the UI elements lack proper identifiers. - -SikuliX, an automation solution, adept at automating interactions with the graphical user interface (GUI) of desktop computers running Windows, Mac, or select Linux/Unix distributions. With its robust image recognition capabilities of OpenCV, SikuliX excels in scenarios where traditional automation frameworks lack access to GUI internals or application source code. - -SikuliX extends its versatility through support for multiple scripting languages, enabling developers to leverage their preferred language for automation tasks. The supported scripting languages include: - -- Python (level 2.7, supported by Jython) -- RobotFramework text-scripts -- Ruby (level 1.9 and 2.0, supported by JRuby) -- JavaScript (supported by the Java Scripting Engine) - -## HyperExecute's Role in SikuliX Automation - -HyperExecute serves as the backbone for managing and executing SikuliX automation workflows efficiently. By leveraging HyperExecute, developers can ensure seamless execution of SikuliX scripts across diverse environments, without being constrained by platform-specific dependencies or configurations. - -## Benefits of Automating SikuliX with HyperExecute - -- **Build Visual UI tests:** SikuliX handles element identification, while Hyperexecute orchestrates test execution and reporting. - -- **Run tests across different machines:** Hyperexecute ensures consistent environments regardless of local configurations. - -- **Automate legacy applications:** SikuliX tackles even outdated UIs, making Hyperexecute your vintage automation champion. - -- **Leverage multiple languages:** Choose your preferred scripting language for SikuliX, knowing Hyperexecute handles the execution infrastructure. diff --git a/static/docs/hyperexecute-vnet-peering-for-private-cloud.md b/static/docs/hyperexecute-vnet-peering-for-private-cloud.md deleted file mode 100644 index dac8c1c61..000000000 --- a/static/docs/hyperexecute-vnet-peering-for-private-cloud.md +++ /dev/null @@ -1,191 +0,0 @@ -# VNet Peering and VNet Links - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Congratulations on getting the HyperExecute Private setup deployed on your Azure Cloud! - -You need to provision a certain public machine inside the Vnet to access this private setup in your environment. This setup cannot access your private applications, URLs, and APIs directly due to the measures set in place for optimal security. - -To solve this issue, we need to perform Vnet peering from the Vnet that is created inside the HyperExecute dedicated resource group to the Vnet that is already present in your account. This account must be the one which contains all of your endpoints, or your test execution account, as the test machines need to access those endpoints. - -You can enable Vnet peering and Vnet linking in your account by following the process documented below. This process uses an ARM template to provision the resources. - -## Steps to Deploy ARM Template - -- Navigate to the search bar in your Azure account and search for **Deploy Custom Template**. Select the option which matches the searched name. - -- You will be able to see **Custom Deployment** page. - -- Click on **Build your own template in the editor** to go to the **Edit template** section. - -- Clear all the text present in the editor and paste the contents of the ARM script provided below. - - **ARM Template** - -```bash -// #################################################################################### -// #### #### -// #### HyperExecute Private Cloud Vnet Peering ARM #### -// #### #### -// #################################################################################### - -{ -"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", -"contentVersion": "1.0.0.0", -"parameters": { -"existingHyperExecuteVirtualNetworkName": { -"type": "String", -"metadata": { -"description": "Set the HyperExecute VNet name" } -}, -"existingHyperExecuteVirtualNetworkResourceGroupName": { -"type": "String", -"metadata": { -"description": "Sets the HyperExecute VNet Resource group" } -}, -"existingRemoteVirtualNetworkName": { -"type": "String", -"metadata": { -"description": "Set the remote VNet name" } -}, -"existingRemoteVirtualNetworkResourceGroupName": { -"type": "String", -"metadata": { -"description": "Sets the remote VNet Resource group" } -}, -"blobPrivateDnsZoneName": { -"type": "String", -"defaultValue": "privatelink.blob.core.windows.net", -"metadata": { -"description": "Sets the Private DNS zone created for Storage account's privatelink" } -}, -"redisPrivateDnsZoneName": { -"type": "String", -"defaultValue": "privatelink.redis.cache.windows.net", -"metadata": { -"description": "SSets the Private DNS zone created for Redis privatelink" } -} -}, -"resources": [ -{ -"type": "Microsoft.Resources/deployments", -"resourceGroup": "[parameters('existingHyperExecuteVirtualNetworkResourceGroupName')]", -"apiVersion": "2020-10-01", -"name": "[guid(parameters('existingHyperExecuteVirtualNetworkName'), parameters('existingHyperExecuteVirtualNetworkResourceGroupName'), 'peering')]", -"properties": { -"mode": "Incremental", -"template": { -"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", -"contentVersion": "1.0.0.0", -"parameters": {}, -"resources": [ -{ -"type": "Microsoft.Network/virtualNetworks/virtualNetworkPeerings", -"apiVersion": "2021-02-01", -"name": "[format('{0}/local-to-remote-vnet', parameters('existingHyperExecuteVirtualNetworkName'))]", -"properties": { -"allowVirtualNetworkAccess": true, -"allowForwardedTraffic": false, -"allowGatewayTransit": false, -"useRemoteGateways": false, -"remoteVirtualNetwork": { -"id": "[resourceId(parameters('existingRemoteVirtualNetworkResourceGroupName'), 'Microsoft.Network/virtualNetworks', parameters('existingRemoteVirtualNetworkName'))]" } -} -}, -{ -"type": "Microsoft.Network/privateDnsZones/virtualNetworkLinks", -"apiVersion": "2020-06-01", -"name": "[format('{0}/{1}', parameters('blobPrivateDnsZoneName'), format('{0}-link', parameters('existingRemoteVirtualNetworkName')))]", -"location": "global", -"properties": { -"registrationEnabled": false, -"virtualNetwork": { -"id": "[resourceId(parameters('existingRemoteVirtualNetworkResourceGroupName'), 'Microsoft.Network/virtualNetworks', parameters('existingRemoteVirtualNetworkName'))]" -} -} -}, -{ -"type": "Microsoft.Network/privateDnsZones/virtualNetworkLinks", -"apiVersion": "2020-06-01", -"name": "[format('{0}/{1}', parameters('redisPrivateDnsZoneName'), format('{0}-link', parameters('existingRemoteVirtualNetworkName')))]", -"location": "global", -"properties": { -"registrationEnabled": false, -"virtualNetwork": { -"id": "[resourceId(parameters('existingRemoteVirtualNetworkResourceGroupName'), 'Microsoft.Network/virtualNetworks', parameters('existingRemoteVirtualNetworkName'))]" -} -} -} -] -} -} -}, -{ -"type": "Microsoft.Resources/deployments", -"resourceGroup": "[parameters('existingRemoteVirtualNetworkResourceGroupName')]", -"apiVersion": "2020-10-01", -"name": "[guid(parameters('existingRemoteVirtualNetworkName'), parameters('existingRemoteVirtualNetworkResourceGroupName'), 'peering')]", -"properties": { -"mode": "Incremental", -"template": { -"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", -"contentVersion": "1.0.0.0", -"parameters": {}, -"resources": [ -{ -"type": "Microsoft.Network/virtualNetworks/virtualNetworkPeerings", -"apiVersion": "2021-02-01", -"name": "[format('{0}/remote-to-local-vnet', parameters('existingRemoteVirtualNetworkName'))]", -"properties": { -"allowVirtualNetworkAccess": true, -"allowForwardedTraffic": false, -"allowGatewayTransit": false, -"useRemoteGateways": false, -"remoteVirtualNetwork": { -"id": "[resourceId(parameters('existingHyperExecuteVirtualNetworkResourceGroupName'), 'Microsoft.Network/virtualNetworks', parameters('existingHyperExecuteVirtualNetworkName'))]" } -} -} -] -} -} -} -] -} - -``` - -- Click on **Save** and fill out the parameters for the **Custom Deployment** template. - - **Parameters Used** - - - **Subscription :** Name of the subscription where you have installed the HyperExecute. - - - **Resource group :** Resource group name where HyperExecute is provisioned. - - - **Region :** Region where you have installed the application. - - - **Existing Hyper Execute Virtual Network Name :** Virtual Network name of the Vnet created in HyperExecute resource group - - - **Existing Hyper Execute Virtual Network Resource Group Name :** HyperExecute Resource group name - - - **Existing Remote Virtual Network Name :** Name of the Virtual Network where you want to establish Peering and Vnet Link. - - - **Existing Remote Virtual Network Resource Group Name :** Name of the resource group where you have the Virtual network which needs to be peered. - - - **Blob Private Dns Zone Name :** Private DNS zone name for the Storage Account Virtual Network Link. Please do not change this unless and until it is required. - - - **Redis Private Dns Zone Name :** Private DNS zone name for the Redis Virtual Network Link. Please do not change this unless and until it is required. - -- Click on the **Review + create** button present at the bottom of the page to navigate to the page shown below. - -- Verify all the details and click on **Create**. Your deployment will start and you will get the status of the page after the deployment is successful. - -If you have any questions about this process, you can reach out to the HyperExecute team and we will get back to you as soon as possible. - -**Note :** - -- In case of any issues in the above setup please reach out to [hyperexecute-privatecloudsupport@testmuai.com](mailto:hyperexecute-privatecloudsupport@testmuai.com) and we will get in touch with you. - -- Please make sure that you have the contributor access for your Vnet and resource group before the setup. - -- Please make sure that spaces are not present in any variable values. diff --git a/static/docs/hyperexecute-vs-traditional-test-grids.md b/static/docs/hyperexecute-vs-traditional-test-grids.md deleted file mode 100644 index d66cdf8ae..000000000 --- a/static/docs/hyperexecute-vs-traditional-test-grids.md +++ /dev/null @@ -1,44 +0,0 @@ -# HyperExecute vs Traditional Test Grids - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Traditional automation testing solutions are slow since the triggered test scenarios are first sent to the Hub, which in turn are scheduled to run on the best-suited Node. This results in unnecessary lag as a number of components are involved in the entire process. Over & above, multiple network hops with the network separated components results in increased test flakiness, a factor that can be a huge hindrance to time-to-market. - -## Traditional Tests Grids -Traditional tests refers to the conventional approach of executing software tests using a hub and node infrastructure. In this method, test scenarios are sent to a central hub, which then schedules and distributes them across multiple nodes (machines or virtual environments) for execution. The test scenarios are typically executed sequentially on the nodes, one after the other. - -Here is a diagrammatic representation of how tests are run using traditional solutions: - -### Limitations of Traditional Tests Grids - -- Slower Execution: Traditional tests often suffer from slower execution times due to the sequential nature of test case execution. - -- Test Flakiness: The separation of network components introduces instability and unreliable test results. - -- Limited Scalability: Traditional test automation environments may face challenges when it comes to scaling. Adding more nodes or expanding the infrastructure requires additional hardware or virtual resources, which can be costly and time-consuming. - -- Limited Test Parallelism: Traditional test automation methods often lack efficient parallel execution capabilities as in sequential execution, tests are unable to take full advantage of available computing resources. - -## HyperExecute - -HyperExecute integrates all test automation components into a single environment, ensuring seamless communication and maximizing efficiency. By leveraging cloud infrastructure, HyperExecute harnesses the power of distributed computing, enabling faster and more scalable test execution. HyperExecute prioritizes security by encrypting test execution logs and leveraging Azure cloud storage, ensuring data protection during transit and at rest. - -Here is the diagrammatic representation of how tests are run on the HyperExecute infrastructure: - -As seen above, tests are securely run on dedicated machines on the Azure cloud. The cloud (for execution) is chosen based on the location from where concierge (i.e. HyperExecute CLI) is triggered. - -### Features and Advantages - -- [**Auto Split Strategy**](/support/docs/hyperexecute-auto-split-strategy/): Enables parallel test execution by intelligently distributing test scenarios based on custom filters, that reduces test execution time. - -- [**Matrix Multiplexing Strategy**](/support/docs/hyperexecute-matrix-multiplexing-strategy/ ): Empowers users to execute a single command across multiple scenarios, conditions, and environments, facilitating efficient regression testing and improving code stability. - -- [**Smart Caching**](/support/docs/hyperexecute-how-smart-caching-boosts-tests-speed/) : Optimizes regression testing by intelligently caching test artifacts, speeding up the subsequent test runs. - -- Auto Purging of Environments: Automates the cleanup of test environments after build completion, maintaining a clean and optimized testing setup. - -- Fine-Grained Concurrency Control: Hyperscale of HyperExecute allows users to have fine grain control over test execution concurrency using YAML files. - -- Linux Autoscaling: HyperExecute automatically scales computing resources up or down based on workload. When test demand increases, additional resources are provisioned dynamically, and when demand decreases, resources are released, ensuring optimal performance without manual intervention. - -> For more details, please refer to our guide on [Key Features of HyperExecute](/support/docs/key-features-of-hyperexecute/) diff --git a/static/docs/hyperexecute-webapp-appium-testing.md b/static/docs/hyperexecute-webapp-appium-testing.md deleted file mode 100644 index a989f712c..000000000 --- a/static/docs/hyperexecute-webapp-appium-testing.md +++ /dev/null @@ -1,325 +0,0 @@ -# Appium Testing On HyperExecute - WebApp - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This page outlines how to execute your Appium tests on HyperExecute for WebApps using TestNG with YAML 0.2 -> HyperExecute uses [YAML 0.2](/support/docs/hyperexecute-yaml-version0.2/) to perform the tests using Appium. - -## Prerequisites - -To run the Tests on HyperExecute from your Local System, you are required: - -- Your TestMu AI [Username and Access key](/support/docs/hyperexecute-how-to-get-my-username-and-access-key/) -- [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) in order to initiate a test execution Job . -- Setup the Environmental Variable -- Ensure you have Appium’s [Java client library](https://github.com/appium/java-client) installed. -- [HyperExecute YAML](/support/docs/hyperexecute-yaml-version0.2/) file which contains all the necessary instructions. - -### Download HyperExecute CLI - -The *HyperExecute CLI* is used for triggering tests on HyperExecute. It is recommend to download the HyperExecute CLI binary on the host system to perform the tests on HyperExecute. The CLI download site for various platforms is displayed below: - -| Platform | HyperExecute CLI download location | -| ---------| --------------------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| macOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -### Setup Environment Variable -Export the environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in the [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). -Run the below mentioned commands in the terminal to setup the CLI and the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -### Steps to Run Your Test -**Step 1:** Write your Automation Script in the client language of your choice from the ones supported by [Appium](/support/docs/appium-languages-and-frameworks/). An automation script for the sample applications have been provided below. - -Here is a sample automation script in Java. Ensure to update the `username` and `accesskey` in the below code. - -```java -package com.lambdatest; - -import io.appium.java_client.MobileBy; -import org.junit.After; -import org.junit.Before; -import org.junit.Test; -import org.openqa.selenium.By; -import org.openqa.selenium.remote.DesiredCapabilities; -import org.openqa.selenium.remote.RemoteWebDriver; -import org.openqa.selenium.support.ui.ExpectedConditions; -import org.openqa.selenium.support.ui.WebDriverWait; - -import java.net.MalformedURLException; -import java.net.URL; -import java.util.concurrent.TimeUnit; - -public class androidWeb { - -String username = System.getenv("LT_USERNAME") == null ? "LT_USERNAME" //Enter the Username here -: System.getenv("LT_USERNAME"); -String accessKey = System.getenv("LT_ACCESS_KEY") == null ? "LT_ACCESS_KEY" //Enter the Access key here -: System.getenv("LT_ACCESS_KEY"); -public static RemoteWebDriver driver = null; -public String gridURL = "@mobile-hub.lambdatest.com/wd/hub"; -public String status = "passed"; -@Before -public void setUp() throws Exception { -DesiredCapabilities capabilities = new DesiredCapabilities(); - -capabilities.setCapability("build", "HYP Web RD Demo"); -capabilities.setCapability("name", "Java Android Web Test"); -capabilities.setCapability("platformName", "android"); -capabilities.setCapability("deviceName", "Galaxy. *,OnePlus. *,Pixel. *"); //Enter the name of the device here -capabilities.setCapability("isRealMobile", true); -capabilities.setCapability("region", "eu"); -// capabilities.setCapability("platformVersion","9"); -capabilities.setCapability("deviceOrientation", "portrait"); -capabilities.setCapability("console",true); -capabilities.setCapability("network",true); -capabilities.setCapability("visual",true); -try -{ -driver = new RemoteWebDriver(new URL("https://" + username + ":" + accessKey + gridURL), capabilities); -} -catch (MalformedURLException e) -{ -System.out.println("Invalid grid URL"); -} catch (Exception e) -{ -System.out.println(e.getMessage()); -} -} - -@Test -public void testSimple() throws Exception -{ -try -{ -driver.get("https://lambdatest.github.io/sample-todo-app/"); -driver.findElement(By.name("li1")).click(); - -System.out.println("Checking Another Box"); -driver.findElement(By.name("li2")).click(); - -System.out.println("Checking Box"); -driver.findElement(By.name("li3")).click(); - -System.out.println("Checking Another Box"); -driver.findElement(By.name("li4")).click(); - -driver.findElement(By.id("sampletodotext")).sendKeys(" List Item 6"); -driver.findElement(By.id("addbutton")).click(); - -driver.findElement(By.id("sampletodotext")).sendKeys(" List Item 7"); -driver.findElement(By.id("addbutton")).click(); - -driver.findElement(By.id("sampletodotext")).sendKeys(" List Item 8"); -driver.findElement(By.id("addbutton")).click(); - -System.out.println("Checking Another Box"); -driver.findElement(By.name("li1")).click(); - -System.out.println("Checking Another Box"); -driver.findElement(By.name("li3")).click(); - -status="passed"; -} -catch (Exception e) -{ -System.out.println(e.getMessage()); -status="failed"; -} -} -@After -public void tearDown() throws Exception -{ -if (driver != null) -{ -driver.executeScript("lambda-status=" + status); -driver.quit(); -} -} - -} -``` - -```java -package com.lambdatest; - -import io.appium.java_client.MobileBy; -import org.junit.After; -import org.junit.Before; -import org.junit.Test; -import org.openqa.selenium.By; -import org.openqa.selenium.remote.DesiredCapabilities; -import org.openqa.selenium.remote.RemoteWebDriver; -import org.openqa.selenium.support.ui.ExpectedConditions; -import org.openqa.selenium.support.ui.WebDriverWait; - -import java.net.MalformedURLException; -import java.net.URL; -import java.util.concurrent.TimeUnit; - -public class iosWeb { - -String username = System.getenv("LT_USERNAME") == null ? "LT_USERNAME" //Enter the Username here -: System.getenv("LT_USERNAME"); -String accessKey = System.getenv("LT_ACCESS_KEY") == null ? "LT_ACCESS_KEY" //Enter the Access key here -: System.getenv("LT_ACCESS_KEY"); -public static RemoteWebDriver driver = null; -public String gridURL = "@hub.lambdatest.com/wd/hub"; -public String status = "passed"; -@Before -public void setUp() throws Exception { -DesiredCapabilities capabilities = new DesiredCapabilities(); - -capabilities.setCapability("build", "HYP Web RD Demo"); -capabilities.setCapability("name", "Java JUnit iOS Web Test"); -capabilities.setCapability("platformName", "ios"); -capabilities.setCapability("deviceName", "iPhone.*"); -capabilities.setCapability("isRealMobile", true); -// capabilities.setCapability("platformVersion","14"); -capabilities.setCapability("deviceOrientation", "portrait"); -capabilities.setCapability("console",true); -capabilities.setCapability("network",true); -capabilities.setCapability("visual",true); - -try -{ -driver = new RemoteWebDriver(new URL("https://" + username + ":" + accessKey + gridURL), capabilities); -} -catch (MalformedURLException e) -{ -System.out.println("Invalid grid URL"); -} catch (Exception e) -{ -System.out.println(e.getMessage()); -} -} - -@Test -public void testSimple() throws Exception -{ -try -{ -driver.manage().timeouts().setScriptTimeout(10, TimeUnit.SECONDS); -driver.get("https://lambdatest.github.io/sample-todo-app/"); -driver.findElement(By.name("li1")).click(); - -System.out.println("Checking Another Box"); -driver.findElement(By.name("li2")).click(); - -System.out.println("Checking Box"); -driver.findElement(By.name("li3")).click(); - -System.out.println("Checking Another Box"); -driver.findElement(By.name("li4")).click(); - -driver.findElement(By.id("sampletodotext")).sendKeys(" List Item 6"); -driver.findElement(By.id("addbutton")).click(); - -driver.findElement(By.id("sampletodotext")).sendKeys(" List Item 7"); -driver.findElement(By.id("addbutton")).click(); - -driver.findElement(By.id("sampletodotext")).sendKeys(" List Item 8"); -driver.findElement(By.id("addbutton")).click(); - -System.out.println("Checking Another Box"); -driver.findElement(By.name("li1")).click(); - -System.out.println("Checking Another Box"); -driver.findElement(By.name("li3")).click(); -status="passed"; -} -catch (Exception e) -{ -System.out.println(e.getMessage()); -status="failed"; -} -} -@After -public void tearDown() throws Exception -{ -if (driver != null) -{ -driver.executeScript("lambda-status=" + status); -driver.quit(); -} -} - -} -``` - -**Step 2:** Execute Your Test Case. Debug and run your code. - -Once you have run your tests, you can view the test execution along with logs. You will be able to see the test cases passing or failing. You can view the same at TestMu AI Automation. - -## Sample YAML 0.2 - -```bash -version: "0.1" -runson: win - -autosplit: true - -concurrency: 2 - -testDiscovery: -command: cat tests.txt -mode: static -type: raw - -testRunnerCommand: mvn test -P $test - -framework: -name: appium -``` - -> The ```region``` parameter specifies the region or location where the Appium tests will be executed. Our platform supports the following three regions: -- ap (Asia-Pacific) -- us (United States) -- eu (European Union) - -## More About Desired Capabilities -Sample Capabilities for both android and iOS web app mentioned below - - -```java -{ -"deviceName": "Galaxy Tab S4", -"platformName": "android", -"platformVersion": "10", -"visual": True, -"console": True, -"deviceOrientation": "PORTRAIT", -"build": "new-12", -"isRealMobile": True, -} -``` - -```java -{ -"deviceName": "iPhone 12 Mini", -"platformName": "ios", -"platformVersion": "14", -"isRealMobile": True, -"visual": True, -"console": True, -"build": "lt-web-4", -"network": True, -} -``` - -> Refer to the [Automation Capabilities Generator](https://www.testmuai.com/capabilities-generator/) to understand more about it. - -## Navigation in Automation Dashboard - -Every test run on the HyperExecute has a unique *jobId* associated with it. Each *jobId* can in turn constitute single (or multiple) *groupId*(s). You can visit [HyperExecute automation dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/hyperexecute/) for checking the status of the test execution. - -The snapshot below shows how to navigate to the respective *testID* for viewing the Selenium logs: - -## Conclusion -By following the instructions in this documentation, you can seamlessly execute the Appium tests on HyperExecute for WebApps, leveraging its secure cloud infrastructure, advanced features, and optimized test execution workflow. diff --git a/static/docs/hyperexecute-widgets.md b/static/docs/hyperexecute-widgets.md deleted file mode 100644 index cd0233693..000000000 --- a/static/docs/hyperexecute-widgets.md +++ /dev/null @@ -1,52 +0,0 @@ -# HyperExecute Widgets - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The HyperExecute Widgets enables you to get all the insights of the Jobs executed in one place. You can easily find information ranging from the count of the Jobs, Stages, Tasks, and Tests run on the platform and their insights. - - Job Widgets - Learn the insights of your triggered Jobs on HyperExecute Platform. - - Stage Widgets - Learn the insights of the Stages of your Jobs on HyperExecute Platform. - - Task Widgets - Learn the insights of the Tasks on HyperExecute Platform. - - Test Widgets - Learn the insights of your Tasks based on the defined parameters in the Tests. - - Resource Utilization Widgets - Learn about the HyperExecute Resource Utilization Widgets. - -## Create Your Widgets Dashboard using Pre-built Template - -To create your widgets dashboard, follow the below mentioned steps: - -**Step 1:** Go to the Insights section from the TestMu AI Dashboard. - -**Step 2:** Click on the **Create New** button, then select the **Pre-built Templates** option. - -**Step 3:** Select HyperExecute Report - -**Step 4:** Enter your Dashboard name and then click on **Create Dashboard** button. - -## Create Your Custom Widgets Dashboard - -To create your widgets dashboard, follow the below mentioned steps: - -**Step 1:** Go to the Insights from the TestMu AI Dashboard. - -**Step 2:** Click on the **Create New** button, then select the **Custom Widgets** option. - -**Step 3:** Now customize your Widget Dashboard - -- Select HyperExecute - -- Give name to your Dashboard - -- Select your desired widgets that you want in your dashboard. - -> Note: You can re-customize the same dashboard by adding more widgets later. - -- Click on **Create Dashboard** button. diff --git a/static/docs/hyperexecute-winapp-integration.md b/static/docs/hyperexecute-winapp-integration.md deleted file mode 100644 index 958302f1a..000000000 --- a/static/docs/hyperexecute-winapp-integration.md +++ /dev/null @@ -1,176 +0,0 @@ -# WinAppDriver Integration with HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -You can use HyperExecute run your tests that are based on Microsoft Office Suite. To do this, we use the TestNG framework. If you need some help with creating a TestNG project, you can go through [this page](https://www.testmuai.com/blog/create-testng-project-in-eclipse-run-selenium-test-script/). - -You can go through the page below to understand how you can use HyperExecute to run your test cases. - -> **Note**: This feature is only available on request. Please contact support@testmuai.com to know more. - -## Configuring the WinAppDriver - -Setting up the WinAppDriver is an important step in the process to run Microsoft Excel via HyperExecute. - -1. Create a function called `start` to initialise the WinAppDriver, and a function called `closeExcel` to terminate Microsoft Excel after the test cases are executed. - -2. Place a Java test file `LtWinDriver.java` that contains the two functions in the `src/main` folder. -A sample `LtWinDriver.java` file: - -```bash -import java.io.File; -import java.io.IOException; -import java.awt.Desktop; - -public class LtWinDriver { -public static void start() { -try { -Desktop d = Desktop.getDesktop(); -d.open(new File("C:\\Program Files (x86)\\Windows Application Driver\\WinAppDriver.exe")); -//d.open(new File("drivers/WinAppDriver.exe")); -} catch (IOException e) { -e.printStackTrace(); -throw new RuntimeException(e); -} -} - -public static void stopExcel() { -try { -ProcessBuilder killexcel =new ProcessBuilder("taskkill ","/f","/IM","Excel.exe"); -killexcel.start(); -} catch (IOException e) { -e.printStackTrace(); -throw new RuntimeException(e); -} -} -} -``` - -## Writing the Test Cases - -You can write your own test cases on Microsoft Excel. However, an example of a test case can be to add two numbers in different cells on Excel. - -You can use the sum formula to add these two numbers, but if you want to run it through HyperExecute, you need to initialize the WinAppDriver. This can be accomplished by calling the start function that we created in the `LtWinDriver.java` file. - -1. Call the `start` function in the `@BeforeClass` tag. - -2. Write your test cases in the `@Test` tag. - -3. Terminate Excel in the `@AfterClass` tag by calling the `stopExcel` function that was created earlier. - -Here is a sample java test file `Test.java` that accomplishes the function mentioned above. All of your test files should be placed in the `src/test` folder. - -```bash -import java.util.concurrent.TimeUnit; -import io.appium.java_client.windows.WindowsDriver; -import org.openqa.selenium.*; -import org.openqa.selenium.remote.DesiredCapabilities; -import org.testng.annotations.*; -import java.net.*; -import java.util.*; - -public class Test1 { -    public static WindowsDriver driver = null; -    -    @BeforeClass -    public void setUp() throws Exception { -        LtWinDriver.start(); -    } -    -    @Test -    public static void test() throws Exception { -        String appPath = "testing.xlsx"; -        DesiredCapabilities cap = new DesiredCapabilities(); -        cap.setCapability("app", appPath); -        cap.setCapability("ms:waitForAppLaunch", 30); -        cap.setCapability("appArguments", "/e "); -        cap.setCapability("ms:experimental-webdriver", true); - -        try { -            // Block of code to try -            driver = new WindowsDriver(new URL("http://127.0.0.1:4723"), cap); -        } catch (Exception e) { -            // Block of code to handle errors -            System.out.println("Something went wrong."); -            driver = new WindowsDriver(new URL("http://127.0.0.1:4723"), cap); -        } -        -        driver.manage().timeouts().implicitlyWait(5, TimeUnit.SECONDS); -        -        try { -            driver.findElement(By.name("Not now")).click(); -        } catch (Exception e) { -            System.out.println(e); -        } - -        driver.findElement(By.name("B2")).click(); -        driver.findElement(By.name("Formula Bar")).sendKeys("10"); -        driver.findElement(By.name("B3")).click(); -        driver.findElement(By.name("Formula Bar")).sendKeys("20"); -        driver.findElement(By.name("B4")).click(); -        driver.findElement(By.name("Formula Bar")).sendKeys("=SUM(B2,B3)"); -        driver.findElement(By.name("Formula Bar")).sendKeys(Keys.ENTER); -    } - -    @AfterClass -    public void tearDown() throws Exception { -       if (driver != null) { -            LtWinDriver.stopExcel(); -            driver.quit(); -        } -    } -} -``` - -- To ensure that your excel file opens and runs successfully, use the `ms:waitForAppLaunch` flag. This ensure that there Excel waits for a duration of time before starting to run your test case. You can set the value of this flag to 30 seconds for better results. - -- You can use the `ms:experimental-webdriver` flag and set it to `true` to help the driver locate elements [more efficiently](https://github.com/microsoft/WinAppDriver/releases/tag/v1.2-RC "https://github.com/microsoft/WinAppDriver/releases/tag/v1.2-RC"). - -- Using `try` and `catch` is a good way to avoid any pop-ups that may affect the test case. - -- You can also use `implicitlyWait` in order to wait before throwing any exceptions. - -## HyperExecute YAML file - -A sample YAML file that can run the test scenario mentioned above is shown below. You can learn more about configuring your YAML file by going through [this page](/support/docs/deep-dive-into-hyperexecute-yaml/). - -```yaml ---- -version: 0.1 -globalTimeout: 150 -testSuiteTimeout: 150 -testSuiteStep: 150 - -runson: win -captureScreenRecordingForScenarios: true - -autosplit: true -retryOnFailure: true - -maxRetries: 1 -concurrency: 2 - -betaFeature: -setPartiallyCompletedAsCompleted: true - -env: -CACHE_DIR: m2_cache_dir - -cacheKey: '{{ checksum "pom.xml" }}' -cacheDirectories: -- .m2 - -pre: -- mvn -Dmaven.repo.local=$CACHE_DIR -Dmaven.test.skip=true clean install - -mergeArtifacts: true - -testDiscovery: -type: raw -mode: remote -command: grep 'public class' src/test/java/*.java | awk '{print$3}' - -testRunnerCommand: mvn `-Dplatname=win `-Dmaven.repo.local=m2_cache_dir `-Dtest=$test test -``` - -You can now run your Excel tests on HyperExecute. Happy testing! diff --git a/static/docs/hyperexecute-workflows.md b/static/docs/hyperexecute-workflows.md deleted file mode 100644 index 523f96d35..000000000 --- a/static/docs/hyperexecute-workflows.md +++ /dev/null @@ -1,101 +0,0 @@ -# Workflows in HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Workflows give you the speed, reliability, and platform‑native orchestration of HyperExecute on a schedule. Think of Workflows as a lightweight, cloud-native alternative and companion to CI/CD, with all the functionalities of HyperExecute such as reporting, RCA etc. - -## Why Workflows -- **Schedule-first orchestration:** Run regular suites by just connecting your Git repo, without wiring a CI pipeline. -- **Native to HyperExecute:** Leverage JIT infrastructure, smart test discovery/splitting, and rich artifacts in one place. -- **Chainable runs:** Trigger downstream workflows on success to form full test chains across projects. -- **Customizable:** Use Workflow Variables to reuse the same YAML across environments and scenarios. - -## How Workflows are Used -Workflows are schedule and platform driven, not commit-driven yet. Workflows can be thought of as UI alternative for CLI-based executions, if your project is available on Git. They’re great for: -- Nightly smoke/regression runs across browsers/devices. -- Scheduled environment checks (staging, pre-prod) before business hours. -- Chaining suites across multiple repositories or projects. - -## Prerequisites -- A [HyperExecute Project](/support/docs/hyperexecute-projects/) connected to **GitHub** or **Bitbucket**. -- A HyperExecute YAML present on the specified branch, or provide a Custom YAML during setup. New to YAML? See [Generate YAML](/support/docs/hyperexecute-generate-sample-yaml/) and the [Deep Dive](/support/docs/deep-dive-into-hyperexecute-yaml/). -- Note: Workflows currently work with the [remote mode of test discovery](/support/docs/deep-dive-into-hyperexecute-yaml/#testdiscovery). - -## Create a Workflow -1) In your Project, click **Setup Workflow**. -2) Enter details: - - **Workflow name** - - **Branch name** (source branch containing the YAML) - - **YAML file path** (path in repo) or add Custom YAML -3) (Optional) Configure the **schedule** or skip it if you want to run workflow manually. See [Schedule Configuration](#schedule-configuration) for details. -4) (Optional) You can **link your workflow** by selecting existing workflows to trigger automatically after this workflow completes successfully. -5) (Optional) You may also define **Workflow Variables** which can be used in the YAML to customize behavior at runtime for a specific workflow. -6) Click **Finish** and your workflow is ready. Click on Play button in the workflow list to run the job manually. - -All jobs appear under the same Project. View jobs from the Project’s **Jobs** tab or by clicking the workflow name to open **Past Jobs** view. - -If you don’t see a job after clicking Run or after the scheduled time, open the workflow and check **Past Occurrences**. Runs that failed to create a job (e.g., due to YAML validation, missing file path, or branch mismatch) are listed there with error details. - -## Schedule Configuration - -During workflow creation (Step 2 of 3), you can configure the **Triggering Schedule Details** to automate when and how often your workflow runs. - -### Trigger Frequency - -You can choose from three trigger options: - -- **Once**: The workflow runs automatically as soon as it is created. No manual execution needed. -- **Later**: Schedule the workflow to run at a specific date and time. -- **Every**: Set up recurring execution by selecting specific days of the week (Mon–Sun) and a time. For example, you can schedule a workflow to run every Monday, Wednesday, and Friday at 9:00 AM. - -The schedule is set in your **local timezone** and stored internally in UTC. This ensures workflows trigger at the correct time regardless of where team members are located. - -### Workflow End Date (Expiry) - -Every scheduled workflow requires a **Workflow ends on** date. After this date, the workflow will stop executing automatically. This ensures workflows do not run indefinitely and helps manage resource usage. - -You can update the schedule and end date at any time by editing the workflow from the actions menu. - -## Manage Workflows - -Once a workflow is created, you can manage it from the workflow list within your Project. Each workflow row displays the schedule, last execution status, and the last 5 jobs. - -### Retrigger a Workflow - -Click the **Play** button (▶) next to a workflow to manually trigger it. This re-runs the workflow using the same configuration (branch, YAML, and variables). You can also edit variable values at runtime before triggering. - -### Workflow Actions Menu - -Click the **three dots menu** (⋯) on any workflow to access the following actions: - -- **Edit Workflow**: Modify the workflow name, branch, YAML path, schedule, linked workflows, or variables. -- **Clone Workflow**: Duplicate an existing workflow configuration. This creates a copy with the same settings, which you can then modify as needed. -- **Delete Workflow**: Permanently remove a workflow. This does not delete any jobs that were previously triggered by the workflow. - -## Connected Workflows (Test Chains) -Use Workflow Linking to form test chains. For example, run a daily smoke suite first and trigger a broader regression only if smoke passes. You can link across projects you own to coordinate multi-repo testing. - -- Learn the pattern with real examples: [Test Chains](/support/docs/hyperexecute-test-chains/) - -## Workflow Variables -Variables let you customize job runs without the need of creating multiple workflows with different YAML configurations for the same project or editing YAML before every run. To set workflow variables for a single workflow, you need to follow the following steps: -1. Create or edit a workflow. On Step 3, turn on **Workflow Variables** and add these keys (`key_1`, `key_2` and `key_3`) in the **Key** column. -2. For the values of the corresponding keys, you can: - - **Add single value**: This value will be considered default when workflow runs through schedule. While manually running you can edit the value at runtime too. - - **Add multiple values**: Among these, the first value will be considered default while running the workflow through schedule. At runtime, you will be able to select any value from all these. - - **Do not add any value**: You can keep the value of a key empty too and fill it later at runtime. The value will be passed as `null` if you do not add any value at runtime. -3. Once you have added all the keys and their values, save the workflow. -4. You have the flexibility to edit the values for specific keys at the runtime, whenever you are running the workflow manually. - -> Please note that the variables added from the workflows dashboard will be automatically added to the YAML file. - -**Maximum Limits:** -- Up to **100 variables per workflow**. -- Up to **25 values per variable**. -- Value length limit: **500 characters** (excluding commas/spaces). - -## FAQ -- **Which repos are supported for Projects?** Currently **GitHub** and **Bitbucket**. -- **Is remote test discovery required?** Yes, Workflows currently work with the [remote mode of test discovery](/support/docs/deep-dive-into-hyperexecute-yaml/#testdiscovery). -- **Can I trigger other workflows automatically?** Yes, via Workflow Linking to build [Test Chains](/support/docs/hyperexecute-test-chains/). -- **Among runtime and workflow variables, what takes precedence?** Runtime value > Workflow‑level defaults. diff --git a/static/docs/hyperexecute-xcui-testing.md b/static/docs/hyperexecute-xcui-testing.md deleted file mode 100644 index 9fa35d55b..000000000 --- a/static/docs/hyperexecute-xcui-testing.md +++ /dev/null @@ -1,178 +0,0 @@ -# Execute XCUI Framework Tests on HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This page outlines how to execute your XCUI tests on HyperExecute with [YAML 0.2](/support/docs/hyperexecute-yaml-version0.2/) - -## Prerequisites - -To run the Tests on HyperExecute from your Local System, you are required: - -- Your TestMu AI [Username and Access key](/support/docs/hyperexecute-how-to-get-my-username-and-access-key/) -- [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) in order to initiate a test execution Job . -- Setup the [Environmental Variable](/support/docs/hyperexecute-environment-variable-setup/) -- [HyperExecute YAML](/support/docs/hyperexecute-yaml-version0.2/) file which contains all the necessary instructions. - -## Step 1: Setup Your Test Suite - -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** - -Download or Clone the code sample for the XCUI framework from the TestMu AI GitHub repository to run the tests on the HyperExecute. - - View on GitHub - -## Step 2: Setup the CLI in your Test Suite - -After cloning / downloading the sample repo, you need to setup the CLI and the environment variables. - -### Download the HyperExecute CLI - -The CLI is used for triggering the tests on HyperExecute. It is recommend to download the CLI binary on the host system and keep it in the root directory of the suite to perform the tests on HyperExecute. - -You can download the CLI for your desired platform from the below mentioned links: - -| Platform | HyperExecute CLI | -| ---------| ---------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| MacOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -### Setup Environment Variable - -Now, you need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in the [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). - -Run the below mentioned commands in your terminal to setup the CLI and the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -## Step 3: Upload your Application and Test Suite - -### Upload your App - -Upload your **iOS** application (.ipa file) to the TestMu AI servers using our **REST API**. You need to provide your **Username** and **AccessKey** in the format ```Username:AccessKey``` in the **cURL** command for authentication. Make sure to add the path of the **appFile** in the cURL request. - -Here is an example cURL request to upload your app using our REST API: - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" \\ ---location --request POST 'https://manual-api.lambdatest.com/app/uploadFramework' \\ ---form 'appFile=@"/Users/macuser/Downloads/proverbial.ipa"' \\ ---form 'type="xcuit-ios"'`} - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/uploadFramework" -F "appFile=@"/Users/macuser/Downloads/proverbial.ipa"" -F "type="xcuit-ios""`} - -Response of above cURL will be a **JSON** object containing the `App URL` of the format - `APP123456789123456789` and will be used in the last step as `appId`. - -### Upload your Test Suite - -Upload your **test suite** (.ipa file) to the TestMu AI servers using our **REST API**. You need to provide your **Username** and **AccessKey** in the format `Username:AccessKey` in the **cURL** command for authentication. Make sure to add the path of the **appFile** in the cURL request. Here is an example cURL request to upload your app using our REST API: - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" \\ ---location --request POST 'https://manual-api.lambdatest.com/app/uploadFramework' \ ---form 'appFile=@"/Users/macuser/Downloads/proverbial_ios_xcuitest.ipa"' \ ---form 'type="xcuit-ios"'`} - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" --location --request POST "https://manual-api.lambdatest.com/app/uploadFramework" --form "appFile=@"C:/Users/varunkumarb/Downloads/proverbial_ios_xcuitest.ipa"" --form "type=\"xcuit-ios\""`} - -Response of above cURL will be a **JSON** object containing the `App URL` of the format - `APP123456789123456789` and will be used in the next step as ``testSuiteAppId`` - -## Step 4: Configure YAML in your Test Suite - -Enter your `` and `` in the YAML file - -```yaml ---- -version: "0.2" - -runson: ios - -autosplit: true - -concurrency: 1 - -retryOnFailure: true -maxRetries: 2 - -framework: -name: "ios/xcui" -args: -# highlight-next-line -appPath: -# highlight-next-line -testSuitePath: -video: true -filters: -attributes: -- type: className -values: ["LambdaUiKitIOSUITestsLaunchTests"] -- type: testName -values: ["LambdaUiKitIOSUITests/testverifyAppLaunch"] -deviceSelectionStrategy: any -devices: [".*"] -artifacts: true - -shards: -mappings: -- name: shard1 -strategy: "only-testing/skip-testing" -values: ["/"] -- name: shard2 -strategy: "only-testing/skip-testing" -values: ["/", "/"] - -jobLabel: ['HyperExecute', 'XCUI', 'Real Device'] -``` - -**When shards are added** - -If you are using the `deviceSelectionStrategy: all`, then in that case all the mentioned shards will be executed on all the devices. -**For example:** There are 2 shards and 2 devices mentioned, then the 2 shards will be executed on 2 devices (2 Shards * 2 Devices). - -If you are using the `deviceSelectionStrategy: any`, then in that case all the mentioned shards will be executed on any one device from the list provided. -**For example:** There are 2 shards and 2 devices mentioned, then the 2 shards will be executed on any device (2 shards * any device mentioned). - -**When shards aren't added** - -If you are using the `deviceSelectionStrategy: all`, then in that case all the tests will be executed on all the devices based on the concurrency. -**For example:** There are 6 tests and 10 devices mentioned, then the 6 tests will be auto distributed on the devices based on the max concurrency. - -If you are using the `deviceSelectionStrategy: any`, then in that case all the mentioned tests will be executed on any one device from the list provided. -**For example:** There are 6 tests and 10 devices mentioned, then the 6 tests will be executed on any one device. - -## Step 5: Generate Reports and Artifacts -To generate artifacts for your XCUI tests, add the `artifacts: true` and `report: true` flag in your YAML file: - -```yaml -report: true - -framework: -name: "ios/xcui" -args: -artifacts: true -...// -``` - -To download these artifacts in your local machine, you can pass the `--download-artifacts` and `--download-report` flag with the CLI command to execute the tests as shown in the next step. - -## Step 6: Execute your Test Suite - -> **NOTE :** In case of macOS, if you get a permission denied warning while executing CLI, simply run **`chmod u+x ./hyperexecute`** to allow permission. In case you get a security popup, allow it from your **System Preferences** → **Security & Privacy** → **General tab**. - -Run the below command in your terminal at the root folder of the project: - -```bash -./hyperexecute --config RELATIVE_PATH_OF_YOUR_YAML_FILE --download-artifacts --download-report -``` - -OR use this command if you have not exported your username and access key in the step 2. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE --download-artifacts --download-report`} - -## Step 7: Monitor the Test Execution - -Visit the [HyperExecute Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) and check your Job status. diff --git a/static/docs/hyperexecute-yaml-capability-overrides.md b/static/docs/hyperexecute-yaml-capability-overrides.md deleted file mode 100644 index 88947429f..000000000 --- a/static/docs/hyperexecute-yaml-capability-overrides.md +++ /dev/null @@ -1,46 +0,0 @@ -# YAML-Based Capability Overrides for Selenium Tests on HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -In Selenium-based testing, [capabilities](/support/docs/selenium-automation-capabilities/) are key-value pairs that define how and where your tests should run, such as browser type, version, OS, console logs, video recording, and more. - -Previously, these capabilities were typically defined within the test script. With this new feature, you can now override or inject these directly from your **HyperExecute YAML** file. This gives you greater flexibility and eliminates the need to modify test scripts every time you change execution parameters. - -## 🚀 Why Use YAML-Based Overrides? -| Benefit | Description | -| ---------------------------- | -------------------------------------------------------------------------- | -| No Code Changes Required | Easily change capabilities without modifying your test script. | -| Reusable & Configurable | Use a single script across different environments with varying YAML files. | -| Cleaner Test Scripts | Keeps your scripts capability-agnostic and environment-independent. | - -## ⚙️ How It Works -- Define a new `ltOptions` section inside your HyperExecute YAML file. -- Specify any desired capabilities as key-value pairs under `ltOptions`. -- During test execution, HyperExecute automatically merges these with the final capabilities passed to the Selenium session. -> All the capabilities defined under the `ltOptions` section are fully aligned with TestMu AI's standard capability structure. You can use any capability listed in our [TestMu AI Capabilities Generator](https://www.testmuai.com/capabilities-generator/). - -## 🔑 Key Behavior Rules - -| Case | Behavior | -| --------------------------------------------------- | ---------------------------------------------------------- | -| Capability present in **both test script and YAML** | The **YAML value takes precedence**. | -| Capability present in **YAML but not in script** | It is **automatically added** to the session capabilities. | -| Capability only in script | Used as-is unless overridden in YAML. | - -> **📝 Note:** This feature is only supported for **Selenium**-based tests at the moment. - -## Sample YAML with Capability Overrides - -```yaml reference title="hyperexecute.yaml" -https://github.com/LambdaTest/testng-selenium-hyperexecute-sample/blob/main/yaml/linux/v1/testng_hyperexecute_linux_ltoptions.yaml -``` - -## 💡 Common Use Cases - -| Use Case | Example Capability | -| ----------------------------- | ---------------------------- | -| Changing browser type/version | `browserName`, `version` | -| Controlling visual artifacts | `video`, `console`, `visual` | -| Defining test metadata | `build`, `name` | -| Network capture and debugging | `network`, `tunnel` | -| Switching Selenium versions | `selenium_version` | diff --git a/static/docs/hyperexecute-yaml-creation-for-playwright.md b/static/docs/hyperexecute-yaml-creation-for-playwright.md deleted file mode 100644 index f51023961..000000000 --- a/static/docs/hyperexecute-yaml-creation-for-playwright.md +++ /dev/null @@ -1,280 +0,0 @@ -# HyperExecute YAML Creation for Playwright - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This guide outlines common use cases and solutions for configuring Playwright test executions on **TestMu AI HyperExecute**. It covers dependency management, environment setup, caching, reporting, and troubleshooting common issues. - -## Q: How can private dependencies be accessed via a custom registry? -If your project uses private dependencies hosted on a custom registry, you must configure access before installing the packages. Add the following commands in the `pre` section of your YAML file: - -```yaml title="hyperexecute.yaml" -pre: -- npm config set registry -- npm install -``` - -## Q: How can a specific Playwright project be executed? -When a project has multiple Playwright projects defined, it may be necessary to run a specific one. Using the `--project` flag ensures that only the intended project executes without affecting others. Check that your execution command includes `--project="PROJECTNAME"`and append it to the `testRunnerCommand`. - -```javascript -npx playwright test --project=chromium -``` - -## Q: How can tests be executed with a specific configuration file? -For setups with multiple configuration files, specifying the correct config file during execution ensures that the intended environment and settings are applied, avoiding conflicts or unexpected behavior. - -```javascript -npx playwright test --config=playwright.config.staging.ts -``` - -## Q: How can skipped tests be ignored during test discovery? -To ignore test cases marked with test.skip, create a custom Node.js script. - -- Create a file named `discovery.js` and add the script from the [Gist](https://gist.github.com/mohitsaini28r/453368e52143fa43efa271b1511aa2e7). -- Update the `testDiscovery` block in your YAML: - -```yaml title="hyperexecute.yaml" -testDiscovery: -command: node discovery.js -``` - -## Q: How can private dependencies be accessed through a private network proxy? -When private dependencies require access through a private network, configure HTTP and HTTPS proxies. - -**For npm:** - -```yaml title="hyperexecute.yaml" -pre: -- npm config set proxy http://${LT_PROXY_HOST}:${LT_PROXY_PORT} -- npm config set https-proxy http://${LT_PROXY_HOST}:${LT_PROXY_PORT} -``` - -**For yarn:** - -```yaml title="hyperexecute.yaml" -pre: -- yarn config set proxy http://${LT_PROXY_HOST}:${LT_PROXY_PORT} -- yarn config set https-proxy http://${LT_PROXY_HOST}:${LT_PROXY_PORT} -``` - -## Q: How can scripts be run on each machine after test execution? -Use the `post` parameter in the YAML file. Typical use cases include: - -- Running cleanup scripts -- Closing API connections -- Uploading test results to tools like Report Portal or Zephyr - -```yaml title="hyperexecute.yaml" -post: -- ./scripts/cleanup.sh -- ./scripts/upload-results.sh -``` - -## Q: How can tasks be executed after all test executions are complete? -Use the `globalPost` parameter to execute tasks once all tests have finished. You can run it on local or remote machines and configure caching if needed. - -Common use cases: -- Merging reports -- Sending email notifications -- Posting summaries to APIs or services - -```yaml title="hyperexecute.yaml" -globalPost: -- ./scripts/merge-reports.sh -- ./scripts/send-summary.sh -``` - -## Q: How can scripts be executed before all test executions start? -Use the `globalPre` parameter to prepare environments or generate config files. - -Examples: -- Generate runtime files -- Import data -- Run preparatory commands - -```yaml title="hyperexecute.yaml" -globalPre: -- ./scripts/setup-env.sh -- ./scripts/import-data.sh -``` - -## Q: How can smart caching be enabled in HyperExecute? -Caching dependencies improves efficiency by avoiding repeated installations. Using `cacheKey` and `cacheDirectories` in the YAML file enables caching for npm or Yarn, ensuring that dependencies are reused across executions. - -**For npm:** - -```yaml title="hyperexecute.yaml" -cacheKey: '{{ checksum "package-lock.json" }}' -cacheDirectories: -- node_modules -``` - -**For yarn:** - -```yaml title="hyperexecute.yaml" -cacheKey: '{{ checksum "yarn.lock" }}' -cacheDirectories: -- node_modules -``` - -## Q: What is `testDiscovery` and how can it be configured? -`testDiscovery` identifies test files or methods to execute. It supports `raw` type with `static` or `remote` modes. - -**File-level discovery:** - -```yaml title="hyperexecute.yaml" -testDiscovery: -type: raw -mode: static -command: grep -lr 'describe' tests -``` - -**Test-level discovery:** - -```yaml title="hyperexecute.yaml" -testDiscovery: -type: raw -mode: remote -command: grep -rn "test(" tests | cut -d: -f1,2 -``` - -## Q: How can Playwright reports be configured in HyperExecute? -To generate and access Playwright HTML reports in HyperExecute, the reports must be stored in a known directory. The YAML file should include a `post` section to upload the directory as an artifact. Partial reports can be configured for framework-specific reporting. - -- Update `playwright.config` to specify the report output location: - -```javascript title="playwright.config" -reporter: [["html", { outputFolder: "playwright-report", open: "never" }]] -``` - -- Update `hyperexecute.yaml` file: - -```yaml title="hyperexecute.yaml" -report: true -partialReports: -frameworkName: playwright -location: playwright-report -type: HTML -``` - -## Q: How can tag-level discovery be performed in Playwright tests? -If tests include tags and only specific tags need to be executed, a custom Node.js script can be used for tag-based discovery. This allows filtering tests at the method level and executing only those matching the desired tag expression. - -- Create a file named `discovery.js` at the root level of the project (in the same directory as package.json). -- Copy the script from the [Gist](https://gist.github.com/gauravchawhan/9568ed96d6bc115707d37f695a56a6e7) into that file. -- Update the testDiscovery section in your hyperexecute.yaml to run this script using the node command, and pass your desired tag expression. - -```yaml title="hyperexecute.yaml" -testDiscovery: -type: raw -mode: static -command: node discovery.js '(?=.*@PROD)(?=.*@LOGIN)' -``` - -## Q: Why do tests pass locally and on the automation grid but fail in HyperExecute? -This occurs due to a version mismatch between the Playwright client and server. In automation grid runs, the client is installed by the user, while the server is managed internally. In HyperExecute, both client and server must be explicitly installed and configured in the YAML to ensure compatibility. - -- Verify the required Playwright version by checking the dependency listed in your `package.json` file. -- Once identified, install the specific version of Playwright during the pre step of the YAML: - -```yaml title="hyperexecute.yaml" -pre: -- npx playwright@1.41.0 install -``` -> Replace `1.41.0` with the version specified in the project’s package.json. - -## Q: Why are tests retried multiple times within a single scenario? -Multiple retries occur when retry logic is configured both at the Playwright framework level and in HyperExecute YAML. This can cause duplicate scenarios, incorrect reporting, and multiple executions of the same test. Disabling framework-level retries and using only HyperExecute-level retries avoids this problem. - -```yaml title="hyperexecute.yaml" -retryOnFailure: true -maxRetries: 1 -``` - -## Q: How can environment variables required for test execution be configured? -Certain frameworks or projects require specific environment variables, such as credentials or base URLs. Configuring these variables via the `env` section in the YAML or using a `.env` file ensures that tests execute successfully in HyperExecute. - -```yaml title="hyperexecute.yaml" -env: -BASE_URL: https://example.com -API_KEY: your_api_key_here -``` - -```bash title=".env" -BASE_URL=https://example.com -API_KEY=your_api_key_here -``` - -## Q: Why can tasks get stuck due to reports opening on a local server? -Playwright tests may hang if the HTML report is configured to automatically open on a local server after execution. Since HyperExecute runs in a headless CI environment, attempting to open the report in a browser window causes the process to stall indefinitely. - -To prevent this, update your `playwright.config.ts` file to prevent the report from opening automatically by setting the open option to `'never'`. - -```javascript title=playwright.config.ts" -reporter: [['html', { open: 'never' }]] -``` - -## Q: Why might a dependency work locally but fail on HyperExecute? -Dependencies that work locally may fail in HyperExecute due to OS-specific `package-lock.json` or `yarn.lock` files. These lock files may prevent correct resolution on a different operating system. Deleting or ignoring these files ensures fresh dependency installation. To ignore these files before uploading the project, add them to the `.gitignore` or `.hyperexecuteignore` file - -## Q: Why do unexpected `driver.quit` errors occur when running tests with parameters? -These errors can happen when there is a version mismatch between the Playwright server installed in HyperExecute and the client used in the project. Installing the exact version in the YAML prevents unstable or unexpected behavior. - -For example, to install version `1.50.0`: - -```yaml title="hyperexecute.yaml" -pre: -- npx playwright@1.50.0 install -``` - -## Q: Why might the browser fail to launch on HyperExecute? -Tests may fail to start if required browser binaries are missing or not installed correctly. Installing all Playwright dependencies, including browsers, ensures successful test execution. - -```yaml title="hyperexecute.yaml" -pre: -- npx playwright install --with-deps -``` - -## Q: Why do tests time out on HyperExecute but pass locally? -Tests can time out due to differences in resource availability, execution speed, or default timeouts between local and HyperExecute environments. Increasing the timeout in the Playwright config or test file mitigates this issue. - -```javascript -test.setTimeout(60000); // 60 seconds -``` - -## Q: Why do configuration files fail when using hardcoded absolute paths? -Absolute paths specific to a local environment may not exist in HyperExecute. Using relative paths from the project root ensures that scripts and configuration files remain portable across environments. - -```javascript -// Instead of require("C:/user/folder/test.js") -require("./tests/test.js") -``` - -## Q: Why are screenshots or videos not available in artifacts? -If Playwright is not configured to capture screenshots, videos, or traces, or if these artifacts are not uploaded correctly from HyperExecute, they will not appear in the report. Correct configuration in both `playwright.config.ts` and YAML ensures artifacts are captured and visible. - -- Update the `playwright.config.ts` file: -```javascript title="playwright.config.ts" -use: { -screenshot: 'on', -video: 'on', -trace: 'on-first-retry', -} -``` - -- Update the `hyperexecute.yaml` file: -```yaml title="hyperexecute.yaml" -uploadArtefacts: -- name: FinalReport -path: -- test-results/** -- playwright-report/** -``` - -## Q: Why do "Cannot find module" errors occur during execution? -These errors occur when required modules are missing or the installation step is skipped. Ensuring that all dependencies listed in `package.json` are installed in the `pre` step of the YAML prevents this issue. - -```yaml title="hyperexecute.yaml" -pre: -- npm install -``` diff --git a/static/docs/hyperexecute-yaml-faqs.md b/static/docs/hyperexecute-yaml-faqs.md deleted file mode 100644 index a3cee0498..000000000 --- a/static/docs/hyperexecute-yaml-faqs.md +++ /dev/null @@ -1,189 +0,0 @@ -# HyperExecute YAML FAQs - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -#### 1. I want to run all my feature files and scenarios listed in a folder. How can I do that without listing all of them explicitly in the YAML file? - -HyperExecute offers you the option to use our AutoSplit mode. You can learn more about the AutoSplit mode by going through [this page](/support/docs/hyperexecute-auto-split-strategy/). A sample YAML file that supports AutoSplit looks like this: - -```yaml ---- -version: 0.1 -runson: linux -concurrency: 2 -autosplit: true -pre: -- npm install - -cacheKey: '{{ checksum "package-lock.json" }}' -cacheDirectories: -- node_modules -testDiscovery: -type: automatic -mode: static -args: -featureFilePaths: -frameWork: javascript -specificTags: ["@ToDoOne", "@ToDoTwo", "@ToDoThree"] - -testRunnerCommand: -``` -You can simply add the path to the folder of files that you want to run in the _**featureFilePaths**_ argument, and your test execution command under the _**testRunnerCommand**_ parameter. - -#### 2. What can I do with pre steps and post steps in the YAML file? - -Pre and post steps open doors to large amounts of flexibility. You can add certificates just before running tests, start a background process to facilitate testing, compile applications, download dependencies and everything else that you need to do just before and after running your tests in the CI or your system. - -#### 3. How can I install private artifactory dependencies that can only be accessed on my organization’s internal network on HyperExecute Machines? - -You can use a tunnel to connect with your organization's private network with the HyperExecute machines. You can do this by setting the tunnel flag to true in HyperExecute’s YAML file. - -```bash -tunnel: true -``` - -Moreover, add one of the following parameters in the `preDirectives` field depending on the tech stack that your organization is using. - -- For Maven projects: - `-DproxyHost=${LT_PROXY_HOST}` - `-DproxyPort=${LT_PROXY_PORT}` - -- For Node: - `npm config set proxy http://${LT_PROXY_HOST}:${LT_PROXY_PORT}` - `npm config set https-proxy http://${LT_PROXY_HOST}:${LT_PROXY_PORT}` - -Therefore, if you want to use a tunnel for connecting HyperExecute with your organization (that is working with a maven project), you should configure your YAML file like this: - -```yaml ---- -version: 0.1 -runson: linux -concurrency: 2 -autosplit: true -tunnelOpts: -global: true - -preDirectives: -commands: -- mvn -Dmaven.repo.local=$CACHE_DIR -Dmaven.test.skip=true clean install -DproxyHost=${LT_PROXY_HOST} -DproxyPort=${LT_PROXY_PORT} -``` - -> **Note**: The variables ‘LT_PROXY_HOST’ and ‘LT_PROXY_PORT’ are exposed with the tunnel proxy value by default on HyperExecute machines when the tunnel flag is set to _true_ in the HyperExecute YAML. - -#### 4. How can I install and set a private node registry on the HyperExecute Machine? - -You can do this by adding the following command in the preDirectives section of the HyperExecute YAML file. - -```bash -preDirectives: -commands: -- npm config set registry -``` - -**Note**: Replace the placeholder value <artifactory_URL> with the link to your private node registry. - -#### 5. Can I run WDIO tests on HyperExecute via proxy? - -Yes, you can. Use the following parameters in the testRunnerCommand of the HyperExecute YAML file: - -```bash -testRunnerCommand: $env:GLOBAL_AGENT_NO_PROXY="hub.lambdatest.com";$env:GLOBAL_AGENT_HTTP_PROXY=$env:LT_PROXY -``` - -#### 6. I want to use a specific version of gradle for my project. How can I set that up on HyperExecute machines? - -You can set up your Gradle project with HyperExecute by configuring the runtime flag in the YAML file. If you are using the 7.0 version of Gradle, use the following configurations. - -```bash -runtime: -language: java -version: 17 -addons: -- name: "gradle" -version: "7.0" -``` - -#### 7. I want to pass a specific package through npm in the YAML file, instead of npm picking the package present in the directory. Can I do that with HyperExecute? - -You can accomplish this by running the following command in the preDirectives section of the YAML file: - -```bash -preDirectives: -commands: -- npm --prefix /path/to/project/my_package.json -``` -This command will install a package called **my_package.json** from the path that you have provided. - -#### 8. How can I use the Jenkins job choice parameters in the YAML file? - -You can do this by directly calling the parameter keys in the HyperExecute YAML file and use the Jenkins choice parameters. -A sample testRunnerCommand to accomplish this is: - -```bash -testRunnerCommand: mvn test `-DselectedTests="$test" `-Dmaven.repo.local=./.m2 dependency:resolve `-Dbrowser=${browser} `-Dversion=${version} -``` -In this example, **browser** and **version** are the two choice parameters from Jenkins, as shown in the image below. You can call them in the YAML file as ${browser} and ${version} in the testRunnerCommand or testDiscovery command. - -```bash -testDiscovery: -type: raw -mode: remote -command: grep 'test name' src/test/java/${xml} | awk '{print$2}' | sed 's/name=//g' | sed 's/\x3e//g' - -testRunnerCommand: mvn test `-DselectedTests="$test" `-Dmaven.repo.local=./.m2 dependency:resolve `-Dbrowser=${browser} `-Dversion=${version} -``` - -#### 9. I run a lot of tests with the same YAML configurations. Is there any way where I can run my tests without specifying the same configurations over and over again? - -You can solve this problem by using HyperExecute's inheritance feature. This feature allows you to inherit the configurations of a base YAML file and saves you the time of configuring the YAML file multiple times. - -```bash -base: -yamls: -- ./ -- ./ -``` - -To learn more about how to use this feature, go through [this page](/support/docs/hyperexecute-inherit-config/). - -#### 10. I want to test the code in my Git repository. Is there a way where I can accomplish that with HyperExecute? - -You can use the `sourcePayload` parameter for the same. Your test scripts are directly sourced from your Git provider with the help of secure access tokens and only your HyperExecute YAML file is encrypted and uploaded through the HyperExecute CLI. To learn more about how this feature works, go through [this page](/support/docs/hyperexecute-how-to-configure-sourcePayload/). - -```bash -sourcePayload: -platform: git -link: https://-------- -ref: master -accessToken: -``` - -#### 11. I am running a non-hub based test on HyperExecute. How can I capture a video of it? - -You can use HyperExecute’s video recording feature even while running non-hub based tests (Selenium, Cypress, CDP are all hub-based). All you need to do is, set the `captureScreenRecordingForScenarios` flag to `true` in your [HyperExecute YAML](/support/docs/deep-dive-into-hyperexecute-yaml/) file to capture the video of your test scenarios. - -`captureScreenRecordingForScenarios: true` - -You can access the recorded video on the Tasks page by clicking on the **Watch Video** button on the right-hand side of your test. - -You can use this feature if you want to trigger a command and want to record it. This will also be useful when you want to record any applications that were triggered on your desktop during the test execution process. - -#### 12. How to check if there is any private dependency in testng YAML? - -We can detect any private dependency in testng YAML using `analyze` flag in CLI. - -#### 13. How to handle Maven SSL Cert Error while executing the test? - -Pass this maven arguments which require to handle mvn ssl cert errors - -```bash --Dmaven.wagon.http.ssl.insecure=true -Dmaven.wagon.http.ssl.allowall=true -Dmaven.wagon.http.ssl.ignore.validity.dates=true -``` - -#### 14. Will my YAML parameters overwrite properties in my XML configuration? - -No, your XML configurations are not overwritten by YAML parameters. - -For more information on the HyperExecute YAML file, visit this [page](/support/docs/deep-dive-into-hyperexecute-yaml/). - ->If you have more questions, then just give us a shout and we will answer them for you. diff --git a/static/docs/hyperexecute-yaml-parameters.md b/static/docs/hyperexecute-yaml-parameters.md deleted file mode 100644 index 979ae6c70..000000000 --- a/static/docs/hyperexecute-yaml-parameters.md +++ /dev/null @@ -1,131 +0,0 @@ -# HyperExecute YAML - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The **HyperExecute YAML** file serves as the foundational component for test execution within the HyperExecute Platform. The subsequent list enumerates the diverse configuration options supported by [HyperExecute YAML](/support/docs/hyperexecute-yaml-parameters/#sample-hyperexecute-yaml), providing you with the flexibility to tailor test execution according to the specific requirements of your project. - -## Mandatory Parameters - -| Key | Type | Description| -|-----|------|------------| -| [version](/support/docs/deep-dive-into-hyperexecute-yaml/#version) | String | The version of HyperExecute YAML you are using. | -| [runson](/support/docs/deep-dive-into-hyperexecute-yaml/#runson) | _linux_, _mac_, or _win_ | The OS on which you will run your Test. Can provide [`${matrix.os}`](/support/docs/deep-dive-into-hyperexecute-yaml/#matrix) as well to run a multi OS job.| -| [pre](/support/docs/deep-dive-into-hyperexecute-yaml/#pre) | List of Strings | The actions that are performed before test execution. | - -## AutoSplit Mode Parameters - -| Key | Type | Description| -|-----|------|------------| -| [autosplit](/support/docs/deep-dive-into-hyperexecute-yaml/#autosplit) | Boolean | Autosplit utilizes an AI-based algorithm to intelligently distribute your discovered tests among concurrent virtual machines (VMs). It takes into account the previous test runs and statuses to achieve the most optimal distribution of tests. By doing so, it maximizes the efficiency and effectiveness of your test execution process. Default value is `false`. | -| [concurrency](/support/docs/deep-dive-into-hyperexecute-yaml/#concurrency) | Integer | Indicates the number of concurrent sessions on HyperExecute. | -| [testDiscovery](/support/docs/deep-dive-into-hyperexecute-yaml/#testdiscovery) | Map | This key is used to locate or discover relevant tests via class names, filters, file names, etc. | -| [testRunnerCommand](/support/docs/deep-dive-into-hyperexecute-yaml/#testrunnercommand) | String | A command used to run a single test entity in isolation.This entity could be a file , module, feature or scenario. | - -## Matrix Mode Parameters - -| Key | Type | Description| -|-----|------|------------| -| [matrix](/support/docs/deep-dive-into-hyperexecute-yaml/#matrix) | String | A matrix allows you to create multiple tasks by performing variable substitutions in a single job definition. | -| [combineTasksInMatrixMode](/support/docs/deep-dive-into-hyperexecute-yaml/#combinetasksinmatrixmode) | Boolean | This flag is passed in matrix mode to run the (matrix-multiplied) combinations as scenarios in the specified number of HyperExecute machines, instead of 1 machine per matrix-combination.| -| [testSuites](/support/docs/deep-dive-into-hyperexecute-yaml/#testsuites) | String | A command to run the tests that were mentioned in the scenario key for matrix based test execution. | - -## Hybrid Mode Parameters - -| Key | Type | Description| -|-----|------|------------| -| [parallelism](/support/docs/deep-dive-into-hyperexecute-yaml/#parallelism) | Integer | Parallelism defines number VMs to be spawned in case of [`hybrid`](/support/docs/hyperexecute-hybrid-strategy/) mode, if the user is not defining the parallelism then he must define platform specific parallelism (win , mac and linux). If both are defined the preference will be given to platform base parallelism | -| [macParallelism](/support/docs/deep-dive-into-hyperexecute-yaml/#macparallelism) | Integer | It defines number of mac VM to be spawned for job. | -| [winParallelism](/support/docs/deep-dive-into-hyperexecute-yaml/#winparallelism) | Integer | It defines number of win VM to be spawned for job. | -| [linuxParallelism](/support/docs/deep-dive-into-hyperexecute-yaml/#linuxparallelism) | Integer | It defines number of linux VM to be spawned for job. | -| [testRunnerCommand](/support/docs/deep-dive-into-hyperexecute-yaml/#testrunnercommand) | String | A command used to run a single test entity in isolation.This entity could be a file , module, feature or scenario. | -| [macTestRunnerCommand](/support/docs/deep-dive-into-hyperexecute-yaml/#mactestrunnercommand) | String | Test runner command for MAC. | -| [winTestRunnerCommand](/support/docs/deep-dive-into-hyperexecute-yaml/#wintestrunnercommand) | String | Test runner command for Windows. | -| [linuxTestRunnerCommand](/support/docs/deep-dive-into-hyperexecute-yaml/#linuxtestrunnercommand) | String | Test runner command for Linux. | - -## Basic Parameters - -| Key | Type | Description| -|-----|------|------------| -| [cacheKey](/support/docs/deep-dive-into-hyperexecute-yaml/#cachekey) | String | File which can be used to generate unique key for cache. Example : package.json | -| [cacheDirectories](/support/docs/deep-dive-into-hyperexecute-yaml/#cachedirectories) | List of Strings | Dependency directories that are cached to improve the test execution speed. | -| [env](/support/docs/deep-dive-into-hyperexecute-yaml/#env) | Map | No | Here we can define all the env variables for VM , we can even refer all the key stored in vault by using syntax like `${{ .secrets.KeyName }}`.| -| [runtime](/support/docs/deep-dive-into-hyperexecute-yaml/#runtime) | Map | Install any dependent language/framework with any specific version. | -| [retryOnFailure](/support/docs/deep-dive-into-hyperexecute-yaml/#retryonfailure) | Boolean | If set to true, then it will retry tests based on `maxRetries` key as defined below. Default value is `false`. | -| [maxRetries](/support/docs/deep-dive-into-hyperexecute-yaml/#maxretries) | Integer | The maximum number of times your scenarios can be retried. If `retryOnFailure` is true then Default value will be 2. Value Range 1 to 5.| -| [post](/support/docs/deep-dive-into-hyperexecute-yaml/#post) | List of Strings | The actions that are performed after test execution. | -| [report](/support/docs/deep-dive-into-hyperexecute-yaml/#report) | Boolean | To generate the test report at the end of the job using the framework generated reports, set this value to true. We also need to set partialReports for generating report | -| [partialReports](/support/docs/deep-dive-into-hyperexecute-yaml/#report) | Map | Configurations related to generate report. | -| [errorCategorizedOnFailureOnly](/support/docs/deep-dive-into-hyperexecute-yaml/#errorcategorizedonfailureonly) | Boolean | Control the behavior of error categorization generated on your tests| -|[errorCategorizedReport](/support/docs/deep-dive-into-hyperexecute-yaml/#errorcategorizedreport) | Map, Boolean | Generate a RCA report for your failed tests.| -| [jobLabel](/support/docs/deep-dive-into-hyperexecute-yaml/#joblabel) | List of Strings | Add tags or labels to jobs. This allows you to search your jobs using the labels or tags assigned to them. | - -## Advanced Parameters - -### Artifacts - -| Key | Type | Description| -|-----|------|------------| -| [mergeArtifacts](/support/docs/deep-dive-into-hyperexecute-yaml/#mergeartifacts) | Boolean | Merge all the artifacts generated by each task. | -| [uploadArtefacts](/support/docs/deep-dive-into-hyperexecute-yaml/#uploadartefacts) | Map | The uploadArtifacts key contains the path and the name of the file which you want to download later as an artifact when job finishes.| -| [captureCSVResult](/support/docs/deep-dive-into-hyperexecute-yaml/#capturecsvresult) | Boolean |If this directive is enabled, it will generate the performance stats artifact having aggregated information for the time taken by Selenium Commands. | -| [skipArtifactStageIfNoTest](/support/docs/deep-dive-into-hyperexecute-yaml/#skipartifactstageifnotest) | Boolean |When enabled, it will mark artifact stage as skipped when there is no selenium/cdp test executed in the task. | -|[linkValidity](/support/docs/deep-dive-into-hyperexecute-yaml/#linkvalidity) | Integer | Customize the duration of validity for report and artifact URLs generated after job completion. | -| [stripParentDirectory](/support/docs/deep-dive-into-hyperexecute-yaml/#stripparentdirectory) | Boolean | When `mergeArtifacts` is true, providing this flag will strip the parent directory from all the downloaded artifacts and place the artifacts in base path. | -| [generateArtifactAfterEveryStage](/support/docs/deep-dive-into-hyperexecute-yaml/#generateartifactaftereverystage) | Boolean | With this flag `artifacts` will be generated after every `stage`, without this flag artifacts are generated after every task and avoid any overrides for artifacts of the same name. | -| [taskIdentifierInNonConflictingArtifacts](/support/docs/deep-dive-into-hyperexecute-yaml/#taskidentifierinnonconflictingartifacts) | Boolean |This flag when set to **true** specifies that the task ID should be included in the non-conflicting artifacts. | - -### Timeout - -| Key | Type | Description| -|-----|------|------------| -| [globalTimeout](/support/docs/deep-dive-into-hyperexecute-yaml/#globaltimeout) | Integer | It sets the max duration (1-150 mins) for a HyperExecute job. It terminates a job exceeding this limit. Default is 90 mins, but can be adjusted based on your requirements.| -| [testSuiteTimeout](/support/docs/deep-dive-into-hyperexecute-yaml/#testsuitetimeout) | Integer | It is used to set the timeout on all scenario stages inside a task. If defined, your complete test suite should get executed within this time. Its max value is 150 minutes.| -| [testSuiteStep](/support/docs/deep-dive-into-hyperexecute-yaml/#testsuitestep) | Integer | It is used to timeout individual scenario stages in a task. Each scenario in each task should get completed before this time. | - -### Tunnel - -| Key | Type | Description| -|-----|------|------------| -| [tunnel](/support/docs/deep-dive-into-hyperexecute-yaml/#tunnel) | Boolean | If set to true , tunnel will be spawn on run time , which connects system where we are running the cli to VM where tests are getting executed. | -| [tunnelOpts](/support/docs/deep-dive-into-hyperexecute-yaml/#tunnelopts) | Map | All the configuration related to tunnel will be mention here. Please note tunnelOpts will work along with either `tunnel` or `tunnelNames`. | -| [tunnelNames](/support/docs/deep-dive-into-hyperexecute-yaml/#tunnelnames) | List of Strings | List of tunnels name which has been registered and running with HyperExecute. | - -### Other Advanced Parameters - -| Key | Type | Description| -|-----|------|------------| -| [globalPre](/support/docs/deep-dive-into-hyperexecute-yaml/#globalpre) | Map | This flag runs once before all tasks start, used for global setup such as installing dependencies or configuring environments. | -| [globalPost](/support/docs/deep-dive-into-hyperexecute-yaml/#globalpost) | Map | This flag runs once after all tasks finish, used for global cleanup such as removing logs or finalizing reports. | -| [failFast](/support/docs/deep-dive-into-hyperexecute-yaml/#failfast) | Map | Flag to fail a job faster if there are `x` consecutive failures. | -| [differentialUpload](/support/docs/deep-dive-into-hyperexecute-yaml/#differentialupload) | Map | Minimize the time of upload of code. | -| [background](/support/docs/deep-dive-into-hyperexecute-yaml/#background) | Map |It is used to trigger long-running Application Servers tasks like running WebApps or databases | -| [retryOptions](/support/docs/deep-dive-into-hyperexecute-yaml/#retryoptions) | Array | Retries the test case if found specific errors in the logs. | -| [base](/support/docs/deep-dive-into-hyperexecute-yaml/#base) | Map | Inherit the configurations of a base YAML file into your parent file. | -| [sourcePayload](/support/docs/deep-dive-into-hyperexecute-yaml/#sourcepayload) | Map | You can use this to define the source of your test scripts so that the scripts can directly be fetched from your git directly for execution on HyperExecute. | -| [hostsOverride](/support/docs/deep-dive-into-hyperexecute-yaml/#hostsoverride) | List of Maps |If you want to add custom domain mappings in local DNS entry. | -| [frameworkStatusOnly](/support/docs/deep-dive-into-hyperexecute-yaml/#frameworkstatusonly) | Boolean | Set status of scenario based on it’s tests status / remarks. | -| [vars](/support/docs/deep-dive-into-hyperexecute-yaml/#vars) | Map | Here we define all the variable which can be refer in other parts of yaml. You can use these variables in the YAML file as `${your_variable_name}`. | -| [testRunnerExecutor](/support/docs/deep-dive-into-hyperexecute-yaml/#testrunnerexecutor) | String | To switch the test execution from powershell to the command line. | -|[workingDirectory](/support/docs/deep-dive-into-hyperexecute-yaml/#workingdirectory)| Path of Location |This flag determines where test discovery and execution commands operate, including the creation of associated files or directories. | -| [captureScreenRecordingForScenarios](/support/docs/deep-dive-into-hyperexecute-yaml/#capturescreenrecordingforscenarios) | bool |If this key is set to true, the video will be recorded for whole scenario execution | -| [buildConfig](/support/docs/deep-dive-into-hyperexecute-yaml/#buildconfig) | Map | For managing the Hyperlink behaviour. | -| [matrixEnvPrefix](/support/docs/deep-dive-into-hyperexecute-yaml/#matrixenvprefix) | Boolean |This flag is used to prefix env variables with `HE_ENV_` being set when you run a job in "matrix" mode. Default is false. | -| [dynamicAllocation](/support/docs/deep-dive-into-hyperexecute-yaml/#dynamicallocation) | Boolean |This flag dynamically allocate test cases to workers as they become available and ensures that all available resources are used to run tests. | -| [collectLocalGitData](/support/docs/deep-dive-into-hyperexecute-yaml/#collectlocalgitdata) | Boolean |This flag is used to store the git diff data related to the repository used to run HyperExecute Jobs. | -| [smartGrid](/support/docs/deep-dive-into-hyperexecute-yaml/#smartgrid) | Boolean |When enabled, it will reduce the browser setup time in case of windows. | -| [scenarioCommandStatusOnly](/support/docs/deep-dive-into-hyperexecute-yaml/#scenariocommandstatusonly) | Boolean |When enabled, it will mark the scenario as passed even if no test is associated with it or will mark the status of scenario based on the status of the last test executed. | -| [cypress](/support/docs/deep-dive-into-hyperexecute-yaml/#cypress) | Boolean |This is required to be true in case user is running Cypress test suite. | -| [cypressOps](/support/docs/deep-dive-into-hyperexecute-yaml/#cypressops) | Map |This map is used to pass cypress specific options. | -| [shell](/support/docs/deep-dive-into-hyperexecute-yaml/#shell) | String |Defines the shell on which all your commands should run. | -| [dataJsonPath](/support/docs/deep-dive-into-hyperexecute-yaml/#datajsonpath) | Array of String | DataJsonPaths helps to distribute data over VMs based on different Jsons.| -| [dataJsonBuilder](/support/docs/deep-dive-into-hyperexecute-yaml/#datajsonbuilder) | Array of String | dataJsonBuilder flag reads the data from your specified JSON files for distributed testing scenarios.| -| [strict](/support/docs/deep-dive-into-hyperexecute-yaml/#strict) | Boolean |If strict is set to true in yaml then the variables used in yaml must be present in either vars or environment variables on the user machine. | -| [codeDirectory](/support/docs/deep-dive-into-hyperexecute-yaml/#codedirectory) | String |Defines in which directory all of your commands defined in yaml would run. Your code would be downloaded in this directory. | -| [preDirectives](/support/docs/deep-dive-into-hyperexecute-yaml/#predirectives) | Map | Similar to pre with option to retry and specify shell. | -| [postDirectives](/support/docs/deep-dive-into-hyperexecute-yaml/#postdirectives) | Map |Similar to post with option to retry. | -| [alwaysRunPostSteps](/support/docs/deep-dive-into-hyperexecute-yaml/#alwaysrunpoststeps) | Boolean | Execute the Post Steps in every scenario case | -| [cacheTestURL](/support/docs/deep-dive-into-hyperexecute-yaml/#cachetesturl) | Boolean |It enables users to cache static test files. | -| [project](/support/docs/deep-dive-into-hyperexecute-yaml/#project) | String | Segregate data at a project/repo level | -| [slackChannel](/support/docs/deep-dive-into-hyperexecute-yaml/#slackchannel) | String | Allows you to receive notifications of your Job updates directly in your preferred Slack channel. | -| [afterEachScenario](/support/docs/deep-dive-into-hyperexecute-yaml/#aftereachscenario) | List of Strings | Allows you to trigger your define tasks after each test scenario. | -| [syncStart](/support/docs/deep-dive-into-hyperexecute-yaml/#syncstart) | Map | Ensures all VMs are ready before starting parallel tasks.| diff --git a/static/docs/hyperexecute-yaml-version0.2.md b/static/docs/hyperexecute-yaml-version0.2.md deleted file mode 100644 index 3a77f32a0..000000000 --- a/static/docs/hyperexecute-yaml-version0.2.md +++ /dev/null @@ -1,218 +0,0 @@ -# HyperExecute Yaml Version 0.2 - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This version introduces several new features and improvements over Version 0.1. This documentation outlines the changes and provides guidance on when to use Version 0.2 instead of Version 0.1. - -- Currently supported frameworks are **maven/testng**, **maven/junit4**, **maven/junit5**, **wdio/mocha**, and **wdio/jasmine** framework. -- Version 0.2 supports all the fields available in Version 0.1, except for [`testDiscovery`](/support/docs/deep-dive-into-hyperexecute-yaml/#testdiscovery) and [`testRunnerCommand`](/support/docs/deep-dive-into-hyperexecute-yaml/#testrunnercommand) -- The new [`framework`](/support/docs/hyperexecute-yaml-version0.2/#framework) flag has been introduced to configure the test framework. - -## Why to use HyperExecute YAML Version 0.2? - -- The new framework feature supports caching by default. You do not have to specify any directories to cache for faster performance. If you adds the [`cacheKey`](/support/docs/deep-dive-into-hyperexecute-yaml/#cachekey) and [`cacheDirectories`](/support/docs/deep-dive-into-hyperexecute-yaml/#cachedirectories) keys in your yaml, the default caching gets disabled and preference is given to user specified cache. -- In Version 0.2, the support for [matrix mode](/support/docs/hyperexecute-matrix-multiplexing-strategy/) has been removed, and only [static discovery](/support/docs/deep-dive-into-hyperexecute-yaml/#mode) is available. This means that the discovery command will run on your system rather than in a matrix. -- Since the support for [`testRunnerCommand`](/support/docs/deep-dive-into-hyperexecute-yaml/#testrunnercommand) is removed, the test orchestration will be managed automatically. - -## ```framework``` - -The ```framework``` field in Hyperexecute YAML Version 0.2 allows you to configure the test framework settings. It provides more flexibility and customization options for your testing needs with the following parameters. - -| Parameters | Type | Mandatory | Description| -|:---|:--|:---|:---| -| [name](#name) | String | Yes | You need to specify which testing framework you are using in your repo.| -| [flags](#flags) | Array | No | Command line flags to pass to the custom runner for both test discovery and execution.| -| [discoveryFlags](#discoveryFlags) | Array | No | Command line flags to pass to the custom runner for test discovery only.| -| [runnerFlags](#runnerFlags) | Array | No | Command line flags to pass to the custom runner for test execution only. | -| [discoveryType](#discoveryType) | String | No | Specifies the type of test discovery to use. Supported values are "method" and "class". The default is "method".| -| [workingDirectory](#workingDirectory) | String | No | Specifies the working directory where all discovery and execution commands will be executed.| -| [defaultReports](#defaultReports) | Boolean | No | Specifies whether to create default reports for the specified framework.| -| [region](#region) | String | No | Specifies in which region you want to spin your appium tests.| -| [artifacts](#artifacts) | Boolean | No | Specifies whether to generate artifacts or not | -| [language](#language) | String | No | Specifies the device’s system language for the test session. This determines the language in which your app’s UI and strings will be displayed. | -| [locale](#locale) | String | No | Defines the regional format settings such as date, time, currency, and number conventions. | -| [mitmProxy](#mitmProxy) | Boolean | No | Capture network logs directly from emulator sessions using MITM. | - -### `name` -Specifies the testing framework used in your repository. - -```yaml -framework: -name: "maven/testng" -``` - -To enable maven runner with Appium, you have to pass `appium: true` before the `framework` field - -```yaml -appium: true -framework:  -  name: "maven/testng" -``` - -### `flags` -Specifies the command line flags to pass to the custom runner for both test discovery and execution. - -```yaml -framework: -name: "maven/testng" -flags: ["-Dplatname=win", "-Dgroups=selenium-test"] -``` - -### `discoveryFlags` -Specifies the command line flags to pass to the custom runner for test discovery only. - -```yaml -framework: -name: "maven/testng" -discoveryFlags: ["-Dgroups=selenium-test"] -``` - -### `runnerFlags` -Specifies the command line flags to pass to the custom runner for test execution only. - -```yaml -framework: -name: "maven/testng" -runnerFlags: ["-Dgroups=database"] -``` - -### `discoveryType` -Specifies the level at which user wants to discover the tests. Supported values are "method" and "class". The default is "method". - -```yaml -framework: -name: maven/testng -#highlight-next-line -discoveryType: method -# instead of method you can also use xmltest or class as a discovery type -flags: -- "-Dplatname=win" -``` - -- For **maven/testng** the supported discovery types are **method, class** and **xmltest**. The default is **method**. -- For **maven/junit4** and **maven/junit5** the supported discovery types are **method** and **class**. The default is **method**. -- For **wdio/mocha** and **wdio/jasmine** the supported discovery types are **test, spec, suite** and **wdiosuite**. The default is **spec**. - -### `workingDirectory` - -The `working directory` specifies the location of the directory in which all test discovery and execution commands will be run, as well as the location of any files or directories that are created as a result of the command execution. If the `workingDirectory` option is not specified, then the working directory will be the directory where the YAML file is located. - -```yaml -framework: -name: maven/testng -discoveryType: method -workingDirectory: src/main -flags: -- "-Dplatname=win" -``` - -### `defaultReports` -Specifies whether to create default reports for the specified framework. - -```yaml -framework: -name: maven/testng -defaultReports: false -flags: -- "-Dplatname=win" -``` - -### `region` - -The region parameter specifies the region or location where the Appium tests will be executed. Our platform supports the following three regions: - -- ap (Asia-Pacific) -- us (United States) -- eu (European Union) - -> The region parameter should always be defined under the `args` parameter, as shown in the below sample code. - -```yaml -framework: -args: -region: us -``` - -### `artifacts` - -To generate artifacts for your Espresso tests, add the `artifacts: true` flag in your YAML file: - -```yaml -framework: -args: -artifacts: true -``` - -> 📕 Learn [how to perform group-based test discovery in TestNG](/support/docs/hyperexecute-how-to-perform-group-based-test-discovery-in-testng) - -### `language` - -Specifies the device’s system language for the test session. This determines the language in which your app’s UI and strings will be displayed. - -```yaml -framework: -args: -language: es -``` - -### `locale` - -Defines the regional format settings such as date, time, currency, and number conventions. - -```yaml -framework: -args: -locale: ES -``` - -### `mitmProxy` - -You can now capture network logs directly from emulator sessions using MITM. This enhancement enables deeper debugging of API calls, request/response payloads, and overall network traffic during test execution. - -```yaml -framework: -args: -mitmProxy: true -``` - -## Sample Yaml Version 0.2 - -```yaml ---- -version: 0.2 -runson: win - -autosplit: true -concurrency: 2 - -pre: -# Skip execution of the tests in the pre step -- mvn dependency:resolve - -framework: -name: maven/testng -flags: -- "-Dplatname=win" -discoveryFlags: ["-Dgroups=selenium-test"] -runnerFlags: ["-Dgroups=database"] -discoveryType: method -workingDirectory: src/main -defaultReports: false -args: -region: ap -language: es -locale: es -mitmProxy: true - -retryOnFailure: true -maxRetries: 1 - -post: -- ls target/surefire-reports/ - -mergeArtifacts: true -uploadArtefacts: -- name: ExecutionSnapshots -path: -- target/surefire-reports/html/** -``` diff --git a/static/docs/hyperexecute-zephyr-scale-integration.md b/static/docs/hyperexecute-zephyr-scale-integration.md deleted file mode 100644 index 975c02baf..000000000 --- a/static/docs/hyperexecute-zephyr-scale-integration.md +++ /dev/null @@ -1,145 +0,0 @@ -# Integrate Zephyr Scale with HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Zephyr Scale is a test management platform that helps you plan, manage, and measure your tests inside Jira. It's designed from the ground up to support large test libraries and scaling teams - -This document details the seamless integration between HyperExecute and qTest, enabling you to run your automated tests on a variety of devices and real devices provided by TestMu AI. - -## Prerequisites - -- Install the [Zephyr Scale - Test Management](https://marketplace.atlassian.com/apps/1213259/zephyr-scale-test-management-for-jira?hosting=cloud&tab=overview) for Jira application from the Atlassian Marketplace. -- TestMu AI account. You can [sign up for free](https://www.testmuai.com/register/?redirectTo=https://accounts.lambdatest.com/dashboard). -- TestMu AI [Username and Access Key](/support/docs/hyperexecute-how-to-get-my-username-and-access-key/) -- A Jira Project key. -- A Zephyr Scale Access token. You can get this from the `profiles and settings` option of your Jira Board. - -## Step 1: Setup Your Test Suite - -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** - -Download or Clone the code sample for the Maestro framework from the TestMu AI GitHub repository to run the tests on the HyperExecute. - - View on GitHub - -## Step 2: Setup the CLI in your Test Suite - -After cloning / downloading the sample repo, you need to setup the CLI and the environment variables. - -### Download the HyperExecute CLI - -The CLI is used for triggering the tests on HyperExecute. It is recommend to download the CLI binary on the host system and keep it in the root directory of the suite to perform the tests on HyperExecute. - -You can download the CLI for your desired platform from the below mentioned links: - -| Platform | HyperExecute CLI | -| ---------| ---------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| MacOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -### Setup Environment Variable - -Now, you need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in the [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). - -Run the below mentioned commands in your terminal to setup the CLI and the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -## Step 3: Configure the Test Script - -In the `PostTestResults.java` script, add the following String parameters. - -* File name of your test results. -* File path of the test result file. -* Jira Project key. -* Zephyr Scale Access token. - -```js -public class PostTestResults { - -public void callApi() { - -String fileName = "TEST-com.lambdatest.JUnitTodo.xml" ; //provide the file name of the test results -String filePath = "target/surefire-reports/TEST-com.lambdatest.JUnitTodo.xml"; //provide the file path of the test result file -String projectKey = "ZD"; //provide the Jira project key -String token = "abcdefghijklmnop123456"; // provide Zephyr Scale access token -``` - -## Step 4: Configure YAML in your Test Suite - -```yaml ---- -version: 0.1 -globalTimeout: 150 -testSuiteTimeout: 150 -testSuiteStep: 150 - -runson: linux - -autosplit: true -retryOnFailure: true - -maxRetries: 1 -concurrency: 3 - -env: -# PAT: ${{ .secrets.testKey }} -CACHE_DIR: m2_cache_dir -TARGET_OS: linux - -cacheKey: '{{ checksum "pom.xml" }}' -cacheDirectories: -- ${CACHE_DIR} - -shell: bash - -pre: -# Download and install packages in the CACHE_DIR. -# Skip execution of the tests in the pre step -- mvn -Dmaven.repo.local=${CACHE_DIR} -Dmaven.test.skip=true clean install - -post: -- cat hyperexecute-zephyr-scale.yaml - -testDiscovery: -type: raw -mode: remote -command: grep 'public class' src/test/java/com/lambdatest/*.java | awk '{print$3}' - -testRunnerCommand: mvn -Dplatname=linux -Dmaven.repo.local=m2_cache_dir -Dtest=$test -P single -# test site -jobLabel: ['hyperexecute', 'zephyr'] -``` - -## Step 5: Execute your Test Suite - -> **NOTE :** In case of MacOS, if you get a permission denied warning while executing CLI, simply run **`chmod u+x ./hyperexecute`** to allow permission. In case you get a security popup, allow it from your **System Preferences** → **Security & Privacy** → **General tab**. - -Run the below command in your terminal at the root folder of the project: - -```bash -./hyperexecute --config RELATIVE_PATH_OF_YOUR_YAML_FILE -``` - -OR use this command if you have not exported your username and access key in the step 2. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE `} - -## Step 6: Monitor the Test Execution - -Visit the [HyperExecute Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) and check your Job status. - -### Viewing the Test Results in Zephyr Scale - -To view test results in Zephyr Scale, head over to your Zephyr Scale project. The below screenshot shows the Test Case in Zephyr Scale that you executed on TestMu AI. - -You can also view the executed build by navigating to **Test Cycles**. - -To view the build details, click on the Zephyr project *Key* or *Name*. diff --git a/static/docs/ie-mode-on-edge.md b/static/docs/ie-mode-on-edge.md deleted file mode 100644 index 68a1a776e..000000000 --- a/static/docs/ie-mode-on-edge.md +++ /dev/null @@ -1,98 +0,0 @@ -# Test in IE Mode on Microsoft Edge - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Microsoft retired Internet Explorer 11 in June 2022, but many enterprise applications still require IE rendering for specific workflows. Microsoft Edge includes an IE Mode that loads pages using the IE11 Trident engine inside an Edge tab. TestMu AI supports IE Mode automation on Windows 11, so you can validate legacy apps without maintaining a standalone IE browser. - -## When to Use IE Mode - -Use IE Mode to test legacy applications that depend on Internet Explorer rendering. - -IE Mode is the right choice when: - -- **Your application requires IE-specific rendering.** Intranet portals, older CRM systems, and government apps often depend on ActiveX controls, VBScript, or document modes that only the IE Trident engine supports. -- **You are migrating from IE11.** During the transition to modern browsers, IE Mode lets you verify that pages render identically in compatibility mode before removing IE-specific code. -- **Compliance or policy mandates IE support.** Some organizations require IE compatibility for internal tools until all systems are upgraded. - -IE Mode is available only on Windows. The Edge browser version must support IE Mode, and Microsoft controls which versions include this feature through enterprise policy. - -## Set Up IE Mode Capabilities - -Configure `InternetExplorerOptions` with Edge attachment to run tests in IE Mode on TestMu AI. - -To run tests in IE Mode, use the `InternetExplorerOptions` class with `platformName` set to `Windows 11` and `browserVersion` set to `11.0`. TestMu AI handles the Edge attachment server-side. Pass your credentials in `LT:Options` as usual. - -```java title="IEModeTest.java" -import org.openqa.selenium.ie.InternetExplorerOptions; -import org.openqa.selenium.remote.RemoteWebDriver; -import java.net.URL; -import java.util.HashMap; - -public class IEModeTest { -public static void main(String[] args) throws Exception { -InternetExplorerOptions browserOptions = new InternetExplorerOptions(); -browserOptions.setPlatformName("Windows 11"); -browserOptions.setBrowserVersion("11.0"); - -HashMap ltOptions = new HashMap<>(); -ltOptions.put("username", System.getenv("LT_USERNAME")); -ltOptions.put("accessKey", System.getenv("LT_ACCESS_KEY")); -ltOptions.put("project", "IE Mode Test"); -ltOptions.put("w3c", true); -browserOptions.setCapability("LT:Options", ltOptions); - -RemoteWebDriver driver = new RemoteWebDriver( -new URL("https://hub.lambdatest.com/wd/hub"), browserOptions); -driver.get("https://example.com"); - -System.out.println("Page title: " + driver.getTitle()); -driver.quit(); -} -} -``` - -```csharp title="IEModeTest.cs" -using OpenQA.Selenium; -using OpenQA.Selenium.IE; -using OpenQA.Selenium.Remote; - -InternetExplorerOptions browserOptions = new InternetExplorerOptions(); -browserOptions.PlatformName = "Windows 11"; -browserOptions.BrowserVersion = "11.0"; - -Dictionary ltOptions = new Dictionary(); -ltOptions.Add("username", Environment.GetEnvironmentVariable("LT_USERNAME")); -ltOptions.Add("accessKey", Environment.GetEnvironmentVariable("LT_ACCESS_KEY")); -ltOptions.Add("project", "IE Mode Test"); -ltOptions.Add("w3c", true); -browserOptions.AddAdditionalOption("LT:Options", ltOptions); - -IWebDriver driver = new RemoteWebDriver( -new Uri("https://hub.lambdatest.com/wd/hub"), browserOptions); -driver.Navigate().GoToUrl("https://example.com"); - -Console.WriteLine("Page title: " + driver.Title); -driver.Quit(); -``` - -## Key Configuration Options - -Review the IE Mode capabilities and their expected values. - -| Capability | Value | Description | -|---|---|---| -| `platformName` | `Windows 11` | IE Mode runs on Windows only. | -| `browserVersion` | `11.0` | Set to `11.0` to indicate IE11 compatibility mode. | -| `w3c` | `true` | Enables the W3C WebDriver protocol. | - -## Common Issues and Solutions - -Troubleshoot frequent problems when running IE Mode tests. - -### Elements not interactable in IE Mode -IE Mode uses the Trident rendering engine, which handles focus and click events differently from modern browsers. Add explicit waits before interacting with elements. Avoid CSS selectors that rely on modern pseudo-classes not supported by IE11. - -### Session timeout errors -IE Mode sessions can take longer to initialize than standard browser sessions. Increase your session timeout in `LT:Options` if you encounter timeout errors during startup. - -IE Mode does not support all Selenium features available in modern browsers. Features like shadow DOM access, modern CSS selectors, and certain JavaScript APIs are unavailable when the page renders through the Trident engine. diff --git a/static/docs/image-analyzer.md b/static/docs/image-analyzer.md deleted file mode 100644 index c37b7025b..000000000 --- a/static/docs/image-analyzer.md +++ /dev/null @@ -1,41 +0,0 @@ -# Image Analyzer Testing With TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The Agent Testing Platform tests an image agent by submitting prompts or images and scoring the returned output against the prompt and any criteria you define. Each image gets a Quality Score from 0 to 100, plus a breakdown of what matched and what did not. It covers image generation, content moderation, and photo validation. - -You upload images by file or URL, define what a correct image looks like, and the platform scores them. A single run analyzes up to 50 images, which suits regression testing after a model update. - -## Features - -**Image Analysis.** Upload single images or batch-process up to 50 at once, by file upload, URL, or drag and drop. Supported formats are JPG, JPEG, PNG, GIF, WEBP, and BMP, with a maximum of 20 MB per image. - -**Custom Evaluation Criteria.** Score images against your own rules, in three types, each toggleable active or inactive: - -- **Brand guidelines:** allowed and prohibited colors, required fonts, and logo requirements. -- **Technical specifications:** dimensions (width by height), aspect ratio, allowed formats, maximum file size, and minimum resolution. -- **Custom rules:** freeform rule text and checklist items. - -All criteria support create, edit, delete, and search by name, description, or type. - -**Analysis History.** Search past analyses by image name or prompt, track status (Pending, Completed, Failed), open any analysis for full results, and bookmark important ones. - -**Analytics Dashboard.** View overall statistics (average, highest, and lowest score, and total count), a 30-day quality trend with a daily bar chart, and the top 20 prompts ranked by average score. - -## Metrics - -Each image is scored on a single Quality Score from 0 to 100, plus a set of qualitative outputs. - -| Metric | Scale | What it measures | -|--------|-------|------------------| -| **Quality Score** | 0 to 100 | Overall image quality and prompt adherence | -| **Matches** | List | Elements that correctly match the original prompt | -| **Discrepancies** | List | Missing or incorrect elements versus the prompt | -| **Overall Assessment** | Text | Summary of how well the image matches the prompt | -| **Detailed Observations** | Text | In-depth analysis of specific image aspects | - -Quality Score bands: 90 to 100 excellent, 80 to 89 good, 60 to 79 fair, and 0 to 59 poor. For each active custom criterion, results show a Pass, Fail, or Partial status with compliance details. - -## Related TestMu AI Guides - -- See how the platform [runs an evaluation end to end](/support/docs/architecture-and-how-evaluation-works/). diff --git a/static/docs/inbound-phone-agent.md b/static/docs/inbound-phone-agent.md deleted file mode 100644 index 07c698ea7..000000000 --- a/static/docs/inbound-phone-agent.md +++ /dev/null @@ -1,121 +0,0 @@ -# Inbound Phone Agent Testing With TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -An inbound phone agent answers calls. To test one, the Agent Testing Platform places a real call to the agent's phone number, and a simulated caller drives the scenario. Typical use cases are IVR menus, inbound support, appointment scheduling, and billing. - -Inbound testing runs in two modes: **pre-evaluation** with live simulated calls, and **post-evaluation** on recordings from real production calls. - -## Pre-Evaluation: Live Test Calls - -In pre-evaluation, the platform simulates customers calling your voice agent, then evaluates the resulting conversations. - -**Phone Number Management.** Register the numbers your agent answers on, with country code selection (20+ countries), a default number, masked display, and edit or delete. - -**Scenario Management.** Generate up to 20 inbound scenarios with configurable personas, languages, and special instructions, or create them manually. Choose from available personas or create custom ones. - -**Voice Configuration (per scenario).** Select a voice from the library with audio preview, enable one of 15 background-noise presets, set the response timing (0.5 to 5.0 seconds) and a maximum call duration (60 to 1800 seconds), and choose who speaks first. - -**Agent Profiles.** Create reusable caller personas with name, phone number, voice, and background noise, stored in an organization-level library with an active or inactive toggle. - -**Test Suites.** Group scenarios with per-scenario voice and phone configuration, associate test and agent profiles, and run the whole suite with one action. - -**Call Execution and Monitoring.** Initiate live test calls, track status in real time, watch a live duration counter, and terminate a call in progress. - -## Post-Evaluation: Recording Analysis - -In post-evaluation, you upload recordings from real production calls and score them with the same metrics, without placing new calls. - -**Voice Analytics.** Upload production recordings (MP3, WAV) and transcripts, analyze them in parallel batches, select which metric categories or individual metrics to run, and bookmark, tag, search, and filter recordings. - -**Recording Playback.** Play any call with play, pause, and duration controls, follow a speaker-identified transcript, see DTMF keypad inputs (0 to 9, star, pound) captured in the transcript, and download the audio and transcript. - -## Shared Across Both Modes - -**Go-Live Assessment.** Get a Green (score at least 80), Yellow (65 to 79), or Red (below 65) verdict, with confidence based on call volume, dimension scores, scenario coverage, failure pattern analysis, validation-criteria compliance, and prioritized action items. - -**Metric Configuration.** Select which metric categories or individual metrics to run per project. - -**Scheduled Runs.** Automate runs with cron-based scheduling, IANA timezones, pause and resume, and run history. - -## Metrics - -Phone agents are evaluated across 8 metric categories with 30+ individual metrics. - -**A. Conversation Flow and Interaction Dynamics** - -| Metric | Unit | What it measures | -|--------|------|------------------| -| Average Latency | ms | Time to respond after the user stops speaking | -| Words Per Minute | wpm | Agent speaking speed | -| AI Talk Ratio | % | Share of call time the agent is speaking | -| User Talk Ratio | % | Share of call time the user is speaking | -| AI Interrupting User | % | How often the agent interrupts the user | -| User Interrupting AI | % | How often the user interrupts the agent | - -**B. Accuracy and Effectiveness** - -| Metric | Unit | What it measures | -|--------|------|------------------| -| First Call Resolution | % | Whether the issue was resolved in a single call | -| Intent Recognition Accuracy | % | How accurately the agent understood intent | -| Task Completion Success Rate | % | Share of assigned tasks completed | -| Instruction Following | % | Adherence to configured instructions | -| Response Consistency | % | Consistency of responses to similar inputs | - -**C. User Experience and Satisfaction** - -| Metric | Unit | What it measures | -|--------|------|------------------| -| CSAT | % | Overall customer satisfaction score | -| CSAT Reason | Text | Explanation for the satisfaction score | -| User Sentiment | Text | Detected emotional sentiment from user speech | -| Early Termination | % | Share of calls not terminated prematurely | - -**D. Business Operational Metrics** - -| Metric | Unit | What it measures | -|--------|------|------------------| -| Containment Rate | % | Issues resolved without human escalation | -| AI to Human Handoff Rate | % | Frequency of escalation to a human agent | - -**E. Audio Voice Quality** - -| Metric | Unit | What it measures | -|--------|------|------------------| -| Average Pitch | Hz | Voice pitch (normal: 85 to 300 Hz) | -| Voice Quality Index | 0 to 5 | Composite voice quality score | -| Signal-to-Noise Ratio | % | Audio clarity versus background noise | - -**F. Speech-to-Text Evaluation** - -| Metric | Unit | What it measures | -|--------|------|------------------| -| STT Accuracy | % | Transcription accuracy | -| STT Verdict | Pass/Fail | Overall transcription quality judgment | -| STT Summary | Text | Detailed transcription quality notes | -| Mismatch Examples | List | Instances where transcription differed from speech | - -**G. Validation Results** - -| Metric | Unit | What it measures | -|--------|------|------------------| -| Compliance | % | Compliance rate against custom validation criteria | -| Pass/Fail/Unable to Verify | Count | Per-criterion validation breakdown | - -**H. Detected Issue Tags (automated).** Every recording is auto-scanned for: latency issues, hallucination in call flow, transcript issues, patchy audio, running in a loop, incorrect STT, interruption handling, number issues, background noise, no response, and blank or empty STT. - -**Threshold reference** - -| Metric | Excellent | Good | Poor | -|--------|-----------|------|------| -| Average Latency | at most 1000 ms | at most 2500 ms | over 2500 ms | -| Words Per Minute | at least 160 (fast) | 131 to 160 | under 110 (slow) | -| Voice Quality Index | at least 2.5 / 5 | n/a | under 2.5 / 5 | -| Average Pitch | 85 to 300 Hz | n/a | under 85 or over 300 Hz | - -## Related TestMu AI Guides - -- See the [phone agent testing overview](/support/docs/phone-agent/) for both testing modes. -- See how to [test an outbound phone agent](/support/docs/outbound-phone-agent/). -- See how to [run these tests from the terminal](/support/docs/testmu-a2a-cli/) with the A2A CLI. diff --git a/static/docs/inside-testmu-platform.md b/static/docs/inside-testmu-platform.md deleted file mode 100644 index 13d98e31f..000000000 --- a/static/docs/inside-testmu-platform.md +++ /dev/null @@ -1,62 +0,0 @@ -# Explore the Automation Dashboard - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) is where you view results, debug failures, and track every test you run on the grid. This page walks you through each section of the dashboard. - -## Build and Test Views - -Filter and browse your tests using either the Build view or the Test view. - -The dashboard groups your tests by build. Click any build to see its individual test cases. You can switch between Build and Test views depending on whether you want to see results grouped by build or listed individually. - -## Test Detail Page - -Click any test to open its detail page with full execution data. - -Each test detail page shows the build name, test status (pass/fail), execution time, browser and OS configuration, resolution, and framework used. - -### Test List Panel - -The left panel lists all test cases within a build. Each entry shows the test status, execution time, and configuration. - -You can sort by date, status, or duration. Use the search bar to find a specific test, or filter by browser, OS, status, or tags. - -### Test Summary - -The summary section gives you a quick overview of the selected test. - -It displays: -- **Build name** and **test name** -- **Status** - passed or failed -- **Execution time** -- **Configuration** - resolution, browser, OS, framework -- **Last updated** timestamp - -Use the action buttons next to the test ID to copy the ID, create an issue, rename, share, or delete the test. - -Use the **Re-Run** button to execute the test again from the dashboard. See [Re-Run Automation Tests](/support/docs/rerun-automation-tests/). - -### Command Logs and Network - -View every command executed during the test and all associated logs. - -The right panel contains three tabs: - -- **All Commands** - Every Selenium command that ran during the test. Hover on a command to see its parameters. -- **Logs** - Toggle between Console Logs, Terminal Logs, and Test Framework Logs. -- **Network** - View all network requests and responses captured during execution. - -### Video Recording - -Watch a full video playback of your test execution to see exactly what happened in the browser. - -## Next Steps - -Now that you know how to read the dashboard, explore more ways to manage and debug your tests. - -- [Filter your tests](/support/docs/filter-your-selenium-tests/) - narrow down results by browser, OS, status, or tags -- [Share test results](/support/docs/sharing-test-results/) - send results to teammates -- [Mark as bug](/support/docs/mark-as-bug-in-automation-testing/) - file a bug directly from a test session -- [Debug your tests](/support/docs/debugging-options/) - enable extended logging for deeper analysis -- [Group tests with tags](/support/docs/group-tests-using-custom-tags/) - organize tests using custom tags diff --git a/static/docs/insights-app-profiling.md b/static/docs/insights-app-profiling.md deleted file mode 100644 index 6ef568933..000000000 --- a/static/docs/insights-app-profiling.md +++ /dev/null @@ -1,342 +0,0 @@ -# App Profiling Insights - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Overview - -The App Profiling dashboard provides comprehensive performance metrics to help you identify bottlenecks and optimize your application. This documentation explains each widget's purpose and how to interpret the data to improve app performance. - -The data on this dashboard comes from the App Profiling capability on Appium tests. To enable App Profiling on your test runs and review per-session metrics, see [App Performance Analytics](/support/docs/appium-app-performance-analytics/). - -## How to access - -1. Open the **Insights** section from the left sidebar of the TestMu AI dashboard. -2. Select **App Profiling**. The landing page lists every test that has App Profiling data. -3. Click a test name to open its dashboard. - -The page is always scoped to the test you opened. Use the **Test Name** filter to add more tests to the comparison without leaving the dashboard or reset to revert. - -## Filter bar - -The filter bar at the top of the dashboard applies to every widget on the page. - -| Filter | Values | Default | -|---|---|---| -| **Test Name** | Multi-select, up to 25 tests | The test you opened (pre-selected) | -| **OS** | Android, iOS | All | -| **Device** | Specific device models recorded across the selected tests | All | -| **App Build Version** | Build versions captured during the sessions | All | -| **Page Label** | Page transition labels recorded by the test | All | -| **Date Range** | Preset windows or a custom range with a time-of-day picker | Last 30 days | - -## Compare mode - -Each per-metric trend widget and Label Page Load Time supports **Compare** mode. Click **Compare** in the widget header, pick a dimension, then select up to five values to overlay on the chart. - -Compare dimensions: - -- **OS**: Android vs iOS -- **Device**: for example Galaxy S23 vs Pixel 8 vs iPhone 15 Pro -- **App Build Version**: for build-over-build comparisons -- **Label**: for page-transition-level breakdowns - -Compare is scoped to the widget. Enabling it on one chart does not affect others. Performance Overview, Performance Trends and Device Performance Matrix do not expose Compare: the first two are summary widgets (use filters to change the data scope instead), and the matrix already breaks data down per device. - -## SLA thresholds - -SLA thresholds are configured at the **organisation level** by an admin. Once set, the same threshold is applied everywhere a metric is rendered: Performance Overview cards, Performance Trends overlays, Device Performance Matrix cells, Label Page Load Time, and the per-metric trend widgets. - -Default thresholds: - -| Metric | Green(Ok) | Amber(Warning) | Red(Critical) | -|---|---|---|---| -| CPU (App) | < 15% | 15–30% | > 30% | -| Memory (App) | < 300 MB | 300–400 MB | > 400 MB | -| FPS | > 50 | 30–50 | < 30 | -| Cold Startup | < 2000 ms | 2000–3000 ms | > 3000 ms | -| Hot Startup | < 500 ms | 500–1000 ms | > 1000 ms | -| Battery Temperature | < 40 °C | 40–45 °C | > 45 °C | -| Label Page Load | < 2.5 s | 2.5–3.0 s | > 3.0 s | - -Thresholds render as: - -- Horizontal reference lines on trend charts. -- Coloured background bands behind the chart area. -- Cell-level heatmap colouring on Device Performance Matrix. -- Coloured value text on Performance Overview cards. - -Admins can override the defaults at the organisation level. Regular users see the configured thresholds applied but cannot modify them. - -## Understanding Performance Metrics Widgets - -### 1. Performance Overview - -A strip of KPI cards summarising the headline metrics over the selected window. - -- **Avg CPU (%)**: application CPU usage averaged across sessions in scope. -- **Avg Memory (MB)**: application memory usage averaged across sessions in scope. -- **Avg FPS**: frames per second averaged across sessions in scope. -- **Avg Cold Startup (ms)**: cold-start duration averaged across sessions in scope. -- **Avg Crashes per Crashed Session**: `avg(crash_count)` over sessions that reported at least one crash. This is the mean crash count among already-crashed sessions, not a crash rate. - -> Cards are hidden automatically when the underlying metric are not recorded for the test session. - -### 2. Performance Trends - -A single time-series chart that overlays selected metrics on dual Y-axes. The **KPIs** selector in the widget header controls which series are visible: CPU, FPS, Memory, Temperature, Battery, Network Upload and Network Download are available; CPU and Memory are on by default. Each visible KPI gets a Min / Max / Avg row in the stats panel below the chart. - -### 3. Device Performance Matrix - -A table comparing performance metrics across the devices that ran in the filtered scope. One row per device. - -| Column | Description | -|---|---| -| **Device** | Device model | -| **CPU App (%)** | Application CPU usage, colour-coded against the configured SLA threshold | -| **CPU System (%)** | System-level CPU usage | -| **Memory App (MB)** | Application memory, colour-coded against the configured SLA threshold | -| **Memory System (MB)** | System-level memory | -| **FPS** | Frame rate, colour-coded against the configured SLA threshold | -| **Sessions** | Session count for that device in the filtered scope | - -Click any column header to sort. The dashboard-level **Device** filter intentionally does not narrow this widget. The matrix is itself the device breakdown. - -### 4. Label Page Load Time - -Duration per page transition label, captured from the `label` events recorded inside the test. Each visible label is plotted as a horizontal bar showing its average duration; the widget shows the top labels by session count by default, and a label selector inside the widget lets you add or remove labels from the chart. - -SLA thresholds render as vertical green/amber/red bands behind the bars, with the configured threshold drawn as a dashed reference line. Bars that extend into the red band are out of SLA. The stats panel below the chart reports Min / Max / Avg duration across the visible labels. - -### 5. CPU Utilization Trend - -**Widget Purpose:** -This graph tracks CPU consumption over time, separating system-level processing from app-specific usage. - -**Metrics Explained:** -- **CPU System (%)**: Total system CPU resources being used -- **CPU App (%)**: CPU resources specifically consumed by your application - -**How to Analyze:** -- Look for consistent app CPU usage above 15-20%, which may indicate inefficient algorithms or background tasks -- Watch for correlations between high CPU usage and other performance issues -- Identify patterns: gradual increases could signal memory leaks; sudden spikes might indicate intensive operations - -**Optimization Strategies:** -- Offload intensive tasks to background threads -- Implement lazy loading for resource-heavy features -- Use caching mechanisms for repetitive operations -- Optimize algorithms with high computational complexity -- Consider using more efficient data structures - -### 6. Frame Rate Trends - -**Widget Purpose:** -Visualizes rendering performance, highlighting both smooth operation and problematic frames. - -**Metrics Explained:** -- **Current FPS**: Frames rendered per second (higher is better, 60+ ideal for smooth animation) -- **Jank Frames**: Frames taking excessive time to render, causing visual stuttering -- **Frozen Frames**: Completely dropped frames that cause noticeable pauses - -**How to Analyze:** -- Identify sections where FPS consistently drops below target (60 FPS for most applications) -- Pay attention to clusters of jank frames, which indicate UI thread blockage -- Look for correlations between frozen frames and specific user interactions - -**Optimization Strategies:** -- Move heavy operations off the UI thread -- Simplify complex layouts and view hierarchies -- Reduce overdraw by flattening view layers -- Optimize or defer expensive drawing operations -- Implement hardware acceleration where appropriate -- Use profiling tools to identify specific rendering bottlenecks - -### 7. Memory Usage - -**Widget Purpose:** -Monitors memory allocation patterns to identify potential leaks and inefficient resource usage. - -**Metrics Explained:** -- **App Memory (MB)**: Direct memory consumed by your application -- **System Memory (MB)**: Overall system memory usage - -**How to Analyze:** -- Look for steadily increasing app memory over time (indicates potential leaks) -- Identify memory spikes during specific operations -- Pay attention to how memory behaves after screen transitions or intensive tasks -- Watch for memory that doesn't return to baseline after operations complete - -**Optimization Strategies:** -- Implement proper object lifecycle management -- Use weak references for observer patterns -- Optimize image loading and caching -- Dispose of resources when moving between screens -- Consider using object pools for frequently created/destroyed objects -- Implement memory leak detection in development builds - -### 8. Battery Utilization - -**Widget Purpose:** -Tracks energy consumption to identify processes that may drain battery excessively. This data is for Android only and rendered only when the session recorded battery samples - -**Metrics Explained:** -- **Battery (mAh)**: Energy consumption rate in milliampere-hours - -**How to Analyze:** -- Identify patterns of high battery usage -- Look for correlation between battery drain and specific app features -- Compare battery usage across different app states (active, background, idle) - -**Optimization Strategies:** -- Optimize network calls (batch requests, compress data) -- Reduce location service usage when not essential -- Implement efficient background processing -- Optimize wake locks and sensor usage -- Use dark mode or darker UI for OLED screens -- Batch CPU-intensive operations - -### 9. Network Utilization - -**Widget Purpose:** -Monitors data transfer to identify inefficient network operations. - -**Metrics Explained:** -- **Network Upload (KB)**: Data sent by the application -- **Network Download (KB)**: Data received by the application - -**How to Analyze:** -- Look for unexpected or excessive data transfers -- Identify patterns in network activity (constant polling vs. event-based) -- Compare network usage against expected data requirements - -**Optimization Strategies:** -- Implement efficient caching strategies -- Compress network payloads -- Use incremental data loading -- Optimize API requests (GraphQL, partial responses) -- Batch network requests when possible -- Implement offline capabilities -- Use efficient image formats and compression - -### 10. Battery Temperature - -**Widget Purpose:** -Measures device thermal performance to identify processes causing overheating. - -**Metrics Explained:** -- **Avg. Temperature (°C)**: Current average temperature of the device - -**How to Analyze:** -- Track temperature increases during specific operations -- Identify features that consistently raise device temperature -- Look for sustained high temperatures that could lead to thermal throttling - -**Optimization Strategies:** -- Optimize CPU-intensive algorithms -- Reduce sustained high CPU/GPU operations -- Space out intensive tasks rather than running simultaneously -- Implement adaptive performance based on device temperature -- Consider lower quality graphics or processing when temperature is high - -### 11. Cold Startup Time - -**Widget Purpose:** -Measures application launch performance from a completely shut down state. - -**Metrics Explained:** -- **Cold Startup (ms)**: Time taken to launch app when not in memory - -**How to Analyze:** -- Look for consistently high startup times -- Identify variations in startup performance -- Compare against industry benchmarks (1-2 seconds is typically acceptable) - -**Optimization Strategies:** -- Implement lazy initialization of non-critical components -- Defer heavy operations until after UI is visible -- Optimize database and storage access during startup -- Use app startup libraries to manage initialization -- Consider using a splash screen for perceived performance -- Reduce app dependencies and initialization chain - -### 12. Hot Startup Time - -**Widget Purpose:** -Measures application launch performance when re-opening from background. - -**Metrics Explained:** -- **Hot Startup (ms)**: Time taken to resume app when already in memory - -**How to Analyze:** -- Look for hot startup times exceeding 500ms -- Identify inconsistency in resume performance -- Compare against cold startup to ensure significant improvement - -**Optimization Strategies:** -- Optimize saved state management -- Implement efficient view restoration -- Use lightweight persistence mechanisms -- Consider UI state caching strategies -- Prioritize restoring visible elements first - -## Advanced Analysis Techniques - -### Correlation Analysis - -To gain deeper insights, analyze relationships between different metrics: - -1. **CPU vs. Frame Rate**: High CPU often correlates with frame drops -2. **Memory vs. Startup Time**: Increasing memory usage may slow startup -3. **Battery vs. Network**: Excessive network activity typically increases battery consumption -4. **Temperature vs. Performance**: High temperatures often lead to throttling and reduced performance - -### Benchmark Comparison - -Establish baseline metrics for your application: - -1. **Competitor Analysis**: Compare your metrics against similar apps -2. **Version Comparison**: Track metrics across your app versions -3. **Device Variation**: Compare performance across different device models -4. **User Scenario Testing**: Create specific user flows and measure performance - -### Performance Budgeting - -Set target thresholds for critical metrics: - -1. **Startup Budget**: Cold start < 2 seconds, hot start < 500ms -2. **Frame Rate Budget**: Maintain 60+ FPS during animations -3. **Memory Budget**: Keep peak memory under device-specific thresholds -4. **Network Budget**: Limit payload sizes and request frequency -5. **Battery Budget**: Limit battery consumption per hour of active use - -## Implementing Performance Improvements - -### Prioritization Framework - -When addressing performance issues: - -1. **User Impact**: Prioritize issues directly affecting user experience -2. **Frequency**: Address problems that occur most frequently -3. **Severity**: Focus on severe performance degradations first -4. **Complexity**: Balance effort required against potential improvements -5. **Business Impact**: Consider effects on retention, conversion, and engagement - -### Testing Methodology - -Verify improvements through rigorous testing: - -1. **A/B Testing**: Compare metrics between old and new implementations -2. **Progressive Rollout**: Deploy changes to a small percentage of users first -3. **Real-World Testing**: Test across various network conditions and devices -4. **Automated Performance Testing**: Implement CI/CD performance checks - -## Continuous Monitoring - -For ongoing performance optimization: - -1. **Real User Monitoring**: Collect performance data from production users -2. **Performance Regressions**: Set up alerts for metric degradations -3. **Periodic Audits**: Schedule regular performance reviews -4. **User Feedback Analysis**: Correlate performance metrics with user sentiment - -By leveraging these widgets and analysis techniques, you can systematically identify and address performance bottlenecks, resulting in a faster, more efficient, and battery-friendly application that provides an excellent user experience. diff --git a/static/docs/insights-best-practices.md b/static/docs/insights-best-practices.md deleted file mode 100644 index 8eb9565b1..000000000 --- a/static/docs/insights-best-practices.md +++ /dev/null @@ -1,103 +0,0 @@ -# Best Practices for TestMu AI Insights - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Unlock the full potential of TestMu AI Insights by following these best practices for dashboards, widgets, and insights modules. These guidelines are distilled from the current knowledge base and are designed to help you optimize your testing process, improve reporting, and drive actionable insights. - -## 1. Dashboard and Widget Design - -- **Define Clear Objectives:** Before creating dashboards or widgets, identify the key questions you want to answer or the metrics you need to track. Align dashboards with business and QA goals. -- **Leverage Pre-built Templates:** Use dashboard templates for common use cases (e.g., Test Summary, Error Insights, Resource Consumption) to save time and ensure standardized reporting. -- **Customize for Stakeholders:** Create separate dashboards for different teams (QA, Dev, Product) with relevant widgets and layouts. Use the layout editor to prioritize and group widgets based on stakeholder needs. -- **Optimize Layout:** Arrange high-priority widgets at the top, resize widgets for visibility, and group similar widgets together for clarity. Optimize dashboards for mobile if needed. - -**Use Case:** -A QA manager wants to monitor both test execution and resource usage. By creating a dashboard with high-priority widgets (like Test Summary and Concurrency Trends) at the top and grouping related widgets, the manager can quickly assess project health and resource allocation. This setup saves time and ensures that critical information is always visible, enabling faster decision-making. - -**How it helps:** -Well-designed dashboards reduce information overload, highlight key metrics, and make it easier for users to focus on what matters most. Customizing layouts for different stakeholders ensures everyone gets the insights they need in the most effective format. - -## 2. Data Visualization and Filtering - -- **Choose the Right Visualization:** Select bar, line, pie, table, or billboard widgets based on the data and the story you want to tell. Use trends and distribution charts for time-series and coverage analysis. -- **Apply Advanced Filters:** Use key, operator, and value filters to focus on relevant data. Filter by product, project, browser, OS, device, or custom fields to drill down into specific insights. -- **Use Custom Grouping and Labels:** Group data by relevant categories (e.g., test status, device type) and provide clear labels for axes and widgets to aid interpretation. - -**Use Case:** -A team needs to analyze test failures by browser and OS. By applying filters and grouping data, they can visualize failure rates for each environment using a bar chart. This helps them quickly identify problematic platforms and prioritize fixes. - -**How it helps:** -Effective visualization and filtering make complex data understandable, enabling users to spot trends, outliers, and actionable insights. Custom grouping and labels ensure clarity and relevance for all viewers. - -## 3. Performance and Resource Optimization - -- **Monitor Resource Utilization:** Use concurrency and resource widgets to track usage trends and optimize allocation. Identify bottlenecks and plan for future capacity. -- **Analyze Test Trends:** Regularly review test execution, pass/fail rates, and error patterns to identify areas for improvement and maintain test suite health. -- **Leverage Performance Profiling:** For app profiling, monitor CPU, memory, battery, network, and startup metrics. Use optimization strategies such as offloading heavy tasks, optimizing network calls, and managing memory efficiently. - -**Use Case:** -A DevOps engineer notices a spike in resource usage during nightly test runs. By monitoring concurrency widgets and analyzing trends, they identify inefficient test scheduling and adjust the pipeline to balance load, reducing costs and improving test reliability. - -**How it helps:** -Proactive resource and performance monitoring prevents bottlenecks, optimizes infrastructure costs, and ensures smooth, reliable test execution. - -## 4. Continuous Improvement and Collaboration - -- **Track and Share Insights:** Use dashboards to monitor progress, share insights with stakeholders, and support data-driven decision-making. Export and share reports as needed. -- **Iterate and Refine:** Continuously review dashboard effectiveness, update widgets, and refine filters as testing needs evolve. -- **Enable Collaboration:** Use dashboard sharing and permissions to foster collaboration between QA, Dev, and Product teams. - -**Use Case:** -A product team wants regular updates on test coverage and bug trends. The QA lead shares a dashboard with tailored widgets and exports weekly reports. As requirements change, the dashboard is updated to reflect new priorities. - -**How it helps:** -Continuous improvement ensures dashboards remain relevant and actionable. Sharing and collaboration break down silos, keeping all teams aligned and informed. - -## 5. Troubleshooting and Error Analysis - -- **Utilize Error Insights:** Use error-focused widgets and templates to quickly identify, analyze, and resolve test failures. Track error trends across browsers, OS, and devices. -- **Monitor Test Health:** Use health snapshot and status ratio widgets to assess test reliability and flakiness. Prioritize fixing frequently failing or flaky tests. - -**Use Case:** -A QA engineer sees a sudden increase in test failures. By using error insights and health snapshot widgets, they quickly pinpoint a new browser version as the cause and coordinate a fix. - -**How it helps:** -Rapid error identification and analysis reduce downtime, improve test reliability, and speed up the feedback loop for development teams. - -## 6. Custom Widgets and Advanced Use Cases - -- **Personalize Visualizations:** Create custom widgets for unique metrics or KPIs not covered by standard widgets. Use custom data sources and axes for tailored insights. -- **Compare Data Sources:** Switch between sources (e.g., Tests, Auto Heal) to analyze different aspects of your testing process. -- **Automate and Integrate:** Where possible, integrate analytics with CI/CD and automate report generation for continuous feedback. - -**Use Case:** -A team wants to track the impact of the Auto Heal feature on test stability. By creating a custom widget that compares healed vs. non-healed test outcomes, they can measure effectiveness and justify further investment. - -**How it helps:** -Custom widgets and integrations provide flexibility for unique business needs, enabling deeper analysis and automation for continuous improvement. - -## 7. Performance Monitoring (App Profiling) - -- **CPU & Memory:** Watch for high or increasing usage; optimize algorithms and resource management. -- **Battery & Network:** Minimize battery drain and network usage by batching operations and compressing data. -- **Startup Time:** Optimize cold and hot startup times by deferring non-critical operations and managing state efficiently. -- **Temperature:** Monitor device temperature and adapt performance strategies to avoid throttling. - -**Use Case:** -A mobile app team uses profiling widgets to monitor memory and battery usage during feature development. They spot a memory leak early and optimize network calls, resulting in better app performance and user reviews. - -**How it helps:** -Performance monitoring ensures high-quality, efficient applications, reduces user complaints, and supports proactive optimization. - -## 8. General Recommendations - -- **Stay Updated:** Regularly check for new widgets, features, and best practices in the TestMu AI documentation. -- **Seek Support:** Contact TestMu AI support for help with advanced configurations or troubleshooting. - -**Use Case:** -A QA lead stays informed about new analytics features and best practices, quickly adopting improvements that streamline reporting and boost team productivity. - -**How it helps:** -Staying current and seeking support ensures you always get the most value from TestMu AI Analytics and can resolve issues efficiently. - -By following these best practices, you can maximize the value of TestMu AI Analytics, drive continuous improvement, and deliver high-quality, data-driven testing outcomes. diff --git a/static/docs/insights-dashboard.md b/static/docs/insights-dashboard.md deleted file mode 100644 index e3eee8d06..000000000 --- a/static/docs/insights-dashboard.md +++ /dev/null @@ -1,24 +0,0 @@ -# Insights Dashboard - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The insights section provides a comprehensive project level data overview of key Test Manager features, including Projects, Test Plans, Builds, and Test Cases, available at both the project and organizational levels. - -1. The insights is segregated on various parameters such as - -- **Total Test Cases:** Provides a count of all test cases created. -- **Manual Test Cases** Number of manual test cases -- **Automated Test Cases** Number of automated test cases. -- **Automation Coverage** Displays the percentage of test cases that are automated. - -2. You can filter test cases based on specific date ranges. For instance, you can view data from July 8th to July 25th to analyze trends over that period. - -3. **Build Summary** provides details of test cases based on status that is whether the test cases have passed, Failed, Skipped or Not Started. - -4. **Test Case Summary** provides information on the type of testing that has been chosen for different test cases. - -5. **Test Case Trend** graphically represents the count of test cases created over the selected duration, offering a visual insight into test case progression and status. - -6. **Issues Trend** provides an overview of issues linked to all test cases over the selected time frame. This helps in identifying areas that may require attention. - -These insights help in monitoring and improving the testing process, ensuring comprehensive coverage and efficient test management. diff --git a/static/docs/insights-private-desktop.md b/static/docs/insights-private-desktop.md deleted file mode 100644 index d0842ca2e..000000000 --- a/static/docs/insights-private-desktop.md +++ /dev/null @@ -1,49 +0,0 @@ -# Private Desktop Insights - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI Private desktop Insights provide comprehensive insights into your private desktop testing efforts. These insights help you optimize your testing efforts by providing detailed insights into your desktop device usage insights, and more. - -The Private desktop Insights is currently in . If you have any feedback or suggestions, please feel free to reach out to us at [support@testmuai.com](mailto:support@testmuai.com). - -## Device Usage Insights - -The Device Usage Insights is a powerful widget that provides a comprehensive overview of your private desktop testing efforts. It displays the total number of devices used, the total number of tests run, and the total duration of devices in use. - -### How it works? - -* The Device Usage Insights widget provides a comprehensive overview of your private desktop testing efforts. -* The widget helps you optimize your testing efforts by providing detailed insights into your desktop device usage insights. - -### Value Proposition - -This widget helps you understand the overall device usage status of your private desktop. It provides a detailed breakdown of the devices used, the devices available, and the devices in use. - -## Usage by Users, Groups and Sub-Organizations - -The Usage by Users, Groups, and Sub-Organizations widget provides a comprehensive overview of your private desktop testing efforts. It displays the total number of users, groups, and sub-organizations using the private desktop. - -### How it works? - -* This widget helps you understand the overall usage status of your private desktop. -* It provides a detailed breakdown of the users, groups, and sub-organizations using the private desktop. - -### Value Proposition - -This widget helps you optimize your testing efforts by providing detailed insights into your private desktop usage by users, groups, and sub-organizations. - -## Filters - -We provide a range of filters to help you customize your insights dashboard. You can filter your insights based on the following parameters: - -* Date Range - Filter your insights based on a specific date range. -* Users - Filter your insights based on specific users. -* Host Name - Filter your insights based on specific host names. -* Browsers - Filter your insights based on specific browser names. -* OS - Filter your insights based on specific device platforms. -* Project Name - Filter your insights based on specific project names. -* Group Name - Filter your insights based on specific group names. - -## Support - -If you have any questions or need further assistance, please feel free to reach out to our [24/7 customer support](mailto:support@testmuai.com). Our team of experts is always here to help you with your testing efforts. diff --git a/static/docs/insights-private-real-devices.md b/static/docs/insights-private-real-devices.md deleted file mode 100644 index cbc2887c9..000000000 --- a/static/docs/insights-private-real-devices.md +++ /dev/null @@ -1,60 +0,0 @@ -# Private Real Devices Insights - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI Private Real Devices Insights provide comprehensive insights into your private real devices testing efforts. These insights help you optimize your testing efforts by providing detailed insights into your real devices device usage insights, and more. - -The Private Real Devices Insights is currently in . If you have any feedback or suggestions, please feel free to reach out to us at [support@testmuai.com](mailto:support@testmuai.com). - -## Device Usage Insights - -The Device Usage Insights is a powerful widget that provides a comprehensive overview of your private real devices testing efforts. It displays the total number of devices used, the total number of tests run, and the total duration of devices in use. - -### How it works? - -* The Device Usage Insights widget provides a comprehensive overview of your private real devices testing efforts. -* The widget helps you optimize your testing efforts by providing detailed insights into your real devices device usage insights. - -### Value Proposition - -This widget helps you understand the overall device usage status of your private real devices. It provides a detailed breakdown of the devices used, the devices available, and the devices in use. - -## Usage by Users, Groups and Sub-Organizations - -The Usage by Users, Groups, and Sub-Organizations widget provides a comprehensive overview of your private real devices testing efforts. It displays the total number of users, groups, and sub-organizations using the private real devices. - -### How it works? - -* This widget helps you understand the overall usage status of your private real devices. -* It provides a detailed breakdown of the users, groups, and sub-organizations using the private real devices. - -### Value Proposition - -This widget helps you optimize your testing efforts by providing detailed insights into your private real devices usage by users, groups, and sub-organizations. - -## Filters - -We provide a range of filters to help you customize your insights dashboard. You can filter your insights based on the following parameters: - -* Date Range - Filter your insights based on a specific date range. -* Users - Filter your insights based on specific users. -* Test Name - Filter your insights based on specific test names. -* Build Name - Filter your insights based on specific build names. -* Tags - Filter your insights based on specific tags. -* UUID - Filter your insights based on specific UUIDs of devices. -* Device Name - Filter your insights based on specific device names. -* Device OS - Filter your insights based on specific device platforms. -* Test Status - Filter your insights based on specific test statuses. -* Project Name - Filter your insights based on specific project names. - -## Custom Widgets for Private Devices - -The widgets above provide standard, pre-built dashboards for private real device insights. If you need more tailored views (such as per-device test counts, duration breakdowns by product, or bar chart comparisons), you can build **custom widgets** using the Custom Widget builder. - -All custom widgets for private devices use the filter `is_dedicated = Equals = true` to scope data to your dedicated devices only. - -For step-by-step instructions and recommended widget configurations, see [Custom Widgets: Private/Dedicated Device Widgets](/support/docs/dashboards-custom-widgets/#privatededicated-device-widgets). - -## Support - -If you have any questions or need further assistance, please feel free to reach out to our [24/7 customer support](mailto:support@testmuai.com). Our team of experts is always here to help you with your testing efforts. diff --git a/static/docs/insights-usage-report.md b/static/docs/insights-usage-report.md deleted file mode 100644 index 2a30d6fed..000000000 --- a/static/docs/insights-usage-report.md +++ /dev/null @@ -1,51 +0,0 @@ -# TestMu AI products usage insights - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI Usage Report provides comprehensive insights into your product usage. These insights help you optimize your testing efforts by providing detailed insights into your product usage, test count, and more. - -The Usage Report is currently in . If you have any feedback or suggestions, please feel free to reach out to us at [support@testmuai.com](mailto:support@testmuai.com). - -## Usage Frequency per product - -The usage frequency widget helps you in understanding the usage of each product. It provides insights into the number of times each product has been used in a given time frame. This helps you in understanding the popularity of each product and helps you in optimizing your testing efforts. - -### How it works? - -* Check for the product subscription you want to view the usage frequency for. -* Select the time frame for which you want to view the usage frequency. -* The widget will display the usage frequency for the selected product and time frame. -* Further filter the data by selecting the project, browser, operating system, etc. - -### Value Proposition - -The usage frequency helps in understand which of the products are being used more frequently. This helps in optimizing your testing efforts by focusing on the products that are being used more frequently. - -## Total Duration Spent per product - -The total duration spent widget provides insights into the total duration spent on each product. It helps you in understanding the time spent on each product and helps you in optimizing your testing efforts. - -### How it works? - -* Check for the product subscription you want to view the total duration spent for. -* Select the time frame for which you want to view the total duration spent. -* The widget will display the total duration spent for the selected product and time frame. -* Further filter the data by selecting the project, browser, operating system, etc. - -### Value Proposition - -The total duration spent helps in understanding the time spent on each product. This helps in optimizing your testing efforts by focusing on the products that are being used more frequently. - -## Unique Applications Insights - -The unique applications insights widget provides insights into the unique applications that have been tested. It helps you in understanding the number of unique applications that have been tested and helps you in optimizing your testing efforts. - -### How it works? - -* Select the time frame for which you want to view the unique applications insights. -* The widget will display the unique applications insights for the selected product and time frame. -* Further filter the data by selecting the project, browser, operating system, etc. - -### Value Proposition - -The unique applications insights help in understanding the number of unique applications that have been tested. This helps in optimizing your testing efforts by focusing on the unique applications that have been tested. diff --git a/static/docs/install-apps-from-app-center.md b/static/docs/install-apps-from-app-center.md deleted file mode 100644 index 4f4524da0..000000000 --- a/static/docs/install-apps-from-app-center.md +++ /dev/null @@ -1,22 +0,0 @@ -# How to Test Apps Installed via App Center? - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -**note** -This method of uploading applications will be deprecated on [31st March 2025](https://learn.microsoft.com/en-us/appcenter/retirement). - -TestMu AI enables developers to test their mobile applications and websites across a range of real devices and browsers. It allows you to install apps directly from the App Center to test them on real android or iOS devices, ensuring that they perform well in actual user environments. - -Let's dive in to learn how to test apps installed from App Center. - -**Step 1**: Login to your TestMu AI account. Visit **Real Device** from the left panel and navigate to the **App Testing**. - -**Step 2**: Choose the OS version of your choice. In this case lets choose **Android**. - -**Step 3**: From the uploading apps section choose **Install from App Center** and enter the **API Token**. - -**Step 4**: You will be able to view the **Configure App Center** detail box, enter the **API Token**, select the **Build** of your choice and click on **Sync**. - -**Step 5**: You can also **Reconfigure** and share the Builds with other team members. - -**Step 6**: Select the **Device** and **OS Version** of your choice to launch the test session. diff --git a/static/docs/install-apps-from-app-store.md b/static/docs/install-apps-from-app-store.md deleted file mode 100644 index 290e49d1c..000000000 --- a/static/docs/install-apps-from-app-store.md +++ /dev/null @@ -1,21 +0,0 @@ -# How to Test Apps Installed via App Store? - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The Apple App Store is a digital marketplace for purchasing and downloading mobile applications on iOS devices. TestMu AI allows you to install apps directly from the App Store and test them on real iOS devices, ensuring that they perform well in actual user environments. - -Let's dive in to learn how to test apps installed from App Store. - -**Step 1**: Login to your TestMu AI account. Visit **Real Device** from the left panel and navigate to the **App Testing**. - -**Step 2**: Next, choose the OS version. In this case lets choose **iOS**. - -**Step 3**: From the uploading apps section choose **Install from App Store**. - -**Step 4**: Select the **iOS device** and **resolution** of your choice to launch the test session. - -**Step 5**: Use your Apple ID to log into the App Store, install the app and begin testing. - -- If you can't find an app on the Apple App Store, ensure you're signed in with an Apple ID from the region where the app is published. For example, if the app is available only in India, sign in with an Indian Apple ID and search again. - -- App availability may also depend on the device's region. If the app isn't visible on devices from one region, try launching a device from the region where the app is available (for example, a US-region device for apps published only in the United States). diff --git a/static/docs/install-apps-from-firebase.md b/static/docs/install-apps-from-firebase.md deleted file mode 100644 index 3fc0f89a8..000000000 --- a/static/docs/install-apps-from-firebase.md +++ /dev/null @@ -1,64 +0,0 @@ -# How to Test Apps Installed via Firebase? - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI offers real device testing capabilities, enabling developers and QA teams to test on actual Android and iOS devices in the cloud. With Firebase App Distribution, you can easily configure your Firebase account within TestMu AI to seamlessly distribute apps from Firebase to TestMu AI App Live. This integration allows you to collaborate by sharing projects within your team on TestMu AI and test on real devices. - -Let’s dive in to learn how to test apps installed from Firebase. - -**Prerequisites:** - -Create a new project on Firebase and open it in Google Cloud Platform (GCP). Using a new project is preferred, though an existing one can be used. - -Here, create an [OAuth Client ID](https://developers.google.com/identity/protocols/oauth2). - -In this step, update the following URLs as - -| Input Field | URL | -|-----------------|-------------| -| JavaScript origins | https://applive.lambdatest.com | -| Redirect URIs| https://applive.lambdatest.com/app | - -Download the OAuth client config JSON file, or simply take note of the Project ID, Client ID, and Client Secret Key. - -Next, add an app, navigate to Firebase, and proceed to App Distribution to upload the application releases. - -Access to the project is granted only if- -1. You are the owner. -2. You are a tester for the app. -3. The app is published. - -These settings can be updated in the OAuth consent screen. - -## Steps to Test apps: - -**Step 1:** -Login to your TestMu AI account. Visit **Real Device** from the left panel and navigate to **App Testing** and click on Install from Firebase. - -**Step 2:** -When you click on Add Project, you can integrate your Firebase account with TestMu AI either by uploading a **config file** or connecting **using credentials**. You should have these credentials from the prerequisite step. - -**Upload a Config file -** - -**Connect with credentials -** - -**Step 3:** -After entering these details, the option to **sign in with Google** will appear. Follow the steps and choose the Google account through which you would like to integrate Firebase with TestMu AI. - -**Step 4:** -You will have successfully integrated TestMu AI with Firebase. You can now click on **Share Now** to share the project with the testers you added in GCP. - -Projects can also be shared later directly from the menu. - -**Step 5:** -Now, you can view all the apps you’ve uploaded along with their respective versions, which can be synced with TestMu AI. Select the application, choose the version, pick the device for testing, and **start the session.** - -## Key Actions Overview - -1. **Add new project:** Click this button to add and sync new Firebase projects to your workspace. -2. **Refresh:** This button refreshes the project list, apps, and releases to their latest state without affecting your synced releases. -3. **Project Menu:** Click here to manage your project. You can share it with team members or delete it from the list. -4. **Sync:** Sync your release. This is necessary to start any session and generates a unique app ID for the release, which will be used for automation test cases. - -Please verify if your app is in testing mode. - -If it is, ensure that you add your email as a test user in GCP. This option is available in the 'Audience' section of GCP. diff --git a/static/docs/install-apps-from-play-store.md b/static/docs/install-apps-from-play-store.md deleted file mode 100644 index f9e82ece1..000000000 --- a/static/docs/install-apps-from-play-store.md +++ /dev/null @@ -1,21 +0,0 @@ -# How to Test Apps Installed via Play Store? - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI enables developers to test their mobile applications and websites across a range of real devices and browsers. It allows for direct installation of apps from the Google Play Store, facilitating testing of apps on real android devices to ensure optimal performance and compatibility. - -Let's dive in to learn how to test apps installed from Play Store. - -**Step 1**: Login to your TestMu AI account. Visit **Real Device** from the left panel and navigate to the **App Testing**. - -**Step 2**: Next, choose the OS version. In this case lets choose **Android**. - -**Step 3**: From the uploading apps section choose **Install from Play Store**. - -**Step 4**: Select the **Device** and **OS Version** of your choice to launch the test session. - -**Step 5**: Log in to the Play Store with your Google account to install and test your app. - -- If you can't find an app on the Google Play Store, ensure you're signed in with a Google account from the region where the app is published. For example, if the app is available only in India, sign in with an Indian Google account and search again. - -- App availability may also depend on the device's region. If the app isn't visible on devices from one region, try launching a device from the region where the app is available (for example, a US-region device for apps published only in the United States). diff --git a/static/docs/install-apps-from-test-flight.md b/static/docs/install-apps-from-test-flight.md deleted file mode 100644 index 430b182d2..000000000 --- a/static/docs/install-apps-from-test-flight.md +++ /dev/null @@ -1,17 +0,0 @@ -# How to Test Apps Installed via Test flight? - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI provides a seamless testing environment for beta versions of mobile applications on real iOS devices. With TestFlight preinstalled, developers can log into TestFlight, install their beta apps on iOS devices, and test before the apps are released on the App Store. This allows developers to effectively assess and improve app performance and user experience prior to launch. - -Let's dive in to learn how to test apps installed from Test Flight. - -**Step 1**: Login to your TestMu AI account. Visit **Real Device** from the left panel and navigate to the **App Testing**. - -**Step 2**: Next, choose the OS version. In this case lets choose **iOS**. - -**Step 3**: From the uploading apps section choose **Install from Test Flight**. - -**Step 4**: Select the **iOS device** and **resolution** of your choice to launch the test session. - -**Step 5**: Login to TestFlight with your Apple ID and password, install your app, and begin testing. diff --git a/static/docs/integrate-azure-pipelines-with-testmu.md b/static/docs/integrate-azure-pipelines-with-testmu.md deleted file mode 100644 index 1e6907873..000000000 --- a/static/docs/integrate-azure-pipelines-with-testmu.md +++ /dev/null @@ -1,109 +0,0 @@ -# Integrate Azure Pipelines With TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Azure Pipelines is a service presented by Azure DevOps to help projects ship faster with the help of a robust pipeline. With Azure Pipelines you can easily generate a build, test it before deploying the changes live in any language you are comfortable with i.e. Node.js, Python, Java, PHP, Ruby, C/C++, .NET etc. The platform offers unlimited CI/CD minutes for open-source projects, along with 10 free parallel jobs. Azure pipelines offers great cross-platform support using which you can run jobs in parallel across Windows, macOS, Linux. - -TestMu AI now integrates with Azure to boost your go-to-market delivery. Perform automated cross browser testing with TestMu AI to ensure your development code renders seamlessly through an online Selenium grid providing 3000+ real browsers running through machines, on the cloud. Perform automation testing in parallel with TestMu AI’s Selenium grid to drastically trim down your test cycles. - -## Pre-requisite - -1. An [Azure CI account](https://dev.azure.com/). - -2. A [Git or GitHub](https://github.com/) repository. - -3. **TestMu AI Authentication Credentials** - - Be aware of your TestMu AI authentication credentials i.e. your TestMu AI username, access key and HubURL. You need to set them up as your environment variables. You can retrieve them from your [TestMu AI automation dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/) by clicking on the key icon near the help button. - - * **For Linux/Mac:** - - $ export LT_USERNAME= `{YOUR_LAMBDATEST_USERNAME}` - - $ export LT_ACCESS_KEY= `{YOUR_LAMBDATEST_ACCESS_KEY}` - - * **For Windows:** - - $ set LT_USERNAME= `{YOUR_LAMBDATEST_USERNAME}` - - $ set LT_ACCESS_KEY= `{YOUR_LAMBDATEST_ACCESS_KEY}` - -## Integrating TestMu AI with Azure Pipelines - -> First and foremost, you need to clone our GitHub repository for downloading the relevant code needed to integrate [Azure Pipelines with LambdaTest](https://github.com/LambdaTest/protractor-selenium-azure-sample). - -**Step 1:** First and foremost, we need to create a project. Populate the relevant fields as you wish and hit the create button. - -**Step 2:** Now, Azure Pipelines will ask you about the location of your code. If you have your code repository in GitHub then select the option for GitHub from the values. - -**Step 3:** After you choose an option for your code repository, you will be asked to login. Post login, you need to choose the particular repository. The repository will pick the azure-pipelines.yml file. - -``` -# Node.js -# Build a general Node.js project with npm. -# Add steps that analyze code, save build artifacts, deploy, and more: -# https://docs.microsoft.com/azure/devops/pipelines/languages/javascript - -trigger: -- master - -pool: -vmImage: 'Ubuntu-16.04' - -variables: -LT_USERNAME: 'Your_LambdaTest_Username' -LT_ACCESS_KEY: 'Your_LambdaTest_Access_Key' - -steps: -- task: NodeTool@0 -inputs: -versionSpec: '10.x' -displayName: 'Install Node.js' - -- script: | -npm install -npm install -g protractor -cd conf -protractor single.conf.js -displayName: 'npm install and build' -``` - -**Step 4:** Executing the above file will run the script in TestMu AI Grid successfully. See the below image of the output. - -## Parallel Testing - -For running the test in parallel, replace protractor single.conf.js with protractor parallel.conf.js in same YML file. - -``` -# Node.js -# Build a general Node.js project with npm. -# Add steps that analyze code, save build artifacts, deploy, and more: -# https://docs.microsoft.com/azure/devops/pipelines/languages/javascript - -trigger: -- master - -pool: -vmImage: 'Ubuntu-16.04' - -variables: -LT_USERNAME: 'Your_LambdaTest_Username' -LT_ACCESS_KEY: 'Your_LambdaTest_Access_Key' - -steps: -- task: NodeTool@0 -inputs: -versionSpec: '10.x' -displayName: 'Install Node.js' - -- script: | -npm install -npm install -g protractor -cd conf -protractor parallel.conf.js //for executing parallel testing -displayName: 'npm install and build' -``` - -Monitor and analyze your test result on the TestMu AI Automation Dashboard. - -Deploy your code in a reliable manner at scale using Azure integration with TestMu AI, and ensure it looks robust across every browser to provide a seamless user experience to all your visitors. Happy Testing! diff --git a/static/docs/integrate-test-reporting-test-management-tools.md b/static/docs/integrate-test-reporting-test-management-tools.md deleted file mode 100644 index 96d4b6f5c..000000000 --- a/static/docs/integrate-test-reporting-test-management-tools.md +++ /dev/null @@ -1,25 +0,0 @@ -# Test Reporting & Test Management Tools - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI offers integration with test reporting and test management tools, allowing you to conveniently manage your test execution reports, as you perform cross browser testing using TestMu AI. Using the integration with your favourite test reporting and test management tools, you can view and analyze the results of your automated browser testing on 3000+ real browsers and operating systems, executed over TestMu AI. - -| [PractiTest Integration With TestMu AI](/docs/practitest-integration/) | -|:--------------------------------------------------------------------------------------------:| - -| [Test Rail Integration With TestMu AI](/docs/testrail-integration-with-testmu-selenium-grid/) | -|:--------------------------------------------------------------------------------------------:| - -| [Calliope Integration With LambdaTest](https://github.com/LambdaTest/LambdaTest-Calliope-Integration/tree/master) | -|:-----------------------------------------------------------------------------------------------------------------:| - -| [Zebrunner Integration With TestMu AI](/docs/zebrunner-integration/) | -|:-----------------------------------------------------------------------------------------------------------------:| - -| [Zephyr Scale Integration With TestMu AI](/docs/zephyr-scale-integration/) | -|:-----------------------------------------------------------------------------------------------------------------:| - -| [TestMo Integration With TestMu AI](/docs/testmo-integration/) | -|:-----------------------------------------------------------------------------------------------------------------:| - -If your favourite test reporting and test management tool is not in the above mentioned, then just give us a shout and we will have it ready shortly for you. diff --git a/static/docs/integrate-testmu-extension-with-azure-pipelines.md b/static/docs/integrate-testmu-extension-with-azure-pipelines.md deleted file mode 100644 index 455b142b5..000000000 --- a/static/docs/integrate-testmu-extension-with-azure-pipelines.md +++ /dev/null @@ -1,169 +0,0 @@ -# Integrate TestMu AI Extension With Azure Pipelines - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Azure Pipelines, by Microsoft, is a feature of Azure DevOps. It helps to automatically build, test, and deploy your projects, with Continuous Integration or CI, upon various production environments. So your Selenium automation test scripts can run successfully on Azure Pipeline. However, sometimes running your scripts can be a bit tedious. This is where the TestMu AI plugin can be used in the Azure Pipeline to ease the workflow and automate the tests. TestMu AI allows you to perform live interactive and automated cross-browser testing on 3000+ real browsers and operating systems online. - -Check TestMu AI Extension On Azure DevOps Marketplace - -## Benefits Of Using TestMu AI Plugin In Azure Pipeline - -* Setup your TestMu AI account in Azure Pipeline using your account credentials. -* Set up, and use the [TestMu AI Tunnel](/docs/testing-locally-hosted-pages/) for testing various production environments like internal, development and staging. -* Embed or fetch TestMu AI test results in your project's job results. -* Witness the execution of the tests, capture screenshots, mark bugs, plot graph and much more, on your automation dashboard on TestMu AI. - -## Prerequisites - -* Azure DevOps (Azure Pipelines) account -* TestMu AI account credentials. - 1. TestMu AI Username - 2. TestMu AI Access KeyYou can fetch your account credentials on TestMu AI from your [profile section](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). - -## How To Install TestMu AI Extension - -Follow the below steps to install TestMu AI Extension from Azure DevOps marketplace: - -1. Go to [Azure DevOps marketplace](https://marketplace.visualstudio.com/) and search for "TestMu AI" in the search box. - -2. In the results window, you will see an option showing the TestMu AI Extension. - -3. Open the found result and click on "Get it free", to get the TestMu AI Extension in your organization for free. - -4. Select an Azure DevOps organization and press "Install". TestMu AI Extension will be installed in this organization. - -5. Once finished, you can see "Done" at the top. It means that TestMu AI Extension has been installed successfully. You can proceed to your organization to see the extension. - -## How To Integrate TestMu AI Plugin In Azure Pipelines - -**Step 1:** Sign in to your Azure DevOps account to access the Azure Pipelines. Create one if you don't have any. - -**Step 2:** Create a project and download the TestMu AI extension from the [Azure marketplace](https://marketplace.visualstudio.com/search?term=lambdatest&target=AzureDevOps&category=All%20categories&sortBy=Relevance) as shown above. - -**Step 3:** In order to use the TestMu AI extension now, select **Pipelines** from the left side menu bar as shown below: - -**Step 4:** Select **New Pipelines** from the top right corner of your screen. - -**Step 5:** Switch to Class Editor from the footer. - -**Step 6:** Select your choice of versioning control software you use, from the given options. For this demo, we have used "Github". - -**Step 7:** Select the template as **Empty Job** to begin with. We will create the job as we go. - -**Step 8:** Add a new task in the job with the help of **+** button, and fill the required details. - - For this demo purpose, we have used following tasks: - -* Search *TestMu AI* in the search menu, and select **TestMu AI Configuration Setup**, **Stop Tunnel**, and **TestMu AI Results** from the available tasks. - -* Similarly, add **Command Line Script** and **Python Script** in to this job. -* Once added, arrange the tasks in below shown order: - -**Step 9:** Click on **TestMu AI Configuration Setup** and then click on **New**. - -**Step 10:** Fill the required details. For this demo purpose, we have used following details: - -* Select Basic Authentication mode of integration from the given authentication methods. -* Enter your TestMu AI username as "Username" and TestMu AI Access Key as "Password" and then click on verify button. -* Enter the desired Service connection name in the Details section and then click Verify and Save button. - -**Note**: You can also activate the TestMu AI Tunnel for localhost testing. Just select Service connection from the drop down list and Select TestMu AI Tunnel. - -**Step 11:** Now we need to install Selenium in our job to perform the automation tests. Click on Command Line Script and enter the command `pip install selenium` to install Selenium: - -**Step 12:** Select **Run a Python script** as we have selected Python in the above steps. - -Select your script to execute either from your repository using the "File Path" option, or you can also write the script inline using the "Inline" option. Once the script is ready, click on "OK" - -**Note**: In order to run the tests, you will need to get your build name from the environment variables. Shown below is the syntax of getting the build name from the environment variables for Python test scripts. - -```py -build_name = os.getenv("LT_BUILD_NAME") -``` - -**Step 13:** Now we have imported everything to run our tests. Click on "Save & Queue" to run your test. - -**Step 14:** Select OS from Agent Specification as shown below and click "Save and Run". - -**Step 15:** The pipeline will get activated and you can see your job queued to be executed. - -**Step 16:** To view the test results, click on the Return option. - -**Step 17:** Click the **TestMu AI** tab. - -You'll find your executed tests results. - -## How To Find Test Execution Results On TestMu AI - -The output of the tests executed above can be seen in your authenticated TestMu AI account as well. Follow the below steps to see your test execution result: - -**Step 1:** Login into your TestMu AI account using the credentials used above in the Azure Pipelines. - -**Step 2:** Go to "Automations" tab from your left menu. - -**Step 3:** Here you can see the executed tests along with their builds, details and status, as shown below: - -Not only this, but you can also use this automation dashboard to obtain Test logs, Analytics, Mark Bugs, Integrate apps from our diverse app store, fetch build details, and much more. TestMu AI Selenium Automation Grid enables you to perform end-to-end automation tests on a secure, reliable, and scalable Selenium infrastructure. You can perform automated cross-browser testing with Selenium scripts on 3000+ browsers and operating systems environments giving you higher test coverage and at significantly less built times. TestMu AI will help you evaluate how well your web application renders when accessed through a variety of different browsers. You can perform automated cross-browser testing of your locally hosted web pages using TestMu AI Tunnel. You could also run a single test across multiple browser/OS configurations, simultaneously. - -## LocalHost Testing Using TestMu AI Extension - -TestMu AI extension also allows you to perform testing of your localhost project securely with the [TestMu AI Tunnel](/docs/testing-locally-hosted-pages/) help. The below steps will help you configure the TestMu AI Tunnel using the TestMu AI extension in your Azure Pipelines: - -### Starting The TestMu AI Tunnel Using TestMu AI Extension - -While configuring the TestMu AI Configuration Setup job, as shown above, check the TestMu AI Tunnel checkbox to activate it. - -Once you activate it, a TestMu AI Tunnel Options box will appear, which require the [modifiers/arguments](/docs/lambda-tunnel-modifiers/) to configure the tunnel as per requirement. Pass the TestMu AI Tunnel modifiers in JSON format in the options box. - -> Note: Make sure to use the camel case for passing the tunnel modifiers. For example, use sharedTunnel if you want to use the modifier --shared-tunnel. For our demo purpose, we will be using simple modifiers, such as --tunnel-name and --proxy-port: {"tunnelName":"azure tunnel", "proxyPort":8050} - -The TestMu AI Tunnel has been successfully configured now. You can test your locally hosted pages and privately hosted pages at the TestMu AI Platform using TestMu AI Tunnel. - -### Stopping The TestMu AI Tunnel Using LambaTest Extension - -To stop the TestMu AI Tunnel, you need to add the _TestMu AI Stop Tunnel_ task in your job. Once this task is executed, the TestMu AI Tunnel will stop successfully. - -You must keep this task at the end of your all tests, to let your tests execute successfully before stopping the TestMu AI Tunnel. - -To setup your azure pipeline using TestMu AI Azure Pipeline Extension, you can refer to the below example: - -``` -resources: -repositories: -- repository: self -type: git -ref: refs/heads/main -jobs: -- job: Job_1 -displayName: Agent job 1 -pool: -vmImage: windows-2019 -steps: -- checkout: self -clean: true -fetchTags: false -- task: Lambdatest.lambda-azure-pipeline-extention.configuration-task.configuration@1 -displayName: LambdaTest configuration setup -inputs: -connection: 'connect endpoint' // provide your connection name -isTunnelActivate: true -isAppAutomate: false // set true if you want to run your app automate test scripts - -- task: PythonScript@0 -displayName: Run a Python script -inputs: -scriptSource: inline -script: >- -print('Hello world') // provide your python script or command to run your test scripts - -- task: Lambdatest.lambda-azure-pipeline-extention.stopLambdaTunnel-task.stoptunnel@1 -displayName: LambdaTest Stop Tunnel -- task: Lambdatest.lambda-azure-pipeline-extention.showResults-task.LambdatestResults@1 -displayName: Lambdatest Results -... -``` -Also you may use the YAMl assistant to add your tasks in the pipeline YAML file - -Click on the ```show assistant``` button and search for lambdatest extension tasks and add them in you pipeline YAML. - -> That was all you need to know for integrating TestMu AI Extension with Azure Pipelines. Increase your productivity with our integrations. If you still have any questions for us, please feel free to let us know. Our experts are always **available on chat** to help you out with any roadblock regarding our product. Happy testing! :) diff --git a/static/docs/integrate-testmu-with-cypress.md b/static/docs/integrate-testmu-with-cypress.md deleted file mode 100644 index 4b0dadef7..000000000 --- a/static/docs/integrate-testmu-with-cypress.md +++ /dev/null @@ -1,87 +0,0 @@ -# Integrate TestMu AI with Cypress Dashboard - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This article guides you on how to integrate **TestMu AI platform** with **The Cypress Dashboard** - -> In this demo we are demonstrating using [TestMu AI's Sample Cypress Cloud Repo](https://github.com/LambdaTest/Cypress-Cloud). You can find all the resources used for this article in the linked repo. - -## Steps To Integrate TestMu AI With Cypress Dashboard - -Assuming that you have run a Cypress test on TestMu AI platform ( if not, this article [here](/support/docs/getting-started-with-cypress-testing/) will guide you in running your first Cypress test on TestMu AI), you need to follow these steps below: - -### Update TestMu AI Cypress CLI - -- Before getting started, you would have to update the **lambdatest-cypress cli** using the command below: - -```bash -npm install -g lambdatest-cypress-cli -``` - -- The CLI version should be the latest i.e. is 2.3.0. You can check the CLI version by running the below command. - -```bash -lambdatest-cypress --version -``` - -### Create a project - -Create a project on Cypress Dashboard as shown below: - -### Enter Project Name - -### Define project ID - -Once to create the project on Cypress Dashboard, you will get a `project ID` and a `record key`. Define the **project ID** in the `cypress.json` file of your project. - -### Define Record Key - -Define the record key (Generated by Cypress Dashboard) in the run command as mentioned below: - -```bash -lambdatest-cypress run --cy="--record;--key " -``` - -### TestMu AI-Cypress Integration - -The integration is now done, just open the **TestMu AI Dashboard** and **Cypress Dashboard** to view the results. - -> That's all! In case you have any questions or need any additional information, you could reach out at our **[24X7 Chat Support]** or mail us directly at support@testmuai.com. - -## Using the Cypress Agent Skill with TestMu AI - -The [cypress-skill](https://github.com/LambdaTest/agent-skills/tree/main/cypress-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The cypress-skill package includes: - -``` -cypress-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing Cypress Agent Skill - -Install a Cypress Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/cypress-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/cypress-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only cypress-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). diff --git a/static/docs/integrating-mattermost-with-testmu.md b/static/docs/integrating-mattermost-with-testmu.md deleted file mode 100644 index f54f665ac..000000000 --- a/static/docs/integrating-mattermost-with-testmu.md +++ /dev/null @@ -1,71 +0,0 @@ -# Integrating Mattermost With TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Mattermost is a Communication platform, written in Golang and React, and is open-source, self-hostable, easy to deploy, manage, and upgrade. It is a secure workplace which allows messaging from behind your firewall. Mattermost is a famous alternative to proprietary SaaS messaging apps, as it allows you to bring all your team communication into a single place, making it anywhere searchable and accessible. - -The TestMu AI Mattermost Integration allows you to share or push a bug to your specified Mattermost channel from the TestMu AI platform in a single click. You can share any UI observations and input with your team on Mattermost anytime, by just capturing a screenshot in the middle of your test session through TestMu AI. You can also annotate the screenshot or highlight your issue or input. The fields populated by you while marking as a bug through TestMu AI gets displayed as information on Mattermost for that testing instance. - -**Mattermost Integration with TestMu AI, like all of the integrations to 3rd party applications, is available for freemium as well as premium plan.** - -## Integrate Mattermost With Your TestMu AI Account - -**Step 1:** Login to your TestMu AI account. Make sure you have an Admin or User level access to see and install 3rd party integrations. - -**Step 2:** Select **'Integrations'** tab from the left navigation menu bar. This will guide you to a screen where you will find a list of 3rd party applications, that have already been integrated as well as that are available to integrate with your TestMu AI account. - -**Step 3:** Hover to the 'Mattermost' block under "Communications" category, and click on "**Add**" button that appears. - -**Step 4:** You need a webhook URL to integrate your workspace with TestMu AI. So copy the webhook URL of the proposed TestMu AI integration on Mattermost. - ->In case you don't have a webhook ready, you can find read about how to create a webhook [here](https://docs.mattermost.com/developer/webhooks-incoming.html). - -**Step 5:** Enter your **webhook URL** copied from the above step, in the box that appears after clicking on Add in step 3, and then press "**Install**" - -**Step 6:** Once you press on Install, Mattermost will be integrated with TestMu AI on the same webhook URL, and an alert-box will appear on the top right corner of the screen stating that the integration is successful. - -**Step 7:** Voila! Go to Integrations tab again and you will see a Mattermost under "My Integrations". - -Also, you can see a **green tick** in front of Mattermost under Communication category, indicating that Mattermost has been successfully **installed**. - -## How To Log Your First Bug Through Mattermost Integration? - -**Step 1:** Select any of the test from the left navigation menu of TestMu AI. For this instance, we will be taking "**Real Time Testing**" option. - -**Step 2:** Enter the URL of the web-app to be tested in the URL box. Press **'Start'** after selecting the configuration of browser and operating system as per your choice. - -**Step 3:** After the VM is launched and operable, you can perform testing on your web-app as per your choice, such as for finding bugs. Suppose if you come across any bug, then you need to click on the **Bug icon** from the left panel, for capturing a screenshot of the current VM instance. - -**Step 4:** A screenshot will be captured and appear on your screen in an **in-built image editor**. Annotate any issue or a task as per your choice in this editor, and then press **"Mark as Bug"** button. - -**Step 5:** Complete the **Bug/Task/Issue Form** that appears upon clicking the **"Mark as Bug"** button, as per your requirement: ->* Select the **Issue type** as Bug/Task/Issue. Here, we have selected it as Bug. -* Set the **Priority** of the bug as per convenience. -* You can assign the bug to a member of your team by populating the field '**Assignee**'. -* Write the '**Summary**' of the bug that would be sent to the assigned member. -* You also get to post an optional **description** to help with the bug. - -At the bottom of the form, please make sure to tick the check button -stating **"Send to Mattermost"**. - ->Note: By default, it would be **checked**, if Mattermost is already integrated. This button helps send the screenshot, as per the filled form, directly to your workspace, when the Bug is created. You can uncheck the button anytime if you do not wish the bug/task/issue on Mattermost - -Now click on **"Create Bug"** button to send the bug directly to Mattermost workspace. - -**Step 6:** After you click on **"Create Bug"**, you will be able to observe it being successfully marked through a single click effort. You will get **prompt messages** on top of your Virtual Machine indicating the progress of bug logging. After few seconds you will be notified with a prompt message "**Bug successfully marked**" indicating that the screenshot has been pushed to your Mattermost workspace. - -Login to your workspace on Mattermost and you will be able to notice the marked bug in the set channel assigned to the set person. All the data you provided through Lambdatest form would already be presented in it. TestMu AI automatically includes test environment details and related screenshots in the screenshot as attachments. - -A clickable link will be provided in the Mattermost channel along with the screenshot to instantly route users on their browser for a full screen view of the bug. - -## How To Remove Mattermost Integration? - ->You can work with one integration at a time. So if you would want to integrate to a similar 3rd party application, then you would have to **remove** your current integration. Here is how you can do that. - -**Step 1:** Login to your TestMu AI account. - -**Step 2:** Select 'Integrations' from the left navigation menu bar & go to 'Communications' category. This will guide you to a screen where you will find 3rd party applications, available to integrate with your TestMu AI account. - -**Step 3:** Hover to Mattermost block. A **"REMOVE"** button will appear. Click on **'REMOVE'**. - ->That was all you need to know for TestMu AI + Mattermost Integration. Increase your productivity multifold with our integrations. If you still have any questions for us, please feel free to let us know. Our experts are always **24/7 chat support** to help you out with any roadblock regarding our product. Happy testing! diff --git a/static/docs/integrating-testmu-report-with-jenkins-pipeline.md b/static/docs/integrating-testmu-report-with-jenkins-pipeline.md deleted file mode 100644 index 3d1c767f1..000000000 --- a/static/docs/integrating-testmu-report-with-jenkins-pipeline.md +++ /dev/null @@ -1,42 +0,0 @@ -# Integrating TestMu AI Report With Jenkins Pipeline - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This documentation provides a comprehensive guide on how to integrate TestMu AI Report with your Jenkins Pipeline for web and app test automation. - -> **Note**: Before you proceed, make sure to integrate [TestMu AI Plugin for Jenkins](/docs/jenkins-with-testmu/). - -1. Login to your Jenkins account. From the left menu, navigate to **New Item**. - -2. Enter the Item name, select **Pipeline** and click **OK**. - -It will route you to a Configuration dashboard. - -4. From the left menu, click **Pipeline**. - -5. In the **Script** field, enter your Jenkins pipeline script and click **Pipeline Syntax**. - -**Best Practices** -**Build naming** - -Define `LT_BUILD_NAME` in your Jenkins pipeline using `${BUILD_TAG}`, and reference the same in your test capabilities. This ensures each TestMu AI run is correctly mapped to its corresponding Jenkins build. - -**Report publishing** - -Add a `post { always { ... } }` block to your pipeline so that the report is generated after every run, regardless of the status. - -6. From the *Sample Step* dropdown, select `lambdaTestReportPublisher:LambdaTest Pipeline Report`. - -7. Select either of the options: *Automation* or *App Automation* from the **Select Product** dropdown. - -8. After selecting the product, click **Generate Pipeline Script** and copy the generated pipeline syntax. - -9. To initiate the steps for TestMu AI Report creation, copy and paste the pipeline syntax in the Stage section of your Jenkins pipeline script. After that, click **Save** or **Apply**. - -10. To initiate the build process, navigate to the left menu and select **Build Now**. This will start the build process. - -11. Upon successful completion of the build process, click on the corresponding build. - -12. To view the test report, click on the **TestMu AI Pipeline Report**. - -The test report will show up, containing build details such as **Name**, **Status**, **Browser/Device**, **OS**, and **Duration**. diff --git a/static/docs/integrating-testmu-with-jenkins-freestyle-project.md b/static/docs/integrating-testmu-with-jenkins-freestyle-project.md deleted file mode 100644 index da2bc2db0..000000000 --- a/static/docs/integrating-testmu-with-jenkins-freestyle-project.md +++ /dev/null @@ -1,39 +0,0 @@ -# Integrate TestMu AI With Jenkins Freestyle Project - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Integrate TestMu AI Jenkins plugin with your Jenkins Freestyle project for: -* Configure your TestMu AI credentials for your Jenkins jobs. -* Set up and tear down the Lambda Tunnel binary file to perform automated cross browser testing even on your locally hosted web apps. -* Embed all the test results such as video logs, network logs, and screenshots of every step performed through TestMu AI with your Jenkins job results. - -> Note: Before you proceed, make sure to integrate [TestMu AI plugin for Jenkins](/docs/jenkins-with-testmu/). - -**Step 1:** Create a **New Item** in Jenkins and select **Freestyle Project**. - -**Step 2:** Go to Source Code Management and feed the repository URL. - -**Step 3:** Under the Built Environments, select TestMu AI and specify your desired capabilities. If you wish to run multiple jobs in parallel then you can **Add Selenium Test Capability**. - ->**Note:** It is recommended to check your network connectivity by **pinging TestMu AI**. - -**Step 4:** Add a build step to **Execute Shell**. - -**Step 5:** Hit the below command to **Execute Shell** and hit **save**. - -``` -python google-search-lambdatest.py -``` - -**Best Practices** -**Build naming** - -Define `LT_BUILD_NAME` in your script using `${BUILD_TAG}`, and reference the same in your test capabilities. This ensures that each TestMu AI run is correctly mapped to its corresponding Jenkins build. - -**Step 6:** Click on **Apply & Save** to set your configurations and then click on **"Build Now"** to execute your test. - -## Test On Locally Hosted Web Applications - -While specifying build environments, check the box to **Use Local Tunnel**. You can then specify a Tunnel name if you like, your TestMu AI Credentials will be automatically fetched. - -Kudos! That is all you need to know for integrating your Jenkins freestyle project with TestMu AI. In case of any questions, feel free to give us a shout. You can also drop an email to [support@testmuai.com](mailto:support@testmuai.com). Happy testing! :) diff --git a/static/docs/integrating-wrike-with-testmu.md b/static/docs/integrating-wrike-with-testmu.md deleted file mode 100644 index 8d9e6583f..000000000 --- a/static/docs/integrating-wrike-with-testmu.md +++ /dev/null @@ -1,67 +0,0 @@ -# Integrating Wrike With TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - ->Wrike is a SaaS(Software-As-A-Service) Product Management software. You can manage and track your projects, deadlines, and other workflow processes. It also allows inter-team communications, and therefore used by many developments, as well as testing teams to keep track of their work and teams. Wrike also provides the ability to track bugs effectively. - -TestMu AI Wrike integration helps you in creating issues in Wrike directly from the TestMu AI platform. TestMu AI now allows you to integrate your Wrike workspace with simple one-click. You can push annotated issues to a project of your choice, assign it to the required teammate, set the priority, and attach screenshots in the middle of the test session. After creating the issue, it gets reflected directly in your project's workspace on Wrike with the details of the test session along with the annotated screenshot. - -**Wrike Integration with TestMu AI, like all of the integrations to 3rd party applications, is available for freemium as well as premium plan.** - -## Integrate Wrike With Your TestMu AI Account - -**Step 1:** Login to your TestMu AI account. You need to have an Admin or User level access to see and install 3rd party integrations. - -**Step 2:** Select **‘Integrations’** tab from the left navigation menu bar. This will guide you to a screen where you will find a list of 3rd party applications, that have already been integrated as well as that are available to integrate with your TestMu AI account. - -**Step 3:** Hover to the 'Wrike' block under "Project Management" category, and click on "**Add**" button that appears. - -**Step 4:** You need to [create an account on Wrike](https://login.wrike.com/login/) to integrate your workspace with TestMu AI. If you don't have an account, you can create one even in the middle of this integration in the this step. If you are already logged in, this step won't appear in the integration process. - -**Step 5:** Review the privacy policy and click on "Accept". - -**Step 6:** Once you press on "Accept", Wrike will be integrated with TestMu AI on the same account, and an alert-box will appear on the top right corner of the screen stating that the integration is successful. - -**Step 7:** Congrats! Wrike has been successfully integrated. Go to Integrations tab again and you will see Wrike under the "My Integrations" tab. - -Also, you can see a **green tick** in front of Wrike under Project Management category, indicating that Wrike has been successfully **installed**. - -## How To Log Your First Bug Through Wrike Integration? - -**Step 1:** Select any of the test from the left navigation menu of TestMu AI. For this instance, we will be taking "**Real Time Testing**" option. - -**Step 2:** Enter the URL of the web-app to be tested in the URL box. Press **'Start'** after selecting the configuration of browser and operating system as per your choice. - -**Step 3:** After the VM is launched and operable, you can perform testing on your web-app as per your choice, such as for finding issues. Suppose if you come across any bug, then you need to click on the **Bug icon** from the left panel, for capturing a screenshot of the current VM instance. - -**Step 4:** A screenshot will be captured and appear on your screen in an **in-built image editor**. Annotate any issue as per your choice in this editor, and then press **"Mark as Bug"** button. - -**Step 5:** Complete the **Bug/Task/Issue Form** that appears upon clicking the **"Mark as Bug"** button, as per your requirement: - -> * Select the **Folder** from your Wrike workspace, in which you want to create this issue. -> * You can assign the bug to a member of your team or yourself, by selecting from the drop-down of field '**Assignee**'. -> * Set the **Priority** of the bug as per convenience. -> * Give a title to your bug in the "Title" field. -> * Also, post an optional **description** to help the assignee with the bug tracking. - -Now click on **"Create Bug"** button create the bug and send the bug directly to selected project. - -**Step 6:** After you click on **"Create Bug"**, you will be able to observe it being successfully marked through a single click effort. You will get **prompt messages** on top of your Virtual Machine indicating the progress of bug logging. After few seconds you will be notified with a prompt message "**Bug successfully marked**" indicating that the screenshot has been pushed to your Wrike workspace. - -**Step 7:** Login to your workspace on Wrike and you will be able to notice the marked bug in the set channel assigned to the selected person. All the data you provided through TestMu AI form would already be presented in it. TestMu AI automatically includes test environment details and related screenshots in the screenshot as attachments. - -A clickable link will be provided in the Wrike channel along with the screenshot to instantly route users on their browser for a full screen view of the bug. - -## How To Remove Wrike Integration? - -> You can work with one integration at a time. So if you would want to integrate to a similar 3rd party application, then you would have to **remove** your current integration. Here is how you can do that. - -**Step 1:** Login to your TestMu AI account. - -**Step 2:** Select ‘Integrations’ from the left navigation menu bar & go to 'Product Management' category. This will guide you to a screen where you will find 3rd party applications, available to integrate with your TestMu AI account. You can also remove it using the 'My Integrations' tab on the Integrations tab. - -**Step 3:** On the Wrike block. A **"REMOVE"** button will appear. Click on **‘REMOVE’**. - -**Step 4:** Upon pressing 'REMOVE', a message will prompt on the top right corner of the app stating the same. - -> That was all you need to know for TestMu AI + Wrike Integration. Increase your productivity multifold with our integrations. If you still have any questions for us, please feel free to let us know. Our experts are always **available on chat** to help you out with any roadblock regarding our product. Happy testing! diff --git a/static/docs/integrating-your-jenkins-pipeline-with-testmu.md b/static/docs/integrating-your-jenkins-pipeline-with-testmu.md deleted file mode 100644 index dd9ea516c..000000000 --- a/static/docs/integrating-your-jenkins-pipeline-with-testmu.md +++ /dev/null @@ -1,99 +0,0 @@ -# Integrating Your Jenkins Pipeline With TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Jenkins Pipeline is also referred to as "Pipeline" offers a suite of plugins to help integrate your continuous delivery pipeline into Jenkins. Jenkins Pipeline does so with the help of Pipeline DSL(Domain Specific Language) syntax that facilitates easy modelling of even the most complex delivery pipeline. - -## Prerequisites For Configuring Jenkins Pipeline With TestMu AI - -1. Jenkins 2.X or greater version. -2. A Jenkins User with root access. -3. Ensure you have the Pipeline plugin, although, it is displayed under the "suggested plugins" during the post-installation setup of Jenkins. -4. **TestMu AI Authentication Credentials** - -Be aware of your TestMu AI authentication credentials i.e. your TestMu AI username, access key and HubURL. You need to set them up as your environment variables. You can retrieve them from your **[TestMu AI automation dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com)** by clicking on the key icon near the help button. - -* For Linux/Mac: - -```javascript -$ export LT_USERNAME= {YOUR_LAMBDATEST_USERNAME} -$ export LT_ACCESS_KEY= {YOUR_LAMBDATEST_ACCESS_KEY} -``` - -* For Windows: - -```javascript -$ set LT_USERNAME= {YOUR_LAMBDATEST_USERNAME} -$ set LT_ACCESS_KEY= {YOUR_LAMBDATEST_ACCESS_KEY} -``` - -## Setting Up Jenkins Pipeline - -You can find the Jenkins file for the Pipeline [here](https://github.com/LambdaTest/nightwatch-selenium-sample/blob/master/Jenkinsfile) or you can find the code for the Pipeline below. Below is the code for the Jenkins Pipeline. - -```javascript -#!/usr/bin/env groovy - -node { -withEnv(["LT_USERNAME=Your LambdaTest UserName", -"LT_ACCESS_KEY=Your LambdaTest Access Key", -"LT_TUNNEL=true"]){ - -echo env.LT_USERNAME -echo env.LT_ACCESS_KEY - -stage('setup') { - -// Get some code from a GitHub repository -try{ -git 'https://github.com/LambdaTest/nightwatch-selenium-sample.git' - -//Download Tunnel Binary -sh "wget https://downloads.lambdatest.com/tunnel/v3/linux/64bit/LT_Linux.zip" - -//Required if unzip is not installed -sh 'sudo apt-get install --no-act unzip' -sh 'unzip -o LT_Linux.zip' - -//Starting Tunnel Process -sh "./LT -user ${env.LT_USERNAME} -key ${env.LT_ACCESS_KEY} &" -sh "rm -rf LT_Linux.zip" -} -catch (err){ -echo err -} - -} -stage('build') { -// Installing Dependencies -sh 'npm install' -} - -stage('test') { -try{ -sh './node_modules/.bin/nightwatch -e chrome,edge tests' -} -catch (err){ -echo err -} -} -stage('end') { -echo "Success" -} -} -} -``` - -Refer to our [Github sample repository](https://github.com/LambdaTest/nightwatch-selenium-sample). Now, follow the below steps for the setup. - -**Step 1:** Create a new project for Pipeline by going to Jenkins --> New Item. - - **Step 2:** Select the Pipeline. - - **Step 3:** Scroll down to Advanced Project Options. Paste the Code in the code pane or fetch it via SCM & hit the **Save** button. - **Note:** To run on the tunnel, Either you can use LT_TUNNEL Environment variable to set the tunnelling capability or you can pass in the code. Instructions on the tunnel are written in Repo Readme file. - -## Additional Links - -- [Integrate TestMu AI With Jenkins Freestyle Project](/support/docs/integrating-testmu-with-jenkins-freestyle-project/) -- [Integrating TestMu AI Report With Jenkins Pipeline](/support/docs/integrating-testmu-report-with-jenkins-pipeline/) diff --git a/static/docs/integration-with-hyperexecute.md b/static/docs/integration-with-hyperexecute.md deleted file mode 100644 index 63d05059f..000000000 --- a/static/docs/integration-with-hyperexecute.md +++ /dev/null @@ -1,13 +0,0 @@ -# Integrations - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Integrate HyperExecute with your favourite CI/CD tools, products and TestMu AI tools to accelerate and optimize your testing ecosystem. - -Integration With Products - -Integration With CI/CD Tools - -Integration With TestMu AI Products - ->If your favourite integrations are not mentioned in the list above, then just give us a shout and we will have them ready for you. diff --git a/static/docs/integrations-with-ci-cd-tools.md b/static/docs/integrations-with-ci-cd-tools.md deleted file mode 100644 index 1a23c9245..000000000 --- a/static/docs/integrations-with-ci-cd-tools.md +++ /dev/null @@ -1,45 +0,0 @@ -# Integrations With CI/CD Tools - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI offers integrations with numerous CI/CD tools to help you fast-track your robust delivery pipeline for pacing up your go-to-market launch. Below are the step-by-step documentation for all our integrations with CI/CD tools to help you leverage the maximum output from TestMu AI Selenium Grid. - -Bitbucket Pipeline Integration With TestMu AI - -AWS CodePipeline Integration With TestMu AI - -Bamboo Integration With TestMu AI - -Jenkins Integration With TestMu AI - -CircleCI Integration With TestMu AI - -Travis CI Integration With TestMu AI - -GitLab CI Integration With TestMu AI - -Semaphore Integration With TestMu AI - -Codeship Pipelines Integration With TestMu AI - -Codefresh CI Pipelines Integration With TestMu AI - -TeamCity Integration With TestMu AI - -Azure Pipelines Integration With TestMu AI - -Concourse CI Integration With TestMu AI - -GOCD Integration With TestMu AI - -Drone CI Integration With TestMu AI - -Appveyor CI Integration With TestMu AI - -Google Cloud CI Integration With TestMu AI - -Bitrise CI Integration With TestMu AI - -Buildkite Integration With TestMu AI - ->If your favourite CI/CD tools is not in the above mentioned, then just give us a shout and we will have it ready shortly for you. diff --git a/static/docs/integrations-with-codeless-automation-tools.md b/static/docs/integrations-with-codeless-automation-tools.md deleted file mode 100644 index 4aae36a5b..000000000 --- a/static/docs/integrations-with-codeless-automation-tools.md +++ /dev/null @@ -1,35 +0,0 @@ -# Codeless Automation Tools - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI offers integration with codeless automation tools to help you expand your cross browser testing coverage. Using the integration with your favorite codeless automation tool, you can perform automated browser testing on 3000+ real browsers and operating systems. - -Run Selenium IDE Tests With TestMu AI - -Katalon Studio Integration With TestMu AI - -Ranorex Integration With TestMu AI - -Oxygen Integration With TestMu AI - -LEAPWORK Integration With TestMu AI - -TestingWhiz Integration With TestMu AI - -Cerberus Studio Integration With TestMu AI - -QMetry Integration With TestMu AI - -Vividus Integration With TestMu AI - -Testsigma Integration With TestMu AI - -Tricentis Tosca Integration With TestMu AI - -Ghost Inspector Integration With TestMu AI - -Accelq Integration With TestMu AI - -algoQA Integration With TestMu AI - -If your favorite Codeless automation tool is not in the above-mentioned, just give us a **shout** and we will have it ready shortly for you. diff --git a/static/docs/integrations-with-project-management-tools.md b/static/docs/integrations-with-project-management-tools.md deleted file mode 100644 index 8bcb3fb60..000000000 --- a/static/docs/integrations-with-project-management-tools.md +++ /dev/null @@ -1,75 +0,0 @@ -# Project Management Tools - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI offers integration to numerous third-party tools for project management and bug tracking to help you easily collaborate with your colleagues as you encounter a bug while performing cross browser testing. After integrating your TestMu AI account with your favourite project management tool or bug tracking tool, you can send any UI bug that you may encounter on your website, directly to your task management dashboard, in just a single click from TestMu AI. - -Asana Integration With TestMu AI - -GitHub Integration With TestMu AI - -GitLab Integration With TestMu AI - -Trello Integration With TestMu AI - -Bitbucket Integration With TestMu AI - -Paymo Integration With TestMu AI - -Teamwork Integration With TestMu AI - -Hive Integration With TestMu AI - -Shortcut Integration With TestMu AI - -monday Integration With TestMu AI - -Breeze Integration With TestMu AI - -Clickup Integration With TestMu AI - -Jira Integration With TestMu AI - -Self-Hosted Jira Integration With TestMu AI - -Azure DevOps Integration With TestMu AI - -BugHerd Integration With TestMu AI - -Mantis Integration With TestMu AI - -zipBoard Integration With TestMu AI - -Backlog Integration With TestMu AI - -Pivotal Integration With TestMu AI - -Practitest Integration With TestMu AI - -Miro Integration - -Airtable Integration - -Wrike Integration - -TargetProcess Integration - -YouTrack Integration - -Linear App Integration - -Bugzilla Integration - -Notion Integration - -Datadog Integration - -PagerDuty Integration - -Bugsnag Integration - -Zoho BugTracker Integration - -GoodDay Integration - ->If your favourite project management or bug tracking tool is not in the above mentioned, then just give us a **shout** and we will have it ready shortly for you. diff --git a/static/docs/ios-entitlements.md b/static/docs/ios-entitlements.md deleted file mode 100644 index c9b8fd51d..000000000 --- a/static/docs/ios-entitlements.md +++ /dev/null @@ -1,33 +0,0 @@ -# iOS Entitlements - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -We support all ipa files generated from different Certificates & Profiles. - -However, to install these apps on our devices, we resign your applications with our Resigning Certificate. - -This is true for all Certificates & Profiles except the applications generated using an Enterprise Account. - -## Disabling Re-Signing In Manual App Testing - -**Note** - -- Your application must be an Enterprise Application. - -When you upload an application that is generated through an Enterprise Account, you have the option at the beginning of your test to disable re-signing by toggling iOS Entitlements. - -Once you toggle the iOS Entitlements option, your application's certificates will be preserved. - -## Disabling Re-Signing In App Automation - -**Note** - -- Your application must be an Enterprise Application. - -You can prevent re-signing your application with the "resignApp" capability. Upload your enterprise application & specify the capability as "false" to prevent it from being re-signed. - -KEY|VALUE|DESCRIPTION ---|--|-- -| resignApp | TYPE: BOOLEAN DEFAULT: True `resignApp = true` | 1. By default, if this capability is not passed, your app will be re-signed. 2. You can pass "false" as a capability to prevent your apps from being re-signed. 3. This is only for iOS-specific applications and devices | -> -If you still have any questions for us, please feel free to let us know via our **24X7 Chat Portal** or mail us to support@testmuai.com diff --git a/static/docs/ios-ipa-files-xcui.md b/static/docs/ios-ipa-files-xcui.md deleted file mode 100644 index c0624acfe..000000000 --- a/static/docs/ios-ipa-files-xcui.md +++ /dev/null @@ -1,87 +0,0 @@ -# Creating iOS Applications - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This document outlines the process of creating and testing iOS apps using Xcode. It covers two methods for creating.ipa files for testing: directly creating the.ipa file or creating an.app bundle and converting it into an.ipa file. The first method involves creating a.ipa file using various export methods, such as Ad Hoc, Enterprise, or Development. The second method involves creating an.app bundle and converting it into an.ipa file. - -## Step 1: Create .ipa files for Testing - -In Xcode, there are two common methods to create **.ipa** files for iOS app distribution. The first method directly creates the .ipa file, while the second method involves creating an .app bundle and then converting it into an .ipa file. - -### Method 1: Using Export Methods - -You can directly create .ipa files using various export methods (**Ad Hoc, Enterprise,** or **Development**) for distribution. The App Store type is not included in this method. - -**Step 1:** Open your project in Xcode. Choose **Generic iOS Device** or **Any iOS Device (arm64).** for the project's device target. - -**Step 2:** Go to the **Product** -> **Clean** to ensure a clean build. Again, go to the **Product** -> **Archive.** Your app will be listed under the **Archives** after the process is completed. - -**Step 3:** Choose your app from the list and click **Distribute App.** - -**Step 4:** Select an export method and configure the Distribution options as follows: - > - **App Thinning:** None - - Unselect the option **Rebuild from Bitcode.** - - **Strip swift symbols** is optional. - - Unselect the option **Include manifest for over-the-air installation.** - -**Step 5:** Choose your Distribution Certificate and Provisioning Profile (Automatic or Manual). This will create the **.ipa** file. - -**Step 6:** Once the export is complete, you will be prompted to specify the location where you want to save the **.ipa** file. Choose the desired location for saving the generated .ipa file. - -You can use the generated .ipa file for testing on the **TestMu AI** Platform. - -### Method 2: Using .app Bundles - -**Step 1:** Open your app project in Xcode. - -**Step 2:** For your Product's device target, select either **Generic iOS Device** or **Any iOS Device (arm64).** - -**Step 3:** Go to the **Product** menu -> **Clean** to ensure a clean build. Again, click on **Product** menu -> **Build.** - -**note** -The **.app file** is generated in your Xcode project's **Products Directory**. - -**Step 4:** After generating the **.app** bundle, proceed with building an **.ipa** file: - -> - Firstly, create an empty directory with the name `Payload`. -- Move the .app file to the `Payload` directory. -- Make an archive (.zip file) by compressing the `Payload` directory. -- Rename this archive and append `.ipa` at the end. This creates an **.ipa file**. - -## Step 2: Create an XCUITest Package (.ipa) - -Follow the below mentioned steps to create your Test package: - -**Step 1:** Open your application project in Xcode. Choose **Generic iOS Device** or **Any iOS Device (arm64)** for the project's device target. - -**Step 2:** Confirm that your UI Tests are included in **Target Membership** and the said targets are selected to be built in your **Xcode Build Scheme.** _UI Tests Targets are specially selected to be built at the "Test" build action._ - -**Step 3:** Click on **Product** -> **Build For** -> **Testing** to create your test package. - -**Step 4:** Go to the **Products Directory** of your XCode project to find the generated **.app file**. - -**Step 5:** Now, create **.ipa** from the **.app** bundle. After generating the **.app** bundle, proceed with building an **.ipa** file: - -> - Firstly, create an empty directory with the name `Payload`. -- Move the .app files to the `Payload` directory. -- Make an archive (.zip file) by compressing the `Payload` directory. -- Rename this archive and append `.ipa` at the end. This creates an **.ipa file**. - -This allow you to transform the generated **.app** bundle into an **.ipa** file, suitable for iOS app distribution. - -## Step 3: Aligning iOS Deployment Target for XCUITest Testing. - -Match the iOS version of your app and test runner **iOS Deployment Target**. If they don't match up, the tests will run locally, but won't be able to run on TestMu AI Real Devices. - -To match the version in your Xcode Project: - -- Go to the Project that you wish to build. -- Go to the **Build Settings**, set the **iOS Deployment Target** to the iOS version of your app. This will align the app and test runner to the same iOS version. - -To match the version in your Xcode Target. - -- Choose the Target of your Project. -- Go to the **Build Settings**, set the **iOS Deployment Target** to the iOS version you want to use in your test. - -**Warning** -This process will overwrite the **Build settings** at the Project Level to the given iOS version. While using this method, take note that the Targets can be out of sync with each other and the project settings, which will cause the tests to break. You would need to re-build the Project if you change the iOS version for one target output in order to keep all your targets in sync. diff --git a/static/docs/ios-keychain-cleanup.md b/static/docs/ios-keychain-cleanup.md deleted file mode 100644 index ea5647c9a..000000000 --- a/static/docs/ios-keychain-cleanup.md +++ /dev/null @@ -1,63 +0,0 @@ -# Support for iOS keychain cleanup and access groups - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Apple’s [Keychain](https://developer.apple.com/documentation/security/keychain_services) offers a secure system for apps to store sensitive information like passwords, certificates, authentication tokens, and other small data pieces. -TestMu AI now offers enhanced Keychain management on real iOS devices, enabling you to test app flows that rely on secure storage and Keychain-specific use cases more effectively. - -## Keychain Cleanup After Sessions - -Certain iOS apps may retain user data such as login details across different test sessions. -This happens because when an app is uninstalled, any Keychain data associated with it **remains on the device**, as iOS does not automatically clear it. -While this data is isolated from other apps, it can persist unless specifically removed. - -To prevent data from carrying over across sessions, TestMu AI offers an option to automatically clear all Keychain entries after your test ends, ensuring a clean environment for every run. - -## Keychain Access Groups During App Resigning - -When TestMu AI resigns your iOS application using a wildcard provisioning profile (to enable installation on real devices), the app’s **keychain-access-groups** entitlement is preserved. -However, the **Bundle Seed ID** (also known as Team ID), a critical part of access groups, gets replaced during resigning. - -As a result, app functionalities that depend on the original access group may break if not handled properly. -By enabling Keychain support, TestMu AI takes care of these changes, allowing your app to continue using Keychain securely even after resigning. - ->This feature is currently in **Beta**. We are actively refining it based on real-world usage and feedback. - -## How to Enable iOS Keychain Support During App Upload - -To activate Keychain cleanup and access group handling, simply pass the `ios_keychain_enabled` parameter during app upload. - -Example using **cURL**: - -```bash -curl -u "USERNAME:ACCESS_KEY" \ --X POST "https://manual-api.lambdatest.com/app/upload/realDevice" \ --F "appFile=@/path/to/your-app.ipa" \ --F "name=YourAppName" \ --F "ios_keychain_enabled=true" -``` -Enabling `ios_keychain_enabled=true` will -- Clear Keychain data after each test session. -- Preserve Keychain access across resigning. - -- Supported on **iOS 13 and above** only. Apps targeting earlier iOS versions are not compatible. -- Requires app resigning (`resignApp=true`). Apps signed with **Enterprise certificates** are not supported. LT framework needs to be injected in this case. - -## Frequently Asked Questions (FAQs) - -### 1. Can Keychain cleanup be managed without `ios_keychain_enabled`? - -Yes. Developers have the following options: -- Proactively delete Keychain entries during user logout or similar app events. -- Reset all Keychain data during the app's first launch after installation. - -You can refer to Apple's guide for best practices: [Manage Keychain Data](https://developer.apple.com/documentation/security/keychain_services). - -Additionally, apps can provide an in-app setting to manually clear Keychain data if needed during testing. - -### 2. Can Keychain access groups be handled without `ios_keychain_enabled`? - -Yes. -Developers can dynamically retrieve the access group information (`kSecAttrAccessGroup`) from existing Keychain entries instead of hardcoding Bundle Seed IDs. - -This approach ensures your app remains functional even if the Team ID changes after resigning. diff --git a/static/docs/ios-shake-gesture-appautomation.md b/static/docs/ios-shake-gesture-appautomation.md deleted file mode 100644 index d79f79509..000000000 --- a/static/docs/ios-shake-gesture-appautomation.md +++ /dev/null @@ -1,59 +0,0 @@ -# Trigger Shake Gesture on iOS Devices - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI supports simulating a **shake gesture** on iOS Real Devices and iOS Simulators during Appium automation sessions. Apps often bind the shake gesture to actions such as switching between production and staging builds, opening debug menus, surfacing feedback prompts, or undoing the last action. - -By the end of this document, you will be able to: -- Trigger a shake gesture on iOS Real Devices and Simulators -- Validate the response returned by the `lambda_executor` hook -- Use the same hook inside Web Automation tests running on iOS Real Devices - -This feature also works in **Web Automation tests on Real Devices**. The same `lambda_executor` script can be used inside your Appium-based web automation session. - -## Supported Platforms - -| Platform | OS Version | -|--------------------|---------------------| -| iOS Real Device | iOS 16 and above | -| iOS Simulator (Virtual) | All OS | - -## Support for Automation on Mobile Devices - -To trigger a shake gesture, use the `lambda_executor` Appium hook with the `gestures` action. No additional capability is required. - -## Supported Arguments - -| Argument | Type | Description | -| ----------------- | --------- | --------------------------------------------------------------------------- | -| `action` | String | Must be `"gestures"` (lowercase). | -| `arguments.shake` | Boolean | Set to `true` to trigger the shake. Set to `false` to skip without erroring.| - -> 💡 **Note:** The hook **can be invoked multiple times** in the same session. - -### Appium Hook Example - -```python -response = driver.execute_script( -'lambda_executor: {"action": "gestures", "arguments": {"shake": true}}' -) -# response → "shake gesture triggered successfully" -``` - -```java -String response = (String) ((JavascriptExecutor) driver).executeScript( -"lambda_executor: {\"action\": \"gestures\", \"arguments\": {\"shake\": true}}" -); -// response → "shake gesture triggered successfully" -``` - -## Response References - -| Scenario | Response | -|----------------------------|---------------------------------------------------| -| `shake: true`| `shake gesture triggered successfully` | -| `shake: false` | `shake gesture skipped (shake=false)` | -| Wrong type for `shake` | `argument 'shake' must be a boolean, got string` | -| `shake` argument missing | `missing required argument: shake` | -| Wrong action casing | `Please send a valid action for LambdaHook` | -| Android session | `shake gesture is supported on iOS only` | diff --git a/static/docs/ip-geolocation-on-real-devices-app.md b/static/docs/ip-geolocation-on-real-devices-app.md deleted file mode 100644 index e25de4363..000000000 --- a/static/docs/ip-geolocation-on-real-devices-app.md +++ /dev/null @@ -1,45 +0,0 @@ -# IP Geolocation on Real Devices App - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Location-specific behavior is crucial to ensure your mobile apps and websites deliver accurate and localized experiences to users worldwide. Whether it's testing region-based content, geo-restricted features, or local SEO implementations, manually simulating different geolocations is essential for a robust QA process. - -TestMu AI's Geolocation Simulation feature allows you to manually set and update the GPS location on real Android devices during live app testing sessions. This empowers you to verify how your app behaves for users across various regions, without needing to physically move devices or use external GPS spoofing tools. - -## Update Geolocation for Real Devices on TestMu AI - -**Step 1:** Click on the **Real Devices** > **App Testing** - -**Step 2:** Select your required device and upload your application and click on Start button. - -**Step 3:** Once your live session begins, click the **IP Geolocation** option from the top toolbar. - -**Step 4:** Use the search bar to find the country or location you want to simulate. Select the desired location from the results. - -## Supported Android - - IP Geolocation is available on below android : - -| Android Version | -|----------------------| -|Android 15| -|Android 14| -|Android 13| -|Android 13| -|Android 12| -|Android 11| -|Android 10| -|Android 09| -|Android 08| - -## Supported Country - -You can simulate geolocation for the following countries in TestMu AI's Real Device App Testing. Set the geolocation [country code](/support/docs/appium-ip-geolocation/#supported-ip-geolocations) if you want to enable the same in your test. - -## Use Cases - -- Localized pricing or currency display -- Regional push notifications or offers -- Language-specific UI behavior -- App feature availability by country -- Compliance with local regulations or restrictions diff --git a/static/docs/ip-geolocation-on-real-devices-browser.md b/static/docs/ip-geolocation-on-real-devices-browser.md deleted file mode 100644 index 77c8bb74c..000000000 --- a/static/docs/ip-geolocation-on-real-devices-browser.md +++ /dev/null @@ -1,43 +0,0 @@ -# IP Geolocation on Real Devices Browser - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Understanding how your website behaves across different geographies is essential for delivering a personalized and compliant user experience. Whether you're targeting region-specific pricing, content, or legal requirements, geolocation simulation is a critical part of cross-border testing. - -With TestMu AI’s Geolocation Change feature for Real Device Browsers, you can manually simulate a different country during a live browser testing session, on actual Android devices. This allows you to test how your mobile website responds to users from specific locations without using a VPN or emulator. - -## Update Geolocation for Real Devices on TestMu AI - -**Step 1:** Click on the **Real Devices** > **Browser Testing** - -**Step 2:** First, select the **web browser** you want to test on. Then, choose the real Android device you’d like to run the session on. - -**Step 3:** Once the session launches, open the **IP Geolocation** option from the top toolbar. - -**Step 4:** Use the search bar to find the country or location you want to simulate. Select the desired location from the results. - -## Supported Web Browsers - - IP Geolocation is available on below Browsers : - -| Browser | -|----------------------| -|Google Chrome| -|Firefox| -|Samsung Internet| -|Microsoft Edge| -|UC Browser Turbo| -|Opera| -|Yandex| - -## Supported Country - -You can simulate geolocation for the following countries in TestMu AI's Real Device Browser Testing. Set the geolocation [country code](/support/docs/appium-ip-geolocation/#supported-ip-geolocations) if you want to enable the same in your test. - -## Use Cases - -- Validate localized content like pricing, languages, or offers -- Test regional cookie consent banners or data privacy flows -- Verify geo-targeted ads and tracking scripts -- Check SEO rendering for different countries -- Compliance with local regulations or restrictions diff --git a/static/docs/jasmine-with-karma-running-jasmine-tests-on-testmu-selenium-grid.md b/static/docs/jasmine-with-karma-running-jasmine-tests-on-testmu-selenium-grid.md deleted file mode 100644 index 1d3628fa3..000000000 --- a/static/docs/jasmine-with-karma-running-jasmine-tests-on-testmu-selenium-grid.md +++ /dev/null @@ -1,106 +0,0 @@ -# Selenium With Jasmine and Karma - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Run Jasmine BDD tests with Karma on the TestMu AI cloud grid. This guide covers setup, running a sample test, configuring capabilities, and testing locally hosted pages. - -**Sample repo** -All the code used in this guide is available in the sample repository. - - View on GitHub - -## Prerequisites - -Complete these steps before running Jasmine tests with Karma on TestMu AI. - -1. Create a [TestMu AI account](https://www.testmuai.com/register/?redirectTo=https://accounts.lambdatest.com/dashboard) and get your username and access key from the dashboard. -2. Install **NodeJS** and **npm** from [nodejs.org](https://nodejs.org/en/). -3. Install [Node.js](https://nodejs.org/) (latest LTS version recommended). - -## Step 1: Clone the Sample Project - -Clone the TestMu AI Jasmine Karma sample repository to your local machine. - -```bash -git clone https://github.com/LambdaTest/karma-jasmine-sample -cd karma-jasmine-sample -``` - -Install the required dependencies: -```bash -npm install -``` - -## Step 2: Set Your Credentials - -Set your TestMu AI username and access key as environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -## Step 3: Configure Your Test Capabilities - -Update the capabilities in the `karma.conf.js` file to define the browser and platform settings. - -``` js -customLaunchers: { -chrome: { -base: 'WebDriver', -config: webdriverConfig, -browserName: 'chrome', -platform: 'windows 10', -version: '71.0', -name: 'Karma With Heartbeat', -user: process.env.LT_USERNAME, -accessKey: process.env.LT_ACCESS_KEY, -pseudoActivityInterval: 15000 // 15000 ms heartbeat to avoid timeouts -} -} -``` - -Generate capabilities for your test requirements with the [Capabilities Generator](https://www.testmuai.com/capabilities-generator/). - -## Step 4: Run the Test - -Execute the test using the following command. - -```bash -karma start karma.conf.js -``` - -Or run the test using: -```bash -npm test -``` - -## Step 5: View Your Results - -After running the test, view your results on the [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build). - -The dashboard provides: -- Video recordings of each test session -- Screenshots captured at each step -- Console logs from the browser -- Network logs for debugging -- Detailed command logs - -## Run Jasmine Tests Using Agent Skills - -Use AI coding assistants to generate and run Jasmine tests with the TestMu AI Agent Skill. - -The [jasmine-skill](https://github.com/LambdaTest/agent-skills/tree/main/jasmine-skill) is part of [TestMu AI Agent Skills](https://github.com/LambdaTest/agent-skills/) - structured packages that teach AI coding assistants how to write production-grade test automation. - -Install the skill: - -```bash -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/jasmine-skill .claude/skills/ - -# For Cursor / Copilot -cp -r agent-skills/jasmine-skill .cursor/skills/ -``` - -Install all available framework skills at once by cloning the repository directly into your tool's skills directory (e.g., `.claude/skills/`, `.cursor/skills/`). diff --git a/static/docs/java-with-playwright.md b/static/docs/java-with-playwright.md deleted file mode 100644 index 6eb0ec145..000000000 --- a/static/docs/java-with-playwright.md +++ /dev/null @@ -1,171 +0,0 @@ -# Java with Playwright: Running Your First Test - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Run Playwright tests written in Java, either with plain Playwright or with JUnit, across 50+ real browsers and operating systems on the TestMu AI cloud platform. The setup is the same for both: you connect Playwright to the grid over a CDP WebSocket. This guide covers the shared steps once, then gives you a per-framework setup in the tabs below. - -## Prerequisites - -1. A TestMu AI **Username** and **Access Key**. Get them from your TestMu AI Profile. Don't have an account? Sign up for free. -2. [JDK](https://www.oracle.com/java/technologies/downloads/) 8 or later and [Apache Maven](https://maven.apache.org/) installed. The sample projects are Maven projects, so Maven resolves the dependencies when you build. -3. Clone the sample repository (each framework lives in its own subdirectory): - -**Sample repo** - View on GitHub - -```bash -git clone https://github.com/LambdaTest/playwright-sample.git -cd playwright-sample -``` - -## Set your credentials - -Set your Username and Access Key as environment variables. Pick your operating system: - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME=${ YOUR_LAMBDATEST_USERNAME()} -set LT_ACCESS_KEY=${ YOUR_LAMBDATEST_ACCESS_KEY()}`} - -## Set Up and Run in Your Framework - -Both frameworks build a `capabilities` object (browser, version, platform, and your `LT:Options`) and connect Playwright to the grid at `wss://cdp.lambdatest.com/playwright`. What differs is how the test is structured. Pick your framework: - -Plain Playwright with a `main()` method and no test runner. - -1. Go to the sample directory: - -```bash -cd playwright-java -``` - -2. The test lives in `PlaywrightTestSingle.java`. It builds capabilities from your env vars, connects to the grid, runs a search, and reports status: - -```java title="PlaywrightTestSingle.java" -package com.lambdatest; - -import com.google.gson.JsonObject; -import com.microsoft.playwright.*; - -import java.net.URLEncoder; - -public class PlaywrightTestSingle { -public static void main(String[] args) { -try (Playwright playwright = Playwright.create()) { -JsonObject capabilities = new JsonObject(); -JsonObject ltOptions = new JsonObject(); - -String user = System.getenv("LT_USERNAME"); -String accessKey = System.getenv("LT_ACCESS_KEY"); - -capabilities.addProperty("browsername", "Chrome"); // Browsers allowed: `Chrome`, `MicrosoftEdge`, `pw-chromium`, `pw-firefox` and `pw-webkit` -capabilities.addProperty("browserVersion", "latest"); -ltOptions.addProperty("platform", "Windows 10"); -ltOptions.addProperty("name", "Playwright Test"); -ltOptions.addProperty("build", "Playwright Testing in Java"); -ltOptions.addProperty("user", user); -ltOptions.addProperty("accessKey", accessKey); -capabilities.add("LT:Options", ltOptions); - -BrowserType chromium = playwright.chromium(); -String caps = URLEncoder.encode(capabilities.toString(), "utf-8"); -String cdpUrl = "wss://cdp.lambdatest.com/playwright?capabilities=" + capabilities; -Browser browser = chromium.connect(cdpUrl); -Page page = browser.newPage(); -try { -page.navigate("https://www.duckduckgo.com"); -Locator locator = page.locator("#search_form_input_homepage"); -locator.click(); -page.fill("#search_form_input_homepage", "LambdaTest"); -page.keyboard().press("Enter"); -String title = page.title(); - -if (title.equals("LambdaTest at DuckDuckGo")) { -// Use the following code to mark the test status. -setTestStatus("passed", "Title matched", page); -} else { -setTestStatus("failed", "Title not matched", page); -} - -} catch (Exception err) { -setTestStatus("failed", err.getMessage(), page); -err.printStackTrace(); -} -browser.close(); -} catch (Exception err) { -err.printStackTrace(); -} -} - -public static void setTestStatus(String status, String remark, Page page) { -Object result; -result = page.evaluate("_ => {}", "lambdatest_action: { \"action\": \"setTestStatus\", \"arguments\": { \"status\": \"" + status + "\", \"remark\": \"" + remark + "\"}}"); -} -} -``` - -3. Run the test: - -```bash -mvn -Dexec.mainClass="com.lambdatest.PlaywrightTestSingle" -Dexec.classpathScope=test test-compile exec:java -``` - -JUnit runs the same connection through a `@DataProvider`, so a single test runs across multiple browser configurations (here Chrome and Microsoft Edge). - -1. Go to the sample directory: - -```bash -cd playwright-java-junit -``` - -2. `LTCapability.java` supplies the parameterized capabilities: - -```java title="LTCapability.java" -import com.google.gson.JsonObject; -import com.tngtech.JUnit.junit.dataprovider.DataProvider; - -public class LTCapability { -@DataProvider -public static Object[] getDefaultTestCapability() { -JsonObject capabilities1 = new JsonObject(); -JsonObject ltOptions1 = new JsonObject(); - -String user = System.getenv("LT_USERNAME"); -String accessKey = System.getenv("LT_ACCESS_KEY"); - -capabilities1.addProperty("browserName", "Chrome"); // Browsers allowed: `Chrome`, `MicrosoftEdge`, `pw-chromium`, `pw-firefox` and `pw-webkit` -capabilities1.addProperty("browserVersion", "latest"); -ltOptions1.addProperty("platform", "Windows 10"); -ltOptions1.addProperty("name", "Playwright Test"); -ltOptions1.addProperty("build", "Playwright Testing using Junit"); -ltOptions1.addProperty("user", user); -ltOptions1.addProperty("accessKey", accessKey); -capabilities1.add("LT:Options", ltOptions1); - -JsonObject capabilities2 = new JsonObject(); -JsonObject ltOptions2 = new JsonObject(); -capabilities2.addProperty("browserName", "MicrosoftEdge"); // Browsers allowed: `Chrome`, `MicrosoftEdge`, `pw-chromium`, `pw-firefox` and `pw-webkit` -capabilities2.addProperty("browserVersion", "latest"); -ltOptions2.addProperty("platform", "Windows 10"); -ltOptions2.addProperty("name", "Playwright Test"); -ltOptions2.addProperty("build", "Playwright Testing using Junit"); -ltOptions2.addProperty("user", user); -ltOptions2.addProperty("accessKey", accessKey); -capabilities2.add("LT:Options", ltOptions2); -return new Object[]{ -capabilities1,capabilities2 -}; -} -} -``` - -3. Run the test: - -```bash -mvn clean test -``` - -## View your test results - -Go to the [TestMu AI Web Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) to see your Playwright Java test results. diff --git a/static/docs/java-with-selenium-running-java-automation-scripts-on-testmu-selenium-grid.md b/static/docs/java-with-selenium-running-java-automation-scripts-on-testmu-selenium-grid.md deleted file mode 100644 index 51472c244..000000000 --- a/static/docs/java-with-selenium-running-java-automation-scripts-on-testmu-selenium-grid.md +++ /dev/null @@ -1,374 +0,0 @@ -# Selenium With Java - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Run your Java Selenium tests on the TestMu AI cloud grid across 3000+ browser and OS combinations. The setup is the same for every framework: you connect to the grid, pass your capabilities, and run. This guide walks through that shared flow once, then gives you a per-framework quickstart for **TestNG, JUnit, Cucumber, Selenide, Gauge, Geb, Serenity**, and the **Java SDK**. - -## Prerequisites - -1. [Create a TestMu AI account](https://www.testmuai.com/register/) if you don't have one. -2. Get your **Username** and **Access Key** from the [TestMu AI Dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard). -3. Install the [Java Development Kit (JDK)](https://www.oracle.com/java/technologies/downloads/) 11 or later. -4. Install [Apache Maven](https://maven.apache.org/). The framework sample projects below all build with Maven. - -## Set your credentials - -Every framework authenticates the same way: your Username and Access Key are passed in the grid URL. Set them as environment variables so you don't hard-code them. Pick your operating system: - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME=${ YOUR_LAMBDATEST_USERNAME()} -set LT_ACCESS_KEY=${ YOUR_LAMBDATEST_ACCESS_KEY()}`} - -## How the sample test works - -All the framework repos below run the **same** sample test, so you only need to understand it once. The test opens the [to-do app](https://lambdatest.github.io/sample-todo-app/), marks the first two items done, adds a new item, and verifies it appears: - -```java -driver.get("https://lambdatest.github.io/sample-todo-app/"); -driver.findElement(By.name("li1")).click(); -driver.findElement(By.name("li2")).click(); -driver.findElement(By.id("sampletodotext")).sendKeys("Yey, Let's add it to list"); -driver.findElement(By.id("addbutton")).click(); -``` - -The driver is a `RemoteWebDriver` pointed at the grid, with your browser/OS choices passed through `LT:Options`: - -```java -ChromeOptions browserOptions = new ChromeOptions(); -browserOptions.setPlatformName("Windows 10"); -browserOptions.setBrowserVersion("latest"); - -HashMap ltOptions = new HashMap(); -ltOptions.put("build", "Java Selenium Build"); -ltOptions.put("name", "Java Selenium Test"); -ltOptions.put("w3c", true); -browserOptions.setCapability("LT:Options", ltOptions); - -driver = new RemoteWebDriver( -new URL("https://" + username + ":" + accesskey + "@hub.lambdatest.com/wd/hub"), -browserOptions); -``` - -Use the [Capabilities Generator](https://www.testmuai.com/capabilities-generator/) to build an `LT:Options` block for any browser, version, and OS combination. - -**What changes between frameworks is only how that test is *structured and run***: the runner, its setup/teardown hooks, and any config files. That's what each tab below covers. - -## Run a test in your framework - -Each tab lists just the framework-specific pieces. Clone the matching repo (it contains the full, ready-to-run project), then run the command shown. - -TestNG is the most common choice. It wraps the shared test with `@BeforeClass` (create the driver) and `@AfterClass` (report status and quit), and drives cross-browser runs from a `testng.xml` suite. - -1. Clone the [sample GitHub project](https://github.com/LambdaTest/Java-TestNG-Selenium): - -```bash -git clone https://github.com/LambdaTest/Java-TestNG-Selenium -cd Java-TestNG-Selenium -``` - -2. Set your browser and OS in the `testng.xml` suite. Listing several environments with `parallel="tests"` and a `thread-count` runs them concurrently: - -```xml title="testng.xml" - - - - - - - - - - - - -``` - -3. Run a single test, or the parallel suite: - -```bash -mvn test -D suite=single.xml -mvn test -D suite=parallel.xml -``` - -JUnit runs the same test as TestNG. The only difference is the lifecycle annotations: it uses `@Before` / `@After` instead of `@BeforeClass` / `@AfterClass`. - -1. Clone the [sample GitHub project](https://github.com/LambdaTest/junit-selenium-sample): - -```bash -git clone https://github.com/LambdaTest/junit-selenium-sample -cd junit-selenium-sample -``` - -2. Set your browser and OS in the `LT:Options` block: - -```java -ChromeOptions browserOptions = new ChromeOptions(); -browserOptions.setPlatformName("Windows 10"); -browserOptions.setBrowserVersion("latest"); - -HashMap ltOptions = new HashMap(); -ltOptions.put("build", "LambdaTestSampleApp"); -ltOptions.put("name", "LambdaTestJavaSample"); -ltOptions.put("w3c", true); -browserOptions.setCapability("LT:Options", ltOptions); -``` - -3. Run a single test or the parallel profile: - -```bash -mvn test -P single -mvn test -P parallel -``` - -Parallel JUnit runs need a custom `Parallelized` runner (extends `Parameterized` with a `ThreadPoolScheduler`). The cloned repo includes it; see [the sample](https://github.com/LambdaTest/junit-selenium-sample) for the full class. - -Cucumber is BDD on top of TestNG: you write scenarios in plain-language **feature files**, and a runner maps each step to WebDriver code. The grid setup lives in the runner. - -1. Clone the [sample GitHub project](https://github.com/LambdaTest/cucumber-testng-sample): - -```bash -git clone https://github.com/LambdaTest/cucumber-testng-sample -cd cucumber-testng-sample -``` - -2. Write your scenario in Gherkin: - -```gherkin title="todo.feature" -Feature: Add new item to ToDo list -Scenario: Add an item to the list -Given user is on home Page -When select First Item -Then select second item -Then add new item -Then verify added item -``` - -Each step maps to a WebDriver action in the step definitions, and the runner (`TestRunner.java`, annotated with `@CucumberOptions`) creates the `RemoteWebDriver` against the grid. - -3. Set your browser and OS in the runner (`TestRunner.java`): - -```java -ChromeOptions browserOptions = new ChromeOptions(); -browserOptions.setPlatformName(platform); -browserOptions.setBrowserVersion("latest"); - -HashMap ltOptions = new HashMap(); -ltOptions.put("build", "Your Build Name"); -ltOptions.put("w3c", true); -browserOptions.setCapability("LT:Options", ltOptions); -``` - -4. Run the test: - -```bash -mvn test -``` - -`@CucumberOptions` sets the `features` and `glue` paths; `TestNGCucumberRunner` feeds scenarios into TestNG. The full runner and step definitions are in the cloned repo. - -Selenide wraps WebDriver with a concise API (`$()`, `open()`) and reads browser/OS choices from **JSON config files** instead of inline capabilities. - -1. Clone and compile the [sample GitHub project](https://github.com/LambdaTest/selenide-testng-sample): - -```bash -git clone https://github.com/LambdaTest/selenide-testng-sample -cd selenide-testng-sample -mvn compile -``` - -2. Set your browser and OS. Environments are declared in a config file, one block per browser: - -```json title="parallel.conf.json" -{ -"server": "hub.lambdatest.com", -"user": "YOUR_USERNAME", -"key": "YOUR_ACCESS_KEY", -"capabilities": { "build": "Java Selenide Parallel" }, -"environments": { -"chrome": { "platformName": "Windows 10", "browserName": "chrome", "browserVersion": "latest" }, -"firefox": { "platformName": "Windows 10", "browserName": "firefox", "browserVersion": "latest" }, -"safari": { "platformName": "macOS Mojave", "browserName": "safari", "browserVersion": "latest" } -} -} -``` - -The test reads that config in `@BeforeMethod`, calls `WebDriverRunner.setWebDriver(driver)`, then uses Selenide syntax. - -3. Run a single test or the parallel profile: - -```bash -mvn test -P single -mvn test -P parallel -``` - -Gauge is a BDD runner where steps are Java methods annotated with `@Step`, and the driver comes from a `DriverFactory`. - -**Prerequisite:** install the [Gauge CLI](https://docs.gauge.org/). - -1. Clone and compile the [sample GitHub project](https://github.com/LambdaTest/gauge-selenium-sample): - -```bash -git clone https://github.com/LambdaTest/gauge-selenium-sample -cd gauge-selenium-sample -mvn compile -``` - -2. Define your steps. Steps map plain-language names to WebDriver actions: - -```java title="StepImplementation_ToDo.java (excerpt)" -@Step("Open the todo app") -public void gotoApp() { -driver.get("https://lambdatest.github.io/sample-todo-app/"); -assertEquals(driver.getTitle(), "Sample page - lambdatest.com"); -} - -@Step("Add new item ") -public void addNewItem(String itemName) { -driver.findElement(By.id("sampletodotext")).sendKeys(itemName); -driver.findElement(By.id("addbutton")).click(); -} -``` - -3. Set your browser and OS in the `DriverFactory` capabilities: - -```java -ChromeOptions browserOptions = new ChromeOptions(); -browserOptions.setPlatformName("Windows 10"); -browserOptions.setBrowserVersion("latest"); - -HashMap ltOptions = new HashMap(); -ltOptions.put("build", "LambdaTestSampleApp"); -ltOptions.put("name", "LambdaTestJavaSample"); -ltOptions.put("w3c", true); -browserOptions.setCapability("LT:Options", ltOptions); -``` - -4. Run the test: - -```bash -mvn test -``` - -Parallel runs are configured by adding multiple browser specs under the `env` folder of the cloned project. - -Geb is a Groovy wrapper around WebDriver. Its distinguishing trait is that capabilities live in a **separate JSON file** passed via a Maven property. - -1. Clone and compile the [sample GitHub project](https://github.com/LambdaTest/geb-parallel-sample-code): - -```bash -git clone https://github.com/LambdaTest/geb-parallel-sample-code -cd geb-parallel-sample-code -mvn compile -``` - -2. Set your browser and OS in the capabilities file: - -```json title="capabilities.json" -{ -"build": "GebFirstTest", -"platformName": "Windows 10", -"browserName": "firefox", -"browserVersion": "latest" -} -``` - -3. Run the test, passing that file via the `lambdageb.capabilities` property: - -```bash -mvn -Dlambdageb.capabilities=capabilities clean test -``` - -Serenity is BDD on top of Cucumber with rich reporting. You connect to the grid with a custom `DriverSource` and drive scenarios with page objects. - -1. Clone the [sample GitHub project](https://github.com/LambdaTest/Serenity-Selenium-Sample): - -```bash -git clone https://github.com/LambdaTest/Serenity-Selenium-Sample -cd Serenity-Selenium-Sample -``` - -Scenarios are written in Gherkin and the grid connection lives in a `DriverSource` implementation (`LambdaTestSerenityDriver`) that builds `LT:Options` from Serenity's environment variables. - -2. Run a single scenario or the parallel profile: - -```bash -mvn verify -P single -mvn verify -P parallel -``` - -Parallel Serenity runs use one runner class per browser, each extending a shared base test. The full `DriverSource` and runners are in the cloned repo. - -Tesbo is a keyword-driven hybrid framework on Selenium: you write scripts in plain English, and the grid setup lives in `config.json`. - -1. Clone the [sample GitHub project](https://github.com/LambdaTest/LamdaTest_Tesbo_Demo): - -```bash -git clone https://github.com/LambdaTest/LamdaTest_Tesbo_Demo -cd LamdaTest_Tesbo_Demo -``` - -2. Set your browser and OS in `config.json` (set `IsGrid` to `true` and point `seleniumAddress` at the grid): - -```json title="config.json" -{ -"run": { -"seleniumAddress": "https://{userName}:{ApiKey}@hub.lambdatest.com/wd/hub", -"browser": { "name": ["chrome"] }, -"capabilities": { -"chrome": { -"build": "Tesbo_With_TestMuAI", -"name": "Tesbo", -"platformName": "Windows 10", -"browserName": "Chrome", -"browserVersion": "latest" -} -}, -"IsGrid": true -} -} -``` - -3. Run the test from your IDE. Hit the run button, and your scripts execute on the grid. - -The **Java SDK** is the zero-code option: it runs your **existing** TestNG tests on the grid unchanged, injecting credentials and capabilities from a YAML file via a Java agent. Use this when you already have a local suite and don't want to edit test code. See the [SDK reference project on GitHub](https://github.com/Lambdatest/lambdatest-java-selenium-sdk). - -1. Add the SDK dependency and agent to your `pom.xml` (the agent is attached to Surefire via `-javaagent`): - -```xml title="pom.xml (key additions)" - -io.github.lambdatest -lambdatest-selenium-java-sdk -1.0.1 - - -``` - -2. Create `lambdatest.yml` with your credentials, platforms, and features. The SDK loads it automatically: - -```yaml title="lambdatest.yml" -username: YOUR_LAMBDATEST_USERNAME -accesskey: YOUR_LAMBDATEST_ACCESS_KEY - -platforms: -- browserName: Chrome -browserVersion: latest -platformName: Windows 10 - -build: SDK Build v1 -name: SDK Test -video: true -tunnel: true -``` - -3. Register the SDK's TestNG listener in `testng.xml`, then run. Your existing test classes execute as-is: - -```bash -mvn test -D suite=testng.xml -``` - -## View your results - -Your test results, including video, network logs, and command-by-command execution, appear on the [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build). diff --git a/static/docs/javascript-with-playwright.md b/static/docs/javascript-with-playwright.md deleted file mode 100644 index 509b405aa..000000000 --- a/static/docs/javascript-with-playwright.md +++ /dev/null @@ -1,332 +0,0 @@ -# JavaScript and TypeScript with Playwright - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Run Playwright tests written in plain JavaScript, with Jest, or in TypeScript across 50+ real browsers and operating systems on the TestMu AI cloud platform. The connection to the grid is the same whichever you use: you connect Playwright over a CDP WebSocket. This guide covers the shared steps once, then gives you the setup for each in the tabs below. - -## Prerequisites - -1. A TestMu AI **Username** and **Access Key**. Get them from your TestMu AI Profile. Don't have an account? Sign up for free. -2. [Node.js](https://nodejs.org/en/) and npm installed. -3. Clone the sample repository (each option lives in its own subdirectory): - -**Sample repo** - View on GitHub - -```bash -git clone https://github.com/LambdaTest/playwright-sample.git -cd playwright-sample -``` - -## Set your credentials - -Set your Username and Access Key as environment variables. Pick your operating system: - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME=${ YOUR_LAMBDATEST_USERNAME()} -set LT_ACCESS_KEY=${ YOUR_LAMBDATEST_ACCESS_KEY()}`} - -## How the test connects - -Each option builds a `capabilities` object (browser, version, platform, and your `LT:Options`) and connects Playwright to the grid at `wss://cdp.lambdatest.com/playwright`: - -```js -wsEndpoint: `wss://cdp.lambdatest.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` -``` - -What changes between them is only the setup file that holds this connection. Pick yours below. - -## Set Up and Run Your Tests - -1. Go to the sample directory and install dependencies: - -```bash -cd playwright-test-js -npm install -``` - -2. The grid connection lives in `lambdatest-setup.js`, a Playwright test fixture that patches the capabilities per project and reports test status: - -```js title="lambdatest-setup.js" -/** -* Add the file in your test suite to run tests on LambdaTest. -* Import `test` object from this file in the tests. -*/ -const base = require('@playwright/test') -const path = require('path') -const { chromium } = require('playwright') -const cp = require('child_process'); -const playwrightClientVersion = cp.execSync('npx playwright --version').toString().trim().split(' ')[1]; - -// LambdaTest capabilities -const capabilities = { -'browserName': 'Chrome', // Browsers allowed: `Chrome`, `MicrosoftEdge`, `pw-chromium`, `pw-firefox` and `pw-webkit` -'browserVersion': 'latest', -'LT:Options': { -'platform': 'Windows 10', -'build': 'Playwright Build', -'name': 'Playwright Test', -'user': process.env.LT_USERNAME, -'accessKey': process.env.LT_ACCESS_KEY, -'network': true, -'video': true, -'console': true, -'tunnel': false, // Add tunnel configuration if testing locally hosted webpage -'tunnelName': '', // Optional -'geoLocation': '', // country code can be fetched from https://www.lambdatest.com/capabilities-generator/ -'playwrightClientVersion': playwrightClientVersion -} -} - -// Patching the capabilities dynamically according to the project name. -const modifyCapabilities = (configName, testName) => { -let config = configName.split('@lambdatest')[0] -let [browserName, browserVersion, platform] = config.split(':') -capabilities.browserName = browserName ? browserName : capabilities.browserName -capabilities.browserVersion = browserVersion ? browserVersion : capabilities.browserVersion -capabilities['LT:Options']['platform'] = platform ? platform : capabilities['LT:Options']['platform'] -capabilities['LT:Options']['name'] = testName -} - -const getErrorMessage = (obj, keys) => keys.reduce((obj, key) => (typeof obj == 'object' ? obj[key] : undefined), obj) - -exports.test = base.test.extend({ -page: async ({ page, playwright }, use, testInfo) => { -// Configure LambdaTest platform for cross-browser testing -let fileName = testInfo.file.split(path.sep).pop() -if (testInfo.project.name.match(/lambdatest/)) { -modifyCapabilities(testInfo.project.name, `${testInfo.title} - ${fileName}`) - -const browser = await chromium.connect({ -wsEndpoint: `wss://cdp.lambdatest.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` -}) - -const ltPage = await browser.newPage(testInfo.project.use) -await use(ltPage) - -const testStatus = { -action: 'setTestStatus', -arguments: { -status: testInfo.status, -remark: getErrorMessage(testInfo, ['error', 'message']) -} -} -await ltPage.evaluate(() => {}, -`lambdatest_action: ${JSON.stringify(testStatus)}`) -await ltPage.close() -await browser.close() -} else { -// Run tests in local in case of local config provided -await use(page) -} -} -}) -``` - -3. Run the test: - -```bash -npm run test -``` - -1. Go to the sample directory and install dependencies: - -```bash -cd playwright-jest-js -npm install -``` - -2. `jest-playwright.config.js` defines a `connectOptions` endpoint per browser (Chromium, Firefox, WebKit), each with its own capabilities: - -```js title="jest-playwright.config.js" -const cp = require('child_process'); -const playwrightClientVersion = cp.execSync('npx playwright --version').toString().trim().split(' ')[1]; - -const caps_chromium = { -'browserName': 'pw-chromium', // Browsers allowed: `Chrome`, `MicrosoftEdge`, `pw-chromium`, `pw-firefox` and `pw-webkit` -'browserVersion': 'latest', -'LT:Options': { -'platform': 'Windows 10', -'build': 'Playwright-Jest Build', -'name': 'Playwright-Jest Test on Chromium', -'user': process.env.LT_USERNAME, -'accessKey': process.env.LT_ACCESS_KEY, -'network': true, -'video': true, -'console': true, -'tunnel': false, // Add tunnel configuration if testing locally hosted webpage -'tunnelName': '', // Optional -'geoLocation': '', // country code can be fetched from https://www.lambdatest.com/capabilities-generator/ -'playwrightClientVersion': playwrightClientVersion -} -}; - -const caps_firefox = { -'browserName': 'pw-firefox', -'browserVersion': 'latest', -'LT:Options': { -'platform': 'Windows 10', -'build': 'Playwright-Jest Build ', -'name': 'Playwright-Jest Test on Firefox', -'user': process.env.LT_USERNAME, -'accessKey': process.env.LT_ACCESS_KEY, -'network': true, -'video': true, -'console': true, -'tunnel': false, -'tunnelName': '', -'geoLocation': '', -} -}; - -const caps_webkit = { -'browserName': 'pw-webkit', -'browserVersion': 'latest', -'LT:Options': { -'platform': 'Windows 10', -'build': 'Playwright-Jest Build', -'name': 'Playwright-Jest Test on WebKit', -'user': process.env.LT_USERNAME, -'accessKey': process.env.LT_ACCESS_KEY, -'network': true, -'video': true, -'console': true, -'tunnel': false, -'tunnelName': '', -'geoLocation': '', -} -}; - -module.exports = { -connectOptions: { -chromium: { -wsEndpoint: `wss://cdp.lambdatest.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(caps_chromium))}` -}, -firefox: { -wsEndpoint: `wss://cdp.lambdatest.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(caps_firefox))}` -}, -webkit: { -wsEndpoint: `wss://cdp.lambdatest.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(caps_webkit))}` -} -}, -browsers: ['chromium', 'firefox', 'webkit'], -} -``` - -3. Run the test: - -```bash -npm run test -``` - -1. Go to the sample directory and install dependencies: - -```bash -cd playwright-test-ts -npm install -``` - -2. The grid connection lives in `lambdatest-setup.ts` (the TypeScript version of the fixture): - -```ts title="lambdatest-setup.ts" -/** -* Add the file in your test suite to run tests on LambdaTest. -* Import `test` object from this file in the tests. -*/ - -import * as base from "@playwright/test"; -import path from "path"; -import { chromium } from "playwright"; - -// LambdaTest capabilities -const capabilities = { -browserName: "Chrome", // Browsers allowed: `Chrome`, `MicrosoftEdge`, `pw-chromium`, `pw-firefox` and `pw-webkit` -browserVersion: "latest", -"LT:Options": { -platform: "Windows 10", -build: "Playwright TypeScript Build", -name: "Playwright TypeScript Test", -user: process.env.LT_USERNAME, -accessKey: process.env.LT_ACCESS_KEY, -network: true, -video: true, -console: true, -tunnel: false, // Add tunnel configuration if testing locally hosted webpage -tunnelName: "", // Optional -geoLocation: '', // country code can be fetched from https://www.lambdatest.com/capabilities-generator/ -}, -}; - -// Patching the capabilities dynamically according to the project name. -const modifyCapabilities = (configName, testName) => { -let config = configName.split("@lambdatest")[0]; -let [browserName, browserVersion, platform] = config.split(":"); -capabilities.browserName = browserName -? browserName -: capabilities.browserName; -capabilities.browserVersion = browserVersion -? browserVersion -: capabilities.browserVersion; -capabilities["LT:Options"]["platform"] = platform -? platform -: capabilities["LT:Options"]["platform"]; -capabilities["LT:Options"]["name"] = testName; -}; - -const getErrorMessage = (obj, keys) => -keys.reduce( -(obj, key) => (typeof obj == "object" ? obj[key] : undefined), -obj -); - -const test = base.test.extend({ -page: async ({ page, playwright }, use, testInfo) => { -// Configure LambdaTest platform for cross-browser testing -let fileName = testInfo.file.split(path.sep).pop(); -if (testInfo.project.name.match(/lambdatest/)) { -modifyCapabilities( -testInfo.project.name, -`${testInfo.title} - ${fileName}` -); - -const browser = await chromium.connect({ -wsEndpoint: `wss://cdp.lambdatest.com/playwright?capabilities=${encodeURIComponent( -JSON.stringify(capabilities) -)}`, -}); - -const ltPage = await browser.newPage(testInfo.project.use); -await use(ltPage); - -const testStatus = { -action: "setTestStatus", -arguments: { -status: testInfo.status, -remark: getErrorMessage(testInfo, ["error", "message"]), -}, -}; -await ltPage.evaluate(() => {}, -`lambdatest_action: ${JSON.stringify(testStatus)}`); -await ltPage.close(); -await browser.close(); -} else { -// Run tests in local in case of local config provided -await use(page); -} -}, -}); - -export default test; -``` - -3. Run the test: - -```bash -npm run test -``` - -## View your test results - -Go to the [TestMu AI Web Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) to see your Playwright test results. diff --git a/static/docs/javascript-with-selenium-running-javascript-automation-scripts-on-testmu-selenium-grid.md b/static/docs/javascript-with-selenium-running-javascript-automation-scripts-on-testmu-selenium-grid.md deleted file mode 100644 index f69b24474..000000000 --- a/static/docs/javascript-with-selenium-running-javascript-automation-scripts-on-testmu-selenium-grid.md +++ /dev/null @@ -1,343 +0,0 @@ -# Selenium With JavaScript - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Run your JavaScript Selenium tests on the TestMu AI cloud grid across 3000+ browser and OS combinations. The setup is the same for every framework: you connect to the grid and pass your capabilities. This guide covers that shared flow once, then gives you a per-framework quickstart in the tabs below. - -## Prerequisites - -1. [Create a TestMu AI account](https://www.testmuai.com/register/) if you don't have one. -2. Get your **Username** and **Access Key** from the [TestMu AI Dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard). -3. Install [Node.js](https://nodejs.org/en/) (v6 or newer) and npm. -4. Install the [Selenium JavaScript bindings](https://www.selenium.dev/downloads/). - -## Set your credentials - -Every framework authenticates the same way: your Username and Access Key are read from environment variables. Set them once. Pick your operating system: - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME=${ YOUR_LAMBDATEST_USERNAME()} -set LT_ACCESS_KEY=${ YOUR_LAMBDATEST_ACCESS_KEY()}`} - -## How the sample test works - -Every framework below connects to the grid at `hub.lambdatest.com/wd/hub` and passes your browser and OS choices through a capabilities object. A minimal one looks like this: - -```js -const capabilities = { -build: 'NodeJS build', -name: 'Test 1', -platformName: 'Windows 10', -browserName: 'chrome', -browserVersion: 'latest', -network: true, -visual: true, -console: true, -video: true -} -``` - -What changes between frameworks is only how those capabilities are wired in: an inline object, a `.conf.js` file, or a runner config. That is what each tab covers. - -Use the [Capabilities Generator](https://www.testmuai.com/capabilities-generator/) to build a capabilities block for any browser, version, and OS combination. - -## Run a test in your framework - -Each tab lists the framework-specific pieces. Clone the matching repo (it contains the full, ready-to-run project), set your browser and OS in its config, then run. - -Mocha reads its capabilities from a `conf/single.conf.js` file. - -1. Clone the [sample GitHub project](https://github.com/LambdaTest/mocha-selenium-sample): - -```bash -git clone https://github.com/LambdaTest/mocha-selenium-sample -cd mocha-selenium-sample -``` - -2. Set your browser and OS in `conf/single.conf.js`: - -```js title="conf/single.conf.js" -exports.capabilities = { -'build': 'Mocha-Selenium-Sample', -'name': 'Your Test Name', -'platformName':'Windows 10', -'browserName': 'chrome', -'browserVersion': 'latest', -'visual': false, -'network':false, -'console':false, -'tunnel': false -}; -``` - -3. Run the test: - -```bash -npm run single -``` - -Nightwatch reads its grid setup from `nightwatch.conf.js` in the cloned project. - -1. Clone the [sample GitHub project](https://github.com/LambdaTest/nightwatch-selenium-sample): - -```bash -git clone https://github.com/LambdaTest/nightwatch-selenium-sample -cd nightwatch-selenium-sample -``` - -2. Set your browser and OS in `nightwatch.conf.js`. -3. Run the test: - -```bash -npm run single -``` - -WebdriverIO reads user, key, and capabilities from `conf/single.conf.js`. - -1. Clone the [sample GitHub project](https://github.com/LambdaTest/webdriverio-selenium): - -```bash -git clone https://github.com/LambdaTest/webdriverio-selenium -cd webdriverio-selenium -``` - -2. Set your browser and OS in `conf/single.conf.js`: - -```js title="conf/single.conf.js" -exports.config = { -user: process.env.LT_USERNAME || "", -key: process.env.LT_ACCESS_KEY || "", -specs: ['./tests/specs/single_test.js'], -capabilities: [{ -browserName: 'chrome', -browserVersion: 'latest', -platformName: 'Windows 10', -name: "Test webdriverio", -build: "build 1", -}], -path: '/wd/hub', -hostname: 'hub.lambdatest.com', -port: 80, -framework: 'mocha', -mochaOpts: { ui: 'bdd' } -} -``` - -3. Run the test: - -```bash -npm run single -``` - -Jest defines capabilities inline in the test file, so there is no separate config to edit. - -1. Clone the [sample GitHub project](https://github.com/LambdaTest/jest-selenium-webdriver-sample): - -```bash -git clone https://github.com/LambdaTest/jest-selenium-webdriver-sample -cd jest-selenium-webdriver-sample -``` - -2. Set your browser and OS in the capabilities object: - -```js -const capabilities = { -build: 'jest-LambdaTest-Single', -browserName: 'chrome', -browserVersion: 'latest', -platformName: 'Windows 10', -}; -``` - -3. Run the test: - -```bash -npm test single.test.js -``` - -CucumberJS runs BDD feature files, with grid capabilities set in `conf/single.conf.js`. - -1. Clone the [sample GitHub project](https://github.com/LambdaTest/NodeJs-Cucumber-Selenium): - -```bash -git clone https://github.com/LambdaTest/NodeJs-Cucumber-Selenium -cd NodeJs-Cucumber-Selenium -``` - -2. Set your browser and OS in `conf/single.conf.js`: - -```js title="conf/single.conf.js" -capabilities: [{ -browserName: 'chrome', -platformName: 'Windows 10', -browserVersion: 'latest', -name: "cucumber-js-single-test", -build: "cucumber-js-LambdaTest-single" -}] -``` - -3. Run the test: - -```bash -npm run single -``` - -Jasmine runs through Karma, which defines cloud browsers in `customLaunchers` inside `karma.conf.js`. - -1. Clone the [sample GitHub project](https://github.com/LambdaTest/karma-jasmine-sample): - -```bash -git clone https://github.com/LambdaTest/karma-jasmine-sample -cd karma-jasmine-sample -``` - -2. Set your browser and OS in `karma.conf.js`: - -```js title="karma.conf.js" -customLaunchers: { -chrome: { -base: 'WebDriver', -config: webdriverConfig, -browserName: 'chrome', -platform: 'windows 10', -version: '71.0', -name: 'Karma With Heartbeat', -user: process.env.LT_USERNAME, -accessKey: process.env.LT_ACCESS_KEY, -pseudoActivityInterval: 15000 -} -} -``` - -3. Run the test: - -```bash -karma start karma.conf.js -``` - -Karma with Angular CLI defines cloud browsers in `customLaunchers` inside `karma.conf.js`. - -1. Clone the [sample GitHub project](https://github.com/LambdaTest/angular-karma-sample): - -```bash -git clone https://github.com/LambdaTest/angular-karma-sample -cd angular-karma-sample -``` - -2. Set your browser and OS in `karma.conf.js`: - -```js title="karma.conf.js" -customLaunchers: { -chrome: { -base: 'WebDriver', -config: webdriverConfig, -browserName: 'chrome', -platform: 'windows 10', -version: '71.0', -name: 'Karma With Heartbeat', -user: process.env.LT_USERNAME, -accessKey: process.env.LT_ACCESS_KEY, -pseudoActivityInterval: 5000 -} -} -``` - -3. Run the test: - -```bash -karma start karma.conf.js -``` - -TestCafe connects through an npm plugin rather than a sample repo, so there is nothing to clone. - -1. Install the plugin in your TestCafe project: - -```bash -npm install testcafe-browser-provider-lambdatest -``` - -2. Run your test against the grid, naming the browser and OS in the command: - -```bash -testcafe "lambdatest:Chrome@74.0:Windows 8" 'path/to/test/file.js' -``` - -NemoJS reads its capabilities from `nemo.config.js`. - -1. Clone the [sample GitHub project](https://github.com/LambdaTest/Nemo-Lambdatest-sample): - -```bash -git clone https://github.com/LambdaTest/Nemo-Lambdatest-sample -cd Nemo-Lambdatest-sample -``` - -2. Set your browser and OS in `nemo.config.js`: - -```js title="nemo.config.js" -"withCapabilities": [{ -"build": "LT Nemo Sample Tests", -"name": "LT Nemo sample test", -"platformName": "Windows 10", -"browserName": "Chrome", -"browserVersion": "latest", -"user": "env:LT_USERNAME", -"accessKey": "env:LT_ACCESS_KEY" -}] -``` - -3. Run the test: - -```bash -npx nemo -P chrome -``` - -CodeceptJS uses a WebDriver helper in `codecept.conf.js`, with an optional service to sync test names and statuses to the dashboard. - -1. Clone the [sample GitHub project](https://github.com/LambdaTest/lambdatest-codeceptjs-sample): - -```bash -git clone https://github.com/LambdaTest/lambdatest-codeceptjs-sample -cd lambdatest-codeceptjs-sample -``` - -2. Set your browser and OS in `codecept.conf.js`: - -```js title="codecept.conf.js" -helpers: { -WebDriver: { -url: 'http://google.com/ncr', -browser: 'chrome', -host: 'hub.lambdatest.com', -port: 80, -user: process.env.LT_USERNAME, -key: process.env.LT_ACCESS_KEY, -desiredCapabilities: { -name: '[CodeceptJS] Automation Sample', -build: '[CodeceptJS] Automation Sample', -platformName: 'Windows 11', -browserName: 'Chrome', -browserVersion: 'dev' -} -} -} -``` - -3. Run the test: - -```bash -npm test -``` - -## Legacy frameworks - -These frameworks are deprecated and kept only for existing suites. For new projects, use one of the frameworks above. - -- **Protractor** (end-of-life August 2023): [sample GitHub project](https://github.com/LambdaTest/protractor-selenium-sample). Set capabilities in the config file, then run `npm run single`. Migrate to WebdriverIO or [Playwright](/support/docs/playwright-testing/). -- **WD** (unmaintained): [sample GitHub project](https://github.com/LambdaTest/wd-selenium-sample). Set capabilities in `conf/single.conf.js`, then run `npm run single`. Migrate to WebdriverIO. -- **AngularJS** (end-of-life December 2021): [sample GitHub project](https://github.com/LambdaTest/angular-karma-sample). Set browsers in `karma.conf.js`, then run `karma start karma.conf.js`. Migrate to Angular with Karma. - -## View your results - -Your test results, including video, network logs, and command-by-command execution, appear on the [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build). diff --git a/static/docs/jenkins-with-hyperexecute.md b/static/docs/jenkins-with-hyperexecute.md deleted file mode 100644 index 2c38c81b1..000000000 --- a/static/docs/jenkins-with-hyperexecute.md +++ /dev/null @@ -1,78 +0,0 @@ -# Jenkins Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Jenkins is an open-source automation server that is easy to install and configure. It can also be used as a simple CI server or turned into a continuous delivery hub for any project. - -You can now integrate HyperExecute with your Jenkins project to shorten your test cycles. - -**How to Integrate HyperExecute with Jenkins** - -To integrate HyperExecute with your Jenkins project, follow the steps listed below. - -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** -Download or Clone the code sample from the TestMu AI GitHub repository to run the tests on the HyperExecute. - - View on GitHub - -1. Log Into your Jenkins account. You will be directed to your Jenkins dashboard. - -2. Select the project you want to integrate with HyperExecute. Or else, click on **+ New Item** on the left panel to create a new project. - -3. After opening your project, click on **Configure**. - -4. Click on **Build Steps**. - -5. Add a new build step and select **Execute Shell** if you are running your tests on MacOS. Alternatively, select **Execute Windows** batch command if you want to run your test on Windows. - -6. Copy and paste the following code snippet in the **Command** section of your Build Step. Click on **Save** to add your Build Step. - - - **For MacOS**: - -``` -curl -O https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute -chmod u+x hyperexecute -./hyperexecute --user --key --config hyperexecuteStatic.yaml -``` - - **For Windows**: - -``` -curl -O https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe -hyperexecute.exe -u -k -i -``` - - **Note**: Replace the placeholder values in the code with your username and access key for HyperExecute. - -7. Build your Jenkins project by clicking on **Build Now** and wait for a few minutes. - - After your build is successful, go to the [HyperExecute dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) to monitor the progress of your tests. - - The **HyperExecute Dashboard**: - - Click on the job to find out more details about your tests. - -You have successfully integrated your Jenkins project with HyperExecute. - -## Sample Jenkins File - -```bash -pipeline { -agent any - -stages { -# Download and Run Hyperexecute stage (optional) -stage('Download & Run Hyperexecute') { -steps { -# Download Hyperexecute CLI for macOS (adjust for other OS) -sh name: 'Download Hyperexecute CLI', script: 'wget https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute' -# Grant execute permission to the downloaded binary -sh name: 'Make Hyperexecute executable', script: 'chmod +x hyperexecute' -# Run Hyperexecute with user credentials and configuration file -sh name: 'Run Hyperexecute Tests', script: './hyperexecute --user --key --config -} -} -} -} -``` diff --git a/static/docs/jenkins-with-testmu.md b/static/docs/jenkins-with-testmu.md deleted file mode 100644 index bae7762e2..000000000 --- a/static/docs/jenkins-with-testmu.md +++ /dev/null @@ -1,114 +0,0 @@ -# Jenkins Plugin for Integration With TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Jenkins is a widely popular Continuous Integration (CI) tool. It is free to use, open-source software written in Java. Jenkins helps to boost your release agility by providing Continuous Integration services. You can trigger these services by a web application server or merely using a command line to hasten the deployment pipeline & release migration activities. Jenkins's popularity offers thousands of plugins to fast track your productivity. One such plugin to help you pace up automated cross browser testing is the TestMu AI Jenkins plugin. With TestMu AI Jenkins plugin, you will easily automate your Selenium test scripts by connecting your Jenkins CI instance to TestMu AI [Selenium grid](https://www.testmuai.com/blog/why-selenium-grid-is-ideal-for-automated-browser-testing/). TestMu AI Selenium grid will offer you a comprehensive library of 3000+ browsers and browser versions so you could achieve a higher test coverage while performing automation testing using the Selenium test suite. - -> TestMu AI Jenkins plugin will help you to: -> ->* Configure your TestMu AI credentials for your Jenkins jobs. ->* Set up and tear down the Lambda Tunnel binary file to perform automated cross browser testing even on your locally hosted web apps ->* Embed all the test results such as video logs, network logs, and screenshots of every step performed through TestMu AI with your Jenkins job results. - -## Prerequisites - -* A TestMu AI account -* An existing Jenkins CI server (version 2.138.2+) -* A Jenkins User with root access. - -## Installing the Jenkins Plugin - -Download TestMu AI Jenkins plugin to run your Selenium tests with Jenkins on TestMu AI. You need to be granted administrator-level access before proceeding with Jenkins installation. - -Make sure to have no active build jobs in execution or queue before heading forward with the TestMu AI Jenkins plugin installation. - -1. Click **Manage Jenkins**, then **Manage Plugins**. -2. Click the **Available** tab. -3. In the **Filter** box, type TestMu AI. -4. You will encounter a list of plugins where you need to select **TestMu AI**. -5. To install TestMu AI Jenkins plugin, you would have to select the checkbox in front of TestMu AI. After successful installation of the plugin, you will be able to find the TestMu AI Jenkins plugin under your installed plugins - -Sometimes your recently installed plugins fail to synchronize with your available plugins on Jenkins. If you don't find TestMu AI plugin under your available plugins, you can force Jenkins to refresh the list by simply clicking the **Check Now** button to fetch the latest plugins from the Jenkins update center. - -## How To Configure Your TestMu AI Credentials? - -After installing the TestMu AI Jenkins plugin to perform automated cross browser testing, you would need to configure the TestMu AI credentials in your Jenkins CI server for integrating your Selenium WebDriver tests with Jenkins. You would need to work with Jenkins UI to proceed with the configuration. - -## To Configure TestMu AI Credentials Using Jenkins UI - -Follow these steps to configure TestMu AI with Jenkins: - -1. On the Home page of Jenkins, click **Credentials**. -2. Under **Credentials** click **System**. System page displays. -3. In the System, click **Global credentials** (Unrestricted) domain. **Global credentials** page opens. -4. Click **Add Credentials**. Add Credentials page opens. -5. Enter the relevant data in the fields and click **Verify Credentials**. After verification, click the **OK** button. Jenkins will generate the ID, which is visible on the **Credential** page. -6. Save your changes. - -Once you successfully add your credentials, then Jenkins will generate an ID. To retrieve this ID for TestMu AI Credentials, you would have to go to Jenkins home page and click on **Credentials** from the left navigation menu. - -From the Jenkins home page, click **Credentials** from the left menu. You can copy the ID for TestMu AI credentials. - -## Configuring The Build Environment Through Jenkins UI - -To enable Lambda Tunnel to perform tests on your locally hosted web application or website: Visit the tab for **Build Environment** and mark the tick on the box left to **LAMBDATEST**. - -That is all you need for performing automated cross browser testing on your locally hosted web pages or web files using the TestMu AI Jenkins plugin with Lambda Tunnel. In the next section, you will learn how to enable the Lambda Tunnel. - -## Configure TestMu AI Tunnel - -Here we will look at the process of enabling Lambda Tunnel to test your locally hosted web apps. Visit the tab for **Build Environment** and mark the tick on the box left to **LAMBDATEST**. - -1. We would recommend keeping the TestMu AI Local Path empty; that way, the plugin will automatically download the LT binary file. However, if you wish to use a binary by downloading externally, you need to provide the file's location as the 'TestMu AI Local Path.' But if your Jenkins works on the master-slave configuration set up, then it would be best to leave the path empty for allowing the plugin to download the relevant binary file for the operating system of your local machine. -2. You can configure additional settings with Lambda Tunnel by using the option to Use Local Tunnel. - -## Environment Variable For TestMu AI Jenkins Plugin -|Environment Variables|Description| -|--- |--- | -|$LT_USERNAME|Your TestMu AI username.| -|$LT_ACCESS_KEY|Your TestMu AI Access key.| -|$LT_GRID_URL|Your TestMu AI hub URL| -|$LT_PLATFORM|OS you wish to test your website with.| -|$LT_BROWSER_NAME|Name of the browser on which you wish to test.| -|$LT_BROWSER_VERSION|Particular browser version on which you wish to test.| -|$LT_RESOLUTION|Specify the screen resolution.| -|$LT_BUILD_NAME|Represents a particular build name.| -|$LT_BUILD_NUMBER|Represents a particular build number.| -|$LT_BROWSERS|An array of combinations.| -|$LT_TUNNEL_NAME|Name of your Lambda Tunnel for local testing.| - -Using the above variables you will receive an output in the below format. - -```javascript -sample Output $LT_BROWSERS: -[ -{ -"operatingSystem": "win10", //OS you wish to test your website with. -"browserName": "Chrome", //Name of the browser on which you wish to test. -"browserVersion": "71.0", //Particular browser version on which you wish to test. -"resolution": "1024x768" //Specify the screen resolution. -}, -{ -"operatingSystem": "win8", //OS you wish to test your website with. -"browserName": "Firefox", //Name of the browser on which you wish to test. -"browserVersion": "63.0", //Particular browser version on which you wish to test. -"resolution": "1024x768" //Specify the screen resolution. -} -``` - -## Running Your First Test - -To run your first test using Jenkins TestMu AI plugin, you need to set a test build. You can perform that easily through Jenkins UI by clicking on **Build Now**. - -As you click on **Build Now**, you will find a new build generating under the build history. - -If you click on the build you will find the logs for console output. You will also notice TestMu AI icon on the left which will provide you in-depth details regarding your Selenium test through an i-frame. - -When you click on the TestMu AI Report icon from the left side, a report table with details for the Selenium test will load. - -Kudos! You successfully executed your Selenium test suite using Jenkins with TestMu AI plugin. - -## Additional Links - -- [Integrate TestMu AI With Jenkins Freestyle Project](/support/docs/integrating-testmu-with-jenkins-freestyle-project/) -- [Integrating TestMu AI Report With Jenkins Pipeline](/support/docs/integrating-testmu-report-with-jenkins-pipeline/) diff --git a/static/docs/jest-on-hyperexecute.md b/static/docs/jest-on-hyperexecute.md deleted file mode 100644 index 77bbdbcc2..000000000 --- a/static/docs/jest-on-hyperexecute.md +++ /dev/null @@ -1,221 +0,0 @@ -# Running Jest Framework Tests on HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Jest is a JavaScript testing framework commonly paired with Puppeteer for testing web applications. Specifically designed for JavaScript, Jest provides a simple and efficient platform for writing unit tests, integration tests, and end-to-end tests using Puppeteer. - -HyperExecute is an AI Native Test Orchestration Cloud Platform that empowers you to run **end-to-end** tests **quickly** and **efficiently**. It provides Just-in-Time (JIT) testing infrastructure with fast execution **speeds**, **smart orchestration**, and **detailed logs**. - -This guide details how to execute your **Jest** framework tests on **HyperExecute** via two different methods: - -- [**Using Local System**](/support/docs/jest-on-hyperexecute/#1-testing-using-local-system) - You can use your own local machine to execute tests. -- [**Using Gitpod Platform**](/support/docs/jest-on-hyperexecute/#2-testing-using-gitpod) - Execute tests using GitPod. (Requires a [Gitpod](https://gitpod.io/login/) account) - -## 1. Testing Using Local System - -Follow the step-by-step guide to execute your test on HyperExecute. - -### Prerequisites - -To run the Tests on HyperExecute from your Local System, you are required: - -- Your TestMu AI [Username and Access key](/support/docs/hyperexecute-how-to-get-my-username-and-access-key/) -- [HyperExecute YAML](/support/docs/hyperexecute-yaml-version0.2/) file which contains all the necessary instructions. -- [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) in order to initiate a test execution Job . -- Setup the [Environmental Variable](/support/docs/hyperexecute-environment-variable-setup/) - -### Step 1: Configure Your Test Suite - -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** - -Download or Clone the code sample for the JUnit from the TestMu AI GitHub repository to run the tests on the HyperExecute. - - View on GitHub - -If you are using your own project, make sure you update the **Hub endpoint** in your tests file. - -By setting up the Hub endpoint, you establish the communication channel between your tests and the browser nodes, enabling effective test distribution and execution. - -Configure the desired capabilities based on your test requirements. For example: - -```bash -const capability = { -"browserName": "Chrome", -"browserVersion": "122.0", -"LT:Options": { -"username": "YOUR_USERNAME", -"accessKey": "YOUR_ACCESS_KEY", -"platformName": "Windows 10", -"project": "Untitled", -"w3c": true, -"plugin": "node_js-jest" -} -} -``` - -> You can also use **dev** and **beta** browser versions. To generate capabilities for your test requirements, you can use our inbuilt 🔗 Capabilities Generator. - -### Step 2: Setup the CLI in your Test Suite - -After cloning / downloading the sample repo, you need to setup the CLI and the environment variables. - -#### Download the HyperExecute CLI - -The CLI is used for triggering the tests on HyperExecute. It is recommend to download the CLI binary on the host system and keep it in the root directory of the suite to perform the tests on HyperExecute. - -You can download the CLI for your desired platform from the below mentioned links: - -| Platform | HyperExecute CLI | -| ---------| ---------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| MacOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -#### Setup Environment Variable - -Now, you need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in the [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). - -Run the below mentioned commands in your terminal to setup the CLI and the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -### Step 3: Configure YAML in your Test Suite - -Configure your YAML file as per your use cases using **key value** pairs. - -In this sample YAML file, we have mentioned: - -- **version** of the YAML file -- **Timeouts** for executing your project -- **Mode of execution** is [Autosplit](/support/docs/hyperexecute-auto-split-strategy/). You can also opt for [Matrix](/support/docs/hyperexecute-matrix-multiplexing-strategy/) or [Hybrid](/support/docs/hyperexecute-hybrid-strategy/) mode. -- **Pre and Post** commands -- **Reports and Artefacts** that will be generated after the completion of tests -- and other necessary YAML Parameters - -```bash -version: 0.1 - -runson: linux - -globalTimeout: 90 -testSuiteTimeout: 90 -testSuiteStep: 90 - -autosplit: true - -retryOnFailure: true -maxRetries: 1 - -concurrency: 2 - -pre: -- npm install - -cacheKey: '{{ checksum "package.json" }}' -cacheDirectories: -- node_modules - -testDiscovery: -type: raw -mode: remote -command: grep -lr 'describe' specs/*.spec.js - -testRunnerCommand: npm test -- $test - -jobLabel: [puppeteer-JestJS, linux, autosplit] -``` - -### Step 4: Execute your Test Suite - -> **NOTE :** In case of MacOS, if you get a permission denied warning while executing CLI, simply run **`chmod u+x ./hyperexecute`** to allow permission. In case you get a security popup, allow it from your **System Preferences** → **Security & Privacy** → **General tab**. - -Run the below command in your terminal at the root folder of the project: - -```bash -./hyperexecute --config RELATIVE_PATH_OF_YOUR_YAML_FILE -``` - -OR use this command if you have not exported your username and access key in the step 2. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE `} - -### Step 5: Monitor the Test Execution - -Visit the [HyperExecute Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) and check your Job status. - -### Step 6: Download Artifacts and Reports - -HyperExecute also facilitates the provision to download the [Artifacts](/support/docs/hyperexecute-artifacts/) and [Reports](/support/docs/hyperexecute-reports/) on your local machine. Click on the corresponding button to download your generated artifacts and reports. - -## 2. Testing Using Gitpod - -You can also use the Gitpod platform to execute our sample repository. It will fetch all the sample codebases and trigger the CLI to execute the tests. - -Follow the below steps to run Test using Gitpod: - -**Step 1:** Click '**Open in Gitpod**' button. You will be redirected to Login/Signup page. This button is configured to redirect you to the Gitpod platform where you will be able to execute our sample repository. - -**Step 2:** Login with TestMu AI credentials. Once logged in, a pop-up confirmation will appear, asking you to **'Proceed'** to the Gitpod editor in a new tab. The current tab will display the HyperExecute Dashboard. - -**Step 3:** Choose your preferred editor (we recommend VS Code Editor) - -**Step 4:** As you are running a sample project, Fetching of the Test Scripts, [HyperExecute YAML](/support/docs/deep-dive-into-hyperexecute-yaml/), [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) and Triggering your tests using the `Execution Command` will be automated. - -**Step 5:** Once you see the `Job Link` in the logs, you can visit the [HyperExecute dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) to see the tests getting executed. - -You can also implement [Secret Keys](/support/docs/hyperexecute-how-to-save-and-manage-secrets/) in your YAML file. - -## Navigation in Automation Dashboard - -Every test run on the HyperExecute has a unique *jobId* associated with it. Each *jobId* can in turn constitute single (or multiple) *groupId*(s). You can visit [HyperExecute Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) for checking the status of the test execution. - -You can seamlessly navigate between JobId's and taskId's. You need to click on the *testID* to navigate from the HyperExecute logs to the Automation Dashboard. - -The snapshot below shows the videos, logs and other meta data for that specific *test_ID* - -> For any query or doubt, please feel free to contact us via **24×7 chat support** or you can also drop a mail to **support@testmuai.com**. -Happy testing! - -## Using the Jest Agent Skill with TestMu AI - -The [jest-skill](https://github.com/LambdaTest/agent-skills/tree/main/jest-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The jest-skill package includes: - -``` -jest-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing Jest Agent Skill - -Install a Jest Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/jest-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/jest-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only jest-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). diff --git a/static/docs/jest-with-playwright.md b/static/docs/jest-with-playwright.md deleted file mode 100644 index c7f613de5..000000000 --- a/static/docs/jest-with-playwright.md +++ /dev/null @@ -1,157 +0,0 @@ -# Jest with Playwright: Running Your First Test - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Learn how to use Playwright with Jest framework to automate web application testing across 50+ real browsers and operating systems on TestMu AI cloud platform. - -## Prerequisites - -1. You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** -Download or clone the code sample for the Playwright Jest from the TestMu AI GitHub repository to run the tests. - - View on GitHub - -```js -git clone https://github.com/LambdaTest/playwright-sample.git -cd playwright-sample -cd playwright-jest-js -``` - -2. Install the npm dependencies. - -``` -npm install -``` - -3. A TestMu AI Username and Access key. You can get it from your TestMu AI Profile section. Don't have an account, sign up for free. - -4. To run Playwright tests, set your TestMu AI Username and Access key in the Environment Variables. - -## Run your Playwright tests with Jest - -Navigate to the `jest-playwright.config.js` file in the `playwright-jest-js` directory and provide your TestMu AI Username and Access key. - -```js -const cp = require('child_process'); -const playwrightClientVersion = cp.execSync('npx playwright --version').toString().trim().split(' ')[1]; - -const caps_chromium = { -'browserName': 'pw-chromium', // Browsers allowed: `Chrome`, `MicrosoftEdge`, `pw-chromium`, `pw-firefox` and `pw-webkit` -'browserVersion': 'latest', -'LT:Options': { -'platform': 'Windows 10', -'build': 'Playwright-Jest Build', -'name': 'Playwright-Jest Test on Chromium', -'user': process.env.LT_USERNAME, -'accessKey': process.env.LT_ACCESS_KEY, -'network': true, -'video': true, -'console': true, -'tunnel': false, // Add tunnel configuration if testing locally hosted webpage -'tunnelName': '', // Optional -'geoLocation': '', // country code can be fetched from https://www.lambdatest.com/capabilities-generator/ -'playwrightClientVersion': playwrightClientVersion -} -}; - -const caps_firefox = { -'browserName': 'pw-firefox', // Browsers allowed: `Chrome`, `MicrosoftEdge`, `pw-chromium`, `pw-firefox` and `pw-webkit` -'browserVersion': 'latest', -'LT:Options': { -'platform': 'Windows 10', -'build': 'Playwright-Jest Build ', -'name': 'Playwright-Jest Test on Firefox', -'user': process.env.LT_USERNAME, -'accessKey': process.env.LT_ACCESS_KEY, -'network': true, -'video': true, -'console': true, -'tunnel': false, // Add tunnel configuration if testing locally hosted webpage -'tunnelName': '', // Optional -'geoLocation': '', // country code can be fetched from https://www.lambdatest.com/capabilities-generator/ -} -}; - -const caps_webkit = { -'browserName': 'pw-webkit', // Browsers allowed: `Chrome`, `MicrosoftEdge`, `pw-chromium`, `pw-firefox` and `pw-webkit` -'browserVersion': 'latest', -'LT:Options': { -'platform': 'Windows 10', -'build': 'Playwright-Jest Build', -'name': 'Playwright-Jest Test on WebKit', -'user': process.env.LT_USERNAME, -'accessKey': process.env.LT_ACCESS_KEY, -'network': true, -'video': true, -'console': true, -'tunnel': false, // Add tunnel configuration if testing locally hosted webpage -'tunnelName': '', // Optional -'geoLocation': '', // country code can be fetched from https://www.lambdatest.com/capabilities-generator/ -} -}; - -module.exports = { -connectOptions: { -chromium: { -wsEndpoint: `wss://cdp.lambdatest.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(caps_chromium))}` -}, -firefox: { -wsEndpoint: `wss://cdp.lambdatest.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(caps_firefox))}` -}, -webkit: { -wsEndpoint: `wss://cdp.lambdatest.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(caps_webkit))}` -} -}, -browsers: ['chromium', 'firefox', 'webkit'], -} -``` - -Pass the below command in the terminal to run the test. - -```js -npm run test -``` - -## View your test results - -Go to the [TestMu AI Web Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) to see your Playwright JavaScript test results. - -## Using the Jest Agent Skill with TestMu AI - -The [jest-skill](https://github.com/LambdaTest/agent-skills/tree/main/jest-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The jest-skill package includes: - -``` -jest-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing Jest Agent Skill - -Install a Jest Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/jest-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/jest-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only jest-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). diff --git a/static/docs/jira-integration.md b/static/docs/jira-integration.md deleted file mode 100644 index 83a33b52f..000000000 --- a/static/docs/jira-integration.md +++ /dev/null @@ -1,61 +0,0 @@ -# Jira Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The TestMu AI Jira integration helps you create issues in Jira directly from the TestMu AI platform itself. With simple one-click integration, you can push annotated issues to the project of your choice, assign them to the required teammate, and attach screenshots. You can do all that while in the middle of a test session in the TestMu AI platform. The fields populated by you when marking as a bug through TestMu AI are displayed as information on the Jira ticket for a testing instance. - -**Jira Integration with TestMu AI, like all of the integrations to 3rd party applications, is available for freemium as well as premium plan.** - -## How to Establish Integration with Jira from Your TestMu AI Account? - -**Step 1:** Login to your TestMu AI account. You would need to have Admin or User level access to see and install integrations. Don't have an account, register for free. - -**Step 2:** Select 'Integration' from the left navigation menu bar. This will guide you to a screen where you will find a list of 3rd party applications, available to integrate with your TestMu AI account. - -**Step 3:** Click on 'ADD' under the block that says 'Jira'. - -**Step 4:** Once you click on **Connect**, you will be redirected to the below screen. Where You have to select Instance type. If you have a **Self Hosted Jira** instance, you can prefer this document [Self-hosted](/support/docs/jira-self-hosted-integration/). - -**Step 5:** Click on the Install button to be redirected to the Jira dashboard. - -**Step 6:** Click on Accept. - -**Step 7:** Once you click on Accept, you will be redirected to the screen below, where you have to select your site and click on the Select Site and Proceed button. - -**Step 8:** When you click on the **Select and Proceed button**, you will be redirected to the below screen, where you have to select projects that you want to integrate and click on the **Integrate Projects and Install button.** - -That's it! Go to Integrations again, and you will be able to notice a **green tick** indicating that Jira is successfully **installed**. You are all set to experience **one-click bug logging** to share your issues directly from your TestMu AI account with our teammates on the Jira project. - -## How To Log Your First Bug Through Jira Integration? - -**Step 1:** Go for any of the tests from the left navigation menu. For the demo, we will be taking the **"Real Time Testing"** option. - -**Step 2:** Present the URL of the web app you need to test in the dialog box. After that, select any configuration for the browser and operating system of your choice & hit '**Start**'. - -**Step 3:** After the VM is launched and operable. You can perform testing on your web-app for finding bugs. If a bug gets revealed, then you need to click on the Bug icon from the left panel for capturing a screenshot of the same. We have highlighted that option with yellow in the below image. - -**Step 4:** After a screenshot is captured, you can annotate any issue or a task with an in-built image editor. Once you are done highlighting the bug, click on the **bug icon** which is beside Jira. - -**Step 5:** After clicking on "**Mark as Bug**" button a Jira specific form would open up. Fill the fields as per your requirement. - -- You can select which project should the ticket go under, along with the **Issue type**. -- You can set the **priority** of the bug. -- You can assign the bug to a colleague by populating the field '**Assignee**'. -- You also get to post a **description** to help relate the cause of the issue or the task in Summary section. - -> Note: Only projects with work type as BUG enabled will be seen in the project dropdown. If you don't see a project in the dropdown, please add **work type as BUG** for your Jira project and resync the integration from the integrations page. - -**Step 6:** Click on "**Mark As Bug**" & observe it being successfully marked through a single click effort. Once you hit the button, you will get **prompt messages** on top of your Virtual Machine indicating the progress of bug logging. You will get a prompt message in few seconds mentioning "**Your bug has been logged successfully**" indicating that the issue has been successfully pushed to the respective workspace. - -**Step 7:** Log in to your Jira dashboard. You will be able to notice the logged issue right away! All the data you provided through TestMu AI would already be presented in it. TestMu AI automatically includes test environment details and related screenshots in the ticket as attachments. - -## How To Remove Jira Integration? - -**Step 1:** Login to your TestMu AI account. - -**Step 2:** Select '**Integration**' from the left navigation menu bar. This will guide you to a screen where you will -find 3rd party applications, available to integrate with your TestMu AI account. - -**Step 3:** Click on '**Remove**'. You can find the **Remove** button when you click on ellipsis in **‘Jira’** block. - -That was all you need to know for TestMu AI + Jira Integration. Increase your productivity with our integrations. If you still have any questions for us, please feel free to let us know. Our experts are always available on **chat** to help you out with any roadblock regarding our product. Happy testing! diff --git a/static/docs/jira-self-hosted-integration.md b/static/docs/jira-self-hosted-integration.md deleted file mode 100644 index 07670ed27..000000000 --- a/static/docs/jira-self-hosted-integration.md +++ /dev/null @@ -1,56 +0,0 @@ -# Self-Hosted Jira Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Many organizations consistently choose to self host their application as it gives them complete control of almost everything. If you want to place Jira behind your corporate firewall and restrict access to someone who isn't on your private network, you can do the same. - -TestMu AI-Jira integration allows you to integrate your self-hosted Jira instance and create the issues directly from the TestMu AI platform. You can send annotated issues to the project of your choice, assign them to a colleague, and add a screenshot straight from the TestMu AI platform. - -## How To Integrate Self-Hosted Jira With TestMu AI - -You can integrate your self-hosted Jira with the TestMu AI platform. To view and install integrations, you would need to have admin or user-level access. - -**Step 1**: Login to your TestMu AI account and select **'Integrations'** from the left menu bar. You will be taken to the integration page where you'll find third-party apps available to integrate. Click on **'Jira'** icon under the **'Bug Tracker'** category. - -**Step 2**: You will be guided to the below screen. Click on **'Self Hosted'**. - -**Step 3**: Enter your Jira URL, Jira username and password in the provided fields then Click on **'Install'**. - -**Step 4**: Visit the Integration page, you'll find the Jira integration under the category **'My Integrations'**. - -## How To Mark Your Bug Through Self-Hosted Jira Integration? - -Once you have established TestMu AI-Jira integration, you can directly mark and push your bugs from TestMu AI to your Jira dashboard. - -**Step 1**: Select the test that you want to perform from the left menu bar. For demonstration, we will perform Real-time testing. - -**Step 2**: Enter the test URL of the website/web app in the URL field. Select the browser and OS combinations that you wish to perform the test on and then click on **'Start'**. - -**Step 3**: A TestMu AI virtual machine will fire up where you can perform real-time testing of your websites and web-apps to find bugs/issues. If you encounter a bug, you can click on the Spider icon from the left panel for [capturing a screenshot](https://www.testmuai.com/full-page-screen-capture) of the same. - -**Step 4**: Once the screenshot is captured, you can annotate any bugs or a task with an in-built image editor. After highlighting the bug, click on **'Mark as Bug'**. - -**Step 5**: After clicking on Mark as Bug, a Jira-specific ticket will open up. Fill in the fields as per your requirement and click on **'Create Issue'**. - -* Select the **'Project'** to which your ticket will go under. -* Assign the bug to a teammate by populating the field **'Assignee'**. -* Select the **'Issue Type'**. -* Fill the **'Summary'** about the bug's details. - -Your bugs will get marked successfully to your Jira dashboard. You'll also receive the prompt message confirming the same. - -**Step 6**: Visit the Jira dashboard and you'll find your marked bugs right away. All the data such as test environment details, screenshots that you provided through TestMu AI will be present there. - -## How To Remove Self-Hosted Jira Integration? - -> You can integrate one bug-tracking tool at a time. If you want to integrate to a similar third party application, then need to remove your current integration. Here is how to do that. - -**Step 1:** Login to your TestMu AI account. - -**Step 2:** Navigate to **'Integrations'** tab from the left menu bar. - -**Step 3:** Under the category **'My Integrations'**. You can find the **'Remove'** button adjacent to Jira icon. - - You will receive the confirmation message upon the successful uninstallation of Jira Self Hosted integration. - -> That's all you need to know for TestMu AI + Jira Self Hosted Integration. If you have any questions for us, please feel free to let us know. Our experts are always **available on chat** to help you out. Happy testing! diff --git a/static/docs/jumpcloud-scim.md b/static/docs/jumpcloud-scim.md deleted file mode 100644 index 0bfcddc7d..000000000 --- a/static/docs/jumpcloud-scim.md +++ /dev/null @@ -1,90 +0,0 @@ -# JumpCloud User Provisioning - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -JumpCloud is a cloud-based directory-as-a-service (DaaS) platform that provides comprehensive identity and access management (IAM) capabilities for organizations. It offers a centralized platform to manage and secure user identities, devices, and applications across a variety of environments, including cloud, on-premises, and hybrid infrastructures. - -In the documentation, we will discuss how to integrate JumpCloud with TestMu AI: - -## Prerequisites​ -- You will need an Enterprise plan with TestMu AI. -- The SSO connection must be already established with TestMu AI. Learn how to [create a SSO Connection with TestMu AI](/support/docs/testmu-sso-self-serve/). - -## Steps to Integrate SCIM with JumpCloud - -- **Step 1:** Sign in to your TestMu AI account. Don't have an account, register for free. - -- **Step 2:** Head to **Settings** > **Organization Settings** from the dropdown. - -- **Step 3:** Go to TestMu AI [Authentication & SSO settings](https://accounts.lambdatest.com/auth/sso) and click on Setup SSO. Head to the **Authentication and SSO** tab and click and copy the **SCIM Base URL and Bearer Token** option. - -- **Step 4:** Log in to JumpCloud. Go to [JumpCloud Admin Console](https://console.jumpcloud.com/login/admin). Log in with your admin credentials. Sign in to the Jump-cloud. Select SSO Applications, then select Get Started. - -- **Step 5:** Search for SCIM and then click on "**Create Custom Integration**" and then click next. - -- **Step 6:** Check the box for "**Manage Single Sign-On (SSO)"** and then click "**Start**." - -- **Step 7:** Enter the name for **Display Label**, then click the **Save Application** and click **configure Application**. - -- **Step 8:** Go to TestMu AI [**Authentication & SSO**](https://accounts.lambdatest.com/auth/sso) settings and click on **Setup SSO**. - -- Enter the **Connection Name** and Click on Add New Connection. and Select your **Identity Provider**. - -- **Step 9:** Copy the required SSO details from TestMu AI, including: - - IdP Entity ID - - SP Entity ID - - ACS URL - - Subject Name - - Login URL - -- **Step 10:** Configure TestMu AI SSO Integration Using SAML. -- Go to TestMu AI Dashboard > **Authentication & SSO Settings**. -- Click **Setup SSO**. Enter a **Connection Name** and click **Add New Connection**. -- Select **JumpCloud** as your Identity Provider (IdP) -- Copy the required SSO details from TestMu AI, including: - - IdP Entity ID - - SP Entity ID - - ACS URL - - Subject Name - - Login URL -- Click **Add Attribute** in jumpcloud, enter the necessary details, and save the configuration. (IdP) -- Copy the **Metadata URL** from JumpCloud. - -- **Step 11:** Finalize the Integration -- Copy the **Metadata URL** from JumpCloud. -- Go to TestMu AI Dashboard and paste the Metadata URL. -- Add the **SSO domain** in TestMu AI and click **Create Connection**. -- Assign users to the SSO connection in JumpCloud and **Save** the configuration. - -- **Step 12:** Go to TestMu AI Dashboard and go **SSO connection**. Copy **Bearer Token** and **SCIM Base URL**. - -- **Step 13:** Go **Identity Management** and paste **SCIM Base URL** and **Bearer Token**. and then click Save button. - -## Provisioning Groups from JumpCloud - -Once SCIM user provisioning is working, you can also push JumpCloud user groups to TestMu AI. - -**Prerequisites** -Group Provisioning must be enabled for your org. Contact **24/7 chat support** to activate it. - -**Step 1:** In JumpCloud Admin Console, go to your TestMu AI SSO Application > **User Groups** tab. - -**Step 2:** Select the user groups you want to provision to TestMu AI. - -**Step 3:** Click **Activate** to start pushing group membership via SCIM. - -**Step 4:** In TestMu AI, go to **Settings** > **Organization Settings** > **SCIM Group Provisioning** to view the synced groups and configure mappings. - -### What Happens After Provisioning - -| JumpCloud Action | TestMu AI Effect | -|---|---| -| Group activated for provisioning | Group created, mapping rules evaluated, members synced | -| User added to group | Member added to all mapped TestMu AI entities | -| User removed from group | Member removed (if no other group maps them there), role recomputed | -| Group renamed | Group renamed, mapped entity renamed to match, rules re-evaluated | -| Group deactivated/deleted | Group soft-deleted, members safely unassigned, roles recomputed | - -> For details on mapping, conflicts, and rules, see the [SCIM Provisioning guide](/support/docs/scim/#group-provisioning). - -- **Step 14:** Enter email that does not exist on TestMu AI platform and provide test email details, and click **Test Connection** and activate button. diff --git a/static/docs/junit-on-hyperexecute-grid.md b/static/docs/junit-on-hyperexecute-grid.md deleted file mode 100644 index d4e19dd79..000000000 --- a/static/docs/junit-on-hyperexecute-grid.md +++ /dev/null @@ -1,247 +0,0 @@ -# Running JUnit Framework Tests on HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -JUnit is a widely-used testing framework for Java that is extensively employed in Selenium test automation. It provides annotations and assertions to structure and validate test cases, making it an integral part of the Selenium testing ecosystem. - -HyperExecute is an AI Native Test Orchestration Cloud Platform that empowers you to run **end-to-end** tests **quickly** and **efficiently**. It provides Just-in-Time (JIT) testing infrastructure with fast execution **speeds**, **smart orchestration**, and **detailed logs**. - -This guide details how to execute your **JUnit** framework tests on **HyperExecute** via two different methods: - -- [**Using Local System**](/support/docs/junit-on-hyperexecute-grid/#1-testing-using-local-system) - You can use your own local machine to execute tests. -- [**Using Gitpod Platform**](/support/docs/junit-on-hyperexecute-grid/#2-testing-using-gitpod) - Execute tests using GitPod. (Requires a [Gitpod](https://gitpod.io/login/) account) - -## 1. Testing Using Local System - -Follow the step-by-step guide to execute your test on HyperExecute. - -### Prerequisites - -To run the Tests on HyperExecute from your Local System, you are required: - -- Your TestMu AI [Username and Access key](/support/docs/hyperexecute-how-to-get-my-username-and-access-key/) -- [HyperExecute YAML](/support/docs/hyperexecute-yaml-version0.2/) file which contains all the necessary instructions. -- [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) in order to initiate a test execution Job . -- Setup the [Environmental Variable](/support/docs/hyperexecute-environment-variable-setup/) - -### Step 1: Configure Your Test Suite - -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** - -Download or Clone the code sample for the JUnit from the TestMu AI GitHub repository to run the tests on the HyperExecute. - - View on GitHub - -If you are using your own project, make sure you update the **Hub endpoint** in your tests file. - -By setting up the Hub endpoint, you establish the communication channel between your tests and the browser nodes, enabling effective test distribution and execution. - -Configure the desired capabilities based on your test requirements. For example: - -```bash -DesiredCapabilities capabilities = new DesiredCapabilities(); -capabilities.setCapability("browserName", "chrome"); -capabilities.setCapability("version", "70.0"); -capabilities.setCapability("platform", "win10"); # If this cap isn't specified, it will just get the any available one -capabilities.setCapability("build", "LambdaTestSampleApp"); -capabilities.setCapability("name", "LambdaTestJavaSample"); -capabilities.setCapability("network", true); # To enable network logs -capabilities.setCapability("visual", true); # To enable step by step screenshot -capabilities.setCapability("video", true); # To enable video recording -capabilities.setCapability("console", true); # To capture console logs -``` - -> You can also use **dev** and **beta** browser versions. To generate capabilities for your test requirements, you can use our inbuilt 🔗 Capabilities Generator. - -### Step 2: Setup the CLI in your Test Suite - -After cloning / downloading the sample repo, you need to setup the CLI and the environment variables. - -#### Download the HyperExecute CLI - -The CLI is used for triggering the tests on HyperExecute. It is recommend to download the CLI binary on the host system and keep it in the root directory of the suite to perform the tests on HyperExecute. - -You can download the CLI for your desired platform from the below mentioned links: - -| Platform | HyperExecute CLI | -| ---------| ---------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| MacOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -#### Setup Environment Variable - -Now, you need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in the [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). - -Run the below mentioned commands in your terminal to setup the CLI and the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -### Step 3: Configure YAML in your Test Suite - -Configure your YAML file as per your use cases using **key value** pairs. - -In this sample YAML file, we have mentioned: - -- **version** of the YAML file -- **Timeouts** for executing your project -- **Mode of execution** is [Autosplit](/support/docs/hyperexecute-auto-split-strategy/). You can also opt for [Matrix](/support/docs/hyperexecute-matrix-multiplexing-strategy/) or [Hybrid](/support/docs/hyperexecute-hybrid-strategy/) mode. -- **Pre and Post** commands -- **Reports and Artefacts** that will be generated after the completion of tests -- and other necessary YAML Parameters - -```yaml ---- -version: 0.1 -globalTimeout: 150 -testSuiteTimeout: 150 -testSuiteStep: 150 - -runson: linux - -autosplit: true -retryOnFailure: true - -maxRetries: 1 -concurrency: 3 - -env: -# PAT: ${{ .secrets.testKey }} -CACHE_DIR: m2_cache_dir -TARGET_OS: linux - -cacheKey: '{{ checksum "pom.xml" }}' -cacheDirectories: -- ${CACHE_DIR} - -shell: bash - -pre: -# Download and install packages in the CACHE_DIR. -# Skip execution of the tests in the pre step -- mvn -Dmaven.repo.local=${CACHE_DIR} -Dmaven.test.skip=true clean install - -post: -- cat yaml/linux/junit_hyperexecute_autosplit_sample.yaml - -mergeArtifacts: true - -uploadArtefacts: -- name: Final-Report -path: -- target/site/** -- name: Surefire-Report -path: -- target/surefire-reports/** - -report: true -partialReports: -location: target/surefire-reports/html -type: html -frameworkName: extent - -testDiscovery: -type: raw -mode: remote -command: grep 'public class' src/test/java/hyperexecute/*.java | awk '{print$3}' - -testRunnerCommand: mvn -Dplatname=linux -Dmaven.repo.local=m2_cache_dir -Dtest=$test test site surefire-report:report - -jobLabel: [selenium-junit, linux, autosplit] -``` - -### Step 4: Execute your Test Suite - -> **NOTE :** In case of MacOS, if you get a permission denied warning while executing CLI, simply run **`chmod u+x ./hyperexecute`** to allow permission. In case you get a security popup, allow it from your **System Preferences** → **Security & Privacy** → **General tab**. - -Run the below command in your terminal at the root folder of the project: - -```bash -./hyperexecute --config RELATIVE_PATH_OF_YOUR_YAML_FILE -``` - -OR use this command if you have not exported your username and access key in the step 2. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE `} - -### Step 5: Monitor the Test Execution - -Visit the [HyperExecute Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) and check your Job status. - -### Step 6: Download Artifacts and Reports - -HyperExecute also facilitates the provision to download the [Artifacts](/support/docs/hyperexecute-artifacts/) and [Reports](/support/docs/hyperexecute-reports/) on your local machine. Click on the corresponding button to download your generated artifacts and reports. - -## 2. Testing Using Gitpod - -You can also use the Gitpod platform to execute our sample repository. It will fetch all the sample codebases and trigger the CLI to execute the tests. - -Follow the below steps to run Test using Gitpod: - -**Step 1:** Click '**Open in Gitpod**' button. You will be redirected to Login/Signup page. This button is configured to redirect you to the Gitpod platform where you will be able to execute our sample repository. - -**Step 2:** Login with TestMu AI credentials. Once logged in, a pop-up confirmation will appear, asking you to **'Proceed'** to the Gitpod editor in a new tab. The current tab will display the HyperExecute Dashboard. - -**Step 3:** Choose your preferred editor (we recommend VS Code Editor) - -**Step 4:** As you are running a sample project, Fetching of the Test Scripts, [HyperExecute YAML](/support/docs/deep-dive-into-hyperexecute-yaml/), [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) and Triggering your tests using the `Execution Command` will be automated. - -**Step 5:** Once you see the `Job Link` in the logs, you can visit the [HyperExecute dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) to see the tests getting executed. - -You can also implement [Secret Keys](/support/docs/hyperexecute-how-to-save-and-manage-secrets/) in your YAML file. - -## Navigation in Automation Dashboard - -Every test run on the HyperExecute has a unique *jobId* associated with it. Each *jobId* can in turn constitute single (or multiple) *groupId*(s). You can visit [HyperExecute Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) for checking the status of the test execution. - -You can seamlessly navigate between JobId's and taskId's. You need to click on the *testID* to navigate from the HyperExecute logs to the Automation Dashboard. - -The snapshot below shows the videos, logs and other meta data for that specific *test_ID* - -> -For any query or doubt, please feel free to contact us via **24×7 chat support** or you can also drop a mail to **support@testmuai.com**. -Happy testing! - -## Using the JUnit 5 Agent Skill with TestMu AI - -The [junit-5-skill](https://github.com/LambdaTest/agent-skills/tree/main/junit-5-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The junit-5-skill package includes: - -``` -junit-5-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing JUnit 5 Agent Skill - -Install a JUnit 5 Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/junit-5-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/junit-5-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only junit-5-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). diff --git a/static/docs/junit-with-playwright.md b/static/docs/junit-with-playwright.md deleted file mode 100644 index 74a839afa..000000000 --- a/static/docs/junit-with-playwright.md +++ /dev/null @@ -1,119 +0,0 @@ -# JUnit with Playwright: Running Your First Test - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Learn how to use Playwright with JUnit to automate web application testing across 50+ real browsers and operating systems on TestMu AI cloud platform. - -## Prerequisites - -1. You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** -Download or clone the code sample for the Playwright JUnit from the TestMu AI GitHub repository to run the tests. - - View on GitHub - -```js -git clone https://github.com/LambdaTest/playwright-sample.git -cd playwright-sample -cd playwright-java-junit -``` - -2. Install the npm dependencies. - -``` -npm install -``` - -3. A TestMu AI Username and Access key. You can get it from your TestMu AI Profile section. Don't have an account, sign up for free. - -## Run your Playwright tests with JUnit - -Navigate to the `LTCapability.JUnit` file and provide your TestMu AI Username and Access key. - -```java -import com.google.gson.JsonObject; -import com.tngtech.JUnit.junit.dataprovider.DataProvider; - -public class LTCapability { -@DataProvider -public static Object[] getDefaultTestCapability() { -JsonObject capabilities1 = new JsonObject(); -JsonObject ltOptions1 = new JsonObject(); - -String user = System.getenv("LT_USERNAME"); -String accessKey = System.getenv("LT_ACCESS_KEY"); - -capabilities1.addProperty("browserName", "Chrome"); // Browsers allowed: `Chrome`, `MicrosoftEdge`, `pw-chromium`, `pw-firefox` and `pw-webkit` -capabilities1.addProperty("browserVersion", "latest"); -ltOptions1.addProperty("platform", "Windows 10"); -ltOptions1.addProperty("name", "Playwright Test"); -ltOptions1.addProperty("build", "Playwright Testing using Junit"); -ltOptions1.addProperty("user", user); -ltOptions1.addProperty("accessKey", accessKey); -capabilities1.add("LT:Options", ltOptions1); - -JsonObject capabilities2 = new JsonObject(); -JsonObject ltOptions2 = new JsonObject(); -capabilities2.addProperty("browserName", "MicrosoftEdge"); // Browsers allowed: `Chrome`, `MicrosoftEdge`, `pw-chromium`, `pw-firefox` and `pw-webkit` -capabilities2.addProperty("browserVersion", "latest"); -ltOptions2.addProperty("platform", "Windows 10"); -ltOptions2.addProperty("name", "Playwright Test"); -ltOptions2.addProperty("build", "Playwright Testing using Junit"); -ltOptions2.addProperty("user", user); -ltOptions2.addProperty("accessKey", accessKey); -capabilities2.add("LT:Options", ltOptions2); -return new Object[]{ -capabilities1,capabilities2 -}; -} -} -``` - -Pass the below command in the terminal to run the test. - -```java -mvn clean test -``` - -## View your test results - -Go to the [TestMu AI Web Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) to see your Playwright JUnit test results. - -## Using the JUnit 5 Agent Skill with TestMu AI - -The [junit-5-skill](https://github.com/LambdaTest/agent-skills/tree/main/junit-5-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The junit-5-skill package includes: - -``` -junit-5-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing JUnit 5 Agent Skill - -Install a JUnit 5 Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/junit-5-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/junit-5-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only junit-5-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). diff --git a/static/docs/junit-with-selenium-running-junit-automation-scripts-on-testmu-selenium-grid.md b/static/docs/junit-with-selenium-running-junit-automation-scripts-on-testmu-selenium-grid.md deleted file mode 100644 index 163f8f975..000000000 --- a/static/docs/junit-with-selenium-running-junit-automation-scripts-on-testmu-selenium-grid.md +++ /dev/null @@ -1,320 +0,0 @@ -# Selenium With JUnit - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Run JUnit tests on the TestMu AI cloud grid. This guide covers setup, running a sample test, configuring capabilities, and testing locally hosted pages. - -**Sample repo** -All the code used in this guide is available in the sample repository. - - View on GitHub - -## Prerequisites - -Make sure you have the following set up before you start. - -1. [Create a TestMu AI account](https://www.testmuai.com/register/) if you don't have one. -2. Get your **Username** and **Access Key** from the [TestMu AI Dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard). -3. Install the **Java development environment** (Java 11 recommended). -4. Install **Maven**. Download it from [the official website](https://maven.apache.org/) or install it on **Linux/MacOS** using [**Homebrew**](https://brew.sh/). - -## Step 1: Clone the Sample Project - -Pull the sample repo to your local machine and navigate into the project directory. - -```bash -git clone https://github.com/LambdaTest/junit-selenium-sample -cd junit-selenium-sample -``` - -You may also want to run the command below to check for outdated dependencies. - -```bash -mvn versions:display-dependency-updates -``` - -## Step 2: Set Your Credentials - -Add your TestMu AI credentials as environment variables so the test can authenticate with the grid. - -Visit the [TestMu AI Dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard), navigate to the left sidebar, and select **Credentials**. Copy your **Username** and **Access Key**, then set them as environment variables: - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -## Step 3: Configure Your Test Capabilities - -Define the browser, version, and OS for your test run. - -```java -ChromeOptions browserOptions = new ChromeOptions(); -browserOptions.setPlatformName("Windows 10"); -browserOptions.setBrowserVersion("latest"); - -HashMap ltOptions = new HashMap(); -ltOptions.put("build", "LambdaTestSampleApp"); -ltOptions.put("name", "LambdaTestJavaSample"); -ltOptions.put("network", true); // To enable network logs -ltOptions.put("visual", true); // To enable step by step screenshot -ltOptions.put("video", true); // To enable video recording -ltOptions.put("console", true); // To capture console logs -ltOptions.put("w3c", true); -browserOptions.setCapability("LT:Options", ltOptions); -``` - -Use the [Capabilities Generator](https://www.testmuai.com/capabilities-generator/) to auto-generate capabilities for any browser, version, and OS combination. - -## Step 4: Run the Test - -Trigger the test from your terminal. - -```bash -mvn test -P single -``` - -```bash -mvn test -P parallel -``` - -Below is the sample JUnit test that marks items as done, adds a new item to the list, and validates the result: - -```java title="JUnit To Do" -import org.junit.After; -import org.junit.Before; -import org.junit.Test; -import org.junit.runner.RunWith; -import org.openqa.selenium.JavascriptExecutor; -import org.openqa.selenium.WebDriver; -import org.openqa.selenium.chrome.ChromeOptions; -import org.openqa.selenium.remote.RemoteWebDriver; -import java.net.URL; -import java.util.HashMap; -public class JUnitTodo { -public String username = "YOUR_USERNAME"; -public String accesskey = "YOUR_ACCESS_KEY"; -public static RemoteWebDriver driver = null; -public String gridURL = "@hub.lambdatest.com/wd/hub"; -boolean status = false; -@Before -public void setUp() throws Exception { -ChromeOptions browserOptions = new ChromeOptions(); -browserOptions.setPlatformName("Windows 10"); -browserOptions.setBrowserVersion("latest"); - -HashMap ltOptions = new HashMap(); -ltOptions.put("build", "LambdaTestSampleApp"); -ltOptions.put("name", "LambdaTestJavaSample"); -ltOptions.put("network", true); // To enable network logs -ltOptions.put("visual", true); // To enable step by step screenshot -ltOptions.put("video", true); // To enable video recording -ltOptions.put("console", true); // To capture console logs -ltOptions.put("w3c", true); -browserOptions.setCapability("LT:Options", ltOptions); -try { -driver = new RemoteWebDriver(new URL("https://" + username + ":" + accesskey + gridURL), browserOptions); -} catch (MalformedURLException e) { -System.out.println("Invalid grid URL"); -} catch (Exception e) { -System.out.println(e.getMessage()); -} -} - -@Test -public void testSimple() throws Exception { -try { -//Change it to production page -driver.get("https://lambdatest.github.io/sample-todo-app/"); - -//Let's mark done first two items in the list. -driver.findElement(By.name("li1")).click(); -driver.findElement(By.name("li2")).click(); - -// Let's add an item in the list. -driver.findElement(By.id("sampletodotext")).sendKeys("Yey, Let's add it to list"); -driver.findElement(By.id("addbutton")).click(); - -// Let's check that the item we added is added in the list. -String enteredText = driver.findElementByXPath("/html/body/div/div/div/ul/li[6]/span").getText(); -if (enteredText.equals("Yey, Let's add it to list")) { -status = true; -} -} catch (Exception e) { -System.out.println(e.getMessage()); -} -} -@After -public void tearDown() throws Exception { -if (driver != null) { -((JavascriptExecutor) driver).executeScript("lambda-status=" + status); -driver.quit(); -} -} -} -``` - -For parallel execution, the sample project uses a `Parallelized` helper class and parameterized tests: - -```java title="Parallelized Class" -import java.util.concurrent.ExecutorService; -import java.util.concurrent.Executors; -import java.util.concurrent.TimeUnit; - -import org.junit.runners.Parameterized; -import org.junit.runners.model.RunnerScheduler; - -public class Parallelized extends Parameterized { - -private static class ThreadPoolScheduler implements RunnerScheduler { -private ExecutorService executor; - -public ThreadPoolScheduler() { -String threads = System.getProperty("junit.parallel.threads", "15"); -int numThreads = Integer.parseInt(threads); -executor = Executors.newFixedThreadPool(numThreads); -} - -@Override -public void finished() { -executor.shutdown(); -try { -executor.awaitTermination(10, TimeUnit.MINUTES); -} catch (InterruptedException exc) { -throw new RuntimeException(exc); -} -} - -@Override -public void schedule(Runnable childStatement) { -executor.submit(childStatement); -} -} - -public Parallelized(Class klass) throws Throwable { -super(klass); -setScheduler(new ThreadPoolScheduler()); -} -} -``` - -```java title="Concurrent Todo App" -import org.openqa.selenium.By; -import org.openqa.selenium.Platform; -import org.openqa.selenium.WebDriver; -import org.openqa.selenium.chrome.ChromeOptions; -import org.openqa.selenium.remote.RemoteWebDriver; -import org.junit.After; -import org.junit.Before; -import org.junit.Test; -import org.junit.runner.RunWith; -import org.junit.runners.Parameterized; -import java.net.MalformedURLException; -import java.net.URL; -import java.util.HashMap; -import java.util.LinkedList; -@RunWith(Parallelized.class) -public class JUnitConcurrentTodo { -public String username = "YOUR_LT_USERNAME"; -public String accesskey = "YOUR_LT_ACCESS_KEY"; -public String gridURL = "@hub.lambdatest.com/wd/hub"; -public String platform; -public String browserName; -public String browserVersion; -public RemoteWebDriver driver = null; -boolean status = false; -@Parameterized.Parameters -public static LinkedList getEnvironments() throws Exception { -LinkedList env = new LinkedList(); -env.add(new String[]{"Windows 10", "chrome", "latest"}); -env.add(new String[]{"macOS Monterey","firefox","latest"}); -env.add(new String[]{"Windows 10","internet explorer","latest"}); -return env; -} -public JUnitConcurrentTodo(String platform, String browserName, String browserVersion) { -this.platform = platform; -this.browserName = browserName; -this.browserVersion = browserVersion; -} -@Before -public void setUp() throws Exception { -ChromeOptions browserOptions = new ChromeOptions(); -browserOptions.setPlatformName(platform); -browserOptions.setBrowserVersion(browserVersion); - -HashMap ltOptions = new HashMap(); -ltOptions.put("build", "JUnitParallelSample"); -ltOptions.put("name", "JUnitParallelSampleTest"); -ltOptions.put("w3c", true); -browserOptions.setCapability("LT:Options", ltOptions); -try { -driver = new RemoteWebDriver(new URL("https://" + username + ":" + accesskey + gridURL), browserOptions); -} catch (MalformedURLException e) { -System.out.println("Invalid grid URL"); -} catch (Exception e) { -System.out.println(e.getMessage()); -} -} -@Test -public void testParallel() throws Exception { -try { -//Change it to production page -driver.get("https://lambdatest.github.io/sample-todo-app/"); - -//Let's mark done first two items in the list. -driver.findElement(By.name("li1")).click(); -driver.findElement(By.name("li2")).click(); - -// Let's add an item in the list. -driver.findElement(By.id("sampletodotext")).sendKeys("Yey, Let's add it to list"); -driver.findElement(By.id("addbutton")).click(); - -// Let's check that the item we added is added in the list. -String enteredText = driver.findElementByXPath("/html/body/div/div/div/ul/li[6]/span").getText(); -if (enteredText.equals("Yey, Let's add it to list")) { -status = true; -} -} catch (Exception e) { -System.out.println(e.getMessage()); -} -} -@After -public void tearDown() throws Exception { -if (driver != null) { -((JavascriptExecutor) driver).executeScript("lambda-status=" + status); -driver.quit(); -} -} -} -``` - -## Step 5: View Your Results - -Check the Automation Dashboard to see exactly what happened during your test. - -Visit the [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) to see your test results. Each session includes: - -- **Video recording** of the full test execution -- **Screenshots** captured at each step -- **Console logs** from the browser -- **Network logs** for every request and response -- **Selenium command logs** showing each driver action - -## Run JUnit Tests Using Agent Skills - -Use AI coding assistants to generate and run JUnit tests with the TestMu AI Agent Skill. - -The [junit-5-skill](https://github.com/LambdaTest/agent-skills/tree/main/junit-5-skill) is part of [TestMu AI Agent Skills](https://github.com/LambdaTest/agent-skills/) - structured packages that teach AI coding assistants how to write production-grade test automation. - -Install the skill: - -```bash -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/junit-5-skill .claude/skills/ - -# For Cursor / Copilot -cp -r agent-skills/junit-5-skill .cursor/skills/ -``` - -Install all available framework skills at once by cloning the repository directly into your tool's skills directory (e.g., `.claude/skills/`, `.cursor/skills/`). diff --git a/static/docs/junit-xml-report.md b/static/docs/junit-xml-report.md deleted file mode 100644 index e0331908d..000000000 --- a/static/docs/junit-xml-report.md +++ /dev/null @@ -1,25 +0,0 @@ -# JUnit XML Report - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -A JUnit XML report is an XML-based test result file format used to represent the results of test executions, particularly unit tests. The XML report provides detailed information about the tests run, including which tests passed, failed, were skipped, and any associated error messages. - -## Steps to Generate Consolidated JUnit XML Report -Follow these steps to generate a consolidated JUnit XML report: - -### Step 1: Update Your Project Configuration -Configure all the necessary dependencies and configurations to your project. - -### Step 2: Configure the HyperExecute YAML File -In your HyperExecute YAML configuration, define the [report](/support/docs/deep-dive-into-hyperexecute-yaml/#report) parameters like this: - -```yaml -report: true -partialReports: -frameworkName: junit -location: reports/ -type: xml -``` - -### Step 3: Execute Your Tests -Run your tests on HyperExecute using the CLI. After your job completes, you can visit the HyperExecute dashboard to download and view the consolidated junit xml report. diff --git a/static/docs/k6-browser-testing.md b/static/docs/k6-browser-testing.md deleted file mode 100644 index 4031dffe4..000000000 --- a/static/docs/k6-browser-testing.md +++ /dev/null @@ -1,125 +0,0 @@ -# Getting Started With k6 Browser Testing - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The [k6 Browser module](https://github.com/grafana/xk6-browser) brings browser automation and end-to-end web testing to k6 while supporting core k6 features. The Browser module introduces browser-level APIs that enable seamless interaction with browsers and facilitate the collection of frontend performance metrics as an integral part of your k6 tests. - -k6 browser module aims to provide rough compatibility with the Playwright API, so you don’t need to learn a completely new API. - -TestMu AI allows you to run k6 Browser tests on a browser farm of 40+ real browsers and operating system combinations. This guide will cover the basics of getting started with K6 testing on the TestMu AI platform. - -**Note**: k6 Browser is an experimental module that supports browser testing through the Chrome DevTools Protocol (CDP). - -## Prerequisites - -> Note: All the code samples in this documentation can be found in the TestMu AI's Repository on GitHub. You can either -> download or clone the repository to quickly run your tests. - src={require('../assets/images/icons/github.png').default} alt="Image" className="doc_img"/> View on GitHub - -1. Install k6. Refer the installation guide here: https://k6.io/docs/get-started/installation/ - -2. In order to run your k6 tests, you will need to set your TestMu AI username and access key in the environment - variables. Click the **Access Key** button at the top-right of the Automation Dashboard to access it. - -**Windows** - -```js -set LT_USERNAME = "YOUR_LAMBDATEST_USERNAME" -set LT_ACCESS_KEY = "YOUR_LAMBDATEST_ACCESS_KEY" -``` - -**macOS/Linux** - -```js -export LT_USERNAME = "YOUR_LAMBDATEST_USERNAME" -export LT_ACCESS_KEY = "YOUR_LAMBDATEST_ACCESS_KEY" -``` - -## Run Your First k6 Test - -Shown below are the steps on running k6 tests on the TestMu AI platform. - -1. Clone the [k6-browser-tests-sample GitHub repository](https://github.com/LambdaTest/k6-browser-tests-sample) and - switch to the cloned directory. - -```sh -git clone https://github.com/LambdaTest/k6-browser-tests-sample.git -cd k6-browser-tests-sample -``` - -2. Ensure you have K6 installed. - -3. Configure your TestMu AI authentication credentials. - -Once you are done with the above-mentioned steps, you can initiate your first k6 test on TestMu AI. - -> **Test Scenario**: The below test script runs on Chrome browser on macOS Ventura. It visits the DuckDuckGo search engine and searches for Playwright. - -```js -import {chromium} from 'k6/experimental/browser'; -import {expect} from 'https://jslib.k6.io/k6chaijs/4.3.4.3/index.js'; - -export default async function() { -const capabilities = { -"browserName": "Chrome", -"browserVersion": "latest", -"LT:Options": { -"platform": "MacOS Ventura", -"build": "K6 Build", -"name": "K6 Test", -"user": `${__ENV.LT_USERNAME}`, -"accessKey": `${__ENV.LT_ACCESS_KEY}`, -"network": true, -"video": true, -"console": true, -'tunnel': false, // Add tunnel configuration if testing locally hosted webpage -'tunnelName': '', // Optional -'geoLocation': '', // country code can be fetched from https://www.testmuai.com/capabilities-generator/ -}, -}; - -const wsURL = `wss://cdp.lambdatest.com/k6?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` -const browser = chromium.connect(wsURL); - -const page = browser.newPage(); - -try { -await page.goto("https://duckduckgo.com"); -await page.screenshot({path: 'screenshots/k6Screenshot.png'}); - -let element = await page.$("[name=\"q\"]"); -await element.click(); -await element.type("K6"); -await element.press("Enter"); -let title = await page.title(); - -try { -expect(title).to.equal("K6 at DuckDuckGo"); -// Mark the test as passed or failed -await page.evaluate(_ => {}, `lambdatest_action: ${JSON.stringify( -{action: "setTestStatus", arguments: {status: "passed", remark: "Assertions passed"},})}`); -} catch (e) { -await page.evaluate(_ => {}, `lambdatest_action: ${JSON.stringify( -{action: "setTestStatus", arguments: {status: "failed", remark: e.stack}})}`); -console.log("Error:: ", e.stack); -} -} finally { -page.close(); -browser.close(); -} -}; -``` - -4. Pass the below command to run the test. - -``` -K6_BROWSER_ENABLED=true k6 run k6_sample.js -``` - -## View your k6 test results - -The TestMu AI Automation Dashboard is where you can see the results of your k6 tests after running them on the TestMu AI platform. - -The below screenshot of TestMu AI Automation Dashboard shows the k6 build on the left and the build sessions associated with the selected build on the right. - -On clicking the session name of the respective test, you can view the details of k6 test session that you just executed. For example, the below screenshot shows a test execution details of k6 test like Test Name, Test ID, selected configurations, test logs, basic info, input config, and test session video. diff --git a/static/docs/kane-ai-api-testing.md b/static/docs/kane-ai-api-testing.md deleted file mode 100644 index 453e7e332..000000000 --- a/static/docs/kane-ai-api-testing.md +++ /dev/null @@ -1,49 +0,0 @@ -# KaneAI - API Testing - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This document provides a detailed guide to performing API testing using KaneAI. The API testing feature allows for comprehensive backend testing, complementing existing UI testing capabilities. Follow the instructions below to execute API tests using the PetStore API as an example. - -## 1. Adding an API in a Web Test - -To start API testing on KaneAI, create a web test using the PetStore API, a commonly available sample. This will allow you to demonstrate API testing capabilities effectively. - -- **Step**: Add an API through the slash command and navigate to the API module. - -## 2. Adding a Curl Command - -Within the API module, you can input the curl command to configure the API settings automatically. - -- **Step**: Paste your curl command into the designated area. KaneAI will populate all necessary details. You may choose to validate the API response or add it directly to the test steps. - -## 3. Validating API Response on KaneAI - -To ensure the API works as expected, use the validation feature. This step confirms that the API responds correctly and can be added to test steps. - -- **Step**: Click the 'validate' option to check the API response. A 200 response status indicates successful validation, automatically adding the API to your test steps. You can now proceed to submit multiple APIs simultaneously if needed. - -## 4. Adding Non-Success APIs in Test Steps - -For APIs that do not return a 200 status, you can review the response body and manually add them to the test steps as required. - -- **Step**: If the API returns a 400 Bad Request or another error, it will not be added automatically. Review the response and add the API manually if needed. - -## 5. Adding Multiple APIs in One Go - -KaneAI allows batch processing of multiple APIs to streamline testing. This feature is helpful for scenarios requiring the execution of several API calls in succession. - -- **Step**: Add multiple APIs by clicking the plus icon and selecting each API, or paste multiple curl commands to add them automatically to the test steps. - -## 6. Handling Different HTTP Methods - -KaneAI supports various HTTP methods like POST, PUT, GET, and DELETE, allowing you to test diverse API interactions. - -- **Step**: Add APIs using different HTTP methods, such as a PUT or DELETE command. Validate each API as before, and if successful, they will be automatically included in the test steps. - -## 7. Executing and Reviewing Test Steps - -Once all APIs are added, KaneAI enables simultaneous execution, with details available on methods used, response statuses, and execution times. - -- **Step**: Click to execute all added APIs in one go and review the response details for insights into API performance and data returned. - -This structure provides logical groupings for different aspects of API testing with KaneAI, making it easier to follow each type of action required for comprehensive API testing. diff --git a/static/docs/kane-ai-app-test-writing-guidelines.md b/static/docs/kane-ai-app-test-writing-guidelines.md deleted file mode 100644 index a2270df56..000000000 --- a/static/docs/kane-ai-app-test-writing-guidelines.md +++ /dev/null @@ -1,100 +0,0 @@ -# Best Practices for Authoring Mobile Native App Tests in KaneAI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Authoring mobile native app tests in KaneAI can be significantly enhanced by following these best practices. These ensure improved stability, reliability, and accuracy of the generated steps and ultimately, the automation. - -## OTP and PIN Fields - -For OTP or PIN fields where each digit is in a separate box: - -- Use the keyword: `OTP` -- Example: `Enter OTP "1234"` - -## Partially Clickable Text (Spannable Text) - -For labels or descriptions containing partially clickable links/texts: - -- Use the keyword: `spannable text` -- Example: `Click on spannable text: "Terms and Conditions" in bottom info modal` - -## Wheels, Sliders, Seekbars, and Date Pickers - -Supported element classes: - -- **Android**: `android.widget.NumberPicker`, `android.widget.SeekBar` -- **iOS**: `XCUIElementTypePickerWheel`, `XCUIElementTypePicker`, `XCUIElementTypeDatePicker`, `XCUIElementTypeSlider` - -Use instruction: - -- `Enter "2" in first column` - -If issues arise: - -- Use TestMu AI App Live [UI Inspector](/support/docs/ui-inspector/) to verify if the element class is supported. - -## Dismissing Popups - -To dismiss popups without visible close buttons: - -- Use instruction: `dismiss dialog` - -## Hiding Keyboard - -- **Android**: Use instruction: `hide keyboard` -- **iOS**: Prefer `press enter` after `type` or use: `click on cancel` button in keyboard if applicable - -Example: - -- `Type abcd and press enter` - -> Note: iOS keyboards are often persistent, and `press enter` helps dismiss them. On Android, this is usually not required. - -## Typing and Special Keys - -- For deleting characters: use `press Del` - -## Wait Instructions - -- Always insert explicit `wait` commands if a screen or element load takes time after an action. Refer [this doc](http://localhost:3001/support/docs/kane-ai-command-guide/#explicit-waits). - -Example: - -- `Click on Submit button` -- `Wait for 3 seconds` -- `Assert confirmation text is visible` - -### Step Timeout (per-step wait) - -By default, each App step waits up to **10 seconds** for its target element to appear. On slow-loading screens, you can override this on a **per-step** basis instead of adding fixed `wait` steps: - -1. Click the **three-dot (⋯) menu** on the step. -2. Select **Step Timeout**. -3. Enter the timeout value in seconds (min **1**, max **300**) and save the step. - -Step Timeout is a **dynamic ceiling**: the step proceeds as soon as the element is ready and only uses the extra time on genuinely slow loads. It applies to element-based interactions (Tap/Click, Type/Input, Search, Clear) and has no effect on fixed `wait` steps, assertions, or API steps. Leaving it unset keeps the default 10-second behavior. - -Step Timeout is supported on KaneAI App automation for **Android and iOS** real devices, bringing App testing in line with Web and Mobile Web, which already support per-step timeouts. - -## Scroll and Wait Pattern - -If scrolling is needed before performing an action: - -- Use: `Scroll down 2 times` -- Add wait after scroll: `Wait for 2 seconds` - -## Explicit Element Identifiers - -Use specific element identifiers to improve accuracy: - -- `Click on abc 'button'` -- `Click on continue 'text'` -- `Assert abc button 'background color' is red` - -## Typing in Specific Input Fields - -Be explicit when typing: - -- `Type "2" in "username" input field` - -By following these practices, users can significantly reduce errors and improve clarity when authoring native mobile app tests in KaneAI. diff --git a/static/docs/kane-ai-app-test.md b/static/docs/kane-ai-app-test.md deleted file mode 100644 index b94a6a6b9..000000000 --- a/static/docs/kane-ai-app-test.md +++ /dev/null @@ -1,30 +0,0 @@ -# Automate your App Testing with KaneAI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -In this guide, we'll walk you through the process of generating, and executing automated tests for your mobile application using KaneAI. - -## Steps to generate the test cases - -### Step 1: Create an App Test -Navigate to the [KaneAI homepage](https://www.testmuai.com/login/?redirectTo=https://kaneai.lambdatest.com/objective) and click on the **Create an App Test** button to initiate the process of generating test cases for your mobile app. Now select your desired operating system (Android, iOS) and upload your application to the platform to start testing. - -### Step 2: Write Your Test Cases in Plain English -In this step, you'll describe your desired test outcomes using plain English. Don't worry about writing specific code or scripts – KaneAI will handle that for you. - -For example: -- Click on the hamburger menu button. -- Write the test in the text field "This is a sample app". -- Click on the "+" button. - -Continue to add instructions/commands in plain English as you go; KaneAI will automatically queue them for execution, eliminating the need to wait for one test to finish before writing the next. Click the **Finish Test**. - -> **Remember:** The more specific you are, the more accurate KaneAI's generated test cases will be. - -### Step 3: Save your Test Case -Now, save all of the test cases you've generated so far. - -Select your Project and the folder where you want to save your test cases. The **Test Name** and **Description** will be generated automatically, but you can also edit them. Select your test's **Type** and **Status**, as well as the necessary tags that will allow you to search for it later. - -### Step 4: Execute your test -To execute your test, click on the **HyperExecute** button and you will be redirected to the [HyperExecute dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute/jobs) where your tests will be executed. diff --git a/static/docs/kane-ai-automation-code-generation.md b/static/docs/kane-ai-automation-code-generation.md deleted file mode 100644 index bb526081e..000000000 --- a/static/docs/kane-ai-automation-code-generation.md +++ /dev/null @@ -1,107 +0,0 @@ -# KaneAI Automation Code Generation - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -KaneAI generates automation scripts in multiple frameworks and languages based on natural language inputs from users. KaneAI is available for desktop web applications as well as native mobile applications currently. - -KaneAI offers two authoring experiences, Classic and New Experience. New Experience is being rolled out in phases. To enable New Experience for your organization, reach out to our support team. The frameworks and languages supported by each experience are listed below. - -Click the `Generate New Code` button to go to code generation page where you can select desired framework and language. - -## Supported Frameworks and Languages - -The table below shows the frameworks and languages KaneAI can generate code in, and the experience each is available under. - - Framework - Language - Availability - Classic - New Experience - - Selenium - Python - Generally available (default) - ✓ - ✗ - - Java - Available on request * - ✓ - ✗ - - Playwright - Python - Available on request * - ✓ - ✓ - - C# - Available on request * - ✓ - ✗ - - JavaScript - Available on request * - ✗ - ✓ - - Cypress - JavaScript - Coming soon - ✗ - ✗ - - WebdriverIO - JavaScript - Coming soon - ✗ - ✗ - - Appium - Python - Generally available (default) - ✓ - ✗ - -**✓** Supported    **✗** Not available. New Experience currently covers Desktop Web only. Mobile is supported in Classic. - -**Available on request.** Please reach out to Support to enable it. These features are partially rolled out and will soon be generally available for all users. - -## Understanding the Code Tab - -After code generation completes, the **Code** tab shows the status of each generated code entry and lets you run a Sample Run Execution to validate it before adding the test case to a full Test Run. - -### Code Generation States - -| State | What it means | -|---|---| -| **Generating** | Code is being created. No actions are available until generation finishes. | -| **Unverified** | Code generation is complete, but a Sample Run Execution has not been run, or the test case has changed since the last run. | -| **Verifying** | A Sample Run Execution is in progress via HyperExecute. | -| **Verified** | The most recent Sample Run Execution completed successfully. | - -### Actions Available After Code is Generated - -Once code is generated, the following actions are available: - -- **Execute & Verify**: Triggers a Sample Run Execution on HyperExecute. This runs the generated code the same way a real KaneAI test run would, including respecting assertion outcomes. While the run is in progress, the status changes to **Verifying** and a **Sample execution in progress · View in HyperExecute** link appears. Once complete, the status updates to **Verified** and the button label changes to **Execute**. -- **Download**: Downloads the generated code to your local machine. - -### Sample Run Execution Behavior - -- Sample Run Execution can only be initiated from the **Code** tab of a test case. -- The **Execute** button is always enabled. You can proceed to run the test on HyperExecute regardless of whether a Sample Run Execution has passed, failed, or not been triggered. - -### Adding Test Cases to a Test Run - -Test cases with generated code can be added to a Test Run regardless of the Sample Run Execution status, whether it passed, failed, or was never triggered. The only cases where a test case is not available for selection in a Test Run are: - -- Code generation failed or was not initiated. -- Code generation is still in progress. - -KaneAI code export leverages public libraries. The generated automation scripts depend on these published packages: - -- Playwright Python: [testmu-playwright-python](https://pypi.org/project/testmu-playwright-python/) -- Playwright C#: [TestmuAI.Playwright.Bindings](https://libraries.io/nuget/TestmuAI.Playwright.Bindings) -- Playwright JavaScript: [@testmuai/playwright-bindings](https://www.npmjs.com/package/@testmuai/playwright-bindings) -- Selenium Java: [testmuai-selenium-bindings](https://mvnrepository.com/artifact/io.github.lambdatest/testmuai-selenium-bindings) diff --git a/static/docs/kane-ai-click-interactions.md b/static/docs/kane-ai-click-interactions.md deleted file mode 100644 index 44fb94293..000000000 --- a/static/docs/kane-ai-click-interactions.md +++ /dev/null @@ -1,251 +0,0 @@ -# Advanced Click Interactions in KaneAI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -KaneAI supports advanced click variants beyond a standard single click: **press and hold (long press)**, **multi-click (double / triple / N-click)**, and **right click (context click)**, across Desktop Web, Android apps, iOS apps, and Mobile Web. Each can be authored with natural language or captured via Manual Interaction, and renders as a distinct step with its own icon and pill label. - -## Overview - -You can author any of the three click variants in two ways: - -- **Natural Language (NL)**: describe the click in plain English (e.g. `long press the menu icon for 5 seconds`). -- **Manual Interaction**: perform the gesture on the device or browser viewport and have it captured as a step. - -| Click Type | Pill Label | Typical Use | -|------------|------------|-------------| -| **Press and Hold** | LONG PRESS | Developer mode entry, context menus, multi-select, hidden settings, push-to-talk | -| **Multi-Click** | MULTI CLICK | Text selection, zoom, list opening, counter interactions | -| **Right Click** | RIGHT CLICK | Context menus on web: duplicate, rename, delete, custom actions | - -## Supported Platforms - -| Platform | Press and Hold | Multi-Click | Right Click | Manual Interaction | -|----------|:--------------:|:-----------:|:-----------:|:------------------:| -| **Desktop Web** | ✅ | ✅ | ✅ | ✅ | -| **Android App** | ✅ | ✅ | ❌ | ✅ | -| **iOS App** | ✅ | ✅ | ❌ | ✅ (Portrait only) | -| **Mobile Web** | ✅ | ✅ | ❌ | ❌ (NL only) | - -**Right click is web-only.** On mobile, use **long press** to open context menus. KaneAI does not silently convert one to the other because they have different semantics. - -## Press and Hold (Long Press) - -### Author with Natural Language - -``` -long press the menu icon -press and hold the build number for 10 seconds -tap and hold the profile avatar -hold down the record button for 5 seconds -long press the Submit button for 30 seconds -``` - -#### Supported NL Patterns - -| Pattern | Example | Duration | -|---------|---------|----------| -| Default duration | `long press the menu icon` | 1 second | -| Custom duration | `press and hold the color swatch for 3 seconds` | 3 seconds | -| Alternative phrasing | `tap and hold the profile avatar` | 1 second | -| Hold down | `hold down the record button for 5 seconds` | 5 seconds | - -#### Duration Rules - -- **Default:** 1 second when no duration is specified. -- **Range:** 0.5 seconds (minimum) to 30 seconds (maximum). -- **Units:** Seconds only. Minutes, hours, and days are rejected. -- **Accuracy:** Within ±200 ms of the specified duration. - -#### Disambiguation - -Only phrases containing **`long press`**, **`press and hold`**, **`tap and hold`**, or **`hold down`** followed by a UI element trigger a long press. - -| Phrase | Interpretation | -|--------|---------------| -| `press Enter` | Keyboard key press | -| `press the login button` | Normal click (bare "press") | -| `press Tab` | Keyboard key press | - -### Manual Interaction Capture - -In Recording mode, the capture layer classifies touch gestures by duration and movement: - -| Gesture | Rule | -|---------|------| -| Tap | Touch < 1 second, no movement | -| **Long Press** | **Touch ≥ 1 second, no movement (< 10 px drift)** | -| Swipe | Movement ≥ 10 px | -| Drag | Hold ≥ 1 second + Movement ≥ 10 px | - -### Common Use Cases - -| Use Case | Typical Duration | -|----------|------------------| -| Developer mode entry | 5–10 seconds | -| Context menu (mobile) | 0.5–2 seconds | -| Delete confirmation | 1–3 seconds | -| Multi-select mode | 0.5–1 second | -| Hidden settings access | 3–15 seconds | - -## Multi-Click (Double / N-Click) - -### Author with Natural Language - -``` -double click on the submit button -click the increment button 5 times -tap twice on the like button -tap thrice on the notification bell -``` - -#### Supported NL Patterns - -| Pattern | Example | Frequency | -|---------|---------|-----------| -| Double click | `double click on the submit button` | 2 | -| Double tap | `double tap to zoom in` | 2 | -| Click N times | `click the button 5 times` | N | -| Tap twice / thrice | `tap twice on the like button` | 2 / 3 | - -On mobile, **`click`** and **`tap`** are interchangeable. `double click X` maps to the same multi-click action as `double tap X`. - -#### Frequency Rules - -- **Maximum:** 20 clicks per instruction. Higher values are rejected as `INVALID_PARAMETER`. -- **Phrasing:** Use `click X N times` for N ≥ 3. The literal phrase `triple click X` is not supported. - -### Manual Interaction Capture - -The capture layer detects multi-click via a **debounce window**: - -- **Window:** 200 ms between consecutive clicks -- **Proximity:** Clicks must land within 10 px of each other -- **Classification:** Two clicks within the window at the same spot = double click, three = triple click, and so on -- **Inter-click interval:** Recorded for accurate replay - -## Right Click (Context Click) - -Right click is supported on **Desktop Web only**. On mobile, use long press instead. - -### Author with Natural Language - -``` -right click on the file item -right-click the table row -context click on the selected text -secondary click on the canvas element -``` - -#### Supported NL Patterns - -| Pattern | Example | -|---------|---------| -| Right click | `right click on the file row` | -| Right-click (hyphenated) | `right-click the document` | -| Context click | `context click on the table row` | -| Secondary click | `secondary click on the canvas` | - -#### Disambiguation - -The word **`right`** must directly modify the verb (`click` / `tap` / `press`) to trigger a right click. When `right` modifies a noun or is part of an adverb, it is treated as a normal click. - -| Phrase | Interpretation | -|--------|---------------| -| `click the right panel` | Normal click (positional) | -| `click the button on the right` | Normal click (positional) | -| `click the submit button right now` | Normal click (`right now` is temporal) | -| `right away click the link` | Normal click (`right away` is temporal) | - -### Manual Interaction Capture - -On Desktop Web, right-clicking fires a `contextmenu` event that is captured as a right-click step automatically. - -## Mutual Exclusivity Rules - -The three click modifiers cannot be combined on a single operation. Attempting to combine them returns `INVALID_PARAMETER`. - -| Combination | Result | -|-------------|--------| -| Long press + Multi-click | Invalid: duration and frequency cannot both be set | -| Right click + Long press | Invalid: right click cannot combine with duration | -| Right click + Multi-click | Invalid: right click cannot combine with frequency | - -**NL examples that fail:** -- `long press the button 3 times` -- `right double-click on X` -- `right click and hold X` - -## UI Rendering - -Each click type displays a distinct icon and pill label in the **Sidebar**, **Test Case Summary**, and **Module view**: - -| Click Type | Pill Label | Icon | -|------------|------------|------| -| Normal click | CLICK | Default cursor | -| Long press | LONG PRESS | Hold / timer | -| Double click | MULTI CLICK | Double-click | -| N-click (3+) | MULTI CLICK | Multi-click | -| Right click | RIGHT CLICK | Context menu | - -## Best Practices - -- **Use NL for most interactions**: fastest authoring path; produces element-first steps. -- **Use Manual Interaction for precise timing**: when exact hold duration matters (e.g. 10 s developer mode). -- **On mobile, use `long press` instead of `right click`** to open context menus. -- **Don't combine modifiers** in a single instruction, they are mutually exclusive. -- For `click N times`, ensure the target element stays **stable** (doesn't move, disappear, or change) between clicks. -- Allow **1–2 seconds after navigation** before performing a click, gives the page time to stabilize. -- Use **variables** to parameterize: `long press the button for ${hold_duration} seconds`. -- Use **conditionals** to apply click types contextually: `if popup is visible then right click on it`. - -## FAQs - -**What is the default long press duration?** -1 second. For example, `long press the menu icon` holds for 1 second. - -**Does `press the button` trigger a long press?** -No. Bare "press" is interpreted as a normal click. Use `long press`, `press and hold`, `tap and hold`, or `hold down` to trigger a long press. - -**Is `double click` different from clicking twice in two separate steps?** -Yes. `double click` (frequency = 2) fires a native `dblclick` event. Two separate `click` instructions are independent single clicks with a pause between them. - -**Can I double tap on mobile?** -Yes. `double click` and `double tap` are interchangeable on mobile. - -**What is the maximum click frequency?** -20 clicks per instruction. Higher values are rejected as `INVALID_PARAMETER`. - -**Why does right click fail on mobile?** -Right click is a mouse-specific interaction that doesn't exist on touchscreens. Use `long press` instead. It opens context menus in most apps. - -**Does `click the right panel` trigger a right click?** -No. KaneAI treats positional `right` as a description of the panel, not a gesture modifier. - -**How does KaneAI tell a tap from a long press during recording?** -By duration: under 1 second = tap, ≥ 1 second with no movement = long press, movement over 10 px = swipe (or drag if held ≥ 1 second). - -**How does KaneAI detect a double click during recording?** -Two clicks within 200 ms at the same location (within 10 px). - -**Can I use variables for the long press duration or click count?** -Yes. `long press the button for ${hold_duration} seconds` and `click the button ${click_count} times` both work. - -**Do click modifiers survive autoheal?** -Yes. Autoheal re-locates the element on a modified page; the click modifier (duration, frequency, right-click flag) is preserved. - -**Can I use these click types inside a Module?** -Yes. All three work inside Modules: create, import, edit, and version-bump as usual. - -## Limitations - -1. **Right click is web-only.** Returns `UNSUPPORTED_OPERATION` on mobile. -2. **Mutual exclusivity.** Long press, multi-click, and right click cannot be combined in a single instruction. -3. **Long press duration range.** Limited to 0.5–30 seconds only. -4. **Multi-click frequency cap.** Maximum 20 clicks per instruction. The literal phrase `triple click X` is not supported. Use `click X 3 times` instead. -5. **Mobile Web: no Manual Interaction.** Only NL instructions are available for long press and multi-click on mobile browsers. -6. **iOS Landscape: no Manual Interaction.** Long press and multi-click MI capture are not supported in Landscape orientation. -7. **Duration accuracy.** Long press is accurate to ±200 ms. Use cases requiring millisecond precision should account for this tolerance. -8. **Multi-click on dynamic elements.** If the target moves, disappears, or changes between clicks, later clicks may miss. Ensure element stability. -9. **Nested if-else not supported.** Single-level if-else with click modifiers works (e.g. `if popup is visible then right click on it`), but nested if-else inside another conditional is not supported. -10. **Secrets as duration values.** `long press for {{secrets.user.DURATION}} seconds` is not supported. Secret values cannot be parsed as numeric durations. -11. **No silent conversion.** Right click is not auto-converted to long press on mobile, and long press is not auto-converted to right click on web. Each gesture must be authored explicitly. diff --git a/static/docs/kane-ai-command-guide.md b/static/docs/kane-ai-command-guide.md deleted file mode 100644 index d78bee9e3..000000000 --- a/static/docs/kane-ai-command-guide.md +++ /dev/null @@ -1,192 +0,0 @@ -# KaneAI - Command Guide - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -KaneAI lets you write **natural language instructions** that it converts into executable automation steps. This guide covers a wide range of commands that help you interact with web pages, control tabs, handle waits, perform assertions, and build dynamic, reliable tests without writing code, best practices and troubleshooting tips to help you get the most out of KaneAI. - -## Navigation & Page Interaction - -Use the below commands to move around the page: - -| Action | Example Commands | -|--------------|------------------------------------------------| -| Open URL | `go to https://example.com` , `open https://example.com` | -| Click | `click on login button` | -| Double / N-Click | `double click the title` , `click the button 5 times`, see [Click Interactions](/support/docs/kane-ai-click-interactions/) | -| Right Click | `right click on the file row` (web only), see [Click Interactions](/support/docs/kane-ai-click-interactions/) | -| Long Press | `long press the menu icon for 3 seconds`, see [Click Interactions](/support/docs/kane-ai-click-interactions/) | -| Type Text | `enter "user@example.com"` , `type "password123"` | -| Clear Field | `clear email input field` | -| Select Date | `select 21st september date` | -| Hover | `hover on shop by category option` | -| Search | `search for "laptop"` | -| Refresh | `refresh the page` | -| Navigation | `go back` , `navigate back` , `go forward` , `navigate forward` | - -## Waits & Timeouts - -**Wait until element** - -Every element interaction automatically includes a **default maximum wait timeout of 10 seconds**, giving the element time to become interactable. This ensures that commands with locators such as click or type don't fail due to slow loading or page transitions. - -**Explicit Waits** - -You can **define timeouts explicitly** when you expect a slower response or need to wait longer than the default. Use simple, natural language commands like: - -- **wait for 5 seconds** - This will pause the script for exactly 5 seconds before proceeding to the next step. - -This is useful when you need precise control over wait times for specific actions or page loads. - -**Custom Timeouts** - -If you prefer, you can also set custom wait timeouts directly in the step menu: - -1. Click the step menu -2. Select **Step Timeout** -3. Enter your preferred timeout value (min 1 second, max 300 seconds) - -Use custom wait timeout when dealing with slow page loads or dynamic content. Step Timeout is a dynamic ceiling. The step proceeds as soon as the element is ready and only uses the extra time on genuinely slow loads. - -Step Timeout is supported across **Web**, **Mobile Web**, and **App (Android and iOS)** testing. - -**When to Use Explicit & Custom Waits** - -- **Custom waits** are ideal for cases where you need to adjust the wait time for certain elements to load (such as dynamic content or external API calls). [RECOMMENDED] -- **Use explicit waits** when you know an action (like a button click or form submission) will take longer than usual. - -Custom waits apply only to element-based interactions. - -## Tab Management - -KaneAI makes it easy to manage browser tabs during automated tests, whether you’re opening new tabs, switching between them, or closing them. Here's how to control your tabs with simple commands: - -**Opening a New Tab** - -To open a new tab in your browser, use the following command: - -- Open amazon.com in a new tab. This will open a new tab, allowing you to work in multiple tabs during your test. -- **IMPORTANT:** Highlight that even when a new tab is open, we need to explicitly switch context of the application to that new tab by adding an instruction for “Switch to Amazon tab” or “Switch to 2nd tab” - -**Switching Between Tabs** - -Switching between tabs lets you interact with different pages or elements within the same session. You can switch by tab number or by the tab's name: - -- **Switch to 2nd tab:** This switches focus to the second tab (index starts at 1), allowing you to perform actions on that page. -- **Switch to Lambdatest tab:** Switch to the tab named "Lambdatest", regardless of its position in the tab order. - -Use these commands when you have multiple pages open and need to shift focus between them. - -**Closing Tabs** - -You can close tabs either by position or by name when you're done with them: - -- **Close Lambdatest tab:** Closes the tab named "Lambdatest". This is useful when you’ve finished interacting with a specific page and want to clean up the session. -- **Close 2nd tab:** Closes the second tab in the current session. This command is useful when you want to manage tab resources and focus on a single page. - -**Tips for Managing Tabs** - -- When switching tabs, be sure the tab is open before using a **switch** command. If the tab doesn’t exist or has already been closed, the command will fail. -- Tabs are indexed in the order they were opened. Keep track of the open tabs to avoid confusion, especially when switching by position. -- Use **explicit waits** if the tab change involves loading content or dynamic changes. - -## Scrolling Commands - -KaneAI allows you to easily control scrolling behavior on your web pages. Here’s how you can scroll to elements, pages, or specific sections: - -**Scroll Until Element is Visible:** This command scrolls the page until a specific element, like a button, becomes visible in the viewport. It's perfect for revealing hidden elements that are off-screen initially. [Currently only supported on Web] - -**Scroll by a Set Amount** - -- **Scroll by 100px:** Scrolls the page down by 100 pixels, allowing for fine-grained control of the viewport position. -- **Scroll by 20 percent:** Scrolls the page down by 20% of the current page height, which is useful for relative scrolling in dynamic pages. - -**Scroll to Specific Positions** - -- **Scroll to the bottom:** Scrolls to the very bottom of the page, making the last section or content visible. -- **Scroll to the top:** Brings the page back to the top, useful for navigating or resetting the viewport. - -**Scroll Multiple Times** - -- **Scroll 3 times:** Repeats the scroll action three times, which can be helpful for interacting with infinite scroll pages or loading dynamic content. - -Use `scroll until` for elements not yet visible in the viewport. Works for entire pages and scrollable subsections. - -## Assertions - -Assertions are used to validate whether your expectations match the actual outcomes in the application. KaneAI supports several types of assertions to check various elements during test execution: - -| Category | Example | -| :--- | :--- | -| **Driver Assertions** | Validate **browser state** ( current URL, window dimensions).Example: Assert if current URL is `https://example.com`Validate window height and width | -| **Text Assertions** | Check for **presence/absence of text**.Example: Assert “Welcome Back!” is visibleExample: Assert error message “Invalid password” is displayed | -| **Visual Assertions** | Confirm **image or logo visibility**.Example: Assert company logo appears in headerExample: Assert product image is visible | -| **Relative Assertions** | Validate **element relationships**.Example: Assert login button is in same column as usernameExample: Assert submit & cancel buttons are in same row | -| **Mathematical Assertions** | Check **numeric calculations**.Example: Assert `3 + 4 = 7` | -| **Element State Assertions** | Verify **element states** like enabled, disabled, visible, hidden, checked, clickable, and more.Example: Assert the "Submit" button is enabledExample: Assert the "Terms" checkbox is checked | -| **DOM Attribute Assertions** | Assert **HTML, ARIA, or data attribute** values on elements.Example: Assert the aria-expanded of the "Menu" equals "true"Example: Assert the href of the "Dashboard" link contains "/dashboard" | -| **CSS Property Assertions** | Validate **computed CSS properties** like color, font-size, opacity, and more.Example: Assert the font-size of the "Heading" equals "24px"Example: Assert the background-color of the "Banner" equals "blue" | - -Since **July 20, 2025**, all assertions fail tests immediately by default. -Older tests may still use *warn-and-continue* until edited. -More info: [Failure Conditions](/support/docs/kaneai-failure-conditions) - -## Unsupported Assertions - -The following assertion types are not yet available via natural language. Use **JS snippets** as workarounds where applicable. - -| Category | Example | Description | -| :--- | :--- | :--- | -| **Spatial** | Assert the 5th column of the table contains "Jordan.Mathews" | Requires detailed knowledge of the DOM structure to assert based on element positions and arrangement. | -| **Action-based** | Assert tooltip appears after hover | Assertions based on actions (e.g., hover or click) need to be split into multiple steps (hover → wait → assert). | -| **Nested Assertions** | Assert login button enabled AND welcome message visible | Nested assertions involving multiple layers of validation within a single assertion are not supported. Split into separate assertions. | - -## Query Information - -KaneAI allows you to retrieve specific information from your web page during tests: - -- **query the current URL:** Retrieves the current URL of the active page -- **query the time mentioned in the poster:** Extracts specific text or data, such as the time listed on a poster or other page element - -## Conditional Actions - -Use conditional logic to execute actions based on specific conditions on the page. KaneAI supports **If / Else‑If / Else** blocks with multiple conditions combined via AND / OR logic. You can add modules, JavaScript, API, and DB steps inside each branch. For the full guide, see [Conditional Logic](/support/docs/kaneai-conditional-logic/). - -Nested conditions (an If / Else block inside another If / Else block) are not supported. - -**Conditional Click** -Example: - **if pop up is present in the viewport then click on it**: clicks on a pop-up element if it is visible within the current view. - -**Conditional Selection** -Example: -**if price < 200 then click add to cart else select second product**: selects an action based on the price condition. If the price is less than $200, it adds the item to the cart. Otherwise, it selects the second product. - -## Do's and Don'ts - -Follow these examples for better clarity and performance during test writing: - -| **Do (Good Example)** | **Don’t (Bad Example)** | -| ----------------------------------------------------------------- | -------------------------------------------------------------------------------------- | -| `Assert text "Submit" on form footer is visible` | `Assert that the submit button works and the layout is correct` | -| `Assert textbox contains the exact value 'heading1'` | `Assert the textbox has heading` (vague, unclear what “heading” refers to) | -| `Assert logo is visible` | `Assert 3 search results are shown` | -| `Assert search results appear after a wait step` | `Wait for search results and then assert they are visible` (assertion contains action) | -| `Assert the divider looks red` | `Assert divider color is #ff5733` (too detail‑specific, fragile visual check) | -| `Assert no text in table overflows its cell` | `Assert the page looks well designed` (subjective, unclear) | - -**Key Takeaways:** - -- **Be specific and clear:** Avoid vague assertions like "heading" and ensure conditions are well-defined. -- **Don’t mix actions with assertions:** Keep actions (like waiting for an element) separate from assertions. -- **Avoid overly specific checks:** Instead of checking for exact visual details (like a color code), focus on functionality (e.g., "Assert divider is visible"). -- **Limit the number of assertions:** Avoid making too many assertions in a single step. Test one clear outcome per assertion. - -## Best Practices - -Follow these best practices to write effective and reliable tests: - -- **Write clear, specific commands:** Ensure your instructions are precise and unambiguous to improve test reliability. -- **Combine command types for richer tests:** Use a mix of commands like waits, clicks, and queries to create more complex and dynamic test flows. -- **Use conditional logic for dynamic scenarios:** When a page changes dynamically (e.g., based on user interaction), use if-else logic to handle different conditions effectively. -- **Apply waits thoughtfully for stability:** Use waits when necessary to ensure elements are loaded and ready for interaction, preventing timing issues. -- **Always add assertions to validate outcomes:** After actions, always verify expected outcomes (e.g., element visibility, text presence) to ensure that the desired result has been achieved. diff --git a/static/docs/kane-ai-deeplink-support.md b/static/docs/kane-ai-deeplink-support.md deleted file mode 100644 index d9c8ee864..000000000 --- a/static/docs/kane-ai-deeplink-support.md +++ /dev/null @@ -1,76 +0,0 @@ -# Deeplinks Support for Mobile App Authoring - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## What are Deeplinks? - -Deeplinks are URL-based mechanisms that allow you to directly access specific screens and functionalities within your mobile application, bypassing standard navigation flows. In the context of mobile app testing with KaneAI, deeplinks enable testers to quickly navigate to targeted screens and perform context-aware actions without manually navigating through the app interface. - -## How Deeplinks Work in KaneAI - -When you use deeplinks in KaneAI during mobile app test authoring, the system: - -1. Accepts a deeplink URL from the user -2. Identifies the target application package (Android) or bundle ID (iOS) -3. Invokes the deeplink within the active app session -4. Navigates directly to the specified screen or performs the defined action - -KaneAI automatically pre-fills the application package or bundle ID based on the app currently installed in your test session, making the process seamless and intuitive. - -## Adding Deeplinks in KaneAI - -### Step 1: Access the Deeplink Feature - -During your mobile app test authoring session in KaneAI, you can add a deeplink instruction using the following method: - -- Type a **slash (/)** in the input box and select the **Deeplink** option from the menu - -### Step 2: Configure Deeplink Parameters - -Once the deeplink modal appears, provide the following information: - -1. **URL*** (Required) - - Enter the complete deeplink URL - - Example: `kaneai://settings` or `myapp://product/123` - -2. **App package for target*** (Required) - - For **Android**: Enter the application package name (e.g., `com.example.QAapp`) - - For **iOS**: Enter the bundle ID (e.g., `com.example.QAapp`) - - KaneAI automatically pre-fills this field with the currently installed app's package or bundle ID - -### Step 3: Add to Test Steps - -Click the **Add in steps** button to include the deeplink instruction in your test case. KaneAI will execute this step during test runs, invoking the specified deeplink within your target application. - -## Best Practices - -1. **Verify Deeplink Support:** Ensure your mobile application has proper deeplink handling implemented before using this feature in tests. - -2. **Test Deeplink Validity:** Manually verify deeplinks work correctly in your app before adding them to automated test cases. - -3. **Handle Edge Cases:** Consider scenarios where deeplinks might fail (e.g., user not logged in, invalid parameters) and add appropriate assertions. - -4. **Combine with Assertions:** After invoking a deeplink, add assertion steps to verify the correct screen was loaded and displays expected content. - -## Troubleshooting - -### Deeplink Not Working - -- Verify the deeplink URL format is correct -- Ensure the target app package/bundle ID matches the installed application -- Check if the app has proper deeplink handling configured -- Confirm the app is already running in the test session - -### Wrong Screen Opens - -- Double-check the deeplink path and parameters -- Verify your app's deeplink routing configuration -- Ensure no conflicting deeplink schemes exist - -### App Crashes on Deeplink - -- Validate the deeplink with your development team -- Check if required data or permissions are available -- Review app logs for specific error messages - -> **Need Help?** If you have questions about deeplink implementation or encounter issues, contact our support team for assistance. diff --git a/static/docs/kane-ai-drag-drop.md b/static/docs/kane-ai-drag-drop.md deleted file mode 100644 index 9cd84831b..000000000 --- a/static/docs/kane-ai-drag-drop.md +++ /dev/null @@ -1,207 +0,0 @@ -# Drag and Drop in KaneAI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -KaneAI lets you author drag interactions across **Desktop Web, Android apps, iOS apps, and Mobile Web** using either natural language or by performing the gesture yourself. Use it for Kanban boards and any other drag-driven UI flow. - -## Overview - -You can author a drag step in two ways: - -- **Natural Language (NL)**: describe the drag in plain English (e.g. `drag "Card A" to "Column B"`). -- **Manual Interaction**: perform the gesture on the device or browser viewport and have it captured as a step. - -## Drag and Drop vs Click and Drag - -KaneAI supports two distinct drag interactions. They look similar but behave differently, and they are **not interchangeable**: - -- **Drag and Drop**: the element is first **long-pressed** to pick it up, and only then moved to the target. This is the standard gesture for moving items between containers and works on **all platforms**. -- **Click and Drag**: the element is pressed and moved **immediately, without any long press**. This is how interactions such as sliders, canvas drawing, and element resizing work, and it is available on **Desktop Web only**. - -| | Drag and Drop | Click and Drag | -| --- | --- | --- | -| **Gesture** | Long press on the element first, then move it to the target | Press and move in one continuous motion, no long press | -| **Platforms** | Desktop Web, Android App, iOS App, Mobile Web | Desktop Web only | -| **Typical scenarios** | Kanban cards, list reordering, container-to-container transfer | Sliders, canvas drawing, element resizing, range selection | - -If a drag step fails on an element that doesn't respond to a long press (for example a slider thumb or a canvas), the element likely expects **Click and Drag** rather than Drag and Drop. Click and Drag has its own constraints, see [Limitations](#limitations). - -## Supported Platforms - -| Platform | Natural Language | Manual Interaction | -| --------------- | :--------------: | :----------------: | -| **Desktop Web** | ✅ | ✅ | -| **Android App** | ✅ | ✅ | -| **iOS App** | ✅ | ✅ | -| **Mobile Web** | ✅ | ❌ | - -## When to Use Which Mode - -| Use Case | Recommended Mode | -| -------------------------------------------------------------------------- | ---------------------- | -| Kanban / multi-container drag | NL or Manual | -| Sortable grid | NL or Manual | -| Container-to-container transfer | NL (recommended) | -| **Sliders** (volume, range, vertical, payment) | **Manual only** | -| **Confirmation gestures** (slide-to-confirm, drag-to-pay, swipe-to-unlock) | **Manual only** | -| Dynamic / moving drop targets | **Manual only** | -| Element resizing / canvas operations / flow charts | **Manual (recommended)** | -| Drag with no stable element identifier | Manual (recommended) | - -**Sliders and confirmation gestures cannot be authored with NL.** Use Manual Interaction to capture the gesture directly. KaneAI records source, target, and drag vector in a coordinate-safe form. - -## Author with Natural Language - -Type the instruction into the KaneAI authoring panel. - -### Supported NL Patterns - -| Pattern | Example | -| ---------------- | ---------------------------------- | -| Name-based | `drag "Card A" to "Column B"` | -| Index-based | `drag the 3rd item to the top` | -| Positional | `drag the top card to the bottom` | -| Anchor reference | `drag X to the bottom of the list` | -| Drop-on phrasing | `drop "Cart" on "Checkout button"` | - -**More examples:** - -``` -drag "Task Card" to "Done column" -move the 3rd item to the top -drop "iPhone 15" on the comparison table -drag the top card to the bottom of the list -move task card from "To Do" column to "In Progress" column -``` - -Natural language drag and drop works best when: -- Drop zones are **explicitly defined and visible**. -- You're moving items between containers. -- The target location is **static** (not dynamically generated). - -## Author with Manual Interaction - -Switch to **Manual Interaction** mode in the KaneAI authoring panel and perform the drag directly on the device viewport. - -### Gesture Classification - -| Input | Captured As | -| -------------------------------------- | ----------- | -| Touch < 1 second, no movement | Tap | -| Touch ≥ 1 second, no movement (< 10 px drift) | Long press | -| Movement ≥ 10 px | Swipe | -| **Hold ≥ 1 second + Movement ≥ 10 px** | **Drag** | - -For tap, long-press, multi-click, and right-click authoring, see [Click Interactions](/support/docs/kane-ai-click-interactions/). - -### When Manual Interaction is Required - -- **Sliders**: volume, range, vertical, payment-style (e.g., slide-to-pay). -- **Confirmation gestures**: slide-to-confirm, drag-to-pay, swipe-to-unlock. -- **Custom drag handles** without a stable accessibility ID or selector. -- **Canvas-based interactions**, drawing tools, and flow chart manipulation. -- **Element resizing** and precise positioning. - -Manual recording is recommended for any scenario where the **drop location isn't explicitly defined** or the target **changes during the drag**. - -## Replay Behavior - -- Elements are **re-resolved at runtime**. KaneAI does **not** use cached coordinates when an element is resolvable. -- **Stale element on replay** → 1 automatic retry, then hard fail. -- **Minor coordinate drift** → auto-corrected on Desktop Web. - -**Mobile drag steps do not auto-heal.** Manual Interaction drags on Android and iOS rely on the captured drag vector and coordinates from the recording device. Replays on a device with a **different screen resolution or aspect ratio** may fail. To maximize cross-device reliability: - -- Prefer NL drags wherever possible. -- Replay Manual Interaction drags on devices with the **same resolution / form factor** as the recording device. -- Capture the gesture once and reuse it in a Module scoped to a specific device profile. - -## Best Practices - -- **Use NL for static layouts**: Kanban boards, sortable grids, multi-container drags. Fastest authoring path. -- **Use Manual for sliders, confirmation gestures, and dynamic targets.** These require touch-driven capture. -- Prefer elements with stable **accessibility IDs / resource IDs** to maximize cross-device replay success. -- For long lists, **scroll the source element into view** before recording the drag. -- Allow **1–2 seconds after a navigation step** before recording the next drag, gives the page time to stabilize. -- For payment / KYC slider flows, capture the gesture once via Manual and **reuse the step inside a Module**. -- Use `{{variable_name}}` syntax to **parameterize** source / target references for data-driven runs. - -## Example Use Cases - -### E-commerce Product Sorting - -``` -drag product "iPhone 15" from available items to comparison table -``` - -### Project Management Board - -``` -move task card from "To Do" column to "In Progress" column -``` - -### Mobile Slider (Manual Interaction) - -Use Manual Interaction to capture brightness, volume, or price-range sliders on Android and iOS. NL cannot resolve a moving slider thumb. - -### Payment Confirmation Gesture (Manual Interaction) - -Capture a `slide-to-pay` or `swipe-to-unlock` gesture once via Manual Interaction, then reuse the step inside a Module across test cases. - -### Form Builder (Desktop Web) - -Use Manual Interaction to drag form fields from a palette to a canvas, resize input fields, and reorder form sections. - -### Data Visualization Dashboard - -Use Manual Interaction to adjust date-range sliders, resize chart panels, and rearrange dashboard widgets. - -## Limitations - -- **Multi-touch gestures** (two-finger drag, pinch-drag): not supported. -- **Drag path waypoints**: only start and end coordinates are captured; intermediate path points are not preserved. -- **Mobile Browser manual recording**: not supported. Use NL only. -- **Cross-context dragging**: drags between iframes or shadow DOMs are not supported. -- **Multi-element dragging**: cannot drag multiple elements simultaneously. -- **Advanced NL** (e.g., `drag X up by 50px`, offset-based reorder): on the roadmap; currently rejected with a graceful error. -- **Click and Drag via NL**: not supported as a natural language test step; supported only via Manual Interaction. Desktop Web only. -- **Click and Drag speed**: executes slowly by design so the dragged element stays in focus during the movement; faster execution could cause the element to lose focus mid-drag. -- **NL slider authoring**: not supported. Use Manual Interaction. -- **NL confirmation gestures** (slide-to-confirm): not supported. Use Manual Interaction. -- **Drag and drop on canvas-based elements via NL**: canvas elements rely on custom rendering; use Manual Interaction. -- **Editing manual drag steps**: source/target locators and step-level config can be edited; the drag vector and gesture timing are immutable to preserve replay fidelity. - -## FAQs - -**What is the difference between Drag and Drop and Click and Drag?** -Drag and Drop long-presses the element to pick it up before moving it, and works on all platforms. Click and Drag presses and moves the element immediately without a long press, and is available on Desktop Web only. See [Drag and Drop vs Click and Drag](#drag-and-drop-vs-click-and-drag). - -**Why is my Click and Drag step slow?** -This is expected. Click and Drag executes deliberately slowly to keep the dragged element in focus throughout the movement. A faster gesture could cause the element to lose focus mid-drag and fail the step. - -**Can I author a slider drag with natural language?** -No. Sliders have moving targets that NL cannot resolve reliably. Use Manual Interaction to capture the slider gesture directly. The captured step replays at ≥ 95% success rate across devices. - -**Can I author a slide-to-confirm or drag-to-pay step with NL?** -No. Confirmation gestures depend on dynamic UI state and must be captured via Manual Interaction. - -**My drag step passes on the recording device but fails on another device. What's wrong?** -KaneAI replays use element resolution by default. Cross-device failures usually indicate that the source or target element identifier changed across builds. Inspect the step logs to see the resolution path used (element vs. coordinate) and ensure the elements expose stable accessibility IDs. - -**Why isn't manual recording available on Mobile Web?** -The KaneAI agent does not enter Recording state for mobile browser sessions. Use natural language instructions or slash commands instead. - -**Can I edit a captured Manual Interaction drag step?** -You can edit the source / target locators and step-level configuration. The drag vector and gesture timing are immutable for Manual steps to preserve replay fidelity. - -**Does drag work inside a KaneAI Module?** -Yes, drag steps can be saved into Modules and reused across test cases. Module versioning applies as usual. - -**Can I parameterize a drag step?** -Yes. Use `{{variable_name}}` in the source or target reference. Local variables, global variables, smart variables, parameters, and dataset rows are all supported. - -**Does drag work inside if/else and while constructs?** -Yes. Drag steps can be placed inside conditional blocks (`if X is visible then drag Y to Z`) and while loops. Each iteration re-resolves elements at runtime. - -**Is drag and drop supported on real devices?** -Yes, drag works on both real devices and the device cloud for App testing. Real Device Web also supports manual drag. diff --git a/static/docs/kane-ai-free-trial-is-here.md b/static/docs/kane-ai-free-trial-is-here.md deleted file mode 100644 index e2219d134..000000000 --- a/static/docs/kane-ai-free-trial-is-here.md +++ /dev/null @@ -1,55 +0,0 @@ -# KaneAI Free Trial is Here! - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Every great product begins with a vision, but that vision can only become reality when it works flawlessly for real users. Whether you're a solo developer launching your first app, a startup team racing to validate your breakthrough idea, or a growing company looking to scale without breaking things, you face the same fundamental challenge: How do you ensure quality without slowing down innovation? - -Traditional testing approaches often force teams to choose between speed and reliability. Complex tools require months to master. Sophisticated frameworks demand specialized expertise. Enterprise solutions come with enterprise price tags that put them out of reach for teams just getting started. - -With KaneAI's free trial access, every team can access GenAI-Native testing capabilities and ship flawless products. - -[**Start testing with KaneAI today →**](https://www.testmuai.com/kane-ai) - -## Here's What it Has to Offer - -With KaneAI free trial, you get access to: - -- **2 AI Agent Sessions:** Any two users can automate in parallel -- **2 Test Manager Seats:** Collaborate, review, and oversee test projects -- **30-Day Free Trial Access:** Upgrade anytime to unlock more power - -| Feature | Free Trial Access | -|----------|----------| -| AI Test Case Generation | Based on credit availability | -| Author/Edit Sessions | 10 sessions max | -| Max Instructions per Session | 40 instructions max | -| Session Time | 10 minutes per session | -| Parallel Executions | Up to 2 | -| HyperExecute| 300 minutes per Essential license | -| Browser Tabs/Windows | Single-session only (no multi-tab or multi-device use) | -|Frameworks | Selenium-Python (Web), Appium-Python (App) | -| Mobile | Device access restricted (only RD freemium devices available) | -| Idle Time | 2 min timeout per session | -| **KaneAI Browser** | | -| Tunneling | Available | -| Geolocation | [Upgrade Now](https://billing.lambdatest.com/billing/plans) | -| Network throttling | [Upgrade Now](https://billing.lambdatest.com/billing/plans) | -| Smart variables | [Upgrade Now](https://billing.lambdatest.com/billing/plans) | -| **KaneAI Actions** | | -| JavaScript | Available | -| API Call | Available | -| Database Query | Available | -| Module (multi-step) | Available | -| Parameter | [Upgrade Now](https://billing.lambdatest.com/billing/plans) | -| Secret | [Upgrade Now](https://billing.lambdatest.com/billing/plans) | -| TOTP Auth Key | [Upgrade Now](https://billing.lambdatest.com/billing/plans) | -| File Upload | [Upgrade Now](https://billing.lambdatest.com/billing/plans) | -| **KaneAI App** | | -| Android app testing | Available | -| iOS app testing | Available | -| Number apk/ipa upload | Upto 5 | -| Device restriction | Available | -| Geolocation testing | [Upgrade Now](https://billing.lambdatest.com/billing/plans) | -| Network throttling | [Upgrade Now](https://billing.lambdatest.com/billing/plans) | - -[**Start testing with KaneAI today →**](https://www.testmuai.com/kane-ai) diff --git a/static/docs/kane-ai-geolocation-tunnel-proxy.md b/static/docs/kane-ai-geolocation-tunnel-proxy.md deleted file mode 100644 index e1d919a0e..000000000 --- a/static/docs/kane-ai-geolocation-tunnel-proxy.md +++ /dev/null @@ -1,46 +0,0 @@ -# KaneAI - Geolocation, Tunnel and Proxy Support - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Now you can start your web tests on Kane AI using advanced configurations like geolocation, tunnel, and dedicated proxy support. - -## Geolocation Support - -With KaneAI's geolocation feature, you can simulate user interactions from different regions to ensure your website works as expected worldwide. - -### Steps to Use Geolocation - -**Step 1 :** Select the **Geolocation** option from the **Advanced Settings** and choose your desired region. - -**Step 2 :** Run your web test on KaneAI with traffic proxied through the selected region. - -**Step 3 :** Once the test is saved, the generated code will include geolocation details automatically, making it easier to -replicate the test across different regions. - -## Tunnel Support - -KaneAI also supports tunneling to allow you to test websites hosted locally or behind a firewall. - -### Steps to Use Tunnel - -**Step 1 :** Configure the tunnel using the [LT tunnel binary](/support/docs/testing-locally-hosted-pages/#lambdatest-tunnel-guide) with `--env ht-prod` mode enabled. You can use the following command to trigger your tunnel: - -{`./LT --user ${YOUR_LAMBDATEST_USERNAME()} --key ${YOUR_LAMBDATEST_ACCESS_KEY()} --env ht-prod -v -n TUNNEL_NAME`} - -**Step 2 :** Select the tunnel from the **Advanced Settings** on KaneAI and choose the tunnel that you have spun up. - -**Step 3 :** Run your web test on KaneAI from a locally hosted webpage. - -**Step 4 :** Once the test is saved, the generated code will include tunnel details automatically for easy replication. - -## Dedicated Proxy Support (Enterprise only) - -KaneAI offers support for dedicated proxy usage, exclusively available for enterprise accounts with dedicated proxy enabled. - -### Steps to Use Dedicated Proxy - -**Step 1 :** Select the **Dedicated Proxy** option from the **Advanced Settings** and choose the region. - -**Step 2 :** Run your web test on KaneAI with traffic routed through the dedicated proxy IP configured for your organization. - -**Step 3 :** Once the test is saved, the generated code will include proxy details automatically, allowing seamless future executions. diff --git a/static/docs/kane-ai-guided-walkthrough.md b/static/docs/kane-ai-guided-walkthrough.md deleted file mode 100644 index e816971a1..000000000 --- a/static/docs/kane-ai-guided-walkthrough.md +++ /dev/null @@ -1,46 +0,0 @@ -# Guided Walkthrough of KaneAI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This guided walkthrough will help you understand the various components and features available, ensuring you can efficiently navigate and use the platform for managing your automated tests after you have successfully [run your first test](/support/docs/kane-ai-web-test/). - -> Navigate to the [Test Manager](https://www.testmuai.com/login/?redirectTo=https://test-manager.lambdatest.com/projects) and [select your project and folder](/support/docs/kane-ai-web-test/#step-3-save-your-test-case) into which you have saved your tests to manage your tests. - -## Projects Page -All the Test Cases generated are accessible from the Test Manager page in the Test Cases section. Each of the Test Case has a unique `Test Case ID` and `Labels` to uniquely identify them. The top section displays the project name, number of Test Cases in that particular project folder, Test Plans, Builds and its creation details. - -On this page you have other things such as **filters**, **adding test cases**, **importing csv files** and much more. You can also filter through all of the test cases on the basis of the status, creator, and tags. - -Click on the three dots (...) of your desired Test Case to Edit or Delete that particular Test Case. - -## Understanding Test Case Details -The test case dashboard provides a comprehensive view of a specific test case, including its summary, code, runs, issues, and version history. - -### Summary - -- **Project:** The name of the project to which the test case belongs. -- **Creator:** The person who created the test case. -- **Created:** When was the test case created. -- **Description:** A brief overview of the test case's purpose and steps. -- **Steps Performed:** List of all the steps performed during that session. -- **HyperExecute:** This button will trigger your steps to the [HyperExecute](/support/docs/getting-started-with-hyperexecute/). -- **Edit Test Steps:** This button will open the playground where you can edit your test cases. - -### Code - -The code page within the test manager provides a comprehensive view of the code used to execute a specific test case. This page typically includes the language and the framework used to create the test case code. - -- **Generate New Code:** If you want to generate another codebase in different language or framework for the same test case. -- **HyperExecute:** This button will trigger your steps to the HyperExecute. -- **View Code:** This opens the code in a built-in separate editor from where you can edit your code files and Download it as well. -- **Download:** To download the entire test case code files. - -### Runs -It shows the history of the test executed with meta data like time for the test completion, who has executed the test, browser and the Operating System. - -### Issues -The number of reported issues or defects during test execution. - -### Version History - -This provides a detailed overview of the changes made to a test case over time. This functionality allows you to track the evolution of your test cases, identify the root causes of issues, and revert to specific versions if necessary. diff --git a/static/docs/kane-ai-javascript-execution.md b/static/docs/kane-ai-javascript-execution.md deleted file mode 100644 index a6eff5ca8..000000000 --- a/static/docs/kane-ai-javascript-execution.md +++ /dev/null @@ -1,74 +0,0 @@ -# KaneAI - JavaScript Execution - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -With KaneAI’s new JavaScript Execution feature, you now have the flexibility to write or paste custom JavaScript code snippets to perform tests. This allows for more advanced and customized test scenarios alongside KaneAI’s plain English test-writing format. - -This can be very useful for: -- **Advanced Interactions:** If you need more granular control over interactions that can’t be expressed in plain English. -- **Complex Test Scenarios:** For scenarios that involve manipulating the DOM, testing specific elements, or interacting with APIs directly. -- **Flexibility:** Switch between natural language and JavaScript to suit your testing needs. -- **Precision:** Perform tests that require more detailed code logic. -- **Seamless Execution:** Test JavaScript code directly without any setup. - -## How to Execute Custom JavaScript Tests -### Step 1: Click on the "Write a Step" Field -In KaneAI’s test editor, navigate to the "Write a Step" field where you typically write your test scenarios in plain English. - -### Step 2: Trigger JavaScript Mode -To switch to JavaScript execution mode, type the forward slash `/` or click on the `+` icon. You will see a JavaScript Pop-up Option appear. Click on the JavaScript option to switch the input mode to custom JavaScript. - -### Step 3: Write or Paste Your JavaScript Code -Now, you can write or paste your custom JavaScript code snippet directly into the text field. A sample JavaScript code: - -```javascript -function simulateTyping(element, text) { -element.value = ''; // Clear any existing value - -for (let i = 0; i < text.length; i++) { -let char = text[i]; -let keyDownEvent = new KeyboardEvent('keydown', {key: char}); -element.dispatchEvent(keyDownEvent); -element.value += char; - -let inputEvent = new Event('input', {bubbles: true}); -element.dispatchEvent(inputEvent); -let keyUpEvent = new KeyboardEvent('keyup', {key: char}); -element.dispatchEvent(keyUpEvent); -} - -let changeEvent = new Event('change', {bubbles: true}); -element.dispatchEvent(changeEvent); -} - -// Example usage -let inputElement1 = document.querySelector('[aria-label="First Name"]'); -let inputElement2 = document.querySelector('[aria-label="Last Name"]'); -let inputElement3= document.querySelector('[aria-label="Email"]'); -simulateTyping(inputElement1, 'Aman'); -simulateTyping(inputElement2, 'Chopra'); -simulateTyping(inputElement3, 'amanc@lambdatest.com'); - -return {"First name": "Aman", "Last name": "Chopra", "email": "amanc@lambdatest.com"}; -``` - -The provided code snippet simulates typing into text fields programmatically by dispatching key-related events such as keydown, input, keyup, and change. It mimics real user interaction by typing each character into the field and updating its value accordingly. The example provided fills in the first name, last name, and email fields with specific values and returns them in an object format. - -### Step 4: Execute the Test -Once your code is ready, simply press Enter. KaneAI will run your custom JavaScript snippet in the current testing environment. - -#### Handling Return Values and Variable Usage -If your JavaScript snippet contains return values, KaneAI will automatically save these values in a variable that can be used later in the test. The output from your JavaScript code will be captured and assigned to a variable. This variable will be available for use throughout your test execution. - -For example, in the code snippet provided: - -```javascript -return {"First name": "Aman", "Last name": "Chopra", "email": "amanc@lambdatest.com"}; -``` - -The returned object with the keys `First name`, `Last name`, and `email` will be stored in a variable. You can reference this variable later in your test to validate or reuse the values. - -> Variables created from return values can be used in subsequent test steps. To use the variable, simply refer to the specific property (e.g., First name) in your test steps for verification or further actions. - -#### JavaScript Snippets in Generated Code -When your test is executed, the JavaScript code snippet you provided will be added directly into the generated test code. This ensures that any custom interactions performed with JavaScript are part of the overall test logic. The inclusion of JavaScript in the generated code allows for easy tracking, debugging, and modification if needed. diff --git a/static/docs/kane-ai-jira-integration.md b/static/docs/kane-ai-jira-integration.md deleted file mode 100644 index b581e9410..000000000 --- a/static/docs/kane-ai-jira-integration.md +++ /dev/null @@ -1,48 +0,0 @@ -# KaneAI - Jira Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The "Jira integration" feature allows you to generate test cases from your Jira tickets automatically. This feature leverages AI to analyze the content of Jira tickets and transform written test case descriptions into executable test cases on the TestMu AI platform. This documentation provides a detailed guide to help you understand and utilize this feature effectively. - -## Key Benefits of the KaneAI Jira Integration: -- **Automated Test Creation :** Quickly transform test case descriptions in Jira into executable test cases without manual setup. -- **Efficient QA Workflow :** Simplify testing processes by managing test creation directly from Jira, saving time and reducing errors. -- **Reduced Bug Risk :** AI Native test creation helps reduce the likelihood of bugs by ensuring all outlined test cases are covered and refined. - -## Step-by-Step Guide -### Step 1: Install TestMu AI Application for Jira -Visit the Atlassian Marketplace, locate the **TestMu AI Cloud** application for Jira and install app on your Jira instance. - -Currently in the **Beta**. Please reach out to our support team to enable the KaneAI for your organization. - -### Step 2: Define Test Cases in a Jira Ticket -- **Create or Open a Jira Ticket :** Start by creating a Jira ticket or accessing an existing ticket where you want to outline test cases. -- **Write Test Case Details :** In the comments section, add detailed information about the test cases you want to execute. Clearly describe each test case's steps, conditions, expected outcomes, and any other relevant information to help KaneAI generate accurate tests. - -### Step 3: Generate Test Cases Using KaneAI -To trigger KaneAI, add the comment to your Jira ticket calling TestMu AI application, for example: - -```bash -@TestMu AI Cloud create test case based on the details of this Jira -``` - -> **Note :** This comment signals KaneAI to analyze the Jira ticket content and automatically create test cases. - -#### What Context is Used for Test Case Generation? - -KaneAI uses the following fields from the Jira ticket as context to generate test cases: - -- **Summary** and **Description** of the ticket -- **Comments** on the ticket -- **Textual custom fields**: You can configure which custom fields are included from the TestMu AI Cloud app settings in Jira. Only text-based custom fields are supported; non-textual fields (e.g., dropdowns, number fields, user pickers, linked tickets) are not used for generation. - -To configure which custom fields are used as context, go to the TestMu AI Cloud app settings in your Jira instance and select the desired textual custom fields. - -After you post the comment, KaneAI will process the request and respond with a confirmation message. The response includes a link to TestMu AI where you can review, modify, and schedule the generated test case. - -### Step 4: Review and Refine the Generated Test Cases -Click on the link provided by KaneAI in the Jira comment to open the [AI Test Case Generator](/support/docs/generate-test-cases-with-ai/). The AI will begin [analyzing your Jira ticket content and generating test cases](/support/docs/generate-test-cases-with-ai/#step-3-generate-test-cases) in real time. You can watch scenarios and test cases stream in as they are created. - -Once the generation is complete, review the test cases grouped across scenarios. Each scenario represents a theme or functional area and is labeled with priority tags such as **Must have**, **Should have**, and **Could have**. Individual test cases are categorized as **Positive**, **Negative**, or **Edge** to indicate their test type. - -You can further refine the generated test cases using the [Conversation Layer](/support/docs/generate-test-cases-with-ai/#conversation-layer-refine-your-test-cases): describe changes in natural language and the AI applies them in real time. Once satisfied, save the test cases to your Test Manager repository or automate them with KaneAI. diff --git a/static/docs/kane-ai-mobile-app-capabilities.md b/static/docs/kane-ai-mobile-app-capabilities.md deleted file mode 100644 index 8978d3e85..000000000 --- a/static/docs/kane-ai-mobile-app-capabilities.md +++ /dev/null @@ -1,41 +0,0 @@ -# Mobile App advanced authoring capabilities - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -> This document provides detailed instructions for utilizing advanced features in KaneAI's mobile app authoring. It covers enabling video and image injections, configuring biometrics, and other advanced settings to enhance test authoring. - -## Introduction - -KaneAI's mobile app authoring support various advanced capabilities for authoring mobile app tests. Users can navigate to advanced settings to enable options such as disabling screenshot block, enabling biometric authentication, image injection, and video injection for creating tests within KaneAI. - -## Disable Screenshot Block - -If your app doesn't allow screenshot to be taken of the app and you want to test your app's performance without any hindrance, you can use this capability. It will disable the screenshot block check added in the application to allow authoring test cases in TestMu AI. - -You need to ensure to enable it from advanced settings before authoring your test on KaneAI. You can find more details about this capability [here](/support/docs/disable-screenshot-block/). - -## Biometrics Authentication - -Biometric Authentication is a security process that leverages the unique biological characteristics of individuals to verify their identities. In native applications, it is often used to provide a seamless, secure, and efficient user experience. This capability allows you to test your application's biometric authentication functionality more accurately and ensures your app provides an optimal user experience across a range of devices and platforms. - -You can find more details about this capability [here](/support/docs/biometric-authentication/). - -## Image Injection - -Camera Image Injection feature allows you to author tests with **image capturing, QR code scanning**, and **barcode scanning** functionalities in your app across **real devices** on the TestMu AI. - -You need to ensure to enable it from advanced settings before authoring your test on KaneAI. Once enabled, you will get an option to inject image in "slash" commands in-session which needs to be used before you select the option to trigger camera API in your application workflow. - -You can find more details about this capability [here](/support/docs/camera-image-injection/). - -## Video Injection - -Video Injection feature enables you to test video-based functionalities such as **live streaming, real-time video capture,** and **video processing** in your app across **real devices** on the TestMu AI. - -You need to ensure to enable it from advanced settings before authoring your test on KaneAI. Once enabled, you will get an option to inject video in "slash" commands in-session which needs to be used before you select the option to trigger camera API in your application workflow. - -You can find more details about this capability [here](/support/docs/video-injection/). - -## Video Explanation - -🎥 **[Watch how mobile app advanced capabilities can be leveraged in KaneAI authoring](https://app.trupeer.ai/view/SZxMvBbih)** diff --git a/static/docs/kane-ai-modules.md b/static/docs/kane-ai-modules.md deleted file mode 100644 index f4ddb1775..000000000 --- a/static/docs/kane-ai-modules.md +++ /dev/null @@ -1,135 +0,0 @@ -# Modules - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Modules in KaneAI let you group a sequence of test steps into a single reusable unit. Instead of re-authoring the same steps in every test case, you create a module once and import it wherever it is needed. This reduces duplication, improves consistency, and makes maintenance easier. When a workflow changes, you update the module and propagate the change across test cases. - -Modules support **variables**, **parameters**, and **secrets**, so you can pass dynamic data into each module execution rather than hard-coding values. - -## Module Listing Page - -The **Modules** section shows all available modules in your project. From here you can: - -- Browse existing modules and see how many test cases each module is linked to -- Click on any module to view its details, version history, and linked test cases -- Create a new module - -## Create a Module - -You create modules by extracting test steps from an active KaneAI session. - -### Step 1: Author Test Steps - -Begin by authoring your test in KaneAI and executing the steps that you want to turn into a module. - -### Step 2: Select Test Steps - -1. Click the **Pause Test** button. -2. Select the test steps you want to include in the module. For example, highlight the steps that handle login functionality. - -### Step 3: Define Module Properties - -1. Click **Create Module**. -2. Enter the module details: - - **Name**: A descriptive name (e.g., `[Web] Login Flow`) - - **Description**: What the module does and when to use it - - **Tags**: Keywords for easier discovery -3. Click **Create Module** to save. - -The selected steps are now saved as a reusable module with an initial version. - -## Use a Module in a Test - -Once a module exists, you can import it into any KaneAI test. - -### Step 1: Open a Test - -Create a new test or open an existing one in the KaneAI web agent. - -### Step 2: Import the Module - -Use either method to access the modules menu: - -- Click the **plus (+)** icon in the test interface -- Type a **slash (/)** to invoke the command menu and select **Add Module** - -Browse the list of available modules and select the one you need. - -### Step 3: Execute - -Click **Add in Queue** to execute the module's test steps. The steps run automatically as part of the test. - -## Edit a Module - -Click on an existing module from the Module listing page to modify its steps or properties. - -When you save changes to a module, a new version is created automatically. See [Versioning and Enhancements](/support/docs/kaneai-modules-versions-and-enhancement/) for details on how version history works. - -## Delete a Module - -You can delete modules that are no longer needed from the module listing page. - -1. Navigate to the **Modules** section. -2. Locate the module you want to delete. -3. Click the meatball menu (**...**) icon on the module card. -4. Select **Delete**. - -**Important** -Only modules that are **not linked to any test cases** can be deleted. If a module is currently used in one or more test cases, the delete option will be disabled. You must first remove the module from all linked test cases before deleting it. - -## Best Practices - -### Create Platform-Specific Modules - -Modules in KaneAI are **platform-specific**. Each platform has distinct interaction patterns, element selectors, and navigation flows, which means the same user action (e.g., tapping vs. clicking, scrolling behavior, keyboard input) behaves differently across platforms. A module authored on one platform may produce unexpected results when used on another. - -**Create separate module copies for each target platform.** For example, if you have a login workflow that needs to run across platforms, create a distinct module for each: - -| Module Name | Platform | -|---|---| -| `[Web] Login Flow` | Desktop Web | -| `[Android-App] Login Flow` | Android App | -| `[Android-Browser] Login Flow` | Android Browser | -| `[iOS-App] Login Flow` | iOS App | -| `[iOS-Browser] Login Flow` | iOS Browser | - -### Use Prefixes and Tags Together - -Adopt a consistent **name prefix** and **tag** strategy to make platform context immediately clear and modules easy to filter. - -**Name prefixes**: Add a platform prefix at the start of every module name: - -- **`[Web]`**: Desktop Web modules -- **`[Android-App]`**: Android native app modules -- **`[Android-Browser]`**: Android browser modules -- **`[iOS-App]`**: iOS native app modules -- **`[iOS-Browser]`**: iOS browser modules - -**Tags**: Add platform tags to each module so you can filter the module listing by platform: - -- `desktop-web`, `android-app`, `android-browser`, `ios-app`, `ios-browser` - -Using both prefixes and tags together gives you the best of both worlds: prefixes make the platform obvious at a glance when importing a module into a test, while tags let you filter the module listing page to show only modules for a specific platform. - -You can extend this convention for other distinctions: - -- **`[Web][Staging]`**: Environment-specific modules -- **`[Android-App][Auth]`**: Feature-area grouping -- Tags like `login`, `checkout`, `onboarding`: Workflow-based categorization - -### Keep Modules Focused - -Each module should represent a **single, cohesive workflow** (e.g., login, checkout, form submission). Avoid creating overly large modules that combine unrelated steps. They become harder to maintain and less reusable. - -### Use Variables and Parameters - -Instead of hard-coding values like usernames, URLs, or test data into module steps, use **variables** and **parameters**. This lets you reuse the same module across different test scenarios by passing in different data at execution time. - -### Write Clear Descriptions - -Add a meaningful description to every module so other team members understand what it does without reading every step. Include the target platform, expected preconditions, and what state the application will be in after the module runs. - -## Related Guides - -- [Versioning and Enhancements](/support/docs/kaneai-modules-versions-and-enhancement/): Track changes, compare versions, and revert modules -- [Bulk Module Update](/support/docs/kaneai-bulk-module-update/): Update a module version across multiple test cases in one action diff --git a/static/docs/kane-ai-network-assertions.md b/static/docs/kane-ai-network-assertions.md deleted file mode 100644 index f1c7dcbc0..000000000 --- a/static/docs/kane-ai-network-assertions.md +++ /dev/null @@ -1,28 +0,0 @@ -# Network Logs Assertions - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The **Network Logs Assertions** feature in KaneAI enables you to test and validate the behavior of your application at the **network layer**. -This is useful when you need to ensure that API calls, response codes, payloads, or request timings meet expected conditions during your test executions. - -# Let's see how to add Network Logs Assertions in a Test - -- **Start a test session** (Desktop Web, Mobile Web, or Mobile App). - -- For **Mobile Web** and **Mobile App** sessions, you need to enable the **Capture Network Logs** toggle before starting your session. Navigate to **Advanced Settings** > **Session Settings** and turn on the **Capture Network Logs** option. - -- All network calls are automatically recorded in the background as you navigate through your application. You can see the Network calls made so far through the **Assert Network Logs** option available in the " / " commands. - -- Here you will find all the Network requests made in this session so far and **having the Json data format**, you can filter the requests on basis of **Status Code** or **HTTP Methods**: - -- Select the specific key's that you want to add in the assertion from the **Request Headers**. - -- To add **Request** or **Response blocks** in assertions click on the `Add Assertion` button. All the chosen values will be validated in a single step. - -- Once added, the request will be stored as a **variable**. KaneAI then makes assertions based on this variable. - -#### How to validate a specific object in request or response of an API? - -- You can reuse the request later in your test using the familiar `{{variable}}` syntax. - -- The stored variable is in **JSON format**, so you can easily access nested values with dot-notation (e.g., `header.status`). diff --git a/static/docs/kane-ai-scroll-in-feature.md b/static/docs/kane-ai-scroll-in-feature.md deleted file mode 100644 index a20b8cc66..000000000 --- a/static/docs/kane-ai-scroll-in-feature.md +++ /dev/null @@ -1,30 +0,0 @@ -# KaneAI - Scroll in Element - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The "Scroll in Element" feature in Kane AI allows users to perform scrolling actions on web elements during testing by using natural language commands. With this feature, you can easily navigate through elements like sidebars, lists, or any scrollable content on a web page. - -## Supported Commands -You can pass the following commands in English to scroll within the web test: - -- **"Scroll left" :** Moves the content horizontally to the left. -- **"Scroll right" :** Moves the content horizontally to the right. -- **"Scroll up" :** Moves the content vertically upwards. -- **"Scroll down" :** Moves the content vertically downwards. - -## Example Use Cases - -- **Scroll a Sidebar:** If a web page has a left or right sidebar, you can command Kane AI to scroll within the sidebar to access hidden elements. -- **Navigate Lists:** Use scrolling commands to move through long lists or dropdown menus. -- **Access Off-screen Content:** When elements are outside the viewport, scrolling commands help bring them into view. - -## How to Use - -**Step 1:** Identify the element you want to scroll. - -**Step 2:** Use the appropriate command based on the direction you need to scroll. For example: - -- "Scroll right in the sidebar" to move right in a horizontal sidebar. -- "Scroll down" if you want to scroll downwards on a long page. - -**Step 3:** Verify the content is in view after the scrolling action. diff --git a/static/docs/kane-ai-secrets.md b/static/docs/kane-ai-secrets.md deleted file mode 100644 index 2563d925e..000000000 --- a/static/docs/kane-ai-secrets.md +++ /dev/null @@ -1,57 +0,0 @@ -# Utilizing KaneAI Secrets Feature: A Step-by-Step Guide - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -KaneAI now supports the use of secret variables to enhance security and streamline functionality. This feature leverages HashiCorp Vault for secure management of sensitive information, ensuring robust protection for your data while maintaining ease of use. This guide outlines the process for creating, managing, and utilizing secret variables within KaneAI. - -- In Web/App Agent, users will see list of all secrets created at user or org level for that user. -- A user role can create only user level secrets, while an admin can create either user-level or org-level secrets. -- A user can delete their own secrets, but only an admin can delete org-level secrets. -- Secret variables will be accessible in test steps, modules, and test cases. - -## How to Create Secrets and use them in your tests? -You can create Secrets variables via: -- Test Manager -- Web/App Agent - -### Create Secrets via Test Manager -**Step 1:** Navigate to the Test Manager. -**Step 2:** Click on the settings icon > Secrets. -**Step 3:** Click on the **Create new secret** button. -**Step 4:** Provide the secret key and value and click on the **Create Secret** button. - -### Create Secrets via Web/App Agent -**Step 1:** Navigate to the **"Write a Step"** field where you typically write your test scenarios in plain English. -**Step 2:** To switch to Secrets mode, type the forward slash / or click on the + icon. You will see a Secret pop-up option appear. -**Step 3:** Provide the secret key and value and click on the **Create Secret** button. - -## How to use your Secrets? -Once created, you can now reference any secret variables in your test steps using **double-curly braces syntax (e.g.,`{{email}}`)**. - -> To use a previously created variable in any subsequent step or test case, reference the variable using the same `{{` syntax. This will list down all existing variables for this test. - -Format: -```bash -use the value of {{secrets.user.email}} in the email text field -``` - -## How to Update your Secrets? -**Step 1:** Navigate to the Test Manager. -**Step 2:** Click on the settings icon > Secrets. -**Step 3:** Click on the **pencil** icon to update that respective secret value. -**Step 4:** Provide the secret value and click on the **Update secret** button. - -## How to Delete your Secrets? -**Step 1:** Navigate to the Test Manager. -**Step 2:** Click on the settings icon > Secrets. -**Step 3:** Click on the **delete** icon to update that respective secret value. -**Step 4:** Provide the secret value and click on the confirmation button. - -# Data Masking & Anonymization - -TestMu AI protects sensitive data in tests through synthetic substitution and capture-time masking. - -- **AI Autofill (synthetic substitution)** - generates realistic substitute values against your real dataset structure, so production values are replaced rather than encoded. No mapping between original and substituted values is retained, removing the re-identification vector. -- **Secret variables (HashiCorp Vault)** - values marked as secret are resolved at runtime from HashiCorp Vault and are never written to action logs, session logs, screenshots (known secret values are redacted from screenshot OCR before upload), video recordings, run summaries, or reports. -- **Lambda Masking** - hides sensitive data in recorded test commands and session recordings. -- **SmartUI PII masking** - masks PII regions in visual snapshots. Masking is enforced at capture time, not applied post-hoc, so masked data is never persisted in artifacts. diff --git a/static/docs/kane-ai-smart-variables.md b/static/docs/kane-ai-smart-variables.md deleted file mode 100644 index 99872f6f6..000000000 --- a/static/docs/kane-ai-smart-variables.md +++ /dev/null @@ -1,79 +0,0 @@ -# Utilizing KaneAI Smart Variables Feature: A Step-by-Step Guide - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -KaneAI's Smart Variables feature enhances test automation by providing predefined, dynamic variables that can be used across test steps, modules, and test cases. These variables enable dynamic adaptation to various environments and conditions, improving test reliability, reusability, and efficiency. - -They cover categories such as date & time, system information, location, connectivity, randomization, and device-specific details, allowing users to create flexible and robust test scenarios that adapt to real-time conditions without manual input. - - Smart variables are predefined and cannot be created or deleted by users. They are accessible in test steps, modules, and test cases. - When typing `{{` in the Web/App Agent, an overlay displays all available variables, including regular, secret, and smart variables, with smart variables labeled as "Smart" and accompanied by descriptions. - -Smart Variables are listed under the variables section within an authoring session. - -## How to Use Smart Variables in Your Tests? - -Smart variables can be referenced in test steps, modules, and test cases using the `{{smart.variable_name}}` syntax. Follow these steps to integrate them into your tests: - -### Step 1: Access the Test Environment - -Enter the Authoring or Editing mode for a KaneAI test case - -### Step 2: Reference a Smart Variable - -In the "Write a Step" field type `{{` to trigger the variable overlay. This will display a list of all available variables, including smart variables labeled with "Smart". - -For example, to use the current date in a test step: - -Use `{{` syntax to get the list of smart variables listed and type to search and filter them out then select the desired one. -``` -Enter the value of {{smart.current_date}} in the date field -``` - -### Step 3: Select a Smart Variable - -From the overlay, select the desired smart variable. The variable will be inserted into the test step in the correct syntax (e.g., `{{smart.current_date}}`). - -### Step 4: Run Your Test - -Once the test step is executed the smart variable will dynamically resolve to its corresponding value based on the current environment or condition (e.g., `2025-04-29` for `{{smart.current_date}}`). - -## Supported Smart Variables - -Below is a comprehensive list of supported smart variables, organized by category, along with their syntax and descriptions: - -| Category | Variable Name | Syntax | Description | -|-----------------------|-----------------------|-------------------------------|-----------------------------------------------------------------------------| -| **Date & Time** | `current_date` | `{{smart.current_date}}` | Current date in YYYY-MM-DD format | -| | `current_day` | `{{smart.current_day}}` | Current day of the week (Monday, Tuesday, etc.) | -| | `current_month` | `{{smart.current_month}}` | Current month name (January, February, etc.) | -| | `current_month_number`| `{{smart.current_month_number}}`| Current month number (01, 02, ..., 12) | -| | `current_year` | `{{smart.current_year}}` | Current year (YYYY) | -| | `current_hour` | `{{smart.current_hour}}` | Current hour in 24-hour format (00 to 23) | -| | `current_minute` | `{{smart.current_minute}}` | Current minute (00 to 59) | -| | `current_timestamp` | `{{smart.current_timestamp}}` | Current timestamp in epoch milliseconds | -| | `current_timezone` | `{{smart.current_timezone}}` | Current timezone name (e.g., GMT, UTC+5:30) | -| | `next_day` | `{{smart.next_day}}` | Date of the next day in YYYY-MM-DD format | -| | `previous_day` | `{{smart.previous_day}}` | Date of the previous day in YYYY-MM-DD format | -| | `start_of_week` | `{{smart.start_of_week}}` | Date of the first day of the current week (YYYY-MM-DD) | -| | `end_of_week` | `{{smart.end_of_week}}` | Date of the last day of the current week (YYYY-MM-DD) | -| | `start_of_month` | `{{smart.start_of_month}}` | Date of the first day of the current month (YYYY-MM-DD) | -| | `end_of_month` | `{{smart.end_of_month}}` | Date of the last day of the current month (YYYY-MM-DD) | -| **User/System Information** | `os_type` | `{{smart.os_type}}` | Operating system name on which test is running (e.g., Windows, macOS, Linux)| -| | `os_version` | `{{smart.os_version}}` | Operating system version on which test is running (e.g., 10.0, 14.2) | -| | `browser_name` | `{{smart.browser_name}}` | Browser name on which test is running (e.g., Chrome, Firefox, Safari) | -| | `browser_version` | `{{smart.browser_version}}` | Browser version on which test is running (e.g., 112.0, 101.0.1) | -| | `user_name` | `{{smart.user_name}}` | Current user name on TestMu AI who trigger the test | -| **Randomized** | `random_int` | `{{smart.random_int}}` | Random 3 digit integer | -| | `random_float` | `{{smart.random_float}}` | Random 2 digit float | -| | `random_string_8` | `{{smart.random_string_8}}` | Random alphanumeric string of specified length | -| | `random_string_56` | `{{smart.random_string_56}}` | Random alphanumeric string of specified length | -| | `random_email` | `{{smart.random_email}}` | Random email address | -| | `random_phone` | `{{smart.random_phone}}` | Random phone number | -| **Device-Specific** | `device_name` | `{{smart.device_name}}` | Name of the mobile device under test | -| | `device_model` | `{{smart.device_model}}` | Model of the mobile device | -| | `device_os` | `{{smart.device_os}}` | Operating system of the mobile device (iOS, Android) | -| | `device_os_version` | `{{smart.device_os_version}}` | Version of the operating system (14.4, 11.0) | -| | `device_orientation` | `{{smart.device_orientation}}`| Orientation of the device (Portrait, Landscape) | -| | `app_version` | `{{smart.app_version}}` | Version of the app under test | -| | `app_package_name` | `{{smart.app_package_name}}` | Package name of the app under test | diff --git a/static/docs/kane-ai-test-plan.md b/static/docs/kane-ai-test-plan.md deleted file mode 100644 index 43fc4b7c9..000000000 --- a/static/docs/kane-ai-test-plan.md +++ /dev/null @@ -1,43 +0,0 @@ -# Intelligent Test Planner - Automate your Test Generation - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The Intelligent Test Planner feature in KaneAI empowers you by simplifying the test generation process. Instead of manually writing test steps or scenarios, you can now provide high-level objectives, and KaneAI will automatically translate them into executable test scripts. - -This document will guide you through the end-to-end implementation for your web applications. - -## Steps to generate and execute your test cases -### Step 1: Define your test objective -You have to clearly state your goal or define the functionality that you want to test. This could be anything from validating a login process to ensuring the functionality of a shopping cart. The Intelligent Test Planner uses this objective as the basis for automatically generating the necessary test steps. - -#### Examples: -- `Assess the auto-suggestions functionality in the search bar of the "https://www.testmuai.com/support/docs"` -- `Verify user login functionality for "https://www.example-ecommerce.com/login"` -- `Verify the add-to-cart functionality on the "https://www.example.com"` -- `Evaluate the user signup process for a new user on https://www.examplesocial.com` -- `Track the add-to-wishlist functionality for a product on "https://www.example.com"` - -Once your objective is defined, press enter and KaneAI will start analyzing the required actions to achieve the desired outcome. - -### Step 2: Define your State, Outcomes, Error Messages, Constraints, and User Inputs - -- **Outcomes :** Mention the desired outcomes or behaviors that you expect to observe after performing a test. For example, if testing a login functionality, the expected outcome might be that the user is successfully logged in and redirected to the dashboard. -- **Error Message :** These are the error messages or notifications that you expect to see under certain conditions. For example, if a user enters invalid login credentials, an error message might indicate that the credentials are incorrect. -- **Constraints :** These are the conditions or limitations under which the test is conducted. For example, you might constrain the testing to specific browsers, devices, or operating systems. -- **User Inputs :** This refers to any information or data that the user needs to provide during the test. For example, in a login test, the user input would be the username and password. - -Now click on the **Review and Scheule test** button. - -### Step 3: Review and Schedule your test -In the left hand side, you can review your test data and verify if something needs to be updated here. - -In the right side, you willget an option to select your Project, Folder in which you want to save your test. You can also select the Type of test and the status of it. You can also assign label to your test for unique identification of it. - -Click on **Schedule Test** button. - -### Step 4: Check the test execution -Click on the Live Preview button to enter the playground and check the live execution of your test based on your defined objective. - -You will be redirected to playground where your tests will be getting executed and you can check the complete process. - -> You can now go to the **Test Manager** where you have saved your test case and trigger it on HyperExecute. diff --git a/static/docs/kane-ai-using-datasets.md b/static/docs/kane-ai-using-datasets.md deleted file mode 100644 index 545ef12dc..000000000 --- a/static/docs/kane-ai-using-datasets.md +++ /dev/null @@ -1,37 +0,0 @@ -# KaneAI - Using Datasets on Test Manager - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Datasets in KaneAI-TestMu AI allow you to manage and reuse input values for test cases efficiently. This guide explains how to create, edit, and manage these data sets while utilizing parameters to run test cases with a variety of inputs, improving flexibility and reducing manual data entry. - -## Accessing Datasets within a Project -Navigate to the Parameters section within your project and review existing data sets under this section. -> **Note :** Parameters act as placeholders for data, enabling test cases to use varying inputs. - -Every test case which uses parameters in KaneAI automatically generates a default data set during authoring. Example: A default data set might capture initial values like form submissions. -> Default Datasets are immutable in nature i.e. they cannot be modified. - -## Creating and Editing Datasets -- To create a dataset, navigate to the Datasets section, click "Create Dataset," and define parameters and values as needed. -- For existing datasets, access the version history of a default dataset, create a copy, and edit the copied dataset to update or add values seamlessly. - -## Adding Data to Datasets -Open the editable Dataset. Click on "**Add a Row After**". Enter your desired values (e.g., 5 or Mike). - -### Autofill with AI - -Alternatively, you can use the **Autofill with AI** option to automatically populate data fields. AI generates values based on the parameter's name, saving time and effort. - -### Importing Data via CSV Files -Prepare your data set in CSV format. Use the Import Data feature in the data set editor and verify imported data and save the test case to ensure synchronization. - -## Utilizing Version History -- **Version Tracking :** All changes to data sets are recorded. -- **Revert or Restore :** Easily revert to a previous version if needed. -- **Backup Assurance :** Ensure data integrity with a maintained history of edits. - -## Executing Test Cases with Parameters -Combine multiple data sets with their respective parameters. -Execute test cases using these data sets to simulate diverse input scenarios. - -## Video Explanation diff --git a/static/docs/kane-ai-using-json-variables.md b/static/docs/kane-ai-using-json-variables.md deleted file mode 100644 index df1d46846..000000000 --- a/static/docs/kane-ai-using-json-variables.md +++ /dev/null @@ -1,55 +0,0 @@ -# KaneAI - Utilizing JSON Variables for Assertions - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This document outlines the process of using JSON variables in KaneAI to assert specific objects within an API response. The guide provides a step-by-step procedure for executing API calls, examining responses, and performing assertions on both status and body of the response. - -## Prerequisites -- Access to KaneAI platform -- Basic understanding of API testing -- Familiarity with JSON structures - -## Step 1: Understanding JSON Variable Assertions -In KaneAI, you can use JSON variables to perform detailed assertions on API responses. This allows you to validate specific objects and elements within the response structure. - -## Step 2: Initiating an API Call -- Navigate to your desired API endpoint (e.g., a pet store website) -- Configure the necessary API call parameters -- Execute the API request - -### Example Workflow: -- Select the appropriate HTTP method (GET, POST, etc.) -- Add required headers -- Input any necessary request body -- Send the request - -## Step 3: Accessing Response Variables -After executing the API call, KaneAI automatically generates variables containing: -- Response status code -- Response body -- Response headers -- Other relevant metadata - -## Step 4: Asserting Response Status -To assert the response status: - -- Use double curly braces `{{` to access the variable list -- Navigate using arrow keys to select the status variable -- Define your expected status code - -### Example: -- Expected status: 200 -- If actual status is 500, the assertion will fail - -## Step 5: Asserting Response Body -Navigate through the response body using the same double curly brace method: -- Open the variable selection menu with `{{` -- Browse to the specific JSON element you want to validate -- Create an assertion for that element - -### Example Validation: -- Check if 'dogs' element exists in the response -- Verify specific property values -- Validate nested JSON structures - -## Video Explanation diff --git a/static/docs/kane-ai-using-parameters.md b/static/docs/kane-ai-using-parameters.md deleted file mode 100644 index 9a6ebd118..000000000 --- a/static/docs/kane-ai-using-parameters.md +++ /dev/null @@ -1,67 +0,0 @@ -# KaneAI - Using Parameters - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This guide provides a detailed walkthrough for utilizing parameters in KaneAI to streamline test case management and execute tests with various data inputs. By leveraging data parameters, testers and developers can manage configurations effortlessly. Follow the step-by-step instructions below for successful implementation. - -## Key Features of Parameters in KaneAI -- **Dynamic Data Input :** Use parameters to define and input test data dynamically during test execution. -- **Multiple Configurations :** Simplify the execution of tests with different data sets. -- **Seamless Test Management :** Centralized management of parameters improves efficiency and reduces redundancy. - -## Define Parameters in a Web or App Agent Test -- Navigate to the specific web agent test URL and configure the test to accept input parameters. - -## Create a parameter in a test -- Confirm that parameters are already defined in the system. To add a new parameter, use one of the following methods to define the parameter: - - Type a **`slash (/)`** and select the Parameter option. - - Click the Plus icon and choose the Parameter icon. - -Click **Create Parameter** to save. - -## Reference the parameter in test step -Reference any parameter using **dollar-curly braces syntax** (e.g.,`${phone_number}`). Enter the name in its designated input field. - -## From a Manual Interaction step - -You can also convert a hard-coded value in a Manual Interaction step into a Parameter directly, without deleting or re-recording the step. This flow is available on both **Desktop** and **Mobile App** Manual Interaction sessions. - -1. Open the recorded Manual Interaction step. -2. **Select the portion** of the instruction text you want to parameterize (for example, highlight `test.com`). - -3. In the **Convert as** popover that appears, switch to the **`{$}` Parameter** tab. -4. Either pick an existing parameter from the searchable list, or click **`+` Create Parameter** to define a new one inline. -5. The selected text in the step is replaced with the parameter reference (for example, `{{search_query}}`). The parameterized step is now ready for data-driven execution. - -This flow is only supported for `Click` and `Type` commands. For the full Convert as popover behavior across Parameters, Variables, and Secrets, see [Creating variables from a Manual Interaction step](/support/docs/kane-ai-using-variables/#from-a-manual-interaction-step). - -## Test Summary Page -Upon completing the test, review the Test Summary Page, where all utilized parameters are clearly displayed, and the corresponding generated code is readily available for seamless integration into future test scenarios. - -## Using parameters for test runs - -### Step 1: Navigate to Test Project -Open the Test Project dashboard and go to the **Test Run** section. - -### Step 2: Create a New Test Run -- Click **Create New Test Run** and name it (e.g., parameters test run). -- Select the KaneAI-generated test cases. -- Optionally, add a description or relevant tags. -- Click **Create Test Run**. - -### Step 3: Add Test Cases and Data Sets -- Select and add the desired test cases to the run. -- Specify configurations (e.g., Windows 10 Chrome). -- Attach the required data sets for the test run. - -### Step 4: Apply Data Sets -- Ensure the data sets containing necessary parameters are pre-filtered and accessible. -- Click **Apply** to confirm the Dataset selection. - -### Step 5: Save and Prepare for Execution -- Save the test run with the selected parameters and Datasets. The system will create test instances ready for execution in a hybrid environment. - -### Step 6: Execute Tests with Parameters -Utilize the parameters to run your test cases with different data sets and inputs. You can manage and monitor execution using the TestMu AI Test Manager interface. - -## Video Explanation diff --git a/static/docs/kane-ai-using-variables.md b/static/docs/kane-ai-using-variables.md deleted file mode 100644 index 82f6bb34e..000000000 --- a/static/docs/kane-ai-using-variables.md +++ /dev/null @@ -1,135 +0,0 @@ -# Variables in KaneAI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Variables are placeholders that store values, which can be reused across different parts of a test case. In automation, using variables helps make the tests more flexible, reusable, and maintainable. Instead of hardcoding values in test steps, you can define variables, assign them values, and reference them throughout your tests. This reduces redundancy and makes the tests easier to modify, especially when dealing with frequently changing parameters like URLs, credentials, or device configurations. Variables can be accessed directly from https://kaneai.lambdatest.com/variables. - -Using variables enables you to: -- Avoid repetitive entries in test cases. -- Get values from within the application under test & store it in variable to validate the accuracy of data. -- Increase the reusability and adaptability of test scripts, especially useful for parameters that might change frequently, such as URLs, credentials, or environmental settings. - -## Types of variables -- **String Variables**: Store simple text values. They can be used for things like usernames, or any textual data. String variables can be created via KaneAI or manually within the session. - -- **JSON Variables**: Store structured data in the form of key-value pairs. JSON variables are especially useful for passing complex objects such as API responses. JSON variables are auto-generated from API calls and DB queries used in KaneAI and cannot be created manually. - -## Scope of variables - -### Local Variables -Local variables can only be defined using natural language or used as a part of an operation when an instruction is provided by the user. They are only accessible within the specific test case where they are defined. They help ensure that values are isolated and can be used temporarily within a specific context. A local variable can be converted to a global or an environment variable within the authoring session of KaneAI. - -### Global Variables -Global variables are accessible across multiple test cases within the entire organisation. They allow values to be reused in multiple places, enhancing consistency and reducing redundancy. Any variable which is converted from local to global has an option to persist value across sessions if the value changes within a session. More details about persist check are added below. - -### Environment Variables -Environment variables are variables that are typically set for a specific environment (e.g., staging, production). They are useful for managing values like URLs, or credentials that differ based on the environment in which the tests are being executed. Using environment variables you can execute your test on multiple environments easily. More details about environments are added below. - -## How to create & edit variables -Variables can be created using the following three methods: - -### Using Slash "/" Command in authoring session -You can define variables directly by using the `/` command in the step input and select "Add a variable" option. You are allowed to create _global_ and _environment_ variables using this approach and define their values. -The value of variables can be edited by clicking on the variables listing icon on the top right. - -### Using Natural Language in authoring session -KaneAI allows you to create variables using natural language. For instance, you can write: - -```bash -Set username as John -``` -This will automatically create a _local_ variable `{{username}}` with the value "John". -For such _local_ variables, only the scope of the variable can be changed to a global with an option to enable the persist check which ensures that the value when updated in one session or test execution gets persistes across sessions and test executions. - -### Via UI from Variables page -By visiting the [variables page](https://www.testmuai.com/login/?redirectTo=https://kaneai.lambdatest.com/variables), you can click on "Create new" option and add relevant details and scope. You can list of all global and environment variables on this page. -You can edit the session value for the variables here which will lead to updation of the value in all new test executions and sessions for that variable. - -### From a Manual Interaction step - -While reviewing a Manual Interaction step, you can replace any hard-coded value in the step's instruction with a Parameter, Variable, or Secret, without deleting or re-recording the step. This flow is available on both **Desktop** and **Mobile App** Manual Interaction sessions. - -#### Steps to convert a value - -1. Open the recorded Manual Interaction step (for example, *Type in main search input field 'test.com'*). -2. **Select the portion** of the instruction text you want to replace (for example, highlight `test.com`). - -3. In the **Convert as** popover that appears, choose one of the three tabs: - - **`{$}` Parameter**: test-level variable, supports selecting an existing parameter or creating a new one. - - **`{x}` Variable**: Global or Environment variable, supports selecting an existing variable or creating a new one. - - **🔒 Secret**: encrypted value that resolves only at execution time. Only new secrets can be created from this tab. -4. Either pick an existing entry from the searchable list, or click **`+` Create Parameter / Variable / Secret** to define a new one inline. -5. The selected text in the step is replaced with the variable reference (for example, `{{search_query}}`). The step updates in place. - -#### Support matrix - -| Variable Type | Use Existing | Create New | Scope | -|---------------|--------------|------------|-------| -| `{$}` Parameter | Supported | Supported | Test-level | -| `{x}` Variable | Supported | Supported | Global / Environment | -| 🔒 Secret | Not supported | Supported | Vault-level | - -- Existing secrets cannot be reused from the Convert as popover. The Secret tab only exposes the **`+` Create Secret** action. This is enforced for security, so a fresh secret must be created each time one is assigned to a Manual Interaction step. -- This is only supported for `Click` and `Type` Commands - -For details on secrets and parameters, see [KaneAI Secrets](/support/docs/kane-ai-secrets/) and [KaneAI Parameters](/support/docs/kane-ai-using-parameters/). - -## How to leverage variables during authoring -Variables can be used to enhance your test cases, making them more dynamic. - -### Syntax for Assertions -Variable usage can be done using the `{{` prefix to list all existing variables in the KaneAI session input box. When making assertions in your test case, you can use variables for inputing a value or as part of the assertion logic. - -#### String variables example: - -```bash -Enter {{username}} in user input field -``` -This will input the value of the username variable in the user input field. - -```bash -assert {{username}} contains 'John' -``` -This will assert if the username variable contains the value "John". - -#### JSON variables example: -JSON variables are generated from the response of an API used via the slash command or from the DB query output. We can use JSON variables to reach a particular object which might be inside the JSON hierarchy via the UI when the `{{` prefix is used in the input box inside the KaneAI session. For instance, we can assert that the email field inside the API variable response matches "john.doe@example.com". - -```bash -assert {{api_variable.response.email}} is 'john.doe@example.com' -``` -Here is a video to further explain the JSON variables usage: - -## Persist Value in Variables - -Persist value check is a mechanism used to ensure that variables retain their values across multiple test executions. This is important for situations where a variable’s value should be preserved between test cases and test executions to ensure consistency. For example, if you fetch a value from the AUT in a test while authoring, you may want to persist it for the other test cases even during test executions. - -The persist check option becomes available only when you change a variable's scope from local to global. - -## Environments - -### What are Environments? -Environments in KaneAI refer to the different configurations under which the tests are executed. Environments typically represent different stages of deployment (e.g., development, staging, production). Each environment can have its own set of variables tailored to the respective deployment scenario. - -### How to Use Environments During Test Runs -Once a test case is authored using environment variables, it can then be executed in any environment as long as the variables used in the test case, exist in the requested environment. - -#### Leveraging UI -While executing a test run or scheduling one, you can choose which environment they want to execute a test run on. All the test cases inside a test run can be executed on a single environment only at a point of time. - -#### Leveraging API -You can pass environment name programmatically when initiating test runs via the API. This allows you to automate the execution of tests in different environments without manual intervention. For instance, - -```bash -curl --location 'https://test-manager-api.lambdatest.com/api/atm/v1/hyperexecute' \ ---header 'Content-Type: application/json' \ ---header 'Authorization: Basic ' \ ---data '{ -"test_run_id": "YOUR_TEST_RUN_ID", #enter test run id -"concurrency": 1, -"environment": "staging" #Optional -``` - -This will run the test case in the "staging" environment. - -If you have any feedback or suggestions, feel free to reach out at support@testmuai.com with your comments. diff --git a/static/docs/kane-ai-web-test-writing-guidelines.md b/static/docs/kane-ai-web-test-writing-guidelines.md deleted file mode 100644 index 1f5d13ac7..000000000 --- a/static/docs/kane-ai-web-test-writing-guidelines.md +++ /dev/null @@ -1,65 +0,0 @@ -# KaneAI Web Agent - Guidelines for Writing Instructions - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The KaneAI Web Agent is an automation tool that executes web interactions based on natural language instructions. This guide provides best practices for writing clear, user-friendly instructions when using the KaneAI Web Agent. The goal is to ensure accurate execution of tasks by the AI and improve the quality of interactions with web elements. - -## General Instruction - -### 1. Clarity and Specificity -Always provide clear and specific instructions for the action you wish to perform. Use the appropriate terminology from the list of supported commands. Avoid vague terms such as `do this` or `click that`. Be explicit about which element to interact with. - -### 2. Context -When the action is dependent on a specific element or section of a webpage, provide enough context to help identify the element. Example: `Click the 'Submit' button on the top right corner of the form`. - -### 3. Step-by-Step Instructions -Break down complex tasks into smaller, manageable steps. Use logical connectors like `then` or `after that` to indicate the sequence of actions. Clearly specify the flow, such as `Click 'Login', then type your email in the input field.` - -### 4. Use Examples -Refer to the examples provided in the list of supported commands for guidance on structuring your instructions. -Example: `Type 'username' in the search bar and press 'Enter' to submit.` - -### 5. Wait Command -Use the wait command when needed to pause execution, allowing operations or page loads to complete before proceeding to the next action. Example: `Click 'Submit' and wait for 5 seconds before proceeding to the next step.` - -### 6. Refining Prompts -If the AI response is not as expected, refine your prompt by adding more details or increasing clarity. Use iterative refinement to achieve the desired outcome. - -### 7. Tab Targeting -When interacting with elements that open in a new tab, use the prompt `switch to the tab` to ensure actions stay on the newly opened tab. - -## DO's & DON'Ts - -### DO's -- **Specify the Exact Element :** Clearly indicate which element you want to interact with. For instance, use the element's name, position, or attributes. -Example: `Click on the second product in the list.` - -- **Use Action Verbs :** Start your instructions with action verbs like "Click," "Type," "Hover," etc. -Example: `Hover over the navigation bar.` - -- **Provide Context for Conditional Actions :** If the action depends on an element’s visibility or existence, include this in the instruction. Example: `If the 'Login' button is visible, click it.` - -- **Use Numbers to Indicate Positions or Quantities :** When specifying positions or quantities, use numbers for clarity. Example: `Scroll down 100 pixels.` - -- **Verify Steps Before Saving Test Cases :** Use the re-run option to ensure KaneAI executes all steps correctly and without errors before saving the test case. - -- **Validate Test Flow While Editing :** Let the test run to validate if the flow is correct while editing a test case. Click "Resume" only after all steps are completed. - -- **Use Manual Interaction When Necessary :** If KaneAI struggles to execute the desired action, manually intervene to guide the AI appropriately. - -### DON'Ts -- **Avoid Vague Terms :** Do not use terms like "Click that" or "Do this" without specifying which element or action. - -- **Do Not Assume Context Without Detail :** Ensure instructions are detailed enough for the AI to understand which elements are involved. Example of what to avoid: `Click the button" (without specifying which button)`. - -- **Avoid Overloading Instructions :** Do not combine too many actions into a single instruction unless they are logically sequenced. Example of what to avoid: `Click the button and type in the field and then hover.` - -- **Do Not Mix Unrelated Commands Without Logical Sequence :** Keep commands logically connected. Avoid mixing different actions without specifying the sequence. Example of what to avoid: `Click and then go to a new tab (without a clear sequence)`. - -- **Always Verify Results for Assertions :** When using assertions, confirm that the expected result occurs to ensure success. - -- **Acknowledge AI Limitations :** If the AI doesn’t perform the desired action, refine the prompt instead of expecting perfect results on the first try. - -- **Use Iterative Approach for Accuracy :** Allow the AI to process, execute the action, and respond. If the initial outcome isn’t correct, refine and reissue the prompt iteratively. - -- **Avoid Unnecessary Jargon :** Use simple language to communicate instructions. Only use technical terms when necessary. diff --git a/static/docs/kane-ai-web-test.md b/static/docs/kane-ai-web-test.md deleted file mode 100644 index 6090c99af..000000000 --- a/static/docs/kane-ai-web-test.md +++ /dev/null @@ -1,27 +0,0 @@ -# Automate your Web App Testing with KaneAI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -In this guide, we'll walk you through the process of generating, and executing automated tests for your web application using KaneAI. - -## Steps to generate the test cases - -### Step 1: Create a Web Test -Navigate to the [KaneAI homepage](https://www.testmuai.com/login/?redirectTo=https://kaneai.lambdatest.com/objective) and click on the **Create a Web Test** button to initiate the process of generating test cases for your web app. In the right side, a window will open with which you can navigate to your platform to perform the testing. - -### Step 2: Write Your Test Cases in Plain English -**Let KaneAI do the coding!** In this step, you'll describe your desired test outcomes using plain English. Don't worry about writing specific code or scripts – KaneAI will handle that for you. - -For example: -- Visit the website "www.lambdatest.com" -- Click on the Login button. - -Keep adding tests in plain English as you go; KaneAI will automatically queue them up for execution, so you don't have to wait for one test to finish before writing the next. Click on the **Fininsh Test** button. -> **Remember:** The more specific you are, the more accurate KaneAI's generated test cases will be. - -### Step 3: Save your Test Case -Now, save your test cases that you have generated till now. - -Select your Project, and the folder in which you want to store your test cases. The **Test Name** and the **Description** will be generated automatically, and you can also edit it. Select the **Type** of your test, and the status, and define the necessary tags which will help ypu to search your tests later. - -### Step 4: Execute your test diff --git a/static/docs/kane-cli-agent-mode.md b/static/docs/kane-cli-agent-mode.md deleted file mode 100644 index 3aa3d6427..000000000 --- a/static/docs/kane-cli-agent-mode.md +++ /dev/null @@ -1,167 +0,0 @@ -# Agent Mode - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Agent Mode outputs structured NDJSON instead of the interactive terminal UI. It's how AI coding agents (Claude Code, Codex CLI, Gemini CLI) consume Kane CLI results: parse events programmatically, extract the final result, and present it to the user. - -## Enable Agent Mode - -Add `--agent` to any run command: - -```bash -kane-cli run "Verify the checkout flow completes successfully" \ ---url https://myapp.com \ ---agent \ ---headless -``` - -When `--agent` is active: -- The interactive TUI is fully suppressed (no boot screen, no menus) -- Stdin is not used (no interactive prompts) - -Always combine `--agent` with `--headless` in non-interactive environments (CI/CD, agent contexts) to avoid display server errors. - -## Output Format - -Kane CLI outputs one JSON object per line (NDJSON) to stdout: - -``` -{"type":"run_start","objective":"Verify checkout","timestamp":"2026-04-14T10:30:45Z"} -{"type":"step_start","index":0,"objective":"Navigate to cart page"} -{"type":"step_event","index":0,"event":"action","detail":"Navigated to /cart","success":true} -{"type":"step_event","index":0,"event":"screenshot","detail":"Screenshot captured"} -{"type":"step_end","index":0,"status":"passed","duration":2.3,"summary":"Navigated to cart"} -{"type":"step_start","index":1,"objective":"Click checkout button"} -... -{"type":"run_end","status":"passed","summary":"...","duration":45.2,...} -``` - -## Event Schema - -### Step Events - -| Type | Key Fields | Description | -|------|-----------|-------------| -| `step_start` | `index`, `objective`, `child_id`* | Step began | -| `step_event` | `index`, `event`**, `detail`, `success`* | Activity within a step | -| `step_end` | `index`, `status` (passed\|failed), `duration`, `summary` | Step completed | - -*optional -**`event` values: `screenshot`, `reasoning`, `action`, `vision`, `assertion`, `evaluation` - -### Flow Events - -| Type | Key Fields | Description | -|------|-----------|-------------| -| `run_start` | `objective`, `timestamp` | Run started | -| `bifurcation` | `flows[]`, `count` | Agent branched into sub-flows | -| `child_agent_start` | `child_id`, `objective`, `parent_step` | Child agent spawned | -| `child_agent_end` | `child_id`, `success`, `steps_taken`, `summary` | Child agent finished | -| `ask_user` | `question`, `step_index`, `options`* | Agent needs input (auto-disabled when stdin is not TTY) | -| `error` | `message` | Error occurred | -| `run_end` | *(see below)* | Terminal event: always the last line | - -## The `run_end` Event - -`run_end` is always the last line. It contains the complete test result: - -```json -{ -"type": "run_end", -"status": "passed", -"summary": "Searched for laptop and added first result to cart", -"one_liner": "Searched for laptop on Amazon and added to cart", -"reason": "Objective completed", -"duration": 45.2, -"final_state": { -"price": "$29.99", -"product_name": "Wireless Headphones" -}, -"context": { -"memory": {}, -"variables": {}, -"pointer": "(passed) Searched for laptop on Amazon" -}, -"token_usage": { -"reasoning_input": 12000, -"reasoning_output": 800, -"vision_input": 5000, -"vision_output": 200 -}, -"session_dir": "~/.testmuai/kaneai/sessions/2026-04-14_10-30-45_a1b2c3", -"run_dir": "~/.testmuai/kaneai/sessions/2026-04-14_10-30-45_a1b2c3/runs/0", -"test_url": "https://test-manager.lambdatest.com/projects/123/test-cases/456" -} -``` - -### Key Fields - -| Field | Description | -|-------|-------------| -| `status` | `"passed"` or `"failed"` | -| `summary` | Full description of what the agent did | -| `one_liner` | Short single-sentence summary | -| `reason` | Why the run stopped (present on failure) | -| `final_state` | Values extracted via "store as" objectives | -| `test_url` | Link to the TestMu AI dashboard for this run | -| `session_dir` | Path to session-level logs | -| `run_dir` | Path to run-level logs, step JSON, screenshots | -| `token_usage` | Token consumption breakdown | - -# The .evidence proof pack - -Every Kane CLI run produces an `.evidence` pack stored natively in the results folder, so failure context travels with the run instead of living in a separate dashboard. Each pack contains the full execution proof for the run: step-by-step traces, screenshots, video, and command, network and console logs, plus the NDJSON event stream and result codes. Coding agents and humans read the same evidence to see exactly what happened, where it failed, and why. - -## Parsing Output - -Get the `run_end` event: - -```bash -# Get just the final result -kane-cli run "..." --agent 2>/dev/null | tail -1 | jq . - -# Extract status -kane-cli run "..." --agent 2>/dev/null | tail -1 | jq -r '.status' - -# Extract a stored value -kane-cli run "go to example.com, store the price as 'price'" --agent 2>/dev/null \ -| tail -1 | jq -r '.final_state.price' -``` - -## Handling `ask_user` Events - -If an objective requires user input mid-run, Kane CLI fires `ask_user`: - -```json -{"type": "ask_user", "question": "Which item should I select?", "options": ["Small", "Medium", "Large"]} -``` - -`ask_user` is **auto-disabled when stdin is not a TTY**. In CI/CD and AI agent contexts, stdin is never a TTY, so `ask_user` never fires. Write objectives that don't require interactive prompts. - -If stdin IS a TTY, respond by writing JSON to stdin: - -```json -{"type": "user_response", "answer": "Medium"} -``` - -To cancel: - -```json -{"type": "cancel"} -``` - -## Agent Mode vs Interactive Mode - -| | Agent Mode (`--agent`) | Interactive Mode | -|--|----------------------|------------------| -| TUI | Suppressed | Full terminal UI | -| Output | NDJSON to stdout | Formatted text | -| Stderr | Logs only | Logs + progress UI | -| Use case | AI agents, programmatic tools | Human development | - -## Next Steps - -- [Error Codes](/support/docs/kane-cli-error-codes/): Complete reference of `result_code` values and recommended actions -- [Parallel Execution](/support/docs/kane-cli-parallel-execution/): Run multiple tests concurrently -- [Skills](/support/docs/kane-cli-skills/): Install the Kane CLI skill for Claude, Codex, or Gemini -- [CLI Reference](/support/docs/kane-cli-cli-reference/): Full flag and command reference diff --git a/static/docs/kane-cli-agent-output.md b/static/docs/kane-cli-agent-output.md deleted file mode 100644 index 4bdf8ee74..000000000 --- a/static/docs/kane-cli-agent-output.md +++ /dev/null @@ -1,3 +0,0 @@ -# Agent Mode Output - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). diff --git a/static/docs/kane-cli-api-calls.md b/static/docs/kane-cli-api-calls.md deleted file mode 100644 index ebf5eaaf7..000000000 --- a/static/docs/kane-cli-api-calls.md +++ /dev/null @@ -1,62 +0,0 @@ -# API Calls - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Objectives can have the agent **make an API call directly**, not just observe the requests a page makes. This is useful for seeding data before a flow, hitting a backend to set up state, or checking a service, then asserting on or reusing the response. - -## Making a call - -Phrase an explicit HTTP request and name its response with "save the response as …": - -``` -Call POST https://api.example.com/orders with body {"item": "sku_42", "qty": 1}, save the response as order -Hit GET https://api.example.com/orders/123, save the response as fetched -Call DELETE https://api.example.com/orders/123 -``` - -A pasted `curl` works too and is kept exactly as written: method, headers, body, and auth: - -``` -curl -X POST https://api.example.com/login -H 'Content-Type: application/json' -d '{"u":"a","p":"b"}', save the response as login -``` - -## Using the response - -Once you've saved a response under a name, reference it elsewhere in the objective: - -| Reference | Resolves to | -|-----------|-------------| -| `{{order.status}}` | the HTTP status code (e.g. `201`) | -| `{{order.response_body}}` | the whole response body | -| `{{order.response_body.}}` | a field from the JSON response body | - -Assert on it, or feed it into later actions. API calls and browser actions mix freely in one objective: - -``` -Call POST https://api.example.com/login with body {"u": "{{user}}", "p": "{{password}}"}, save the response as login, -assert {{login.status}} is 200, -then open https://app.example.com and verify the dashboard loads -``` - -``` -Call POST https://api.example.com/orders with body {"item": "sku_42", "qty": 1}, save the response as order, -assert {{order.status}} is 201, -then open https://app.example.com/orders and verify an order for "sku_42" is visible -``` - -## Tokens and secrets - -Put any API token or credential in a variable marked `secret: true` (see [Variables and Context](/support/docs/kane-cli-variables-and-context/)) so it is masked in logs and never stored in plain text: - -``` -curl -X DELETE https://api.example.com/records/42 -H "Authorization: Bearer {{api_token}}" -``` - -## Make a call vs. observe page traffic - -These are two different things: - -- **Make a call (this page)**: *you* tell the agent to send a request: seed a record, call a backend, set up state. -- **[Network assertions](/support/docs/kane-cli-checkpoint-devtools-network/)**: *observe* the requests the page itself makes during a UI flow (status codes, bodies, timing). - -They compose: seed state with a direct call, drive the UI, then assert on the page's own network traffic. diff --git a/static/docs/kane-cli-assurance-automation.md b/static/docs/kane-cli-assurance-automation.md deleted file mode 100644 index 487169b64..000000000 --- a/static/docs/kane-cli-assurance-automation.md +++ /dev/null @@ -1,174 +0,0 @@ -# Assurance in CI and from Agents - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The conversational assurance commands — `context extract`, `design tests`, and `maintain reconcile` — are interactive by default. This page is the contract for running them **headless**: from CI, from a script, or from an AI agent driving kane-cli. - -## The ask policy: `--mode` - -On a terminal, extract and design open a chat. Headless is an explicit opt-in — a bare non-TTY invocation **exits `2`** and mutates nothing: - -``` -extract: no TTY — pass an explicit --mode agent|ci|override to run headless -``` - -`--mode` decides both what happens when the agent has a question, and what the command writes to stdout: - -| Mode | Questions | stdout | -|---|---|---| -| `interactive` | asked in the chat (TTY default) | the Ink chat UI | -| `agent` | low/medium-risk defaults are auto-taken (each reported); a **high-risk** question pauses the session — exit `3`, resumable | **NDJSON events** (one JSON object per line); prose diagnostics go to stderr | -| `ci` | any high-risk question **fails closed** — exit `1`, error code `HIGH_RISK_CI` | prose transcript | -| `override` | every default is auto-taken, including high-risk (each flagged in the commit record) | prose transcript | - -Rule of thumb: `agent` when something can read the pause and answer (an AI agent, a human on the next shift); `ci` when a pipeline must never guess; `override` when you accept the recommended defaults wholesale and want one unattended pass. - -The same matrix drives `maintain reconcile`, with two reconcile-specific rules: no headless mode ever archives anything — ARCHIVE decisions wait for an interactive session — and a `ci`-mode run that hits a decision needing a human **stores the plan and exits `2`** (the work isn't lost; walk the stored plan interactively or apply it in `agent` mode). - -## Exit codes - -Consistent across extract, design, and the maintain commands that embed them: - -| Code | Meaning | -|---|---| -| `0` | Complete. | -| `1` | Runtime failure. For extract and design, a `ci`-mode fail-close on a high-risk question also exits `1`; reconcile's `ci` fail-close stores the plan and exits `2` instead. | -| `2` | Usage / auth / refusal — bad flags, failed input validation, no store, bare non-TTY without `--mode`, missing `--yes` on a destructive command. Nothing was mutated. | -| `3` | **Paused and resumable** — the only meaning of 3. A session is saved; resume it within 24 hours. | - -## The NDJSON stream (`--mode agent`) - -With `--mode agent`, stdout speaks a versioned NDJSON vocabulary — envelope `{"type": "", "v": 1, "verb": "extract"|"design", ...}`, one object per line. The vocabulary is open: new event types may appear, so **tolerate unknown types**. - -| type | payload highlights | -|---|---| -| `run_start` | `mode`, `trace` (the per-run log path); design adds `use_case` | -| `corpus` | extract: the `sources[]` this run covers + already-extracted `skipped[]` | -| `source_start` / `source_skipped` | `source_id`, `index`/`total`, `resumed` / `reason` | -| `plan` | the `--plan` transcription payload | -| `assumed_default` | a question auto-answered with its recommended default: `id`, `selected_index`, `risk` | -| `agent_activity` | progress: `kind` (`tool` / `decision` / `progress` / `thinking_done`) + a display `label` | -| `usage` | per agent turn: `credits` + running `total_credits` | -| `validate_failed` | a proposal failed kane-side validation: `codes[]`, `repairing` (the agent self-repairs) | -| `commit` | what landed: counts + `minted[]` (`cid` + `logical_id`); extract adds `proposal_id` | -| `receipt` | design: per-phase commit receipt — `commit_n`, `phase`, `committed[]`, `warnings[]`, `parity`, and a human-readable `next` hint | -| `message_sent` | your `--message` was delivered: `sid`, `chars` | -| `session_paused` | `sid`, the verbatim `resume` command, `expires_at`, and **`pending_questions[]`** in full | -| `session_complete` | `sid` | -| `gate_refused` | a design gate refused the run (may be the first event) | -| `error` | `message` + a stable `code` where one exists (`NO_STORE`, `PREFLIGHT`, `SOURCE_MISSING`, `BLOB_MISSING`, `HIGH_RISK_CI`, `STALE_BASIS`) | -| `done` | **always the last event**: `status` (`complete`/`paused`/`error`/`refused`/`interrupted`/`aborted`) + `exit_code` | - -**The `done` guarantee:** every `--mode agent` invocation ends its stream with exactly one `done` event — including refusals and graceful interrupts. The one exception is operator force: a second Ctrl+C can hard-kill the process (exit `130`) without a `done`. Any other stream that ends without `done` should be treated as a crash. One more parsing note: the agent may also repair a draft mid-turn on its own — that surfaces only as `agent_activity` lines (labels like `validation failed`, `refining the draft`); treat activity labels as display text, never script against them. - -### Reconcile's stream - -`maintain reconcile --mode agent` speaks the same envelope with `verb: "reconcile"` and its own event set: - -| type | payload highlights | -|---|---| -| `reconcile_plan` | the triage ahead: `source_id`, `plan_path`, `rows[]` (`kind`, `ref`, `why`), `archive[]` (proposed archivals with their evidence-decay reasons) | -| `reconcile_row_start` | per row: `kind`, `ref`, plus the impact counts where they apply (`stale`, `direct`) | -| `reconcile_row_end` | the row's `outcome` (`applied` \| `failed` \| `skipped` \| `plan-only` \| `paused`) + `exit_code`, and an additive `detail` carrying a failure's reason and hint. A row's embedded design run is folded in here, so the stream stays single-writer with exactly one `done` | -| `reconcile_paused` | `plan_path` + `pending[]` (`ref`, `why`) — resume with the same reconcile command (or `--apply`) | -| `reconcile_summary` | the honest totals, always the same field set: `applied`, `skipped`, `deferred`, `plan_only`, `failed`, `paused`, `stale_created` | -| `done` | always last — same guarantee as above | - -Validation failures (bad inputs, unknown source, the fork guard) ride the stream as `error` + `done` with exit `2` — never stderr alone. - -## The pause → answer → resume loop - -This is the heart of driving assurance from an agent. A real exchange (events abridged, payloads shortened): - -```bash -$ kane-cli context extract --mode agent -{"type":"run_start","v":1,"verb":"extract","mode":"agent","trace":".context/logs/extract-….log"} -{"type":"corpus","v":1,"verb":"extract","sources":[{"source_id":"prd-online-store","cid":"sha256:0661…"}],"skipped":[]} -{"type":"agent_activity","v":1,"verb":"extract","kind":"decision","label":"asking to resolve an ambiguity"} -{"type":"session_paused","v":1,"verb":"extract","sid":"ext-20260716T140742-prd-online-store", -"resume":"kane-cli context extract --resume ext-20260716T140742-prd-online-store --mode agent", -"expires_at":"2026-07-17T14:07:53Z", -"pending_questions":[{"id":"q1", -"text":"The PRD conflicts on guest checkout; should I treat checkout as account-required or guest-allowed?", -"risk":"high", -"rationale":"Lines L20-L21 say all customers must create an account, but L35 says guest checkout is allowed.", -"options":[{"label":"Account required","detail":"…"},{"label":"Guest allowed","detail":"…"}], -"recommended_index":0,"allow_free_text":true}]} -{"type":"done","v":1,"verb":"extract","status":"paused","exit_code":3} -``` - -The pause event carries everything needed to decide: the question, why it matters, the options, and the recommendation. Answer **in plain words** — no question ids, no option indexes. After the resumed run's usual `run_start`, `corpus`, and `source_start` (with `"resumed": true`) events, the stream continues: - -```bash -$ kane-cli context extract --resume ext-20260716T140742-prd-online-store --mode agent \ ---message "Account required — treat the update section as superseding: no guest checkout" -{"type":"message_sent","v":1,"verb":"extract","sid":"ext-…","chars":115} -{"type":"usage","v":1,"verb":"extract","credits":2.45,"total_credits":2.45} -{"type":"commit","v":1,"verb":"extract","derived":5,"minted":[{"cid":"sha256:6d68…","logical_id":"uc-create-an-account-to-order"}, …]} -{"type":"session_complete","v":1,"verb":"extract","sid":"ext-…"} -{"type":"done","v":1,"verb":"extract","status":"complete","exit_code":0} -``` - -The agent maps your statement to its own pending questions. A statement that answers nothing pending is treated as steering ("also cover the coupon path"); if it leaves a high-risk ambiguity standing, the run pauses again with refreshed questions. - -Between the pause and the resume, everything is inspectable without contending the session: - -```bash -kane-cli context sessions --json # one row per resumable session, with its resume command -kane-cli context sessions show --json # the pending questions in wire shape + any assumed defaults -``` - -Abandoned sessions expire after 24 hours; `kane-cli context sessions clean` garbage-collects them. - -## Headless review - -Trust promotion deliberately has **no auto-approve** — but it does have a non-interactive path. Prepare verdicts as JSON and land them atomically: - -```bash -cat > verdicts.json <<'EOF' -[ -{"ref": "uc-create-an-account-to-order", "resolution": "approved"}, -{"ref": "uc-manage-the-cart", "resolution": "approved"} -] -EOF -kane-cli context review --verdicts verdicts.json --json -``` - -`resolution` is one of `approved | edited | rejected | skipped | supersede` (optional `reason`, `edit`, `supersede_target`). One unresolvable ref fails the whole file (exit `2`, nothing committed). With `--json`, each landed verdict echoes as one NDJSON row. - -## Machine-readable reads - -These read commands have structured forms: `context list --json` and `context sessions --json` (one JSON object per line), `context explain --json`, `context view --json` (the full computed graph payload), `context view --no-open --out graph.html` (render without a browser), and `cover --json`. - -## Headless maintain - -- `maintain reconcile --from --source-id --plan` — safe preview: records the source change, stages every proposed row into a stored plan, touches nothing else. Exit `0`; when the source actually changed, the plan path is the last stdout line (an unchanged source is a no-op that stores nothing). -- `maintain reconcile … --mode override` (or `--mode ci`) — unattended application: ADD and MODIFY rows apply, archiving never happens headless, and `ci` fail-closes the moment human judgement is needed (the plan is stored; exit `2`). -- Re-running the same reconcile command is idempotent — it resumes a pending plan, reports an applied one, and recomputes a superseded one ([details](/support/docs/kane-cli-assurance-maintain/#running-again)). -- Bare headless runs without an explicit `--mode` refuse with exit `2` — by design. - -## A CI shape that works - -```bash -# fail the pipeline on unresolved high-risk ambiguity, never guess: -kane-cli context extract --mode ci - -# or: let it pause, surface the questions as a build artifact, resume in a follow-up job: -kane-cli context extract --mode agent > extract.ndjson; code=$? -if [ "$code" -eq 3 ]; then -kane-cli context sessions --json > pending-sessions.ndjson # hand to a human or an agent -fi - -# design a specific use-case unattended, bounded: -kane-cli design tests --use-case uc-checkout --max 8 --mode ci - -# keep the suite honest on requirement changes: -kane-cli maintain reconcile --from ./docs/prd.md --source-id prd --plan -``` - -Author and batch the resulting tests with the same CI patterns as any other test — see the [CI/CD recipes](/support/docs/kane-cli-cicd/). - -## Next steps - -- [The assurance overview](/support/docs/kane-cli-assurance/) — where each command sits. -- [Building the context graph](/support/docs/kane-cli-assurance-context/) · [Designing tests](/support/docs/kane-cli-assurance-design/) · [Maintaining the suite](/support/docs/kane-cli-assurance-maintain/). diff --git a/static/docs/kane-cli-assurance-context.md b/static/docs/kane-cli-assurance-context.md deleted file mode 100644 index 75f86f990..000000000 --- a/static/docs/kane-cli-assurance-context.md +++ /dev/null @@ -1,240 +0,0 @@ -# Building the Context Graph - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -`kane-cli context` builds a local, content-addressed knowledge store (`.context/` in your project directory) from your requirement documents, and extracts **use-cases** from them with an AI agent. It is the first stage of the [assurance lifecycle](/support/docs/kane-cli-assurance/): Source → Use-case → Scenario → AC → Test. - -```bash -kane-cli context ingest ./prd-online-store.md # snapshot a source -kane-cli context extract # extract use-cases (interactive chat) -kane-cli context review # promote proposals to trusted -kane-cli context list # see what you have -``` - -## `context ingest` — snapshot your sources {#ingest} - -```bash -kane-cli context ingest [--as ] -``` - -Snapshots one or more files into `.context/` (the store is created on first use). Each source gets a stable id — by default the filename slug (`prd-online-store.md` → `prd-online-store`), or pass `--as ` to name it yourself. - -Ingest is deterministic about identity: - -| You ingest… | Result | -|---|---| -| same id, same bytes | `unchanged` — nothing written | -| same id, new bytes | `versioned` — the source's head moves; everything extracted from the old snapshot goes **stale** | -| a new id | `created` | -| identical bytes already ingested under a different id | interactive relocate offer (default yes); non-TTY mints the new id and prints a hint | - -``` -$ kane-cli context ingest ./prd-online-store.md -created prd-online-store source sha256:0661… blob sha256:3db8… -``` - -Accepted media: text (`.txt`) and markdown (`.md`, `.markdown`) up to 2 MB — cited verbatim by line; PNG/JPEG/WebP images up to 5 MB — cited whole-image. Anything else is rejected with `UNSUPPORTED_MEDIA`; oversized files with `FILE_TOO_LARGE`. - -When the new bytes are a **changed version of a source you already extracted from**, prefer [`kane-cli maintain reconcile`](/support/docs/kane-cli-assurance-maintain/) over a bare re-ingest — it records the same head move *and* triages what the change means for your suite, in one step. - -## `context extract` — propose use-cases {#extract} - -```bash -kane-cli context extract [--plan] [--force] [--source ] [--mode ] [--resume [--message ""]] -``` - -Runs the extraction agent over every ingested source whose current snapshot has no committed extraction yet (already-extracted snapshots are skipped — re-run with `--force` to redo one). - -On a terminal the default is an **interactive chat**. Headless use is an explicit opt-in: a bare non-TTY invocation exits `2` and asks you to pass `--mode agent|ci|override` — see [Automation](/support/docs/kane-cli-assurance-automation/) for the headless contract. - -What the agent does: - -- Reads each source and proposes use-cases with **verbose descriptions** (flows, inputs, states, boundaries) and **`criteria[]`** — short, cited sketches of the acceptance-relevant promises the source states. Criteria are hints for [`kane-cli design`](/support/docs/kane-cli-assurance-design/), never test oracles. -- **Cites everything.** Every proposal must quote an exact line from the source; fabricated evidence is rejected before anything is written. (Image sources cite the whole image — there is no text to quote.) -- **Asks when the source is ambiguous.** Conflicting requirements become clarifying questions with options, a recommended default, and a risk level — low/medium-risk questions can be defaulted, high-risk ones want a real answer. -- **Grounds itself in what you already have.** The agent explores the existing graph read-only before proposing, so a use-case you already committed becomes new evidence on the existing node (shown as `≈ matches`) instead of a duplicate. - -Extraction stops at the use-case. Scenarios, ACs, and tests are minted by the [design engine](/support/docs/kane-cli-assurance-design/) — each stage can only create its own kinds. - -Flags: - -| Flag | Meaning | -|---|---| -| `--plan` | Stop after the proposal: print it (with `≈ matches` dedup flags) and commit nothing | -| `--force` | Re-extract sources even if their current snapshot was already extracted | -| `--source ` | Extract exactly this ingested source instead of the whole corpus | -| `--mode ` | Ask policy for headless runs: `agent` \| `ci` \| `override` — see [Automation](/support/docs/kane-cli-assurance-automation/) | -| `--resume ` | Resume a paused session ([sessions](#sessions)) | -| `--message ""` | With `--resume`: answer the pending questions in plain words | - -### The interactive chat - -The chat has two zones. The scrollback is the complete session journey — your words, the agent's narrative, its reasoning segments and tool lines, each proposal list as it arrived, answer receipts, commit receipts, and per-turn credit costs. The live region below shows only what is happening now: the current thinking line (**ctrl+t** expands it), the question panel, and the composer. - -**Answering questions** happens right in the composer: - -- Bare input answers the active question (marked `›`), and the cursor advances: `1` picks option 1, typing an option's name matches it, anything else is free text where allowed. -- `N:` targets question N explicitly (`2: yes`). Without the colon, `2 business days` stays free text. -- **Empty Enter** accepts the recommended defaults for **low/medium-risk questions only** — each is echoed back as `↳ assumed … (flagged)`. High-risk questions are never bulk-defaulted: answer them, or type `defer` to leave the rest with the agent. -- The batch submits when every question is answered, assumed, or deferred — never silently partial. - -**Ctrl+C asks for a pause, not a crash.** The first press asks the agent to save the session; on success you get a pause card with the exact resume command (including the `--message` form) and the run exits `3`. If the save can't complete within a few seconds you get an honest `interrupted — session not saved` (exit `130`, not resumable), and a second Ctrl+C at any point is an immediate hard exit. `/pause` does the same from the composer; `/done` ends the session cleanly. - -Slash commands (everything else you type is conversation for the agent): - -| Command | Action | -|---|---| -| `/view [N \| acs\|scenarios\|tests]` | browse this session's proposals in a view-only explorer | -| `/explain ` | why a committed item exists — replayed from the record, no agent turn | -| `/pause` | save + exit, resumable | -| `/done` | finish the session cleanly | - -### The review checklist - -After the agent proposes, the same session walks you through a review checklist: space cycles **approve / edit / reject / skip** per item, `e` opens an edit form, and Enter commits the whole batch as one record: - -- **approve / edit** → committed as `trusted` -- **reject** → committed + archived (kept on record, not deleted) -- **skip** → committed as `derived` — queued for later review - -### Pausing and resuming - -When the agent needs an answer you're not there to give (or you Ctrl+C), the session is saved and the run exits `3`. Resume it any time within 24 hours: - -```bash -kane-cli context sessions # list resumable sessions + their resume commands -kane-cli context extract --resume # re-presents the pending questions -kane-cli context extract --resume --message "Account required — the update supersedes the old section" -``` - -`--message` answers **in plain words** — no question ids, no option indexes. The agent maps your statement to its own pending questions. A statement that answers nothing pending is treated as steering ("also cover the coupon path"). If your answer leaves a high-risk ambiguity standing, the run pauses again with refreshed questions. - -## `context review` — review outside the extract session {#review} - -```bash -kane-cli context review [--queue derived|skipped|archived|drift] [--verdicts ] [--json] -``` - -Walks existing nodes through the same review checklist, landing every verdict as one batched record: - -- `derived` (default) — everything unreviewed -- `skipped` — strictly the items you skipped during an extract review -- `archived` — resurrection candidates: an explicit approve restores trust -- `drift` — a **listing only** (works without a TTY): nodes whose evidence is stale or orphaned, with their pinned sources — the re-extract worklist - -In any queue: approve promotes, reject archives (a trusted node *can* be demoted), and edit mints a new version that supersedes the old one — nodes are immutable, so edits never rewrite history and existing references never break. - -**Headless verdicts** — `--verdicts ` is the one non-TTY write path: a JSON array of - -```json -[{ "ref": "uc-manage-the-cart", "resolution": "approved" }] -``` - -with `resolution` one of `approved | edited | rejected | skipped | supersede` (plus optional `reason`, `edit`, `supersede_target`). It is atomic: every ref must resolve and sit in a verdict queue, or nothing commits (exit `2`). With `--json`, each landed verdict echoes as one NDJSON row. There is deliberately no auto-approve mode for review — trust requires a human decision. - -## Inspecting the graph {#inspect} - -### `context list` - -```bash -kane-cli context list [--type source|usecase] [--inferred] [--stale] [--all] [--json] -``` - -Lists nodes with their trust and freshness. `--inferred` shows only unreviewed (`derived`) nodes, `--stale` only stale or orphaned nodes (evidence pinned to an outdated snapshot, or no live source at all), `--all` includes superseded versions (hidden by default). `--json` emits one JSON object per line. - -### `context view` - -```bash -kane-cli context view [--out ] [--open|--no-open] [--json] -``` - -Renders the whole graph as a **single self-contained HTML page** — swimlanes per use-case, provenance edges back to the source, trust and staleness at a glance, a commit rail along the bottom, and click-through detail panels with each node's lineage. It is a snapshot (no server; works offline); re-run to refresh. Piped runs write the file and print its path instead of opening a browser; `--json` prints the computed payload for scripting. - -### `context explain` - -```bash -kane-cli context explain [--json] -``` - -Replays a node's recorded history straight from the store — **no model call, ever**: when it was minted and why, every review verdict, edits and supersessions, name assignments. `` is a logical id (`uc-manage-the-cart`) or a cid. - -### `context sessions` {#sessions} - -```bash -kane-cli context sessions [list|show|clean] [] [--all] [--json] -``` - -Paused extract *and* design sessions live under `.context/sessions/` for 24 hours. `list` shows each with its pending-question count, expiry, and ready-to-paste resume command. `show ` prints everything the paused agent is waiting on — the questions in full, any defaults it assumed in your absence, and both resume forms. `clean` garbage-collects expired sessions (`clean ` removes one; `--all` removes everything). - -## Housekeeping - -### `context retire` - -```bash -kane-cli context retire [--reason ] [--yes] -``` - -Retires a source. Its use-cases are **not** deleted — they read `orphaned` once no live source evidences them. Fully reversible via `revert`. - -### `context name` - -```bash -kane-cli context name # name one node -kane-cli context name --backfill [--yes] # assign ids to every unnamed node -``` - -Assigns a stable kebab-case name. Names are never part of a node's identity — renaming never re-addresses — and names follow edits, so a name assigned to version 1 keeps resolving to the current version. - -### `context revert` - -```bash -kane-cli context revert [--reason ] [--yes] -``` - -Inverts a record's effects by appending a compensation record — mints are tombstoned, heads move back, trust states are restored. History is never rewritten: the store keeps both the mistake and its correction. Reverting a revert restores the original effects. - -### `context fsck` / `context rebuild` - -`fsck` verifies the full record chain and checks the read caches for drift (exit `1` on any issue) — run it whenever hands touched `.context/` directly. `rebuild` wipes the derived caches and regenerates them from the verified records; it is always safe. - -**Destructive-verb rule:** `retire`, `revert`, `name --backfill`, and `rebuild` prompt for confirmation on a terminal (default No) and require an explicit `--yes` headless. Read commands never create a `.context/` store in a directory that has none — only `ingest` and `extract` do. - -## Trust and freshness - -| Trust | Meaning | How you get there | -|---|---|---| -| `derived` | machine-proposed, unreviewed | extraction commit (or a skip verdict) | -| `trusted` | human-confirmed | approve or edit in review | -| `archived` | human-rejected | reject in review | - -Freshness is orthogonal: `fresh` / `stale` (the source snapshot moved) / `orphaned` (no live source evidences it) / `superseded` (this version was replaced). A stale use-case is still trusted — it just needs re-verification against the new snapshot, which is exactly what [`kane-cli maintain reconcile`](/support/docs/kane-cli-assurance-maintain/#reconcile) is for. - -## The store on disk - -``` -.context/ -├── meta.json # store identity + format version -├── commits/ # append-only records — the truth -├── blobs/ # write-once source snapshots -├── derived/ # regenerable read caches (delete any time; rebuild restores) -├── proposals// # proposal + review artifacts per extract run -├── sessions// # resumable paused sessions (expire after 24h) -├── logs/ # per-run trace files -├── design/ # design rationale sidecars + technique overrides -├── reconcile/plans/ # stored reconcile plans -└── signals.ndjson # internal review bookkeeping (appears once recorded) -``` - -Two rules worth repeating from the [overview](/support/docs/kane-cli-assurance/#the-store-context): the store is **single-writer**, and it is **not git-mergeable** — gitignore it and share by re-ingesting sources. - -Every extract run also writes a per-run trace to `.context/logs/extract-.log` (the path is printed at the start of the run) — the first place to look when a run surprises you. - -## For agents and CI - -Headless extraction (`--mode agent|ci|override`), the NDJSON event stream, exit codes, and the pause/resume contract are documented in [Automation](/support/docs/kane-cli-assurance-automation/). - -## Next steps - -- [Designing tests](/support/docs/kane-cli-assurance-design/) — turn a trusted use-case into ACs, scenarios, and runnable tests. -- [Maintaining the suite](/support/docs/kane-cli-assurance-maintain/) — what to do when a source changes. -- [Automation](/support/docs/kane-cli-assurance-automation/) — the headless contract. diff --git a/static/docs/kane-cli-assurance-coverage.md b/static/docs/kane-cli-assurance-coverage.md deleted file mode 100644 index 0e096225b..000000000 --- a/static/docs/kane-cli-assurance-coverage.md +++ /dev/null @@ -1,76 +0,0 @@ -# Coverage: Proven vs Owed - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -`kane-cli cover` measures coverage on two independent axes over the same store: - -- **Depth** — what a real execution **proved**, read from an evidence pack's coverage records. Facts only: the pack was sealed with these verdicts inside it; `cover` never recomputes them. -- **Completeness** — what the design still **owes**, computed live from the `.context/` graph. A perfect pack can still ship with an unverified criterion or a happy-path-only use-case; this axis never reads packs. - -A run can look green and still owe you coverage — that's exactly the situation the two axes make visible. - -```bash -kane-cli cover [--from ] [--json] # the two-axis panel -kane-cli cover gaps [--stage design|cover|all] [--top ] [--from ] # the ranked worklist -``` - -## The panel - -``` -coverage — 8f0e…f2.evidence - -depth (proven by the pack): -◐ ███░░░░░░░ 38% uc-buy-as-a-guest — partial (1/4 ACs proven, 1 failed, 1 blocked) -✔ ██████████ 100% uc-mobile-sign-in — covered (1/1 ACs proven) · 1 stale - -completeness (live graph): -[high] create ac-payment-declined-message — no test verifies this AC -→ kane-cli design tests --use-case uc-buy-as-a-guest -``` - -- The default pack is the newest in `/.testmuai/evidence`; `--from` takes a pack directory, a sealed `.evidence` file, or an execution id. -- Depth is **risk-weighted and lenient**: a high-risk criterion weighs more, and a passed-but-stale criterion still counts as proven — staleness is surfaced (`· N stale`), never silently demoted. Per-use-case status is `covered` (every AC proved) · `blocked` (something couldn't run, nothing failed) · `partial` · `uncovered`. -- Coverage reflects **this run**: sealed packs cover only what the run touched. Project-wide coverage lives in the graph axis, unaffected by any single pack. -- `--json` emits the full panel as structured data. - -## `cover gaps` — the worklist - -One ranked list (risk first) of what to do next, each row with a ready-to-paste command: - -``` -gaps — stage design (5) -1. [high] create uc-checkout-while-signed-in — use-case has no scenarios -→ kane-cli design tests --use-case uc-checkout-while-signed-in -2. [med] create ac-the-cart-displays-an-order-subtotal — no live test verifies this acceptance criterion -→ kane-cli design tests --use-case uc-manage-the-cart -``` - -- `--stage design` (default, no pack needed) — criteria no test verifies, use-cases with no or only-happy scenarios, recorded gap nodes from design runs, stale designed entities. -- `--stage cover` (needs a pack) — a covered criterion whose **execution** disappointed: `failed` → re-design that slice; `blocked` or never-run → the test exists, run it. -- `--stage all` — both, one ranking. `--top ` trims the list. - -## The join: how a pack knows your graph - -Every per-test result in an evidence pack carries a `definition_id` — a hash of the resolved test definition, identical to the one design stamps on each test it emits. The pack↔graph join is this hash equality and nothing else: no ids to sync, no registry to maintain. A hand-edited test hashes differently and simply stops joining — honest, not broken (a redesign — [`design tests --force`](/support/docs/kane-cli-assurance-design/#re-runs-and-force) or [`maintain evolve`](/support/docs/kane-cli-assurance-maintain/#evolve) — re-stamps the link). - -Coverage records land in packs automatically whenever the project has a `.context/` store — the inline `run`/`testmd` path and `testrun` both write them before sealing. A project without a store gets byte-identical packs to before; a coverage-write failure never costs the seal. - -## The authoring bridge - -A freshly designed test has never been executed, and the tooling is honest about that: - -1. `kane-cli design tests` writes `t-…_test.md` files — runnable, but with no recording yet. -2. `kane-cli testrun` preflight refuses never-authored members (`missing_meta`). -3. So the first run of each designed test is [`kane-cli testmd run `](/support/docs/kane-cli-testmd/) — the agent authors it in a real browser and commits the recording. -4. From then on the test replays like any other: batch it with `testrun`, and its verdicts join the pack via `definition_id`. - -Until step 3 happens, `cover` reads the test's criteria as *covered on paper, unproven in execution* (`covered_by` present, execution `not-run`). That is a deliberate reading, not a bug — a designed test is a claim until a run proves it. - -## Inside the pack: `coverage/usecases.yaml` - -The pack's coverage record is one YAML file you can read, diff, and archive — one row per live use-case: identity and risk, sources and provenance, scenarios, and each acceptance criterion with its verdict join (`covered_by`, `execution: passed|failed|blocked|not-run`, `fresh`, the expected answer, and what satisfied it). Diff two packs' `usecases.yaml` to see exactly what a release changed in proven coverage. - -## Next steps - -- [The authoring bridge in practice](/support/docs/kane-cli-assurance-design/#from-design-to-execution) — design → author → batch. -- [Maintaining the suite](/support/docs/kane-cli-assurance-maintain/) — act on what the gaps list tells you. diff --git a/static/docs/kane-cli-assurance-design.md b/static/docs/kane-cli-assurance-design.md deleted file mode 100644 index 66d9609fd..000000000 --- a/static/docs/kane-cli-assurance-design.md +++ /dev/null @@ -1,126 +0,0 @@ -# Designing Tests from Use-Cases - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -`kane-cli design tests` turns **one committed use-case** into everything that proves it: acceptance criteria (ACs), scenarios, and exactly one runnable test per scenario — conversationally, on the same chat surface [`kane-cli context extract`](/support/docs/kane-cli-assurance-context/#extract) uses. Everything the engine emits is **derived** knowledge you review; approvals promote it, nothing is silently trusted. - -```bash -kane-cli design tests --use-case uc-manage-the-cart # design one use-case (chat) -kane-cli design explain t-add-first-item # replay WHY — zero fresh AI -``` - -`` is a short logical id (`uc-manage-the-cart`, `t-login-smoke`) or a full cid. - -## Flags - -| Flag | Meaning | -|---|---| -| `--use-case ` | The use-case to design (optional with `--resume` — the session remembers it) | -| `--max ` | Budget ceiling: max scenario+test pairs kept. **No ceiling when omitted** — the agent estimates the right size and asks you to confirm. Every eviction becomes a named `budget-evicted` gap | -| `--strength pairwise\|3-wise` | Manual covering-array strength override; absent = risk-judged (3-wise when money, auth, or data loss is involved) | -| `--mode ` | Ask policy for headless runs: `agent` \| `ci` \| `override` — bare non-TTY exits `2`. See [Automation](/support/docs/kane-cli-assurance-automation/) | -| `--force` | Redesign a use-case that already has a live design (supersedes its scenario+test pairs; equivalent ACs are reused) | -| `--resume ` | Resume a paused session ([sessions](/support/docs/kane-cli-assurance-context/#sessions)) | -| `--message ""` | With `--resume`: answer the pending questions (or steer) in plain words | -| `--plan` | Transcription only — print each finalize payload, commit nothing | - -## The session — five phases - -Interactive runs are a chat. The engine works phase by phase and parks between phases for your approval; you steer in plain words. - -0. **Grounding** — reads the use-case and its cited criteria, follows the cites into the actual source text, and surveys what already exists for reuse. -1. **Invariant ACs** — promises that hold across every path, each with a complete, machine-checkable oracle. A promise whose expected result the source never states becomes a `missing-expected-result` gap with a recommended default rather than an invented answer. -2. **Scenarios** — technique-driven path expansion: happy, negative, boundary, edge, and — where the use-case warrants them — security, accessibility, performance, and i18n paths. Existing scenarios are linked, never duplicated; preconditions become dependencies. -3. **Path ACs + wiring** — per-scenario criteria, plus the record of which scenario exercises which invariant. An invariant nothing exercises becomes an `invariant-unexercised` gap. -4. **Tests** — exactly **one test per scenario** (strict 1:1). Pairs are scored and cut at the budget; each kept test carries a runnable body, a baseline-capture step where a check needs a before/after delta, and a written check whose expected value comes from its AC — a check that disagrees with its criterion is rejected, so a test can't quietly assert something weaker than the requirement. - -Between phases you steer in plain words: `looks good` (approve) · rename or correct an item (edit) · `drop 3` (reject) · `show 2` / `hide` (drill in) — plus the local slash commands `/explain ` (why an item exists — free, replayed from the record), `/done` (end the session), and `/pause` (save + exit `3`). The chat shell — the question panel, the composer grammar, ctrl+t, ctrl+c-to-pause — is exactly the one [extract uses](/support/docs/kane-cli-assurance-context/#the-interactive-chat). - -Headless modes run all phases without parking and emit one combined result; a high-risk question pauses an `agent`-mode run (resumable) and fails a `ci`-mode run closed. See [Automation](/support/docs/kane-cli-assurance-automation/). - -## What you get - -A design run commits to the graph **and writes files**. Each kept test lands as a normal, runnable `*_test.md` under `/.testmuai/tests/`: - -```markdown ---- -assurance: -id: t-add-one-in-stock-product-and-verify-minimum-valid-cart -base: sha256:00f8… ---- -# Add one in-stock product and verify minimum valid cart pricing - -> Prove the customer can create the minimum valid cart and see a line total and subtotal. - -## Step 1 - -Open {{store_url}} in a fresh browser session and navigate to the product listing… - -## Step 4 — assert @verifies ac-a-valid-cart-contains-at-least-1-item, ac-the-cart-displays-an-order-subtotal - -Confirm count check: 1 (equals) — the stated promise: after adding one in-stock product, the cart contains exactly 1 item. -``` - -Three things to notice: - -- **`@verifies` tags** bind each assert step to the acceptance criteria it proves. This is the link [`kane-cli cover`](/support/docs/kane-cli-assurance-coverage/) measures against — captured at authoring time, permanent, auditable. -- **`{{variables}}`** appear wherever the requirements didn't pin a value (the store URL, a known in-stock product). Each unknown is also recorded as a gap so it can't be forgotten. Supply values the normal way — see [Variables & context](/support/docs/kane-cli-variables-and-context/). -- The `assurance:` frontmatter links the file to its design entry in the graph, so coverage lookups are exact even after the file moves. - -Alongside the tests, the run records the ACs and scenarios themselves, the wiring between them, **gap nodes** with full context for everything it could not resolve, and a rationale sidecar per test (under `.context/design/rationale/`) that `design explain` replays. You'll also see **warnings** at commit time — for example when a test's `@verifies` list claims more criteria than its written check actually asserts. - -Design output is derived like everything else — review it with [`kane-cli context review`](/support/docs/kane-cli-assurance-context/#review), or browse it with [`kane-cli context view`](/support/docs/kane-cli-assurance-context/#inspect). - -### From design to execution - -A designed test is a normal test file — but it is still `derived`, and it has never been *run*. First review the design output like anything else the engine emits (approve, edit, or reject the generated ACs, scenarios, and tests with [`kane-cli context review`](/support/docs/kane-cli-assurance-context/#review) — the commit-time warnings resurface there). Then author each kept test once, and it batches like any other test: - -```bash -kane-cli testmd run .testmuai/tests/t-add-one-…_test.md # author it (first run, agent works it out) -kane-cli testrun run --match 't-' # from then on: batch replay -``` - -Until a test has been authored, `kane-cli testrun` preflight reports it as `missing_meta` and [`kane-cli cover`](/support/docs/kane-cli-assurance-coverage/) reads its criteria as covered-on-paper but unproven. That reading is deliberate — see [Coverage](/support/docs/kane-cli-assurance-coverage/#the-authoring-bridge). - -## Re-runs and `--force` {#re-runs-and-force} - -A use-case with a live design refuses a re-run, staleness-aware: - -``` -'uc-manage-the-cart' is already designed @ v1 — current; use --force to redesign -'uc-manage-the-cart' was designed @ v1 — the use-case is now @ v2 (STALE); use --force to redesign -``` - -`--force` regenerates the scenario+test pairs (superseding the old ones); ACs are dedup-first — an equivalent AC re-emitted by the engine reuses the existing node instead of piling up copies. When the staleness comes from a source document you just changed, [`kane-cli maintain reconcile`](/support/docs/kane-cli-assurance-maintain/) surfaces the same re-design as part of its changed-source triage; for staleness from older changes, [`kane-cli maintain evolve`](/support/docs/kane-cli-assurance-maintain/#evolve) re-designs the use-case with the blast radius stated first. - -## `design explain` — replay the why - -```bash -kane-cli design explain -``` - -Replays the recorded rationale — never a model call: - -- a **test** → the technique that produced it, the boundary values considered, the covering-array strength and why, the criteria it verifies and the scenario it automates; -- a **scenario / AC / gap** → its content plus every recorded judgement and review verdict. - -Ask it "why does this test exist?" six months later and the answer is the one recorded at design time, not a reconstruction. - -## Extending the technique catalog - -The design engine ships with an embedded catalog of test-design techniques and surface profiles. Drop replacement or additional YAML under `.context/design/` (same id overrides, new ids append) to extend it per store — no upgrade needed. - -## Exit codes - -| Code | Meaning | -|---|---| -| `0` | Design complete (or `--plan` transcription complete). | -| `1` | Runtime failure. | -| `2` | Usage / refusal — unknown use-case, already-designed without `--force`, bare non-TTY without `--mode`. | -| `3` | Session paused and resumable — see [sessions](/support/docs/kane-cli-assurance-context/#sessions). | - -## Next steps - -- [Coverage](/support/docs/kane-cli-assurance-coverage/) — measure what the designed tests prove. -- [Maintaining the suite](/support/docs/kane-cli-assurance-maintain/) — reconcile the suite when requirements change. -- [Automation](/support/docs/kane-cli-assurance-automation/) — headless design in CI or from an agent. diff --git a/static/docs/kane-cli-assurance-maintain.md b/static/docs/kane-cli-assurance-maintain.md deleted file mode 100644 index 19e1e1901..000000000 --- a/static/docs/kane-cli-assurance-maintain.md +++ /dev/null @@ -1,135 +0,0 @@ -# Maintaining the Suite as Sources Change - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Products change; tests shouldn't rot. `kane-cli maintain` closes the [assurance loop](/support/docs/kane-cli-assurance/): when a requirement document changes, `maintain reconcile` turns that one changed source into an honest, row-by-row update plan for your suite, and `maintain evolve` re-designs a use-case whose design went stale. Everything works over the same `.context/` store — maintain adds no new knowledge kinds, it moves the existing ones. - -```bash -kane-cli maintain reconcile --from --source-id # the interactive session (TTY default) -kane-cli maintain reconcile --from --source-id --plan # preview: stage + store the plan -kane-cli maintain reconcile --apply [path] # continue a stored plan -kane-cli maintain reconcile --from --source-id --mode agent # headless — see Automation -kane-cli maintain evolve [--because ""] # re-design one stale use-case (interactive) -kane-cli maintain evolve --from-stale # …or every use-case with stale designs -``` - -## `maintain reconcile` — one changed source, one triage {#reconcile} - -Reconcile is the on-change front door: a requirement document changed — what should the suite do about it? It re-ingests the source, re-extracts use-cases over the new snapshot, leads with a changeset (what the change did to your knowledge), and then triages the resulting rows with you. - -It takes **two explicit inputs** — reconcile never guesses which source a file belongs to: - -- `--from ` — the **new** version of the document (a file path). -- `--source-id ` — the **existing** source this file succeeds; its head moves. Find ids with `kane-cli context list --type source`. - -Both are required on a fresh run. `--apply ` alone is enough to continue a stored plan — the plan remembers its source. - -Hand reconcile the changed file directly — don't `context ingest` the new version first. Reconcile does the re-ingest itself, and [re-running the same command](#running-again) is always safe. - -### Fail-fast validations - -Before anything runs — no questions asked, nothing written, identical in every mode — reconcile validates its inputs, in order: - -1. both `--from` and `--source-id` are present; -2. the file exists and is a regular file; -3. it is an ingestable document type; -4. the source id names a known source; -5. that source isn't retired (restore it first with [`kane-cli context revert`](/support/docs/kane-cli-assurance-context/#housekeeping)); -6. the file doesn't already back a **different** live source — the fork guard: the error suggests the `--source-id` you probably meant, so one document's history never silently forks into another's. - -Any failure exits `2` with a message naming the next command to run. In `--mode agent`, validation failures ride the NDJSON stream (`error` + `done`), never stderr alone. - -### The changeset — what the change did - -Rendered first, before any actions: - -``` -changeset: 3 item(s) -[MODIFY] uc-manage-the-cart — updated: title, criteria -[ADD] uc-save-cart-for-later -[ARCHIVE] uc-legacy-flow — evidence decayed: no quote from the source relocates into the new text, no other live source, no fresh evidence this run -``` - -- **MODIFY** — the re-extract matched an existing use-case whose content moved. Each MODIFY knows *why*: a content change in the source, or a structural break the change caused. -- **ADD** — a use-case newly extracted from the changed source. -- **ARCHIVE** — a strict, three-part evidence decay: every quote fails to relocate into the new text, *and* no other live source evidences the node, *and* this run attached no fresh evidence. All three, or it isn't proposed for archiving. - -### The session (the default in a terminal) - -Interactive reconcile is a **card walk**: one ADD / MODIFY / ARCHIVE card at a time, each with its why — ARCHIVE cards carry the full evidence-decay reasoning, and MODIFY and ARCHIVE cards state their honest downstream cost up front (`impact: approving marks 14 item(s) stale`). While a card is up, every keystroke belongs to the card: - -- **Arrow keys** move through the options, **Enter** takes the highlighted one, and **digits** jump straight to an option. -- **Typing anything else opens an inline editor** seeded with your words — they become the steering the re-design sees, and approving applies both in one gesture. - -The verdicts per card: - -- **ADD / MODIFY** — **approve** (an ADD runs a design session for the new use-case right there; a MODIFY commits the update, or re-designs via [`maintain evolve`](#evolve) when the break is structural, blast radius stated first) · **reject** (drop the staged proposal) · **defer** (park it — the stored plan keeps it and a later run re-offers it) · or type to **steer** the re-design in your own words. -- **ARCHIVE** — **retire** (the explicit verdict; reversible any time with [`kane-cli context revert`](/support/docs/kane-cli-assurance-context/#housekeeping) — nothing is ever deleted) · **skip** · **defer**. - -After the last card the composer wakes: type ` ` to route one more re-design through the same session. **Nothing lands unapproved** — beyond recording the source change itself, everything a reconcile proposes is staged until you decide. Ctrl+C pauses cleanly (pending work lives in the stored plan, and the same reconcile command picks it back up), and the session ends with an honest summary of what was applied, rejected, deferred, and retired. - -### `--plan` — a preview that doesn't touch the suite - -`--plan` records the source change and **stages everything downstream**: the proposed rows are held in a stored plan (`plan stored: `, under `.context/reconcile/plans/`), and no tests or designs are touched. Two things do land, disclosed in the output: the head move (the change fact is true regardless of what you decide), and a matched use-case whose source content moved is updated as part of the re-extract itself. Every MODIFY and ARCHIVE row in the plan carries its impact line (`impact: approving marks N item(s) stale`), and a `skipped arms` line names every analysis this release does not run. - -Walk the plan later with `--apply ` — or bare `--apply`, which picks the latest plan behind an approval prompt (headless modes accept it silently). `--apply --from --source-id ` recomputes live instead. `--plan` and `--apply` together is a usage error (exit `2`). A repeated `--plan` re-renders the stored plan; an unchanged source is a truthful no-op (`nothing to reconcile`). - -### Running again — reconcile converges {#running-again} - -The same command is safe to repeat; it picks up where things stand: - -| State on a re-run | What happens | -|---|---| -| the file's bytes changed again | a fresh reconcile of the new change | -| unchanged, and the stored plan has pending rows | the plan is **resumed** in your chosen mode | -| unchanged, and the plan was fully applied | `already reconciled` — clean exit | -| unchanged, no stored plan | `nothing to reconcile` | -| the graph moved since the plan was stored | `graph moved since this plan — recomputing` (pending work is re-staged, not re-billed) | - -A plan stored by an earlier kane-cli version is refused with a hint to recompute — plans don't survive format changes silently. - -### Headless modes - -`--mode agent|ci|override` is the same ask-policy matrix extract and design use — see [Automation](/support/docs/kane-cli-assurance-automation/) for the full contract and reconcile's NDJSON stream. Two things are specific to reconcile: - -- Headless runs don't stage: the re-extract commits as it goes, and rows apply per mode — `override` and `ci` auto-apply ADD and MODIFY rows; `ci` fail-closes when a run needs human judgement; `agent` streams typed events and pauses. -- **Archiving is never automatic.** No headless mode archives anything; ARCHIVE decisions wait for an interactive session. - -A bare non-TTY run refuses (exit `2`) and asks for an explicit `--mode` — or `--plan` for a preview. - -### The rows - -| Kind | Fact behind it | Action on approve | -|---|---|---| -| `ADD` | a use-case newly extracted from the changed source, or an uncovered criterion of a touched use-case | a design run for that use-case (`kane-cli design tests --use-case `) | -| `MODIFY` | matched-but-changed content, or an entity whose pins this change broke | commit the update, or re-design via `kane-cli maintain evolve ` when the break is structural | -| `REMOVE` | a use-case now orphaned — no live source evidences it | plan-only — never executed in this release | - -## `maintain evolve` — re-design a stale use-case {#evolve} - -```bash -kane-cli maintain evolve [--because ""] # any designed entity → its parent use-case -kane-cli maintain evolve --from-stale # every use-case with stale designed entities -``` - -Evolve re-designs the **parent use-case** of whatever you point it at — a test, scenario, criterion, or the use-case itself. It is interactive-only, and the blast radius is always stated before anything runs; declining is a clean exit. - -- **Staleness-gated:** a fresh target refuses. `--because ""` is the sanctioned override — your reason becomes the change context the re-design sees, on the record. -- `--from-stale` collects every use-case with stale designed entities and walks them one confirm at a time. -- After a clean run, evolve reports the diff between the two design generations — what was superseded, what was minted, what was **retained** unchanged, and which criteria's verifying tests moved. A re-design doesn't break what it didn't change. -- Reconcile's MODIFY rows route here automatically — reach for evolve directly when staleness arrived outside a reconcile (an older change, a retired source). [`kane-cli cover gaps`](/support/docs/kane-cli-assurance-coverage/) lists stale designed entities in its ranked worklist. - -## Exit codes - -| Code | Meaning | -|---|---| -| `0` | Session, plan, or resume complete — or a friendly no-op (unchanged source). | -| `1` | The reconcile chain failed, or another live reconcile holds the lock (a dead run's lock clears itself — never delete it by hand). | -| `2` | Usage or validation failure — nothing was mutated. | -| `3` | Paused — pending work is in the stored plan; the same command (or `--apply`) continues it. | - -## Next steps - -- [Coverage](/support/docs/kane-cli-assurance-coverage/) — `cover gaps --stage design` is the standing worklist between reconciles. -- [Designing tests](/support/docs/kane-cli-assurance-design/) — what an approved ADD row actually runs. -- [Automation](/support/docs/kane-cli-assurance-automation/) — reconcile in CI, and its NDJSON stream. diff --git a/static/docs/kane-cli-assurance.md b/static/docs/kane-cli-assurance.md deleted file mode 100644 index 507d95445..000000000 --- a/static/docs/kane-cli-assurance.md +++ /dev/null @@ -1,83 +0,0 @@ -# The Assurance Lifecycle - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -kane-cli began as a way to author and replay browser tests. The **assurance** commands take on the step before and after: describe what your product must do, and kane-cli designs the tests that prove it — each one permanently linked to the requirement it verifies. Run them, and coverage stops being a guess: every run reports exactly what it proved and what it still owes. And as your product changes, the suite is reconciled instead of quietly rotting. - -Requires kane-cli **0.6.1 or later** (`kane-cli --version`). On 0.6.0 these commands fail after a fresh install — upgrade. - -## The loop - -``` -requirement docs product changes -│ │ -▼ ▼ -context ingest ──► context extract ──► context review ──► design tests -(snapshot the (agent proposes (promote to (ACs, scenarios, -sources) use-cases, cites trusted) one test per -every claim) scenario — written -as *_test.md files) -│ -▼ -maintain ◄── cover ◄── evidence pack ◄── testrun run ◄── testmd run ◄── context review -(reconcile (proven (sealed proof) (batch replay) (author each (approve the -a changed vs owed) test once) design output) -source) -``` - -Every stage is a separate command, so you can stop, review, and resume at any point — nothing downstream happens without the upstream commit. - -| Stage | Command | What it does | -|---|---|---| -| Capture | [`kane-cli context ingest`](/support/docs/kane-cli-assurance-context/#ingest) | Snapshot requirement documents into a local, content-addressed store (`.context/`) | -| Extract | [`kane-cli context extract`](/support/docs/kane-cli-assurance-context/#extract) | An agent reads the sources and proposes **use-cases**, citing the exact lines it read | -| Review | [`kane-cli context review`](/support/docs/kane-cli-assurance-context/#review) | You promote proposals to **trusted**, edit them, or reject them | -| Design | [`kane-cli design tests`](/support/docs/kane-cli-assurance-design/) | Turn one use-case into acceptance criteria, scenarios, and runnable tests — each test tagged with the criteria it verifies | -| Review the design | [`kane-cli context review`](/support/docs/kane-cli-assurance-context/#review) | Design output is `derived` too — approve, edit, or reject the generated ACs, scenarios, and tests | -| Execute | [`kane-cli testmd run`](/support/docs/kane-cli-testmd/), `kane-cli testrun run` | Author and replay the designed tests; every run seals an evidence pack | -| Measure | [`kane-cli cover`](/support/docs/kane-cli-assurance-coverage/) | Two axes: what a pack **proved** vs what the design still **owes** | -| Maintain | [`kane-cli maintain`](/support/docs/kane-cli-assurance-maintain/) | Reconcile the suite when a source document changes | - -## The vocabulary - -| Term | Meaning | -|---|---| -| **Source** | A requirement document you ingested — a PRD, a spec, a policy page. Content-addressed: editing the file and re-ingesting creates a new version. | -| **Use-case** | One thing a user needs to accomplish, extracted from sources with cited evidence. | -| **Acceptance criterion (AC)** | A single verifiable promise ("the cart holds at most 10 items"), with a machine-checkable oracle. | -| **Scenario** | One path through a use-case — happy, negative, boundary, edge. | -| **Test** | Exactly one runnable test per scenario (strict 1:1), written as a normal `*_test.md` file. | -| **derived / trusted / archived** | Trust states. Everything an agent proposes starts `derived` (unreviewed); your review promotes it to `trusted` or rejects it to `archived`. Nothing is silently trusted. | -| **fresh / stale / orphaned** | Freshness. When a source document changes, everything extracted from the old snapshot reads `stale` until re-verified; a node whose sources are all retired reads `orphaned`. | -| **Gap** | A recorded, ranked piece of missing coverage — a criterion no test verifies, a question nobody answered, a scenario cut by budget. Gaps are first-class output, not silence. | -| **Evidence pack** | The sealed `.evidence` file every run produces — the proof coverage is measured from. | - -## Assurance vs `generate` - -kane-cli has two ways to author tests, for two different jobs: - -- **[`kane-cli generate`](/support/docs/kane-cli-generate/)** — quick test cases from a plain-language description. One prompt in, scenarios and cases out. Great for exploring coverage of a feature you can describe in a sentence. -- **The assurance lifecycle** — tests derived from your actual requirement documents, with every claim cited, every proposal reviewed, and a permanent, auditable link from each test back to the criteria it verifies. Use it when you need to answer "what exactly is covered, and how do we know?" - -If you have a PRD and care about coverage accounting, start with assurance. If you want ten good test ideas in a minute, start with `generate`. - -## The store: `.context/` {#the-store-context} - -The assurance commands work over a local store in your project directory, created on first `ingest`: - -- It is **append-only**: nothing is ever deleted or rewritten. Edits create new versions; mistakes are reverted with compensation records. `kane-cli context explain` can replay the full history of any node. -- It is **yours and local**: sources, use-cases, designs, and review verdicts live in your project, not on a server. The extract and design agents run against the KaneAI service using your login, but the store they commit to is on your disk. -- **Keep `.context/` out of git merges.** The store is single-writer and not git-mergeable — two branches appending records will corrupt it on the next read. Gitignore it; share by re-ingesting sources. -- `kane-cli context fsck` verifies the whole store; `kane-cli context rebuild` regenerates the read caches from the verified records. - -## What costs credits - -`context extract`, `design tests`, and `maintain reconcile` (which embeds them) call the KaneAI agent and consume credits (`kane-cli balance` to check; each agent turn's cost is reported as it happens). Everything else — list, view, review, explain, cover, fsck — is local and free. - -## Next steps - -- [Building the context graph](/support/docs/kane-cli-assurance-context/) — ingest, extract, review. -- [Designing tests](/support/docs/kane-cli-assurance-design/) — from a use-case to runnable `*_test.md` files. -- [Coverage](/support/docs/kane-cli-assurance-coverage/) — proven vs owed, and how designed tests join execution. -- [Maintaining the suite](/support/docs/kane-cli-assurance-maintain/) — reconcile a changed source. -- [Automation](/support/docs/kane-cli-assurance-automation/) — running all of this headless in CI or from an agent. diff --git a/static/docs/kane-cli-authentication.md b/static/docs/kane-cli-authentication.md deleted file mode 100644 index f7167e879..000000000 --- a/static/docs/kane-cli-authentication.md +++ /dev/null @@ -1,115 +0,0 @@ -# Kane CLI Authentication - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Kane CLI authenticates against your TestMu AI account before it can run tests, upload sessions, or interact with Test Manager. There are two authentication methods: - -- **OAuth**: recommended for everyday local use. Opens a browser, you approve once, and tokens are stored on your machine. -- **Basic auth**: your TestMu AI username and access key. Use this in CI and other non-interactive environments where no browser is available. - -## OAuth Login - -```bash -kane-cli login --oauth -``` - -Kane CLI opens your default browser to the TestMu AI consent page. Sign in and approve the request. When the browser hands control back, Kane CLI stores your tokens locally and you are signed in. You usually do not need to log in again on the same machine. Kane CLI reuses the stored session on subsequent runs. - -If you run `kane-cli login` interactively without flags, Kane CLI launches a guided login wizard that walks you through choosing a method, profile, and (for basic auth) entering credentials. - -## Basic Auth - -Basic auth uses your TestMu AI username and account access key. It is the right choice for CI runners, Docker containers, and any other environment where opening a browser is not possible. - -### Per-Run Flags - -Pass credentials directly on the command you are running. They take precedence over any stored credentials: - -{`kane-cli run "Search for a product" \\ - --username "${ YOUR_LAMBDATEST_USERNAME()}" \\ - --access-key "${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -Both `kane-cli run` and `kane-cli feedback` accept `--username` and `--access-key`. - -### Persistent Basic Auth - -To save basic auth credentials for a profile so you do not have to pass them every time, log in with the basic auth flags: - -{`kane-cli login --username "${ YOUR_LAMBDATEST_USERNAME()}" --access-key "${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -Saved basic auth is used automatically for subsequent commands run under that profile. - -### Where to Find Your Access Key - -Sign in to the TestMu AI [dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard) > **Credentials** and copy your access key. Treat it like a password. Anyone with your username and access key can run tests on your account. - -## Profiles - -A profile is a named login. Profiles are useful when you work with more than one TestMu AI account or organisation, or when you want separate credentials for personal and team use without re-authenticating each time you switch. - -### Create a Profile - -Pass `--profile ` to `login` to authenticate under a named profile. If you omit `--profile`, Kane CLI uses the profile named `default`. - -```bash -kane-cli login --oauth --profile work -kane-cli login --oauth --profile personal -``` - -You can mix methods. One profile can use OAuth and another can use basic auth. - -### List Profiles - -```bash -kane-cli profiles list -``` - -Prints each saved profile with its environment, marking the active one. - -### Switch the Active Profile - -```bash -kane-cli profiles switch work -``` - -Subsequent commands run under `work` until you switch again. - -### Delete a Profile - -```bash -kane-cli profiles delete personal -``` - -Removes the stored credentials for that profile. - -### Run Against a Specific Profile Without Switching - -A few commands accept `--profile ` so you can target a profile for a single invocation without changing the active one. This is supported on `kane-cli login`, `kane-cli whoami`, and `kane-cli balance`. For other commands, use `kane-cli profiles switch` first. - -## Check Who You Are Logged In As - -```bash -kane-cli whoami -``` - -`whoami` prints an identity card showing the active profile, environment, authentication method (OAuth or basic), the username (when known), and (for OAuth) whether the stored token is valid, expired, or missing, along with its expiry date. - -Pass `--profile ` to inspect a profile other than the active one. - -## Log Out - -```bash -kane-cli logout -``` - -`logout` signs out of the active profile. For OAuth profiles, Kane CLI revokes the stored tokens with TestMu AI before deleting them. The on-disk credentials for the profile are removed. If you have other profiles configured, Kane CLI automatically switches to one of them; otherwise you are left with no active profile. - -## Where Credentials Are Stored - -Credentials live under your home directory: - -```text -~/.testmuai/kaneai/profiles///credentials -``` - -The file is created with restricted permissions (mode `0600`) so only your user account can read it. There is no need to inspect or edit this file by hand. Use `kane-cli login`, `kane-cli logout`, and the `kane-cli profiles` commands to manage it. diff --git a/static/docs/kane-cli-browser-state.md b/static/docs/kane-cli-browser-state.md deleted file mode 100644 index 51ccf4f21..000000000 --- a/static/docs/kane-cli-browser-state.md +++ /dev/null @@ -1,55 +0,0 @@ -# Browser State Actions - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Objectives can directly manage cookies, localStorage, and the clipboard, useful for seeding state before a flow (skip a login, dismiss a consent banner) and for testing copy/paste behavior. - -## Cookies - -``` -Set a cookie named session with value abc123 -Set cookies consent=yes and tracking=off, then reload the page -Delete the consent cookie -Clear all cookies -``` - -- Values accept `{{variables}}`: `set a cookie named session with value {{auth_token}}` -- A cookie without an explicit domain applies to the **current page's site**. Navigate first -- Reload or navigate after setting if the page must pick the cookie up -- Provide `path` together with a domain; a path alone is rejected by the browser - -## localStorage - -``` -Set localStorage keys theme=dark and lang=en -Delete the lang key from localStorage -Clear localStorage -``` - -- Storage is **per-site**. Navigate to the target site before setting -- Reload after setting if the app only reads storage on page load -- Values accept `{{variables}}` - -## Clipboard - -The run uses an **isolated test clipboard**. Your real OS clipboard is never read or written. Site Copy buttons are captured into it automatically. - -``` -Write "John Tester" to the clipboard -Click the message field, then paste from the clipboard -Clear the clipboard -``` - -- **Paste targets the focused field**: click or focus the field first, then paste (`Ctrl/Cmd+V` in an objective works the same way) -- Text and images both paste; rich editors receive a real paste event -- Typical flows: *write → click field → paste*, or *click the site's Copy button → click field → paste* - -## Verifying state - -Each of these has a matching assertion family. See [Cookies](/support/docs/kane-cli-checkpoint-devtools-cookies/), [localStorage](/support/docs/kane-cli-checkpoint-devtools-localstorage/), and [Clipboard](/support/docs/kane-cli-checkpoint-devtools-clipboard/): - -``` -Set a cookie named session with value abc123, reload, and verify the page shows you as logged in -Set localStorage theme=dark, reload, and verify the dark theme is active -Click the Copy link button, then verify the clipboard contains "/invoice/42" -``` diff --git a/static/docs/kane-cli-changelog.md b/static/docs/kane-cli-changelog.md deleted file mode 100644 index 480ebbfcb..000000000 --- a/static/docs/kane-cli-changelog.md +++ /dev/null @@ -1,34 +0,0 @@ -# Changelog - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Full release notes and version history are published on GitHub: - -**[github.com/LambdaTest/kane-cli/releases](https://github.com/LambdaTest/kane-cli/releases)** - -## Stay Updated - -Subscribe to releases on GitHub to get notified when a new version ships: - -1. Go to [github.com/LambdaTest/kane-cli](https://github.com/LambdaTest/kane-cli) -2. Click **Watch → Custom → Releases** - -Update to the latest version at any time: - -```bash -npm update -g @testmuai/kane-cli -``` - -Check your current version: - -```bash -kane-cli --version -``` - -## Reporting Issues - -Found a bug or unexpected behavior? Open an issue on GitHub: - -**[github.com/LambdaTest/kane-cli/issues](https://github.com/LambdaTest/kane-cli/issues)** - -See [Troubleshooting](/support/docs/kane-cli-troubleshooting/#filing-a-bug-report) for the information to include in your report. diff --git a/static/docs/kane-cli-checkpoint-devtools-clipboard.md b/static/docs/kane-cli-checkpoint-devtools-clipboard.md deleted file mode 100644 index 957aa86c3..000000000 --- a/static/docs/kane-cli-checkpoint-devtools-clipboard.md +++ /dev/null @@ -1,63 +0,0 @@ -# Clipboard Assertions - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Clipboard assertions let you verify what a "Copy" button actually copied, extract copied values into variables, and confirm clipboard state after your test writes or clears it. - -## The Test Clipboard - -Every run uses an **isolated test clipboard**: - -- **Your real clipboard is safe**: the OS clipboard is never read and never written. Tests can't leak your copied data into results, and nothing you copy mid-run interferes with the test -- **Automatic capture**: when the page copies something (a Copy button, Ctrl/Cmd+C on a selection), it lands in the test clipboard -- **One current entry**: like a real clipboard, every copy or clipboard write **replaces** the previous content -- **Multi-format**: a single entry can carry plain text, HTML, and an image together (e.g. a "Copy image" button) -- **Same behavior everywhere**: headless, headed, and CI runs all behave identically - -### Planning for Multi-Step Tests - -Because the clipboard holds one entry at a time: - -- Verify a copied value **immediately after** the copy or write that produced it. A later clipboard write/clear replaces it -- To check several copies, interleave: copy A → verify → copy B → verify - -## What You Can Query - -| Query | Description | -|-------|-------------| -| text content | The plain-text content of the clipboard ("" when empty) | -| HTML content | The rich-text representation, when present | -| formats present | Which content types the entry carries (text, HTML, image) | -| image presence/size | Whether an image was copied, and its byte size | - -## Example Assertions - -``` -Click the "Copy link" button, then verify the clipboard contains "/invoice/42" -Verify the clipboard text is "INV-2026-042" -Verify an image was copied to the clipboard -Verify the clipboard text is empty -``` - -## Example Extractions - -``` -Click the Copy button, store the copied coupon code as 'coupon' -Store the clipboard text as 'copied_link' -``` - -Stored values work like any other variable. Fill `{{coupon}}` into a field later, or assert on it. - -## Writing and Pasting (actions) - -The clipboard isn't read-only. Objectives can also drive it. See [Browser State Actions](/support/docs/kane-cli-browser-state/): - -``` -Write "John Tester" to the clipboard, click the message field, then paste from the clipboard -``` - -## Tips - -- **Don't paste to verify**: assert on the clipboard directly instead of pasting into a field and reading it back -- **Prefer positive phrasing** for empties: "verify the clipboard text is empty" rather than "verify X is not on the clipboard" -- Clipboard actions need a page loaded first. Navigate before writing or pasting diff --git a/static/docs/kane-cli-checkpoint-devtools-console.md b/static/docs/kane-cli-checkpoint-devtools-console.md deleted file mode 100644 index d8a3960ed..000000000 --- a/static/docs/kane-cli-checkpoint-devtools-console.md +++ /dev/null @@ -1,77 +0,0 @@ -# Console Assertions - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Console assertions let you verify browser console output: error messages, warnings, log messages, and uncaught JavaScript exceptions. - -## How Capture Works - -KaneAI captures all browser console output automatically during each test step: - -- **Continuous capture**: Every `console.log()`, `console.warn()`, `console.error()`, and uncaught JS exception is recorded -- **Per-step scope**: Each test step starts with a fresh capture. Console messages from previous steps are not carried over -- **Limits**: Up to 50,000 messages are stored per step. When the limit is reached, the oldest 10% are dropped -- **No truncation**: Message text is stored in full. Unlike network response bodies, console messages are never truncated -- **Object resolution**: When JavaScript logs an object (`console.log({status: "ok"})`), KaneAI resolves it to the actual value instead of storing "JSHandle@object" -- **Multi-tab support**: Console messages from new tabs and popups are also captured -- **Top-frame only**: Messages from embedded iframes (payment widgets, third-party components) are not captured - -### Planning for Multi-Step Tests - -Because console data resets each step, plan accordingly: - -- If you need to verify a console message **later**, extract and store it in the same step it appears -- Console output from step 1 won't be visible in step 3's console log - -### Level Normalization - -Console message levels are normalized to 5 values: - -| Level | What triggers it | -|-------|-----------------| -| `log` | `console.log()`, `console.dir()`, `console.table()`, and other info-level calls | -| `warning` | `console.warn()` | -| `error` | `console.error()` and uncaught exceptions | -| `info` | `console.info()` | -| `debug` | `console.debug()` | - -## What You Can Query - -| Field | Type | Description | -|-------|------|-------------| -| `level` | string | Message level: "log", "warning", "error", "info", "debug" | -| `text` | string | Full message text (not truncated) | -| `url` | string | Source file URL | -| `line_number` | int | Source line number | -| `is_exception` | bool | True for uncaught JS exceptions (pageerror events) | -| `stack_trace` | string | Stack trace (exceptions only) | - -### Errors vs Exceptions - -- **`errors`** includes ALL error-level messages: both `console.error()` calls AND uncaught exceptions -- **`exceptions`** is a subset of errors: only uncaught JavaScript exceptions -- To check for app-level errors without exceptions: query errors where `is_exception` is false - -## Example Assertions - -``` -Assert: no console errors on the page -Assert: no uncaught JavaScript exceptions -Assert: console contains "Amplitude SDK triggered" -Assert: no console warnings -Assert: no JS errors after clicking Submit -``` - -## Example Extractions - -``` -Store all console error messages -Extract the first console error text -Store all console log output -``` - -## Example If/Else - -``` -If console contains "feature_flag_enabled" then use new flow, else use legacy flow -``` diff --git a/static/docs/kane-cli-checkpoint-devtools-cookies.md b/static/docs/kane-cli-checkpoint-devtools-cookies.md deleted file mode 100644 index 7817e2e0e..000000000 --- a/static/docs/kane-cli-checkpoint-devtools-cookies.md +++ /dev/null @@ -1,60 +0,0 @@ -# Cookies Assertions - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Cookie assertions let you verify browser cookies: check existence, values, and security attributes like httpOnly, secure, and sameSite. - -## How Capture Works - -Cookies are captured as a **point-in-time snapshot** when the checkpoint triggers: - -- **On-demand capture**: Cookies are read from the browser context at the moment the assertion runs, not accumulated over time -- **Current state only**: You see exactly what cookies exist right now, including any set by the page's JavaScript or server responses -- **All cookies visible**: Unlike `document.cookie` in JavaScript, KaneAI can see httpOnly cookies too -- **Domain-scoped**: Cookies are captured for all domains the browser has visited in this session - -### Planning for Multi-Step Tests - -Because cookies are captured at assertion time: - -- If you need to check cookies set by a specific page, assert on the **same page** or **after** you've visited it -- If cookies are needed in a later step (e.g., after navigating away), extract and store them first -- Cookies persist in the browser across steps (unlike network/console which reset), but asserting on a different domain may show different cookies - -## What You Can Query - -| Field | Type | Description | -|-------|------|-------------| -| `name` | string | Cookie name (e.g., "session_id") | -| `value` | string | Cookie value | -| `domain` | string | Domain (e.g., ".example.com") | -| `path` | string | Cookie path (e.g., "/") | -| `expires` | float | Expiry as epoch seconds (-1 for session cookies) | -| `http_only` | bool | True if HttpOnly flag is set | -| `secure` | bool | True if Secure flag is set | -| `same_site` | string | "Strict", "Lax", or "None" | - -## Example Assertions - -``` -Assert: a cookie named "session_id" exists -Assert: the session cookie is httpOnly -Assert: no cookies are set without the Secure flag -Assert: at least 3 cookies are set on the page -Assert: the auth cookie has sameSite set to "Strict" -``` - -## Example Extractions - -``` -Store all cookies -Extract the value of the "session_id" cookie -Store all cookie names -Get all cookies for the example.com domain -``` - -## Example If/Else - -``` -If a cookie named "auth_token" exists then go to dashboard, else go to login -``` diff --git a/static/docs/kane-cli-checkpoint-devtools-localstorage.md b/static/docs/kane-cli-checkpoint-devtools-localstorage.md deleted file mode 100644 index 7f495a650..000000000 --- a/static/docs/kane-cli-checkpoint-devtools-localstorage.md +++ /dev/null @@ -1,72 +0,0 @@ -# localStorage Assertions - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -localStorage assertions let you verify data stored in the browser's `window.localStorage`: check key existence, values, and item counts. - -## How Capture Works - -localStorage is captured as a **point-in-time snapshot** when the checkpoint triggers: - -- **On-demand capture**: localStorage is read from the current page at the moment the assertion runs -- **Current state only**: You see exactly what's in localStorage right now -- **Domain-scoped**: localStorage is per-origin (protocol + domain + port). You only see data for the current page's origin -- **String values**: All localStorage values are strings. If the application stores JSON objects, they're stored as JSON strings - -### Planning for Multi-Step Tests - -Because localStorage is captured at assertion time: - -- Assert on localStorage while you're **on the page** that set the values. Navigating to a different domain means a different localStorage -- If values are needed later, extract and store them before navigating away -- localStorage persists across steps (unlike network/console) as long as you stay on the same origin - -### JSON Values - -Applications often store structured data in localStorage as JSON strings: - -```javascript -// Application code -localStorage.setItem("user_prefs", JSON.stringify({theme: "dark", lang: "en"})); -``` - -In assertions, the value is the raw JSON string. You can parse it to check individual fields: - -``` -Assert: the "theme" field in the user_prefs localStorage item is "dark" -``` - -KaneAI will parse the JSON and drill into the value automatically. - -## What You Can Query - -| Method | Returns | Description | -|--------|---------|-------------| -| `storage.all()` | dict | All key-value pairs | -| `storage.get(key)` | string or None | Value for a specific key | -| `storage.keys()` | list of strings | All key names | -| `storage.has(key)` | bool | Whether a key exists | - -## Example Assertions - -``` -Assert: auth_token exists in localStorage -Assert: the theme preference in localStorage is "dark" -Assert: localStorage has fewer than 10 items -Assert: the user_id value in localStorage is not empty -``` - -## Example Extractions - -``` -Store all localStorage items -Extract the auth_token from localStorage -Store the user preferences from localStorage -Get all localStorage keys -``` - -## Example If/Else - -``` -If localStorage has "onboarding_complete" then show dashboard, else start onboarding -``` diff --git a/static/docs/kane-cli-checkpoint-devtools-network.md b/static/docs/kane-cli-checkpoint-devtools-network.md deleted file mode 100644 index 9c3596222..000000000 --- a/static/docs/kane-cli-checkpoint-devtools-network.md +++ /dev/null @@ -1,67 +0,0 @@ -# Network Assertions - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Network assertions let you verify HTTP traffic: API responses, status codes, headers, response bodies, and request timing. - -## How Capture Works - -KaneAI captures all HTTP network traffic automatically during each test step: - -- **Continuous capture**: Every HTTP request and response is recorded as it happens -- **Per-step scope**: Each test step starts with a fresh capture. Traffic from previous steps is not carried over -- **Limits**: Up to 5,000 requests are stored per step. When the limit is reached, the oldest 10% of entries are dropped to make room. Response bodies are capped at 64KB per entry -- **Text bodies only**: Response bodies are captured for text-based content types (JSON, HTML, XML, CSS, JavaScript). Binary content (images, fonts, videos) is skipped -- **Multi-tab support**: Traffic from new tabs and popups is also captured - -### Planning for Multi-Step Tests - -Because network data resets each step, plan accordingly: - -- If you need to assert on an API response **later**, extract and store it in the same step the request happens -- Navigation and API calls in step 1 won't be visible in step 3's network log -- Use extraction checkpoints to save values across steps - -## What You Can Query - -| Field | Type | Description | -|-------|------|-------------| -| `method` | string | HTTP method (GET, POST, PUT, DELETE, ...) | -| `url` | string | Full request URL | -| `domain` | string | Domain (e.g., "api.example.com") | -| `path` | string | URL path without query string | -| `query_params` | dict | Query parameters | -| `resource_type` | string | xhr, fetch, document, script, image, ... | -| `request_headers` | dict | Request headers | -| `request_body` | string | Request body (may be truncated) | -| `response_status` | int | HTTP status code (200, 404, 500, ...) | -| `response_headers` | dict | Response headers | -| `response_body` | string | Response body (text types only, may be truncated) | -| `timing.duration_ms` | float | Total request duration in milliseconds | -| `timing.ttfb_ms` | float | Time to first byte in milliseconds | -| `failed` | bool | True if request failed at network level | -| `failure_reason` | string | Error reason (e.g., "net::ERR_CONNECTION_REFUSED") | - -## Example Assertions - -``` -Assert: no API calls returned 5xx status codes -Assert: the POST /api/login returned HTTP status 200 -Assert: all API responses completed in under 2 seconds -Assert: no network requests failed with connection errors -Assert: the /posts endpoint returned at least 10 items in the response body -``` - -## Example Extractions - -``` -Store the response body of the POST /api/login request -Extract the status code of the last API call to /api/users -Store all API request URLs -``` - -## Example If/Else - -``` -If the /api/auth returned 200 then proceed to dashboard, else show error message -``` diff --git a/static/docs/kane-cli-checkpoint-devtools-performance.md b/static/docs/kane-cli-checkpoint-devtools-performance.md deleted file mode 100644 index 4a6bfac4a..000000000 --- a/static/docs/kane-cli-checkpoint-devtools-performance.md +++ /dev/null @@ -1,61 +0,0 @@ -# Performance Assertions - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Performance assertions let you verify [Core Web Vitals](https://web.dev/articles/vitals) and other key performance metrics for the current page. - -## How Capture Works - -Performance data is **navigation-based**. Metrics are measured for the most recent page navigation: - -- **Automatic measurement**: KaneAI uses the [web-vitals](https://github.com/GoogleChrome/web-vitals) library to capture metrics -- **Per-navigation scope**: Metrics reflect the last full page load. If you navigate to a new page, the metrics reset for that navigation -- **Point-in-time snapshot**: When a performance checkpoint triggers, KaneAI captures the current metrics at that moment - -### What This Means for Your Tests - -- Performance metrics describe the **last navigation**. If you navigate to Page A then Page B, the metrics reflect Page B -- Place performance assertions **after** the page you want to measure has fully loaded -- Use a wait step if the page needs time to settle before measuring - -## Available Metrics - -| Metric | What It Measures | Good Threshold | Learn More | -|--------|-----------------|----------------|------------| -| **LCP** | Largest Contentful Paint: when the largest visible element finishes rendering | < 2,500ms | [web.dev/lcp](https://web.dev/articles/lcp) | -| **CLS** | Cumulative Layout Shift: visual stability, how much the page layout shifts | < 0.1 | [web.dev/cls](https://web.dev/articles/cls) | -| **INP** | Interaction to Next Paint: responsiveness to user input | < 200ms | [web.dev/inp](https://web.dev/articles/inp) | -| **FCP** | First Contentful Paint: when the first content appears on screen | < 1,800ms | [web.dev/fcp](https://web.dev/articles/fcp) | -| **TTFB** | Time to First Byte: server response time | < 800ms | [web.dev/ttfb](https://web.dev/articles/ttfb) | - -> **Note**: Not all metrics are available for every page. INP requires user interaction to trigger. Some metrics may be `null` if the browser hasn't measured them yet. - -## Example Assertions - -``` -Assert: page LCP is under 2500ms -Assert: CLS is below 0.1 -Assert: TTFB is under 800ms -Assert: FCP is less than 1800ms -Assert: page performance meets Core Web Vitals thresholds -``` - -## Example Extractions - -``` -Store the page LCP value -Extract all web vitals metrics -Store the TTFB for this page -``` - -## Example If/Else - -``` -If LCP is under 2500ms then continue, else report performance issue -``` - -## Tips - -- **Wait for load**: Place a wait step before performance assertions to ensure the page has fully loaded and metrics are available -- **Navigate first**: Metrics are per-navigation. Make sure you've navigated to the target page before asserting -- **Not all metrics are instant**: CLS accumulates over time, INP requires interaction. LCP and FCP are typically available after the page visually completes loading diff --git a/static/docs/kane-cli-checkpoint-devtools.md b/static/docs/kane-cli-checkpoint-devtools.md deleted file mode 100644 index b1520a415..000000000 --- a/static/docs/kane-cli-checkpoint-devtools.md +++ /dev/null @@ -1,51 +0,0 @@ -# DevTools Assertions - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -DevTools assertions let you verify data that isn't visible on the page: HTTP network traffic, browser console output, performance metrics, cookies, localStorage, and clipboard. KaneAI captures this data automatically in the background; you just write what to check. - -## Available Domains - -| Domain | What It Captures | Documentation | -|--------|-----------------|---------------| -| [Network](/support/docs/kane-cli-checkpoint-devtools-network/) | HTTP requests and responses | Status codes, headers, response bodies, timing | -| [Console](/support/docs/kane-cli-checkpoint-devtools-console/) | Browser console messages | Errors, warnings, log messages, JS exceptions | -| [Performance](/support/docs/kane-cli-checkpoint-devtools-performance/) | Core Web Vitals | LCP, CLS, INP, FCP, TTFB | -| [Cookies](/support/docs/kane-cli-checkpoint-devtools-cookies/) | Browser cookies | Names, values, flags (httpOnly, secure, sameSite) | -| [localStorage](/support/docs/kane-cli-checkpoint-devtools-localstorage/) | Browser localStorage | Key-value pairs stored in the browser | -| [Clipboard](/support/docs/kane-cli-checkpoint-devtools-clipboard/) | Browser clipboard | Copied text, HTML, image, and formats present | - -## How It Works - -Each DevTools domain follows the same pattern: - -1. **Capture**: KaneAI captures the data automatically during your test run -2. **Generate**: When a checkpoint triggers, the AI generates code to query the captured data -3. **Execute**: The code runs in an isolated sandbox and returns a result -4. **Assert**: The result is compared against your expected value - -You don't write code. You write natural language objectives, and KaneAI handles the rest. - -## Examples - -``` -Assert: no API calls returned 5xx -Assert: no console errors on the page -Assert: page LCP is under 2500ms -Assert: session cookie exists and is httpOnly -Assert: auth_token is stored in localStorage -``` - -## All Checkpoint Types Work - -DevTools assertions support all three checkpoint types: - -- **Assert**: "Assert: no console errors", fails the test if there are errors -- **Extract**: "Store all cookies", saves the data for later steps -- **If/Else**: "If the API returned 200 then proceed, else retry", branch on the result - -## Important Notes - -- DevTools data is **not visible in a screenshot**. KaneAI will never try to open the browser DevTools panel -- Each domain captures data differently. See individual pages for details on timing and scope -- All assertions run in an isolated sandbox. Generated code cannot access the file system or network diff --git a/static/docs/kane-cli-checkpoint-textual.md b/static/docs/kane-cli-checkpoint-textual.md deleted file mode 100644 index 7b3647f0e..000000000 --- a/static/docs/kane-cli-checkpoint-textual.md +++ /dev/null @@ -1,45 +0,0 @@ -# Textual (DOM) Assertions - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Textual assertions extract data from the page's DOM: element states, attributes, and computed styles that aren't always visible in a screenshot. - -## When It's Used - -- Element states: disabled, enabled, checked, readonly, expanded -- CSS properties with exact values: `font-size: 16px`, `opacity: 0.5`, `display: none` -- HTML attributes: `placeholder`, `aria-*`, `data-*`, `class`, `id`, `href`, `src`, `type`, `value` -- Attribute existence: "has placeholder", "has aria-label" -- Exact CSS color values: `rgb(255,0,0)`, `#ff0000` - -## Examples - -### Assertions - -``` -Assert: the submit button is disabled -Assert: the checkbox is checked -Assert: the input field has placeholder "Enter email" -Assert: the element has aria-label "Close dialog" -Assert: the font-size of the heading is 24px -``` - -### Extractions - -``` -Extract the href of the first link -Store the value attribute of the email input -Get the class of the error message element -``` - -## When NOT to Use - -- For visible text content (prices, labels) → use [Visual](/support/docs/kane-cli-checkpoint-visual/) -- For color names like "red background" → use [Visual](/support/docs/kane-cli-checkpoint-visual/) (DOM may return `transparent` or inherited values) -- For network/console/cookie data → use [DevTools](/support/docs/kane-cli-checkpoint-devtools/) - -## How It Works - -1. KaneAI captures the DOM snapshot of the page -2. The AI model identifies the target element and extracts the requested property -3. The value is compared against the expected value diff --git a/static/docs/kane-cli-checkpoint-title.md b/static/docs/kane-cli-checkpoint-title.md deleted file mode 100644 index 14dcd65f2..000000000 --- a/static/docs/kane-cli-checkpoint-title.md +++ /dev/null @@ -1,31 +0,0 @@ -# Title Assertions - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Title assertions check the browser tab's document title (`document.title`). - -## When It's Used - -- Page title verification: "title contains Dashboard" -- Navigation confirmation: "title is Home Page" - -## Examples - -### Assertions - -``` -Assert: page title contains "Dashboard" -Assert: title is "My Account - Settings" -``` - -### Extractions - -``` -Store the page title -``` - -## How It Works - -1. KaneAI reads `page.title()` directly -2. The title string is compared against the expected value -3. No screenshot or DOM analysis needed, this is a direct read diff --git a/static/docs/kane-cli-checkpoint-url.md b/static/docs/kane-cli-checkpoint-url.md deleted file mode 100644 index 26339017f..000000000 --- a/static/docs/kane-cli-checkpoint-url.md +++ /dev/null @@ -1,41 +0,0 @@ -# URL Assertions - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -URL assertions check values in the browser's address bar: the current URL path, query parameters, fragments, and redirect targets. - -## When It's Used - -- URL path: "URL contains /checkout" -- Query parameters: "URL has param `sort=price`" -- Redirect verification: "redirected to /login" -- Fragment/hash: "URL hash is #section-2" - -## Examples - -### Assertions - -``` -Assert: URL contains /checkout -Assert: the page redirected to /dashboard -Assert: URL path is /products/42 -``` - -### Extractions - -``` -Store the current URL -Extract the URL path -``` - -### If/Else - -``` -If URL contains /login then enter credentials, else go to profile -``` - -## How It Works - -1. KaneAI reads the current `page.url` value directly -2. The URL string is compared against the expected value using the specified operator -3. No screenshot or DOM analysis needed, this is a direct read diff --git a/static/docs/kane-cli-checkpoint-visual.md b/static/docs/kane-cli-checkpoint-visual.md deleted file mode 100644 index 4d4557b7c..000000000 --- a/static/docs/kane-cli-checkpoint-visual.md +++ /dev/null @@ -1,50 +0,0 @@ -# Visual Assertions - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Visual assertions verify what's visible on screen by analyzing the current screenshot. This is the default method, when in doubt, KaneAI uses visual analysis. - -## When It's Used - -- Text content: prices, labels, headings, counts, messages -- Visibility: "is the login button visible", "are search results displayed" -- Color checks using color names: "has red background", "blue text" -- Any content that appears visually on the page - -## Examples - -### Assertions - -``` -Assert: the product price is $29.99 -Assert: the search results show at least 5 items -Assert: the error message is visible -Assert: the hero section displays "Welcome back" -``` - -### Extractions - -``` -Store the product price -Extract the heading text -Get the number of items in the cart -``` - -### If/Else - -``` -If the login button is visible then click it, else click Sign Up -``` - -## How It Works - -1. KaneAI takes a screenshot of the current page -2. The AI model analyzes the screenshot to find the requested information -3. The extracted value is compared against the expected value using the specified operator - -## Best Practices - -- Use visual assertions for any text or content you can see on screen -- Be specific about what to look for: "the price in the cart summary" not just "the price" -- For exact CSS values (like `rgb(255,0,0)` or `#ff0000`), use [Textual (DOM)](/support/docs/kane-cli-checkpoint-textual/) instead -- For element states (disabled, checked), use [Textual (DOM)](/support/docs/kane-cli-checkpoint-textual/) instead diff --git a/static/docs/kane-cli-checkpoints.md b/static/docs/kane-cli-checkpoints.md deleted file mode 100644 index 2bd0bfc12..000000000 --- a/static/docs/kane-cli-checkpoints.md +++ /dev/null @@ -1,74 +0,0 @@ -# Checkpoints - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Checkpoints are verification points that KaneAI evaluates during test execution. They let you assert conditions, branch on results, or extract values for later use. - -## Checkpoint Types - -| Type | What it does | -|------|-------------| -| **Assertion** | Verify a condition is true, fails the test if not | -| **If/Else** | Branch execution based on a condition | -| **Extraction** | Store a value for use in later steps | - -All three types work with every analyze method below. - -## Analyze Methods - -Each checkpoint uses an analyze method to determine *where* to look for the data: - -| Method | Data Source | When to Use | -|--------|-----------|-------------| -| [Visual](/support/docs/kane-cli-checkpoint-visual/) | Screenshot (what you see on screen) | Text, labels, prices, counts, colors, visibility checks | -| [Textual (DOM)](/support/docs/kane-cli-checkpoint-textual/) | Page DOM elements | Element states (disabled, checked), CSS properties, HTML attributes | -| [URL](/support/docs/kane-cli-checkpoint-url/) | Browser URL bar | URL path, query params, redirects | -| [Title](/support/docs/kane-cli-checkpoint-title/) | Page title | Document title verification | -| [DevTools](/support/docs/kane-cli-checkpoint-devtools/) | Browser internals | Network traffic, console logs, performance, cookies, localStorage, clipboard | - -## How to Use - -Write your assertions naturally in the objective. KaneAI automatically picks the right analyze method: - -``` -Assert: the price is $29.99 → Visual -Assert: the submit button is disabled → Textual (DOM) -Assert: URL contains /checkout → URL -Assert: page title contains "Dashboard" → Title -Assert: no API calls returned 5xx → DevTools (Network) -Assert: no console errors → DevTools (Console) -Assert: page LCP is under 2500ms → DevTools (Performance) -Assert: session cookie exists → DevTools (Cookies) -Assert: auth_token exists in localStorage → DevTools (localStorage) -Assert: the clipboard has the copied link → DevTools (Clipboard) -``` - -Extractions work the same way: - -``` -Store the product price → Visual -Store the current URL → URL -Store all cookies → DevTools (Cookies) -Store the API response body → DevTools (Network) -``` - -## Operators - -Assertions support these comparison operators: - -| Operator | Meaning | Example | -|----------|---------|---------| -| `equals` | Exact match | price equals "29.99" | -| `contains` | Substring match | URL contains "/checkout" | -| `not_contains` | Does not contain | title not contains "Error" | -| `gt` / `gte` | Greater than / or equal | items greater than 5 | -| `lt` / `lte` | Less than / or equal | LCP less than 2500 | -| `not_equals` | Not equal | status not equals "failed" | - -## Learn More - -- [Visual Assertions](/support/docs/kane-cli-checkpoint-visual/): screenshot-based text and visibility checks -- [Textual (DOM) Assertions](/support/docs/kane-cli-checkpoint-textual/): element states and attributes -- [URL Assertions](/support/docs/kane-cli-checkpoint-url/): URL-based checks -- [Title Assertions](/support/docs/kane-cli-checkpoint-title/): page title checks -- [DevTools Assertions](/support/docs/kane-cli-checkpoint-devtools/): network, console, performance, cookies, localStorage, clipboard diff --git a/static/docs/kane-cli-cicd.md b/static/docs/kane-cli-cicd.md deleted file mode 100644 index fad9a3af0..000000000 --- a/static/docs/kane-cli-cicd.md +++ /dev/null @@ -1,281 +0,0 @@ -# CI/CD Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Kane CLI runs headlessly in CI/CD pipelines using credentials passed as environment variables or inline flags. Tests fail fast on assertion errors and return standard exit codes for pipeline control flow. - -## Common Patterns - -These patterns apply to every CI system; the platform-specific recipes below differ only in how they wire up the secrets. - -- **Always pass `--headless`**. CI runners have no display. -- **Always set `--timeout `**. A hung run cannot be allowed to block the pipeline. -- **Authenticate with `--username` and `--access-key`** from CI secrets. Do not call `kane-cli login` in CI. That flow opens a browser for OAuth and will not work on a runner. -- **Load test data with `--variables-file `**. Check the file into your repo (without secret values), or generate it before the step. -- **Check the exit code**. `0` passed, `1` failed, `2` error, `3` timeout or cancellation. - -**Chrome Requirement** -The runner spawns Chrome itself, so the CI image must have Chrome available on `PATH`. If your runner image cannot install Chrome, point Kane CLI at a remote browser with `--cdp-endpoint ` or `--ws-endpoint ` (for example, a TestMu AI `wss://` endpoint). - -## Authentication in CI/CD - -Pass credentials directly on the run command using environment variables from your secrets store: - -{`kane-cli run "Verify checkout flow completes" \\ - --url https://staging.myapp.com \\ - --username "${ YOUR_LAMBDATEST_USERNAME()}" \\ - --access-key "${ YOUR_LAMBDATEST_ACCESS_KEY()}" \\ - --headless \\ - --agent \\ - --timeout 300`} - -Get your username and access key from the TestMu AI [dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard) > **Credentials**. - -## Exit Codes - -| Code | Meaning | Pipeline Behavior | -|------|---------|-------------------| -| `0` | Test passed | Pipeline continues | -| `1` | Test failed (assertion not met) | Pipeline stops | -| `2` | Error (auth failure, Chrome crash) | Pipeline stops | -| `3` | Timeout or cancelled | Pipeline stops | - -## CI/CD Checklist - -- Always use `--headless` and `--agent`: no display server in CI -- Set `--timeout`: prevents pipeline hangs (e.g. `--timeout 300`) -- Set `--max-steps`: caps run length (e.g. `--max-steps 50`) -- Use `--variables-file`: load test data from a committed config file -- Store credentials as secrets: never hardcode in pipeline files - -## Platform Guides - -Store `LT_USERNAME` and `LT_ACCESS_KEY` as repository secrets under **Settings > Secrets and variables > Actions**. - -```yaml -# .github/workflows/browser-tests.yml -name: Browser Tests -on: [push, pull_request] - -jobs: -kane-tests: -runs-on: ubuntu-latest -steps: -- uses: actions/checkout@v4 - -- uses: actions/setup-node@v4 -with: -node-version: '20' - -- name: Install Chrome -uses: browser-actions/setup-chrome@v1 - -- name: Install Kane CLI -run: npm install -g @testmuai/kane-cli - -- name: Run browser tests -env: -LT_USERNAME: ${{ secrets.LT_USERNAME }} -LT_ACCESS_KEY: ${{ secrets.LT_ACCESS_KEY }} -run: | -kane-cli run \ -"Search for 'wireless headphones' on Amazon and open the first result" \ ---headless \ ---timeout 300 \ ---username "$LT_USERNAME" \ ---access-key "$LT_ACCESS_KEY" \ ---variables-file ./tests/variables.json - -- name: Upload test logs -if: always() -uses: actions/upload-artifact@v4 -with: -name: kane-test-logs -path: ~/.testmuai/kaneai/sessions/ -``` - -Define `LT_USERNAME` and `LT_ACCESS_KEY` as masked CI/CD variables in your project settings. - -```yaml -# .gitlab-ci.yml -stages: -- test - -kane-cli: -stage: test -image: node:20 -before_script: -- apt-get update && apt-get install -y wget gnupg -- wget -qO- https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - -- echo "deb http://dl.google.com/linux/chrome/deb/ stable main" > /etc/apt/sources.list.d/google-chrome.list -- apt-get update && apt-get install -y google-chrome-stable -- npm install -g @testmuai/kane-cli -script: -- | -kane-cli run "Verify the homepage loads and the login button is visible" \ ---headless \ ---timeout 300 \ ---username "$LT_USERNAME" \ ---access-key "$LT_ACCESS_KEY" \ ---variables-file ./tests/variables.json -variables: -LT_USERNAME: $LT_USERNAME -LT_ACCESS_KEY: $LT_ACCESS_KEY -artifacts: -paths: -- ~/.testmuai/kaneai/sessions/ -when: always -expire_in: 7 days -``` - -Store credentials in **Jenkins > Manage Jenkins > Manage Credentials**. The two `credentials(...)` IDs refer to Username/Password or Secret Text credentials configured in Jenkins. - -```groovy -// Jenkinsfile -pipeline { -agent any -environment { -LT_USERNAME = credentials('lt-username') -LT_ACCESS_KEY = credentials('lt-access-key') -} -stages { -stage('Install') { -steps { -sh 'npm install -g @testmuai/kane-cli' -} -} -stage('Run kane-cli') { -steps { -sh ''' -kane-cli run "Sign in and confirm the dashboard renders" \ ---headless \ ---timeout 300 \ ---username "$LT_USERNAME" \ ---access-key "$LT_ACCESS_KEY" \ ---variables-file ./tests/variables.json -''' -} -} -} -post { -always { -archiveArtifacts artifacts: '~/.testmuai/kaneai/sessions/**', -allowEmptyArchive: true -} -} -} -``` - -The pipeline fails on any non-zero exit code from the `sh` step, which matches Kane CLI's exit-code semantics. - -Add `LT_USERNAME` and `LT_ACCESS_KEY` as repository variables under **Repository settings > Repository variables**. - -```yaml -# bitbucket-pipelines.yml -pipelines: -default: -- step: -name: Browser Tests -image: node:20 -script: -- npm install -g @testmuai/kane-cli -- kane-cli run ---url https://staging.myapp.com ---username $LT_USERNAME ---access-key $LT_ACCESS_KEY ---headless ---agent ---timeout 300 ---max-steps 50 -"Complete the checkout flow and verify order confirmation" -artifacts: -- ~/.testmuai/kaneai/sessions/** -``` - -The shell command below works in any CI that can run a Linux container with Chrome installed: - -```bash -kane-cli run "Open the pricing page and verify the Pro plan is listed" \ ---headless \ ---timeout 300 \ ---username "$LT_USERNAME" \ ---access-key "$LT_ACCESS_KEY" \ ---variables-file ./tests/variables.json -``` - -If your CI image cannot install Chrome (for example, a minimal Node Alpine image), point Kane CLI at a remote browser instead: - -```bash -kane-cli run "Open the pricing page and verify the Pro plan is listed" \ ---headless \ ---timeout 300 \ ---ws-endpoint "$LT_BROWSER_WSS" \ ---username "$LT_USERNAME" \ ---access-key "$LT_ACCESS_KEY" \ ---variables-file ./tests/variables.json -``` - -`--cdp-endpoint ` works the same way for browsers that expose a Chrome DevTools Protocol URL. With either flag, Kane CLI skips its own Chrome launch and connects to the endpoint you provide. - -## Running Multiple Tests - -Run several tests and fail the pipeline if any fail: - -```bash -#!/bin/bash -set -e - -PASS=0 -FAIL=0 -FAILED_TESTS=() - -run_test() { -local name="$1" -local objective="$2" -echo "Running: $name" -if kane-cli run "$objective" \ ---url https://staging.myapp.com \ ---username $LT_USERNAME \ ---access-key $LT_ACCESS_KEY \ ---headless --agent --timeout 120; then -((PASS++)) -else -((FAIL++)) -FAILED_TESTS+=("$name") -fi -} - -run_test "Login" "Log in with valid credentials and verify dashboard appears" -run_test "Search" "Search for 'laptop' and verify at least one result appears" -run_test "Checkout" "Add first product to cart and complete checkout" -run_test "Settings" "Open account settings and verify profile page loads" - -echo "" -echo "Results: $PASS passed, $FAIL failed" -if [[ $FAIL -gt 0 ]]; then -echo "Failed tests: ${FAILED_TESTS[*]}" -exit 1 -fi -``` - -## Variables in CI/CD - -Commit a non-secret variables file to your repo, and inject secrets at runtime: - -```json -{ -"app_url": { "value": "https://staging.myapp.com" }, -"test_product_sku": { "value": "PROD-001" } -} -``` - -Merge with secrets in your pipeline: - -```bash -kane-cli run "Log in as {{email}} with {{password}} and verify dashboard" \ ---variables-file ./test-variables.json \ ---variables "{\"email\": {\"value\": \"$TEST_EMAIL\"}, \"password\": {\"value\": \"$TEST_PASSWORD\", \"secret\": true}}" \ ---username $LT_USERNAME \ ---access-key $LT_ACCESS_KEY \ ---headless --agent -``` diff --git a/static/docs/kane-cli-cli-reference.md b/static/docs/kane-cli-cli-reference.md deleted file mode 100644 index ad1693756..000000000 --- a/static/docs/kane-cli-cli-reference.md +++ /dev/null @@ -1,211 +0,0 @@ -# CLI Reference - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Commands - -### `kane-cli --tui` - -Launch the interactive TUI. - -```bash -kane-cli --tui -``` - -### `kane-cli run` - -Run a browser automation test. - -```bash -kane-cli run "" [options] -``` - -| Flag | Description | Default | -|------|-------------|---------| -| `--url ` | Starting URL for the browser | Last used or configured URL | -| `--headless` | Run Chrome without a visible window | Off | -| `--max-steps ` | Maximum agent reasoning steps | `30` | -| `--timeout ` | Abort after N seconds | No limit | -| `--cdp-endpoint ` | Connect to an existing Chrome via CDP | Auto-launch Chrome | -| `--ws-endpoint ` | Connect via WebSocket (e.g. TestMu AI remote grid `wss://`) | Local Chrome | -| `--variables ` | Inline variable JSON | None | -| `--variables-file ` | Load variables from a JSON file | None | -| `--global-context ` | Override global context markdown | `~/.testmuai/kaneai/global-memory.md` | -| `--local-context ` | Override local context markdown | `.testmuai/context.md` | -| `--session-context ` | Prior runs context JSON | None | -| `--agent` | Output structured NDJSON (for AI coding agents) | Off | -| `--mode ` | Run mode: `action` (strict) or `testing` (lenient) | Config value, otherwise `testing` | -| `--code-export` | Generate code export after run uploads | Off | -| `--code-language ` | Code export language: `python` or `javascript` | `python` | -| `--skip-code-validation` | Skip post-codegen worker-side validation | On | -| `--no-skip-code-validation` | Force post-codegen worker-side validation | Off | -| `--username ` | Basic auth username (overrides stored profile) | N/A | -| `--access-key ` | Basic auth access key (overrides stored profile) | N/A | -| `--env ` | Environment (`prod`) | Active profile's env | - -### `kane-cli login` - -Authenticate with TestMu AI. Opens a browser for OAuth, or accepts credentials for basic auth. - -```bash -# OAuth (interactive) -kane-cli login [--profile ] - -# Basic auth (non-interactive) -kane-cli login --username --access-key [--profile ] -``` - -See [Authentication](/support/docs/kane-cli-authentication/) for details on profiles and auth methods. - -### `kane-cli logout` - -Revoke tokens and remove stored credentials for the active profile. - -```bash -kane-cli logout -``` - -### `kane-cli whoami` - -Show the active profile and authentication status. - -```bash -kane-cli whoami [--profile ] -``` - -Prints profile, environment, auth method, username, and token state (for OAuth). - -### `kane-cli balance` - -Show credit balance for the active profile. - -```bash -kane-cli balance [--profile ] -``` - -### `kane-cli profiles` - -Manage named authentication profiles. - -```bash -kane-cli profiles list # List all profiles -kane-cli profiles switch # Switch the active profile -kane-cli profiles delete # Delete a profile -``` - -See [Authentication: Profiles](/support/docs/kane-cli-authentication/#profiles) for details. - -### `kane-cli config` - -View and modify persistent settings. - -```bash -kane-cli config show # Show all settings -kane-cli config set-window x # Set browser window size -kane-cli config set-mode # Set run mode -kane-cli config chrome-profile [path] # Set Chrome profile (interactive picker if no path) -kane-cli config project [id] # Set Test Manager project (interactive picker if no id) -kane-cli config folder [id] # Set Test Manager folder (interactive picker if no id) -``` - -Commands without arguments (`chrome-profile`, `project`, `folder`) launch an interactive picker UI. AI agents cannot run these. Ask the user to run them directly. - -See [Configuration](/support/docs/kane-cli-configuration/) for the full settings reference. - -### `kane-cli feedback` - -Submit feedback on a completed test run. - -```bash -kane-cli feedback \ ---test-id \ ---feedback-type \ ---details "optional message" -``` - -## Exit Codes - -| Code | Meaning | -|------|---------| -| `0` | Test passed | -| `1` | Test failed (assertion not met) | -| `2` | Error (auth failure, Chrome crash, infra issue) | -| `3` | Timeout or cancelled | - -## TUI Slash Commands - -| Command | Args | Description | -|---------|------|-------------| -| `/run` | `"objective"` | Execute a test run | -| `/login` | `[--profile name]` | OAuth login | -| `/logout` | `[--profile name]` | Logout and revoke tokens | -| `/whoami` | `[--profile name]` | Show profile info | -| `/balance` | | Show credit balance | -| `/profiles` | `list\|switch\|delete` | Manage profiles | -| `/config` | `show\|set-window\|set-mode\|chrome-profile\|project\|folder` | Manage configuration | -| `/new` | | Start a fresh session (uploads current session first) | -| `/summary` | `[index]` | View detailed run summaries | -| `/cancel` | | Abort the current run | -| `/help` | | Show the command reference | -| `/clear` | | Clear chat history | -| `/exit` | | Quit Kane CLI | - -## Keyboard Shortcuts - -| Shortcut | Action | -|----------|--------| -| Enter | Submit objective | -| Ctrl+C | Cancel current run | -| Ctrl+C (twice) | Exit TUI | -| Ctrl+R | Reverse history search | -| Esc | Go back / close picker | -| Up / Down | Navigate menu or input history | -| Tab | Accept autocomplete | - -## Settings Reference - -| Setting | Default | Command | -|---------|---------|---------| -| `window_size` | `1920x1080` | `config set-window` | -| `default_url` | `https://kaneai-playground.lambdatest.io` | Internal default | -| `model` | `v16-alpha` | Internal default | -| `mode` | `testing` | `config set-mode` | -| `chrome_profile_path` | Empty (temporary per run) | `config chrome-profile` | -| `project_id` / `project_name` | None | `config project` | -| `folder_id` / `folder_name` | None | `config folder` | -| `code_export.enabled` | `false` | TUI menu or `--code-export` flag | -| `code_export.language` | `python` | `--code-language` flag | -| `code_export.skip_validation` | `true` | TUI menu or `--skip-code-validation` flag | - -Settings are stored at `~/.testmuai/kaneai/tui-config.json`. See [Configuration](/support/docs/kane-cli-configuration/) for details. - -## Directory Structure - -``` -~/.testmuai/kaneai/ -├── tui-config.json # Persistent settings -├── global-memory.md # Global agent context -├── chrome-profiles/ # Named Chrome user profiles -│ └── / -├── profiles/ # Stored credentials -│ └── / -│ └── / -│ └── credentials # OAuth tokens or basic auth (mode 0600) -├── sessions/ # All session history -│ └── / -│ ├── session.json # Session metadata and run list -│ ├── tui.log # Session event log -│ ├── runs/ -│ │ └── / # Per-run directory -│ │ └── run-test/ -│ │ ├── actions.ndjson -│ │ └── screenshots/ -│ └── code-export/ # Generated code (when enabled) -└── variables/ # Global variable files -└── *.json - -.testmuai/ # Project-local (in cwd) -├── context.md # Project-specific agent context -└── variables/ -└── *.json # Project-specific variables -``` diff --git a/static/docs/kane-cli-configuration.md b/static/docs/kane-cli-configuration.md deleted file mode 100644 index d66ac8364..000000000 --- a/static/docs/kane-cli-configuration.md +++ /dev/null @@ -1,180 +0,0 @@ -# Configuration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Kane CLI stores persistent settings at `~/.testmuai/kaneai/tui-config.json`. Most settings are managed through `kane-cli config` subcommands; a few are managed through interactive pickers in TUI mode, and one (code export) is toggled from the TUI menu. - -Authentication credentials are managed separately under `~/.testmuai/kaneai/profiles/`. See [Authentication](/support/docs/kane-cli-authentication/). - -## Viewing Settings - -Print the current configuration: - -```bash -kane-cli config show -``` - -The output groups settings under three headings: - -```text -Configuration - -Auth -method oauth | basic (user@example.com) | not configured -profile default -env prod - -Defaults -url https://kaneai-playground.lambdatest.io -model v16-alpha -mode testing -window 1920x1080 -project (none) -folder (none) - -Paths -chrome /Users/you/.testmuai/kaneai/chrome-profiles/work -``` - -Empty fields are shown as `(none)`. The `chrome` path is empty by default, in which case Kane CLI launches Chrome with a temporary profile each run. - -## Settings Reference - -| Field | Type | Default | Description | How to Change | -|-------|------|---------|-------------|---------------| -| `window_size.width` | integer | `1920` | Chrome window width in pixels (800–3840) | `kane-cli config set-window ` | -| `window_size.height` | integer | `1080` | Chrome window height in pixels (600–2160) | `kane-cli config set-window ` | -| `chrome_profile_path` | string | `""` | Path to a Chrome user-data dir. Empty means a fresh profile per run. | `kane-cli config chrome-profile [path]` | -| `default_url` | string \| null | `https://kaneai-playground.lambdatest.io` | Starting URL when a run begins. | `kane-cli config set-url ` | -| `model` | string | `"v16-alpha"` | Reasoning + vision model used by the agent. | Internal default | -| `project_id` | string \| null | `null` | TestMu AI Test Manager project ID for upload | `kane-cli config project [id]` | -| `project_name` | string \| null | `null` | Display name of the selected project | Set by `kane-cli config project` | -| `folder_id` | string \| null | `null` | TestMu AI Test Manager folder ID for upload | `kane-cli config folder [id]` | -| `folder_name` | string \| null | `null` | Display name of the selected folder | Set by `kane-cli config folder` | -| `mode` | `"action"` \| `"testing"` | `"testing"` | Agent behaviour on auth walls, blocked pages, or error pages. | `kane-cli config set-mode ` | -| `code_export.enabled` | boolean | `false` | Generate code export after upload completes. | TUI menu, or `--code-export` flag | -| `code_export.language` | `"python"` \| `"javascript"` | `"python"` | Output language for generated code. Accepts `python` or `javascript`. | `--code-language ` | -| `code_export.skip_validation` | boolean | `true` | Skip post-codegen worker-side validation. | TUI menu, or `--skip-code-validation` | - -## Updating Settings - -### Window Size - -The Chrome window is launched at the configured resolution: - -```bash -kane-cli config set-window 1280x800 -``` - -The format is `WIDTHxHEIGHT` (lowercase `x` separator). Width must be between 800 and 3840; height must be between 600 and 2160. Invalid values are rejected without changing the saved config. - -In TUI mode, the same setting can be edited through an interactive window-size picker. - -### Test Manager Project - -```bash -kane-cli config project -``` - -In a TTY, this opens an interactive project picker. The picker fetches the projects available to your active profile, lets you search and arrow-key through them, and saves the chosen `project_id` and `project_name`. Login is required before the picker can fetch projects. - -You can also set a project ID directly without the picker: - -```bash -kane-cli config project -``` - -See [Test Manager Integration](/support/docs/kane-cli-tms-integration/) for how project selection feeds into uploads. - -### Test Manager Folder - -```bash -kane-cli config folder -``` - -Opens an interactive folder picker for the currently selected project. Folders are searchable and shown with their hierarchy. You must have a project selected first. - -To set a folder ID without the picker: - -```bash -kane-cli config folder -``` - -See [Test Manager Integration](/support/docs/kane-cli-tms-integration/) for how folder selection feeds into uploads. - -### Mode - -```bash -kane-cli config set-mode action -kane-cli config set-mode testing -``` - -`mode` controls how the agent behaves when a run hits an authentication wall, a blocked page, or an error page: - -- **`testing`** (default): the agent treats those pages as part of the run and continues. Use this when you expect the agent to push through gates that would otherwise stop a real user. -- **`action`**: the agent hard-stops on authentication, blocked, and error pages so you can intervene manually before the run proceeds. - -You can override the saved mode for a single run with `--mode ` on `kane-cli run`. - -### Code Export - -The `code_export` block enables and configures generated code output produced after a successful Test Manager upload. There is no `kane-cli config` subcommand for this block. Set it from one of: - -- **The TUI**: open the config menu, choose Code Export, and toggle the `enabled` and `skip_validation` switches. -- **Per-run flags** on `kane-cli run`: - - `--code-export` to enable for this run only - - `--code-language ` to pick the output language (`python` or `javascript`) - - `--skip-code-validation` / `--no-skip-code-validation` to control post-codegen validation - -Code export requires a Test Manager upload, so it is only meaningful when a project is configured. See [Test Manager Integration](/support/docs/kane-cli-tms-integration/) for the full upload pipeline. - -## Chrome Management - -### Chrome Profile - -By default, `chrome_profile_path` is empty and Kane CLI launches Chrome with a fresh, temporary user-data directory each run. A clean per-run profile isolates cookies and storage between runs and prevents extensions, password autofill, or signed-in sessions from leaking into automation. - -When you select a named Chrome profile, Kane CLI stores it under `~/.testmuai/kaneai/chrome-profiles/` and reuses that directory across runs. This is useful when a test depends on having a logged-in session, a saved address, or a specific extension installed. - -### Choosing a Different Profile - -```bash -kane-cli config chrome-profile -``` - -In a TTY, this opens an interactive Chrome-profile picker. The picker lists every profile under `~/.testmuai/kaneai/chrome-profiles/` plus a "temporary" entry that clears the path back to empty (per-run fresh profiles). You can also create a new named profile from the picker. - -To set a path directly without the picker: - -```bash -kane-cli config chrome-profile /absolute/path/to/profile -``` - -### Headless Mode - -To run Chrome without a visible window, pass `--headless` on `kane-cli run`: - -```bash -kane-cli run "Verify the home page loads" --headless -``` - -Headless mode is per-run; there is no persistent setting. It is the right choice for CI and other environments without a display. - -### Window Size - -The Chrome window dimensions for both headed and headless modes come from the `window_size` setting. See [Window Size](#window-size) above to update them. - -## Resetting Settings - -There is no `kane-cli config reset` subcommand. To reset persistent settings to defaults, delete the config file: - -```bash -rm ~/.testmuai/kaneai/tui-config.json -``` - -Kane CLI recreates the file with defaults the next time it writes a setting. This only resets `tui-config.json`. It does **not** affect: - -- Authentication credentials under `~/.testmuai/kaneai/profiles/` (use `kane-cli logout`) -- Session history under `~/.testmuai/kaneai/sessions/` -- Variables under `~/.testmuai/kaneai/variables/` and `.testmuai/variables/` -- Chrome profiles under `~/.testmuai/kaneai/chrome-profiles/` diff --git a/static/docs/kane-cli-error-codes.md b/static/docs/kane-cli-error-codes.md deleted file mode 100644 index eaeb64f0d..000000000 --- a/static/docs/kane-cli-error-codes.md +++ /dev/null @@ -1,329 +0,0 @@ -# Error Codes Reference - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Every Kane CLI run ends with a `result_code` in the `run_end` event. This page lists every code, explains what triggered it, and tells you what to do next. - -## How to Read Result Codes - -Result codes follow HTTP-style groupings: - -| Range | Category | Meaning | -|-------|----------|---------| -| **1xx** | Success | The objective completed successfully. | -| **2xx** | Cancelled | The run was cancelled before completion. | -| **3xx** | Stuck | The agent could not make further progress. | -| **4xx** | Agent Error | An internal agent failure prevented completion. | -| **5xx** | Infra Error | A platform or infrastructure issue stopped the run. | -| **6xx** | Blocker | An external obstacle (CAPTCHA, paywall, etc.) blocked the agent. | -| **7xx** | Assertion Error | The objective's assertions or checkpoints failed. | - -The `run_end` event includes both `result_code` (numeric) and `reason_code` (descriptive string): - -```json -{ -"type": "run_end", -"status": "failed", -"result_code": 310, -"reason_code": "stuck.max_steps", -"summary": "Agent reached maximum step limit..." -} -``` - -## 1xx -- Success - -### 100 COMPLETE - -| Field | Value | -|-------|-------| -| **Reason code** | `success.complete` | -| **Status** | `COMPLETE` | -| **What happened** | All checkpoints passed and the agent confirmed the objective is achieved. | -| **Action** | None required. | - -## 2xx -- Cancelled - -### 200 USER_CANCELLED - -| Field | Value | -|-------|-------| -| **Reason code** | `cancelled.user` | -| **Status** | `CANCELLED` | -| **What happened** | The run was cancelled by the caller (e.g., Ctrl+C, `cancel` event, or API cancellation). | -| **Action** | No action required. Re-run the objective if needed. | - -## 3xx -- Stuck - -The agent ran into a dead end. These are not crashes - the agent recognized it could not make further progress. - -### 310 MAX_STEPS - -| Field | Value | -|-------|-------| -| **Reason code** | `stuck.max_steps` | -| **Status** | `STUCK` | -| **What happened** | The agent exceeded the configured `--max-steps` limit before completing the objective. | -| **Action** | Increase `--max-steps` (e.g., `--max-steps 60`), or break the objective into smaller, sequential runs. | - -### 320 DAG_CYCLE - -| Field | Value | -|-------|-------| -| **Reason code** | `stuck.dag_cycle` | -| **Status** | `STUCK` | -| **What happened** | The agent detected 3 repeated page-state cycles (it keeps returning to the same state after each action). | -| **Action** | Rephrase the objective with a different approach. The current path leads to a loop - try an alternative navigation route or add explicit intermediate steps. | - -### 330 AP_STUCK - -| Field | Value | -|-------|-------| -| **Reason code** | `stuck.ap_stuck` | -| **Status** | `STUCK` | -| **What happened** | The action planner determined there are no viable actions available on the current page to progress toward the objective. | -| **Action** | Manual intervention is needed. Check the screenshot at `{session_id}/runs/{run_dir}/run-test/screenshots/` to understand what page state the agent ended on. You may need to adjust the objective, provide credentials, or navigate to a different starting URL. | - -## 4xx -- Agent Error - -An internal agent component failed. These typically indicate a transient issue or a problem with the session state. - -### 410 AP_NO_ACTION - -| Field | Value | -|-------|-------| -| **Reason code** | `agent_error.ap_no_action` | -| **Status** | `ERROR` | -| **What happened** | The action planner returned 3 consecutive empty decisions (no action selected). | -| **Action** | Retry the run. If the error persists, simplify the objective or file a [bug report](https://github.com/LambdaTest/kane-cli/issues). | - -### 420 REASONING_FAILURE - -| Field | Value | -|-------|-------| -| **Reason code** | `agent_error.reasoning_failure` | -| **Status** | `ERROR` | -| **What happened** | An unhandled exception occurred in the reasoning engine. | -| **Action** | Check `{session_id}/runs/{run_dir}/run.log` for the full stack trace. Retry the run - this is usually transient. If it persists, file a [bug report](https://github.com/LambdaTest/kane-cli/issues) with the log file. | - -### 430 CLEAN_START_INVALID - -| Field | Value | -|-------|-------| -| **Reason code** | `agent_error.clean_start_invalid` | -| **Status** | `ERROR` | -| **What happened** | The `clean_start` validation failed - the agent could not establish a valid initial session state (e.g., the starting URL did not load, or the browser context is corrupted). | -| **Action** | Verify the `--url` is accessible and loads correctly. If using `--cdp-endpoint`, ensure the browser is in a clean state. | - -### 440 CHILD_FAILED - -| Field | Value | -|-------|-------| -| **Reason code** | `agent_error.child_failed` | -| **Status** | `ERROR` | -| **What happened** | A child agent (spawned for a sub-objective during analysis) failed. The parent run cannot continue. | -| **Action** | Check the `child_agent_end` event in the NDJSON output for the child's failure details. Fix the child objective, then re-run the parent. | - -## 5xx -- Infra Error - -Platform or infrastructure failures. These are independent of the objective itself. - -### 510 SCREENSHOT_FAILED - -| Field | Value | -|-------|-------| -| **Reason code** | `infra_error.screenshot_failed` | -| **Status** | `ERROR` | -| **What happened** | The browser failed to capture a screenshot (Playwright exception). The agent cannot proceed without visual input. | -| **Action** | Check that the browser is still running. If using `--cdp-endpoint`, verify the connection is alive. Restart the browser and retry. | - -### 520 BROWSER_CRASHED - -| Field | Value | -|-------|-------| -| **Reason code** | `infra_error.browser_crashed` | -| **Status** | `ERROR` | -| **What happened** | The browser process crashed or became unresponsive. | -| **Action** | Restart the browser and retry the run. If this happens repeatedly, check available system memory and ensure no other processes are competing for resources. | - -### 530 TIMEOUT - -| Field | Value | -|-------|-------| -| **Reason code** | `infra_error.timeout` | -| **Status** | `ERROR` | -| **What happened** | The run exceeded its configured timeout. | -| **Action** | Increase `--timeout` (in seconds): `--timeout 300`. If the objective is complex, break it into smaller runs. | - -### 540 EVENT_DISPATCH - -| Field | Value | -|-------|-------| -| **Reason code** | `infra_error.event_dispatch` | -| **Status** | `ERROR` | -| **What happened** | A custom event dispatch (e.g., browser event injection) failed. | -| **Action** | Check `{session_id}/runs/{run_dir}/run.log` for the failing event handler. This usually indicates a page script error or an incompatible browser state. Retry the run, and if it persists, file a [bug report](https://github.com/LambdaTest/kane-cli/issues). | - -### 550 CONTROLLER_AUTH - -| Field | Value | -|-------|-------| -| **Reason code** | `infra_error.controller_auth` | -| **Status** | `ERROR` | -| **What happened** | The TestMu AI controller returned a `401 Unauthorized` response. Your authentication token has expired or is invalid. | -| **Action** | Refresh your authentication token: | - -{`kane-cli login -# or re-run login with basic auth -kane-cli login --username "${ YOUR_LAMBDATEST_USERNAME()}" --access-key "${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -Verify with `kane-cli whoami`, then retry the run. - -### 560 CREDITS_EXHAUSTED - -| Field | Value | -|-------|-------| -| **Reason code** | `infra_error.credits_exhausted` | -| **Status** | `ERROR` | -| **What happened** | The TestMu AI controller returned a `402 Payment Required` response. Your account has no remaining credits for AI agent runs. | -| **Action** | Purchase additional credits or upgrade your plan on the [TestMu AI billing page](https://billing.lambdatest.com/billing/plans/). Contact your account administrator if you are on a team plan. | - -## 6xx -- Blocker - -An external obstacle on the target website prevented the agent from completing the objective. These are not agent bugs - they require changes to the environment or objective. - -### 610 CAPTCHA - -| Field | Value | -|-------|-------| -| **Reason code** | `blocker.captcha` | -| **Status** | `ERROR` | -| **What happened** | The page presented a CAPTCHA challenge that the agent cannot solve. | -| **Action** | Solve the CAPTCHA manually in an interactive session first, then re-run. Alternatively, use a test environment that has CAPTCHAs disabled, or allowlist the test IP. | - -### 620 PAYWALL - -| Field | Value | -|-------|-------| -| **Reason code** | `blocker.paywall` | -| **Status** | `ERROR` | -| **What happened** | The page requires a paid subscription to access the content the objective needs. | -| **Action** | Provide credentials for a subscribed account via [variables](/support/docs/kane-cli-variables-and-context/), or test against a non-paywalled environment. | - -### 640 BLOCKED - -| Field | Value | -|-------|-------| -| **Reason code** | `blocker.blocked` | -| **Status** | `ERROR` | -| **What happened** | The target site blocked access (IP ban, WAF rule, geo-restriction, or bot detection). | -| **Action** | Use a different IP address, allowlist the test IP on the target site, or use a different test account. Check whether the site has bot-detection that needs to be configured for testing. | - -### 650 ERROR_PAGE - -| Field | Value | -|-------|-------| -| **Reason code** | `blocker.error_page` | -| **Status** | `ERROR` | -| **What happened** | The page returned an HTTP error (404 Not Found, 500 Internal Server Error, etc.) instead of the expected content. | -| **Action** | Verify the `--url` is correct and the target site is operational. Check `{session_id}/runs/{run_dir}/run-test/screenshots/` to see the error page. | - -### 660 AUTH_NO_CREDENTIALS - -| Field | Value | -|-------|-------| -| **Reason code** | `blocker.auth_no_credentials` | -| **Status** | `ERROR` | -| **What happened** | The agent encountered a login page but no credentials were provided in the objective or variables. | -| **Action** | Provide login credentials using [variables](/support/docs/kane-cli-variables-and-context/): | - -```bash -kane-cli run "Log in and navigate to dashboard" \ ---url https://myapp.com/login \ ---variables '{"username": {"value": "test@example.com"}, "password": {"value": "s3cret", "secret": true}}' \ ---agent -``` - -## 7xx -- Assertion Error - -The agent completed its actions but one or more assertions or checkpoints did not pass. - -### 710 FINAL_VERIFICATION - -| Field | Value | -|-------|-------| -| **Reason code** | `assertion_error.final_verification` | -| **Status** | `ASSERTION_ERROR` | -| **What happened** | The final checkpoint failed. The agent performed all actions but the end state did not match the expected outcome. | -| **Action** | This means the objective was **not achieved**. Review the screenshot at `{session_id}/runs/{run_dir}/run-test/screenshots/` to see the final page state. The objective itself may need adjustment, or the application under test may have a bug. | - -### 720 CHECKPOINT_FAILED - -| Field | Value | -|-------|-------| -| **Reason code** | `assertion_error.checkpoint_failed` | -| **Status** | `ASSERTION_ERROR` | -| **What happened** | An inline checkpoint (mid-run assertion) failed. The agent detected that an intermediate condition was not met. | -| **Action** | Check which checkpoint failed in the step logs at `{session_id}/runs/{run_dir}/run-test/step_N.json`. The assertion condition may be too strict, or the application did not reach the expected state at that point. | - -### 730 NO_BRANCH_MATCHED - -| Field | Value | -|-------|-------| -| **Reason code** | `assertion_error.no_branch_matched` | -| **Status** | `ASSERTION_ERROR` | -| **What happened** | An if-else objective had multiple branches, but none of the conditions matched the actual page state. | -| **Action** | Review your branching conditions. Add a fallback branch, or check the screenshot to see what state the page was actually in. Adjust conditions to cover the observed state. | - -## Quick Lookup Table - -| Code | Name | Reason Code | Action | -|------|------|-------------|--------| -| 100 | COMPLETE | `success.complete` | Done | -| 200 | USER_CANCELLED | `cancelled.user` | Re-run if needed | -| 310 | MAX_STEPS | `stuck.max_steps` | Increase `--max-steps` or simplify | -| 320 | DAG_CYCLE | `stuck.dag_cycle` | Try a different approach | -| 330 | AP_STUCK | `stuck.ap_stuck` | Manual intervention needed | -| 410 | AP_NO_ACTION | `agent_error.ap_no_action` | Retry the run | -| 420 | REASONING_FAILURE | `agent_error.reasoning_failure` | Check logs, retry | -| 430 | CLEAN_START_INVALID | `agent_error.clean_start_invalid` | Fix session state | -| 440 | CHILD_FAILED | `agent_error.child_failed` | Check child agent details | -| 510 | SCREENSHOT_FAILED | `infra_error.screenshot_failed` | Check browser | -| 520 | BROWSER_CRASHED | `infra_error.browser_crashed` | Restart browser | -| 530 | TIMEOUT | `infra_error.timeout` | Increase `--timeout` | -| 540 | EVENT_DISPATCH | `infra_error.event_dispatch` | Check event handler | -| 550 | CONTROLLER_AUTH | `infra_error.controller_auth` | Run `kane-cli login` | -| 560 | CREDITS_EXHAUSTED | `infra_error.credits_exhausted` | Purchase credits | -| 610 | CAPTCHA | `blocker.captcha` | Solve manually or disable | -| 620 | PAYWALL | `blocker.paywall` | Provide subscription | -| 640 | BLOCKED | `blocker.blocked` | Change IP or account | -| 650 | ERROR_PAGE | `blocker.error_page` | Fix the URL | -| 660 | AUTH_NO_CREDENTIALS | `blocker.auth_no_credentials` | Provide credentials | -| 710 | FINAL_VERIFICATION | `assertion_error.final_verification` | Objective not achieved | -| 720 | CHECKPOINT_FAILED | `assertion_error.checkpoint_failed` | Check assertion condition | -| 730 | NO_BRANCH_MATCHED | `assertion_error.no_branch_matched` | Update branch conditions | - -## Handling Errors in Agent Mode - -When consuming Kane CLI output programmatically, use `result_code` ranges to determine your next action: - -```bash -# Run and capture the result -RESULT=$(kane-cli run "..." --agent --headless 2>/dev/null | tail -1) -CODE=$(echo "$RESULT" | jq -r '.result_code') - -case $CODE in -100) echo "Passed" ;; -2*) echo "Cancelled" ;; -3*) echo "Stuck: $(echo $RESULT | jq -r '.reason_code')" ;; -4*) echo "Agent error: retry recommended" ;; -5*) echo "Infra error: check environment" ;; -6*) echo "Blocked: manual intervention needed" ;; -7*) echo "Assertion failed: check test conditions" ;; -esac -``` - -## Related Resources - -- [Agent Mode](/support/docs/kane-cli-agent-mode/): NDJSON output format and event schema -- [Troubleshooting](/support/docs/kane-cli-troubleshooting/): Common issues and fixes -- [CLI Reference](/support/docs/kane-cli-cli-reference/): Full flag and command reference diff --git a/static/docs/kane-cli-generate-workflow.md b/static/docs/kane-cli-generate-workflow.md deleted file mode 100644 index e78d4da8f..000000000 --- a/static/docs/kane-cli-generate-workflow.md +++ /dev/null @@ -1,130 +0,0 @@ -# The Generate Workflow - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -`kane-cli generate` is built around a simple loop: **generate → refine → save → run**. Each command is one turn that exits when done; you move between turns with the request id. This page walks the loop end to end. For the feature overview and option reference, see [Generating test cases with AI](/support/docs/kane-cli-generate/). - -## 1. Generate - -Start with a plain-language description of what you want covered: - -```bash -kane-cli generate "checkout flow on a shopping site" -``` - -kane-cli generates scenarios and cases and prints the result, ending with a **request id** and the exact commands to refine or save it: - -``` -✓ Generated 3 scenarios · 11 cases (request 23271) - -▸ Login -- Valid credentials [Positive] -- Wrong password [Negative] -- Empty fields [Edge] -▸ Checkout -- Guest checkout [Positive] -- Expired card [Negative] -... - -Refine: kane-cli generate "" --refine --req 23271 -Save: kane-cli generate --save --req 23271 -``` - -Keep the request id, every later command uses it. - -Bound the size with limits when you want a tighter or broader set: - -```bash -kane-cli generate "checkout flow on a shopping site" --scenario-limit 4 --per-scenario-limit 6 -``` - -Add `--memory` to reuse relevant existing cases and avoid duplicating coverage you already have. - -## 2. Refine - -Refinement is a plain-language conversation. Each refine is a fresh command with `--refine --req `: - -```bash -kane-cli generate "also cover an expired card and an out-of-stock item" --refine --req 23271 -kane-cli generate "drop the social-login scenario, focus on guest checkout" --refine --req 23271 -``` - -Each refine returns the updated result. Repeat until the set looks right. - -### When generation asks a question - -Sometimes a turn ends by asking you something instead of finishing, for example, *"Which environment should these target, staging or production?"* This is a normal outcome, not a failure (the command exits `0`). Answer it by refining with your answer: - -```bash -kane-cli generate "target staging" --refine --req 23271 -``` - -(Driving this from a script or agent? Add `--agent` (or rely on it being auto-on when stdin is not a TTY) and read the answer-needed signal from the NDJSON, then re-invoke the same way.) - -## 3. Save - -When you are happy with the set, save it. `--save` writes the **functional** cases as `_test.md` files: - -```bash -kane-cli generate --save --req 23271 -``` - -By default this writes under `/.testmuai/tests`: - -``` -.testmuai/tests/ -checkout-23271/ -login/ -valid-credentials_test.md -wrong-password_test.md -checkout/ -guest-checkout_test.md -... -``` - -Choose a different location with `--out`, and name the suite with `--name`: - -```bash -kane-cli generate --save --req 23271 --out ./tests --name checkout-suite -``` - -Only functional cases are written: non-functional cases (Security, Performance, …) are part of the generated result but are not saved as runnable tests. See [Saving is functional-only](/support/docs/kane-cli-generate/#saving-is-functional-only). - -## 4. Run - -The saved files are ordinary `_test.md` tests. Run any of them with `testmd`: - -```bash -kane-cli testmd run .testmuai/tests/checkout-23271/checkout/guest-checkout_test.md -``` - -From here, everything in the [testmd docs](/support/docs/kane-cli-testmd/) applies: replay from cache, edit steps, compose with `@import`, and commit the output to git. - -## Automating it (agents / CI) - -Pass `--agent` (auto-on when stdin is not a TTY) to get structured NDJSON on stdout instead of the human display, so a script or coding agent can drive the loop: - -```bash -kane-cli generate "checkout flow on a shopping site" --agent -kane-cli generate "add an expired-card case" --refine --req 23271 --agent -kane-cli generate --save --req 23271 --agent -``` - -Each command prints one JSON object per line; the final line is the terminal event carrying the request id, the status, and the refine/save commands to run next. - -## Exit codes - -| Code | Meaning | -|---|---| -| `0` | Turn completed, including a turn that ended with a clarification question | -| `1` | Generation failed | -| `2` | Error: authentication / setup / transport, or an invalid combination of flags | -| `3` | Generation stopped or cancelled | -| `130` | Interrupted (Ctrl-C) | - -Invalid flag combinations exit `2` with a message explaining the fix, for example using `--refine` without `--req`, passing a description with `--save`, using `--out` without `--save`, or `--req` without `--refine` or `--save`. - -## Next steps - -- [Generating test cases with AI](/support/docs/kane-cli-generate/): overview, modes, and the full option reference. -- [Running tests with testmd](/support/docs/kane-cli-testmd/): run and replay the files `--save` produces. diff --git a/static/docs/kane-cli-generate.md b/static/docs/kane-cli-generate.md deleted file mode 100644 index dbae073b9..000000000 --- a/static/docs/kane-cli-generate.md +++ /dev/null @@ -1,127 +0,0 @@ -# Generating Test Cases with AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -`kane-cli generate` turns a plain-language description of *what you want to test* into structured **test scenarios** and **test cases**, without writing them by hand and without launching a browser. This page covers AI test-case generation from both the command line and the interactive kane-cli TUI. - -A generation produces: - -- **Test Scenarios**: logical groupings of related checks (e.g. "Login", "Checkout"). -- **Test Cases**: the individual checks inside each scenario, each typed **Positive**, **Negative**, or **Edge**. - -You generate, review the result, **refine** it in plain language as many times as you like, and optionally **save** the functional cases as runnable `_test.md` files that [`kane-cli testmd`](/support/docs/kane-cli-testmd/) can execute and replay. - -For the full picture of AI test-case generation, including richer inputs (files, PDFs, issue links) and pushing cases into Test Manager or automation, see the [AI test-case generation guide](https://www.testmuai.com/support/docs/generate-test-cases-with-ai/). The **CLI here takes a text description** (optionally with local files attached via [`--files`](#attaching-files-for-context)) and writes local `_test.md` files. See [Limits and scope](#limits-and-scope). - -## Quick start - -```bash -# 1. Generate from a description -kane-cli generate "checkout flow on a shopping site" - -# 2. Refine it (repeat as needed) -kane-cli generate "also cover an expired card and an out-of-stock item" --refine --req 23271 - -# 3. Save the functional cases as runnable tests -kane-cli generate --save --req 23271 -# → .testmuai/tests///_test.md - -# 4. Run them -kane-cli testmd run .testmuai/tests/checkout-23271/login/valid-credentials_test.md -``` - -The request id (`23271` above) is printed at the end of each generation and is how you continue a request across commands. - -## The three modes - -`generate` runs **one turn per command, then exits**. You continue a request by running the command again with `--req `. (That's the scripted/headless surface; run interactively, kane-cli opens a live TUI session instead, see [Interactive mode (TUI)](#interactive-mode-tui).) - -| Mode | Command | What it does | -|---|---|---| -| **New** | `kane-cli generate ""` | Starts a fresh request and generates scenarios + cases. Prints a request id. | -| **Refine** | `kane-cli generate "" --refine --req ` | Adjusts an existing request in plain language: add coverage, narrow scope, change focus. | -| **Save** | `kane-cli generate --save --req [--out ]` | Writes the request's **functional** cases to `_test.md` files. No new generation. | - -`--refine` and `--save` both require `--req`. `--refine` needs a change description; `--save` takes no description. - -## Options - -| Option | Purpose | -|---|---| -| `--req ` | The request id to refine or save. | -| `--out ` | Where `--save` writes. Defaults to `/.testmuai/tests` (the same location the TUI uses). | -| `--name ` | Names the run and the saved suite folder. | -| `--scenario-limit ` | Maximum number of scenarios to generate. | -| `--per-scenario-limit ` | Maximum test cases per scenario. | -| `--memory` | Use the **memory layer**: reuse relevant existing test cases and reduce duplicates. | -| `--files ` | Comma-separated local files to attach as context (new generations and refines only). See [Attaching files for context](#attaching-files-for-context). | -| `--project ` / `--folder ` | Test Manager project / folder. | -| `--agent` | Emit structured NDJSON on stdout (auto-on when run non-interactively / piped). | -| `--env`, `--username`, `--access-key` | Environment and authentication, same as [`kane-cli run`](/support/docs/kane-cli-quickstart/). See [Authentication](/support/docs/kane-cli-authentication/). | - -## Attaching files for context - -Give the generator more to work from by attaching local files with `--files` (a comma-separated list of paths) on a **new** generation or a **refine**: - -```bash -kane-cli generate "test the login flow described in the attached spec" --files ./login-spec.pdf,./wireframe.png -``` - -The files are sent along with your description and reflected in the generated scenarios and cases: attach a spec, a screenshot of the UI, a PDF or Word document, or a CSV of inputs. - -- **Supported types**: documents (`.txt`, `.json`, `.xml`, `.csv`, `.pdf`, `.docx`, `.xlsx`), images (`.jpg`, `.jpeg`, `.png`, `.gif`, `.bmp`, `.webp`), audio (`.mp3`, `.wav`, `.m4a`), and video (`.mp4`, `.mov`, `.webm`, `.mpeg`, `.mpga`). -- **Limits**: up to **10 files**, each **50 MB** or smaller. -- **Checked before anything is sent**: all paths are validated as a set. If any is missing, an unsupported type, too large, or over the count, the command stops and lists the offending paths so nothing is uploaded by mistake. Files outside the current directory are allowed but flagged with a warning. -- **Not with `--save`**: files attach to a generation or refine, not a save (`--files` with `--save` is rejected). - -In the interactive TUI, type `@` in the generate prompt to attach a file inline instead of passing `--files`. - -## Interactive mode (TUI) - -The commands above are the scripted surface. The kane-cli TUI **launches in Run mode** (for running tests) by default. Switch to **Generate mode** with **`/generate`**, and back with `/run`. Or jump straight in: running `kane-cli generate ""` in a terminal (without `--agent`) opens the TUI directly in Generate mode and submits your objective. - -Unlike the one-turn command-line surface, the TUI is a **live session**: generated scenarios stay pinned in a **Scenarios** box that you refine and browse in place: - -| Input | Action | -|---|---| -| *(type any text)* | **Refine**: describe a change in plain language; the set updates in place | -| `@` | **Attach a file**: type `@` to pick a local file to add as context (same files as `--files`) | -| `/view [S]` | Open the **scenario browser**: drill scenarios → cases → case detail | -| `/save` | Save the functional cases to `/.testmuai/tests/…` | -| `/cancel` | Cancel the current generation | -| `/run` | Switch back to Run mode | - -In the scenario browser: `↑↓` move · `↵` open · `◂ ▸` previous/next sibling · `x` remove a case · `←`/`esc` back. **Non-functional cases** (Security, Performance, …) show a **gray `✓`** with a "won't be saved" note. `/save` writes only functional cases (see [Saving is functional-only](#saving-is-functional-only)). - -If generation asks a **clarifying question**, just type your answer to continue. After `/save`, the files are ordinary `_test.md` tests under `.testmuai/tests/`, run them with [`kane-cli testmd run`](/support/docs/kane-cli-testmd/). - -## How a result is shaped - -Each generated case carries: - -- a **title** and **steps**, -- a **type**: Positive (expected to pass), Negative (tests failure handling), or Edge (corner cases), -- a **category**: e.g. *Functional*, *Security*, *Performance*, -- a **priority**. - -Scenarios are ordered by importance. The full result is returned at the end of the turn so it can be reviewed before you refine or save. - -## Saving is functional-only - -`--save` writes **only test cases whose category is *Functional***: those are the ones that translate into runnable `_test.md` tests. Non-functional cases (Security, Performance, and so on) are still generated and shown, but they are **not** written to disk. If a request has no functional cases, `--save` writes nothing and tells you so. - -Saved files are ordinary `_test.md` tests in the standard format, identical to a hand-written one. From there everything in [the testmd docs](/support/docs/kane-cli-testmd/) applies: run them, edit them, replay them from cache, commit them to git. - -This is the intended path: **generate authors test cases → testmd runs them.** - -## Limits and scope - -- **Input is a text description, optionally with attached files.** Attach local files with `--files` (a spec, screenshot, PDF, or CSV) to give the generator more context. Issue-link inputs and the web product's "Create / Create and Automate" are not part of the CLI. -- **Refinement is whole-request.** You refine the request as a whole in plain language; targeting an individual scenario or case from the CLI is not yet supported. -- **Scenario and per-scenario counts** are bounded by `--scenario-limit` / `--per-scenario-limit`. - -## Next steps - -- [The generate workflow](/support/docs/kane-cli-generate-workflow/): the new → refine → save → run loop, worked examples, and exit codes. -- [Running tests with testmd](/support/docs/kane-cli-testmd/): run and replay the `_test.md` files `--save` produces. -- [Authentication](/support/docs/kane-cli-authentication/): logging in and choosing an environment. diff --git a/static/docs/kane-cli-getting-started.md b/static/docs/kane-cli-getting-started.md deleted file mode 100644 index 0d92fe770..000000000 --- a/static/docs/kane-cli-getting-started.md +++ /dev/null @@ -1,3 +0,0 @@ -# Getting Started with Kane CLI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). diff --git a/static/docs/kane-cli-installation.md b/static/docs/kane-cli-installation.md deleted file mode 100644 index 57a1c8b4f..000000000 --- a/static/docs/kane-cli-installation.md +++ /dev/null @@ -1,95 +0,0 @@ -# Installing Kane CLI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Kane CLI is published to the public npm registry as `@testmuai/kane-cli` and to a Homebrew tap. Install it with `npm` or `brew` to get the `kane-cli` command on your `PATH`. - -## Install - -Requires Node.js 18 or higher. - -```bash -npm install -g @testmuai/kane-cli -``` - -Platform-specific native binaries are installed automatically for your OS. No additional configuration is needed. - -For macOS and Linux. Node.js is not required. - -```bash -brew install LambdaTest/kane/kane-cli -``` - -Homebrew also installs Google Chrome automatically through the `google-chrome` cask, so no additional setup is needed. - -Kane CLI requires **Google Chrome** installed on your system. Chrome is used as the automation browser and is launched automatically when you run a test. The npm install additionally requires **Node.js 18 or higher**. The Homebrew install bundles Chrome for you and needs no Node.js. - -## Verify - -```bash -kane-cli --version -``` - -If the command is not found, your shell is not seeing the npm global `bin` directory. Open a new terminal or update `PATH`, then try again. - -## Platform Support - -| Platform | Architecture | Supported | -|----------|-------------|-----------| -| macOS | Apple Silicon (ARM) | ✅ | -| macOS | Intel (x64) | ✅ | -| Linux | x64 | ✅ | -| Windows | x64 | ✅ | - -## Update - -Kane CLI checks the npm registry once every 24 hours when you launch it. When a newer version is available, the CLI prints a one-line notification on startup with the current and latest versions. The check runs in the background and never blocks startup. - -Upgrade with: - -```bash -npm install -g @testmuai/kane-cli@latest -``` - -After upgrading, run `kane-cli --version` to confirm the new version is active. - -## Uninstall - -```bash -npm uninstall -g @testmuai/kane-cli -``` - -This removes the `kane-cli` binary but leaves your local data in place. Kane CLI stores credentials, configuration, sessions, and Chrome profile data under `~/.testmuai/kaneai/`. To wipe that state as well: - -```bash -rm -rf ~/.testmuai/kaneai -``` - -Only do this if you want a clean reset, it logs you out of all profiles and deletes saved configuration, session history, and command history. - -## Troubleshooting Installation - -**`kane-cli: command not found` after install** - -Your npm global bin directory is not in your PATH. Find it and add it: - -```bash -# Find your npm global bin directory -npm config get prefix - -# The kane-cli binary lives in /bin on macOS/Linux, and on Windows -# Add it to your PATH in ~/.zshrc or ~/.bashrc -export PATH="$(npm config get prefix)/bin:$PATH" -``` - -**Installation fails on Node 16** - -Kane CLI requires Node.js 18+. Check your version and upgrade if needed: - -```bash -node --version -``` - -## Next Step - -[Quick Start](/support/docs/kane-cli-quickstart/): Authenticate and run your first test. diff --git a/static/docs/kane-cli-introduction.md b/static/docs/kane-cli-introduction.md deleted file mode 100644 index dc473bbba..000000000 --- a/static/docs/kane-cli-introduction.md +++ /dev/null @@ -1,52 +0,0 @@ -# Kane CLI Documentation - Getting Started - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -**Kane CLI** `kane-cli` is an AI-powered browser automation tool that runs from your terminal. Describe what you want to test in plain English: Kane CLI navigates websites, clicks elements, fills forms, extracts data, and validates outcomes in a real Chrome browser. - -- **Run browser tests from any terminal or IDE**: no test scripts, no selectors, no framework boilerplate -- **Integrate into CI/CD pipelines**: headless mode with structured JSON output and standard exit codes -- **Use as a skill in AI coding agents**: Claude Code, Codex CLI, and Gemini CLI can invoke Kane CLI directly to test and verify web UIs on your behalf - -```bash -# Install -npm install -g @testmuai/kane-cli - -# Authenticate -kane-cli login - -# Run your first test -kane-cli run --url https://example.com "Click the 'More information' link and verify the page loads" -``` - -## Supported IDEs - -Kane CLI runs from the integrated terminal of your editor, so you can drive browser tests without leaving your development environment. It is supported in: - -* Cursor -* Antigravity -* VS Code -* Codex -* Kiro (AWS) -* Eclipse-based IDEs -* Web-based IDEs - -## Three Modes - -| Mode | Command | Best For | -|------|---------|----------| -| **Interactive TUI** | `kane-cli --tui` | Development, exploration, chained multi-run sessions | -| **Non-Interactive CLI Mode** | `kane-cli run "..." --headless --agent` | CI/CD pipelines, shell scripts | -| **Agent Mode** | `kane-cli run "..." --agent` | AI coding agents (Claude, Codex, Gemini) | - -## Next Steps - -- [Installation](/support/docs/kane-cli-installation/): Install Kane CLI and verify your setup -- [Quick Start](/support/docs/kane-cli-quickstart/): Authenticate and run your first test in 5 minutes -- [Authentication](/support/docs/kane-cli-authentication/): OAuth, basic auth, and profile management -- [Writing Objectives](/support/docs/kane-cli-writing-objectives/): Learn how to write effective natural language objectives -- [Configuration](/support/docs/kane-cli-configuration/): Window size, Chrome profiles, Test Manager project, and run mode -- [Test Manager Integration](/support/docs/kane-cli-tms-integration/): Uploads, share links, code export, and session history -- [Agent Mode](/support/docs/kane-cli-agent-mode/): Use Kane CLI with AI coding agents -- [CI/CD Integration](/support/docs/kane-cli-cicd/): Add Kane CLI to your pipeline -- [Skills](/support/docs/kane-cli-skills/): Install the Kane CLI skill for Claude, Codex, or Gemini diff --git a/static/docs/kane-cli-modes.md b/static/docs/kane-cli-modes.md deleted file mode 100644 index 0ef2f3d02..000000000 --- a/static/docs/kane-cli-modes.md +++ /dev/null @@ -1,204 +0,0 @@ -# Modes of Operation - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Kane CLI has three modes. Choose based on who (or what) is running the test. - -| Mode | Command | Output | Best For | -|------|---------|--------|----------| -| **Interactive TUI** | `kane-cli --tui` | Terminal UI | Development, exploration, chained sessions | -| **Non-Interactive CLI Mode** | `kane-cli run "..." --headless` | Formatted text + JSON | CI/CD, shell scripts | -| **Agent Mode** | `kane-cli run "..." --agent` | NDJSON on stdout | AI coding agents (Claude, Codex, Gemini) | - -## Interactive TUI - -Launch the TUI: - -```bash -kane-cli --tui -``` - -The TUI is the right surface when you are exploring objectives, debugging failures, or working through a multi-run flow that should share browser state. - -### Boot and Menu - -On launch, Kane CLI runs a short boot sequence (auth check, environment resolution, mascot animation), then drops into the **main menu**. The top-level entries are: - -| Entry | Purpose | -|-------|---------| -| Run | Start a run or adjust per-run options | -| Auth | Login, logout, switch profile, view identity, check credit balance | -| Config | View and change settings (mode, project, folder, Chrome profile, window size) | -| Exit | Graceful shutdown (uploads the session if applicable) | - -Use the arrow keys to navigate, Enter to select, and Esc to back out of a submenu. - -### TUI Mode - -In your terminal, type `kane-cli --tui` and this switches to TUI mode. Type your objective at the prompt and press Enter. The agent begins streaming steps into the scrollback: each step shows the action taken, a short rationale, and a status icon. When the run finishes, a result summary block appears. - -Subsequent runs in the same TUI session reuse the same browser, so you can iterate on objectives without re-logging in or re-navigating. - -### Slash Commands - -Typing `/` in chat mode opens an autocomplete palette. Continue typing to filter, use the arrow keys to select, and press Enter to insert the command. - -| Command | Args | Description | -|---------|------|-------------| -| `/run` | `"objective"` | Execute a test run | -| `/login` | `[--profile name]` | OAuth login | -| `/logout` | `[--profile name]` | Logout and revoke tokens | -| `/whoami` | `[--profile name]` | Show profile info | -| `/balance` | | Show credit balance | -| `/profiles` | `list\|switch\|delete` | Manage profiles | -| `/config` | `show\|set-window\|set-url\|set-mode\|chrome-profile\|project\|folder` | Manage configuration | -| `/new` | | Start a fresh session (uploads the current session first) | -| `/summary` | `[index]` | View detailed run summaries | -| `/cancel` | | Abort the current run | -| `/help` | | Show the command reference | -| `/clear` | | Clear chat history | -| `/exit` | | Quit Kane CLI | - -You can also send a bare line of text without a leading `/`. It is treated as the objective for `/run`. - -### History Search - -Press **Ctrl+R** in the input prompt to open reverse history search across past inputs in this and previous sessions. Type to filter, use the arrow keys to move between matches, Enter to accept, and Esc to dismiss. - -The prompt also offers ghost-text completion: if your current input is a prefix of a recent entry or a slash command, the rest is shown dimmed and Tab accepts it. - -### Status Bar - -A two-row status bar sits at the bottom of the TUI: - -| Indicator | Meaning | -|-----------|---------| -| Model | The model in use (default `v16-alpha`) | -| Session | Last six characters of the current session ID | -| Auth dot | Green when authenticated, red when not logged in | -| Profile | Active profile name (or `no profile`) | -| Environment | `prod` (green) or a yellow `stage` warning | -| Runs | Number of runs completed in this session | -| Hint line | Context-aware shortcuts | - -### Multi-Run Sessions - -Every run launched from the same TUI invocation shares one Chrome instance and one session directory. Cookies, login state, and tabs persist across runs, so an early run can log in and a later run can land mid-application without re-authenticating. Starting a fresh session from inside the TUI is done with `/new`, which uploads the current session and then resets state. - -``` -> go to https://myapp.com and log in as admin -✓ PASSED (5 steps, 8.2s) - -> navigate to User Management and create a new user "testuser@example.com" -✓ PASSED (7 steps, 12.1s) - -> verify the new user appears in the users table -✓ PASSED (3 steps, 4.5s) -``` - -### Interactive Follow-Ups - -If the agent needs information mid-run (for example, a one-time code or a clarifying choice), it pauses and asks at the input prompt. Type your answer and press Enter; the agent resumes from where it left off. Use Ctrl+C to cancel the run instead of answering. - -### Keyboard Shortcuts - -| Shortcut | Action | -|----------|--------| -| Enter | Submit objective | -| Ctrl+C | Cancel current run | -| Ctrl+C (twice) | Exit TUI | -| Ctrl+R | Reverse history search | -| Esc | Go back / close picker | -| Up / Down | Navigate menu or history | -| Tab | Accept autocomplete in chat mode | - -### Cancelling and Exiting - -| Action | Shortcut | -|--------|----------| -| Cancel the current run | `/cancel`, or **Ctrl+C** once during a run | -| Exit the TUI | `/exit`, or **Ctrl+C** twice in quick succession | -| Force exit during shutdown upload | **Ctrl+C** twice while exit is in progress | - -A graceful `/exit` runs the upload pipeline (if applicable) and prints any final links to your terminal scrollback before the process ends. - -## Non-Interactive CLI Mode - -```bash -kane-cli run "Search for 'automation testing' on Google" \ ---url https://google.com \ ---headless -``` - -This mode is best for shell scripts, CI/CD pipelines, and any scenario where the interactive TUI is not needed. - -### Output Streams - -| Stream | Contents | -|--------|----------| -| stderr | Live progress (banner, step tree, result box, links, upload progress, feedback prompt) | -| stdout | The final JSON `run_end` payload, including the share URL when an upload succeeds | - -This separation lets you capture each independently: - -```bash -kane-cli run "..." > result.json 2> progress.log -``` - -In CI, redirect stdout to a file your job can parse and let stderr stream to the build log. - -When stdin is not a TTY, Kane CLI automatically switches to plain NDJSON mode (the same as `--agent`). Each line on stdout is one JSON event terminated by a newline. - -### Exit Codes - -| Exit Code | Meaning | -|-----------|---------| -| `0` | Test passed | -| `1` | Test failed (agent reached an unrecoverable failure, or the upload failed) | -| `2` | Error (auth failure, Chrome crash) | -| `3` | Timeout or cancelled | - -### What You See at the End of a Run - -When a run finishes, Kane CLI prints a result summary: - -| Field | Meaning | -|-------|---------| -| Status | `PASSED` (green check) or `FAILED` (red cross) | -| Steps | Total step count, with a `(N passed, M failed)` breakdown when there were failures | -| Duration | Wall-clock time in seconds (or minutes and seconds for longer runs) | -| Credits | Credits consumed, when reported | -| Summary | Bullet-point summary of what the agent did | -| Reason | Failure reason (failed runs only) | - -Below the summary, Kane CLI prints any of the following links: - -| Label | Points to | -|-------|-----------| -| `ShareLink` | A shareable session URL on TestMu AI Test Manager | -| `TestCase` | The test case detail page in TestMu AI Test Manager | -| `CodeExport` | The local directory containing generated code (when code export is enabled) | - -### Feedback Prompt - -After the result and links print, Kane CLI prompts you to rate the session with thumbs up or thumbs down. Use the left and right arrow keys to choose, Enter to submit, or Esc to skip. See [Test Manager Integration](/support/docs/kane-cli-tms-integration/) for details. - -## Agent Mode - -Add `--agent` to get structured NDJSON output designed for AI coding agents: - -```bash -kane-cli run "Verify login page loads" --url https://myapp.com --agent --headless -``` - -With `--agent`: -- The interactive TUI is **fully suppressed**: no boot screen, no menus -- Each event is a **single JSON line** on stdout -- Progress UI renders to **stderr** (clean separation) -- Stdin is not used (no interactive prompts) - -This mode is how Claude Code, Codex CLI, and Gemini CLI consume Kane CLI results. The agent reads the NDJSON stream, parses the final `run_end` event, and presents results to you. - -See [Agent Mode](/support/docs/kane-cli-agent-mode/) for the full event schema and parsing guide. - -Always combine `--agent` with `--headless` in non-interactive environments to prevent display server errors. diff --git a/static/docs/kane-cli-parallel-execution.md b/static/docs/kane-cli-parallel-execution.md deleted file mode 100644 index 769d24124..000000000 --- a/static/docs/kane-cli-parallel-execution.md +++ /dev/null @@ -1,122 +0,0 @@ -# Parallel Test Execution - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Run multiple independent browser tests concurrently to reduce total execution time. Instead of running tests sequentially (sum of all durations), parallel execution runs them simultaneously: total time equals the longest single test. - -## When to Run in Parallel - -Split objectives and run in parallel when: - -- An objective has **more than 15 steps**: long objectives drift; split them -- Tasks are **independent**: no shared browser state or sequential dependencies -- You're testing **different features**: login vs. checkout vs. settings -- You're testing **different user roles**: admin flow vs. customer flow - -Each sub-objective must be **self-contained**: it navigates to its own URL, authenticates independently, and asserts its own outcomes. - -## Sequential Pattern - -Run three tests one after another: - -```bash -kane-cli run "Log in and verify dashboard" --url https://myapp.com --headless --timeout 120 -kane-cli run "Search for products and verify results" --url https://myapp.com --headless --timeout 120 -kane-cli run "Complete checkout and confirm order" --url https://myapp.com --headless --timeout 120 -``` - -**Total time:** sum of all three durations. - -## Parallel Pattern (Shell Background Processes) - -```bash -#!/bin/bash -RESULTS_DIR=$(mktemp -d) - -# Start all tests in background -kane-cli run "Log in and verify dashboard" \ ---url https://myapp.com --agent --headless --timeout 120 \ -> "$RESULTS_DIR/test1.ndjson" 2>&1 & - -kane-cli run "Search for products and verify results" \ ---url https://myapp.com --agent --headless --timeout 120 \ -> "$RESULTS_DIR/test2.ndjson" 2>&1 & - -kane-cli run "Complete checkout and confirm order" \ ---url https://myapp.com --agent --headless --timeout 120 \ -> "$RESULTS_DIR/test3.ndjson" 2>&1 & - -kane-cli run "Verify admin user management page" \ ---url https://myapp.com --agent --headless --timeout 120 \ -> "$RESULTS_DIR/test4.ndjson" 2>&1 & - -# Wait for all to finish -wait - -# Parse and print results -echo "" -echo "| # | Test | Status | Steps | Time | Summary |" -echo "|---|------|--------|-------|------|---------|" - -i=1 -for f in "$RESULTS_DIR"/test*.ndjson; do -result=$(tail -1 "$f") -status=$(echo "$result" | jq -r '.status') -duration=$(echo "$result" | jq -r '.duration') -summary=$(echo "$result" | jq -r '.one_liner') -echo "| $i | $(basename $f .ndjson) | $status | - | ${duration}s | $summary |" -((i++)) -done - -rm -rf "$RESULTS_DIR" -``` - -**Total time:** duration of the longest test. - -## Batch Summary Format - -After all tests complete, present results like this: - -``` -🧪 Test Suite: Core Flows -📅 Run at: 2026-04-14 14:30 UTC - -| # | Test | Status | Steps | Time | Summary | -|---|------------------------|--------|-------|------|--------------------------------| -| 1 | Login + dashboard | ✅ | 5 | 12s | Welcome banner visible | -| 2 | Product search | ✅ | 7 | 18s | 3 results for 'shoes' | -| 3 | Checkout flow | ❌ | 9 | 25s | Payment form did not load | -| 4 | Admin user management | ✅ | 6 | 15s | Users table loaded (12 rows) | - -📊 Pass rate: 3/4 (75%) · Total steps: 27 · Total time: 25s (longest test) -``` - -## AI Agent Sub-Task Pattern - -For AI coding agents (Claude Code, Codex CLI, Gemini CLI), use the agent's parallel task mechanism to spawn multiple sub-agents simultaneously. Each sub-agent runs one Kane CLI command and returns structured results. - -**Agent prompt template** (give this to each sub-agent): - -``` -Run this Kane CLI browser test and report the results: - -kane-cli run "" --agent --headless --timeout 120 - -After the command completes: -1. Capture the exit code -2. Parse the run_end event (last line of stdout) -3. If status is "failed", read the failing step's screenshot from run_dir -4. Return: { status, steps, duration, summary, session_dir, failure_step, screenshot_path } -``` - -## Resource Limits - -Each parallel Kane CLI instance opens its own Chrome browser. Monitor resources when running many parallel tests: - -- Each instance uses ~150–300 MB RAM -- 4 concurrent tests = ~800 MB – 1.2 GB RAM -- Each instance uses its own CDP port (9222–9230) - -Start with 4 parallel tests and scale up based on your environment. - -Always use `--headless` for parallel runs: multiple visible browser windows cause confusion and unnecessary resource overhead. diff --git a/static/docs/kane-cli-quickstart.md b/static/docs/kane-cli-quickstart.md deleted file mode 100644 index 83e7c4a34..000000000 --- a/static/docs/kane-cli-quickstart.md +++ /dev/null @@ -1,107 +0,0 @@ -# Quick Start - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This guide takes you from a fresh install to a passing run in under five minutes. - -## Step 1: Install - -```bash -npm install -g @testmuai/kane-cli -``` - -Once `kane-cli --version` prints a version number, you are ready. See [Installation](/support/docs/kane-cli-installation/) for platform requirements and troubleshooting. - -## Step 2: Authenticate - -```bash -kane-cli login -``` - -Your default browser opens on a TestMu AI consent page. Sign in (or confirm, if you are already signed in), authorise the CLI, and the page will tell you it is safe to close the tab. Control returns to your terminal and your credentials are stored under `~/.testmuai/kaneai/profiles/`. - -{`kane-cli login \\ - --username "${ YOUR_LAMBDATEST_USERNAME()}" \\ - --access-key "${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -Find your username and access key on the TestMu AI [dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard) > **Credentials**. - -Use this method for environments without a browser (servers, containers, CI/CD). - -For non-interactive CI pipelines, pass credentials inline: - -{`kane-cli run "your objective" \\ - --username "${ YOUR_LAMBDATEST_USERNAME()}" \\ - --access-key "${ YOUR_LAMBDATEST_ACCESS_KEY()}" \\ - --headless --agent`} - -Store `LT_USERNAME` and `LT_ACCESS_KEY` as secrets in your CI/CD platform. See [CI/CD Integration](/support/docs/kane-cli-cicd/) for platform-specific guides. - -Verify authentication at any time: - -```bash -kane-cli whoami -``` - -If you prefer username and access key, or need to manage multiple accounts, see [Authentication](/support/docs/kane-cli-authentication/). - -## Step 3: Open the TUI - -Run `kane-cli --tui`: - -```bash -kane-cli --tui -``` - -A short boot animation renders, then the main view drops you at a chat prompt with a status bar at the bottom showing your model, session ID, environment, and run count. From there, anything you type is treated as a test objective. Lines that begin with `/` are slash commands such as `/help`, `/config`, `/cancel`, and `/exit`. - -## Step 4: Run Your First Test - -At the chat prompt, type a natural-language objective and press Enter: - -```text -Search for "wireless headphones" on Amazon and add the first result to cart -``` - -Kane CLI launches Chrome, hands the objective to the agent, and starts streaming progress. You see a step tree fill in as the agent thinks, observes the page, and acts: clicking, typing, scrolling, and verifying. When the agent is done, a result block reports whether the objective passed or failed and how many steps it took. - -If you ever need to stop a run, press `Ctrl+C` once. Pressing it twice exits Kane CLI. - -## Step 5: Or Run from the Command Line - -For scripting and CI, skip the TUI and use the `run` subcommand: - -```bash -kane-cli run "Click the 'More information' link" --url https://example.com -``` - -In CLI mode, Kane CLI streams progress to **stderr**, prints the final result as a single JSON object on **stdout**, and exits with a status code that reflects the outcome: - -| Exit Code | Meaning | -|-----------|---------| -| `0` | Test passed | -| `1` | Test failed (assertion not met) | -| `2` | Error (auth failure, Chrome crash) | -| `3` | Timeout or cancelled | - -Capture results in a shell script: - -```bash -kane-cli run "Verify the homepage loads" > result.json -``` - -By default each run starts on the KaneAI playground site. The agent navigates from there based on your objective. - -## What Happens at the End of a Run - -When the session ends, Kane CLI uploads the run to TestMu AI Test Manager and prints a share link. For details on the upload, the share-link experience, and the run mode toggle, see [Test Manager Integration](/support/docs/kane-cli-tms-integration/). To change settings like window size, Chrome profile, or the active project and folder, see [Configuration](/support/docs/kane-cli-configuration/). - -Run `kane-cli --tui` to open the interactive TUI: a full terminal UI where you can type objectives, see step-by-step progress, and chain multiple tests in one session with persistent browser state. - -## Next Steps - -- [Writing Objectives](/support/docs/kane-cli-writing-objectives/): Learn the three patterns: actions, assertions, extractions -- [Modes of Operation](/support/docs/kane-cli-modes/): Understand Interactive TUI, Headless CLI, and Agent Mode -- [Authentication](/support/docs/kane-cli-authentication/): Manage profiles and credential methods -- [Variables & Context](/support/docs/kane-cli-variables-and-context/): Parameterize tests with credentials and project-specific context -- [CI/CD Integration](/support/docs/kane-cli-cicd/): Add Kane CLI to your pipeline diff --git a/static/docs/kane-cli-skills.md b/static/docs/kane-cli-skills.md deleted file mode 100644 index 4e6c734f9..000000000 --- a/static/docs/kane-cli-skills.md +++ /dev/null @@ -1,157 +0,0 @@ -# Kane CLI Skills for AI Agents - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -A Kane CLI **skill** is a markdown instruction file that teaches an AI coding agent how to use `kane-cli`: when to invoke it, how to build commands, how to parse NDJSON output, how to present results, and how to handle failures. Install the skill once, and your agent handles browser automation tasks automatically whenever you ask. - -Skills work on top of [Agent Mode](/support/docs/kane-cli-agent-mode/) (`--agent` flag). The skill tells the agent to always use `--agent`, parse the NDJSON stream, and present structured results back to you. - -Skills are plain markdown files. Review and customize them by editing the installed file after download. - -## Install - -The quickest way to install the Kane CLI skill globally for all supported agents (Claude Code, Codex CLI, and Gemini CLI) in one command: - -```bash -npx @testmuai/kane-cli-skill -``` - -If you prefer to install manually for a specific agent, follow the instructions below. - -## Install by Agent - -The Kane CLI skill for Claude Code is a `SKILL.md` file placed in your skills directory. - -**Global install** (available in all projects): - -```bash -mkdir -p ~/.claude/skills/kane-cli -curl -o ~/.claude/skills/kane-cli/SKILL.md \ -https://raw.githubusercontent.com/LambdaTest/kane-cli/main/skill-installer/skills/SKILL.md -``` - -**Project-level install** (available only in this project): - -```bash -mkdir -p .claude/skills/kane-cli -curl -o .claude/skills/kane-cli/SKILL.md \ -https://raw.githubusercontent.com/LambdaTest/kane-cli/main/skill-installer/skills/SKILL.md -``` - -After installing, Claude Code automatically loads the skill. No restart required. - -**What Claude can do with this skill:** - -- Recognize when your request involves a browser and invoke `kane-cli` automatically -- Build properly structured `kane-cli run --agent` commands -- Parse the NDJSON output and present structured results with step counts, duration, extracted values, and assertion results -- Inspect failure logs and screenshots to diagnose issues -- Run multiple browser tests in parallel using sub-agents -- Suggest bug reports for agent failures, linked to the GitHub repo - -**Example prompts:** - -``` -"Can you verify the checkout flow on staging.myapp.com works?" -"Check if our login form shows the right error for wrong passwords" -"Search for 'laptops' on Amazon and store the top 3 prices" -``` - -The Kane CLI skill for Codex CLI is appended to your `AGENTS.md` file. Codex reads `AGENTS.md` at the project root or from `~/.codex/AGENTS.md` globally. - -**Global install** (available in all projects): - -```bash -curl -o /tmp/kane-cli-agents.md \ -https://raw.githubusercontent.com/LambdaTest/kane-cli/main/skill-installer/skills/SKILL.md - -# Append to global AGENTS.md (creates the file if it doesn't exist) -cat /tmp/kane-cli-agents.md >> ~/.codex/AGENTS.md -``` - -**Project-level install**: - -```bash -curl -o /tmp/kane-cli-agents.md \ -https://raw.githubusercontent.com/LambdaTest/kane-cli/main/skill-installer/skills/SKILL.md - -cat /tmp/kane-cli-agents.md >> AGENTS.md -``` - -**What Codex can do with this skill:** - -- Invoke `kane-cli run --agent` for browser tasks -- Parse NDJSON output using `jq` and `tail` -- Run parallel browser tests using shell background processes (`&` + `wait`) -- Inspect failure logs and show screenshot paths - -**Example prompts:** - -``` -"Verify the staging site checkout flow works" -"Run the login test and tell me what happened" -``` - -The Kane CLI skill for Gemini CLI is a `SKILL.md` file placed in your Gemini skills directory. - -**Global install** (available in all projects): - -```bash -mkdir -p ~/.gemini/skills/kane-cli -curl -o ~/.gemini/skills/kane-cli/SKILL.md \ -https://raw.githubusercontent.com/LambdaTest/kane-cli/main/skill-installer/skills/SKILL.md -``` - -**Project-level install**: - -```bash -mkdir -p .gemini/skills/kane-cli -curl -o .gemini/skills/kane-cli/SKILL.md \ -https://raw.githubusercontent.com/LambdaTest/kane-cli/main/skill-installer/skills/SKILL.md -``` - -**What Gemini can do with this skill:** - -- Identify browser automation requests and invoke `kane-cli --agent` -- Parse NDJSON events and surface the final result -- Run tests in parallel using shell subprocesses -- Diagnose failures from log files and screenshots - -**Example prompts:** - -``` -"Test that the homepage loads correctly on staging" -"Verify the registration flow works end to end" -``` - -## First-Time Auth in Agent Contexts - -OAuth login opens a browser window: AI agents cannot complete this flow. Use Basic Auth instead: - -{`kane-cli login \\ - --username "${ YOUR_LAMBDATEST_USERNAME()}" \\ - --access-key "${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -Get credentials from the TestMu AI [dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard) > **Credentials**. Verify with `kane-cli whoami`. - -## How Skills Work - -When you ask your AI agent to test something on the web, the skill: - -1. Checks `kane-cli` is installed (`kane-cli --version`) -2. Verifies authentication (`kane-cli whoami`) -3. Builds a `kane-cli run --agent --headless` command from your request -4. Parses the NDJSON stream, waits for `run_end` -5. Presents structured results: status, steps, duration, extracted values, assertion results -6. If failed: inspects `run_dir` logs and screenshots to diagnose the issue -7. Rates the run and suggests a bug report for low-quality agent failures - -## Skill File Locations - -| Agent | Global | Project-level | -|-------|--------|---------------| -| Claude Code | `~/.claude/skills/kane-cli/SKILL.md` | `.claude/skills/kane-cli/SKILL.md` | -| Codex CLI | `~/.codex/AGENTS.md` (appended section) | `AGENTS.md` (appended section) | -| Gemini CLI | `~/.gemini/skills/kane-cli/SKILL.md` | `.gemini/skills/kane-cli/SKILL.md` | - -Source files: [github.com/LambdaTest/kane-cli/tree/main/skill-installer/skills](https://github.com/LambdaTest/kane-cli/tree/main/skill-installer/skills) diff --git a/static/docs/kane-cli-testmd.md b/static/docs/kane-cli-testmd.md deleted file mode 100644 index 4d588786a..000000000 --- a/static/docs/kane-cli-testmd.md +++ /dev/null @@ -1,305 +0,0 @@ -# Test.md - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -`testmd` lets you write browser tests as Markdown files (`_test.md`) and commit them to your repo. On the first run, the AI agent authors each step and saves a recording. On every subsequent run, each step **replays from cache** with no LLM cost and much faster execution. Commit the test file and its recordings to git so teammates and CI can re-run the same tests without re-authoring. - -## Quick Start - -**Step 1:** Create a test file. The filename must end in `_test.md`: - -```markdown ---- -mode: testing ---- - -# Amazon search - -## Open Amazon -Open https://www.amazon.com. - -## Search for headphones -Type "wireless headphones" into the search box and submit. -Verify at least one product result is visible. -``` - -**Step 2:** Run the test: - -```bash -kane-cli testmd run amazon_test.md --agent -``` - -On the first run, the agent authors each step and caches the recording. On every later run, the steps replay from cache instantly. - -Always combine `--agent` with `--headless` in CI/CD environments to avoid display server errors. - -## When to Use testmd vs run - -`kane-cli run` is one-shot. It runs an objective, uploads results, and exits. It is ideal for quick, one-off verifications like checking if a page loads correctly or extracting a value from a live site. - -`kane-cli testmd run` is for tests you want to persist. Use it when you are building a login flow smoke test, a regression suite, or any test you plan to re-run across builds. The test file lives in your repo, recordings are cached and committed alongside it, and every subsequent run replays from cache without consuming LLM credits. Teammates and CI pick up the same recordings and replay them identically. - -If you run an ad-hoc objective with `kane-cli run` and later decide you want to keep it, use the `--name` flag to save it as a `_test.md` file (see [Recording a Test from a Live Session](#recording-a-test-from-a-live-session) below). - -## File Format - -A `_test.md` file has four parts in order: - -### YAML Frontmatter - -Configuration between `---` markers at the top of the file: - -```yaml ---- -mode: testing -max_steps: 30 -headless: true ---- -``` - -**Supported keys:** - -| Key | Scope | Description | -|-----|-------|-------------| -| `mode` | root | `testing` (default) or `action`. Testing mode pushes through auth walls for negative-test assertions. Action mode halts on auth walls. | -| `max_steps` | root + step | Max agent reasoning steps per step. Default: `30` | -| `timeout` | root + step | Hard kill per step, in seconds | -| `headless` | root | Run Chrome with no visible window | -| `variables` | root + step | Variables with `{{name}}` syntax, same format as [Variables & Context](/support/docs/kane-cli-variables-and-context/) | -| `code_export` | root + step | Generate Playwright code after the run | -| `code_language` | root + step | `python` or `javascript` for code export | -| `global_context` / `local_context` | root + step | Inline Markdown or file path for agent context | - -### Title and Steps - -After the frontmatter, add a `# H1` title for the test. This is purely decorative. Kane CLI ignores everything before the first `## ` heading. - -Each `## H2` heading marks a test step. The heading text is a label for your reference; the agent reads the **step body** underneath it. Write the body as either plain English prose describing what the agent should do, or a single `@import ` line to pull in a reusable helper file. Do not mix prose and `@import` in the same step. - -### Per-Step Config Overrides - -You can override frontmatter settings for individual steps by adding a `yaml` fenced block immediately under the step heading: - -````markdown -## Submit the form -```yaml -timeout: 90 -optional: true -``` -Click submit and verify the confirmation banner. -```` - -Setting `optional: true` tells Kane CLI that a failure on this step should not fail the overall test. The run continues to the next step. - -## Replay and Cascade Rule - -This is the most important concept in `testmd`. - -### Replay - -After the first run, each step replays from its cached recording with no AI agent, no LLM cost, and much faster execution. A step replays only if **all** of these hold: - -- A recording for that step exists -- The step's prose is unchanged since the recording -- The step's `yaml` block is unchanged -- No earlier step invalidated it - -### Cascade - -Editing step N **re-authors step N and every step after it**. Each step starts where the previous step left off (URL, login state, open tabs). When step 3 changes, step 4 cannot safely replay against state that no longer exists. - -A one-line tweak at the top of a 20-step test re-authors all 20 steps on the next run. To minimize re-authoring, edit only the steps you need to change. - -**Useful commands:** - -| Action | How | -|--------|-----| -| Re-record one step | Edit only that step (steps after it cascade automatically) | -| Force full re-authoring | Use `--author` flag for one run | -| Wipe cache entirely | Run `rm -rf output-/` | - -## Reusing Flows with `@import` - -Extract repeating flows (login, setup, cookie banner dismissal) into helper files: - -```markdown -## Sign in -@import ./helpers/login.md -``` - -**Rules:** - -- Helper filename **must not** end in `_test.md`. Only `_test.md` files are valid entry points -- Path resolves relative to the **importing file**, not the shell's working directory -- The step body must be exactly `@import ` with no mixed prose or extra lines -- The step's `yaml` block may contain **only** `optional` -- `optional: true` on `@import` is allowed only at the root file level, not on nested imports -- Variables and context propagate into helpers automatically - -Editing a helper re-authors that step in **every test that imports it**, plus everything after the import in those tests. The same cascade rule applies. - -## Recording a Test from a Live Session - -Run an ad-hoc objective with the `--name` flag to save it as a replayable test file: - -```bash -kane-cli run "Search for noise-cancelling headphones on amazon.com" --name amazon-search -``` - -On exit, Kane CLI writes the test file to `.testmuai/tests/amazon-search_test.md`. Move that file into your repo and re-run it with `testmd run`. - -Without `--name`, ad-hoc runs are ephemeral and nothing is written to disk. - -## Commands - -| Command | Purpose | -|---------|---------| -| `kane-cli testmd run --agent` | Run a test file | -| `kane-cli testmd list` | List `*_test.md` files under the current directory | -| `kane-cli testmd status ` | Show Test Manager identity and local sync state | -| `kane-cli testmd export ` | Regenerate code export from existing recordings (no browser launch) | -| `kane-cli testmd delete ` | Delete the test and its `output-/` cache locally. Does NOT delete from Test Manager | - -## Flags for `testmd run` - -All [`kane-cli run` flags](/support/docs/kane-cli-cli-reference/) apply (`--agent`, `--headless`, `--max-steps`, `--timeout`, `--variables`, etc.), plus these additional flags: - -| Flag | Default | Description | -|------|---------|-------------| -| `--name ` | none | Persist the run under this name. Must match `[a-zA-Z0-9_-]+` | -| `--on-lock-conflict ` | none | Behavior when another user holds the test's edit lock: `readonly` (replay-only, no upload), `fail` (exit 2), or `wait` (block until released) | -| `--retry` | off | On replay failure, restart with a shrinking replay window | -| `--retry-count ` | `3` | Max retry restarts before falling back to full re-author | -| `--author` | off | Force authoring every step, skipping replay entirely | -| `--code-language ` | `python` | Code export language: `python` or `javascript` | - -Flag values win over frontmatter for all settings **except** `variables`. The file owns variables. You can add new keys via flags but cannot override file-defined ones. - -## Output Directory - -After a run, Kane CLI creates an output directory next to the test file: - -``` -amazon_test.md -output-amazon/ -Result.md # Human-readable run report -.internal/ # Cached recordings, do not edit -playwright-python-code/ # Only if code_export is enabled -``` - -`output-/` is **commit-safe**. Commit it to git so teammates and CI replay the same recordings. - -For tests using `@import`, helper recordings land next to the helper file in `helper-output---/` directories. These are also commit-safe. - -### Result.md - -After each run, `Result.md` is generated with: - -| Section | Contents | -|---------|----------| -| Frontmatter | `status`, `started`, `duration_s`, `session_id` | -| Step results | One entry per step: `✓ passed`, `✗ failed`, or `⏭ skipped` (suffixed `(optional)` for soft-failing optional steps) | -| Import failure paths | For `@import` steps that failed, the path to the failing sub-step inside the helper | - -To check whether a test passed or where it failed, read `Result.md` instead of re-running the test. - -## CI/CD Usage - -```bash -kane-cli testmd run ./tests/checkout_test.md \ ---agent \ ---headless \ ---on-lock-conflict wait \ ---retry -``` - -| Flag | Purpose | -|------|---------| -| `--agent` | Structured NDJSON to stdout | -| `--headless` | No browser window | -| `--on-lock-conflict wait` | Block instead of failing if a teammate is editing the same test | -| `--retry` | Automatically recover from transient replay failures | - -### Exit Codes - -| Code | Meaning | -|------|---------| -| 0 | ✅ Passed | -| 1 | ❌ Failed (test ran but did not pass) | -| 2 | ⚠️ Error (auth, setup, parse error, or `--on-lock-conflict fail`) | -| 3 | ⏱️ Timeout, cancelled, or `--on-lock-conflict wait` timed out | - -## Common Parse Errors - -Parse errors abort **before** any browser launch with exit code `2`: - -| Error Message | Fix | -|---------------|-----| -| `frontmatter is missing closing '---'` | Add the trailing `---` | -| `invalid YAML in frontmatter` | Validate the YAML block | -| `step body must be exactly one of prose / @import` | Split into two steps | -| `step config on @import may only contain 'optional'` | Remove other keys from the yaml block | -| `cannot @import a test file` | Imports may only reference helpers (not files ending in `_test.md`) | -| `cyclic reference` | Restructure helpers to break the circular dependency | -| `chrome config is global-only` | Move Chrome key to root frontmatter | -| `'' is run-level and cannot be set per-step` | Move `mode` / `on_lock_conflict` to root frontmatter | -| `unknown config key` | Remove or fix the key | -| `auth/identity keys are CLI-only` | Pass `username` / `access_key` as CLI flags, not in frontmatter | - -## Example: Full Test with Imports - -**`tests/checkout_test.md`:** - -```markdown ---- -mode: testing -headless: true -variables: -username: -value: "testuser@example.com" -password: -value: "s3cret!" -secret: true ---- - -# Checkout flow - -## Login -@import ./helpers/login.md - -## Add item to cart -Go to the products page, search for "wireless headphones", and click "Add to Cart" on the first result. - -## Verify cart -Go to the cart page. Assert the cart contains 1 item. Store the total price as 'cart_total'. - -## Complete checkout -Click "Proceed to Checkout", fill in shipping details, and assert the order confirmation page loads. -``` - -**`tests/helpers/login.md`:** - -```markdown -# Login helper - -## Open the login page -Go to https://app.example.com/login. - -## Sign in -Enter {{username}} in the email field and {{password}} in the password field, then click "Sign In". Assert the dashboard loads. -``` - -**Run:** - -```bash -kane-cli testmd run tests/checkout_test.md --agent -``` - -## Next Steps - -- [Writing Objectives](/support/docs/kane-cli-writing-objectives/): Learn how to write effective natural language objectives -- [Variables & Context](/support/docs/kane-cli-variables-and-context/): Parameterize tests with variables, secrets, and context files -- [Agent Mode](/support/docs/kane-cli-agent-mode/): Parse structured NDJSON output from Kane CLI -- [CI/CD Integration](/support/docs/kane-cli-cicd/): Add Kane CLI to your pipeline -- [CLI Reference](/support/docs/kane-cli-cli-reference/): Full flag and command reference -- [Troubleshooting](/support/docs/kane-cli-troubleshooting/): Debug common issues diff --git a/static/docs/kane-cli-tms-integration.md b/static/docs/kane-cli-tms-integration.md deleted file mode 100644 index b1fb9ba83..000000000 --- a/static/docs/kane-cli-tms-integration.md +++ /dev/null @@ -1,143 +0,0 @@ -# Test Manager Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -By default, Kane CLI uploads each session to TestMu AI Test Manager as a test case. This page covers what gets uploaded, where it ends up, and the related features: project and folder selection, code export, share links, the post-session feedback prompt, and the local session directory. - -This is the default behaviour for every session in both the TUI and the CLI. You do not need to opt in. - -## What Gets Uploaded - -When a session ends, Kane CLI runs an upload pipeline that finalises the test case in Test Manager. Two kinds of artefacts move from your machine to TestMu AI: - -- **Screenshots**: uploaded incrementally as the agent runs. By the time the session ends, most of them are already in place. PNG screenshots are converted to WebP before upload. -- **Run artefacts and metadata**: packaged at session exit. This includes per-run action logs, an execution blob describing the run (objective, status, steps, durations, variables in scope), and the zipped contents of the session run directories. - -You see a short progress indicator at exit covering these stages: `convert` (preparing metadata), `zip` (packaging run directories), `presign` (requesting upload URLs), `upload` (sending the zip and metadata), and `finalize` (committing the test case in Test Manager). If code export is enabled, a `code_export` stage runs after `finalize`. If it is disabled, that stage is reported as skipped. - -If the upload fails, Kane CLI prints the error and exits. The local session directory is preserved either way, see [Session History on Disk](#session-history-on-disk). - -## Choosing Where Uploads Land - -Each test case is filed under a Test Manager project. Optionally, you can also choose a folder inside that project. - -### Project - -Configure your project once, and every subsequent session uploads under it: - -```bash -kane-cli config project -``` - -With no value, this opens a search-as-you-type picker. Projects are loaded from your Test Manager account on demand. Type to filter, use the arrow keys to navigate pages of five, press **Enter** to select, or **Tab** to create a new project. If your account has no projects yet, the picker jumps straight to the create flow. - -You can also set a project non-interactively by ID: - -```bash -kane-cli config project -``` - -In the TUI, the same picker is available via the `/config` command. - -### Folder - -Folders are optional. Once a project is set, you can pick a folder the same way: - -```bash -kane-cli config folder -``` - -This opens a search-as-you-type folder picker. Type to filter, **Enter** to select, **Tab** to create a new folder. Each folder is shown with its current test case count. - -```bash -kane-cli config folder -``` - -If you try to set a folder before choosing a project, Kane CLI asks you to pick a project first. - -### Finding Your Test Case in Test Manager - -After a successful upload, Kane CLI prints two links to the terminal (see [Share Links at Session Exit](#share-links-at-session-exit)). Both lead into the TestMu AI Test Manager for the test case that was just created. - -## Code Export - -Code export converts a completed test case into runnable Playwright code that you can check into your repository or extend by hand. - -### What It Produces - -The supported languages are **Python with Playwright** and **JavaScript with Playwright**. Code export runs server-side after the test case is finalised. Kane CLI polls Test Manager until generation is complete, then downloads the resulting files. - -### Enabling Code Export - -Code export is **off by default**. Turn it on either in your stored config or per-run on the CLI. - -In `~/.testmuai/kaneai/tui-config.json`, set the `code_export` block: - -```json -{ -"code_export": { -"enabled": true, -"language": "python", -"skip_validation": true -} -} -``` - -For one-off CLI runs, use flags: - -```bash -kane-cli run "Add an item to the cart" \ ---code-export \ ---code-language python \ ---skip-code-validation -``` - -`--no-skip-code-validation` forces validation on for that run. `--code-language` accepts `python` or `javascript`. - -### Where to Find the Output - -Generated code is downloaded into a local directory under your session, by default `~/.testmuai/kaneai/sessions//code-export/`. At session exit Kane CLI prints a `CodeExport` line in the links box pointing to that directory. - -The same code is also available alongside the test case in Test Manager. - -## Share Links at Session Exit - -When the upload finishes successfully, Kane CLI prints a small links block to the terminal. You see up to three links, depending on what was generated: - -- **ShareLink**: a shareable session URL on TestMu AI Test Manager. Anyone you send this link to can view the run summary without needing access to the project. Links are issued with a seven-day expiry. -- **TestCase**: a direct URL to the test case dashboard inside your Test Manager project. Use this when you are signed in to TestMu AI and want to drill into the run. -- **CodeExport**: a `file://` link to the directory on your machine that holds the generated Playwright code. Only shown when code export is enabled and produced files. - -The links survive the terminal exit and remain in your scrollback. - -## Feedback Prompt - -After a session uploads successfully, Kane CLI shows a thumbs-up / thumbs-down prompt. Use the left and right arrow keys to switch between the two options, **Enter** to submit, or **Esc** to skip. Skipping sends nothing. - -If you submit, Kane CLI posts your choice (positive or negative) for the test case that was just finalised. Both the TUI and the CLI render the same prompt; in the CLI it appears on stderr after the links block. - -You can also submit feedback after the fact for a known test case: - -```bash -kane-cli feedback --test-id --feedback-type positive -kane-cli feedback --test-id --feedback-type negative --details "..." -``` - -## Session History on Disk - -Every session, regardless of upload outcome, leaves a directory on your machine: - -```text -~/.testmuai/kaneai/sessions// -├── session.json # Metadata: started_at, ended_at, model, profile, -│ # test_id, testcase_id, upload_status, run summaries -├── tui.log # Append-only event log for the TUI session -├── runs// # One directory per run in the session -│ └── run-test/ -│ └── actions.ndjson # Step-by-step record of what the agent did -└── code-export/ # (when code export is enabled) Generated code files -``` - -`` is a UUID generated when the session starts. The runner writes its step logs and screenshots into `runs//`, where `n` increments per run within the session. - -This directory is useful when you want to look back at a past run without going to Test Manager, debug a session that failed to upload, or hand a teammate the raw artefacts. diff --git a/static/docs/kane-cli-troubleshooting.md b/static/docs/kane-cli-troubleshooting.md deleted file mode 100644 index f2e8ea84b..000000000 --- a/static/docs/kane-cli-troubleshooting.md +++ /dev/null @@ -1,267 +0,0 @@ -# Troubleshooting - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Log Locations - -Before diagnosing, know where to look: - -| Log | Path | -|-----|------| -| Run text log | `{run_dir}/run.log` | -| Step detail (JSON) | `{run_dir}/run-test/step_NNN.json` | -| Step screenshot | `{run_dir}/run-test/screenshots/step_NNN.png` | -| Run summary | `{run_dir}/run-test/run_summary.json` | -| Session log | `{session_dir}/tui.log` | -| All sessions | `~/.testmuai/kaneai/sessions/` | - -The `run_end` event in Agent Mode provides `session_dir` and `run_dir` directly. - -## Chrome Issues - -### "Chrome failed to launch" - -**Cause:** Chrome is not installed, all CDP ports in the 9222–9230 range are in use, or a profile lock from another running Chrome. - -Kane CLI manages a Chrome process and connects to it over the Chrome DevTools Protocol (CDP). On macOS it looks under `/Applications/Google Chrome.app`; on Linux it looks for `google-chrome`, `google-chrome-stable`, `chromium`, and similar binaries; on Windows it looks under `Program Files\Google\Chrome\Application\chrome.exe` and `AppData\Local`. - -**Fix:** -1. Install Google Chrome if not present -2. Check for processes on CDP ports: -```bash -lsof -i :9222-9230 -``` -3. Quit any extra Chrome processes hoarding the 9222–9230 port range -4. Pick a different Chrome user-data directory, or quit the Chrome instance using it. See [Chrome Management](/support/docs/kane-cli-configuration/#chrome-management) -5. If you only need to connect to an already-running Chrome: -```bash -kane-cli run "..." --cdp-endpoint http://localhost:9222 -``` - -### "CDP endpoint not reachable" - -**Cause:** Using `--cdp-endpoint` but Chrome is not running on that port. - -**Fix:** Remove `--cdp-endpoint` and let Kane CLI manage Chrome automatically. Or start Chrome with remote debugging before running: - -```bash -google-chrome --remote-debugging-port=9222 & -kane-cli run "..." --cdp-endpoint http://localhost:9222 -``` - -### Chrome opens then closes immediately - -**Cause:** Another Kane CLI instance is already running and holds the Chrome profile lock. - -**Fix:** Check for running kane-cli processes: -```bash -ps aux | grep kane-cli -``` -Kill any existing processes, then retry. - -## Authentication Issues - -### "Authentication failed" (exit code 2) - -**Cause:** Expired tokens or incorrect credentials. - -**Fix for interactive use:** -1. Re-run the login flow: -```bash -kane-cli login -``` -2. Confirm which profile, environment, and token state are active: -```bash -kane-cli whoami -``` - If the token is missing or expired and refresh did not succeed, log in again. - -**Fix for CI / non-interactive use:** - -Verify both values against the credentials shown in your TestMu AI dashboard, then pass them on the command line: - -{`kane-cli run "" \\ - --username "${ YOUR_LAMBDATEST_USERNAME()}" \\ - --access-key "${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -If they still do not work, regenerate the access key in the dashboard and retry. - -### "Not configured" on first run - -**Cause:** No profile exists yet. - -**Fix:** Run the login flow: - -{`kane-cli login --username "${ YOUR_LAMBDATEST_USERNAME()}" --access-key "${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -Get credentials from the TestMu AI [dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard) > **Credentials**. - -### Basic auth not working - -**Cause:** Wrong username or access key. - -**Fix:** Verify your credentials on the TestMu AI dashboard. Username and access key are case-sensitive. Make sure you're using the access key (not the password). - -## Run Issues - -### "Run timed out" or "max steps exceeded" - -**Cause:** Objective is too complex, page is slow to load, or `--max-steps` is too low. - -**Fix:** -- Increase `--timeout`: `--timeout 300` -- Increase `--max-steps`: `--max-steps 60` -- Break the work into smaller objectives. Run several sequential `kane-cli run` invocations, each focused on one logical sub-task. The session keeps the same browser between runs, so state carries over. -- Tighten the objective. Vague objectives often cause the agent to wander; describe the target outcome and any required values up front. - -### Agent repeats the same action - -**Cause:** The agent is stuck in a loop: the page didn't change after the action. - -**Fix:** Rephrase the objective to be more explicit. Add an assertion after the action to confirm state changed: -``` -"click the Save button, assert the page shows 'Saved successfully'" -``` - -### "Variables not resolving": `{{key}}` appears literally - -**Cause:** Variable file not loaded, wrong JSON format, or wrong variable key name. - -**Fix:** -1. **JSON syntax.** Variable files are JSON. A missing comma or unquoted key will cause the file to be skipped silently. -2. **File location.** Confirm your file is in the right place, see [loading order](/support/docs/kane-cli-variables-and-context/#loading-order). -3. **Inline test.** Bypass file loading by passing the variable on the command line: -```bash -kane-cli run "log in as {{user}}" \ ---variables '{"user":{"value":"alice"}}' -``` - If the inline form works, the issue is with file loading, not the variable itself. - -### Assertions fail even though the page looks correct - -**Cause:** The assertion phrasing doesn't match what's on the page, or there's a timing issue. - -**Fix:** -1. Check the screenshot at `{run_dir}/run-test/screenshots/step_NNN.png`: see exactly what the agent saw -2. Refine the assertion: use `assert the page contains` (substring) instead of exact text -3. Add a wait: `"wait for the confirmation message to appear, then assert..."` - -## Upload Issues - -### "Upload failed" or "Test Manager error" - -**Cause:** Kane CLI uploads run artifacts to TestMu AI Test Manager at the end of the session. If the upload fails: - -**Fix:** -1. **Authentication.** Re-check `kane-cli whoami` and re-login if needed. Test Manager upload requires a valid token (or basic auth) for the configured environment. -2. **Network connectivity.** The upload talks to the TestMu AI control plane and a cloud storage endpoint. Verify outbound HTTPS is not blocked by a proxy or firewall. -3. **Project is set.** The pipeline will not commit a test case without a project. Confirm one is configured: -```bash -kane-cli config show -``` - If `project_id` is empty, set it with `kane-cli config project` or pick one in the TUI. - -## Agent Mode Issues - -### No NDJSON output / only seeing TUI - -**Cause:** Missing `--agent` flag. - -**Fix:** Add `--agent` to your command: -```bash -kane-cli run "..." --agent --headless -``` - -### NDJSON parsing fails: `jq` errors or unexpected output - -**Cause:** Stderr is mixing with stdout, or you're trying to parse mid-stream events. - -**Fix:** Redirect stderr and use `tail -1` to get only the `run_end` event: -```bash -kane-cli run "..." --agent 2>/dev/null | tail -1 | jq . -``` - -### `ask_user` event fires and blocks the run - -**Cause:** The objective requires human input in an agent context. - -**Fix:** Rewrite the objective to avoid prompts. For example, instead of "navigate through the sign-up flow", be explicit: -``` -"click Sign Up, fill email with '{{email}}', fill password with '{{password}}', click Create Account" -``` - -## Installation Issues - -### `kane-cli: command not found` after install - -**Cause:** npm global bin directory is not in your PATH. - -**Fix:** -```bash -npm config get prefix - -# Add to PATH (adjust path based on above output) -export PATH="$(npm config get prefix)/bin:$PATH" - -# Make permanent: add to ~/.zshrc or ~/.bashrc -echo 'export PATH="$(npm config get prefix)/bin:$PATH"' >> ~/.zshrc -``` - -### Installation fails - -**Cause:** Node.js version is below 18. - -**Fix:** Check your version and upgrade: -```bash -node --version # Must be 18 or higher -``` - -### Install fails with "sharp: Please add node-addon-api" - -**Symptom:** `npm install -g @testmuai/kane-cli` fails with `sharp: Please add node-addon-api to your dependencies` (any Node version, any platform). - -Kane CLI 0.3.4+ treats `sharp` as an optional dependency, so the install still succeeds even if sharp fails. Screenshots simply upload as PNG instead of WebP (about 30% larger, no functional impact). On an older version, upgrade first with `npm install -g @testmuai/kane-cli@latest`. - -**Cause:** `sharp` powers optional PNG to WebP screenshot compression. When it cannot load its prebuilt binary it tries to build from source, which fails. The most common trigger on macOS is a system-wide libvips (often pulled in by `brew install appium`, `imagemagick`, or `gdal`). - -**Fix (most common, macOS):** -```bash -# Diagnose: a printed version means libvips is the cause -pkg-config --modversion vips-cpp - -# Bypass libvips detection. Uninstall first, since npm considers -# kane-cli already installed and will not re-resolve sharp otherwise. -npm uninstall -g @testmuai/kane-cli -SHARP_IGNORE_GLOBAL_LIBVIPS=1 npm install -g @testmuai/kane-cli - -# Make it permanent -echo 'export SHARP_IGNORE_GLOBAL_LIBVIPS=1' >> ~/.zshrc && source ~/.zshrc -``` - -Two other triggers: npm configured to skip optional dependencies (`npm config get omit` should not contain `optional`, so clear it with `npm config delete omit` and reinstall), and a proxy or private registry that does not forward the `@img` scope (add an `@img:registry=https://registry.npmjs.org/` pass-through). If you are fine with PNG screenshots, no action is needed. - -## "Update available" Notice - -Kane CLI checks the public npm registry for a newer release once every 24 hours. The result is cached locally so the check itself is non-blocking and silent on failure. When a newer version exists, Kane CLI surfaces an "update available" notification with the current and latest versions and a severity label (`major`, `minor`, or `patch`). - -The notice is informational, your current version still works. To upgrade, follow the steps in [Updates](/support/docs/kane-cli-installation/#update). - -## Filing a Bug Report - -If you encounter behavior that looks like an agent bug (not auth, timeout, or a vague objective), file an issue: - -**[github.com/LambdaTest/kane-cli/issues](https://github.com/LambdaTest/kane-cli/issues)** - -Include the following: - -| Field | How to Get It | -|-------|---------------| -| Kane CLI version | `kane-cli --version` | -| OS | macOS (ARM/Intel), Linux (x64/ARM64), Windows (x64) | -| What happened | Describe the behavior | -| Reproduction steps | The exact `kane-cli run` command and objective | -| Expected behavior | What should have happened | -| Logs | `run_summary.json` and `step_NNN.json` from `run_dir` | -| Screenshot | `screenshots/step_NNN.png` from `run_dir` | - -Do NOT file bug reports for: auth issues, low timeouts, vague objectives, or site-side errors (CAPTCHAs, 500 errors). diff --git a/static/docs/kane-cli-variables-and-context.md b/static/docs/kane-cli-variables-and-context.md deleted file mode 100644 index ea1302f06..000000000 --- a/static/docs/kane-cli-variables-and-context.md +++ /dev/null @@ -1,199 +0,0 @@ -# Variables & Context - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -**Variables** keep credentials and test data out of your objectives. **Context files** give the agent persistent background information: guidance, conventions, and notes that apply across runs. - -## Variables - -### Format - -Variables are JSON objects keyed by name. Each entry describes a single variable: - -```json -{ -"username": { "value": "alice", "secret": false }, -"api_key": { "value": "sk-live-...", "secret": true } -} -``` - -| Field | Required | Type | Default | Description | -|-------|----------|------|---------|-------------| -| `value` | Yes | string | None | The variable's value. Entries without `value` are ignored. | -| `secret` | No | boolean | `false` | When `true`, the value is masked in logs and routed to the TestMu AI secrets store instead of being synced as plain Test Manager variables. | - -### Usage in Objectives - -Reference variables with `{{key}}` syntax: - -```bash -kane-cli run \ ---url https://myapp.com \ ---variables-file ./creds.json \ -"fill the email field with '{{email}}', -fill the password field with '{{password}}', -click Login, -assert the Dashboard is visible" -``` - -Before the objective is sent to the agent, `{{email}}` and `{{password}}` are rewritten to internal namespaced forms; the agent sees the resolved values at runtime. - -### Loading Order - -Variables are merged from four sources in this order. Later sources override earlier ones for the same key: - -1. **Global directory**: `~/.testmuai/kaneai/variables/*.json` -2. **Local project directory**: `.testmuai/variables/*.json` (relative to where you invoke `kane-cli`) -3. **File flag**: `--variables-file ` -4. **Inline flag**: `--variables ''` (highest priority) - -Within a directory, files are read in alphabetical order; later files override earlier files for duplicate keys. Files that fail to parse as JSON are skipped with a warning. - -### Inline Variables - -Pass a JSON object directly on the command line: - -```bash -kane-cli run "Log in as {{username}}" \ ---variables '{"username": {"value": "alice"}}' -``` - -### Variables from a File - -Point at a single JSON file: - -```bash -kane-cli run "Log in as {{username}}" \ ---variables-file ./vars.json -``` - -### Project-Local Variables - -Drop one or more `*.json` files into `.testmuai/variables/` inside your project's working directory. They load automatically whenever you run `kane-cli` from that directory. - -```text -my-project/ -├── .testmuai/ -│ └── variables/ -│ ├── credentials.json -│ └── urls.json -└── ... -``` - -Project-local variables override global variables but are overridden by file and inline flags. - -### Global Variables - -For values you want available across every project on your machine, place `*.json` files in `~/.testmuai/kaneai/variables/`. - -```text -~/.testmuai/kaneai/variables/ -├── personal.json -└── shared.json -``` - -Global variables have the lowest precedence, anything else with the same key wins. - -### Example Variable File - -```json -{ -"app_url": { "value": "https://staging.myapp.com" }, -"admin_email": { "value": "admin@example.com" }, -"admin_password": { "value": "admin_pass_123", "secret": true }, -"customer_email": { "value": "customer@example.com" }, -"customer_password": { "value": "customer_pass_456", "secret": true }, -"test_product_sku": { "value": "PROD-2024-001" } -} -``` - -### Secrets - -Mark a variable as secret by setting `"secret": true`: - -```json -{ -"api_key": { "value": "sk-live-abc123", "secret": true } -} -``` - -Secret values are masked in displayed output and logs, and are routed to the TestMu AI secrets store instead of being synced to Test Manager as plain variables. Use this for credentials, tokens, and anything else that should not appear in shareable artifacts. - -Do not commit credential files to version control. Add `.testmuai/variables/` to your `.gitignore`, or use environment variable substitution in CI/CD. - -## Context Files - -Context files are plain Markdown files whose contents are passed to the agent alongside your objective. Use them for standing instructions: coding conventions, accounts to use, sites to avoid, or domain knowledge the agent should always have. - -### Two Levels - -| Level | Path | Use For | -|-------|------|---------| -| **Global** | `~/.testmuai/kaneai/global-memory.md` | Company-wide terminology, shared test accounts, universal patterns | -| **Local** | `.testmuai/context.md` (in project directory) | App-specific navigation, known UI quirks, test environment details | - -### Example Local Context File - -```markdown -# MyApp Staging Context - -## Application Overview -MyApp is a SaaS project management tool. Users create projects, invite members, and track tasks. - -## Test Environment -- URL: https://staging.myapp.local -- Database resets daily at 2 AM UTC -- File uploads are disabled in staging - -## Navigation Patterns -- Main menu is in the left sidebar (hover to expand) -- Settings is under the top-right user avatar menu -- Deep links work: /dashboard/projects/123/tasks - -## Known UI Quirks -- The modal close button sometimes needs two clicks -- Date picker defaults to today: click the field to open -- The "Copy Link" toast appears bottom-right for 3 seconds - -## Common Test Flows -1. Create a project: Dashboard > "New Project" > fill name > "Create" -2. Invite a member: Project Settings > "Team" > "Invite" > enter email > "Send" -3. Complete a task: Tasks page > click task > "Mark Complete" > confirm dialog - -## Test Data -- Existing project for testing: "Test Project" (ID: proj_12345) -- Existing user: john@example.com -``` - -### Override Per Run - -Override either context file for a single run: - -```bash -kane-cli run "your objective" \ ---global-context ./custom-global.md \ ---local-context ./custom-local.md -``` - -If a context file is missing or empty, it is silently ignored, no error is raised. - -### What to Put in Context - -**Global context:** -- Company terminology and naming conventions -- Standard test account references (no hardcoded values) -- Performance expectations (e.g., "page loads in under 3 seconds") -- Universal navigation patterns - -**Local context:** -- App overview and purpose -- Test environment URL and known limitations -- Navigation patterns specific to this app -- Known UI quirks and workarounds -- Common test flows -- Test data references (IDs, emails: no passwords) - -**Keep out of context:** -- Hardcoded secrets (use variables) -- Large code blocks or logs -- Information that changes frequently diff --git a/static/docs/kane-cli-writing-objectives.md b/static/docs/kane-cli-writing-objectives.md deleted file mode 100644 index dc78c77e3..000000000 --- a/static/docs/kane-cli-writing-objectives.md +++ /dev/null @@ -1,176 +0,0 @@ -# Writing Objectives - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The objective string is the most important input to Kane CLI. How you phrase it determines what the agent does and whether it succeeds. Objectives follow three patterns that you can combine freely. - -## Three Patterns - -| Pattern | Keywords | Purpose | -|---------|----------|---------| -| 🎯 **Action** | go to, click, type, fill, search, scroll, hover, select, submit | Performs browser actions | -| ✅ **Assertion** | assert, verify, confirm, check that, ensure | Validates a condition: produces pass/fail | -| 📦 **Extraction** | store X as 'name' | Reads a value from the page, persists in output | - -**Assertions** and **extractions** are evaluated as [Checkpoints](/support/docs/kane-cli-checkpoints/). See the reference for all analyze methods (Visual, Textual, URL, Title, DevTools) and comparison operators. - -### Actions - -Use imperative verbs to describe what the agent should do: - -``` -"go to https://example.com" -"click the 'Add to Cart' button" -"fill the email field with user@example.com" -"scroll down to the pricing section" -"select 'Premium' from the plan dropdown" -``` - -### Assertions - -Assertions validate page state. The test fails if the condition is not met. - -``` -"assert the page contains 'Order Confirmed'" -"verify the cart total shows '$29.99'" -"confirm no error message is visible" -"check that the Submit button is disabled" -``` - -### Extractions: The "store as" Pattern - -Extractions read a value from the page and store it in the run output's `final_state` field. - -**Always use the explicit `store X as 'name'` syntax.** Vague phrasing like "tell me" or "read" does not reliably capture data. - -❌ Bad: agent may see it but won't persist it: -``` -"go to example.com and tell me the price" -"read the page title" -``` - -✅ Good: value is captured in `final_state`: -``` -"go to example.com, store the price of the first item as 'price'" -"store the page title as 'page_title'" -``` - -## Combining Patterns - -Chain all three patterns in one objective: - -``` -"go to {{app_url}}/dashboard, -store the welcome message as 'welcome_text', -store the user role in the sidebar as 'role', -assert the role is 'Admin', -click the Settings button, -assert the page contains 'Account Settings'" -``` - -This objective: navigates → extracts two values → validates a condition → performs an action → validates the result. - -## Real Examples - -### Login flow - -```bash -kane-cli run \ ---url https://app.example.com \ ---variables-file ./creds.json \ -"fill the email field with '{{email}}', -fill the password field with '{{password}}', -click the Login button, -assert the page contains 'Dashboard', -store the logged-in username as 'username'" -``` - -### Search with filters - -```bash -kane-cli run \ ---url https://shop.example.com/products \ -"click the Category filter, -select 'Electronics', -click the Sort dropdown, -select 'Price: Low to High', -store the count of products as 'result_count', -assert the result_count is greater than 0" -``` - -### Settings change - -```bash -kane-cli run \ ---url https://app.example.com/account \ -"click the Settings tab, -scroll down to the Notification section, -check the 'Email Notifications' checkbox, -click Save, -assert the page shows 'Settings saved successfully'" -``` - -### Checkout - -```bash -kane-cli run \ ---url https://shop.example.com/cart \ ---variables-file ./payment.json \ -"store the cart total as 'total', -assert the total is greater than 0, -click Checkout, -fill the card number field with '{{card_number}}', -fill the expiry field with '{{card_expiry}}', -fill the CVC field with '{{card_cvc}}', -click Place Order, -assert the page contains 'Order Confirmed', -store the order number as 'order_id'" -``` - -## Assertion Types - -| Type | Example | -|------|---------| -| Exact match | `assert the total shows '$29.99'` | -| Contains | `assert the page contains 'Welcome'` | -| State | `assert the Submit button is disabled` | -| Conditional | `if a cookie banner appears, dismiss it, then assert the homepage loads` | -| Negative | `assert no error message is visible` | -| Positional | `assert 'Settings' appears in the left sidebar` | - -## Dos and Don'ts - -| ✅ Do | ❌ Don't | -|-------|---------| -| Use imperative verbs: "go to", "click", "store as" | Use vague verbs: "check out", "look at", "explore" | -| Be specific: "click the **blue** 'Submit' in the checkout form" | Be ambiguous: "click the button" | -| Name extractions: "store X as 'name'" | Expect implicit capture: "tell me the price" | -| Use `{{variables}}` for credentials and URLs | Hardcode secrets in the objective | -| Always provide `--url` | Assume the agent knows where to start | -| Split objectives with >15 steps | Cram 30 steps into one objective | - -## Splitting Long Objectives - -Objectives with more than 15 steps drift and become unreliable. Split them into multiple runs. - -In **Interactive TUI**, the browser stays open between runs: state carries over automatically: - -``` -> go to https://myapp.com and log in as admin -✓ PASSED - -> navigate to User Management and create a new user "qa@example.com" -✓ PASSED - -> verify the new user appears in the users table -✓ PASSED -``` - -In **Headless CLI**, use `--max-steps` to cap each run: - -```bash -kane-cli run "login flow" --url https://myapp.com --max-steps 10 -kane-cli run "dashboard check" --url https://myapp.com --max-steps 10 -``` - -A good rule: if your objective has more than three `assert` statements or more than five distinct actions, consider splitting it. diff --git a/static/docs/kaneai-advanced-settings.md b/static/docs/kaneai-advanced-settings.md deleted file mode 100644 index 6cec8a1a5..000000000 --- a/static/docs/kaneai-advanced-settings.md +++ /dev/null @@ -1,137 +0,0 @@ -# Advanced Settings in KaneAI Test Case - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -KaneAI allows you to edit the advanced settings of an existing test case directly from the **Test Summary** page. This gives you the flexibility to update configurations like network settings, timezone, Chrome options, and custom headers after a test has already been authored, without having to recreate it from scratch. - -This feature is supported across all platforms: **Desktop Web**, **Mobile App**, and **Mobile Web**. - -## Overview - -When you edit advanced settings for a test case, you have two options: - -- **Apply Settings**: Saves the updated settings and generates a new version of the test case with regenerated code. Use this when the setting changes do not affect the test steps themselves. -- **Edit Test Steps**: Saves the updated settings and opens the playground so you can make relevant changes to your test steps. Use this when the updated settings may require modifications to the test flow. - -## Platform-Specific Advanced Settings - -The available settings vary depending on the platform of your test case. Refer to the relevant section below for the complete list of configurable options. - -### Desktop Web - -| Setting | Control Type | Description | -|---|---|---| -| **Network Configuration** | Dropdown (Tunnel / Inactive) | Select tunnel or proxy for accessing applications behind firewalls | -| **Timezone** | Dropdown | Set the timezone where your tests will run | -| **Chrome Options** | Toggle + multi-input | Add up to 10 Chrome browser command-line flags | -| **Custom Headers** | Toggle + multi-input | Add up to 10 custom HTTP headers to inject in your test case | - -### Android App - -**Top-level settings:** - -| Setting | Control Type | Description | -|---|---|---| -| **Network Configuration** | Dropdown (Tunnel / Inactive) | Select tunnel or proxy for accessing apps behind firewalls | -| **Auto-login through Google** | Toggle + credentials | Start session pre-authenticated with a Google account | -| **GPS Location** | Toggle + lat/long input | Mock GPS coordinates on real device | - -**Session settings:** - -| Setting | Control Type | Description | -|---|---|---| -| **Auto-Grant Permissions** | Toggle | Auto-grant all Android Manifest permissions | -| **Enable Keyboard** | Toggle | Use specialized Unicode-enabled keyboard | -| **Ignore Unimportant Views** | Toggle | Ignore views marked unimportant for accessibility | -| **Disable UI Idle Wait** | Toggle | Skip idle wait before commands (Flutter, React Native) | -| **Enable Secure Screenshot Mocking** | Toggle | Mock secure screenshots (Salesforce use case) | -| **Disable Screenshot Block** | Toggle | Disable secure screenshot blocking | -| **Biometric Authentication** | Toggle | Enable Fingerprint / Touch ID workflows (Android 11+) | -| **Image Injection** | Toggle | Enable image capture, QR code, barcode scanning | -| **Video Injection** | Toggle | Inject video feed for live detection / face recognition | -| **Device Orientation** | Dropdown (Auto) | Set device orientation for session | -| **Capture Network Logs** | Toggle | Capture network logs for network assertions | -| **Network Throttling** | Dropdown | Select throttling profile | -| **Timezone** | Dropdown | Set timezone for test session | -| **Language** | Dropdown | Set language for test session | - -For more details on mobile app capabilities like biometric authentication, image injection, and video injection, see [Advanced Mobile Features](/support/docs/kane-ai-mobile-app-capabilities/). - -### iOS App - -**Top-level settings:** - -| Setting | Control Type | Description | -|---|---|---| -| **Network Configuration** | Dropdown (Tunnel / Inactive) | Select tunnel or proxy | -| **GPS Location** | Toggle + lat/long input | Mock GPS coordinates on real device | - -**Session settings:** - -| Setting | Control Type | Description | -|---|---|---| -| **Upload File** | Toggle | Enable file uploads to the application | -| **Auto-Accept Alerts** | Toggle | Auto-accept system/app-level alerts | -| **Auto-Dismiss Alerts** | Toggle | Auto-dismiss system/app-level alerts | -| **Biometric Authentication** | Toggle | Enable Fingerprint / Touch ID | -| **Image Injection** | Toggle | Image capture, QR code, barcode scanning | -| **Video Injection** | Toggle | Inject recorded video feed | -| **Device Orientation** | Dropdown (Auto) | Set orientation | -| **Capture Network Logs** | Toggle | Capture network logs | -| **Network Throttling** | Dropdown | Select throttling profile | -| **Timezone** | Dropdown | Set timezone | -| **Language** | Dropdown | Set language | - -### Android Web - -Android Web test cases share most settings with [Android App](#android-app), with the following differences: - -- **Custom Headers** is available (Toggle + multi-input, add up to 10 custom HTTP headers) -- **Biometric Authentication**, **Image Injection**, and **Video Injection** are **not** available - -### iOS Web - -iOS Web test cases share most settings with [iOS App](#ios-app), with the following differences: - -- **Custom Headers** is available (Toggle + multi-input, add up to 10 custom HTTP headers) -- **Biometric Authentication**, **Image Injection**, and **Video Injection** are **not** available - -## How to Edit Advanced Settings - -### Step 1: Open the Test Summary Page - -Navigate to your test case in KaneAI and open the **Test Summary** tab. - -### Step 2: Open the Edit Test Steps Dialog - -Click on **Edit Test Steps** to open the advanced settings panel. You will see the current configuration for your test case. - -### Step 3: Modify the Settings - -Update the settings as needed. For example, you can: - -- Change the **Network Configuration** from `None` to `Geolocation` and select a region. -- Update the **Timezone** to a different zone. -- Toggle **Chrome Options** on or off and add command-line switches. -- Enable **Custom Headers** and add HTTP headers for your test. - -Settings that have been modified will display an **Edited** badge next to them. - -### Step 4: Save Your Changes - -Choose one of the two actions at the bottom of the dialog: - -- **Apply Settings**: Saves the settings and generates a new version with updated code. The test case remains on the Test Summary page. -- **Edit Test Steps**: Saves the settings and opens the playground so you can adjust the test steps to reflect the new configuration. - -## Version History - -Every time you apply updated settings, a new version of the test case is created. You can view and compare changes between versions from the **Version History** tab. - -The version comparison view shows a diff of what changed, for example, updated network configuration or timezone values, so you can track exactly what was modified in each version. - -## Limitations - -- **Advanced settings only**: Only advanced settings (network configuration, timezone, Chrome options, custom headers, and mobile session settings) can be edited from the test summary page. To change test steps, OS, browser, or device configurations, you need to open the playground. -- **New version generated on every apply**: Applying updated settings always creates a new version of the test case with regenerated code. There is no way to update settings in-place without generating a new version. -- **Version history diff for pre-rollout versions**: If you compare a version created after this feature was rolled out with a version created before it, the diff will have no advanced settings changes to display since the older version never tracked them. Similarly, if you open a pre-rollout version and click on advanced settings, only the latest settings are shown as the original settings were not recorded for that version. diff --git a/static/docs/kaneai-auto-heal.md b/static/docs/kaneai-auto-heal.md deleted file mode 100644 index 57c7c7440..000000000 --- a/static/docs/kaneai-auto-heal.md +++ /dev/null @@ -1,84 +0,0 @@ -# Auto-Heal for Automation Scripts in KaneAI with HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## What is Auto-Heal in KaneAI & HyperExecute? - -KaneAI, the GenAI Native testing assistant from TestMu AI, generates automation test scripts across multiple languages and frameworks using natural language instructions. When these scripts are executed via **HyperExecute**, TestMu AI’s high-performance test orchestration platform, they come with an added layer of reliability: the **Auto-Heal** feature. - -**Auto-Heal** ensures your automation scripts remain robust, even when the application under test undergoes changes like modified element locators (IDs, XPaths, CSS selectors). Instead of failing the test immediately when locators break, KaneAI dynamically finds new locators at runtime by leveraging the original natural language instruction used to execute the test case. - -## How is Auto-Heal Implemented? - -### Multi-Locator Fallback Approach - -- Every element identified by KaneAI comes with **multiple locators** (XPath, CSS, ID, etc.) at the time of script generation. -- During execution on HyperExecute, if the primary locator fails, the script automatically tries fallback locators sequentially. - -### Locator Healing via Natural Language Understanding - -- If **all locators fail**, KaneAI doesn't stop there. -- It re-evaluates the **original natural language instruction** that generated the test step. -- Using AI models trained on automation scenarios and web element patterns, KaneAI **rebuilds the locator on-the-fly** based on the intent of the instruction (e.g., “Click on the ‘Submit’ button”). -- The test step continues without manual intervention, ensuring minimal test flakiness and faster issue resolution. - -## Complexity of Auto-Heal Logic - -The core logic of Auto-Heal integrates several sophisticated systems working in real-time: - -1. **Locator Redundancy and Priority Mapping:** - - Prioritized fallback locators based on confidence scores. - -2. **Natural Language Parsing with Context Awareness:** - - Re-parsing instructions with contextual understanding of nearby elements, page structure, and DOM hierarchy. - -3. **Visual Query Integration (where applicable):** - - When locators cannot be determined purely through DOM, KaneAI utilizes **visual queries** to identify elements based on their appearance and relative positioning. - -4. **Scope of Auto-Heal:** - - Any command that requires the use of an element locator, such as **Click**, **Type**, or **Hover**, falls under the purview of auto-heal. - -## Human-in-the-loop review & approval - -Self-heal and agent-driven test modifications are surfaced for human review before they become permanent. When KaneAI adapts a locator or refactors a step, the change is captured with a before/after diff and an attributed audit-log entry. A reviewer can accept, reject, or edit the adaptation from the test summary; changes are never silently committed. Confidence-scored element matching automatically blocks low-confidence actions, and every accepted or rejected modification is recorded in Audit Logs for compliance. - -## Benefits of Auto-Heal - -- ✅ **Reduced Test Maintenance Effort:** - No need to manually update locators when the application UI changes. - -- ⚡ **Increased Automation Stability:** - Avoid flaky tests caused by minor DOM updates or attribute changes. - -- 🧠 **AI-Native Healing with Context:** - Uses the original intent behind instructions to intelligently recover from failures. - -- ⏱️ **Faster Execution with HyperExecute:** - Combined with the speed of HyperExecute, the healing process occurs in real-time without significant delays. - -- 📉 **Lower Failure Rate in CI/CD Pipelines:** - Ensures test cases remain green more consistently, even with frequent app changes. - -## Future Additions to Auto-Heal (In roadmap) - -- 🧭 **Detailed Healing Reports:** - Automatic reporting of healed locators, fallback paths taken, and suggestions for locator improvements. - -- 📌 **Auto-Update Healed Locators:** - Automatically update auto-healed locators in a new version of the test case script generated by KaneAI, further reducing maintenance overhead. - -## Video Explanation - -🎥 **[Watch the Demo: How Auto-Heal Works in KaneAI with HyperExecute](https://app.trupeer.ai/view?slug=R9mcuH)** - -The video walkthrough explains: -- How KaneAI generates scripts with fallback locators. -- Demo of locator failure and Auto-Heal in action. -- How KaneAI dynamically finds new locators using the original instruction. - -## Related Docs - -- [Selenium Auto Healing](/support/docs/auto-healing/):Auto-heal for Selenium web automation tests -- [Playwright Auto Healing](/support/docs/playwright-auto-healing/):Auto-heal for Playwright web automation tests -- [Smart Heal for Appium](/support/docs/smart-heal-appium/):AI-powered self-healing for mobile app automation on real devices -- [Auto Healing in HyperExecute](/support/docs/hyperexecute-auto-healing/):Auto-heal for tests executed via HyperExecute diff --git a/static/docs/kaneai-bulk-module-update.md b/static/docs/kaneai-bulk-module-update.md deleted file mode 100644 index 3a2612d70..000000000 --- a/static/docs/kaneai-bulk-module-update.md +++ /dev/null @@ -1,164 +0,0 @@ -# Bulk Module Update - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Maintaining many test cases that share the same module can become slow and error-prone when a module evolves. Each time a module is updated, you would otherwise have to manually open and edit every affected test case and bump the module version one by one. - -For an overview of modules, see [Modules](/support/docs/kane-ai-modules/). - -The **Bulk Module Update** feature solves this by allowing you to: - -- Update a validated module version across multiple eligible KaneAI test cases in one action -- Clearly see which test cases are eligible or ineligible for the update, and why -- Safely roll the new module version into your existing tests while keeping execution verification explicit and controlled - -**Note** -Bulk Module Update is currently supported only for **KaneAI test cases**. Support for non-KaneAI test cases is coming soon. - -**Limited Availability** -This feature is currently being rolled out gradually and may not be available to all users. If you don't see this feature in your account, it will be enabled for you soon. - -## How to Use Bulk Module Update - -### Step 1: Access Bulk Module Update - -You can start a bulk module update from two places: - -#### From the Module Detail Page - -1. Navigate to the module whose version you want to bulk-update -2. Click the **Update Test Cases** button -3. If the module version is not yet authored/validated or there are no eligible test cases, this button will be disabled - -#### From the Test Case Summary Page - -1. Open a test case that uses the module version you want to update -2. If there is a newer version of the same module and other test cases still use the older version, you will see an **Update Module for all Linked Test Cases** button -3. Click this button to open the bulk update flow, pre-filtered for that module and version - -### Step 2: Module Validation Check - -Before the bulk update flow starts, the system checks whether the selected module version is authored/validated and whether there are eligible test cases to update. - -#### When Update is Disabled - -The **Update Test Cases** button will be disabled if: - -- The module version is **not authored/validated** yet -- There are **no eligible test cases** (e.g., all linked test cases are faulty, unverified, or non-KaneAI) - -You will see a message such as: - -> "This module version is not authored yet. Author it in a test to enable bulk update." - -**To resolve:** Use the module in a test case and complete its authoring/validation, then ensure eligible KaneAI test cases are available. - -### Step 3: Select Test Cases - -After starting the flow, you will see a list of all test cases linked to the target module version. - -#### Ineligible Test Cases - -| Case | Description | -|--------|---------| -| **Faulty Test Cases** | Test Cases with **Faulty** status can't be updated | -| **Non-KaneAI Test Cases** | Bulk update supports only KaneAI test cases currently | -| **Unverified Test Cases** | Test Cases with **Unverified** status can't be updated | - -Ineligible test cases are greyed out, disabled for selection, and annotated with the reason. - -Select the eligible test cases you want to update and click **Next** to proceed. - -### Step 4: Review and Confirm - -Before running the bulk update, you will see a final summary screen showing: - -- The **target module version** (the version you are updating to) -- The **count of selected eligible test cases** -- Optionally, a **version movement summary** (e.g., from v1 → v2) - -**Coming Soon** -- **Seamless Module Validation** - Module validation will be performed automatically in future releases -- **Bulk Code-Gen Validation** - Bulk Code-Gen validation will be available in an upcoming update - -Click **Confirm Update** to trigger the bulk job and apply the new module version to all selected test cases. - -After triggering the update, you will see a success banner along with a toast message as shown in the screenshot below - -### Step 5: Post-Update Changes - -Once the bulk update is completed: - -#### Bulk Job Execution - -- A single bulk job is created -- All selected eligible test cases have their module version updated - -#### Changes on Each Updated Test Case - -For each updated test case: - -| Change | Description | -|--------|-------------| -| Module Version | Updated to the new version | -| Test Case Version | Bumped with commit message: "Module updated from vX to vY (Bulk-Update)" | -| Code-Gen | New Code-Gen for the updated module is automatically used for future runs| -| Code-Gen Status | Set to **Unvalidated** | -| Execution Verified | Set to **false** until new Code-Gen is validated | - -## Linked Test Cases Tab - -The module detail page includes a **Linked Test Cases** tab that lists all test cases linked to the module, grouped by module version, with key metadata. From here, you can navigate directly to individual test case summary pages. - -## Current Limitations - -The following limitations apply currently to the Bulk Module Update Feature: - -1. **No Module Validation in Bulk Flow** - - - The bulk update flow **does not perform module validation**. - - You can only bulk‑update to module versions that have already been authored and validated in a KaneAI session. - -2. **No Bulk Code-Gen Validation** - - - Bulk validation of Code‑Gen across test cases is **not supported yet**. - - `code_gen_status` will remain Unvalidated after the bulk module update until the Code-gen is successfully executed through a Test Run. - -## Troubleshooting - -### "Update Test Cases" Button Is Disabled - -**Symptoms:** -- The **Update Test Cases** button on the Module Detail Page is disabled -- You see a message: "This module version is not authored yet. Author it in a test to enable bulk update." - -**Solutions:** -1. Confirm that the module version has been authored/validated at least once in a test case -2. Ensure there are eligible KaneAI test cases linked to this module version (non-faulty, non-unverified) -3. Refresh the Module Detail Page and try again - -### Some Test Cases Are Greyed Out in the Selection List - -**Symptoms:** -- Certain test cases appear greyed out and cannot be selected -- They display reasons such as "Cannot update - test case is faulty" - -**Solutions:** -1. For **faulty** test cases: Fix the underlying issues causing the faulty status, then re-attempt -2. For **non-KaneAI** test cases: Bulk update is not supported; update them manually if needed -3. For **unverified** test cases: Complete verification steps, then revisit the bulk update flow - -### Tests Still Show "Code-Gen Unvalidated" After Update - -**Symptoms:** -- After the bulk update, test cases show: "Code-Gen updated after module update – will be validated on next execution." -- Code-Gen status remains **Unvalidated** - -**Solutions:** -1. Run or re-run the affected test cases following your standard validation workflow -2. Once Code-Gen is validated, the test case's code-gen status will be updated and execution verified will be set to **true** - -## Related Guides - -- [Modules](/support/docs/kane-ai-modules/): Create, use, and manage modules -- [Versioning and Enhancements](/support/docs/kaneai-modules-versions-and-enhancement/): Track changes, compare versions, and revert modules diff --git a/static/docs/kaneai-chrome-options.md b/static/docs/kaneai-chrome-options.md deleted file mode 100644 index c64d8b6f2..000000000 --- a/static/docs/kaneai-chrome-options.md +++ /dev/null @@ -1,31 +0,0 @@ -# Chrome Options - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Chrome options, also known as Chrome arguments, are command-line switches that alter the default behavior of the Chrome browser. These options provide flexibility to users by enabling or disabling certain browser features, modifying settings, and configuring custom behaviors such as headless browsing or disabling GPU acceleration. - -In KaneAI Web Agent, Chrome options are used to tailor the testing environment to suit the specific needs of automated tests. By defining Chrome options, users can configure the browser behavior before initiating a test session, allowing for a more customized and controlled testing setup. This guide will walk you through the steps to configure and use Chrome options in the KaneAI Web Agent: - -## Steps to use Chrome Options in KaneAI Web Agent -Using Chrome options in KaneAI Web Agent is simple and involves defining specific configurations before launching a test. Follow the steps below to add and manage Chrome options for your tests. - -### Step 1: Navigate to the Test Configuration Page -- Begin by accessing the Test Configuration page within the KaneAI Web Agent interface. Click on the **Create a Web Test** button. - -### Step 2: Locate the Chrome Options Section -- Enable the toggle to add Chrome browser command-line options when launching your instance. -- Enter your Command line switches and you can provide the type of argument as well i.e. String or File for that particular command line. -- You can enter up to 10 Chrome options in the provided input fields. -- For example: - - `--headless` - - `--disable-gpu` - - `--use-file-for-fake-audio-capture=/path/to/audio/file` - -### Step 3: Start the Web Agent -- Once the configuration is done, initiate the Web Agent. The browser will launch with the specified Chrome options applied, allowing you to perform your tests under the customized environment. - -## Considerations -- **Maximum Options :** You can configure up to 10 Chrome options per session. Ensure you only input the necessary options for your test case. -- **Supported Options :** KaneAI Web Agent will validate that only supported Chrome options are entered. If you enter an unsupported option, an error message will notify you to correct the configuration. -- **File Paths :** If a Chrome option requires a file path (for instance, `--use-file-for-fake-audio-capture`), KaneAI will automatically inject the Downloads folder path by default. This simplifies the process and removes the need to manually specify file paths for certain features. -- **Default Browser Configuration :** If no Chrome options are provided, KaneAI will default to the standard browser configuration, meaning the browser will launch with its default settings. diff --git a/static/docs/kaneai-ci-cd-automation.md b/static/docs/kaneai-ci-cd-automation.md deleted file mode 100644 index b6e4ae2a6..000000000 --- a/static/docs/kaneai-ci-cd-automation.md +++ /dev/null @@ -1,191 +0,0 @@ -# Automating Test Runs Execution with CI/CD on KaneAI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This guide provides a detailed walkthrough for automating the execution of test runs using CI/CD tools such as GitHub Actions or Jenkins on the KaneAI platform. By following these steps, you can effortlessly integrate test run executions with your pipelines. - -## Prerequisites -- Access to Test Manager. -- A project with KaneAI-generated test cases. -- Credentials for TestMu AI platform. - -## Step-by-Step Guide -### Step 1: Navigate to the Test Manager -- Log in to the KaneAI platform and go to the Test Manager page. -- Access the project where the test run is located. - -### Step 2: Locate the 'Test Runs' Section & Copy Test run ID -Inside your selected project, navigate to the Test Runs section. Select the desired test run for integration or direct triggering from your CI/CD pipeline. - -Open the test run to view its URL. Copy the Test Run ID from the URL. This ID will be used in the API call for integration. - -A list of test instances with various configurations will be displayed for each test run. - -### Step 3: Configure the API Call -Replace `` with the actual ID from the URL and set additional optional parameters: - -- **concurrency**: Select the concurrency you want to set for this test run. Defaults to 1 if not specified. -- **title**: Assign a unique job title; a random one will be generated if omitted. -- **region**: Choose a region for web tests such as _eastus_ or _centralindia_ to select the region from where the VM is allocated. -- **mobile_region**: Choose a region for mobile app tests such as _us_,_ap_ or _eu_ to select the region from where the device is allocated. -- **tunnel**: Add the parameter and tunnel name if you want to run using TestMu AI Tunnel for private applications. See more details for tunnel [here](/support/docs/kane-ai-geolocation-tunnel-proxy/#tunnel-support). -- **dedicated_proxy**: Add the region of the dedicated proxy such as _us_ or _eu_, #Optional, either tunnel or dedicated proxy or geolocation can be used in a single API call. -- **geolocation**: Add the resgion of the geolocation that you want to run your tests from. You can find the list of support geolocations [here](/support/docs/selenium-geolocation-capabilities/). -- **environment_id**: Define the environment on which you want to run the test run if required. -- **retry_on_failure**: Define whether to retry on failure. When enabled, retries are triggered both when the [testRunnerCommand](/support/docs/deep-dive-into-hyperexecute-yaml/#testrunnercommand) execution fails and when individual test cases fail. You can set the maximum number of retries using `max_retries`. You can find more details [here]( https://www.testmuai.com/support/docs/deep-dive-into-hyperexecute-yaml/#retryonfailure). -- **max_retries**: Define the number of maximum retries you want (Max value 5). Default retries 1. You can find more details [here](/support/docs/deep-dive-into-hyperexecute-yaml/#maxretries). -- **timezone**: Define the timezone you want to choose for test run in UTC+01:00 format. -- **app_profiling**: Add true if you want to track app profiling metrics in your test. You can find more details [here](/support/docs/appium-app-performance-analytics/). -- **performance**: Add true if you want generate Lighthouse report for your web tests. Supported on limited OS browser combinations. Additionally, Setting this as true could potentially slow down the execution time. You can find more details [here](/support/docs/view-lighthouse-performance-metrics/). -- **android_app_id**: Specify an Android app ID to override the existing app in the configuration of the Test instances. -- **ios_app_id**: Specify an iOS app ID to override the existing app in the configuration of the Test instances. -- **accessibility**: Set as true if you want to run accessibility test on all your tests in the test run. Setting this as true could potentially slow down the execution time. -- **replaced_url :** To be used to dynamically replace any pattern URL in test cases with the replacement URL for entire test run. -- **report_enabled**: Set to `true` to generate an HTML report for the test run. The report can be accessed from the HyperExecute Job page after execution. See [Reports](/support/docs/kaneai-hyperexecute-test-run-execution/#reports) for details. -- **extent_report_enabled**: Set to `true` to generate an Extent report for the test run. The report can be accessed from the HyperExecute Job page after execution. Only one report type can be enabled at a time. Use either `report_enabled` or `extent_report_enabled`, not both. -- **report_email_to**: An array of email addresses to receive the test run report via email after execution. Maximum 10 email addresses. Only works when `report_enabled` is set to `true`. - -Test case failure retries are supported only for code exported from **May 10, 2026 onwards**. For previously exported code, retries are triggered only on test runner command failure. To use this capability, regenerate the code export for your test cases in Test Manager. - -#### Example API Call: - -```yaml -curl --location 'https://test-manager-api.lambdatest.com/api/atm/v1/hyperexecute' \ ---header 'Content-Type: application/json' \ ---header 'Authorization: Basic ' \ ---data '{ -"test_run_id": "YOUR_TEST_RUN_ID", #enter test run id -"concurrency": 1, # Optional, default 1 -"title": "UNIQUE_BUILD_NAME", #Optional -"console_log": false, #Optional for web tests, options - false, error, warn, info, true -"network_logs": false, #Optional to capture network calls during the test -"network_full_har": false, #Optional, Only for web, to capture complete network requests & response during the test -"region": "YOUR_DESIRED_REGION", #Optional for web tests, options - eastus, centralindia -"mobile_region": "YOUR_DESIRED_REGION", #Optional for mobile tests, options - us,eu,ap -"tunnel": "tunnel-name", #Optional, either tunnel or dedicated proxy or geolocation can be used -"dedicated_proxy": "region", #Optional, either tunnel or dedicated proxy or geolocation can be used -"geolocation": "region", #Optional, either tunnel or dedicated proxy or geolocation can be used -"environment_id": 0, #Optional, default null, to define the environment for test run -"retry_on_failure": true, #Optional, default true, to define if retry is needed in case of failure -"max_retries": 1, #Optional, default 1, to define the number of retries needed if retry_on_failure is enabled -"timezone": {"region":"Indianapolis"}, #Optional, Only for web, to define the timezone using which the test runs should run -"app_profiling": false, #Optional for performance metrics for native mobile apps -"performance": false, #Optional for lighthouse report for web tests -"android_app_id": "lt://", #optional -"ios_app_id": "lt://", #optional -"accessibility": false, #Optional only for web tests -"network_throttle": { -"label": "No Throttling", -"value": "default", -"download_speed": 0, -"upload_speed": 0, -"latency": 0, -"honor_network": true -}, #Optional, only for mobile tests, to define if sessions needs to be enabled with network throttling enabled -"replaced_url": [ -{ -"pattern_url": "TEST_URL_1", -"replacement_url": "REPLACED_TEST_URL_1" -}, -{ -"pattern_url": "TEST_URL_2", -"replacement_url": "REPLACED_TEST_URL_2" -} -], #Optional to be used to dynamically replace any pattern URL in test cases with the replacement URL -"report_enabled": false, #Optional, set true to generate HTML report -"extent_report_enabled": false, #Optional, set true to generate Extent report -"report_email_to": ["email1@example.com"] #Optional, array of email addresses to receive report (max 10) -}' -``` - -#### Sample API Response: - -The API response contains the job ID for both jobs created for desktop web tests as well as mobile tests on real devices. Here is a sample response for a test run which contains both app and web test cases: - -```yaml -{ -"job_id": "abcd1234-abcd-1234-abcd-1234abcd1234", -"app_job_id": "abcd1234-abcd-1234-abcd-1234abcd1234", -"test_run_id": "01LAMBDATEST2024KANEAIJ8NPGDP", -"job_link": "https://hyperexecute.lambdatest.com/hyperexecute/task?jobId=", -"mobile_job_link": "https://hyperexecute.lambdatest.com/hyperexecute/task?jobId=" -} -``` - -### Step 4: Authenticate and Trigger the Job -- Provide your TestMu AI username and access key for Basic Authentication. -- Submit the API call to trigger the job. The process will start within seconds. - -> **Important :** Keep your credentials secure to maintain platform integrity. - -### Step 5: Monitor Test Executions -- Follow the link provided in the API response to view the HyperExecute Job. -- Monitor running executions in real time through the dashboard. - -## Github Actions sample -Here is a sample that you can use on how to integrate the API with Github Actions in your Github repository: - -**Step 1: Create a GitHub Actions Workflow YAML File** -In your Git repository, navigate to .github/workflows/ and create a file named sanity-test.yml. - -**Step 2: Sample for yml file** -In below sample, we are executing a test run using test_run_id and then verifying the result for it using HyperExecute job status API. You can find more details on HyperExecute APIs [here](https://www.testmuai.com/support/api-doc/?key=hyperexecute) and update the API in the .yml file based on your needs. - -```yml -name: Run Sanity Tests on LambdaTest - -on: -push: -branches: -- main -pull_request: -branches: -- main - -jobs: -sanity-test: -runs-on: ubuntu-latest - -steps: -- name: Trigger Sanity Test on LambdaTest -id: trigger-test -run: | -echo "Triggering sanity tests on LambdaTest" -response=$(curl --location 'https://test-manager-api.lambdatest.com/api/atm/v1/hyperexecute' \ ---header 'accept: application/json' \ ---header 'Content-Type: application/json' \ ---header 'Authorization: Basic ' \ ---data '{ -"test_run_id" : "", -"concurrency" : 1, -}') -echo "Response: $response" -job_id=$(echo $response | jq -r '.job_id') -echo "Job ID: $job_id" -echo "::set-output name=job_id::$job_id" - -- name: Check Test Status -run: | -echo "Checking test status" -job_id=${{ steps.trigger-test.outputs.job_id }} -sleep 240 # Wait before checking the status -response=$(curl --location "https://api.hyperexecute.cloud/v2.0/job/$job_id" \ ---header "accept: application/json" \ ---header "Authorization: Basic ") - -echo "Response: $response" -status=$(echo $response | jq -r '.data.status') -echo "Test status: $status" -if [[ "$status" != "completed" ]]; then -echo "Tests failed. Exiting with error." -exit 1 -fi -echo "Sanity tests passed successfully." -``` - -**Step 3: Define Workflow Triggers** -Set the workflow to trigger on push and pull_request events (you can modify the trigger based on your needs) under "on" section of the yaml above. - -## Video Explanation - -The video consists of old configuration with Test Plans and Builds instead of Test Runs. Please keep this in mind while going through the video. The video will be updated soon. diff --git a/static/docs/kaneai-conditional-logic.md b/static/docs/kaneai-conditional-logic.md deleted file mode 100644 index a4d532a82..000000000 --- a/static/docs/kaneai-conditional-logic.md +++ /dev/null @@ -1,146 +0,0 @@ -# Conditional Logic (If / Else‑If / Else) in KaneAI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Introduction - -KaneAI supports conditional branching in your test cases using **If / Else‑If / Else** blocks. This lets you handle dynamic application behavior, for example, clicking a login button only when it is visible, or choosing between different checkout flows based on a variable value. - -Each branch can contain multiple steps, including regular actions, **modules**, **JavaScript**, **API**, and **DB** steps, giving you full flexibility to build complex, real‑world test scenarios. - -## How It Works - -1. Insert a conditional block from the **/** slash command menu while your authoring session is running. -2. Define a condition using variables and comparison operators. -3. Optionally add **Else‑If** branches for additional conditions. -4. Add steps, including modules, JS, API, and DB steps, inside each branch. -5. During authoring, only the branch whose condition is true gets executed. Steps in the remaining branches are queued. -6. In automation, KaneAI evaluates the conditions top‑to‑bottom at runtime and executes the first matching branch automatically. - -**Conditional blocks require a running session** -A conditional block can only be inserted while the authoring session is running, because KaneAI needs a running session to analyze the condition. Adding a conditional block from the **+ Add step** option while a test is paused is not currently supported. Conditional blocks that already exist in the test can still be extended while paused, for example by adding an Else‑If branch. - -## Prerequisites - -- A KaneAI authoring session that is running. Conditional blocks cannot be inserted while the session is paused. -- Variables or element states available to build your conditions (e.g., smart variables, global variables, or extracted values). - -## Step‑by‑step Guide - -### Step 1: Add a Conditional Block - -1. In your authoring session, press **/** to open the slash command menu. -2. Select **Add If-Else** option. -3. KaneAI inserts an **If / Else** block into your test flow. - -### Step 2: Define the Condition - -Click the **If** header to open the condition editor. You can define conditions in two ways. Toggle between them using the **switcher icon** (`&&`). - -#### Option A: Natural Language - -Type your condition as a plain-English sentence (e.g., "login button is visible"). KaneAI interprets the intent and evaluates it at runtime. - -#### Option B: Operand & Operator - -Build the condition explicitly using left operand, operator, and right operand: - -1. Enter a **left operand**: this can be a variable (e.g., `{{login_button_visible}}`), extracted text, or a literal value. -2. Choose a **comparison operator** from the dropdown: - -| Operator | Description | -|---|---| -| `==` | Equals | -| `!=` | Not equals | -| `>` | Greater than | -| `<` | Less than | -| `>=` | Greater than or equal to | -| `<=` | Less than or equal to | -| `contains` | Left operand contains the right operand | -| `not_contains` | Left operand does not contain the right operand | -| `starts_with` | Left operand starts with the right operand | -| `ends_with` | Left operand ends with the right operand | - -3. Enter the **right operand** (e.g., `true`, a number, or another variable). - -#### Adding Multiple Conditions - -You can combine conditions using **AND** / **OR** logic: - -1. After defining the first condition, click the **+** button (labeled **Add condition**). -2. Toggle the logical operator between **AND** and **OR** using the dropdown. -3. Fill in the additional condition's operands and operator. -4. Click the green **checkmark** to confirm. - -### Step 3: Add Steps Inside a Branch - -Each branch (**If**, **Else‑If**, **Else**) supports multiple steps. You can add: - -- **Regular test steps**: type your instruction in the step input field. -- **Modules**: reuse existing modules by adding them inside any branch. -- **JavaScript steps**: execute custom JS within a branch. -- **API steps**: make API calls as part of a conditional flow. -- **DB steps**: run database queries conditionally. -- **Manual interaction**: click the **manual interaction icon** next to the step input field to perform actions directly on the browser within the branch. - -Use the step input field or press **/** inside a branch to access the slash command menu. - -### Step 4: Add Else‑If Branches - -To handle additional conditions beyond the initial **If**: - -1. Click **Add Else‑If** between the **If** and **Else** blocks. -2. Define the condition for the new branch (same process as Step 2). -3. Add steps inside the **Else‑If** block. -4. Repeat to add as many **Else‑If** branches as needed. - -**Authoring vs. Automation behavior** -During authoring only one condition can be true at a time, so only the matching branch's steps are executed. Steps under the remaining branches go into a **queued** state. When the test runs in automation, all queued branches are evaluated automatically and the first matching branch is executed. - -### Step 5: Add Steps to the Else Branch - -The **Else** block executes when none of the preceding conditions are met. Click **+ Add step** inside the **Else** block and add your fallback steps. - -### Step 6: Close the Block - -Click **End If** at the bottom of the conditional block to finalize it. You can then continue adding steps after the conditional block as usual. - -## Using Modules Inside Conditional Branches - -You can embed entire modules within any branch of a conditional block. This is useful for reusing login flows, setup routines, or shared workflows that should only run under specific conditions. - -To add a module inside a branch: - -1. Click into the step input field within the branch. -2. Press **/** and select the module you want to add, or type your instruction. -3. The module and its steps appear inline within the branch. - -## Tips and Recommendations - -- **Keep conditions simple.** Use clear, single‑purpose comparisons. Combine with AND / OR only when necessary. -- **Use variables for dynamic values.** Store extracted values or API responses in variables, then reference them in your conditions. -- **Leverage modules for reusable logic.** Instead of duplicating steps across branches, create a module and embed it. -- **Test each branch independently.** During authoring, verify that each branch executes the correct steps before saving. - -## Limitations - -- **Nested conditions are not supported.** You cannot place an If / Else‑If / Else block inside another conditional block. -- **New conditional blocks cannot be created while a test is paused.** The **/** slash command menu and the **+ Add step** option do not offer **Add If-Else** in the Draft state. Conditional blocks that already exist in the test can still be extended while paused. - -## FAQ - -### Can I add an Else‑If branch after I have already authored the test? - -Yes. On a conditional block that already exists in your test, you can add Else‑If branches, including while the test is paused. What you cannot do while paused is create a new conditional block, because **Add If-Else** is not offered in the **/** slash command menu or the **+ Add step** option in that state. - -### How many Else‑If branches can I add? - -You can add up to **5** Else‑If branches per conditional block. - -### Can I use modules, JS, API, and DB steps inside every branch? - -Yes. All branch types (If, Else‑If, and Else) support the full range of step types including modules, JavaScript, API, and DB steps. - -### Are nested If / Else blocks supported? - -No. Nested conditional blocks are not supported. If you need complex branching, consider splitting the logic across multiple test cases or using Else‑If branches instead. diff --git a/static/docs/kaneai-create-pr.md b/static/docs/kaneai-create-pr.md deleted file mode 100644 index bb727cc5f..000000000 --- a/static/docs/kaneai-create-pr.md +++ /dev/null @@ -1,169 +0,0 @@ -# Create Pull Requests from KaneAI Test Cases - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -KaneAI lets you export generated test code directly to your GitHub or GitLab repository as a pull request (PR) or merge request (MR). Instead of downloading ZIP files and manually copying code, you can raise PRs from the Test Manager in a few clicks, or automatically whenever code is generated. - -This guide covers how to integrate your Git provider, configure PR settings, create pull requests from test cases, and track their status. - -**Beta Feature** -This feature is currently in **beta** and is not generally available. If you would like access, please reach out to [support@testmuai.com](mailto:support@testmuai.com). - -## Prerequisites - -- A TestMu AI account with access to KaneAI and Test Manager. -- A GitHub or GitLab account with at least one repository. -- At least one test case with code generated. -- Admin role to configure integration and PR settings. Users with appropriate permissions can create PRs for their own test cases. - -## Supported Platforms and Frameworks - -| Platform | Framework | File Exported | -|----------|-----------|---------------| -| Desktop Browser | Selenium (Python) | `test.py` | -| Mobile Browser | Selenium (Python) | `test.py` | -| Mobile App | Appium (Python) | `test.py` | - -## Step 1: Integrate Your Git Provider - -Before you can create pull requests, you need to connect your GitHub or GitLab account to TestMu AI. - -### Connect via Integrations Page - -Set up your Git provider from the TestMu AI Integrations page: - -- **GitHub**: Supports OAuth, Repo-based OAuth, Cloud/PAT, and Self-Hosted (GitHub Enterprise) connection methods. See the [GitHub Integration guide](/support/docs/github-integration/) for detailed setup instructions. -- **GitLab**: Supports OAuth, Personal Access Token, and Self-Managed instance connection methods. See the [GitLab Integration guide](/support/docs/gitlab-integration/) for detailed setup instructions. - -### Select Platform from Test Manager - -When you initiate PR creation for the first time from the Test Manager, a **Select Platform to Create PR** dialog appears. This dialog shows the integration status for each provider. - -- If your provider is **not integrated**, click the **Integrate** button to be redirected to the Integrations page. - -- If your provider is already **integrated**, the status shows in green. Click **Setup** to proceed to the repository configuration. - -Bitbucket support is coming soon. Currently, GitHub and GitLab are the supported providers. - -## Step 2: Configure Repository Settings (First-Time Setup) - -After selecting your integrated provider, you are taken to the **Setup Pull Request** page. This is a one-time configuration that defines how test code is organized in your repository. - -### Repository Strategy - -Choose how repositories are mapped to your projects: - -| Strategy | Description | -|----------|-------------| -| **Single Repository** | One repository for the entire organization. All projects push test files to the same repo under project-specific folders. | -| **Multiple Repositories** | Each project can be linked to its own repository. A default repository handles unmapped and new projects. | - -### Configuration Fields - -| Field | Description | -|-------|-------------| -| **Repository** | The target repository where PRs will be created. In Single mode, all projects use this repo. In Multiple mode, this is the default repo for unmapped projects. | -| **Folder Structure** | The file path pattern inside the repository. Default: `{project_name}/{test_id}_{test_name}/{version_no}`. The test file (`test.py`) is placed at this path. | -| **Base Branch (PR Target)** | The branch that PRs are opened against (e.g., `main`, `master`, `develop`). | -| **Auto-PR** | When enabled, a PR is automatically created whenever new code is generated for a test case. Default: Disabled. | - -Click **Complete Setup** to save the configuration. - -You can change all of these settings later from the [Test Manager Settings > Pull Requests](https://www.testmuai.com/login/?redirectTo=https://test-manager.lambdatest.com/settings/pull-requests#) tab. - -### Single Repository Mode - -In Single Repository mode, all test case PRs across all projects go to one repository. The settings page shows the selected repository, folder structure pattern, base branch, and Auto-PR toggle. - -### Multiple Repositories Mode - -In Multiple Repositories mode, you can map individual projects to specific repositories. The configuration table shows columns for **Project Name**, **Repository Name**, **Folder Path**, **Base Branch**, and **Auto-PR**. - -- **Unmapped Projects**: Any project without a specific mapping uses the default repository configuration. -- **+ Add Project**: Click to map a specific project to a different repository. Only KaneAI-enabled projects are available for mapping. - -Click **Update Setup** to save any changes. - -## Step 4: Create a Pull Request - -Once the integration and repository settings are configured, you can create PRs directly from the test case listing page or from the code section of an individual test case details page. - -### PR Status in the Test Case Listing - -Each test case in the listing displays its PR status inline: - -- **PR number with icon** (e.g., `#2`, `#3`, `#4`): A PR exists for this test case. Hover over the PR indicator to see the status (Open, Merged, or Closed), creation details, and a **View on GitHub** link that opens the PR directly. -- **No PR**: No pull request has been created for this test case yet. - -### How to Create a PR - -1. Navigate to your project in **Test Manager**. -2. Find the test case you want to create a PR for in the test case listing. -3. Click the **three-dot menu** (actions) on the test case and select **Create PR**. -4. If this is your first time, you will be prompted to select a platform and complete the setup (see Steps 1-2 above). -5. Confirm the PR details and submit. The PR is created in your configured repository. - -### Diff-Based Updates - -KaneAI automatically tracks whether each code export version already has an associated PR. This detection happens in the background. The **Create PR** button only appears for a version when there is no existing PR for it. - -In the **Code** section of a test case, each code export version shows its PR status alongside the **Create PR** and **Execute** actions. If a PR already exists for that version, the **Create PR** button is disabled with a tooltip: "PR already exists for this version." - -- **New version without a PR**: The **Create PR** button is active. Click it to raise a PR with that version's code. -- **Version with an existing PR**: The PR number is shown (e.g., `#2`) and the **Create PR** button is disabled since the code is already synced. - -### Auto-PR - -When Auto-PR is enabled in your settings, a pull request is automatically created whenever code generation completes for a test case. No manual action is required. - -- The PR uses your saved configuration (repository, folder structure, base branch). - -## Step 5: Filter by PR Status - -Use the **PR Status** filter to quickly find test cases based on their pull request state. Click the **More** filter option in the test case listing toolbar, then select **PR Status**. - -Available filter options: - -| Filter | Description | -|--------|-------------| -| **Open** | Test cases with an open PR that has not been merged or closed. | -| **Merged** | Test cases whose PR has been merged into the base branch. | -| **Closed** | Test cases whose PR was closed without merging. | -| **Diff Available** | Test cases where code has changed since the last PR was created. | -| **No PR** | Test cases that do not have any associated pull request. | - -Select one or more filters and click **Apply** to filter the listing. - -## Additional Details - -### Folder Structure in the Repository - -Test files are organized in your repository following the configured folder structure pattern. The default pattern is: - -``` -your-repo/ -├── {project_name}/ -│ ├── {test_id}_{test_name}/ -│ │ └── {version_no}/ -│ │ └── test.py -│ └── {test_id}_{test_name}/ -│ └── {version_no}/ -│ └── test.py -``` - -- **`{project_name}`**: Derived from the Test Manager project name. -- **`{test_id}_{test_name}`**: The test case ID and name combined for uniqueness. -- **`{version_no}`**: The code export version number. - -You can customize this pattern during setup or in the Pull Requests settings tab. - -### Permissions - -| Action | Admin | User | -|--------|-------|------| -| Configure Git integration | Yes | No | -| Modify repository strategy and settings | Yes | No | -| Toggle Auto-PR | Yes | No | -| Create PR for own test cases | Yes | Yes | -| Create PR for permitted folders | Yes | Yes | -| View PR status | Yes | Yes | diff --git a/static/docs/kaneai-custom-headers.md b/static/docs/kaneai-custom-headers.md deleted file mode 100644 index 14d6cce2f..000000000 --- a/static/docs/kaneai-custom-headers.md +++ /dev/null @@ -1,67 +0,0 @@ -# Custom Headers in KaneAI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Custom headers allow you to inject additional HTTP header information into your test sessions. You can use them to send tokens, session IDs, content-type overrides, or any other custom metadata with requests made during your tests. - -Custom headers are supported on both **Desktop Web** and **Mobile Browser** platforms. - -## Header Value Types - -Each custom header has a **Type** field that determines how the value is handled: - -| Type | Description | -|---|---| -| **String** | A plain-text value that is sent as-is with the request (e.g., `1234567890`). | -| **Secret** | A reference to an organization-level secret stored in KaneAI. Uses the `{{secrets.org.}}` syntax to keep sensitive values like tokens or API keys out of plain text. See [KaneAI Secrets](/support/docs/kane-ai-secrets/) for more details. | - -## Configure Custom Headers for Desktop Web - -### Step 1: Create a Web Test - -Navigate to the [KaneAI page](https://www.testmuai.com/login/?redirectTo=https://kaneai.lambdatest.com/objective) and click the **Desktop Browser** option to create a new web test. - -### Step 2: Enable and Add Custom Headers - -In the **Create a Web Test** dialog, enable the **Custom Headers** toggle. Add your headers by entering a **Header Name** and **Header Value** for each row. Click the **+** button to add more rows. - -You can add up to 10 custom headers per session. - -If you provide a default header (e.g., `Accept`) in the Custom Headers section, it will override the browser's default value for that header. - -### Step 3: Start the Test Session - -Click **Start Testing** to launch the session. The headers you configured will be applied to all HTTP requests made during the session, including navigation requests and API calls. - -## Configure Custom Headers for Mobile Browser - -Custom headers are also available when authoring Mobile Browser tests. The configuration works the same way as Desktop Web, with an additional **Type** column to choose between **String** and **Secret** values. - -Navigate to the advanced settings for your Mobile Browser test and enable the **Custom Headers** toggle. Add your headers with the appropriate type for each value. - -## View Custom Headers During Authoring - -During an active authoring session, click **Advanced Settings** in the top navigation bar to view the custom headers applied to the current session. - -The **Advanced Settings** panel displays all configured custom headers along with their values. Secret values are shown in their `{{secrets.org.}}` format. - -## View Custom Headers on the Test Summary Page - -After the session completes, the custom headers used during the session are displayed on the **Test Summary** page. Click on **Advanced Settings** to expand the panel and view the headers. - -You can also click **View details** in the test case sidebar to see the full list of advanced settings including custom header in a side drawer. - -## Example Use Cases - -- **Session Management**: Send a custom `X-Session-ID` header to track user sessions across requests: -``` -X-Session-ID: 1234567890 -``` -- **Authentication Tokens**: Use the **Secret** type to inject API keys or bearer tokens without exposing them in plain text: -``` -Authorization: {{secrets.org.BearerToken}} -``` -- **Content Negotiation**: Override the `Accept` header to test specific response formats: -``` -Accept: application/json -``` diff --git a/static/docs/kaneai-database.md b/static/docs/kaneai-database.md deleted file mode 100644 index 91a9a4675..000000000 --- a/static/docs/kaneai-database.md +++ /dev/null @@ -1,55 +0,0 @@ -# Database Connection in KaneAI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Managing test data in isolation or manually updating it across different test cases can lead to inconsistencies, increased maintenance efforts, and slower test execution. Integrating database connections within KaneAI allows teams to access and manipulate real-time data directly during test creation and execution. This capability ensures that tests are always aligned with the latest data, reducing manual data management, improving test accuracy, and enabling more efficient and scalable test automation workflows. - -This document provides a step-by-step process to connect databases within the KaneAI platform. It covers the creation of a new connection, selecting database types, entering necessary details, and performing operations like queries and assertions. The guide ensures a smooth setup and usage experience for users looking to integrate their databases seamlessly with KaneAI. - -## Creating a database connection - -To begin, you need to access the [database connections](https://www.testmuai.com/login/?redirectTo=https://kaneai.lambdatest.com/databases "database connections") page within KaneAI and initiate the creation of a new connection by clicking on **Create New Connection**. - -### Add connection details - -Select the connection type as either TCP/IP or over SSH. Enter the database name and choose from available database types, including MySQL, PostgreSQL, MSSQL, Oracle, Mongo DB, and GCP Spanner. - -Provide a description and input your host name, port, username, and other relevant details. You can enter the password using an **organisation-level secret** or directly entering the password. For PostgreSQL, ensure you select the database name. - -**Special Connection Types (GCP Spanner):** -Unlike traditional databases, GCP Spanner does not use standard hostnames, ports, usernames, or passwords. To connect to a Spanner instance, you will instead need to provide: -* **Instance ID** (The Spanner instance name) -* **Database ID** (The database name within the instance) -* **Credentials JSON** (The service account credentials file used for authentication) - -KaneAI supports SQL, NoSQL, and Cloud-native (GCP Spanner) database connections. - -### Connecting local databases - -For local or private databases, select a tunnel if your tunnel is active. The tunnel can be activated easily by following the details available [here](/support/docs/kane-ai-geolocation-tunnel-proxy/#tunnel-support). Test the connection and create it to see the sample database added. - -*Note: GCP Spanner does not support SSH tunneling since it utilizes Google Cloud's native built-in security and networking.* - -Following flag will be additionally required for database connections: `--expose database_type:host:port`. - -So, your command will look like: - -{`./LT --user ${YOUR_LAMBDATEST_USERNAME()} --key ${YOUR_LAMBDATEST_ACCESS_KEY()} --expose mysql:0.0.0.0:3306 --verbose --env ht-prod`} - -## Using Database queries within KaneAI session - -Navigate within your KaneAI session. Use the slash command to add a database query. All connected databases will be visible, allowing you to view the schema or directly enter a query. For instances like GCP Spanner, the schema discovery will automatically show all user-created tables and column data types for the configured database. - -To perform a query, you can select any connected database and input your query to execute. You can leverage variables or parameters as well within the query to make your tests even more robust. - -**Safe Queries Only** -For safety and data integrity, KaneAI only allows `SELECT`, `INSERT`, and `UPDATE` operations. Altering or destructive queries such as `ALTER`, `CREATE`, `DELETE`, or `DROP` commands are not supported and will return an error. - -**Spanner Row Limit** -For **GCP Spanner** queries, a **100-row limit** is enforced on all `SELECT` queries to ensure fast performance and prevent excessive data transfer. It is recommended to use `WHERE` clauses to filter your data. - -Once you Run or Add query, your query will be recorded as a test step within KaneAI, and a JSON variable containing the table response will be generated if the query is successful. This JSON variable can be used to perform various assertions on the data. You can continue adding multiple database queries and validate responses easily. - -## Video explanation - -Have any feedback or request? Reach out to us via [support@testmuai.com](mailto:support@testmuai.com) and we would be happy to hear from you. diff --git a/static/docs/kaneai-dynamic-url-replacement.md b/static/docs/kaneai-dynamic-url-replacement.md deleted file mode 100644 index 4e24884e6..000000000 --- a/static/docs/kaneai-dynamic-url-replacement.md +++ /dev/null @@ -1,114 +0,0 @@ -# Dynamic URL Replacement - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -As a QA Engineer, you may need to run test cases on different environments (e.g., staging, production) without altering your test scripts. With Dynamic URL Replacement you can execute the same test cases across multiple environments without modifying the test scripts, ensuring seamless execution across different environments. - -> This feature is applicable only for Web Tests - -## Prerequisites -You must have [created a test run for KaneAI Generated Tests](/support/docs/kaneai-hyperexecute-test-run-execution/). - -## How to Use Dynamic URL Replacement - -### Step 1: Navigate to the Test Runs Page -- Log in to TestMu AI. -- Go to the Test Manager > Test Runs section. -- Click on your desired **Test Run**. -- Click on **Run with HyperExecute** button. - -### Step 2: Configure the Test Execution URL -- In the Run on HyperExecute side drawer, locate the new field labeled **Replace Test Execution URL**. -- Enter the **Pattern URL** (your original URL used in the tests) and **Replacement URL** (URL of different environment you want to test) in the text field. - -> You can add multiple URLs for all web tests in the run to execute in different environments - -### Step 3: Execute the Test Run -- If the **Replace Test Execution URL** field is filled, the entered URL will override the default (Pattern) URL during execution. -- If left empty, the test will run with the default URL. -- Click on the **Execute** button - -## How to Use Dynamic URL Replacement via API -If you prefer to use the API to configure and execute tests with Dynamic URL Replacement, follow the instructions below: - -### API Endpoint -```yaml title="POST request" -https://test-manager-api.lambdatest.com/api/atm/v1/hyperexecute -``` - -### Request Headers -```javascript -Content-Type: application/json -Authorization: Basic -``` - -### Request Body -The following JSON payload demonstrates how to use the API to configure Dynamic URL Replacement: - -```javascript -{ -"test_run_id": "YOUR_TEST_RUN_ID", // Replace with your test run ID -"concurrency": 1, // Optional, Default: 1 -"title": "UNIQUE_BUILD_NAME", // Optional -"region": "YOUR_DESIRED_REGION", // e.g., eastus, centralindia -"replaced_url": [ -{ -"pattern_url": "TEST_URL_1", -"replacement_url": "REPLACED_TEST_URL_1" -}, -{ -"pattern_url": "TEST_URL_2", -"replacement_url": "REPLACED_TEST_URL_2" -}, -{ -"pattern_url": "TEST_URL_3", -"replacement_url": "REPLACED_TEST_URL_3" -} -], -"accessibility": false -} -``` - -### Parameters Explained -| Parameter | Description | -|-----------|-------------| -| `test_run_id` | The unique identifier for the test run. | -| `concurrency` | (Optional) Number of parallel executions. Default is 1. | -| `title` | (Optional) A unique name for the build. | -| `region` | The region where the tests will be executed (e.g., eastus, centralindia). | -| `replaced_url` | An array of objects containing `pattern_url` and `replacement_url` | -| `accessibility` | (Optional) Enable or disable accessibility testing. | - -### Example cURL Command -Here’s an example cURL command to trigger a test run with Dynamic URL Replacement: - -```javascript -curl --location 'https://test-manager-api.lambdatest.com/api/atm/v1/hyperexecute' \ ---header 'Content-Type: application/json' \ ---header 'Authorization: Basic ' \ ---data '{ -"test_run_id": "YOUR_TEST_RUN_ID", -"concurrency": 1, -"title": "UNIQUE_BUILD_NAME", -"region": "eastus", -"replaced_url": [ -{ -"pattern_url": "lambdatest", -"replacement_url": "ajio" -}, -{ -"pattern_url": "d", -"replacement_url": "flipkart" -}, -{ -"pattern_url": "hello", -"replacement_url": "1234" -} -], -"accessibility": false -}' -``` - -- Replace `` with your actual Base64-encoded authorization credentials. -- Ensure that the `pattern_url` matches the exact URL pattern used in your test scripts. -- Multiple `replaced_url` entries can be added to replace different URLs dynamically. diff --git a/static/docs/kaneai-edit-test-steps.md b/static/docs/kaneai-edit-test-steps.md deleted file mode 100644 index 8e4f244cf..000000000 --- a/static/docs/kaneai-edit-test-steps.md +++ /dev/null @@ -1,5 +0,0 @@ -# Edit and Update Test Steps Using KaneAI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -In this tutorial, learn how with KaneAI, you can efficiently update and edit test steps using automation. diff --git a/static/docs/kaneai-failure-conditions.md b/static/docs/kaneai-failure-conditions.md deleted file mode 100644 index 2791e8456..000000000 --- a/static/docs/kaneai-failure-conditions.md +++ /dev/null @@ -1,34 +0,0 @@ -# Failure Conditions - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Failure conditions in KaneAI allow you to define how the test execution in a Test Run should behave when a particular test step fails. These conditions give you fine-grained control over how failures are handled at the step level and how they impact the overall execution result. - -Failure conditions are configurable for every type of command or step in KaneAI Authoring mode and are essential for aligning your tests with your quality and failure tolerance. - -Failure conditions are set in the Authoring mode but they are applicable only in Test Run your Test's flow while authoring & code generation will not be affected by this. - -Below are the failure conditions available in KaneAI: - -1. **Fail test immediately:** -If the step fails, the test execution is halted immediately, and the overall test status is marked as `Failed`. This ensures that no further steps are executed after a critical failure. This is ideal for scenarios where proceeding further does not make sense or could lead to misleading results. - -2. **Fail but continue execution:** -If the step fails, the test execution continues with the remaining steps, but the overall test is marked as `Failed` after completion. This is useful when you want to capture all failures in a single run for debugging purposes, but still treat the test as failed overall. - -3. **Warn but continue execution:** -If the step fails, the test execution continues with the remaining steps and the overall test result remains `unaffected`. This is particularly useful for non-critical validations where failure should not impact the test outcome. - -You can set the failure condition in the Authoring mode by clicking on the three-dots on the step: - ->This change will not affect your existing test cases or the generated code where you’ve used hard/soft assertions. However, when you edit these tests, the new failure conditions will be applied. - -## Default Failure Condition Setting - -By default, the failure condition for each step is set to **Fail test immediately**, ensuring that the execution stops as soon as a step fails. - -The default failure condition can be configured at the Organization level within the **Org Product Preferences** section of **Organization settings**, [here](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/org-settings/kane-ai/failure-conditions). - - These settings will be applied to all the **new tests** being created in that Organization & can only be edited by the **Admins** of the Organization. - -If you are transitioning from a Hard & Soft Assertions approach, think of **Fail test immediately** as your **Hard Assertion**, which halts on assertion failure, and **Warn but continue execution** as your **Soft Assertion**, which logs the failure but continues execution. diff --git a/static/docs/kaneai-faqs.md b/static/docs/kaneai-faqs.md deleted file mode 100644 index 2cce4b65d..000000000 --- a/static/docs/kaneai-faqs.md +++ /dev/null @@ -1,44 +0,0 @@ -# KaneAI FAQs - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Answers to common questions about authoring and running tests with KaneAI. - -## General - -### What does KaneAI run my tests on? - -KaneAI runs your tests on real browsers in the cloud on HyperExecute, across multiple operating systems. - -### Where can I see my test results? - -Every run is saved to Test Manager, and KaneAI gives you a shareable Test Summary link you can send to anyone. - -## Bot protection and CAPTCHAs - -### Why is my test blocked by a CAPTCHA or Cloudflare challenge? - -KaneAI runs your test on a cloud browser. Bot protection such as Cloudflare Turnstile, Google reCAPTCHA, or hCaptcha can classify that cloud session as automated and block it, so a sign up, log in, or checkout step never completes. The verification token is rejected even when the form is filled correctly. - -### Can KaneAI solve CAPTCHAs automatically? - -No. CAPTCHAs are built to block automation, and no tool can solve them reliably. The reliable approach is to stop the challenge from being served to your test traffic, not to solve it. You do not need to build a separate environment to do this. - -### How do I run tests on a site protected by Cloudflare Turnstile? - -Use whichever option fits your setup: - -1. **Whitelist TestMu AI's cloud IPs (recommended).** Add TestMu AI's cloud IP ranges to an allow rule in your security layer, a Cloudflare IP Access rule, a WAF allowlist, or a Turnstile allowlist, scoped to your test domain. Traffic from our cloud browsers is then trusted and the challenge is never shown. -2. **Use a test environment without the live challenge.** On staging or UAT, disable the CAPTCHA or switch it to a testing key. Cloudflare Turnstile provides test keys that always pass, so automation verifies while production stays fully protected. -3. **Allowlist automation with a request header.** If your application supports it, allow a known custom request header on your test environment so requests carrying it skip the challenge. Keep this to test environments only. - -**Get your IPs whitelisted** -Follow the [TestMu AI Public IP Ranges](/support/docs/testmu-public-ip/) for the current IP ranges and the exact steps. - -### Do I need a separate, CAPTCHA free environment? - -No. Whitelisting TestMu AI's cloud IPs in your existing setup is a configuration change, not new infrastructure. - -### How do I test an app on a private network or localhost? - -Whitelisting only works for applications reachable on the public internet. If your environment is on a private network, behind a VPN, or on localhost, use the [TestMu AI Tunnel](/support/docs/network-whitelisting-and-tunnel-guide/) instead. diff --git a/static/docs/kaneai-gps-location.md b/static/docs/kaneai-gps-location.md deleted file mode 100644 index 709688a3d..000000000 --- a/static/docs/kaneai-gps-location.md +++ /dev/null @@ -1,68 +0,0 @@ -# GPS Location Mocking for Mobile App and Browser Authoring - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -> This document provides detailed instructions for utilizing the GPS Location Override feature in KaneAI for both Mobile App and Mobile Browser testing. It covers how to set GPS coordinates at the start of a session as well as within a running session to test location-based application behavior accurately. - -## Overview - -Testing location-based application behavior (such as geo-restricted content, regional UI, compliance flows, or location-aware features) is challenging when device GPS coordinates are dynamic or tied to the physical location of the tester. - -The GPS Location Override feature solves this problem by allowing users to explicitly define latitude and longitude values both during session initialization and while an authoring session is in progress. This ensures consistent, repeatable, and deterministic testing of location-dependent scenarios on supported mobile devices. - -This feature is supported for both **Mobile App** and **Mobile Browser** sessions. For both sessions, GPS location can also be changed within a running session. - -## How To Use - -### Select GPS location while Authoring App Test - -While creating a test session, go to Advanced Settings and enable GPS location mocking option. Either include the location Coordinates or search for a particular location from the map. Ensure the coordinates fall within the allowed ranges: - -- Longitude: -180 to 180 -- Latitude: -90 to 90 - -### Start the Session - -1. Launch the session after providing the coordinates. -2. The device GPS location is mocked at session start. -3. The configured location remains active until it is updated during the session. - -### Verify the Applied Location - -GPS coordinates can be verified: -- Inside the running session by clicking **Advanced Settings** -- On the Session Summary page after execution -- During Edit Test, the coordinates used in authoring are autoselected in playground - -### Update GPS Location During an Active Session - -You can also set or update the GPS location while an authoring session is already running using the **slash command**. This is useful when your test flow requires the device location to change mid-session (e.g., simulating a user traveling between cities). - -**Step 1:** Inside an active authoring session, type `/` in the action input field to open the slash command menu. Select **Set GPS Location**. - -**Step 2:** In the **Select GPS Location** modal, search for a location by name or enter latitude and longitude coordinates directly. The map preview updates to reflect the selected position. - -**Step 3:** Click **Confirm Location** to apply the new GPS coordinates. A success notification confirms the update, and the new coordinates are recorded as a step in your test. - -The updated GPS location takes effect immediately on the device and remains active until changed again or the session ends. - -## Limitations - -- GPS override is applied only during session initialization and can be changed mid-session. -- Replay/Edit sessions reuse GPS values from the original session. - -## Troubleshooting - -### Invalid Latitude or Longitude Error - -**Issue:** Session fails to start due to a validation error. - -**Cause:** -- Longitude is less than -180 or greater than 180 -- Latitude is less than -90 or greater than 90 - -**Resolution:** -- Verify that latitude and longitude values are within valid ranges -- Ensure values are passed as valid numbers or numeric strings - -> Have any feedback or request? Reach out to us via [support@testmuai.com](mailto:support@testmuai.com) and we would be happy to hear from you. diff --git a/static/docs/kaneai-hyperexecute-test-run-execution.md b/static/docs/kaneai-hyperexecute-test-run-execution.md deleted file mode 100644 index 173aeef18..000000000 --- a/static/docs/kaneai-hyperexecute-test-run-execution.md +++ /dev/null @@ -1,129 +0,0 @@ -# Create and Execute your Test Run for KaneAI Generated Tests on HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This documentation outlines the steps to create a test run for KaneAI generated tests using Test Manager. The steps include selecting test cases, setting configurations, and running tests with HyperExecute for efficient execution. Follow the instructions carefully to ensure seamless execution of the process. - -## Prerequisites -- Access to Test Manager. -- A project with KaneAI-generated test cases. -- Credentials for TestMu AI platform. - -## Step-by-Step Guide -### Step 1: Navigate to the Projects Page -- Log in to Test Manager. Navigate to the Projects page and select the desired project. -- Proceed to the Test Cases section. - -### Step 2: Create a Test Run -- Go to the Test Run section within the project. -- Click on Create Test Run. -- Provide the following details: - - **Name :** e.g., Demo Test Run 123. - - **Description :** A brief overview of the run. - - **Tags :** Any label to differentiate your test run. - - **Type :** Choose KaneAI Generated Test Cases. - -### Step 3: Select and Add Test Cases -- Click Next to view available test cases. -- Select the required test cases for the run. -- Click Add Test Case to include them. - -All KaneAI test cases with generated code are available for selection here, regardless of whether Sample Run Execution has been triggered or its outcome. Test cases where code generation has not started or is still in progress will not be available. - -### Step 4: Assign and Configure Test Cases -- On the test run page, review the added test cases. -- Assign team members in bulk. -- Update configurations as required. - -#### Set Configurations -- Choose from available desktop browser and mobile app configurations. If a new configuration is needed, click on Create Configuration button and choose from thousands of configuration options. - -#### Create New Configuration -- Enter **configuration name** and the following details to create a new Desktop configuration: - - **OS :** e.g., macOS. - - **OS Version :** e.g., Monterey. - - **Browser :** e.g., Chrome. - - **Browser Version :** e.g., 131. - - **Resolution :** e.g., 1440x900. -- Enter **configuration name** and the following details to create a new Real Device configuration: - - **OS :** e.g., android. - - **Manufacturer :** e.g., Samsung. - - **Device :** e.g., Galaxy S24. - - **OS Version :** e.g., 14. - - **App :** e.g., Proverbial. -- Click on Create Configuration - -### Step 5: Run with HyperExecute -- Initiate the run using HyperExecute by clicking **Run with HyperExecute**. -- View all test instances in the dashboard. -- Set the desired concurrency level, e.g., 5. - -## Advanced Configurations - -Before clicking **Execute**, you can optionally click **Advanced Configurations** to customize your test execution. The following settings are available: - -| Setting | Description | -|---|---| -| **Region** | Choose a region for web tests such as _eastus_ or _centralindia_ to select the region from where the VM is allocated. | -| **Mobile Region** | Choose a region for mobile app tests such as _us_, _ap_, or _eu_ to select the region from where the device is allocated. | -| **Tunnel** | Add the tunnel name if you want to run using TestMu AI Tunnel for private applications. Either tunnel, dedicated proxy, or geolocation can be used in a single execution. See [Tunnel Support](/support/docs/kane-ai-geolocation-tunnel-proxy/#tunnel-support) for details. | -| **Dedicated Proxy** | Add the region of the dedicated proxy such as _us_ or _eu_. Either tunnel, dedicated proxy, or geolocation can be used in a single execution. | -| **Geolocation** | Set the geolocation from which you want to run your tests. See the list of supported geolocations [here](/support/docs/selenium-geolocation-capabilities/). Either tunnel, dedicated proxy, or geolocation can be used in a single execution. | -| **Console Log** | Enable console log capture for web tests. Options: `false`, `error`, `warn`, `info`, `true`. | -| **Network Logs** | Enable capture of network calls during the test. | -| **Network Full HAR** | Capture complete network requests and responses during web tests. | -| **Network Throttle** | Enable network throttling for mobile tests by defining download speed, upload speed, and latency. | -| **Timezone** | Define the timezone for web test runs (e.g., UTC+01:00 format). | -| **Retry on Failure** | Enable to retry on failure. When enabled, retries are triggered both on [testRunnerCommand](/support/docs/deep-dive-into-hyperexecute-yaml/#testrunnercommand) failure and individual test case failure. Defaults to true. See [Retry on Failure](/support/docs/deep-dive-into-hyperexecute-yaml/#retryonfailure) for details. | -| **Max Retries** | Define the maximum number of retries when retry on failure is enabled. Max value 5, default 1. See [Max Retries](/support/docs/deep-dive-into-hyperexecute-yaml/#maxretries) for details. | -| **Replace URL** | Dynamically replace any pattern URL in test cases with a replacement URL for the entire test run. See [Dynamic URL Replacement](/support/docs/kaneai-dynamic-url-replacement/) for details. | -| **Performance** | Enable Lighthouse performance reports for web tests. Supported on limited OS browser combinations. May slow down execution time. See [Lighthouse Performance Metrics](/support/docs/view-lighthouse-performance-metrics/) for details. | -| **App Profiling** | Enable app performance metrics tracking for native mobile app tests. See [App Performance Analytics](/support/docs/appium-app-performance-analytics/) for details. | -| **Android App ID** | Specify an Android app ID (`lt://`) to override the existing app in the test instance configuration. | -| **iOS App ID** | Specify an iOS app ID (`lt://`) to override the existing app in the test instance configuration. | -| **Visual Regression** | Add visual testing configuration: select browsers, viewports, devices, and orientation. See [Visual Testing with SmartUI](/support/docs/kaneai-smartui-visual-testing/) for details. | -| **Accessibility** | Enable accessibility checks (WCAG 2.1 AA) with best practices and needs review options. Available for web with Chrome and Edge browsers only. May slow down execution time. | -| **Report Enabled** | Set to true to enable report generation for the test run. Select either HTML or Extent report format. Only one can be active at a time. Reports are accessible from the HyperExecute Job page after execution. See [below](#reports). | -| **Report Email To** | An array of email addresses to receive the test run report via email after execution. Maximum 10 email addresses. Only works when report is enabled. | - -Test case failure retries are supported only for code exported from **May 10, 2026 onwards**. For previously exported code, retries are triggered only on test runner command failure. To use this capability, regenerate the code export for your test cases in Test Manager. - -### Reports - -Enable the **Reports** option in Advanced Configurations to generate reports for your test run. You can enable either an HTML report or an Extent report. Only one can be active at a time. Reports are supported for both web and mobile test executions. - -- **Generate HTML Report**: Toggle to **Yes** to generate an HTML report for the test run, available in the HyperExecute dashboard after execution. -- **Generate Extent Report**: Toggle to **Yes** to generate an Extent report for the test run, available in the HyperExecute dashboard after execution. -- **Email Addresses**: Add one or more email addresses (separated by space or enter) to receive the report via email after execution. You can add up to **10 email addresses**. - -**Accessing the Report** - -Once your test run execution is complete, you can access the generated HTML report from the HyperExecute Job page. Navigate to the job and click on the **Reports** section to view or download the report. - -**HTML Report Overview** - -The generated HTML report includes: - -- **Summary**: Job metadata (job number, labels, username, build time, date), test summary with pass/fail counts and donut charts, scenario summary, task analytics, and browser-level breakdown. -- **Test Cases**: A detailed list of all test cases with their status (pass/fail), OS, OS version, browser, duration, and links to view the test or watch the video recording. - -**Extent Report Overview** - -The generated Extent report provides a rich interactive view of test execution results, accessible from the HyperExecute Job page. It includes: - -- **Dashboard**: A visual summary of the test run with pass/fail statistics and trend data. -- **Test Details**: Step-by-step breakdown of each test case with status and screenshots. - -**Email Report** - -If you added email addresses, recipients receive an email with a summary of the test run results along with the HTML report as an attachment. - -Reports are also available when configuring [Scheduled Test Runs](/support/docs/kaneai-scheduled-test-runs/). - -- Click **Execute** to run the tests. - -### Monitor Execution on HyperExecute -- Navigate to the HyperExecute page. -- Observe the discovery and rapid execution of test cases. - -## Video Explanation diff --git a/static/docs/kaneai-kb-api-testing-and-network-assertions.md b/static/docs/kaneai-kb-api-testing-and-network-assertions.md deleted file mode 100644 index 31e1349bf..000000000 --- a/static/docs/kaneai-kb-api-testing-and-network-assertions.md +++ /dev/null @@ -1,250 +0,0 @@ -# API Testing & Network Assertions - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -KaneAI isn't just a UI testing tool. It also supports API testing and network-level assertions, allowing you to validate both frontend behavior and backend responses in a single test flow. This is critical for scenarios where the UI depends on API data, or where you need to verify that the correct API calls are being made behind the scenes. - -## API Testing in KaneAI - -### Adding an API Call - -You can add API calls to your test using the `/` (slash) command: - -1. Type `/` in the input field -2. Select **API** from the options -3. Paste your **curl command**. KaneAI will auto-populate all request details (URL, method, headers, body) -4. Click **Validate** to execute the API and see the response -5. If the response is successful (2xx), click **Add to Test** to include it as a test step - -### Supported HTTP Methods - -KaneAI supports all standard HTTP methods: - -| Method | Use Case | -|---|---| -| **GET** | Retrieve data (fetch user, list products) | -| **POST** | Create data (register user, create order) | -| **PUT** | Update data (update profile, modify settings) | -| **PATCH** | Partial update (change email, update single field) | -| **DELETE** | Remove data (delete user, cancel order) | - -### Example: GET Request - -```bash -curl -X GET "https://api.example.com/users/123" \ --H "Authorization: Bearer {{secret.api_token}}" \ --H "Content-Type: application/json" -``` - -### Example: POST Request - -```bash -curl -X POST "https://api.example.com/users" \ --H "Authorization: Bearer {{secret.api_token}}" \ --H "Content-Type: application/json" \ --d '{ -"name": "John Doe", -"email": "john@example.com", -"role": "admin" -}' -``` - -## API Response Variables - -After a successful API call, the response is automatically stored as a **JSON variable**. You can reference it in subsequent steps using `{{variable_name}}` syntax. - -### Accessing Response Data - -``` --- After making a GET /users/123 API call -- --- Response stored as {{api_users}} -- - -assert {{api_users.response.name}} is "John Doe" -assert {{api_users.response.email}} is "john@example.com" -assert {{api_users.response.status}} is "active" -``` - -### Accessing Nested Data - -Use dot notation to reach nested properties: - -``` -assert {{api_users.response.address.city}} is "New York" -assert {{api_users.response.orders[0].total}} is "150.00" -``` - -### Using API Data in UI Steps - -Combine API data with UI interactions: - -``` --- Fetch user data from API -- --- (stored as {{api_user}}) -- - --- Navigate to UI and verify data matches -- -go to https://app.example.com/users/123 -wait for 3 seconds -assert the name field shows {{api_user.response.name}} -assert the email field shows {{api_user.response.email}} -``` - -## Network Logs Assertions - -The **Network Logs Assertions** feature lets you validate actual network calls that your application makes during a test session. This is different from making your own API calls. It captures what the app itself is doing behind the scenes. - -### How It Works - -1. During a test session, **all network calls are recorded automatically** in the background -2. Use `/` → **Assert Network Logs** to view them -3. Filter by **Status Code** (200, 404, 500, etc.) or **HTTP Method** (GET, POST, etc.) -4. Select the specific request you want to assert on -5. Choose which **request headers**, **request body**, or **response body** keys to validate -6. Click **Add Assertion**. All chosen values are validated in a single step - -### Accessing Network Data as Variables - -Once you add a network assertion, the request data is stored as a **variable** in JSON format. You can reuse it later with `{{variable}}` syntax and access nested values with dot notation (e.g., `{{network_req.response.data.userId}}`). - -Network log assertions are currently only supported on desktop web authoring in KaneAI. You can find more details in [Network Logs Assertion](/support/docs/kane-ai-network-assertions/) guide. - -## Real-World API + UI Scenarios - -### Scenario: Create User via API, Verify in UI - -Use the API to set up test data, then verify it appears correctly in the UI. - -``` --- Step 1: Create user via API -- --- (Use / → API → Paste curl for POST /api/users) -- --- Response stored as {{new_user}} -- - --- Step 2: Verify in UI -- -go to https://app.example.com/admin/users -wait for 3 seconds -type {{new_user.response.email}} in the search field -wait for 2 seconds -assert {{new_user.response.name}} is visible in the users table -``` - -### Scenario: Verify UI Action Triggers Correct API Call - -Test that clicking "Add to Cart" on the UI actually sends the right API call. - -``` --- Browse to product page -- -go to https://shop.example.com/products/laptop -click "Add to Cart" -wait for 3 seconds - --- Assert the network call was correct -- --- (Use / → Assert Network Logs → Find the POST /cart/items call) -- --- Assert the request body contains the correct product ID -- --- Assert the response status is 200 -- -assert "Added to cart" success message is visible -``` - -### Scenario: E-Commerce Checkout with Backend Validation - -``` --- Step 1: Add items to cart via UI -- -go to https://shop.example.com -click "Add to Cart" on "Wireless Mouse" -click "Add to Cart" on "USB Cable" - --- Step 2: Proceed to checkout -- -click on "Cart" icon -click on "Checkout" - --- Step 3: Verify order total via API -- --- (Use / → Assert Network Logs → Find the POST /api/orders call) -- --- Assert the response total matches what the UI shows -- -assert the order total on the page matches {{order_api.response.total}} -``` - -### Scenario: Test API Error Handling in UI - -Verify that the UI handles API errors gracefully. - -``` --- Navigate to a page that fetches data -- -go to https://app.example.com/reports/nonexistent -wait for 5 seconds - --- Assert Network Logs shows a 404 -- --- (Use / → Assert Network Logs → Find the GET /api/reports/nonexistent call) -- --- Assert status code is 404 -- - --- Verify UI handles the error -- -assert "Report not found" message is visible -assert no unhandled error or stack trace is shown on the page -``` - -### Scenario: API-Driven Test Data Setup - -Before running a UI test, use APIs to set up the exact state you need. - -``` --- Create test product via API -- --- POST /api/products with name "Test Product", price 29.99 -- --- Response stored as {{test_product}} -- - --- Create test user via API -- --- POST /api/users with name "Test Buyer" -- --- Response stored as {{test_user}} -- - --- Now run the UI test -- -go to https://shop.example.com/login -type {{test_user.response.email}} in the email field -type "password123" in the password field -click "Sign In" -wait for 3 seconds - --- Search for the product we created -- -type {{test_product.response.name}} in the search field and press Enter -wait for 3 seconds -assert {{test_product.response.name}} is visible in the results -click "Add to Cart" -assert cart count shows "1" -``` - -### Scenario: Validate API Response Headers - -``` --- Make an API call -- --- GET /api/users with Authorization header -- - --- Assert response headers -- -assert {{api_users.headers.content-type}} contains "application/json" -assert {{api_users.headers.x-ratelimit-remaining}} is not "0" -``` - -## Combining API Testing with Database Queries - -KaneAI also supports database queries. You can combine API calls, UI interactions, and database validation in a single test: - -``` --- Step 1: Create order via UI -- --- ... add items and checkout ... -- - --- Step 2: Verify API response -- --- Assert the POST /orders response contains order ID -- - --- Step 3: Verify in database -- --- (Use / → Database Query) -- --- Query: SELECT * FROM orders WHERE id = {{order_api.response.id}} -- --- Assert the database record matches the API response -- -``` - -For database testing details, see [Database Connections](/support/docs/kaneai-database). - -## Best Practices - -| Practice | Details | -|---|---| -| **Use API calls for test data setup** | Faster and more reliable than creating data through the UI | -| **Store API tokens as secrets** | Never hardcode tokens; use `{{secret.api_token}}` | -| **Validate both UI and API** | A test that only checks UI can miss backend errors | -| **Use network assertions for passive validation** | Verify what the app is doing under the hood without making separate calls | -| **Filter network logs by status code** | Quickly find 4xx/5xx errors your app might be making silently | -| **Chain API response variables** | Use output from one API call as input to the next | -| **Test error scenarios via API** | Set up invalid data via API, then verify UI handles it gracefully | diff --git a/static/docs/kaneai-kb-assertions-and-validation.md b/static/docs/kaneai-kb-assertions-and-validation.md deleted file mode 100644 index c6ca91540..000000000 --- a/static/docs/kaneai-kb-assertions-and-validation.md +++ /dev/null @@ -1,347 +0,0 @@ -# Assertions & Validation - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Assertions verify that your application behaves correctly after performing actions. In KaneAI, you write assertions in natural language, and the AI evaluates them against the current page state. This guide covers every assertion type, shows how to write them reliably, and explains how to handle tricky validation scenarios. - -All assertions have an option for failure behavior. By default, a failure in assertion will fail the test during execution. You can configure the default failure behavior for assertions via Organisation Settings. See more details in [Failure Conditions](/support/docs/kaneai-failure-conditions/#default-failure-condition-setting). - -## Assertion Categories - -### Text Assertions - -Check whether specific text content is present (or absent) on the page. - -``` -assert "Welcome Back, John!" is visible -assert "Your order has been placed" is displayed -assert error message "Invalid email address" is visible -assert "Out of Stock" is not visible on the page -``` - -**With variables:** -``` -assert {{username}} is visible on the page -assert {{api_response.message}} contains "success" -``` - -### Visual Assertions - -Verify that images, logos, icons, or visual elements appear correctly. - -``` -assert the TestMu logo is visible in the header -assert the product image is displayed -assert the green checkmark icon appears next to "Verified" -``` - -Visual assertions rely on screenshots. Elements smaller than 12×12 pixels may not be detected reliably. Avoid asserting on exact hex colors. Use descriptive colors instead. - -### URL & Browser State Assertions - -Validate the browser state such as the current URL or page title. - -``` -assert the current URL is "https://example.com/dashboard" -assert the current URL contains "/dashboard" -assert the page title contains "Dashboard" -``` - -### Relative/Layout Assertions - -Check the spatial relationship between elements. - -``` -assert the "Login" button is in the same row as the "Register" link -assert the "Submit" button is below the form fields -``` - -### Mathematical Assertions - -Validate calculations and numeric values. - -``` -assert 3 + 4 = 7 -assert the cart total equals the sum of item prices visible -``` - -## Element State & Attribute Assertions - -Assert element states, DOM attributes, and computed CSS properties by querying the DOM directly. Describe the element using its visible text or description and state the condition you want to verify. - -**Desktop & Mobile Web Only** -Element State & Attribute Assertions are currently supported on **desktop web** and **mobile web** only. Native mobile app assertions (Android/iOS) are planned for a future phase. - -### Element State Assertions - -Verify the interactive state of UI elements using natural language. - -| State | Positive Example | Negative Example | -|---|---|---| -| **Enabled / Disabled** | `Assert the "Submit Order" button is enabled` | `Assert the "Authorize Payment" button is disabled` | -| **Visible / Hidden** | `Assert the "Dashboard panel" is visible` | `Assert the "Hidden section" is not visible` | -| **Clickable** | `Assert the "Click Me" button is clickable` | `Assert the "View Only" button is not clickable` | -| **Present / Absent** | `Assert the "Account widget" is present` | `Assert the error banner is not present` | -| **Checked / Unchecked** | `Assert the "Terms" checkbox is checked` | `Assert the "Newsletter" checkbox is not checked` | -| **Toggled On / Off** | `Assert the Notifications toggle is toggled on` | `Assert the Dark Mode toggle is toggled off` | - -**Present vs. Visible** -These two assertions behave differently when an element is not found: -- **"Assert element is hidden"**: Element must exist in the DOM but not be displayed. Returns an **error** if the element is not found at all. -- **"Assert element is not present"**: Element does not exist in the DOM. Returns **pass** if the element is not found. - -### DOM Attribute Assertions - -Assert the value of any HTML, ARIA, or data attribute on an element. - -| Category | Example | -|---|---| -| **ARIA** | `Assert the aria-expanded of the "Accordion Header" equals "true"` | -| **ARIA** | `Assert the aria-label of the Close button equals "Close dialog"` | -| **ARIA** | `Assert the role of the alert element equals "alert"` | -| **Data attributes** | `Assert the data-testid of the "Workspace" equals "main-content"` | -| **Data attributes** | `Assert the data-status of the "Sprint tracker" equals "active"` | -| **HTML attributes** | `Assert the placeholder of the search input contains "Search"` | -| **HTML attributes** | `Assert the href of the "Dashboard" link starts with "https://"` | -| **HTML attributes** | `Assert the src of the Logo image ends with ".svg"` | -| **Form attributes** | `Assert the action of the Signup form equals "/api/signup"` | -| **Form attributes** | `Assert the method of the form equals "POST"` | - -**Attribute existence checks:** - -``` -Assert the Sample video has a controls attribute -Assert the input has a required attribute -Assert the NoAlt image does not have an alt attribute -``` - -### CSS Property Assertions - -Assert computed CSS property values for the following properties: - -`color` · `background-color` · `border-color` · `font-size` · `font-family` · `font-weight` · `font-style` · `display` · `visibility` · `opacity` - -| Category | Example | -|---|---| -| **Color** | `Assert the color of the "Alert message" equals "rgb(255, 0, 0)"` | -| **Named color** | `Assert the background-color of the "Banner" equals "blue"` | -| **Font size** | `Assert the font-size of the "Heading" equals "24px"` | -| **Font family** | `Assert the font-family of the "Body text" contains "Arial"` | -| **Font weight** | `Assert the font-weight of the "Bold text" equals "700"` | -| **Font style** | `Assert the font-style of the "Quote" equals "italic"` | -| **Display** | `Assert the display of the "Layout" equals "flex"` | -| **Visibility** | `Assert the visibility of the "Layer" equals "hidden"` | -| **Opacity** | `Assert the opacity of the "Watermark" equals "0.5"` | - -**CSS Value Normalization** -CSS values are automatically normalized before comparison: -- **Colors:** Named colors (e.g., "red") are converted to RGB. All hex/rgb/rgba formats normalize to a canonical form. -- **Font families:** Surrounding quotes are stripped. -- **Whitespace:** Extra whitespace is collapsed. - -Named color normalization is exact-match only: "red" maps to `rgb(255, 0, 0)`, but shades like `#d10000` will **not** match "red". Use exact RGB values for precision. - -### Supported Operators - -**String Operators** - -| Operator | NL Triggers | Example | -|---|---|---| -| Equals | "is", "equals" | `Assert the role equals "alert"` | -| Does not equal | "is not", "does not equal" | `Assert the type does not equal "password"` | -| Contains | "contains", "includes" | `Assert the placeholder contains "Search"` | -| Does not contain | "does not contain" | `Assert the href does not contain "staging"` | -| Starts with | "starts with" | `Assert the href starts with "https://"` | -| Does not start with | "does not start with" | `Assert the href does not start with "http://"` | -| Ends with | "ends with" | `Assert the src ends with ".svg"` | -| Does not end with | "does not end with" | `Assert the src does not end with ".png"` | - -**Numeric Operators** - -| Operator | NL Triggers | Example | -|---|---|---| -| Greater than | "is greater than" | `Assert the data-count is greater than "5"` | -| Less than | "is less than" | `Assert the data-index is less than "10"` | -| Greater than or equal | "is at least" | `Assert the data-score is at least "5"` | -| Less than or equal | "is at most" | `Assert the data-priority is at most "5"` | - -Negation is supported across all assertion types using "NOT", "is not", "isn't", or "does not have". - -### Limitations - -- **Shadow DOM not supported:** Elements inside shadow DOM boundaries are not accessible. -- **No regex matching:** Pattern-based matching (e.g., "Assert data-id matches `[a-f0-9-]{36}`") is not supported. -- **Value length cap:** Attribute or CSS values exceeding 500 characters are truncated with a warning. -- **Hidden element resolution:** Element finding relies on vision. Elements present in the DOM but not visible on the page may not be resolved correctly. - -## Writing Good Assertions: Do's and Don'ts - -| Good Assertion | Bad Assertion | Why | -|---|---|---| -| `assert text "Submit" on the form footer is visible` | `assert that the submit button works and the layout is correct` | Combines multiple concerns; "works" is vague | -| `assert the textbox contains the exact value "heading1"` | `assert the textbox has heading` | "heading" is ambiguous, which heading? | -| `assert the TestMu logo is visible` | `assert 3 search results are shown` (as a visual assertion) | Count assertions need text-based evidence, not visual | -| `assert "Error" is not visible on the page` | `assert there are no errors` | Too vague; specify what kind of error | - -## Key Rules for Reliable Assertions - -### Be Specific, Not Subjective - -The assertion must have a clear true/false answer given the page state. - -**Bad** -``` -assert the page is user-friendly -assert the layout is correct -``` - -**Good** -``` -assert the navigation menu has 5 items -assert the "Search" input field is visible at the top of the page -``` - -### Avoid Asserting on Imperceivable Details - -KaneAI uses a screenshot + DOM combination. Very small visual details may not be detectable. - -**Risky assertions (may be flaky):** -- Exact hex color codes (`#ff5733`) -- Colors of thin lines or indicator dots -- SVG shapes smaller than 12px -- Font sizes or font families - -**Safer alternatives:** -``` -assert the button appears red (instead of assert color is #ff0000) -assert the error border is visible (instead of assert border is 2px solid red) -``` - -## Real-World Assertion Scenarios - -### Scenario: Login Flow Validation - -``` -type "admin@example.com" in the email field -type "wrong_password" in the password field -click on "Sign In" button -assert "Invalid credentials" error message is visible - -clear the password field -type "CorrectPass123" in the password field -click on "Sign In" button -assert the current URL contains "/dashboard" -assert "Welcome, Admin" is visible -``` - -### Scenario: E-Commerce Cart Validation - -``` -click "Add to Cart" for the "Wireless Mouse" product -click on the cart icon -assert "Wireless Mouse" is visible in the cart -assert the cart item count shows "1" -assert the cart total is "$29.99" -click "Remove" next to "Wireless Mouse" -assert "Your cart is empty" is visible -``` - -### Scenario: Form Validation Messages - -``` -click the "Submit" button without filling any fields -assert "First name is required" error is visible -assert "Email is required" error is visible -assert "Password is required" error is visible - -type "J" in the "First Name" field -assert "First name must be at least 2 characters" is visible -``` - -### Scenario: Search Results Verification - -``` -type "laptop" in the search field and press Enter -wait for 3 seconds -assert search results are visible -assert the first result contains the word "laptop" -assert the results count is greater than 0 -``` - -### Scenario: Data Table Content Validation - -``` -assert the "Users" table header is visible -assert the table contains a row with "john@example.com" -assert "Active" status is shown next to "john@example.com" -``` - -## Asserting with Variables - -Variables make assertions dynamic and reusable, especially when combined with API responses or stored values. - -### String Variable Assertions - -``` -set expectedTitle as "Dashboard" -assert {{expectedTitle}} is visible on the page -``` - -### JSON Variable Assertions (from API) - -After making an API call via the `/` command, the response is stored as a JSON variable: - -``` -assert {{api_response.status}} is "200" -assert {{api_response.data.email}} is "john@example.com" -assert {{api_response.data.items}} contains "Laptop" -``` - -### Cross-Referencing UI and API Data - -Store a value from the UI, make an API call, and compare: - -``` -Fetch the order ID displayed on the confirmation page --- (stored as a variable such as {{orderID}}) -- --- Make API call to verify order via /api command -- -assert {{api_response.orderId}} equals {{orderID}} -``` - -## Conditional Assertions - -KaneAI supports basic if-else logic for assertions that depend on page state. - -``` -if "Login" button is visible then click on it -if price < 200 then click "Add to Cart" else select the second product -``` - -Nested conditions (an If / Else block inside another If / Else block) are **not supported**. Keep conditional logic simple and flat. For complex branching, use Else‑If branches or break the test into separate test cases. See the full [Conditional Logic](/support/docs/kaneai-conditional-logic/) guide. - -## Unsupported Assertions (Current Limitations) - -These assertion types are not yet available via natural language. Use **JS snippets** as workarounds where applicable. - -| Category | Example | Workaround | -|---|---|---| -| **Specific DOM Position** | `assert the 5th table column contains "Jordan"` | JS: `document.querySelectorAll('td')[4].textContent` | -| **Action-Based** | `assert tooltip appears after hover` | Split into: hover step → wait → assert tooltip text | -| **Nested Conditions** | `assert A is true AND B is visible` | Split into two separate assertions | -| **Changes Over Time** | `assert the spinner disappears after 5 seconds` | Use wait step, then assert spinner is not visible | - -For the full list of JS execution options, see [JS Snippets](/support/docs/kane-ai-javascript-execution/) guide. - -## Configuring Assertion Failure Behavior - -By default, assertions fail the test immediately (unless configured otherwise in Organisation Settings). You can configure this per step: - -| Mode | Behavior | -|---|---| -| **Fail Immediately** (default) | Test stops at the failed assertion | -| **Fail and Continue** | Assertion failure is recorded, but subsequent steps continue executing | -| **Warn and Continue** | Assertion failure is logged as a warning; test is not marked as failed | - -Configure via: Step menu → **Failure Condition** → Select behavior. - -This is useful when you want to run all assertions in a test and see a complete report of what passed and what failed, rather than stopping at the first failure. diff --git a/static/docs/kaneai-kb-authentication-and-session-management.md b/static/docs/kaneai-kb-authentication-and-session-management.md deleted file mode 100644 index efd913d3d..000000000 --- a/static/docs/kaneai-kb-authentication-and-session-management.md +++ /dev/null @@ -1,277 +0,0 @@ -# Authentication & Session Management - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Authentication testing is one of the most critical areas for any QA team. This guide covers how to test login flows, multi-factor authentication (MFA/2FA), TOTP, session handling, and authentication-related edge cases in KaneAI. - -## Basic Login Flows - -### Standard Username/Password Login - -``` -go to https://app.example.com/login -type "admin@example.com" in the email input field -type "SecurePassword123!" in the password field -click on the "Sign In" button -wait for 5 seconds -assert the current URL contains "/dashboard" -assert "Welcome, Admin" is visible -``` - -### Login with "Remember Me" - -``` -type "admin@example.com" in the email field -type "SecurePassword123!" in the password field -check the "Remember me" checkbox -click on the "Sign In" button -wait for 5 seconds -assert the current URL contains "/dashboard" -``` - -### Reusable Login Module - -If most of your tests start with a login, create a **Module** to avoid repeating the login steps in every test: - -1. Author the login steps in a test -2. Pause the test -3. Select the login steps -4. Click **Create Module** → Name it "Login Flow" -5. In future tests, use `/` → Select your "Login Flow" module - -``` --- In any new test: -- --- Use / command → Add Module → "Login Flow" -- --- The module executes all login steps automatically -- -assert "Welcome" is visible --- Continue with your test-specific steps -- -``` - -Use **environment variables** for credentials so the same login module works across staging, QA, and production environments. Set `{{env.login_email}}` and `{{env.login_password}}` per environment. - -## Using Variables for Credentials - -### Environment Variables (Recommended) - -Create environment-specific variables to test login across different environments: - -``` -type {{env.login_email}} in the email field -type {{env.login_password}} in the password field -click "Sign In" -``` - -Set different values for each environment: -- **Staging**: `login_email` = `test@staging.example.com` -- **Production**: `login_email` = `readonly@example.com` - -You can find more information about environment variables in [this guide](/support/docs/kane-ai-using-variables/#environments). - -### Secrets (For Sensitive Data) - -Store passwords securely using KaneAI Secrets (backed by HashiCorp Vault): - -1. Go to [KaneAI Variables](https://www.testmuai.com/login/?redirectTo=https://kaneai.lambdatest.com/variables) -2. Create a secret variable for the password -3. Reference it in your test: `type {{secret.admin_password}} in the password field` - -Secrets are: -- Encrypted at rest and in transit -- Never exposed in logs or generated code -- Not visible in the test step output - -## TOTP / Multi-Factor Authentication - -KaneAI has native TOTP support, which means you can test applications protected by authenticator apps (Google Authenticator, Okta Verify, Microsoft Authenticator) without any external scripts. - -### How It Works - -1. KaneAI stores your TOTP **secret key** (Base32 encoded) securely -2. At runtime, KaneAI generates the correct 6-digit code every 30 seconds -3. You reference the code as a smart variable in your test steps - -### Step-by-Step: Testing a TOTP Login Flow - -**Step 1: Register the TOTP Secret Key** - -1. Start a web authoring session -2. Type `/` and select **Add TOTP Authentication Key** -3. Enter the Base32 secret key (the one you'd normally scan as a QR code in an authenticator app) -4. You can also use a LambdaTest secret: `{{secret.totp_key}}` -5. Click **Save** -6. A smart variable (e.g., `{{totp}}`) is generated - -**Step 2: Use the TOTP in Your Test** - -``` --- Login with username/password first -- -type "admin@example.com" in the email field -type {{secret.admin_password}} in the password field -click "Sign In" - --- MFA screen appears -- -wait for 3 seconds -enter {{totp}} in the verification code field -click "Verify" - --- Authenticated -- -wait for 5 seconds -assert the current URL contains "/dashboard" -assert "Welcome, Admin" is visible -``` - -The `{{totp}}` smart variable generates a time-based 6-digit code based on the registered secret key. KaneAI calculates the correct code at the moment the step executes, so you get a valid code for each step run. Since TOTP codes refresh every 30 seconds, ensure that the step entering the code and the step submitting it execute in quick succession. Avoid placing long waits between them. - -### TOTP on Mobile - -TOTP smart variables work identically on mobile tests: - -``` -type {{env.login_email}} in the email field -type {{secret.login_password}} in the password field -click "Login" -wait for 3 seconds -enter {{totp}} in the OTP field -click "Verify" -``` - -### Where to Get the TOTP Secret Key - -- Most authenticator setups show a "Can't scan QR?" link that reveals the Base32 secret key -- For enterprise SSO, ask your IT team for the TOTP secret associated with the test account - -## SSO (Single Sign-On) Flows - -SSO flows typically redirect to an identity provider (Okta, Azure AD, Google Workspace) and then back to the application. - -KaneAI does not currently support testing SSO flows. SSO involves cross-domain redirects and third-party identity provider pages, which are outside the scope of KaneAI's natural language automation at this time. - -### Recommended Alternatives - -- **Bypass SSO in test environments:** Configure your application to allow direct login (username/password) in staging/test environments, bypassing the SSO redirect entirely. -- **Use API-based authentication:** Obtain a session token or auth cookie via API calls before launching KaneAI, and inject it into the browser session. -- **Pre-authenticated sessions:** Work with your identity provider to set up test accounts with persistent sessions that don't require interactive login. - -## Testing Login Validation & Edge Cases - -### Invalid Credentials - -``` -type "admin@example.com" in the email field -type "wrong_password" in the password field -click "Sign In" -wait for 2 seconds -assert "Invalid email or password" error message is visible -assert the current URL still contains "/login" -``` - -### Empty Field Validation - -``` -click "Sign In" without entering any credentials -assert "Email is required" error is visible -assert "Password is required" error is visible -``` - -### Account Lockout After Failed Attempts - -``` --- Attempt 1 -- -type "admin@example.com" in the email field -type "wrong1" in the password field -click "Sign In" -wait for 2 seconds - --- Attempt 2 -- -type "wrong2" in the password field -click "Sign In" -wait for 2 seconds - --- Attempt 3 -- -type "wrong3" in the password field -click "Sign In" -wait for 2 seconds - -assert "Account locked" message is visible -``` - -## Session Management Testing - -### Session Persistence After Page Refresh - -``` --- Login first -- -type "admin@example.com" in the email field -type "password123" in the password field -click "Sign In" -wait for 5 seconds -assert "Dashboard" is visible - --- Refresh the page -- -refresh the page -wait for 3 seconds - --- Should still be logged in -- -assert "Dashboard" is visible -assert "Welcome, Admin" is visible -``` - -### Logout Flow - -``` -click on the user avatar in the top right corner -click on "Sign Out" -wait for 3 seconds -assert the current URL contains "/login" - --- Verify session is cleared -- -go to https://app.example.com/dashboard -wait for 3 seconds -assert the current URL contains "/login" -``` - -### Testing Session Timeout - -For applications with session timeouts, combine explicit waits with assertions: - -``` --- Login -- --- ... login steps ... -- -assert "Dashboard" is visible - --- Wait for session timeout (if timeout is 60 seconds in test env) -- -wait for 65 seconds - --- Try to navigate -- -click on "Reports" in the sidebar -wait for 3 seconds -assert "Session expired" is visible -``` - -For session timeout testing, consider setting a short timeout in your test environment (e.g., 60 seconds) rather than waiting for production-length timeouts (30 minutes). - -## Custom Headers for Authentication - -For applications that require auth tokens in headers, use KaneAI's [Custom Headers](/support/docs/kaneai-custom-headers) feature: - -1. In session settings, add custom headers -2. Example: `Authorization: Bearer ` -3. All requests from the browser will include this header - -This is useful for: -- Bypassing login in test environments -- Testing API-authenticated applications -- Injecting test-user tokens - -## Best Practices for Authentication Testing - -| Practice | Details | -|---|---| -| **Use environment variables for credentials** | Never hardcode passwords in test steps | -| **Store passwords as secrets** | Use KaneAI Secrets (HashiCorp Vault) for encrypted storage | -| **Create a login module** | Avoid repeating login steps in every test | -| **Test negative cases** | Always test invalid credentials, empty fields, and lockout | -| **Use TOTP smart variables for MFA** | Don't use hardcoded OTPs that expire | -| **Add waits after SSO redirects** | SSO flows involve multiple page loads | -| **Test session persistence** | Refresh the page after login to verify session | -| **Test logout properly** | Verify that navigating to protected pages redirects to login | diff --git a/static/docs/kaneai-kb-dynamic-content-waits-and-page-state.md b/static/docs/kaneai-kb-dynamic-content-waits-and-page-state.md deleted file mode 100644 index a8405d2df..000000000 --- a/static/docs/kaneai-kb-dynamic-content-waits-and-page-state.md +++ /dev/null @@ -1,250 +0,0 @@ -# Dynamic Content, Waits & Page State - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Modern web applications are heavily asynchronous: data loads from APIs, pages render progressively, notifications pop up and disappear, and content appears based on user interaction. This guide teaches you how to handle these dynamic behaviors reliably in KaneAI tests. - -## Understanding Waits in KaneAI - -KaneAI provides three types of wait mechanisms: - -### 1. Default Auto-Wait (Built-In) - -Every element interaction command (click, type, assert, etc.) automatically waits up to **10 seconds** for the target element to become interactable. You don't need to add explicit waits for most simple interactions. - -**When auto-wait is sufficient:** -- Clicking a button that's already on the page -- Typing in a visible input field -- Asserting text that's already rendered - -### 2. Explicit Waits - -Use explicit waits when you know an action will take longer than the default 10 seconds, or when you need to pause between steps. - -``` -wait for 5 seconds -wait for 10 seconds -``` - -**When to use explicit waits:** -- After form submission that triggers a page redirect -- After clicking a button that triggers an API call -- Between cascading dropdown selections -- After scrolling on infinite-scroll pages -- Before asserting on content loaded from an API - -### 3. Custom Step Timeout - -For individual steps that need more time, set a custom timeout via the step menu: - -1. Click the **step menu** (three dots) -2. Select **Step Timeout** -3. Enter your timeout value (min **1** second, max **300** seconds; default is **10** seconds when unset) - -**When to use custom timeout:** -- Waiting for file uploads to complete -- Waiting for large reports to generate -- Waiting for third-party payment gateways to respond - -**Recommendation** -Prefer **custom step timeout** over explicit waits when you're waiting for a specific element to appear. Custom timeouts are dynamic: they proceed as soon as the element is ready, while explicit waits always wait the full duration. - -**Supported platforms** -Step Timeout is available for **Web**, **Mobile Web**, and **App (Android and iOS)** testing. It applies only to element-based interactions (click, type, search, clear); fixed `wait` steps and assertions are unaffected. - -## Real-World Dynamic Content Scenarios - -### Scenario: Loading Spinner After Form Submit - -Your app shows a spinner after clicking "Submit" and then displays a success message. - -``` -click on the "Submit" button -wait for 5 seconds -assert "Submission Successful" is visible -assert the loading spinner is not visible -``` - -**Better approach using custom timeout:** -1. Set the custom step timeout to 15 seconds for the assertion step -2. Write: `assert "Submission Successful" is visible` -3. KaneAI will wait up to 15 seconds for the message, proceeding immediately when it appears - -### Scenario: Toast Notification (Auto-Dismiss) - -Toast notifications appear briefly (typically 3-5 seconds) and then disappear. KaneAI does not currently support interacting with toast notifications via natural language commands. Use a JS snippet to capture them instead. -```javascript -// Wait briefly and capture the toast content -return document.querySelector('.toast-message')?.textContent || 'NO TOAST FOUND'; -``` - -Adjust the selector (`.toast-message`) to match your application's toast component class or ID. - -### Scenario: Modal/Dialog After Button Click - -``` -click on "Delete Account" button -wait for 2 seconds -assert the confirmation modal is visible -assert "Are you sure you want to delete your account?" is visible in the modal -click "Cancel" inside the modal -assert the modal is not visible -``` - -### Scenario: Auto-Complete Search (Debounced Input) - -Search fields often wait 300-500ms after the user stops typing before making an API call. - -``` -type "react testing" in the search bar -wait for 2 seconds -assert search suggestions are visible -click on the first suggestion -assert the search results page has loaded -``` - -### Scenario: Page Redirect After Action - -After login, the app redirects to the dashboard. - -``` -type "admin@example.com" in the email field -type "password123" in the password field -click "Sign In" -wait for 5 seconds -assert the current URL contains "/dashboard" -assert "Welcome, Admin" is visible -``` - -### Scenario: Lazy-Loaded Images - -Product pages often load images lazily as you scroll. - -``` -scroll down 3 times -wait for 3 seconds -assert the product image for "Winter Jacket" is visible -``` - -### Scenario: Tab Content That Loads on Click - -Tabs that load their content via AJAX when selected: - -``` -click on the "Reviews" tab -wait for 3 seconds -assert customer reviews are visible -assert "5 out of 5 stars" is visible -``` - -### Scenario: Conditional UI Based on API Response - -A dashboard that shows different content based on the user's subscription: - -``` -go to https://app.example.com/dashboard -wait for 5 seconds -if "Upgrade to Pro" button is visible then click on it -``` - -## Handling Popups & Dialogs - -### Browser Alert Dialogs - -KaneAI handles native browser alerts (`alert()`, `confirm()`, `prompt()`) automatically in most cases. - -### Cookie Consent Banners - -``` -if "Accept Cookies" button is visible then click on it -``` - -### Permission Dialogs - -For notification or location permission popups, handle them before they appear using Chrome options or custom headers. See [Chrome Options](/support/docs/kaneai-chrome-options). - -### Dismissing Popups Without Close Buttons (Mobile) - -On mobile apps, use: -``` -dismiss dialog -``` - -This works for popups that don't have visible close buttons. - -## Network-Dependent Content - -### Scenario: Slow API Response - -When your app fetches data from a slow API: - -``` -click on "Generate Report" -wait for 15 seconds -assert "Report ready for download" is visible -``` - -**Or with custom timeout:** Set the assertion step timeout to 30 seconds instead of using explicit wait. - -### Testing Under Poor Network Conditions - -Use KaneAI's [Network Throttling](/support/docs/kaneai-network-throttling) to simulate slow connections: - -1. Before starting the session, set network profile to **3G** or **Custom** -2. Add longer waits to account for slower load times -3. Validate that the app handles slow networks gracefully - -``` --- With 3G throttling enabled -- -go to https://app.example.com -wait for 10 seconds -assert the page content has loaded -assert no "Network Error" message is visible -``` - -## Patterns for Robust Test Flows - -### Pattern: Action → Wait → Assert - -The most reliable pattern for any action that triggers asynchronous behavior: - -``` --- Action -- -click on "Submit" button - --- Wait -- -wait for 3 seconds - --- Assert -- -assert "Success" message is visible -``` - -### Pattern: Scroll → Wait → Interact - -For content below the fold: - -``` -scroll down until "Contact Us" section is visible -wait for 2 seconds -type "feedback@example.com" in the email field -``` - -### Pattern: Conditional Wait for Dynamic Content - -When you're unsure if a popup or overlay will appear: - -``` -if "Close" button is visible then click on "Close" button -wait for 2 seconds -click on "Continue" button -``` - -## Common Mistakes and How to Avoid Them - -| Mistake | What happens | Fix | -|---|---|---| -| No wait after page navigation | Assertion runs on old page, fails or passes incorrectly | Add `wait for 3-5 seconds` after navigation | -| Asserting toast after long wait | Toast has already disappeared | Assert immediately after the triggering action via JS snippet | -| Using explicit wait when custom timeout is better | Test always waits full duration even if element appears early | Use custom step timeout instead | -| Asserting dynamic count without waiting for load | Count is 0 because data hasn't loaded yet | Wait for data to load, then assert count | -| Not handling conditional popups | Test fails when an unexpected cookie banner or dialog appears | Use `if` conditional to dismiss popups | diff --git a/static/docs/kaneai-kb-finding-and-interacting-with-elements.md b/static/docs/kaneai-kb-finding-and-interacting-with-elements.md deleted file mode 100644 index 8b2d84ad9..000000000 --- a/static/docs/kaneai-kb-finding-and-interacting-with-elements.md +++ /dev/null @@ -1,236 +0,0 @@ -# Finding & Interacting with Elements - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -KaneAI converts natural language instructions into automation actions by identifying the right element on the page. The more precisely you describe an element, the more reliably KaneAI will find and interact with it. This guide covers practical techniques for targeting elements in real-world applications. - -## How KaneAI Finds Elements - -KaneAI uses a combination of signals to locate elements: - -- **Text content**: visible labels, button text, placeholder text -- **Element type**: button, link, input field, dropdown, checkbox -- **Position on page**: top, bottom, left sidebar, inside a specific section -- **Visual appearance**: color, size, icons -- **Accessibility attributes**: aria-labels, roles, alt text - -The best instructions combine **what the element is** (type) with **how to identify it** (text or position). - -## Describing Elements Clearly - -### Use the Element Type - -Always mention what kind of element you are targeting. This eliminates ambiguity when multiple elements share the same text. - -| Instruction | Why it works | -|---|---| -| `click on the "Login" button` | Specifies both the text and element type | -| `click on the "Pricing" link in the navigation bar` | Distinguishes the nav link from other "Pricing" text on the page | -| `type "john@example.com" in the email input field` | Targets the input specifically, not a label | - -Think of it like giving directions to a person: "Click the blue **button** that says Submit" is clearer than "Click Submit." - -### Use Quotes for Exact Text Matching - -Wrap element text in quotes when you need an exact match. This prevents KaneAI from matching similar but incorrect elements. - -| Scenario | Instruction | -|---|---| -| Page has both "Sign In" and "Sign Up" buttons | `click on the "Sign In" button` | -| Input field has placeholder "Enter your email" | `type "test@example.com" in the "Enter your email" field` | -| Multiple links with similar names | `click on the "View Details" link` | - -### Use Positional Cues to Disambiguate - -When multiple elements look the same (e.g., a table with "Edit" buttons on every row, or a page with multiple "Submit" buttons), use position or context to narrow down the target. - -**Absolute positions:** -``` -click on the "Submit" button at the bottom of the page -click on the search icon in the top right corner -``` - -**Relative positions (near another element):** -``` -click the "Edit" button next to "john@example.com" -click the "Delete" icon beside the "Quarterly Report" row -``` - -**Ordering (nth element):** -``` -click on the second "Add to Cart" button -select the third item in the dropdown -``` - -**Inside a container:** -``` -click "Submit" inside the payment form -click "Save" inside the modal dialog -``` - -## Real-World Scenarios - -### Scenario: E-Commerce Product Listing - -Your page shows a grid of products, each with an "Add to Cart" button. - -**Bad approach** -`click on Add to Cart`. KaneAI won't know which product you mean. - -**Good approaches** -``` -click the "Add to Cart" button below the "Running Shoes" product -click the "Add to Cart" button for the first product in the list -``` - -### Scenario: Data Table with Repeated Actions - -A user management table has Edit, Delete, and View buttons for each row. - -``` -click the "Edit" button in the row containing "jane.doe@company.com" -click "Delete" next to user "John Smith" -click the "View" link in the third row of the users table -``` - -### Scenario: Navigation Menu with Nested Items - -A multi-level navigation has "Settings" under both "Account" and "Admin" sections. - -``` -hover on the "Account" menu, then click on "Settings" -click on "Settings" inside the "Admin" dropdown -``` - -### Scenario: Modal/Dialog Interactions - -A page has a confirmation modal with "Cancel" and "Confirm" buttons, but the background page also has buttons. - -``` -click "Confirm" inside the confirmation dialog -click the "Cancel" button in the modal -``` - -## Scrolling to Find Elements - -If an element is not visible in the current viewport, KaneAI can scroll to find it. - -| Action | Instruction | -|---|---| -| Scroll until element appears | `scroll until the "Load More" button is visible` | -| Scroll by specific amount | `scroll down by 500 pixels` | -| Scroll by percentage | `scroll down by 30 percent` | -| Scroll to page boundaries | `scroll to the bottom of the page` | -| Scroll multiple times (infinite scroll) | `scroll down 5 times` | -| Scroll within a container | `scroll down inside the chat messages panel` | - -`scroll until` works only for interactable elements. For non-interactable elements, use `scroll down X times` combined with a wait step. - -`scroll until` is currently supported on **Web only**. For mobile app, use `scroll down X times` combined with a wait step. - -## Tab & Window Management - -Many real-world flows open new tabs (e.g., clicking "Terms & Conditions" or a payment gateway). - -**Key rule:** When a new tab opens, you must explicitly switch to it before interacting with elements on that tab. - -| Action | Instruction | -|---|---| -| Open URL in new tab | `open https://example.com in a new tab` | -| Switch to tab by name | `switch to the "Payment Gateway" tab` | -| Switch to tab by position | `switch to the 2nd tab` | -| Close a tab | `close the "Terms" tab` | -| Return to original tab | `switch to the 1st tab` | - -Window management is currently not supported on KaneAI. - -### Scenario: Checkout Flow That Opens Payment in New Tab - -``` -click on "Proceed to Payment" -switch to the 2nd tab -type "4111111111111111" in the card number field -click "Pay Now" -switch to the 1st tab -assert "Payment Successful" is visible -``` - -Always add a `switch to tab` instruction after any action that opens a new tab. Without it, subsequent instructions will execute on the wrong tab and fail. - -## Hover & Multi-Step Interactions - -Some UI elements only appear after hovering (e.g., dropdown menus, tooltips, action icons on table rows). - -``` -hover on the "Products" menu item -click on "Electronics" in the dropdown -``` - -``` -hover on the user avatar in the top right corner -click on "Sign Out" -``` - -### Scenario: Tooltip Verification - -``` -hover on the info icon next to "Annual Revenue" -assert tooltip text "Total revenue for the fiscal year" is visible -``` - -## Drag and Drop - -KaneAI supports drag-and-drop interactions on **Desktop Web, Android apps, iOS apps, and Mobile Web**, for sortable lists, kanban boards, sliders, payment confirmation gestures, and similar UI patterns. - -``` -drag "Task: Fix Bug" and drop it on the "In Progress" column -drag the first item and drop it below the third item in the list -drag "Card A" to "Column B" -``` - -Use **Manual Interaction** to capture sliders, slide-to-confirm gestures, and other dynamic-target drags that NL cannot resolve reliably. Manual recording is available on Desktop Web, Android, and iOS. Mobile Web is **NL-only**. - -See the [Drag and Drop](/support/docs/kane-ai-drag-drop/) guide for the full platform matrix, gesture classification rules, and replay behavior. - -## When Natural Language Isn't Enough - -For complex or highly dynamic pages where natural language targeting is unreliable, KaneAI supports **JS Snippets** as a workaround. Common cases include: - -- Elements with no visible text or distinguishing attributes -- Canvas-based interactions -- Custom web components with non-standard rendering - -Use the `/` command and select **"Add JS Snippet"** to execute JavaScript directly. For example, to click a deeply nested shadow DOM element: - -```javascript -document.querySelector('my-component').shadowRoot.querySelector('.hidden-btn').click(); -``` - -See the [JS Snippets & Workarounds](/support/docs/kaneai-kb-js-snippets-and-workarounds) guide for a comprehensive list of scenarios where JS scripting is the recommended approach. - -## Quick Reference: Element Interaction Commands - -| Action | Example Instructions | -|---|---| -| **Click** | `click on the "Submit" button` | -| **Double / N-Click** | `double click the title` , `click the button 5 times`, see [Click Interactions](/support/docs/kane-ai-click-interactions/) | -| **Right Click** | `right click on the file row` (web only), see [Click Interactions](/support/docs/kane-ai-click-interactions/) | -| **Long Press** | `long press the menu icon for 3 seconds`, see [Click Interactions](/support/docs/kane-ai-click-interactions/) | -| **Type** | `type "hello world" in the search input` | -| **Clear** | `clear the email input field` | -| **Hover** | `hover on the "Products" menu` | -| **Select (dropdown)** | `select "United States" from the country dropdown` | -| **Select (date)** | `select 21st September date` | -| **Check/Uncheck** | `check the "I agree" checkbox` | -| **Drag & Drop** | `drag "Card A" and drop it on "Column B"` | - -## Do's and Don'ts - -| Do | Don't | -|---|---| -| `click the "Save" button inside the settings modal` | `click Save` (which Save button?) | -| `type "admin" in the "Username" input field` | `type admin` (which field?) | -| `click the "Edit" button next to "john@example.com"` | `click the Edit button` (which row?) | -| `hover on "Account" menu, then click "Profile"` | `go to my profile` (too vague for nested nav) | -| `scroll until the "Footer" section is visible` | `scroll to the bottom` (if you need a specific section, name it) | -| `switch to the "Checkout" tab` | `continue on the new tab` (KaneAI needs explicit tab switch) | diff --git a/static/docs/kaneai-kb-forms-inputs-and-data-entry.md b/static/docs/kaneai-kb-forms-inputs-and-data-entry.md deleted file mode 100644 index 98731d05e..000000000 --- a/static/docs/kaneai-kb-forms-inputs-and-data-entry.md +++ /dev/null @@ -1,291 +0,0 @@ -# Forms, Inputs & Data Entry - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Forms are at the heart of most web applications: login pages, registration flows, checkout processes, search filters, and admin panels. This guide covers how to handle every type of form element in KaneAI with real-world testing scenarios. - -## Text Input Fields - -### Basic Text Entry - -Use `type`, `enter` or `input` followed by the value and the target field. - -``` -type "john.doe@example.com" in the email input field -enter "SecurePass123!" in the password field -type "123 Main Street" in the address field -``` - -### Using Variables & Parameters for Dynamic Data - -Instead of hardcoding values, use parameters for reusable and parameterized tests. - -``` -create a parameter using /(slash) command called username -type ${username} in the username field -``` - -Parameters once used in authoring can be used to input different data values via dataset. See the [Parameters](/support/docs/kane-ai-using-parameters/) guide for more information. - -For test data that needs to be unique each run, use **smart variables**: - -``` -type {{smart.random_email}} in the email field -type {{smart.random_phone}} in the phone number field -``` - -Smart variables like `{{smart.random_email}}` generate unique values every run, which is ideal for registration flows where duplicate emails would fail. See the [Smart Variables](/support/docs/kane-ai-smart-variables/) guide for the full list. - -## Dropdown & Select Menus - -### Standard HTML Select Dropdowns - -``` -select "United States" from the country dropdown -select "California" from the state dropdown -``` - -### Custom Dropdowns (Non-Native) - -Many modern applications use custom-built dropdowns that aren't standard HTML `` elements. For these, use a click-then-select pattern: - -``` -click on the "Country" dropdown -click on "United States" in the dropdown list -``` - -If the dropdown requires typing to filter: - -``` -click on the "Country" dropdown -type "United" in the search field inside the dropdown -click on "United States" in the filtered results -``` - -### Multi-Select Dropdowns - -``` -click on the "Tags" dropdown -click on "Urgent" in the dropdown list -click on "Bug" in the dropdown list -``` - -### Scenario: Cascading Dropdowns (Country → State → City) - -A common pattern where the second dropdown options depend on the first. - -``` -select "India" from the country dropdown -wait for 2 seconds -select "Maharashtra" from the state dropdown -wait for 2 seconds -select "Mumbai" from the city dropdown -``` - -Always add a `wait` between cascading dropdowns. The child dropdown needs time to populate after the parent selection changes. - -## Date Pickers - -### Simple Date Pickers - -``` -select 15th March 2026 date -select tomorrow's date -``` - -### Calendar Widget Date Pickers - -For custom calendar components, navigate month by month: - -``` -click on the date input field -click the "Next Month" arrow 3 times -click on day "22" -``` - -### Date Range Pickers - -``` -click on the "Start Date" field -select 1st February 2026 date -click on the "End Date" field -select 28th February 2026 date -``` - -### Using Smart Variables for Dynamic Dates - -``` -type {{smart.current_date}} in the "Start Date" field -type {{smart.end_of_month}} in the "End Date" field -``` - -**JS Workaround** -If the date picker uses a read-only input that doesn't accept typed values, use a JS snippet to set the value directly: - -```javascript -document.querySelector('input[name="startDate"]').value = '2026-03-15'; -document.querySelector('input[name="startDate"]').dispatchEvent(new Event('change', { bubbles: true })); -``` - -Use `/` → **Add JS Snippet** to execute this. - -## Checkboxes & Radio Buttons - -### Checkboxes - -``` -check the "I agree to Terms and Conditions" checkbox -uncheck the "Subscribe to newsletter" checkbox -``` - -### Radio Buttons - -``` -click on the "Express Shipping" radio button -select the "Monthly" billing option -``` - -### Scenario: Preferences Form with Multiple Checkboxes - -``` -check the "Email Notifications" checkbox -check the "SMS Alerts" checkbox -uncheck the "Marketing Communications" checkbox -click the "Save Preferences" button -assert "Preferences saved successfully" is visible -``` - -## File Uploads - -KaneAI supports uploading files during test authoring. Use the `/` command to access the file upload option. - -### Steps to Upload a File - -1. Type `/` in the input field -2. Select **Upload File** -3. Choose the file from your system -4. The file gets uploaded and can be used in the test step as a variable. - -``` -upload file {{FILE_sample_report_pd}} to the file input -``` - -For detailed instructions, refer to [File Upload & Download Testing](/support/docs/kaneai-upload-and-download-files). - -## OTP & PIN Fields - -Applications that use separate input boxes for each digit (e.g., 4-digit or 6-digit OTP fields) require the `OTP` keyword. - -``` -enter OTP "1234" -enter OTP "567890" -``` - -Do **not** try to type into each OTP box individually (e.g., `type "1" in first box, type "2" in second box`). Use the `OTP` keyword as a single instruction. KaneAI handles the per-box distribution automatically. - -### Scenario: Login with OTP Verification - -``` -type "john@example.com" in the email field -click on "Send OTP" button -wait for 5 seconds -enter OTP "123456" -click on "Verify" button -assert "Welcome, John" is visible -``` - -For real TOTP-based MFA, see the [TOTP Authentication](/support/docs/kaneai-totp) guide. - -## Sliders & Range Inputs - -### Web Sliders - -For range sliders on web, describe the target value: - -``` -set the price slider to 500 -drag the volume slider to 75 percent -``` - -If the slider is not responsive to natural language, use a **JS snippet**: - -```javascript -const slider = document.querySelector('input[type="range"]'); -slider.value = 75; -slider.dispatchEvent(new Event('input', { bubbles: true })); -``` - -### Mobile Sliders & Pickers - -On mobile, specific element classes are supported: -- **Android**: `android.widget.SeekBar`, `android.widget.NumberPicker` -- **iOS**: `XCUIElementTypeSlider`, `XCUIElementTypePicker`, `XCUIElementTypePickerWheel` - -``` -enter "5" in the first column -drag the slider to 80 percent -``` - -## Rich Text Editors - -For applications with WYSIWYG editors (TinyMCE, CKEditor, Quill, etc.), the editor content is typically inside an iframe or contenteditable div. - -### Using Natural Language - -``` -click inside the text editor -type "This is a test paragraph for the blog post." -``` - -### If the Editor is Inside an iFrame - -KaneAI handles iFrames automatically in most cases. If not, use a JS snippet: - -```javascript -document.querySelector('iframe.editor').contentDocument.body.innerHTML = '

Test content

'; -``` - -## Search Fields with Autocomplete - -Many search fields show suggestions as you type. Handle these by typing, waiting for suggestions, then selecting. - -``` -type "New Yo" in the search field -wait for 2 seconds -click on "New York, NY" in the suggestions dropdown -``` - -### Scenario: Address Autocomplete (Google Places) - -``` -type "221B Baker" in the address field -wait for 3 seconds -click on "221B Baker Street, London" in the autocomplete suggestions -assert the city field contains "London" -``` - -## Keyboard Interactions in Forms - -| Action | Instruction | -|---|---| -| Submit a form via Enter key | `type "search query" in the search field and press Enter` | -| Tab between fields | `press Tab` | - -### Hiding the Keyboard (Mobile) - -- **Android**: `hide keyboard` -- **iOS**: `press Enter` after typing (preferred), or `click on the "Done" button on the keyboard` - -``` -type "test@example.com" in the email field and press Enter -``` - -## Do's and Don'ts for Form Testing - -| Do | Don't | -|---|---| -| `type "admin" in the "Username" input field` | `type admin` (which field?) | -| `wait for 2 seconds` between cascading dropdowns | Immediately select child dropdown after parent change | -| `enter OTP "1234"` for OTP/PIN fields | `type "1" in the first OTP box` | -| Use `{{smart.random_email}}` for unique registration data | Hardcode `test@test.com` which may already exist | -| Use JS snippets for stubborn date pickers | Fight with natural language for read-only date inputs | diff --git a/static/docs/kaneai-kb-js-snippets-and-workarounds.md b/static/docs/kaneai-kb-js-snippets-and-workarounds.md deleted file mode 100644 index 7c3ff7485..000000000 --- a/static/docs/kaneai-kb-js-snippets-and-workarounds.md +++ /dev/null @@ -1,222 +0,0 @@ -# JS Snippets & Workarounds - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -KaneAI's natural language engine handles the vast majority of web testing scenarios. However, some interactions require direct DOM manipulation or precise programmatic control that natural language can't reliably express. For these cases, KaneAI provides **JS Snippets**: the ability to execute JavaScript code directly on the page. - -This guide is organized by common QA scenario, showing you **when** natural language falls short and **exactly what JS snippet** to use as a workaround. - -## How to Add a JS Snippet - -1. In the KaneAI input field, type `/` or click the `+` icon -2. Select **Add JS Snippet** -3. Write or paste your JavaScript code -4. Click **Add** to execute it as a test step - -The snippet runs in the context of the page and has full access to the DOM, `document`, `window`, and any JavaScript variables the page has defined. - -## Date Picker Interactions - -**Problem:** Many date pickers use read-only inputs that reject typed values. Natural language typing doesn't trigger the custom event handlers these components rely on. - -### Set Date on a Read-Only Date Picker - -```javascript -const dateInput = document.querySelector('input[name="startDate"]'); -// Remove readonly temporarily -dateInput.removeAttribute('readonly'); -// Set the value -dateInput.value = '2026-03-15'; -// Trigger change events so the framework (React, Angular, Vue) picks up the change -dateInput.dispatchEvent(new Event('input', { bubbles: true })); -dateInput.dispatchEvent(new Event('change', { bubbles: true })); -return 'Date set to 2026-03-15'; -``` - -### Set Date on React DatePicker - -React components use synthetic events. Use the native input setter: - -```javascript -const dateInput = document.querySelector('input[name="date"]'); -const nativeInputValueSetter = Object.getOwnPropertyDescriptor( -window.HTMLInputElement.prototype, 'value' -).set; -nativeInputValueSetter.call(dateInput, '2026-03-15'); -dateInput.dispatchEvent(new Event('input', { bubbles: true })); -return 'React date picker value set'; -``` - -### Set Date Range Picker - -```javascript -const startDate = document.querySelector('input[name="start"]'); -const endDate = document.querySelector('input[name="end"]'); - -const setter = Object.getOwnPropertyDescriptor( -window.HTMLInputElement.prototype, 'value' -).set; - -setter.call(startDate, '2026-02-01'); -startDate.dispatchEvent(new Event('input', { bubbles: true })); - -setter.call(endDate, '2026-02-28'); -endDate.dispatchEvent(new Event('input', { bubbles: true })); - -return 'Date range set: Feb 1 - Feb 28, 2026'; -``` - -## String Manipulation & Data Transformation - -**Problem:** Operations like converting text to lowercase, extracting substrings, or formatting data are not reliably handled via natural language. - -### Convert Text to Lowercase and Validate - -```javascript -const title = document.querySelector('h1.page-title').textContent; -const lowercase = title.toLowerCase(); -if (lowercase === 'welcome to dashboard') { -return 'PASS: Title in lowercase matches expected value'; -} else { -throw new Error('FAIL: Got "' + lowercase + '"'); -} -``` - -### Extract and Validate a Substring - -```javascript -const orderText = document.querySelector('.order-id').textContent; -// Text is "Order #12345", extract just the number -const orderId = orderText.replace('Order #', ''); -if (orderId.length === 5 && !isNaN(orderId)) { -return 'PASS: Order ID is valid: ' + orderId; -} else { -throw new Error('FAIL: Invalid order ID format: ' + orderId); -} -``` - -### Trim Whitespace and Compare - -```javascript -const actual = document.querySelector('.username').textContent.trim(); -const expected = 'john_doe'; -if (actual === expected) { -return 'PASS: Username matches'; -} else { -throw new Error('FAIL: Expected "' + expected + '", got "' + actual + '"'); -} -``` - -## Table & List Validation - -**Problem:** KaneAI can't reliably assert specific cell values in complex tables (e.g., "the 5th column of the 3rd row contains X"). - -### Assert Specific Table Cell Value - -```javascript -const rows = document.querySelectorAll('table.users-table tbody tr'); -const thirdRowFifthCol = rows[2].querySelectorAll('td')[4].textContent.trim(); -if (thirdRowFifthCol === 'Active') { -return 'PASS: 3rd row, 5th column is "Active"'; -} else { -throw new Error('FAIL: Got "' + thirdRowFifthCol + '"'); -} -``` - -### Count Table Rows - -```javascript -const rowCount = document.querySelectorAll('table tbody tr').length; -if (rowCount === 10) { -return 'PASS: Table has 10 rows'; -} else { -throw new Error('FAIL: Table has ' + rowCount + ' rows, expected 10'); -} -``` - -### Validate Table is Sorted - -```javascript -const cells = Array.from(document.querySelectorAll('table tbody tr td:first-child')); -const values = cells.map(cell => cell.textContent.trim()); -const sorted = [...values].sort((a, b) => a.localeCompare(b)); -const isSorted = JSON.stringify(values) === JSON.stringify(sorted); -if (isSorted) { -return 'PASS: Table is sorted alphabetically by first column'; -} else { -throw new Error('FAIL: Table is not sorted. Got: ' + values.join(', ')); -} -``` - -## Clipboard Operations - -**Problem:** Natural language can't directly read or write clipboard content. - -### Copy Text to Clipboard - -```javascript -const text = document.querySelector('.referral-code').textContent; -await navigator.clipboard.writeText(text); -return 'Copied to clipboard: ' + text; -``` - -### Read Clipboard Content - -```javascript -const clipText = await navigator.clipboard.readText(); -return 'Clipboard contains: ' + clipText; -``` - -## Local Storage & Session Storage - -**Problem:** You may need to verify, set, or clear browser storage for test setup or validation. - -### Read a Local Storage Value - -```javascript -const token = localStorage.getItem('authToken'); -if (token) { -return 'Auth token exists: ' + token.substring(0, 20) + '...'; -} else { -throw new Error('FAIL: No auth token in localStorage'); -} -``` - -### Set a Local Storage Value (Test Setup) - -```javascript -localStorage.setItem('feature_flag_newUI', 'true'); -return 'Feature flag set'; -``` - -### Clear Session Storage (Logout Simulation) - -```javascript -sessionStorage.clear(); -return 'Session storage cleared'; -``` - -## Quick Reference: When to Use JS Snippets - -| Scenario | Natural Language? | JS Snippet Needed? | -|---|---|---| -| Click a visible button | Yes | No | -| Type in a standard input | Yes | No | -| Assert text is visible | Yes | No | -| Set date on read-only date picker | Unreliable | **Yes** | -| Validate specific table cell | No | **Yes** | -| String manipulation (lowercase, trim) | Unreliable | **Yes** | -| Read/write localStorage | No | **Yes** | -| Clipboard operations | No | **Yes** | -| Check if page is fully loaded | No | **Yes** | -| Count elements on page | Unreliable for exact counts | **Yes** | - -## Best Practices for JS Snippets - -| Practice | Details | -|---|---| -| **Use natural language first** | Only fall back to JS when natural language is unreliable | -| **Keep snippets focused** | One snippet = one action or assertion | -| **Use descriptive return values** | Return "PASS: ..." or throw "FAIL: ..." for clear test output | -| **Null-check elements** | Always check if `querySelector` returns `null` before operating on it | -| **Test snippets in browser DevTools first** | Paste into the Console to verify before adding to KaneAI | diff --git a/static/docs/kaneai-kb-knowledge-base-index.md b/static/docs/kaneai-kb-knowledge-base-index.md deleted file mode 100644 index 55bf9f08a..000000000 --- a/static/docs/kaneai-kb-knowledge-base-index.md +++ /dev/null @@ -1,91 +0,0 @@ -# KaneAI Knowledge Base - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - - const getBrandName = () => 'TestMu AI'; - const getBrandNameLowercase = () => 'testmu ai'; - if (type === 'lowercase') { - return getBrandNameLowercase(); - } - return getBrandName(); -}; - -The KaneAI Knowledge Base is your go-to resource for writing effective test cases, handling complex testing scenarios, and getting the most out of KaneAI's natural language automation. Whether you're authoring your first test or working around advanced edge cases, these guides provide practical, scenario-driven instructions with real-world examples. - -## Authoring Guides - -Start here to learn how to write reliable test instructions for any scenario, from finding elements on the page to handling complex form interactions. - - Element Interactions - Target elements precisely using text, type, position, and context cues. Covers scrolling, tab management, hover, and drag-and-drop. - - Forms & Inputs - Handle text inputs, dropdowns, date pickers, checkboxes, file uploads, OTP fields, sliders, autocomplete, and multi-step form wizards. - - Assertions Guide - Write reliable assertions for text, visual elements, URLs, layout, and math. Includes do's and don'ts, variable-based assertions, and failure configuration. - - Waits & Timing - Handle loading spinners, toast notifications, modals, infinite scroll, lazy loading, and async content with the right wait strategy. - - Mobile Patterns - Android & iOS specific patterns: keyboard handling, OTP fields, native pickers, gestures, deep links, permission dialogs, and spannable text. - -## Advanced Testing - -Guides for complex testing scenarios involving authentication, API validation, and workarounds for natural language limitations. - - Authentication & Sessions - Test login flows, TOTP/MFA with native smart variables, session persistence, logout, and timeout handling. Includes reusable login modules. - - API & Network - Make API calls via curl, validate responses with JSON variables, assert network logs, and combine API + UI testing in a single flow. - - JS Workarounds - When natural language isn't enough: JS snippets for date pickers, CSS validation, string manipulation, table cells, and more. - - Failure Conditions - Configure how assertion failures behave: fail immediately, fail and continue, or warn and continue. Set defaults at organization or step level. - -## Variables and Parameters - -Manage dynamic data, secrets, and parameterized test inputs. - - Variables - Create and use local, global, and environment variables. Store values from the application and reuse them across test steps and executions. - - Secrets - Securely stored sensitive data (e.g., passwords, API tokens) encrypted via HashiCorp Vault. Never exposed in logs or generated code. - - Smart Variables - Predefined dynamic variables for dates, randomization, device info, and system details. Use `{{smart.current_date}}`, `{{smart.random_email}}`, and more. - - Parameters - Values passed into test cases at runtime to customize test execution for different configurations or environments. - - Datasets - Collections of test data used for data-driven testing, allowing you to run the same test with different inputs. - -## Reference - - Command Types - Complete reference of all supported KaneAI commands: navigation, clicks, typing, waits, tab management, scrolling, assertions, conditional logic, and queries. - - Code Generation - Generate automation scripts from KaneAI tests in multiple frameworks and languages including Selenium (Python), Appium (Python), and more. - - Error Handling - Understand error messages in KaneAI, what they mean, and how to resolve common issues during test authoring and execution. - -## Quick Start Guide - -New to KaneAI? Here's the recommended reading order: - -1. **[Author Your First Desktop Browser Test](/support/docs/author-your-first-desktop-browser-test/)**: Get started by authoring your first desktop browser test -2. **[Author Your First Mobile Browser Test](/support/docs/author-your-first-mobile-browser-test/)**: Learn to author mobile browser tests with device selection and configuration -3. **[Author Your First Mobile App Test](/support/docs/author-your-first-mobile-app-test/)**: Build your first mobile app test with app upload and device configuration -4. **[Element Interactions](/support/docs/kaneai-kb-finding-and-interacting-with-elements/)**: Learn how to describe elements so KaneAI can find them reliably -5. **[Forms & Inputs](/support/docs/kaneai-kb-forms-inputs-and-data-entry/)**: Handle every type of form element -6. **[Assertions Guide](/support/docs/kaneai-kb-assertions-and-validation/)**: Verify your app works correctly -7. **[Waits & Timing](/support/docs/kaneai-kb-dynamic-content-waits-and-page-state/)**: Handle async behavior and timing -8. **[JS Workarounds](/support/docs/kaneai-kb-js-snippets-and-workarounds/)**: When natural language isn't enough diff --git a/static/docs/kaneai-kb-mobile-app-testing-patterns.md b/static/docs/kaneai-kb-mobile-app-testing-patterns.md deleted file mode 100644 index 44238bd10..000000000 --- a/static/docs/kaneai-kb-mobile-app-testing-patterns.md +++ /dev/null @@ -1,353 +0,0 @@ -# Mobile App Testing Patterns - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Mobile native app testing in KaneAI follows many of the same principles as web testing, but with platform-specific nuances for Android and iOS. This guide covers the patterns, workarounds, and best practices that will help you write reliable mobile app tests. - -## Getting Started with Mobile Tests - -1. Navigate to the [KaneAI home page](https://www.testmuai.com/login/?redirectTo=https://kaneai.lambdatest.com/objective) -2. Choose **App Test** (Android or iOS) -3. Select a real device from the device cloud -4. Upload your app (APK for Android, IPA for iOS) or select a previously uploaded app -5. Start authoring with natural language instructions - -## Keyboard Handling - -The keyboard is one of the biggest differences between web and mobile testing. On mobile, the on-screen keyboard persists after typing and can block elements below it. - -### Android - -The keyboard typically auto-dismisses on most interactions, but you can explicitly dismiss it: - -``` -hide keyboard -``` - -### iOS - -iOS keyboards are more persistent. The best approach is to use `press Enter` after typing, which both submits the current field and dismisses the keyboard: - -``` -type "john@example.com" in the email field and press Enter -``` - -Alternatively, if the keyboard has a "Done" or "Cancel" button: - -``` -click on "Done" button on the keyboard -click on "Cancel" button on the keyboard -``` - -If you skip keyboard dismissal on iOS, the keyboard may cover the next element you need to interact with, causing the step to fail. Always dismiss the keyboard before moving to the next field. - -### Scenario: Login Form on iOS - -``` -type "admin@example.com" in the email field and press Enter -type "password123" in the password field and press Enter -click on the "Sign In" button -wait for 5 seconds -assert "Welcome" is visible -``` - -## OTP & PIN Fields - -Mobile apps frequently use individual digit boxes for OTP entry. Use the `OTP` keyword. KaneAI automatically distributes digits across the boxes. - -``` -enter OTP "123456" -enter OTP "7890" -``` - -Do **not** type into each box individually. The `OTP` keyword handles multi-box distribution. - -### Scenario: SMS OTP Verification - -``` -type "+1-555-0123" in the phone number field and press Enter -click on "Send Code" -wait for 10 seconds -enter OTP "654321" -click on "Verify" -assert "Phone verified" is visible -``` - -## Scrolling on Mobile - -Mobile scrolling works differently from web. There's no pixel-based scroll. Use count-based or directional scrolling. - -``` -scroll down 3 times -scroll up 2 times -scroll down 1 time -``` - -### The Scroll → Wait Pattern - -Always add a wait after scrolling on mobile. Content may need time to load, and the scroll animation needs to complete. - -``` -scroll down 2 times -wait for 2 seconds -assert "Contact Us" section is visible -``` - -### Scenario: Scrolling Through a Long Settings Page - -``` -scroll down 3 times -wait for 2 seconds -click on "Privacy Settings" -scroll down 2 times -wait for 2 seconds -click on "Delete Account" -assert confirmation dialog is visible -``` - -`scroll until element is visible` is currently supported only on **Web**. On mobile, use `scroll down X times` combined with waits and assertions. - -## Pickers, Wheels & Sliders - -### Native Pickers - -Mobile apps use native picker wheels for dates, numbers, and selections. KaneAI supports the following element classes: - -**Android:** -- `android.widget.NumberPicker` -- `android.widget.SeekBar` - -**iOS:** -- `XCUIElementTypePickerWheel` -- `XCUIElementTypePicker` -- `XCUIElementTypeDatePicker` -- `XCUIElementTypeSlider` - -### Interacting with Picker Wheels - -``` -enter "5" in the first column -enter "March" in the second column -enter "2026" in the third column -``` - -### Slider Interactions - -``` -drag the slider to 80 percent -set the brightness slider to 50 -``` - -### Troubleshooting Pickers - -If a picker doesn't respond to natural language: - -1. Open LambdaTest **App Live** with the [UI Inspector](https://www.testmuai.com/support/docs/ui-inspector/) -2. Inspect the picker element to verify its class name -3. Confirm the class is one of the supported types listed above -4. If the class is a custom implementation, you may need to use **manual interaction** mode - -## Partially Clickable Text (Spannable Text) - -In mobile apps, text labels often contain partially clickable links, for example, "By signing up, you agree to our **Terms and Conditions** and **Privacy Policy**" where only "Terms and Conditions" is tappable. - -Use the `spannable text` keyword: - -``` -click on spannable text: "Terms and Conditions" in the bottom info section -click on spannable text: "Privacy Policy" in the sign-up footer -``` - -## Dismissing Popups & Dialogs - -Mobile apps frequently show permission dialogs, promotional popups, or system alerts that don't have visible close buttons. - -### Generic Dialog Dismissal - -``` -dismiss dialog -``` - -### Permission Dialogs - -``` -if "Allow" button is visible then click on "Allow" -if "Don't Allow" button is visible then click on "Don't Allow" -``` - -### App Rating Dialogs - -``` -if "Rate this app" dialog is visible then dismiss dialog -``` - -### Scenario: Handling Permission Prompts on App Launch - -``` --- App launches and asks for location permission -- -wait for 3 seconds -if "Allow While Using App" is visible then click on "Allow While Using App" - --- App asks for notification permission -- -wait for 2 seconds -if "Allow Notifications" is visible then click on "Allow Notifications" - --- Now proceed with the test -- -assert the home screen is visible -``` - -## Typing & Special Keys - -### Deleting Characters - -``` -press Del -press Del -press Del -``` - -This deletes one character at a time. To clear an entire field, it's more efficient to: - -``` -clear the "Username" field -type "new_username" in the "Username" field -``` - -### Pressing Enter/Return - -``` -press Enter -``` - -This can trigger form submission, keyboard dismissal, or moving to the next field depending on the app's implementation. - -## Using Explicit Element Identifiers - -On mobile, adding element type keywords improves accuracy because there may be multiple elements with similar text. - -``` -click on "Continue" button -click on "Learn More" text -click on "Settings" icon -assert "Welcome" text is visible -assert the profile button's background color is blue -``` - -| Keyword | Purpose | -|---|---| -| `button` | Targets tappable buttons | -| `text` | Targets text labels/views | -| `icon` | Targets image/icon elements | -| `input field` | Targets text inputs | -| `checkbox` | Targets checkable elements | -| `switch` | Targets toggle switches | - -## Deep Links - -KaneAI supports deep links for navigating directly to specific screens in your app, bypassing the normal navigation flow using the slash command. - -Deep links are excellent for test setup. Jump directly to the screen you want to test instead of navigating through the entire app flow. See [Deep Link Support](/support/docs/kane-ai-deeplink-support). - -## Mobile-Specific Testing Scenarios - -### Scenario: E-Commerce App - Browse and Purchase - -``` --- Handle initial permissions -- -if "Allow Notifications" is visible then click on "Allow Notifications" -wait for 2 seconds - --- Search for product -- -click on the search icon -type "wireless earbuds" in the search field and press Enter -wait for 3 seconds - --- Browse results -- -scroll down 2 times -wait for 2 seconds -click on the first "Wireless Earbuds" product - --- Add to cart -- -scroll down 2 times -wait for 2 seconds -click on "Add to Cart" button -assert "Added to cart" toast is visible - --- Go to cart -- -click on the cart icon -assert "Wireless Earbuds" is visible in the cart -click on "Checkout" button -``` - -### Scenario: Banking App - Check Balance with Biometric - -``` --- App launches with biometric authentication enabled from advanced settings -- -wait for 5 seconds --- (Biometrics pop-up visible via LambdaTest device settings) -- -click "pass" in modal -assert the home screen is visible - --- Navigate to accounts -- -click on "Accounts" in the bottom navigation -wait for 3 seconds -assert account balance is visible - --- Check recent transactions -- -scroll down 2 times -wait for 2 seconds -assert "Recent Transactions" section is visible -``` - -### Scenario: Social Media App - Post with Image - -``` -click on the "New Post" button -click on "Choose from Gallery" -wait for 3 seconds -click on the first image in the gallery -click on "Next" -wait for 2 seconds -type "Testing with KaneAI! #automation" in the caption field and press Enter -hide keyboard -click on "Share" button -wait for 5 seconds -assert "Post shared successfully" is visible -``` - -## Network Conditions & Geolocation - -### Testing Under Different Network Conditions - -Set network throttling before or during the test session to simulate real-world conditions: - -- **Offline**: Test no-network behavior -- **2G/3G**: Test on slow connections -- **Custom**: Define specific bandwidth limits - -See [Network Throttling](/support/docs/kaneai-network-throttling) for configuration. - -### GPS Mocking (Mobile) - -Test location-dependent features by mocking GPS coordinates: - -``` --- Set location to New York -- --- (Configured in session settings before test starts) -- -assert "Stores near New York" is visible -``` - -See [Geolocation Testing](/support/docs/kane-ai-geolocation-tunnel-proxy). - -## Best Practices for Mobile App Testing - -| Practice | Details | -|---|---| -| **Always dismiss keyboard after typing (especially iOS)** | Use `press Enter` or `hide keyboard` to prevent keyboard blocking elements | -| **Add waits after scrolls** | Mobile scroll animations need time to complete; content may lazy-load | -| **Use `OTP` keyword for PIN/OTP fields** | Don't type into individual digit boxes | -| **Handle permission dialogs upfront** | Use `if...then` conditionals at the start of your test | -| **Use deep links for test setup** | Skip navigation to jump directly to the screen under test | -| **Use element type keywords** | `button`, `text`, `icon` improve targeting accuracy | -| **Use `dismiss dialog` for stubborn popups** | Works for dialogs without visible close buttons | -| **Use `spannable text` for inline links** | Essential for "Terms and Conditions" style partial links | diff --git a/static/docs/kaneai-manual-interaction-video.md b/static/docs/kaneai-manual-interaction-video.md deleted file mode 100644 index 3a96c896a..000000000 --- a/static/docs/kaneai-manual-interaction-video.md +++ /dev/null @@ -1,5 +0,0 @@ -# Manual Interaction Using KaneAI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -In this tutorial, learn how KaneAI simplifies test automation by allowing manual interaction to create prompts in natural language, analyzing test steps, generating code in various languages, and providing detailed test descriptions. diff --git a/static/docs/kaneai-manual-interaction.md b/static/docs/kaneai-manual-interaction.md deleted file mode 100644 index 940eba25c..000000000 --- a/static/docs/kaneai-manual-interaction.md +++ /dev/null @@ -1,259 +0,0 @@ -# KaneAI Manual Interaction Mode - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Overview -Manual Interaction in KaneAI lets you take direct control of the application during test authoring or while working in a playground session. Instead of relying only on natural language instructions, you can interact with the app in real time and let the agent record everything you do as structured test steps. - -This is especially useful when you need to reach a very specific UI state, handle complex flows, or deal with elements that are difficult to describe in words. You can pause the agent, perform actions like a real user, and then resume automated authoring from that exact point. - -Manual Interaction works for Web and Mobile app testing, making it easier to capture realistic user behavior across platforms. - -You can also pause recording at any time within a Manual Interaction session to navigate or set up app state without those actions being logged as test steps, then resume recording from the same point. - -### Supported platforms -Manual Interaction is available for - - - Web browser test authoring on desktop - - - Mobile app test authoring on Android - - - Mobile app test authoring on iOS - -### How Manual Interaction works -While you are in Authoring mode, you can enable Manual Interaction by clicking the Manual Interaction button. Once enabled, you can directly interact with the application under test. - -Every action you perform such as clicking, typing, scrolling, or navigating through screens is automatically captured by the agent and converted into natural language test steps. You do not need to describe these actions in natural language while Manual Interaction is active. - -When you want to switch back to writing instructions in plain language, simply click Turn off Manual Interaction. From that point onward, the agent will again follow natural language commands. - -### Pause Recording -While you are in a Manual Interaction session, you can temporarily pause recording so that your actions are not captured as test steps. This is useful when you need to log in, navigate to a specific screen, dismiss unrelated dialogs, or explore the app without cluttering the step list. The session enters a draft state where the application remains fully interactable, but no steps are added to the step list. - -**How to use Pause Recording** - -1. Open an active Authoring or playground session. - -2. With Manual Interaction enabled, click **Pause Recording** in the Manual Interaction toolbar. - -3. If a confirmation dialog appears, confirm to proceed. The dialog is shown by default and is suppressed once you select **Don't show again** on it. - -4. Interact with the application freely. Pages load, forms accept input, and navigation works as expected, but the step list stops growing. The session transitions to Draft state. - -5. Click **Start recording** when you are ready to capture steps again. - -**What to expect while paused** - -- The application is fully interactable. Only the recorder is paused. -- The step list stays exactly as it was at the moment you paused. - -### Limitations of Instructions -Manual Interaction focuses only on capturing real-time UI actions, so it has a few limitations you should be aware of. It cannot record any instruction that involves logic, validation, or reusable test components. These need to be added through natural language instructions after you finish interacting with the application. - -The following instruction types are not supported in Manual Interaction: - -**Assertions** - -You can’t record checks or validations while interacting manually. Any “verify,” “confirm,” or “assert” step must be added as a written instruction. - -**If–else logic** - -Conditional flows or branching logic aren't captured during manual actions. These must be written out in natural language if your test needs conditional behavior. - -**Modules and Reusable Blocks** - -Manual Interaction can’t insert or reference modules. You’ll need to add them separately once the manual steps are done. - -**Variables, Parameters, and Secrets** - -These structured inputs aren't detected during manual steps. You must define or call them through natural language instructions. - -**Datasets or Dynamic Data** - -Using dataset-driven values or any form of dynamic data requires a written instruction, since they can’t be recorded through interactions. - -In short, Manual Interaction is great for capturing what you do on the screen, but any logic, validation, or reusable component needs to be added through natural language instructions afterward. - -### Visual indicators in test steps -Each recorded step clearly shows how it was created - via natural language or manual interaction. An icon indicates whether the step was captured through Manual Interaction or written using natural language instructions. This helps you quickly understand and review how the test was authored. - -### Editing Manual Interaction Steps -Once a step is recorded using Manual Interaction, editing options are intentionally limited to preserve the accuracy of the recorded action. Currently, only the locator name can be edited for manual interaction steps. - -You cannot modify the action type, element behavior, or interaction flow after recording. If the recorded step does not match the intended behavior, it is recommended to delete the step and record it again using Manual Interaction, or recreate it using a natural language instruction. - -This ensures that recorded steps remain consistent with the actual UI interaction performed during the session. - -## Manual Interaction Limitations: -### Web Browser on Desktop - -- **Drag and drop on canvas based elements:** Canvas elements rely on custom rendering, which makes drag and drop actions unreliable to record accurately. - -- **Hover:** Mouse hover actions that trigger tooltips or menus cannot be recorded. - -Double click, right click, and press-and-hold (long press) **are now supported** in Manual Interaction. See [Click Interactions](/support/docs/kane-ai-click-interactions/) for capture rules and platform support. - -### Mobile Apps - -- **Two finger scroll:** Multi touch gestures are not captured during recording. - -- **Copy and paste:** Clipboard based actions are not captured during recording. - -- **Recent apps actions:** System navigation actions like opening or switching recent apps are not supported. - -- **Mobile keyboard actions such as backspace:** Special keyboard keys and actions are not recorded accurately. - -- **System level popups:** Popups controlled by the operating system, such as permission dialogs, cannot be captured. - -**Important Note on Text Input Behavior** - -Manual Interaction records all characters you attempt to type as part of the instruction, even if the application does not accept them. - -For example, if an input field allows only numeric values and you try typing alphabets, those characters will still appear in the recorded instruction, even though they do not appear in the input field. - -## Common use cases -### Use Case 1: Recording a complete workflow manually and converting it into a test -This is useful when you already know the exact flow and want KaneAI to capture it as an automated test. - -**Steps:** - -1. Open KaneAI and start a new test in Authoring mode. - -2. Launch the application under test in the browser or mobile device. - -3. Perform the workflow exactly as a real user would. For example, logging in, navigating through pages, and submitting a form. - -4. As you interact with the app, KaneAI records each action automatically as test steps. - -5. Once the workflow is complete, click Turn off Manual Interaction. - -6. Review the generated steps and add assertions using natural language where needed by pausing the test. - -7. Save the test. - -### Use Case 2: Working with dynamic or lazy loaded UI elements -Ideal for infinite scroll lists or carousels that load items on demand. - -**Steps:** - -1. Start Authoring mode and enable Manual Interaction. - -2. Scroll through the list or carousel until the desired element becomes visible. - -3. Click or interact with the loaded element. - -4. Turn off Manual Interaction after the interaction is captured. - -5. Add assertions to verify the correct item or state using natural language. - -### Use Case 3: Combining Manual Interaction with natural language instructions - -This approach gives you the best of both worlds. - -**Steps:** - -1. Start writing your test using natural language instructions. - -2. When you reach a step that is easier to perform manually, enable Manual Interaction. - -3. Perform the required actions directly on the app. - -4. Disable Manual Interaction once done. - -5. Continue the test with natural language instructions. - -6. Add assertions at appropriate steps to validate outcomes. -### Use Case 4: Click and drag to adjust UI elements -This use case applies when you need to move or adjust an element using a click and drag action, such as sliders, scrollbars, or draggable controls that are supported outside of canvas based elements. - -**Steps:** - -1. Start the test in Authoring mode. - -2. Enable Manual Interaction. - -3. Click on the element and drag it to the required position. - -4. Release the mouse once the desired state is reached. - -5. KaneAI records the click and drag action as a test step. - -6. Turn off Manual Interaction. - -7. Add an assertion in natural language to verify the final state. - -### Use Case 5: Adjusting values using sliders (Only works on web) - -This use case applies when you need to change a value using a slider, such as volume controls, range selectors, brightness settings, or price filters. - -Sliders are often easier to handle using Manual Interaction rather than describing exact movements in natural language. - -**Steps:** - -1. Start the test in Authoring mode. - -2. Click on the slider handle. - -3. Drag the slider to the required value or position. - -4. Release the slider once the desired value is set. - -5. KaneAI records the slider movement as a test step. - -6. Turn off Manual Interaction. - -7. Add an assertion in natural language to verify the final value or state. For example, verify that the selected range or value is displayed correctly. - -**Notes specific to sliders** -- Slider interactions are recorded as click and drag actions. -- Exact pixel-level precision may vary depending on the UI implementation. -- Always validate the outcome using an assertion rather than relying only on the recorded movement. - -### Use Case 6: Reaching a setup state without recording it -This use case applies when you need to navigate your app to a specific starting point (for example, logging in, accepting a cookie banner, or opening a particular screen), but you do not want any of those preparatory steps to appear in your test. - -**Steps:** - -1. Start the test in Authoring mode and enable Manual Interaction. - -2. Click Pause Recording in the Manual Interaction toolbar and confirm in the dialog. - -3. Perform the setup actions you want to skip from recording, such as logging in or navigating to the target screen. - -4. Click Start recording once the application is in the desired starting state. - -5. Continue interacting with the app or switch to natural language instructions. From this point onward, your actions are recorded as test steps. - -6. Add assertions in natural language as needed to validate behavior. - -## Best practices - -- Use Manual Interaction to reach complex UI states quickly. -- Switch back to natural language for validations and assertions. -- Review recorded steps to ensure they reflect the intended behavior. -- Avoid unsupported interactions to prevent incomplete steps. -- Use Pause Recording when you need to log in, navigate, or set up app state without those actions being captured as test steps. - -## Troubleshooting: -### Manual actions are not being recorded -Make sure Manual Interaction is turned on before you start interacting with the application. If it is disabled, your actions will not be captured as test steps. - -### Assertions are not getting recorded -This is expected behavior. Assertions are not recorded through Manual Interaction and must be added using natural language instructions after turning it off. - -### Some interactions are missing or incomplete -Check whether the interaction you are trying to perform is currently supported. Actions like drag and drop, hover, or certain mobile gestures are not available yet and will not be recorded. - -### Typed text does not appear in the input field -If the input field restricts certain characters, those characters may still appear in the recorded instruction even though they are not visible in the UI. This does not affect test execution. - -### Unable to interact with system popups -System level popups are not supported through Manual Interaction. These flows need to be handled using alternative approaches or natural language instructions where possible. - -### Application state does not look correct after switching modes -After turning off Manual Interaction, give the application a moment to stabilize before adding new steps or assertions. This helps avoid inconsistencies in recorded steps. - -### My actions stopped getting recorded mid-session -Check whether recording is paused. When Pause Recording is active, the session is in a draft state. Your interactions affect the application but are not captured as test steps. Click Start recording in the Manual Interaction toolbar to resume. - -## Video Tutorial diff --git a/static/docs/kaneai-mobile-app-accessibility.md b/static/docs/kaneai-mobile-app-accessibility.md deleted file mode 100644 index f7f1aaa53..000000000 --- a/static/docs/kaneai-mobile-app-accessibility.md +++ /dev/null @@ -1,81 +0,0 @@ -# Mobile App Accessibility Testing - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Mobile app accessibility testing enables automated accessibility scanning for native mobile applications in KaneAI. This feature performs accessibility checks during test execution, helping identify accessibility issues and ensure compliance with accessibility standards. - -## Product boundary - -This page is for KaneAI-authored mobile flows. If you are triggering scans directly from Appium code, use [Native App Automation Appium (Overview)](/support/docs/accessibility-native-app-automation-test/). If you want manual screen-by-screen validation on a real device, use [Accessibility App Scanner (Overview)](/support/docs/accessibility-app-scanner/). - -### Platform Support - -| Platform | Supported Versions | Notes | -|----------|-------------------|-------| -| **iOS** | 17.0 and above | Versions below 17.0 will pass tests but won't generate accessibility reports | -| **Android** | All versions | Full support for accessibility scanning | - -## How It Works - -### Step 1: Add Accessibility Scan Instruction - -During test authoring, add the accessibility scan instruction: - -1. Navigate to the screen you want to scan -2. Type "/" in the input box and click on **Accessibility scan** -3. The instruction is recorded as part of the test flow -4. The current screen is marked for accessibility scanning - -### Step 2: Instruction Recording - -When the accessibility scan instruction is added: - -- The instruction is recorded in the test execution flow -- The current screen is marked to be scanned using the app scanner hook for accessibility -- The scan hook is triggered during test execution at the corresponding screen - -### Step 3: Test Case Creation - -Once the test is saved: - -- A test case is automatically created with the **`accessibility-scan`** tag -- The accessibility scan instruction is visible in the test case summary -- Use this tag to filter and identify tests that include accessibility scanning - -To view the screenshot of the screen where the accessibility scan was performed, click the (•••) icon. - -### Step 4: Execution and Reporting - -During automated test execution: - -- The accessibility scan is performed at the corresponding screen -- Accessibility issues are detected and categorized -- The accessibility report is available in the test execution results on Automation dashboard - -## Key Features - -| Feature | Description | -|---------|-------------| -| **Native App Scanning** | Uses platform-native accessibility scan hooks | -| **Automatic Tagging** | Tests are automatically tagged with `accessibility-scan` for easy identification | -| **Test Case Integration** | Accessibility instructions are visible in test case summaries | -| **Automated Reporting** | Accessibility reports are generated during test execution | -| **Cross-Platform Support** | Works on iOS 17+ and all Android versions | - -## Troubleshooting - -### No Accessibility Report Generated - -**Possible Cause:** -- iOS version is below 17.0 (tests will execute but no report is generated) - -### Accessibility Scan Instruction Not Recorded - -**Possible Causes:** -- Instruction not properly added during authoring -- Test not saved after adding the instruction - -**Solutions:** -1. Verify the instruction was added correctly during authoring -2. Ensure the test is saved after adding the accessibility instruction -3. Check the test case summary for accessibility instruction visibility diff --git a/static/docs/kaneai-modules-versions-and-enhancement.md b/static/docs/kaneai-modules-versions-and-enhancement.md deleted file mode 100644 index 6ce81b02d..000000000 --- a/static/docs/kaneai-modules-versions-and-enhancement.md +++ /dev/null @@ -1,56 +0,0 @@ -# Module Versioning and Enhancements - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Every time you modify a module in KaneAI, a new version is created automatically. This version history lets you track what changed, compare any two versions side by side, and revert to a previous version when needed. Version control is especially valuable when multiple team members work with the same modules across test cases. - -For an overview of creating, using, and managing modules, see [Modules](/support/docs/kane-ai-modules/). - -## How Versioning Works - -When you edit an existing module (for example, by adding, removing, or modifying a test step), KaneAI automatically increments the version number (e.g., from 1.1 to 1.2). You do not need to manually create versions. - -After a new version is created, test cases that use the module can either: - -- **Accept the new version**: Update the test case to use the latest module version -- **Continue using the current version**: Keep the existing version without changes - -To update a module version across many test cases at once, use the [Bulk Module Update](/support/docs/kaneai-bulk-module-update/) feature. - -## Viewing Version History - -Open any module from the **Modules** listing page to see its complete details, including: - -- Description and tags -- Full version history with timestamps - -The version history shows every change made to the module, who made it, and when. - -## Comparing Versions - -You can compare any two versions of a module to see exactly what changed between them. - -1. Open the module and navigate to the **Version History** section. -2. Select the two versions you want to compare. -3. Review the differences: added, removed, or modified steps are highlighted. - -## Reverting to a Previous Version - -If a module update introduces problems, you can revert to any earlier version. - -1. Open the module and go to **Version History**. -2. Select the version you want to restore. -3. Confirm the revert. - -Reverting creates a new version (it does not delete the intermediate versions), so you maintain a complete audit trail. - -## Linked Test Cases - -The module detail page includes a **Linked Test Cases** tab that shows all test cases using the module, grouped by version. This helps you understand the impact of a version change before making updates. - -From this tab you can navigate directly to individual test case summary pages. - -## Related Guides - -- [Modules](/support/docs/kane-ai-modules/): Create, use, and manage modules -- [Bulk Module Update](/support/docs/kaneai-bulk-module-update/): Update a module version across multiple test cases in one action diff --git a/static/docs/kaneai-network-throttling.md b/static/docs/kaneai-network-throttling.md deleted file mode 100644 index eeaf17209..000000000 --- a/static/docs/kaneai-network-throttling.md +++ /dev/null @@ -1,70 +0,0 @@ -# Network Throttling - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The Network Throttling feature in Kane AI enables users to simulate various network conditions during mobile app testing. -This document outlines how to configure and use network throttling in Kane AI for mobile test creation & execution. - -Following are the Network Throttling settings available: - -| Throttling Setting | Download Speed | Upload Speed | Latency | -| --------- | ------------ | ----------- | ------------ | -| 2G | ↓ 30 - 50kbps | ↑ 1 - 16 kbps | 500ms | -| 3G | ↓ 100 - 200kbps | ↑ 50 - 64 kbps | 400ms | -| 3G+ | ↓ 3 - 5mbps | ↑ 1 - 2 mbps | 100ms | -| 4G LTE | ↓ 13 - 15mps | ↑ 4 - 7 kbps | 70ms | -| 4G Advanced | ↓ 23 - 25mbps | ↑ 9 - 12 mbps | 20ms | -| Custom | ~ kbps | ~ kbps | ~ ms | - -You can create session-specific throttling by selecting the `Custom` option. - -## How to Set Up Network Throttling ? -For your mobile app test session, network throttling can be set up in advance by applying those settings from the start of the Authoring session, or you can modify them at any point during the session. - -### Selecting Network Setting -- Go to `Advanced Settings` in the `Create App Test` section to add Network Throttling setting in advance. - -You can select desired Network setting & your session will start with that setting. - -### Honoring the Network setting -If you want to honor the Network settings in the Authoring session too: - -- Click on the `Honor network profile configuration` toggle. - -This will apply network setting in the Authoring session too, allowing you to experience how your Application works under that particular Network setting. - - Recommended to use this setting only if its absolutely necessary, as this will hamper Authoring experience in low Network Bandwidth settings. - - Network setting will always be Honored while executing the Test Case & code generation, the Honoring option is applicable for Authoring mode only. - -## Changing Network Throttling setting mid session : -To change Network Throttling setting in between the Authoring session: - -- Click on the `Change network throttling` present in Actions Tab, or type `/` to access it. - -- Then select desired Network setting. - -A change in Network throttling setting is also condidered as a step in the Test Case. - -You can edit the Network Throttling setting at that particular step too. To do that: - -- `Pause` the execution flow to enter into the **Draft** state. -- Click on the `Edit Instruction` button & change the Network setting. - -## Changing network throttling in Edit Test Steps mode -Before you begin editing a test case , you can set up network throttling. This Edit mode specific setting will overwrite the network configuration previously established during the app session's initialization in Authoring mode. - - The network throttling for your Editing session is determined by what you set when you start that session. If you don't specify any network settings for the Edit mode, no throttling will be applied. The network settings mentioned while initializing the Authoring session will not be applicable in the Edit session, as saving changes in the Edit mode creates a new test case version. - - And all Test Case executions, like Test Runs and code generation, are performed on the test case's latest saved version. - -## Network Throttling in Test Runs -While executing your Test Cases every Network Throttle step will be honoered. In addition to the Authoring & Editing mode, you can also select the Network setting to initialize your session with : - -- Click on `Advanced Configurations` present in the `Run with HyperExecute` section or `Schedule Test Runs` section while executing a Test Run. - -- Network Throttling setting is available in the `Network Settings` section. - -- You can view where & when Network Throttling commands are executed in the command logs present in the App Automation page. - - Offline Mode is supported only for iOS devices. diff --git a/static/docs/kaneai-release-notes-0-0-1.md b/static/docs/kaneai-release-notes-0-0-1.md deleted file mode 100644 index d1126f440..000000000 --- a/static/docs/kaneai-release-notes-0-0-1.md +++ /dev/null @@ -1,25 +0,0 @@ -# Version 0.0.1 - Beta - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -### Proximity-Based Hub Region Allocation for Code Generation -We’ve introduced proximity-based hub region allocation to optimize code generation performance by selecting the nearest hub region to the user. This enhancement ensures faster execution and reduced latency during test script generation. - -### Proximity-Based VM Allocation for Web Tests -KaneAI now allocates virtual machines (VMs) for web tests based on proximity, improving test execution speed and stability by choosing the closest VM location to the user. - -### Removed Auto Trigger of Code-Export on Test Failures or Queued Instructions -The automatic triggering of code export has been removed when instructions in the test are either failed or queued. This update provides users with better control over the export process and avoids unnecessary actions during troubleshooting. - -### Improved Stability in Manual Interaction with Multiple XPaths -We've enhanced manual interactions by allowing the use of multiple XPaths, providing more stability and accuracy when interacting with elements on web pages. - -### Code Generation -Addressed multiple issues in the code generation process, including: - -- Click interception problems -- Websites failing to load correctly -- Failures in identifying and interacting with XPaths - -### Document and Element Scroll Fix in Manual Interaction -Fixed a bug causing issues with scrolling documents and elements during manual interactions, ensuring smoother navigation and interaction within the testing interface. diff --git a/static/docs/kaneai-release-notes-0-0-2.md b/static/docs/kaneai-release-notes-0-0-2.md deleted file mode 100644 index d2ef4587c..000000000 --- a/static/docs/kaneai-release-notes-0-0-2.md +++ /dev/null @@ -1,36 +0,0 @@ -# Version 0.0.2 - Beta - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## 1. Iframe, Shadow DOM, and Canvas Support -KaneAI now fully supports testing for: - -- Iframe elements 🖼️ -- Shadow DOM elements 🕶️ -- Canvas elements 🎨 - -We've gone the extra mile to bring full iframe, Shadow DOM, and Canvas support in code exports 🖥️. Now, KaneAI handles these intricate UI elements with precision during script generation, making your automation journey even smoother. Whether dealing with complex nested iframes or shadow-rooted elements, KaneAI has you covered! - -## 2. Auto-Scroll for Manual Interaction 🎯 -No more searching for the latest entries after clicking! KaneAI now automatically scrolls to the latest interaction in manual testing sessions, saving time and making testing more efficient. - -## 3. Screenshot Modal Fix 📸 -We’ve fixed an issue where screenshots would appear outside the modal on both KaneAI and TMS. Screenshots will now stay neatly within their designated area for better viewing. - -## 4. URL Update with Objective Changes 🌐 -Whenever you update an objective, the URL will now update along with any other pre-filled fields. This improvement ensures that your URLs stay in sync with your updates for better tracking and organization. - -## 5. Label Overlap Fix in TMS 🏷️ -Fixed the issue of labels overlapping other elements on the TMS UI. Now, we show a concise label such as "+12" when there's no space. Clicking it will open an overlay with all label details, improving readability and UI neatness. - -## 6. Test Case Name Display Issue Fixed 🔍 -Resolved an issue where test case names weren't appearing in certain situations, such as when the screen was zoomed in at 200%. This fix ensures that test case names are visible even in extreme zoom conditions. - -## 7. Manual Interaction Enhancements 🖱️ -We've refined manual interactions to make Click actions faster and more precise, optimizing the overall experience for testers. - -## 8. Stable Template Support 📑 -Our templates have been stabilized for more reliable performance during test creation and execution. - -## 9. Enhancements in Visual and Textual Query for Code Generation 🔍 -We've enhanced the visual and textual query capabilities in code exports, making your scripts even more accurate when handling complex UI elements. diff --git a/static/docs/kaneai-release-notes.md b/static/docs/kaneai-release-notes.md deleted file mode 100644 index 40e1c7e66..000000000 --- a/static/docs/kaneai-release-notes.md +++ /dev/null @@ -1,13 +0,0 @@ -# KaneAI Release Notes - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -KaneAI is an AI-powered next-gen testing assistant that revolutionizes software testing. Whether you're a seasoned QA professional or a developer new to testing, KaneAI simplifies complex testing processes, making them accessible to everyone. - -## KaneAI Releases - -### September, 2024 -| Release Number | Release Date | -|----------------|--------------| -| [Version 0.0.2](/support/docs/kaneai-release-notes-0-0-2) | September 23, 2024| -| [Version 0.0.1](/support/docs/kaneai-release-notes-0-0-1) | September 16, 2024| diff --git a/static/docs/kaneai-rename-instructions.md b/static/docs/kaneai-rename-instructions.md deleted file mode 100644 index 1c6fb60cc..000000000 --- a/static/docs/kaneai-rename-instructions.md +++ /dev/null @@ -1,45 +0,0 @@ -# Rename Slash Command Steps - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Rename test steps for API, JS, Database, Network Assertion and Visual Comparison to improve clarity and consistency without affecting functionality. - -## Overview - -The **Rename Test Steps** feature allows users to change a test step’s name without changing how it works. - -**Benefits:** -- Improves readability of test steps -- No impact on execution or instruction logic -- Renamed step name stays even if instruction changes later -- Works for Web and Mobile slash command instructions -- The step names are updated even in the execution command logs. - -## How to Use - -1. Ensure the authoring session is running and the relevant step is visible. -2. Pause the session. Renaming only works in "DRAFT" session state after session is Paused. -3. Click on the step name to make it editable. -4. Enter the new name and press **Enter** or click outside to save. -5. Resume the session. - -> **Note:** Only the step name changes; the instruction logic and behavior remain the same. - -## Limitations - -- Supported instruction types: - - **Web:** API, DB, JS, Visual comparison, Network assertion - - **Mobile:** API, DB, JS, Visual comparison -- Only the **step name** can be updated (max 255 characters) -- Renaming inside Modules is not supported -- Step rename history is not versioned -- Special characters `{}` appear as `{{}}` in the generated code - -## Troubleshooting - -| Issue | Cause | Solution | -|-------|-------|---------| -| Step name not updated | Save action missed | Re-edit and save | -| Step name resets after refresh | Network/session issue | Refresh page and retry | - -> Have any feedback or request? Reach out to us via [support@testmuai.com](mailto:support@testmuai.com) and we would be happy to hear from you. diff --git a/static/docs/kaneai-scheduled-test-runs.md b/static/docs/kaneai-scheduled-test-runs.md deleted file mode 100644 index 11d4daf41..000000000 --- a/static/docs/kaneai-scheduled-test-runs.md +++ /dev/null @@ -1,21 +0,0 @@ -# Scheduled Test Runs - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -In this documentation, you will learn how to create a scheduled test run using pre-existing test runs. This involves navigating through your project, selecting test configurations, and setting up recurring schedules to automate test executions at specified times. - -**Step 1:** Navigate to **Test Manager** >> select your desired project >> **Test Runs** section. - -**Step 2:** Here, you will see the details of your existing test runs. To create a schedule, click on the three dots next to a test run and select **Create Schedule**. Alternatively, you can go to the **Scheduled runs** section to view all existing schedules. If you don't have any existing schedule, click on **Create a Scheduled Run** button to create a scheduled run. - -> Test Runs must be created before creating a test run schedule. - -**Step 3:** Provide the details related to the test you want to schedule, like **Schedule Name**, select existing test run you want to schedule, and configure the date and time details. - -**Step 4:** After making your selections, click **Next** to review a summary of the schedule, including the scheduled date and time. - -**Step 5:** In the **Build Parameters** box, you can define your desired concurrency for the tests. Click on the **Advanced Configurations** button to configure additional settings such as network throttling, URL replacement, visual regression, accessibility testing, and [report generation](/support/docs/kaneai-hyperexecute-test-run-execution/#reports). Finally, click **Schedule Test Run** to create a recurring schedule. - -**Step 6:** Click on any existing schedule to view detailed information, including a summary of test outcomes, passed and failed cases, and their respective instances. This helps in easily reviewing and proceeding with the tests. - -## Video Explanation diff --git a/static/docs/kaneai-scroll-until-mobile.md b/static/docs/kaneai-scroll-until-mobile.md deleted file mode 100644 index c1f210989..000000000 --- a/static/docs/kaneai-scroll-until-mobile.md +++ /dev/null @@ -1,88 +0,0 @@ -# Scroll Until Element for Mobile Authoring - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -> This document provides a comprehensive guide on using the Scroll Until Element feature in KaneAI. It covers how to define scroll actions using simple natural language commands, eliminating the need for manual coordinate-based scripting for mobile app testing. - -## Overview - -In modern mobile and web applications, important elements are often located below the fold or deep within long, scrollable pages. Writing custom scroll logic or managing scroll coordinates manually can be fragile, difficult to maintain, and time-consuming. - -The **Scroll Until Element** feature allows users to automate scrolling using **natural language commands**. Instead of controlling how to scroll, users simply describe the element they want to reach. - -This feature helps by: -- **Automatically locating elements** by scrolling until the target becomes visible -- **Reducing complexity** by removing the need for technical selectors or manual scroll logic - -## How-To Guide - -### Identify the Target - -Determine the element you need to reach based on its **visual representation**. - -In natural language mode, elements can be referenced using: -- Visible text (e.g., *"Login button"*) -- Element type (e.g., *"input field"*) - -> **Note:** -> Natural language commands do **not** support technical selectors such as IDs, XPaths, or CSS attributes. -> Always describe what is visibly present on the screen. - -### Formulate the Command - -Enter your command into the test editor using **action-oriented language**. -Clearly specify what the agent should find and what action to perform once found. - -**Examples:** -- `"Scroll until search bar is visible and click on it."` -- `"Scroll down until element with text 'Shipping Policy' is visible."` -- `"Scroll inside a subsection until date input field and click on it."` - -## Use Cases - -| Scenario | Example Natural Language Command | -|--------|--------------------------------| -| Visibility Check | `Scroll until search bar is visible and click on it.` | -| Text Search | `Scroll down until element with text XYZ is visible.` | -| Button Interaction | `Scroll until Submit button and click on it.` | -| Subsection Navigation | `Scroll inside a subsection until date input field and click on it.` | -| Dropdown Selection | `Scroll inside the dropdown until 'New option' and select it.` | -| Horizontal Scrolling (Mobile App) | `Scroll right inside the carousel until 'Best Sellers' is visible.` | -| Nested Container (Mobile App) | `Scroll inside the reviews section until 'Load more' button and click on it.` | - -## Limitations - -Please be aware of the following constraints in the current version. - -### Mobile App Authoring - -**Now Supported** -Mobile app authoring now supports **horizontal scrolling** and **scrolling inside nested containers** (element scrolling). You can scroll left or right, for example inside carousels, and scroll within specific container elements such as dropdowns, lists, and subsections instead of only the main viewport. - -- **Scroll Limit**: - Maximum of **20 scroll attempts** are performed. If the element is not found, the action fails. -- **Directional Extremes**: - ❌ Commands such as *“Scroll to top”* or *“Scroll to bottom”* are not supported. - -### Mobile Browser Authoring - -- **Element Eligibility**: - Applicable **only to interactable elements** (buttons, links, dropdowns, inputs). - ❌ Plain text or static labels are not supported. -- **Native Popups**: - ❌ Scrolling inside native browser popups or system alerts is not supported. -- **Horizontal Scrolling**: - ❌ Not supported. - -## Troubleshooting - -If you encounter execution issues, refer to the table below: - -| Error Message / Behavior | Potential Cause | Solution | -|------------------------|-----------------|----------| -| Maximum Scroll Limit Reached | Element is beyond 20 scrolls or not present | Verify element visibility and ensure it appears within 20 scrolls | -| Element Not Found | Non-interactable or plain text element (Mobile Browser) | Target a button, link, or input field instead | -| Scroll Timeout | Element is inside a nested container that was not specified in the command | Name the container in the instruction (e.g., "Scroll inside the reviews section until ...") | -| Command Not Recognized | Usage of IDs, XPaths, or attributes | Rewrite using visible text or element type (e.g., "button") | - -> Have any feedback or request? Reach out to us via [support@testmuai.com](mailto:support@testmuai.com) and we would be happy to hear from you. diff --git a/static/docs/kaneai-sequential-test-runs.md b/static/docs/kaneai-sequential-test-runs.md deleted file mode 100644 index 3de6b5d46..000000000 --- a/static/docs/kaneai-sequential-test-runs.md +++ /dev/null @@ -1,74 +0,0 @@ -# Sequential Test Runs - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Sequential test runs let you execute dependent test cases in a specific order. Unlike parallel execution, where test cases run independently and concurrently, sequential execution ensures each test case completes before the next one begins. This is useful when test cases share state or depend on the outcome of a previous test, for example, a login test that must run before an account settings test. - -## Prerequisites - -- Access to Test Manager with an existing project -- KaneAI-generated test cases with successful code generation -- Credentials for TestMu AI platform - -## Create a Sequential Test Run - -### Step 1: Enable Sequential Execution - -Create a test run and add your test cases following the standard [test run setup](/support/docs/kaneai-hyperexecute-test-run-execution/). After adding the test cases, enable the **Execute instances sequentially** toggle. - -Once enabled, you can arrange the test cases in the order you want them to execute. The test cases listed will be executed in this sequence for each configuration assigned. - -Parameterized test cases must use the default dataset with single values when running sequentially. - -### Step 2: Configure for a Single Platform - -For a **single-platform sequential run** (e.g., desktop web only), you can select multiple configurations at a global level. For example, you can run the same sequence on both Linux with Chrome 137 and Linux with Firefox. A separate sequential run is created for each configuration. - -### Step 3: Preview and Save - -Click **Show execution preview** to review the test sequence before executing. The preview displays the exact order in which test cases will run for each configuration. Once verified, click **Save test run** and proceed to execution. - -## Multi-Platform Sequential Runs - -You can also configure sequential runs that span multiple platforms, for example, running a web test followed by a mobile app test. When multiple platforms are combined (desktop web, mobile browser, mobile app), the global configuration option is replaced with per-test-case configuration. - -### Step 1: Select Configurations Per Test Case - -Assign the appropriate configuration to each test case individually. Choose configurations that match the platform each test case was authored on. - -### Step 2: Arrange Execution Order and Preview - -Set the execution order based on your test dependencies. Use the execution preview to verify the sequence, for example, a web login test running first, followed by an iOS app verification test. - -### Step 3: Execute the Sequential Run - -Save and execute the test run. The test instances will run one after another in the defined order. - -## Key Differences: Single vs. Multi-Platform - -| Aspect | Single-Platform Run | Multi-Platform Run | -|---|---|---| -| **Configurations** | Multiple global configurations allowed (one sequential run per config) | Single configuration per test case only | -| **Global config** | Supported | Not supported: group configuration is removed when multiple platforms are added | -| **Concurrency** | One per configuration sequence | Limited to one across the entire run | - -## Limitations - -- **Multi-platform runs support only a single configuration per test case.** When you add platforms such as mobile browser or mobile app alongside desktop web, the global (group) configuration option is removed. -- **Concurrency is limited to one for multi-platform sequential runs.** Tests always run one after another since they depend on execution order. -- **Execution order must be configured carefully.** The sequence directly impacts dependent test behavior. Verify the order in the execution preview before saving. - -## Troubleshooting - -| Issue | Solution | -|---|---| -| Group configuration option disappears after adding new platforms | This is expected for multi-platform sequential runs. Only single-configuration-per-test-case is supported in this mode. | -| Tests not running in the expected order | Re-check the configured sequence in the execution preview and ensure the test run was saved after making ordering changes. | -| Concurrency appears higher than expected | Confirm the run is configured as sequential (toggle enabled) and not mixed with parallel configurations. | -| Unexpected behavior with platform configurations | Ensure only configurations that match each test case's platform are selected. | - -## Related Guides - -- [Execute Test Runs on HyperExecute](/support/docs/kaneai-hyperexecute-test-run-execution/): Standard test run creation and execution -- [Scheduled Test Runs](/support/docs/kaneai-scheduled-test-runs/): Automate test run scheduling -- [Test Run Configurations](/support/docs/test-runs-configurations/): Manage browser and device configurations diff --git a/static/docs/kaneai-smart-locator-priority.md b/static/docs/kaneai-smart-locator-priority.md deleted file mode 100644 index fe0c2356a..000000000 --- a/static/docs/kaneai-smart-locator-priority.md +++ /dev/null @@ -1,68 +0,0 @@ -# Smart Locator Priority - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Smart Locator Priority lets your organization define the order in which KaneAI should prefer supported locator strategies while authoring test cases in KaneAI. - -This is useful when your application is more stable with one locator type over another and you want that preference applied consistently at the organization level. - -## What It Does - -Smart Locator currently supports prioritizing these locator strategies: - -1. **CSS selector** -2. **XPath** -3. **Full XPath** - -The locator type placed at the top of the list is given higher priority than the locator types below it. - -For example: - -- If **CSS selector** is placed above **XPath**, KaneAI will prefer CSS selector first. -- If **XPath** is moved to the top, KaneAI will prefer XPath first. - -## Why This Matters - -Different applications behave better with different locator strategies. - -You may want to adjust the default order if: - -- your application generates more stable CSS selectors than XPath expressions -- your current pages rely more heavily on XPath-based resolution -- your team wants one consistent Smart Locator strategy across the organization - -## Where to Find It - -- [Open Smart Locator Priority in Organization Settings](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/org-settings/kane-ai/smart-locator-priority) - -The following screen shows where the setting is located: - -## How to Configure Smart Locator Priority - -1. Open **Organization Settings** in TestMu AI. -2. Navigate to **Org Product Preferences**. -3. Expand the **Kane AI** section in the left navigation. -4. Open **Smart locator priority**. -5. Drag and drop the supported locator types into your preferred order. -6. Keep the most preferred locator strategy at the top of the list. - -Once updated, the new order becomes the organization-level Smart Locator preference for any newly authored or edited test cases in KaneAI. - -The change in priority will not reflect in already authored test cases and will only be updated when a test step is authored or re-authored in KaneAI playground. - -## Best Practices - -- Choose one order for your organization and avoid changing it too frequently. -- Validate the preferred strategy on a few representative KaneAI flows before standardizing it across teams. -- If locator behavior changes after a UI update in your product, review whether the current priority order is still appropriate. - -## Limitations - -- This setting currently supports only the locator strategies shown in the UI. -- The priority order controls which locator type is preferred first; it does not replace the need for stable page structure and reliable element identification. - -## Related Docs - -- [Failure Conditions](/support/docs/kaneai-failure-conditions/) -- [KaneAI Command Guide](/support/docs/kane-ai-command-guide/) -- [Custom Instructions in KaneAI Test Generation](/support/docs/generate-test-cases-with-ai/) diff --git a/static/docs/kaneai-smartui-visual-testing.md b/static/docs/kaneai-smartui-visual-testing.md deleted file mode 100644 index 56185fbe7..000000000 --- a/static/docs/kaneai-smartui-visual-testing.md +++ /dev/null @@ -1,73 +0,0 @@ -# Visual Test Creation with KaneAI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This guide provides a detailed walkthrough on how to leverage KaneAI to create, manage, and automate visual regression tests. By integrating with SmartUI, you can capture screenshots, compare visual changes, and maintain UI consistency throughout your development lifecycle. - -## Creating Your First Visual Test - -### Initiate a Web Test -To begin, navigate to the [KaneAI dashboard](https://www.testmuai.com/login/?redirectTo=https://kaneai.lambdatest.com/objective) and click **Author Browser Test**. You can configure network settings before clicking **Start Testing** to launch the test authoring environment. - -### Add a Visual Comparison Step -Author your test by writing steps in plain language, such as `go to lambdatest`. - -To add a visual test step, type `/` in the "Write a step" editor and select **Visual Comparison** from the command menu. You may also use the **Eye Icon** on the dashboard for quick access. - -### Configure Your Screenshot -A configuration modal will appear for your screenshot. -* **Screenshot Name (Required):** You must provide a descriptive name for the screenshot. -* **Mis-Match Threshold (Optional):** Set an acceptable percentage of difference below which changes are auto-approved. The default is 0%. -* **Screenshot Type (Optional):** Choose between **View Port** (default) or **Full Page**. - -Click **Create** to add the step to your test flow. - -## Analyzing Baselines and Regressions - -### Establish the Baseline -The first time the test is executed, it captures an initial screenshot that becomes the **Baseline** for all future comparisons. You can view this image by clicking **Show Baseline**. - -### Rerun and Analyze the Comparison -When the test case is replayed or re-run during authoring, a new screenshot is captured and compared against the baseline. Click **Show Comparison** to open the SmartUI view. This interface highlights any visual differences found between the two images. - -### Approve or Reject Changes -Within the comparison view, you can review the changes and decide on a verdict. -* **Reject Screenshot:** If the changes are unintended defects, you may reject the comparison. -* **Approve Screenshot:** If the changes are intentional, you may accept the comparison. This will update the baseline with the new image. - -You can also adjust project settings like **Compare Type** and **Mis-Match Threshold** directly from this view. - -## Saving and Automating Your Visual Test - -### Save Your Test Case -Once your test flow is complete, click **Finish test**. You will be prompted to save the session, where you can define the **Project**, **Folder**, **Test Name**, and **Description**. - -### Execution and Scheduling - -Once your test case is saved, navigate to the **Test Runs** tab to configure and launch its execution. For a complete guide on the initial setup, refer to our documentation on [Test Run Execution](/support/docs/kaneai-hyperexecute-test-run-execution/). - -You can execute your test using one of two methods: - -* **Run Instance Now** -This option triggers an immediate, one-time execution of your test. It is ideal for quick validations or debugging a specific change. - -* **Schedule Test Run** -This option allows you to automate your visual regression testing by setting up a recurring schedule. Learn more in our guide to [Scheduled Test Runs](/support/docs/kaneai-scheduled-test-runs/). - -### Using Advanced Configurations for Visual Regression -While running **Run Instances Now** user can access **Visual Regression** settings via **Advanced Configurations** - -Here users can enable **Visual Regression** settings to use the advanced configurations: -* Custom Browsers (upto 4) -* Custom Viewports (upto 5) -* Custom Devices (upto 2) with Orientation (Portrait and Landscape) - -For a more flexible usage you may use the **JSON** tab where you can set your own parameters and configuration. - -## View Your Visual Test Results on SmartUI - -After the Test Run job is finished, all screenshots captured using the `/Visual Comparison` command via KaneAI will be available in your SmartUI project for review. - -1. Navigate to the [SmartUI Dashboard](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/). -2. Select your project and the most recent build. -3. Here, you can compare screenshots against the baseline, highlight differences, and approve or reject changes. You can use features like Smart Ignore to mask dynamic areas for more stable comparisons. diff --git a/static/docs/kaneai-test-manager-access-management.md b/static/docs/kaneai-test-manager-access-management.md deleted file mode 100644 index 1c36d4b37..000000000 --- a/static/docs/kaneai-test-manager-access-management.md +++ /dev/null @@ -1,31 +0,0 @@ -# Access Management - KaneAI <> Test Manager - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This document provides guidelines on managing access to KaneAI and Test Manager within TestMu AI. The access management functionality is exclusive to **organization administrators**, allowing them to allocate licenses to users as per the organization's requirements. - -## License Allocation Rules -- **License Limitation :** Administrators can only allocate as many licenses as they have purchased or gained licenses through trials. -- **User Addition :** A user can only be added if there is an available license. -- **License Reassignment :** An admin can revoke a license from one user and assign it to another, ensuring optimal resource utilization. -- **Real-Time Monitoring :** Admins can view the current usage of licenses through the Access Management Dashboard. - -## Step-by-Step Guide for Managing Access - -### Accessing the License Management Panel -**Step 1 :** Log in to your TestMu AI Admin Account. -**Step 2 :** Navigate to **Settings** > **Organization Settings** > **Products**. -**Step 3 :** Select KaneAI or Test Manager tab to allocate the desired product license. - -### Assigning a License to a User -**Step 1 :** Navigate to KaneAI or Test Manager tab for which you want to allocate the license. -**Step 2 :** Click on **Allocate Licenses** button. -**Step 3 :** Select the plan name for which you want to allocate the licenses to your users. -**Step 4 :** Select and add users from your organization to allocate licenses. -**Step 5 :** Click Allocate Licenses to complete the process. - -### Removing or Reassigning a License -**Step 1 :** Locate the user from the User List. -**Step 2 :** You can select multiple users or for individual users click on the **...** button. -**Step 3 :** Click **Revoke License** to remove access. -> The freed-up license will be available for reassignment. To assign it to another user, follow the [Assigning a License to a User](/support/docs/kaneai-test-manager-access-management/#assigning-a-license-to-a-user) steps. diff --git a/static/docs/kaneai-test-run-instance-view.md b/static/docs/kaneai-test-run-instance-view.md deleted file mode 100644 index 6d0e72a1b..000000000 --- a/static/docs/kaneai-test-run-instance-view.md +++ /dev/null @@ -1,144 +0,0 @@ -# Test Run Instance View - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -**Early Access** -This feature is currently being rolled out in phases and may not be available on all accounts. If you do not see the **New View** badge on your test instances, your account has not been enabled yet. - -The Test Run Instance view provides a step-level execution replay that maps each executed action back to the original steps authored in KaneAI. Instead of reviewing raw automation logs, you can walk through the exact sequence of steps - with screenshots, command details, and failure context - in a single unified interface. - -## Test Run Summary Dashboard - -When you open a test run, the summary dashboard gives you an at-a-glance view of the entire run. - -Test instances with the **New View** badge support the enhanced instance view described below. - -## Test Instance Detail View - -Click on any test instance to open the detailed execution view. The page is divided into three main areas: the **Steps Panel** on the left, the **Screenshot/Video Panel** in the center, and the **Inspector Panel** on the right. - -### Execution Metadata - -The top bar displays key information about the test instance: - -- **Status** - Passed or Failed indicator -- **Duration** - Total execution time -- **Configurations** - Browser version, OS, and resolution -- **Version** - Test case version used for this execution -- **Executed by** - The user or scheduler that triggered the run -- **Test Case ID** - Link back to the original test case -- **Labels** - Associated job and task identifiers - -For failed tests, a banner prompts you to **Generate RCA** (Root Cause Analysis) or **View failed step** to jump directly to the point of failure. - -### Steps Panel - -The left panel lists every step that was executed, mapped back to the original steps authored in KaneAI. Each step shows: - -- The **action type** (click, assert, set variable, execute JavaScript, navigate, scroll, etc.) -- The **execution duration** for that step -- A **pass/fail indicator** (green dot for passed, red dot for failed, gray for skipped) -- **Variable values** and **assertion results** inline - -Steps that include sub-actions - such as loops, conditionals, or JavaScript snippets - are expandable to reveal the full execution detail. - -#### Auto-Heal Visibility - -When [Auto-Heal](/support/docs/kaneai-auto-heal/) recovers a step during execution, that step displays an **auto-heal indicator** icon next to its execution duration in the steps panel. This gives you clear, step-level visibility into exactly which steps were healed in a run, without digging through execution logs. - -#### Autoplay Steps - -Click the **Autoplay Steps** button at the top of the steps panel to automatically walk through each step in sequence. As each step is highlighted, the center panel updates to show the corresponding screenshot captured during execution. This provides a visual replay of the entire test execution without needing to click through each step manually. - -### Screenshot and Video Panel - -The center panel shows a screenshot of the application under test at the point when the selected step was executed. You can toggle between: - -- **Screenshot view** - A static capture of the page at that step -- **Video view** - A recorded video of the execution - -At the bottom of the panel, a **step progress bar** displays colored dots representing each step - green for passed, red for failed. Click any dot to jump directly to that step. - -Hovering over a dot shows a preview tooltip with the step number and action name. - -### Compare Original Screenshot - -Click **Compare original screenshot** above the screenshot panel to open a side-by-side comparison of: - -- **Original while authoring** - The screenshot captured when the test was first authored in KaneAI -- **Current execution screenshot** - The screenshot from this execution run - -This helps you identify UI changes, layout shifts, or missing elements that may have caused a test failure - especially useful when a test that previously passed starts failing after a UI update. - -## Debugging Failed Tests - -When a test instance fails, the view highlights the failure clearly so you can diagnose the issue quickly. - -- The **failed step** is highlighted with a red border and shows the assertion result (e.g., "Assertion False"). -- Steps after the failure are **grayed out**, indicating they were not executed. -- The screenshot panel shows the state of the application at the exact moment of failure. -- Click **View failed step** in the top banner to jump directly to the failure point. -- Click **Generate RCA** to trigger an AI-powered root cause analysis that examines the failure context and suggests probable causes. - -## Command Logs - -Click the panel toggle on the right side to open the **All Commands** tab. This panel shows every low-level command that was executed during the test, grouped by step. - -Each command group is expandable and shows: - -- **Command type** - Execute JavaScript, Command, HTTP request, etc. -- **Duration** - Time taken for each individual command -- **Timestamp** - Exact execution time -- **Script content** - For JavaScript commands, the script that was executed - -Use the **Search Commands** bar to find specific commands, or filter using the **View** dropdown to narrow by command type. - -## Execution Logs - -The right panel provides multiple log tabs for in-depth debugging: - -- **Framework** - Logs from the test framework used during execution -- **Console** - Browser console output captured during the test -- **Device** - Device-level logs (useful for mobile test instances) -- **Network** - Network request and response details -- **HyperExecute** - Raw HyperExecute execution logs including runtime information such as network idle times, DOM load events, and script execution output - -Additionally, the following tabs are available for extended analysis: - -- **SmartUI** - Visual regression testing results -- **Performance** - Performance metrics captured during execution -- **Accessibility** - Accessibility audit results for the tested pages - -You can dock the logs panel to the **right** or **bottom** of the screen depending on your preference. - -## Navigate Between Tests - -Click the **View tests** button in the top-right corner to open a panel listing all test instances in the current test run. This panel lets you: - -- **Search** for specific tests by name -- **Filter** by status - view only failed, passed, or stopped tests -- **Switch** between tests without navigating back to the test run listing - -## Share a Test Run Instance - -You can share a test run instance page publicly with stakeholders, including people who do not have a TestMu AI account. Click the **Share** icon in the top-right corner of the instance view to open the share dialog. - -The share dialog provides the following options: - -- **Expiry Duration** - Choose how long the shared link remains accessible (for example, **View: 7 Days**). After the link expires, it can no longer be opened. -- **Email ID(s)** - Enter one or more email addresses and click **Invite** to send the link directly, with an optional **Message** for context. -- **Copy Link** - Copy a shareable link to the clipboard and distribute it yourself. - -Anyone with an active shared link can view the test run instance (including the steps, screenshots, and execution details) until the link expires. - -## Limitations - -- **Mobile Browser not supported**: The enhanced Test Run Instance view is currently not supported for Mobile Browser test executions. -- **Applies to newly generated code only**: This view is available only for test cases whose code was generated after the feature was enabled. For older test cases, the previous automation details page will continue to be shown. - -## Related Guides - -- [Execute Test Runs on HyperExecute](/support/docs/kaneai-hyperexecute-test-run-execution/) - Create and execute test runs -- [Sequential Test Runs](/support/docs/kaneai-sequential-test-runs/) - Run dependent test cases in order -- [Test Run Configurations](/support/docs/test-runs-configurations/) - Manage browser and device configurations -- [Scheduled Test Runs](/support/docs/kaneai-scheduled-test-runs/) - Automate test run scheduling diff --git a/static/docs/kaneai-test.md b/static/docs/kaneai-test.md deleted file mode 100644 index d3843034d..000000000 --- a/static/docs/kaneai-test.md +++ /dev/null @@ -1,5 +0,0 @@ -# Run Your First Web Test Using KaneAI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -In this tutorial, learn how to run your first web test using KaneAI. With KaneAI, you can create, schedule, execute, and debug tests using natural language. You'll learn how to create a web test, highlighting features like if statements and assert queries. diff --git a/static/docs/kaneai-totp.md b/static/docs/kaneai-totp.md deleted file mode 100644 index 3f165753a..000000000 --- a/static/docs/kaneai-totp.md +++ /dev/null @@ -1,82 +0,0 @@ -# TOTP Authentication in KaneAI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Introduction - -Teams often need to test SSO or MFA‑protected applications where the second factor is a six‑digit OTP. Native TOTP support in KaneAI generates those codes on the fly from the shared secret key, keeps the entire login journey inside KaneAI with identical behavior in Replay, generated code, and CI, and lets you inject the code directly into your test steps, no external scripts or servers required. - -TOTP variables are now **global by default**. You create them once and reuse them across all test cases and modules in your organization. - -## How It Works - -1. **Create a global TOTP variable** from the **Variables > TOTP Variables** page or via the `/totp` slash command during authoring. -2. Give the variable a name (e.g., `sso_totp`) and supply the Base32 secret key. -3. At runtime KaneAI calculates the correct six‑digit code every 30 seconds. -4. Reference the code in any test step using `{{totp.variable_name}}`. - -## Prerequisites - -- A TOTP‑compatible login flow (e.g., Google Authenticator, Okta Verify, Microsoft Authenticator). -- The **secret key** (Base32 encoded) associated with the account you want KaneAI to log in as. - -## Step‑by‑step Guide - -### Create a Global TOTP Variable - -You can create a global TOTP variable in two ways: - -#### Option A: From the Variables Page - -1. Open **Test Manager** and navigate to **Variables > TOTP Variables**. -2. Click **+ Create new**. -3. Enter a **Variable Name** (e.g., `sso_totp`). -4. Enter the **TOTP Secret Key**: paste the Base32 secret directly or reference an org secret using `{{` syntax. -5. Click **Create TOTP Variable**. - -#### Option B: From the Authoring Session - -1. Start a KaneAI authoring session and press **/**. -2. Choose **Add TOTP Authentication Key**. -3. Provide a **Variable Name** and the **Base32 secret key**. -4. Click **Save**. The variable is created globally and is immediately available across all test cases and modules. - -### Use a TOTP Variable in Your Test - -1. Navigate to the OTP input field in your application. -2. Reference the variable in your instruction, for example, "Enter `{{totp.sso_totp}}` in verify field". -3. KaneAI generates and inserts the current six‑digit code automatically. - -## Migrating Existing Local TOTP Variables - -If your test cases were created before this update (10th March 2026), they may contain **local** TOTP variables that were scoped to a single session. These local variables continue to work in existing test runs. However, when you open one of these test cases in the KaneAI playground for editing, you will be prompted to convert the local variables to global. - -When you open a test case that contains local TOTP variables, KaneAI displays a **Convert Local TOTP to Global** dialog: - -1. Review the listed local variables. -2. Assign a **TOTP variable name** for each one (this becomes the global identifier). -3. Optionally reference an existing global TOTP variable using `{{` syntax instead of creating a new one. -4. Click **Convert All**. - -**Important** -- **Existing test runs are not affected.** Your scheduled jobs, CI pipelines, and previously generated code will continue to work as before. -- The conversion is only triggered when you **edit the test case in the KaneAI playground**. Until you do, the test case uses its original local variables. -- After conversion, the test case uses `{{totp.variable_name}}` syntax, and the variable becomes available across all test cases and modules. - -## FAQ - -### Does KaneAI store my secret securely? - -Yes. Secrets are encrypted, stored in Vault, and never exposed in logs or generated code. - -### Do mobile tests support TOTP? - -Yes. The `{{totp.variable_name}}` syntax is available in both Web and Mobile test steps. - -### Can I reuse the same TOTP variable across multiple test cases? - -Yes. Because TOTP variables are global, any test case or module can reference the same variable by name. - -### What happens to my existing tests that use the old `{{totp}}` syntax? - -Existing test runs continue to execute without changes. When you open the test case for editing in the KaneAI playground, you will be prompted to convert local variables to global and adopt the new `{{totp.variable_name}}` syntax. diff --git a/static/docs/kaneai-upload-and-download-files.md b/static/docs/kaneai-upload-and-download-files.md deleted file mode 100644 index d12433a6d..000000000 --- a/static/docs/kaneai-upload-and-download-files.md +++ /dev/null @@ -1,70 +0,0 @@ -# Upload and Download Files in KaneAI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This guide provides a step-by-step process for uploading and downloading files within Kane AI, a cloud-based testing platform provided by TestMu AI. Users can easily upload files from their local system, access pre-uploaded files, and download files generated during a test session. - -## Prerequisites - -- A valid TestMu AI account with access to Kane AI. -- An active test session on Kane AI. -- Supported file formats: - - **Images :** JPG, JPEG, PNG, GIF (Maximum size: 10 MB) - - **Videos :** MP4 (Maximum size: 50 MB) - - **Files :** XLS, XLSX, DOC, DOCX, PDF, CSV, TXT, CRT, CER, GED, GPX, XML, PBIX, ZIP, JSON (Maximum size: 15 MB) - -## Steps to Upload a File -### Step 1: Initiate a Test Session -- Log in to your TestMu AI account. -- Click on **Create a Web Test** to start a new session within Kane AI. - -### Step 2: Navigate to the Upload Section -- Within the test session, navigate to the specific webpage where file uploads are required. -- Type a **`slash (/)`** to access the file selection menu. - -### Step 3: Select Files for Upload -- Choose to either: - - Select from pre-uploaded files. - - Upload files from your local system. -> - Ensure the upload limit does not exceed 5 files per session. -> - Supported file formats: -> - **Images :** JPG, JPEG, PNG, GIF (Maximum size: 10 MB) -> - **Videos :** MP4 (Maximum size: 50 MB) -> - **Files :** XLS, XLSX, DOC, DOCX, PDF, CSV, TXT, CRT, CER, GED, GPX, XML, PBIX, ZIP, JSON (Maximum size: 15 MB) -- Click **Add File** to confirm your selection. - -### Step 4: File Upload Process -- Upon clicking **Add File**, the selected files are uploaded successfully to the downloads folder within the session. -- These files are treated as variables and their paths are dynamically assigned. -- These variables are named with prefix `FILE_` - -The `FILE_` variable behavior described above applies to **Web tests**. In **App tests** no file variable is created. See [File Upload in App Tests](#file-upload-in-app-tests) below for more information. - -### Step 5: Accessing Uploaded Files -- In the test environment, type upload in the command field. -- Select the required file from the available list using **double-curly braces syntax** (e.g.,`upload {{FILE_IMAGE_1_PNG}} in upload section`). -- Kane AI will detect the appropriate action for uploading and provide relevant options. -- On the right panel, all downloaded files in the session’s downloads folder will be displayed. - -## File Upload in App Tests - -File upload behaves differently in App tests than in Web tests. In a Web test, TestMu AI creates a file variable for each upload. App tests skip this step entirely. No file variable is created, so you never reference uploaded files with double-curly-brace syntax. - -### How it works - -When you upload files to an App test session, TestMu AI injects them directly into the device's storage. The files become part of the device itself, which is why there is no variable to assign, track, or reference in your instructions. - -### Working with injected files - -Most native apps include their own file controls, such as an upload or attach button. Selecting one of these opens the device's system gallery or file picker, where your injected files are already available. You then choose the file you need and continue through the app's normal flow, exactly as a real user would. - -## Downloading Files from Kane AI -### Step 1: Managing Downloaded Files -- Files added during the session will be visible in the downloads section. -- Users can either download all files at once or select specific files for download. - -### Step 2: Reviewing Uploaded Files Post-Test -- After test completion, all uploaded files are recorded as variables. -- These files are also available as attachments in the test summary page for easy reference and download. - -The file upload and download functionality in Kane AI enhances test automation by providing easy access to necessary files. By following these steps, testers can efficiently manage files during their test sessions. diff --git a/static/docs/kaneai-variables-and-parameters.md b/static/docs/kaneai-variables-and-parameters.md deleted file mode 100644 index 18af1c120..000000000 --- a/static/docs/kaneai-variables-and-parameters.md +++ /dev/null @@ -1,22 +0,0 @@ -# Variables & Parameters in KaneAI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -In KaneAI, one of the core principles of effective test automation is the ability to manage dynamic data efficiently. Testing often requires input that changes frequently or is context-dependent, such as URLs, credentials, or specific device configurations. KaneAI offers a set of powerful tools (variables, secrets, smart variables, parameters and datasets) to simplify the process of managing and using dynamic data in your test cases. Each of these options offers unique functionality that helps you keep your tests flexible, secure, and scalable. - -All variables and parameters in KaneAI can be accessed from https://kaneai.lambdatest.com/variables - - Variables - Placeholders that store dynamic values, enabling reuse and reducing redundancy in test cases. - - Secrets - Securely stored sensitive data (e.g., passwords) that are used in tests without exposing them in the code. - - Smart Variables - Context-aware variables that automatically pull data in real-time during test execution. - - Parameters - Values passed into test cases at runtime to customize test execution for different configurations or environments. - - Datasets - Collections of test data used in a test, allowing for data-driven testing by running the same test with different inputs. diff --git a/static/docs/kaneai-while-loops.md b/static/docs/kaneai-while-loops.md deleted file mode 100644 index ebfa20807..000000000 --- a/static/docs/kaneai-while-loops.md +++ /dev/null @@ -1,331 +0,0 @@ -# While Loops in KaneAI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Introduction - -KaneAI supports repeating a block of steps using **While Loops**. A While Loop executes the steps inside its body for as long as a user‑defined condition evaluates to **true**, and re‑checks the condition before every iteration. - -**Availability** -This feature is being rolled out gradually. If you don't see **While Loop** in the slash command menu, contact your TestMu AI support representative to get it enabled. - -## How It Works - -1. Insert a **While Loop** block from the **/** slash command menu. This is the only entry point for adding a loop. -2. Define the loop condition, either in natural language (default) or using the operand & operator editor. -3. Confirm the condition. The loop body opens. -4. Add steps inside the loop body: natural language steps, modules, JavaScript, API, DB, or manual interaction. -5. Click **End While** to finalize the block. Until you finalize, the loop is an authoring‑only placeholder and does not run. -6. Once finalized, each run of the loop follows the same cycle on every iteration: - - The condition is evaluated. - - If the condition is **true**, the loop body runs once. - - If the condition is **false**, the loop exits successfully and execution moves on to the next step in the test. -7. A hard safety cap of **30 iterations per execution** prevents runaway loops. See [Max Iteration Safety Limit](#max-iteration-safety-limit). - -## Prerequisites - -- An active KaneAI authoring session. -- Familiarity with [Conditional Logic (If / Else)](/support/docs/kaneai-conditional-logic/). While Loops reuse the same condition editor. -- Any variables, smart variables, or dataset parameters you plan to reference in the loop condition or body. - -## Step‑by‑step Guide - -### Step 1: Insert a While Loop Block - -A While Loop can only be added through the **/** slash command menu. - -1. In your authoring session, press **/** to open the slash command menu. -2. Select **Add While Loop**. -3. KaneAI inserts an empty While Loop block at the current position in your test flow. The block contains a header, an empty condition editor, and an **End While** control. - -A newly inserted block starts in an empty state with no condition and no body steps. - -### Step 2: Define the Loop Condition - -The condition editor opens in **natural language mode** by default, matching KaneAI's NL‑first authoring philosophy. You can switch to **operand & operator mode** at any time. - -#### Option A: Natural Language - -Type your condition as a plain‑English sentence. KaneAI interprets the intent and re‑evaluates it at runtime. Examples: - -- `while the loading spinner is visible` -- `while the Next button is present` -- `while {{count}} is greater than 0` -- `while {{job_status}} is not "complete"` - -#### Option B: Operand & Operator - -Build the condition explicitly using a **left operand**, a **comparison operator**, and a **right operand**: - -1. Enter a **left operand**: this can be a variable (e.g., `{{active_count}}`), a dataset parameter (e.g., `${max_attempts}`), extracted text, or a literal value. -2. Choose a **comparison operator** from the dropdown: - -| Operator | Description | -|---|---| -| `==` | Equals | -| `!=` | Not equals | -| `>` | Greater than | -| `<` | Less than | -| `>=` | Greater than or equal to | -| `<=` | Less than or equal to | -| `contains` | Left operand contains the right operand | -| `not_contains` | Left operand does not contain the right operand | -| `starts_with` | Left operand starts with the right operand | -| `ends_with` | Left operand ends with the right operand | -| `is_visible` | Target element is currently visible on the page | -| `is_not_visible` | Target element is not currently visible on the page | - -3. Enter the **right operand**: a literal, variable, dataset parameter, or extracted value. - -Both operands support variables (`{{var_name}}`), dataset parameters (`${param_name}`), extracted text, and literal values. - -#### Confirming the Condition - -Confirm the condition to open the loop body. Once confirmed, the condition remains editable until the loop begins executing. You can reopen it to adjust operators, operands, or the natural‑language expression. - -### Step 3: Add Steps Inside the Loop Body - -The loop body becomes available only after the condition is confirmed. Inside the body you get the standard KaneAI step input, the same one used in the main authoring flow, where you can type a natural‑language step or press **/** to open the slash command menu. - -You can add any of the following inside a loop body: - -- **Natural language steps**: type your instruction in plain English. -- **Modules**: reuse existing modules by adding them via the slash command menu. -- **JavaScript steps**: execute custom JS for data extraction, array handling, or index management. -- **API steps**: make API calls as part of the iteration. -- **DB steps**: run database queries inside each iteration. -- **Manual interaction**: perform actions directly on the browser within the loop body using the manual interaction control. -- **If / Else blocks**: insert a conditional block inside the loop body via the slash command menu to add branching logic within iterations. - -### Step 4: Finalize the Loop with "End While" - -When your loop body is ready, click **End While** at the bottom of the block to finalize it. - -**Finalization is required** -Unlike If / Else blocks (which evaluate the moment you confirm a condition), a While Loop **accumulates** steps first and is only executed once you finalize it with **End While**. Until finalization it exists as an authoring‑only placeholder so you can keep adding body steps. The **End While** button is active only after the condition is confirmed **and** at least one step has been added to the body. - -Clicking **End While** starts execution immediately. KaneAI re‑evaluates the condition before every iteration and stops when the condition becomes false or when the [max iteration safety limit](#max-iteration-safety-limit) is reached. - -## Executing a While Loop - -Once a While Loop has been finalized, KaneAI runs it inline in the Playground. Each iteration evaluates the condition, runs the body steps in order if the condition is true, and then re‑evaluates. Execution finishes the moment the condition becomes false or the safety cap is reached. - -**While the loop is running**, the block header shows the current iteration so you can follow progress as it happens. A **Pause While** control is available during execution, which you can use to interrupt a run and inspect intermediate state. - -**When the loop finishes**, the block switches to a completed state and reports the final outcome of the condition along with the total number of iterations that ran. Every body step inside the block reflects the status it had on the most recent iteration, so you can review what happened in the final pass without re‑running the test. - -## Using the `{{loop_counter}}` Variable - -Every While Loop exposes a built‑in **`{{loop_counter}}`** variable inside its body. The counter is **1‑indexed** and increments by 1 on each iteration (first iteration → `1`, second → `2`, and so on). - -`{{loop_counter}}` is available to all step types inside the loop body, including natural language steps, JavaScript snippets, API and DB steps, and **element selectors**. - -**Example: clicking each row of a table one at a time:** - -> Click the row at position `{{loop_counter}}` in the bookings table - -KaneAI substitutes the counter into the underlying element selector before it is resolved, so a selector like `tr:nth-of-type({{loop_counter}})` targets row 1 on the first iteration, row 2 on the second, and so on. - -## Using Dataset Parameters and Variables in Loops - -All existing KaneAI [variables](/support/docs/kane-ai-using-variables/) and [dataset parameters](/support/docs/kane-ai-using-datasets/) are fully usable inside loop conditions and loop body steps. No special configuration is required. - -- **Variables** use the `{{variable_name}}` syntax, for example, `{{active_count}}`, `{{job_status}}`, or a value extracted from a previous step. -- **Dataset parameters** use the `${parameter_name}` syntax, for example, `${max_retries}`. - -**Dynamic iteration count pattern.** You can extract a count from the page or an API response into a variable using a JavaScript step, then reference that variable in the While condition: - -1. Add a JS step that reads the count and stores it in `{{total}}`. -2. Add a While Loop with the condition `{{loop_counter}} <= {{total}}`. -3. Add the steps you want to repeat inside the body. - -**Operand validation** -Both operands of a condition cannot be parameters at the same time. If one operand is a parameter (for example, a dataset parameter like `${max_retries}`), the other operand must be a runtime‑updated value: a local variable, a smart variable, a value read from the UI, or a literal. If both operands are parameters, KaneAI rejects the condition with the `BOTH_OPERANDS_AS_PARAMETERS` error. See [Error Messages and Troubleshooting](#error-messages-and-troubleshooting) for details. - -## Element Presence / Absence Conditions - -While Loop conditions can reference the visibility of UI elements directly, useful for polling, waits, and spinner handling. - -- **Natural language mode:** - - `while the loading spinner is visible` - - `while the Next button is present` -- **Operand & operator mode:** use the `is_visible` or `is_not_visible` operator with an element reference as the operand. - -Example use case: after triggering a data‑heavy action, repeat a short wait step while a loading spinner element is visible, so the test resumes automatically the moment the spinner disappears. - -**Automatic query refresh** -For UI‑based conditions you do **not** need to add a manual query step inside the loop body. KaneAI automatically inserts the underlying visual / DOM query and re‑runs it at the start of every iteration, so the condition is always evaluated against fresh page state. - -## Max Iteration Safety Limit - -Every While Loop is capped at a hard maximum of **30 iterations per execution**. If the condition has not become false by the 30th iteration, execution stops and the step is marked as failed with the message: - -> While loop exceeded maximum iterations (30) - -This safety cap prevents a mis‑configured or non‑terminating condition from consuming unlimited execution time. - -- The maximum iteration value is **not shown** during normal authoring or execution. Only the current iteration number is displayed. -- When a loop hits the cap, execution stops and an error is reported. The error clearly indicates that termination was caused by the safety limit and not by the exit condition becoming false. -- The underlying error code is `LOOP_MAX_LIMIT_REACHED`. - -Use the safety cap as a last line of defense. Design your condition so the loop exits naturally well before the 30th iteration. If your scenario legitimately requires more than 30 iterations, reach out to your TestMu AI support representative. - -## Common Use Cases - -- **Deactivate active items.** Repeat a deactivation step `while {{active_count}} > 0` to drain a queue of active bookings, alerts, or records. -- **Poll a status endpoint.** Trigger an async operation, then poll `while {{job_status}} != "complete"` with a short wait inside the body and the max iteration cap as a safety net. -- **Wait for a loading spinner.** Use the natural‑language condition `while the loading spinner is visible` to resume the test the moment the spinner disappears. -- **Validate paginated table rows.** Iterate `while the Next button is enabled` (or `while {{has_next_page}} == "true"`) and validate each page's rows inside the body. -- **Click each row by position.** Use `{{loop_counter}}` inside element targeting (`tr:nth-of-type({{loop_counter}})`) to walk a table row by row without hand‑writing selectors. - -## Tips and Recommendations - -- **Design for a natural exit.** Write conditions that terminate deterministically. Treat the max iteration limit as a safety net, not a control flow mechanism. -- **Keep loop bodies small.** Extract repeated logic into a [module](/support/docs/kane-ai-modules/) and call the module inside the loop body instead of duplicating steps. -- **Validate the body first.** Author the steps you intend to repeat outside a loop first, confirm they work, and then move them inside the While block. -- **Use `{{loop_counter}}` for position‑based targeting.** Prefer `{{loop_counter}}` in selectors over hand‑writing `nth-of-type(…)` indexes. It keeps the intent obvious and avoids off‑by‑one errors. -- **Extract dynamic counts into variables.** Read the number of items from the page or API into a variable first, then reference it in your condition, rather than hard‑coding an iteration count. - -## Editing an Existing While Loop - -Once a While Loop has been finalized, you can still change it. The loop itself stays in sync automatically. - -- **Add, remove, or reorder body steps.** Use the edit instruction action on any step inside the loop; the loop updates automatically to reflect the change. -- **Change the condition.** Edit the While Loop step directly. The new condition applies on the next test run. -- **Finalize later.** If you closed the authoring view before clicking **End While**, the loop remains unfinalized. Re‑open the test, add the remaining body steps, and click **End While** when ready. -- **Loops inside modules.** When a While Loop is part of a [module](/support/docs/kane-ai-modules/), any change to the loop condition or body bumps the module version so other tests using that module pick up the new behavior on their next run. - -## Nesting Rules - -While Loops can contain **regular If / Else conditional blocks** inside their body, which lets you branch within each iteration. The maximum supported nesting depth is **two levels**: a While Loop containing a conditional, which in turn contains ordinary steps. - -The following nesting patterns are **not supported**: - -- A While Loop directly inside another While Loop. -- A While Loop directly inside a conditional branch (If, Else‑If, or Else). - -## Limitations - -- **No For‑Each loop construct.** Collection iteration can be handled using a While Loop with JavaScript steps for index management. -- **Hard cap of 30 iterations per execution.** The safety cap is fixed and not user‑configurable. Contact support if you have a scenario that legitimately requires more. -- **Nested While Loops are not supported.** You cannot place a While Loop inside another While Loop (see [Nesting Rules](#nesting-rules)). -- **While Loops cannot live inside conditional branches.** A While Loop cannot be placed inside an If / Else branch. -- **No Break / Continue commands.** There is no way to exit a loop early or skip to the next iteration; structure your condition to terminate naturally. -- **Natural language cannot create a loop.** Phrases like "repeat this 10 times" or "while the spinner is visible, do X" will not create a loop. You must use the slash command and select **While Loop**. KaneAI surfaces this as the `WHILE_NOT_SUPPORTED_VIA_NL` error. -- **New While Loops cannot be created while a test is paused.** The **/** slash command menu and the **+ Add step** option do not offer **Add While Loop** in the Draft state. A While Loop that already exists in the test can still be edited while paused. -- **Both operands in a condition cannot be parameters at the same time.** At least one side must be a runtime‑updated value. See `BOTH_OPERANDS_AS_PARAMETERS` in [Error Messages and Troubleshooting](#error-messages-and-troubleshooting). -- **Local variables must be defined inside the block.** If a local variable referenced inside a While block was created outside the block, KaneAI shows an error when you click **End While**. - -## Error Messages and Troubleshooting - -Below are the user‑facing error codes you may encounter when authoring or running a While Loop, together with what each one means and how to resolve it. - -### 1. `LOOP_MAX_LIMIT_REACHED` - -**Error message** - -> While loop exceeded maximum iterations. This may be due to an issue with the loop condition or the actions within the loop. - -**Suggestion** - -> Review the loop condition and the actions being performed within the loop to ensure they are correct and will allow the loop to terminate properly. - -**When you see it.** The loop ran the maximum allowed number of iterations (**30**) without the condition ever becoming false. - -**How to resolve.** - -- Verify that the condition *can* become false. Typical causes: - - A variable used in the condition is never updated inside the body. - - The UI state being checked (element visible, value present, etc.) is never reached. -- Add or correct a body step that advances the condition toward termination. -- If your scenario legitimately requires more than 30 iterations, contact your TestMu AI support representative. - -### 2. `INFINITE_LOOP_DETECTED` - -**Error message** - -> The while loop appears to be an infinite loop. Please review the loop condition and the actions within the loop to ensure that the loop will terminate properly. - -**Suggestion** - -> Review the loop condition and the actions being performed within the loop to ensure they are correct and will allow the loop to terminate properly. - -**When you see it.** KaneAI detected a condition / body combination that cannot terminate, typically a condition that is independent of anything the body changes, or a comparison that is always true. - -**How to resolve.** - -- Ensure the body contains at least one step that changes a value referenced by the condition, for example, increment a counter, click a control that updates the UI state, or wait for a status transition. -- If the condition uses constants only (e.g. `1 == 1`), rewrite it so it depends on a variable or UI state that evolves during the loop. - -### 3. `BOTH_OPERANDS_AS_PARAMETERS` - -**Error message** - -> Both operands in the while loop condition are parameters. Only one operand should be a parameter. - -**Suggestion** - -> Please modify the loop condition to ensure that only one operand is a parameter. - -**When you see it.** Both sides of the loop condition are test parameters (for example, `${max_retries} > ${default_retries}`). Parameter values are fixed for the lifetime of a run, so such a condition cannot change between iterations. It would either be an infinite loop or never enter. - -**How to resolve.** Replace one operand with a runtime‑updated value: - -- A counter variable incremented in the body. -- A value read from the UI via a query. -- A literal. - -Example valid forms: `{{counter}} < ${max_retries}`, `${status} == "ready"`, `{{cart_empty}} == false`. - -### 4. `WHILE_NOT_SUPPORTED_VIA_NL` - -**Error message** - -> Looping is supported via slash commands only. Type / and select While Loop to add a loop. - -**Suggestion** - -> Use the while loop button in slash commands to create a loop instruction. - -**When you see it.** You added a natural‑language step that describes a loop, for example, *"repeat until the cart is empty"*, *"while the spinner is visible, do X"*, or *"keep clicking Next"*. The natural‑language pipeline does not expand these phrases into loops. - -**How to resolve.** - -1. Remove the looping phrase from the plain‑English step. -2. Open the slash menu (`/`) and select **While Loop**. -3. Enter the loop condition in the While Loop authoring dialog. -4. Add the per‑iteration action as a body step inside the loop. - -## FAQ - -### How do I exit a While Loop early? - -Early exit (Break) and skip‑to‑next‑iteration (Continue) are not supported. Structure your exit condition (for example, combine it with an additional flag variable you set from inside the loop) so the loop terminates naturally on the next re‑evaluation. - -### Can I nest a While Loop inside another While Loop? - -No. Nested loops are not supported, and a While Loop cannot be placed inside an If / Else branch either. A While Loop **can** contain conditional (If / Else) blocks in its body. See [Nesting Rules](#nesting-rules). If you need multi‑level iteration, split the logic across multiple test cases or use a single loop combined with JavaScript for inner bookkeeping. - -### What happens if my condition is already false on the first check? - -The loop body is skipped entirely, zero iterations run, and execution continues with the next step after the block. - -### What is the maximum number of iterations allowed? - -Every While Loop is capped at a hard maximum of **30 iterations per execution**. The limit is fixed and not user‑configurable. If a scenario legitimately requires more, reach out to your TestMu AI support representative. - -### Can I use `{{loop_counter}}` outside the loop body? - -No. `{{loop_counter}}` is only defined inside the While Loop body it belongs to. If you need to reference the final iteration count after the loop completes, store it in a variable of your own inside the loop body. - -### Does the Test Summary page show each iteration? - -No. The live iteration counter is a Playground‑only view. On the Test Summary page, the While block shows only its final completed state along with the total number of iterations that ran. Step‑level screenshots and statuses inside the block reflect data from the most recent iteration that was executed. - -### Do dataset parameters and variables work inside loops? - -Yes. Both `{{variable}}` and `${dataset_parameter}` syntaxes are fully supported inside loop conditions and body steps with no special configuration. The only restriction is that both operands in a single condition cannot be parameters at the same time. At least one must be a runtime‑updated value such as a local variable, a smart variable, a UI‑derived value, or a literal. See the `BOTH_OPERANDS_AS_PARAMETERS` entry in [Error Messages and Troubleshooting](#error-messages-and-troubleshooting). - -### Why is nothing happening when I click "End While"? - -The **End While** action is only available once the condition has been confirmed **and** at least one step exists in the loop body. Confirm the condition and add at least one body step, then **End While** becomes available. diff --git a/static/docs/karate-report.md b/static/docs/karate-report.md deleted file mode 100644 index f984d3722..000000000 --- a/static/docs/karate-report.md +++ /dev/null @@ -1,23 +0,0 @@ -# Karate Report - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Karate framework supports generating detailed test execution reports that help visualize test results and insights. On HyperExecute, you can run your Karate tests and configure it to generate and download these reports seamlessly. - -> Ensure your project is configured with the **Karate framework** and all test scripts are in place. - -## Steps to Generate Karate Reports on HyperExecute - -**Step 1:** Configure all the necessary dependencies and configurations to your project. - -**Step 2:** Configure the report parameters in the HyperExecute YAML file. - -```yaml -report: true -partialReports: -location: target -type: html -frameworkName: karate -``` - -**Step 3:** Now execute your job by triggering the HyperExecute CLI. You can visit the HyperExecute dashboard to download the report after job completion. diff --git a/static/docs/karma-integration-with-testmu.md b/static/docs/karma-integration-with-testmu.md deleted file mode 100644 index 2343efb3f..000000000 --- a/static/docs/karma-integration-with-testmu.md +++ /dev/null @@ -1,108 +0,0 @@ -# Selenium With Karma - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Run Karma tests on the TestMu AI cloud grid. This guide covers setup, running a sample test, configuring capabilities, and testing locally hosted pages. - -**Sample repo** -All the code used in this guide is available in the sample repository. - - View on GitHub - -## Prerequisites - -Complete these steps before integrating Karma with TestMu AI. - -1. Create a [TestMu AI account](https://www.testmuai.com/register/?redirectTo=https://accounts.lambdatest.com/dashboard) and get your username and access key from the dashboard. -2. Install **NodeJS** and **npm** from [nodejs.org](https://nodejs.org/en/). -3. Install the latest version of **JavaScript**. -4. Install [Angular CLI](https://cli.angular.io/): `npm install -g @angular/cli` - -## Step 1: Clone the Sample Project - -Clone the TestMu AI Karma sample repository to your local machine. - -```bash -git clone https://github.com/LambdaTest/angular-karma-sample -cd angular-karma-sample -``` - -Install the required dependencies and Karma CLI globally: -```bash -npm install -g karma-cli -npm install -``` - -## Step 2: Set Your Credentials - -Set your TestMu AI username and access key as environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -## Step 3: Configure Your Test Capabilities - -Update the capabilities in the `karma.conf.js` file to define the browser and platform settings. - -``` js -customLaunchers: { -chrome: { -base: 'WebDriver', -config: webdriverConfig, -browserName: 'chrome', -platform: 'windows 10', -version: '71.0', -name: 'Karma With Heartbeat', -user: process.env.LT_USERNAME, -accessKey: process.env.LT_ACCESS_KEY, -pseudoActivityInterval: 5000 // 5000 ms heartbeat -} -} -``` - -Generate capabilities for your test requirements with the [Capabilities Generator](https://www.testmuai.com/capabilities-generator/). - -## Step 4: Run the Test - -Execute the test using the following command. - -```bash -karma start karma.conf.js -``` - -**Parallel tests:** -```bash -ng test -``` - -## Step 5: View Your Results - -After running the test, view your results on the [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/). - -The dashboard provides: -- Video recordings of each test session -- Screenshots captured at each step -- Console logs from the browser -- Network logs for debugging -- Detailed command logs - -## Run Karma Tests Using Agent Skills - -Use AI coding assistants to generate and run Karma tests with the TestMu AI Agent Skill. - -The [karma-skill](https://github.com/LambdaTest/agent-skills/tree/main/karma-skill) is part of [TestMu AI Agent Skills](https://github.com/LambdaTest/agent-skills/) - structured packages that teach AI coding assistants how to write production-grade test automation. - -Install the skill: - -```bash -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/karma-skill .claude/skills/ - -# For Cursor / Copilot -cp -r agent-skills/karma-skill .cursor/skills/ -``` - -Install all available framework skills at once by cloning the repository directly into your tool's skills directory (e.g., `.claude/skills/`, `.cursor/skills/`). diff --git a/static/docs/katalon-integration-with-hyperexecute.md b/static/docs/katalon-integration-with-hyperexecute.md deleted file mode 100644 index 0e5f1b154..000000000 --- a/static/docs/katalon-integration-with-hyperexecute.md +++ /dev/null @@ -1,70 +0,0 @@ -# Katalon Integration with HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -[Katalon Studio](https://www.katalon.com) is an automation tool for testing web, API, mobile, and desktop applications. It offers a flexible automation platform that fits teams and projects of all sizes and purposes. It also helps uses create tests, execute them and integrate them with the CI/CD ecosystem. - -> Katalon Studio Integration with HyperExecute enables cross-platform testing, from web automation on Windows, macOS, and Linux to mobile automation on Android and iOS. - -It also comes with a built-in interface for codeless automation testing and validating the UI elements of your web application. - -## Prerequisites - -- Ensure that you have integrated Katalon Studio with TestMu AI. Follow the steps on this [page](/support/docs/katalon-integration-with-testmu/) to do so. - -- Ensure that you have the necessary licenses required to use Katalon Studio. You need the Katalon Runtime License to integrate Katalon with HyperExecute. - -## Integrating Katalon Studio with HyperExecute - -1. You must generate a command from Katalon to run it through the command line. To do so: - - a. Generate a command for Console mode through the Katalon UI. Click on the **Command Builder** button on the top of the screen. - - b. Select the test suite that you want to use and click on **Generate Command**. - - c. Click on the **Copy to Clipboard** button to copy the command. - -2. Clear the contents of the .gitignore file in the root directory of your Katalon project. - -3. Download the HyperExecute CLI binary in the root directory of your Katalon project. - -4. Create the hyperexecute.yaml file in the root directory of your Katalon project with the given parameters: - -```yaml -version: 0.1 -globalTimeout: 90 -testSuiteTimeout: 90 -testSuiteStep: 90 - -autosplit: true - -runson: win - -tunnel: true -tunnelOpts: -global: true - -concurrency: 2 - -pre: -- pip install requests # To install pip request -- python installer.py # To install the certs in the Machine - -runtime: katalon - -testDiscovery: -type: raw -mode: remote -command: grep -lr 'testSuiteGuid' */*.ts | sed 's/:.*//' | sed 's/.ts//g' - -testRunnerCommand: katalonc `-noSplash `-runMode=console `-projectPath="G:\foreman\Hyperexecute_Sample\Hyperexecute_Sample.prj" `-retry=0 `-testSuitePath="$test" `-browserType="firefox" `-executionProfile="default" `-apiKey="aaa9402b-6a2e-4621-a4c3-05fe356f5aad" `-`-config `-webui.autoUpdateDrivers=true - -retryOnFailure: false -maxRetries: 1 -``` - -Paste the command that you generated from the Katalon UI to fill the **testRunnerCommand** field in the _hyperexecute.yaml_ file. - -5. Run the Hyperexecute CLI commands from the same path as your Katalon project. - -You have successfully integrated your Katalon Studio project with HyperExecute. diff --git a/static/docs/katalon-integration-with-testmu.md b/static/docs/katalon-integration-with-testmu.md deleted file mode 100644 index 8ea4812a8..000000000 --- a/static/docs/katalon-integration-with-testmu.md +++ /dev/null @@ -1,39 +0,0 @@ -# Katalon Integration With TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -[Katalon Studio](https://www.katalon.com) is an automation tool for web, API, mobile, and desktop applications testing. Katalon Studio offers a flexible automation platform that fits teams and projects of any size, for any purpose. It supports users from creating tests, execution, reports to seamless integration with the CI/CD ecosystem. - -Katalon Studio offers cross-platform testing, from Windows, macOS, and Linux for web automation, to Android and iOS for mobile automation. It also comes with a built-in intuitive interface for codeless automation testing and validating the UI elements of your web application. - -This integration between TestMu AI and Katalon Studio enables you to automate the cross browser testing process through the Selenium Grid of more than 2,000 browsers. - -As a result, you’ll be able to expand your test coverage, analyze and extract insightful test reports, and enhance your team collaboration. - -## How To Integrate TestMu AI With Katalon? - -For integrating TestMu AI with Katalon, make sure to run your test scripts through the ‘Remote’ desired capability. You can select all your desired capabilities from [TestMu AI Capabilities Generator](https://www.testmuai.com/capabilities-generator/). - -**Step 1:** Login to TestMu AI. If you haven’t signed up yet, click here. - -**Step 2:** Visit your Profile on TestMu AI to fetch your Username and Access Key value. - -**Step 3:** You would require these values for generating a secure remote connection with Katalon through TestMu AI remote URL. - -```json -Remote URL = "http://username:accessKey@hub.lambdatest.com/wd/hub" - -/* Below is an example remote URL based on the image shared above in Step 2.*/ - -Example remote URL = "http://harshitp:ABCD1234PQRS@hub.lambdatest.com/wd/hub" -``` - -**Step 4:** Visit TestMu AI Capabilities Generator for declaring desired capabilities. For instance, if you wish to test on macOS High Sierra using Firefox browser then desired capabilities for Java frameworks would look like below. - -The below execution is set up on Katalon version 5.10.1. - -**Step 5:** In Katalon Studio, open ‘Remote’ settings for passing TestMu AI settings. Go to Project → Settings → Desired Capabilities → Remote. - -Once you select ‘Remote’ and provide the basic parameters then you only need to execute your automation tests. You can do that by going to Action → Run → Remote. - -If you have any questions, feel free to reach out to us through our **24×7 chat support**. Happy testing! diff --git a/static/docs/katalon-integration.md b/static/docs/katalon-integration.md deleted file mode 100644 index 38dc639e7..000000000 --- a/static/docs/katalon-integration.md +++ /dev/null @@ -1,133 +0,0 @@ -# Using Katalon Studio with Real Devices - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -[Katalon Studio](https://katalon.com/) is an automation tool for web, API, mobile, and desktop application testing. Katalon Studio offers a flexible automation platform that fits teams and projects of any size, for any purpose. It supports users from creating tests, execution, and reports to seamless integration with the CI/CD ecosystem. - -Katalon Studio offers cross-platform testing, from Windows, macOS, and Linux for web automation, to Android and iOS for mobile automation. It also comes with a built-in intuitive interface for codeless automation testing and validating the UI elements of your web application. - -This integration between TestMu AI and Katalon Studio enables you to automate the cross-browser testing process through the Selenium Grid of more than 2,000 real devices. - -As a result, you’ll be able to expand your test coverage, analyze and extract insightful test reports, and enhance your team collaboration. - -## Objective - -By the end of this topic, you will know: - -1. How can we use Katalon Spy with TestMu AI. -2. How can we record interactions using Katalon Record Mobile and create Automation Scripts. -3. Executing Katalon Scripts using TestMu AI’s real device cloud. - -## Prerequisites for Getting Started - -Before you can start performing App automation using Katalon Studio with Lambdatest, you would need to follow these steps: - -- You will need a TestMu AI username and access key. To obtain your access credentials, [purchase a plan](https://billing.lambdatest.com/billing/plans) or access the [automation dashboard](https://www.testmuai.com/login/?redirectTo=https://appautomation.lambdatest.com/). -- [Katalon Studio](https://katalon.com/) -- Access to an Android app (.apk) or an iOS app (.ipa). - -- If you do not have any **.apk** or **.ipa** file, you can run your sample tests on TestMu AI by using our sample :link: [Android app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk) or sample :link: [iOS app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_ios.ipa). -- Response of above cURL will be a **JSON** object containing the `APP_URL` of the format - ``lt://APP123456789123456789`` and will be used in the next steps. - -## Katalon Spy with TestMu AI - -### 1. Katalon Studio Web Interface -Once you have started Katalon Studio, load your project/ one of the sample projects provided by Katalon using the [web interface](https://my.katalon.com/home). - -Once the project has been created, we can now move to the Katalon Studio on Windows/Mac. - -### 2. Katalon Studio on PC -Open the Katalon Studio on the Windows/Mac: - -Click on File > Open Project. - -### 3. Fetch Project & add credentials -Now we can click on Fetch Project > which will list all the Projects and make a selection. - -Add in the Username & Personal Access Token & click Ok. - -Now we can see that there are 2 icons: - -The first one is the Katalon Spy and the next one is the Katalon Mobile Record. - -### 4. Katalon Spy Kick-start -To start using Katalon Spy with TestMu AI, just click on the Katalon Spy icon dropdown and select Remote Devices. - -### 5. Edit Configuration -Click on Edit Configuration - -#### a. Hub URL -Now we need to fill in the details for Remote Server URL that will be found on your App Automation Dashboard. - -Copy the Hub URL and add it to the Remote Server URL from the App Automation Dashboard. - -For the rest of the details, quickly head over to [Capabilities Generator](https://www.testmuai.com/capabilities-generator/) and now we can proceed with the next steps. - -#### b. Desired Capabilities -Add in all the [Desired Capabilities](https://www.testmuai.com/capabilities-generator/) as shown below, then click Apply and Close. - -**Mandatory Capabilities:** -```java -"platformName", "android"; -"deviceName", "Pixel 6"; -"platformVersion", "12"; -"isRealMobile", true; -``` - -Now for the last step we need the App ID. - -Upload your **_iOS_** application (.ipa file) or **_android_** application (.apk file) to the TestMu AI servers using our **REST API**. You need to provide your TestMu AI **Username** and **AccessKey** in the format `Username:AccessKey` in the **cURL** command for the authentication. Make sure to add the path of the **appFile** in the cURL request if required. Here is an example cURL request to upload your app using our REST API: - - **Using App File from System:** - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "appFile=@"/Users/macuser/Downloads/proverbial_android.apk"" `} - -**Using App URL:** - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "url=:https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk" -F "name=Proverbial_App" `} - -### 6. App ID -Now just add in your App ID and click Start - -### 7. Run Mobile Spy -The session now starts and it’ll look like this: - -We can now begin to select the elements on the screen and Capture them using the Katalon Studio and the captured items will be shown as shown below: - -## Katalon Mobile Recorder with TestMu AI - -### 1. Starting off with Katalon Mobile Recorder -To start using Katalon Mobile Recorder with TestMu AI, just click on the Katalon Mobile Recorder the icon dropdown and select Remote Devices. - -### 2. Kick Start Mobile Recorder -Now we have already added the Configuration while starting the Katalon Mobile Spy, so we can now just add the Cloud Application ID and click Start. - -### 3. Record actions -Now we can start interacting with the device and it will now start to Record all the Actions that we perform on the Device. - -### 3. Save Recorded actions -Here’s how it would look like once all the steps are recorded. Post this we can Save this Recorded Steps in the form of a Script and save it. - -You’ll now be prompted with where do you want to save this script, select the desired location and save it. - -### 4. Saved actions -Once saved, we can now see how it looks like! - -Now for the final task, let’s run this saved script. - -## Executing Katalon Scripts using TestMu AI’s Cloud Devices - -### 1. Run Saved actions -Click on the dropdown next to the Run Test icon on Katalon Studio and click Remote. - -### 2. Test Executing -Now we can see that the test starts running. - -### 3. View test execution - -Once you have run your tests, you can view the test execution along with logs. You will be able to see the test cases passing or failing. You can view the same at [TestMu AI Automation](https://www.testmuai.com/login/?redirectTo=https://appautomation.lambdatest.com/build). - -## Additional Links - -- [Advanced Configuration for Capabilities](/support/docs/desired-capabilities-in-appium/) -- [Automation Capabilities Generator](https://www.testmuai.com/capabilities-generator/) diff --git a/static/docs/key-features-of-hyperexecute.md b/static/docs/key-features-of-hyperexecute.md deleted file mode 100644 index 7759ae18c..000000000 --- a/static/docs/key-features-of-hyperexecute.md +++ /dev/null @@ -1,79 +0,0 @@ -# Key Features of HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -HyperExecute provides you with the tools that you need to run your tests at the fastest speeds possible. In addition to all of the core features that TestMu AI cloud provides to your tests (for example: detailed logs, smart CI features, network insights, video recording, access to browsers & platforms on the cloud), HyperExecute takes a step forward to improve your experience. It supports all [programming languages and major test automation frameworks](/support/docs/hyperexecute-supported-languages-and-frameworks/). - ->Check out [TestMu AI HyperExecute on Microsoft Azure Marketplace](https://azuremarketplace.microsoft.com/en-us/marketplace/apps/lambdatestinc1584019832435.hyper_execute?exp=ubp8&tab=Overview) - - Test Splitting and Multiplexing - Use Smart Auto Split Strategy, Matrix Strategy or Hybrid Strategy modes to split and execute tests as per your requirements. - - Detailed Test Reports and Logs - Get a detailed test report for every job and stream the terminal logs in real time to understand everything that happened with your test. - - FailFast - Run your jobs faster or fail them quickly to receive instant feedback and save your test time. - - Jobs Prioritization - Label the jobs that you want to run first on HyperExecute by giving them a higher priority. - - Test Artifact Management - HyperExecute also manages all of your test artifacts and provides you with the option to implement smart workflows on your test code without modifying it. - - AI Native Root Cause Analysis - Root Cause Analysis and Error Classification features in HyperExecute to view different kinds of errors and directly land on corrective measures or fixes. - - Background Services - Triggers long-running Application Servers tasks like running WebApps or Databases and generates the logs report in the Dashboard. - - Auto Healing - Auto Healing Feature allows you to automatically rеcovеr from cеrtain typеs of failurеs during thе еxеcution of your tеst scripts. - - Projects - Setup your custom projects using HyperExecute GUI itself and initiate the tests execution from the platform only. - - Jobs Archiving - Jobs Archiving allows you to archive the non important jobs so that you and your team can focus only on required ones. - - Test Muting - Mute scenarios that have been continuously failing for a pre-defined number of times, ignore expected failures, achieve better runtimes and faster feedback on the executed Jobs. - - Task Metrics - Track your CPU, and Memory utilization of the virtual machines for a seamless user experience. - - Workflows - Schedule and chain your HyperExecute jobs to automate recurring test runs and orchestrate multi-job pipelines. - - Rerun Failed Tests - Re-execute only the failed tests from a previous job to quickly validate fixes without rerunning the entire suite. - -## Availability - -HyperExecute is available on Windows, Linux and Mac. You can use HyperExecute in more than 60 regions that are supported by Microsoft Azure. - -## Integrations -To ensure that your testing environment can be optimised, HyperExecute provides you with multiple [Integrations](/support/docs/integration-with-hyperexecute/), such as: - - Integrations with CI/CD Tool - Explore a range of CI/CD tool integrations to optimize your testing pipeline. - - Integration with Products - Effortlessly integrate with products like Tosca, Katalon and more for streamlined test execution and management. - - Integration with TestMu AI Products - Integrate seamlessly with other TestMu AI products sucha as Smart UI or Real Device and enhance your testing environment's efficiency. - -## Security -HyperExecute prioritises the safety and security of your data above everything else. The entirety of your data is encrypted for the entire duration of your testing lifecycle. HyperExecute is compliant with the industry leading security standards offering you: - -- **Data Encryption and Compliance :** HyperExecute prioritizes data safety through full encryption, adhering to industry standards like ([SOC2, GDPR, and CCPA](https://www.testmuai.com/security)). -- **Dedicated Proxy Server Option :** Ensure secure connections by utilizing dedicated proxy servers for your testing activities. -- **Automatic Tunnel Management :** Maintain data security for private websites with automatic tunnel management through Network Address Translation (NAT) instances. - -## Enterprise -For enterprises that want to keep everything behind their own firewalls, HyperExecute comes with [private cloud facilities](/support/docs/hyperexecute-private-cloud-setup/). You can set up your own cloud runners and storage which ensures that no data leaves your organisation’s firewalls. - -You can use HyperExecute’s [Quickstart feature](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/quickstart) to run your first test, and go through our Concepts page to understand the ins and out of HyperExecute. - -[Frequently asked questions](/support/docs/hyperexecute-feature-faqs/) about HyperExecute Features. diff --git a/static/docs/lambda-exceptions.md b/static/docs/lambda-exceptions.md deleted file mode 100644 index e8c8ce7ae..000000000 --- a/static/docs/lambda-exceptions.md +++ /dev/null @@ -1,264 +0,0 @@ -# View Assertion Errors On TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -While handling GET requests in your Selenium scripts, assertions might fail. Use the Lambda Exceptions feature to view these assertion errors as exceptions and handle them more effectively. - -## What Are Lambda Exceptions? - -Lambda Exceptions let you manage and display GET request errors like AssertionError on the dashboard. - -Lambda Exception is a feature of TestMu AI that lets you manage and handle GET request errors, like AssertionError. The error gets displayed as an exception in the Exception tab of your test. - -For example: - -## How to Use Lambda Exception - -Catch assertion errors in GET requests and push them to the dashboard using JavascriptExecutor. - -While performing assertions on GET requests, if the assertion fails, an AssertionError exception is thrown. Refer to the below Java syntax to catch this error and push it to TestMu AI using Lambda Exceptions. - -```javascript -try { -// some GET request -Assert.assertEquals(ActualValue, ExpectedValue); -} catch (AssertionError e) { -Status = "failed"; -exceptionCapture.add(e.getMessage()); -((JavascriptExecutor) driver).executeScript("lambda-exceptions", exceptionCapture); -} -``` - -In the above syntax, when the AssertionError is caught in the catch{} block, the exception message is added as an array of Strings. This message is then pushed to TestMu AI using the Lambda Exceptions feature via the JavascriptExecutor command. - -## Lambda Exception Demo - -Walk through a sample script that visits a URL, fetches the title, and asserts it against an expected value. - -For demo purposes, this script will: - -* Visit [www.testmuai.com](https://www.testmuai.com) -* Fetch the title -* Assert the title with the expected value - -### View Exception on the Dashboard Using Lambda Exception - -Below is the full code for this script, using TestNG framework in Java. - -```javascript -package com.lambdatest; - -import java.net.MalformedURLException; -import java.net.URL; -import java.util.ArrayList; - -import org.openqa.selenium.By; -import org.openqa.selenium.JavascriptExecutor; -import org.openqa.selenium.NoSuchElementException; -import org.openqa.selenium.remote.DesiredCapabilities; -import org.openqa.selenium.remote.RemoteWebDriver; -import org.testng.Assert; -import org.testng.annotations.AfterSuite; -import org.testng.annotations.BeforeSuite; -import org.testng.annotations.Test; - -public class LambdaException { - -private static RemoteWebDriver driver; -private static String Status="failed"; - -@BeforeSuite -public void setup() throws MalformedURLException { - -try { -String username = System.getenv("LT_USERNAME"); -String authkey = System.getenv("LT_ACCESS_KEY"); -String hub = "@hub.lambdatest.com/wd/hub"; - -DesiredCapabilities caps = new DesiredCapabilities(); -caps.setCapability("browser", "Chrome"); -caps.setCapability("version", "86"); -caps.setCapability("platform", "MacOS Catalina"); -caps.setCapability("build", "LambdaException Demo"); -caps.setCapability("name", "Test 2"); -caps.setCapability("network", true); -caps.setCapability("visual", true); -caps.setCapability("video", true); -caps.setCapability("console", true); - -System.out.println("Desired Caps: " + caps); -driver = new RemoteWebDriver(new URL("https://" + username + ":" + authkey + hub), caps); -} -catch(Exception e) -{ -System.out.println(e); -} -} - -@Test -public static void testAssertionError() { - -ArrayList exceptionCapture = new ArrayList<>(); -try { -driver.get("https://www.lambdatest.com"); - -String ExpectedTitle = "Most Powerful Cross Browser Testing Tool Online | LambdaT"; -String TitleValue = driver.getTitle(); -if (TitleValue.equals(ExpectedTitle)) { -Status = "passed"; -} - -Assert.assertEquals(TitleValue, ExpectedTitle); -} catch (AssertionError e) { -Status = "failed"; -exceptionCapture.add(e.getMessage()); -((JavascriptExecutor) driver).executeScript("lambda-exceptions", exceptionCapture); -} -} - -@AfterSuite -public void tearDown() { -driver.executeScript("lambda-status=" + Status); -driver.quit(); -} - -} -``` - -When you execute the above test as a TestNG script, the test will fail. This is because we provided an incorrect value in the ExpectedTitle variable for demo purposes. When you open the test view and navigate to the Exception tab, you will find a message displaying this error: - -### View StackTrace on the Dashboard Using Lambda Exception - -You can also view the full StackTrace on the dashboard using Lambda Exceptions. - -You can view the full StackTrace on the TestMu AI platform using the Lambda Exceptions feature. Convert the StackTrace to String and print it as an array of Strings. Below is the full script to print the StackTrace using TestNG framework in Java: - -```javascript -package com.lambdatest; - -import java.io.PrintWriter; -import java.io.StringWriter; -import java.net.MalformedURLException; -import java.net.URL; -import java.util.ArrayList; - -import org.openqa.selenium.By; -import org.openqa.selenium.JavascriptExecutor; -import org.openqa.selenium.NoSuchElementException; -import org.openqa.selenium.remote.DesiredCapabilities; -import org.openqa.selenium.remote.RemoteWebDriver; -import org.testng.Assert; -import org.testng.annotations.AfterSuite; -import org.testng.annotations.BeforeSuite; -import org.testng.annotations.Test; - -public class LambdaException { - -private static RemoteWebDriver driver; -private static String Status="failed"; - -@BeforeSuite -public void setup() throws MalformedURLException { - -try { -String username = System.getenv("LT_USERNAME"); -String authkey = System.getenv("LT_ACCESS_KEY"); -String hub = "@hub.lambdatest.com/wd/hub"; - -DesiredCapabilities caps = new DesiredCapabilities(); -caps.setCapability("browser", "Chrome"); -caps.setCapability("version", "86"); -caps.setCapability("platform", "MacOS Catalina"); -caps.setCapability("build", "LambdaException Demo"); -caps.setCapability("name", "Print StackTrace"); -caps.setCapability("network", true); -caps.setCapability("visual", true); -caps.setCapability("video", true); -caps.setCapability("console", true); - -System.out.println("Desired Caps: " + caps); -driver = new RemoteWebDriver(new URL("https://" + username + ":" + authkey + hub), caps); -} -catch(Exception e) -{ -System.out.println(e); -} -} - -@Test -public static void testAssertionError() { - -ArrayList exceptionCapture = new ArrayList<>(); -try { -driver.get("https://www.lambdatest.com"); -String TitleValue = driver.getTitle(); -String ExpectedTitle = "Most Powerful Cross Browser Testing Tool Online | LambdaT"; -if (TitleValue.equals(ExpectedTitle)) { -Status = "passed"; -} -Assert.assertEquals(TitleValue, ExpectedTitle); -} catch (AssertionError e) { -Status = "failed"; -StringWriter sw = new StringWriter(); -PrintWriter printWriter = new PrintWriter(sw); -PrintWriter pw = printWriter; -e.printStackTrace(pw); -String sStackTrace = sw.toString(); -exceptionCapture.add(sStackTrace); -((JavascriptExecutor) driver).executeScript("lambda-exceptions", exceptionCapture); -} -} - -@AfterSuite -public void tearDown() { -driver.executeScript("lambda-status=" + Status); -driver.quit(); -} - -} -``` - -When you execute the above test as a TestNG script, it produces the same result as the view exception section because of the incorrect ExpectedTitle value. Open the test view and navigate to the Exception tab to find the full StackTrace: - -Below is the full StackTrace printed on the TestMu AI platform from the above script. - -```javascript -java.lang.AssertionError: expected [Most Powerful Cross Browser Testing Tool Online | LambdaT] but found [Most Powerful Cross Browser Testing Tool Online | Lambdatest] at -org.testng.Assert.fail(Assert.java:99) at -org.testng.Assert.failNotEquals(Assert.java:1037) at -org.testng.Assert.assertEqualsImpl(Assert.java:140) at -org.testng.Assert.assertEquals(Assert.java:122) at -org.testng.Assert.assertEquals(Assert.java:629) at -org.testng.Assert.assertEquals(Assert.java:639) at -com.lambdatest.LambdaException.testAssertionError(LambdaException.java:66) at -java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at -java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64) at -java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at -java.base/java.lang.reflect.Method.invoke(Method.java:564) at -org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:132) at -org.testng.internal.TestInvoker.invokeMethod(TestInvoker.java:599) at -org.testng.internal.TestInvoker.invokeTestMethod(TestInvoker.java:174) at -org.testng.internal.MethodRunner.runInSequence(MethodRunner.java:46) at -org.testng.internal.TestInvoker$MethodInvocationAgent.invoke(TestInvoker.java:822) at -org.testng.internal.TestInvoker.invokeTestMethods(TestInvoker.java:147) at -org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:146) at -org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:128) at -java.base/java.util.ArrayList.forEach(ArrayList.java:1511) at -org.testng.TestRunner.privateRun(TestRunner.java:764) at -org.testng.TestRunner.run(TestRunner.java:585) at -org.testng.SuiteRunner.runTest(SuiteRunner.java:384) at -org.testng.SuiteRunner.runSequentially(SuiteRunner.java:378) at -org.testng.SuiteRunner.privateRun(SuiteRunner.java:337) at -org.testng.SuiteRunner.run(SuiteRunner.java:286) at -org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:53) at -org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:96) at -org.testng.TestNG.runSuitesSequentially(TestNG.java:1218) at -org.testng.TestNG.runSuitesLocally(TestNG.java:1140) at -org.testng.TestNG.runSuites(TestNG.java:1069) at -org.testng.TestNG.run(TestNG.java:1037) at -org.testng.remote.AbstractRemoteTestNG.run(AbstractRemoteTestNG.java:115) at -org.testng.remote.RemoteTestNG.initAndRun(RemoteTestNG.java:251) at -org.testng.remote.RemoteTestNG.main(RemoteTestNG.java:77) -``` - -That’s it! You can now easily handle and manage exceptions using the Lambda Exceptions feature. If you still have any doubt, please feel free to reach out to us via **24/7 chat support** or by mailing to us on [support@testmuai.com](mailto:support@testmuai.com). Happy testing! 🙂 diff --git a/static/docs/lambda-hooks.md b/static/docs/lambda-hooks.md deleted file mode 100644 index cbae46c05..000000000 --- a/static/docs/lambda-hooks.md +++ /dev/null @@ -1,38 +0,0 @@ -# Lambda Hooks for Selenium Automation - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Available Lambda Hooks - -Use these hooks via JavascriptExecutor to control test behavior, manage files, and capture logs. - -TestMu AI Selenium Grid offers a set of hooks (Lambda Hooks) that you can use to modify your automation test cases and perform multiple operations in your automation scripts. - -Below is the list of available Lambda Hooks and the function each performs. - -| Lambda Hooks | Descriptions | -| ------------- | ------------ | -| *lambda-status* | Check whether the test is passed or failed.For Passed Test:`driver.executeScript("lambda-status=passed");`For Failed Test:`driver.executeScript('lambda-status=failed');` `((JavascriptExecutor) driver).executeScript("lambda-status=" + "passed");` Here are some newly added status you can use for interpreting test execution results: `skipped`, `ignored`, `unknown`, `error` | -| *lambda-file-exists* | Check whether the downloaded file exists in the test machine.`((JavascriptExecutor) driver).executeScript("lambda-file-exists=file-name.file_format");` | -| *lambda-file-stats* | Retrieve file metadata such as md5 code, modified time, name and size.`((JavascriptExecutor) driver).executeScript("lambda-file-stats=file-name.file_format");` | -| *lambda-file-content* | Download file content using base64 encoding.`((JavascriptExecutor) driver).executeScript("lambda-file-content=file-name.file_format");` | -| *lambda-file-list* | List down the file in download directory.`print driver.execute_script("lambda-file-list={match string with filename}");` `ie:print driver.execute_script("lambda-file-list=sample");Response: List of files in downloads dir starting with sample` | -| *lambda-name* | For changing the test name.`((JavascriptExecutor) driver).executeScript("lambda-name=TestName");` `((JavascriptExecutor) driver).executeScript("lambda-name=" + "name from hooks");` | -| *lambda-build* | For updating the build name.`executeScript("lambda-build=BUILD_NAME");` | -| *lambda-action* | Used to mark a test as passed/failed. Moreover, it allows the option to include a failure reason, which will be visible on the TestMu AI Automation Dashboard inside the session view.`Map action = new HashMap();action.put("status", "failed"); action.put("reason", "tmp reason"); driver.executeScript("lambda-action", action);` `((JavascriptExecutor) driver).executeScript("lambda-action=" + "Lambda Error");` | -| *lambda-perform-keyboard-events* | You can simulate keyboard shortcuts like **ctrl + c**, **ctrl + v** in automation test scenarios. This hook is supported on both Windows and MacOS. `js.executeScript("lambda-perform-keyboard-events:tab");`| -| *lambda_breakpoint* | Aborts the test execution to use the live interaction feature. `driver.executeScript("lambda-breakpoint=true");`| -| *lambda_screenshot* | Captures the async screenshot during test execution. `driver.executeScript("lambda-screenshot=true");`| -| *lambda_files_delete* | Deletes the file in the download directory in the virtual machines (VMs).`driver.executeScript("lambda-files-delete=file1.csv,file2.csv);`| -| *lambda_network_throttle* | Throttles network speed during test execution.`executeScript("lambda-throttle-network","Regular 4G")`| -| *lambda_ping* | Fetches the IPs of the domain.`driver.executeScript("lambda-ping=lambdatest.com");`| -| *lambda_exceptions* | Uploads the exceptions for tests that are captured on the console.`driver.executeScript('lambda-exceptions', [[message]])`| -| *lambda_get_clipboard* | Prints the clipboard data on the console.`driver.executeScript("lambda-get-clipboard");`| -| *lambda_set_clipboard* | Sets the clipboard data.`driver.executeScript("lambda-set-clipboard= Amit");`| -| *lambda_clear_clipboard* | Clears the data of the clipboard.`driver.executeScript("lambda-clear-clipboard");`| -| *lambda_unbound_ping* | Fetches the IPs from the outbound domain.`driver.executeScript("lambda-unbound-ping=lambdatest.com");` -| *lambda:network* | Fetches the network log entries in array format during session.`driver.execute_script("lambda:network");`- Fetch the network log from last fetch request time to current time.`driver.execute_script("lambda:network=all");`- Fetch from start of test session to current time. -| *lambdaUpdateName* | Sets the test name during test execution.`driver.executeScript("lambdaUpdateName=TestName");` | -| *lambda-test-tags* | Dynamically update your test tags for a test session which can be used to organize and filter your test results. **Syntax :** `driver.executeScript("lambda-test-tags", "Tag 1,Tag 3,Tag 2");` **Limitations :** **1. Maximum Character Length per Tag:** Each tag can have up to 50 characters. **2. Maximum Number of Tags:** A maximum of 15 tags can be assigned to a single test session. | - -> **Note**: These hooks only work if you are connected to your [TestMu AI Hub URL](/support/docs/hyperexecute-general-faqs/#17-how-can-i-access-my-lambdatest-hub-url). If you use these hooks on any other platform, you might see the error: `javascript error: Invalid left-hand side in assignment` diff --git a/static/docs/lambda-tunnel-modifiers.md b/static/docs/lambda-tunnel-modifiers.md deleted file mode 100644 index f159a0346..000000000 --- a/static/docs/lambda-tunnel-modifiers.md +++ /dev/null @@ -1,45 +0,0 @@ -# TestMu AI Tunnel Modifiers - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI Tunnel provides several command-line arguments that you can leverage to define TestMu AI Tunnel parameters. You can pass command-line arguments to either a **‘config’** file or an environment variable. It is recommended to use the latest version of TestMu AI tunnel binary to ensure compatibility with variables. - -This document will help you understand all the modifiers/arguments used with TestMu AI Tunnel to help you test your locally hosted web pages. - -| FLAG (SHORT) | FLAG (LONG) | DESCRIPTION | TYPE | -|--------------|-------------|-------------|------| -|| --allowHosts | Comma separated list of hosts to route via tunnel. Everything else will be routed via Internet. Learn how to use the [wildcard expressions](/support/docs/tunnel-allowHost-wildcard-support/) in the `--allowHosts` flag | String | -| | --bypassHosts | Comma separated list of hosts to bypass from tunnel. These will be routed via internet. | String | -| | --callbackURL | Callback URL for tunnel status. | String | -| -c | --config | Path of the config file to use | String | -| | --clientCert | mTLS Client Certificate filepath. Only `pfx` file type is supported| String | -| | --clientKey | mTLS Client Key filepath. | String | -| | --mTLSHosts | Comma separated list of mTLS hosts. | String | -| -d | --dir | Specify the Local Directory that will be server by a file server on port 33000 | String | -| | --dns | Comma Separated List of DNS Servers | String | -| | --egress-only | Uses proxy settings only for outbound requests. | N/A | -| -e | --env | The environment on which the TestMu AI Tunnel will run. | String | -| -h | --help | Help for the TestMu AI Tunnel | N/A | -| | --infoAPIPort | Exposes [Tunnel Info API](/support/docs/advanced-tunnel-features/#tunnelinfoapis) At The Specified Port | Integer | -| | --ingress-only | Routes Only Incoming Traffic Via The Proxy Specified | N/A | -| -k | --key | TestMu AI Access Key. | String | -| -l | --load-balanced | Activates [Load Balancing](/support/docs/load-balancing-in-lambda-tunnel/) for TestMu AI Tunnel | N/A | -| | --logFile | Location of the TestMu AI Tunnel Log File | String | -| -m | --mitm | Enable the [MITM(Man-in-the-middle)](/support/docs/advanced-tunnel-features/#mitmlocaltesting) mode for TestMu AI Tunnel | N/A | -| | --mode | Specifies in which mode tunnel should run [ssh,tcp,ws]. (default "ssh") | String | -| | --ntlm | To use Microsoft NTLM (Windows NT LAN Manager) authentication for communication or transport purposes. | Boolean | -| | --pidfile | Path Of Pidfile, Where Process ID will Be Written | String | -| -p | --port | Port for TestMu AI Tunnel to activate. | Integer | -| | --proxy-host | Specifies the Tunnel Proxy Port hostname. | String | -| | --proxy-pass | Specifies the Tunnel Proxy Port password. | String | -| | --proxy-port | Specifies the port number where Tunnel proxy will activate. | Integer | -| | --proxy-user | Specifies the Tunnel Proxy Port username | String | -| | --pacfile | Path of a PAC file. | String | -| -s | --shared-tunnel | Sharing Tunnel Among Team Members | N/A | -| | --sshConnType | Specify type of ssh connection (over_22, over_443, over_ws).To use –sshConnType, specify ––mode ssh flag first. | String | -| N/A | --maxSSHConnections | Increase the SSH Connection from Tunnel Client to Tunnel Server. Maximum allow is 30 | N/A | -| -n | --tunnelName | Specifies the custom TestMu AI Tunnel name to be used. | String | -| -u | --user | TestMu AI Username. | String | -| | --use-private-ip | Sets remote address to an internal IP of client machine. | Boolean | -| -v | --verbose | Should Every Proxy Request Be Logged To Stdout | N/A | -| | --version | Displays the installed version for TestMu AI Tunnel | N/A | diff --git a/static/docs/laravel-dusk-with-selenium-running-laravel-dusk-automation-scripts-on-testmu-selenium-grid.md b/static/docs/laravel-dusk-with-selenium-running-laravel-dusk-automation-scripts-on-testmu-selenium-grid.md deleted file mode 100644 index cb25676f3..000000000 --- a/static/docs/laravel-dusk-with-selenium-running-laravel-dusk-automation-scripts-on-testmu-selenium-grid.md +++ /dev/null @@ -1,123 +0,0 @@ -# Selenium With Laravel Dusk - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Run Laravel Dusk tests on the TestMu AI cloud grid. This guide covers setup, running a sample test, configuring capabilities, and testing locally hosted pages. - -**Sample repo** -All the code used in this guide is available in the sample repository. - - View on GitHub - -## Prerequisites - -Complete the following steps before you begin automation testing with Laravel Dusk. - -1. Download **PHP 8.1 or later** (current LTS recommended). Use the following commands in the terminal: - - * **MacOS:** Previous versions of **MacOS** have **PHP** installed by default. For the latest **MacOS** versions starting with **Monterey**, download and install **PHP** manually: -```bash -/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" -brew install php -``` - * **Windows:** Download **PHP** from [PHP for Windows](http://windows.php.net/download/) and add it to your system PATH. Refer to the [PHP Windows installation guide](http://php.net/manual/en/install.windows.php) to ensure PHP is accessible through Command Prompt (cmd). - -2. Download **composer** in the project directory ([Linux/MacOS](https://getcomposer.org/download/), [Windows](https://getcomposer.org/doc/00-intro.md#installation-windows)). - - **Note:** To use the **composer** command directly, it either should have been downloaded in the project directory or should be accessible globally which can be done by the command below: -```bash -mv composer.phar /usr/local/bin/composer -``` - -## Step 1: Clone the Sample Project - -Clone the repository and install dependencies. - -```bash -git clone https://github.com/LambdaTest/php-laravel-dusk-todo -cd php-laravel-dusk-todo -``` - -Install the composer dependencies: -```bash -composer install -composer dump-autoload -``` - -## Step 2: Set Your Credentials - -Configure your credentials to connect to the TestMu AI Selenium Grid. - -Create .env from the example file: -```bash -cp .env.example .env -``` - -Update your TestMu AI credentials in the newly created .env file: -```bash -LT_USERNAME="" -LT_ACCESS_KEY="" -``` - -Set TestMu AI `Username` and `Access Key` in environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -## Step 3: Configure Your Test Capabilities - -Define browser, version, and OS settings for your test run. - -Update platform configuration in the driver method of `tests/DuskTestCase.php` to specify the target where tests should run. - -```php -$capabilities = array( -"build" => "LaravelDusk Build", -"name" => "LaravelDusk Build", -"platform" => "Windows 10", -"browserName" => "Chrome", -"version" => "latest" -); -``` - -**Capabilities Generator** -Use the TestMu AI [Capabilities Generator](https://www.testmuai.com/capabilities-generator/) to auto-generate the capabilities class for your test requirements. - -## Step 4: Run the Test - -Execute the Laravel Dusk test from the command line. - -```bash -php artisan dusk -``` - -## Step 5: View Your Results - -Check the test output on the console and the TestMu AI dashboard. - -Visit the [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) to view your test results. The dashboard provides: - -- Text logs for each test step -- Screenshots captured during execution -- Video recordings of the full test session - -## Run Laravel Dusk Tests Using Agent Skills - -Use AI coding assistants to generate and run Laravel Dusk tests with the TestMu AI Agent Skill. - -The [laravel-dusk-skill](https://github.com/LambdaTest/agent-skills/tree/main/laravel-dusk-skill) is part of [TestMu AI Agent Skills](https://github.com/LambdaTest/agent-skills/) - structured packages that teach AI coding assistants how to write production-grade test automation. - -Install the skill: - -```bash -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/laravel-dusk-skill .claude/skills/ - -# For Cursor / Copilot -cp -r agent-skills/laravel-dusk-skill .cursor/skills/ -``` - -Install all available framework skills at once by cloning the repository directly into your tool's skills directory (e.g., `.claude/skills/`, `.cursor/skills/`). diff --git a/static/docs/launch-first-session.md b/static/docs/launch-first-session.md deleted file mode 100644 index 4725cac44..000000000 --- a/static/docs/launch-first-session.md +++ /dev/null @@ -1,159 +0,0 @@ -# Launch Your First Browser Session - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This guide walks you through setting up your TestMu AI account, creating -your first browser session on TestMu AI Browser Cloud, and driving it using -TypeScript/Puppeteer. In just a few minutes, you'll be programmatically -controlling a cloud browser. - -## Prerequisites - -**Step 1**: [Sign up on TestMu AI](https://www.testmuai.com/register/) - -**Step 2**: Get your credentials: - -1. After signing up, navigate to **Settings → Account Settings** -2. Find your **Username** and **Access Key** - -**Step 3**: Set up environment variables: - -1. Create a `.env` file in your project root (if you don't have one) -2. Add your credentials: - -``` -LT_USERNAME=your_username -LT_ACCESS_KEY=your_access_key -``` - -> Make sure to add `.env` to your `.gitignore` file to keep your credentials secure. - -## Installing the TestMu AI Browser SDK - -```bash -npm i @testmuai/browser-cloud -``` - -**Requirements:** Node.js 16+ (Node 18+ required if using the Playwright adapter) - -## Create Your First Session - -Let's create a simple script that launches a cloud browser, navigates to a page, -and cleans up: - -```typescript -// my-first-session.ts - -import { Browser } from '@testmuai/browser-cloud'; - -const client = new Browser(); - -async function main() { -const session = await client.sessions.create({ -adapter: 'puppeteer', -lambdatestOptions: { -build: 'Getting Started', -name: 'My First Session', -'LT:Options': { -username: process.env.LT_USERNAME, -accessKey: process.env.LT_ACCESS_KEY, -} -} -}); - -console.log('Session created:', session.id); -console.log('View live session at:', session.sessionViewerUrl); - -// Connect and use the browser -const browser = await client.puppeteer.connect(session); -const page = (await browser.pages())[0]; - -await page.goto('https://example.com'); -console.log('Title:', await page.title()); - -// Clean up -await browser.close(); -await client.sessions.release(session.id); -console.log('Session released'); -} - -main().catch(console.error); -``` - -Playwright requires **Node.js 18+**. - -```typescript -// my-first-session.ts - -import { Browser } from '@testmuai/browser-cloud'; - -const client = new Browser(); - -async function main() { -const session = await client.sessions.create({ -adapter: 'playwright', -lambdatestOptions: { -build: 'Getting Started', -name: 'My First Session', -'LT:Options': { -username: process.env.LT_USERNAME, -accessKey: process.env.LT_ACCESS_KEY, -} -} -}); - -console.log('Session created:', session.id); -console.log('View live session at:', session.sessionViewerUrl); - -// Connect and use the browser -const { browser, context, page } = await client.playwright.connect(session); - -await page.goto('https://example.com'); -console.log('Title:', await page.title()); - -// Clean up -await browser.close(); -await client.sessions.release(session.id); -console.log('Session released'); -} - -main().catch(console.error); -``` - -```typescript -// my-first-session.ts - -import { Browser } from '@testmuai/browser-cloud'; - -const client = new Browser(); - -async function main() { -const session = await client.sessions.create({ -adapter: 'selenium', -lambdatestOptions: { -build: 'Getting Started', -name: 'My First Session', -'LT:Options': { -username: process.env.LT_USERNAME, -accessKey: process.env.LT_ACCESS_KEY, -} -} -}); - -console.log('Session created:', session.id); -console.log('View live session at:', session.sessionViewerUrl); - -// Connect and use the browser -const driver = await client.selenium.connect(session); - -await driver.get('https://example.com'); -console.log('Title:', await driver.getTitle()); - -// Clean up -await driver.quit(); -await client.sessions.release(session.id); -console.log('Session released'); -} - -main().catch(console.error); -``` diff --git a/static/docs/leapwork-integration-with-testmu.md b/static/docs/leapwork-integration-with-testmu.md deleted file mode 100644 index 76c9135be..000000000 --- a/static/docs/leapwork-integration-with-testmu.md +++ /dev/null @@ -1,81 +0,0 @@ -# How to Integrate TestMu AI With LEAPWORK - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -If you build codeless test automation in LEAPWORK, you can run those same visual flows on real browsers and devices by connecting LEAPWORK to TestMu AI. The integration gives your LEAPWORK agent access to 3,000+ browsers, devices, and OS combinations. You configure TestMu AI once inside LEAPWORK Studio, then run automation flows against the TestMu AI cloud without managing any local machines. - -This integration covers two connection types. Use a **web agent** to run flows on the cross browser cloud, and a **TestMu AI Server** connection to automate native Android and iOS apps on real devices. - -## Prerequisites for Connecting LEAPWORK to TestMu AI - -Before you start, make sure you have the following in place. - -* A TestMu AI account. If you do not have one, [sign up for a free TestMu AI account](https://www.testmuai.com/register/?redirectTo=https://accounts.lambdatest.com/dashboard). -* Your TestMu AI **Username** and **Access Key**, available from the **Account Settings** page after you log in. -* A working LEAPWORK Studio installation. **Admin** access is required to add a TestMu AI Server connection for mobile testing. - -## How to Connect LEAPWORK to TestMu AI for Cross Browser Testing - -LEAPWORK assigns **Agents**, the available machines that execute your automation flows. To run flows on the TestMu AI cross browser cloud, create an agent with web capabilities that points at TestMu AI. The steps below walk through that setup end to end. - -**Step 1:** In LEAPWORK Studio, open the **Agents** section and click **New**. - -**Step 2:** Open the **Agent type** drop-down and select **TestMu AI**. A settings window opens for the new agent. - -**Step 3:** Log in to your TestMu AI account in a browser so you can copy your credentials in the next step. - -**Step 4:** Open **Account Settings** from your profile menu in the top-right corner. Copy the **Username** and **Access Key** shown there. You need both values to authenticate the agent. - -**Step 5:** Return to the TestMu AI agent settings window in LEAPWORK Studio. This window is where you define how the agent connects to the cloud and which environment it runs on. - -Fill in the agent settings as follows. TestMu AI supports both manual and automated browser testing across 3,000+ browser, device, and OS combinations, so the platform, browser, and version you pick here determine the environment your flows run on. - - * **Title**: a name for the agent, for example `TestMu AI`. - * **Hostname**: the IP address, machine name, or DNS name of the TestMu AI hub (see Step 6 for the exact value). - * **Port**: the connection port. The default is `80`. - * **Username** and **Access Key**: the credentials you copied in Step 4. - * **Platform & OS**: select the operating system to run on from the drop-down. - * **Platform version**: select the OS version for the selected platform. - * **Browser**: select the browser to run the flow on. - * **Browser version**: pick a specific version, or select **latest** to always use the newest available version. - * **Resolution**: set the screen resolution the TestMu AI machine uses while running flows. - * **Description**: optional free text to help you identify the agent later. - -**Step 6:** Enter your **Hostname**, **Username**, and **Access Key** in the highlighted fields. These three values are what authenticate the agent, so confirm them carefully. In the **Hostname** field, enter the TestMu AI web hub: - -```text -hub.lambdatest.com -``` - -**Step 7:** Click **Test connection**. When the credentials and hostname are correct, Studio shows **Successfully connected** in green. - -Click **Save** to store the TestMu AI environment in Studio. You can now select this agent whenever you schedule or run automation flows. - -## How to Set Up a TestMu AI Server Connection for Mobile App Testing - -A **TestMu AI Server** connection points your LEAPWORK flows at the TestMu AI mobile cloud. Use it to automate native Android and iOS app flows on real devices without maintaining a physical device lab. This connection is separate from the web agent above and is added on the **Connections** screen. - -Adding a server connection is restricted to LEAPWORK **Admin** users. If you do not see the option, ask your workspace administrator to grant admin access or to create the connection for you. - -**Step 1:** In LEAPWORK Studio, open the **Connections** screen and click **Add**. The **Add Connection** dialog opens. - -**Step 2:** Enter a unique **Title** and an optional **Description** so your team can identify the connection. Open the **Type** drop-down and select **TestMu AI Server**. Selecting it reveals the connection fields. - -**Step 3:** Fill in the connection fields: - - * **Hostname**: the TestMu AI mobile endpoint. Use `https://mobile-hub.lambdatest.com/wd/hub` unless you are on a dedicated or region-specific hub. - * **Username**: your TestMu AI account username (the same value from Step 4 of the web setup above). - * **Access Key**: your TestMu AI access key from the same **Account Settings** page. - * **Custom Capabilities** (optional): a JSON block defining the device, OS version, and other run settings, for example `{"deviceName": "Galaxy S23", "platformVersion": "14", "platformName": "android"}`. - -**Step 4:** Click **Test connection** to validate the credentials. When the values are correct, Studio shows a success message. Click **Save** to store the connection. - -The saved connection appears in the **Connections** list and can be reused across mobile automation flows. To change credentials or remove it, use the **Edit** or **Delete** buttons next to **Add**. - -## Related TestMu AI Codeless Integration Guides - -Use these guides to set up other codeless tools or to compare connection options. - -* [How to choose a codeless automation integration](/support/docs/integrations-with-codeless-automation-tools/) covers the full set of TestMu AI codeless tool integrations. -* [How to run ACCELQ tests on TestMu AI](/support/docs/accelq-integration/) connects ACCELQ as an external device provider. -* [How to run Testsigma tests on TestMu AI](/support/docs/testsigma-integration/) runs Testsigma automation flows on the TestMu AI cloud. diff --git a/static/docs/linear-app-integration.md b/static/docs/linear-app-integration.md deleted file mode 100644 index 37b0ed9d1..000000000 --- a/static/docs/linear-app-integration.md +++ /dev/null @@ -1,65 +0,0 @@ -# Linear App Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Linear App is a bug tracking tool built for high -performance teams. With Linear App, you can perform bug tracking, streamline software projects, sprints, tasks and more. TestMu AI and Linear App integration allows you to perform one-click bug logging from TestMu AI to your Linear App projects. - -In this documentation, we look at how to integrate Linear App with TestMu AI and mark and share bugs from TestMu AI to Linear App projects. - -## How To Integrate Linear App With Your TestMu AI Account? - -1. Login to your TestMu AI account. To install integrations, you need to have admin or user level access. - -2. Visit 'Integrations' from the left menu bar. Once you are in the integration page, select **LINEAR APP** under the **BUG TRACKER** category. - ->**Note**: If you already have a TestMu AI integration with any project management tool then you will be asked to uninstall the existing TestMu AI integration. - -3. You will be asked to enter your Linear App **API Token**. - -> -* **What is an API Token?** -An API token is needed to validate a user's identity from one server to another by retrieving client resources from the server where the connection is made. API tokens are essential when an end user interacts to an application via a two-step verification process to ensure its security and data integrity. - -4. To get the API token, log in to your Linear App account. Click on the Profile avatar and select **Settings**. - -5. From the left sidebar menu, click on **API**. - -6. Under the **Personal API Keys**, provide your desired label name and click on **Create key**. - -7. Your Linear App API token will be generated. Copy the generated API token. - -8. Paste the copied API token in the provided field and click on **Install**. - -The Linear App will be successfully integrated with your TestMu AI account. You'll also receive the confirmation message and an email of the same. - -Visit the Integrations page, under **MY INTEGRATION**, you'll find the installed Linear App. - -## Logging First Bug Through Linear App Integration - -Let's perform one-click bug logging while performing Real time testing. - -1. Select the Real Time Testing from the left menu. - -2. Enter the test URL, select the configurations like browser, browser versions, OS and resolutions and then click on **START**. - -3. A virtual machine will be launched where you can perform live testing of websites and web apps. If you encounter a bug, click on the Bug icon from the left bar to grab the screenshot. - -4. The screenshot will be captured on which you can annotate bugs or tasks by using in-built image editor tools. After annotating the bugs/tasks, click on Mark As Bug icon. - -5. A Linear App ticket will open up where you need to fill the details like Team, Assignee, Summary, Priority and so on. After furnishing the required details, click on **Mark As Bug**. - -Once you click on Mark As Bug, your bugs will be marked successfully. - -6. To view your logged bugs, log into your Linear App account and click on **My Issues** from the left menu bar. - -You will notice the marked issue right away. - -7. To view the issue's details, click on the issue. You can view the issue's attachments like screenshots, test id, reproduce on TestMu AI link, test URL, OS, browser and browser versions. - -## Uninstalling The Linear App Integration - -1. Login to your TestMu AI account. Select Integrations and navigate to 'My Integrations'. - -2. Click on **REMOVE** adjacent to Linear App icon. - -The Linear App integration will be uninstalled successfully. diff --git a/static/docs/link-ado-issues-with-test-manager.md b/static/docs/link-ado-issues-with-test-manager.md deleted file mode 100644 index a29c55393..000000000 --- a/static/docs/link-ado-issues-with-test-manager.md +++ /dev/null @@ -1,34 +0,0 @@ -# Link Azure DevOps Issues with Test Manager - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Link Azure DevOps Issues with Test Manager - -Test Manager enhances its functionality by seamlessly linking test cases with Azure DevOps Work Items. This integration ensures a reliable connection between your testing activities and Azure DevOps Work Items, optimizing project management and collaboration across teams. - -### Prerequisites -- Active Microsoft Azure DevOps account -- Appropriate permissions to access Azure DevOps projects - -### Initial Setup and Configuration - -> **Note:** If you have already integrated Azure DevOps with your TestMu AI account, you can skip this section and proceed directly to [Linking Azure DevOps Work Items](#linking-azure-devops-work-items) - -1. Navigate to the [Integration page](https://integrations.lambdatest.com/) to integrate your TestMu AI account with Azure DevOps. Click on **Connect** as shown in the image. - -2. A side drawer will open to initiate Azure DevOps integration. Click on **Install** to proceed. - -3. Select your Azure DevOps Organization and click **Install**. Azure DevOps is now integrated with your TestMu AI account. - -### Linking Azure DevOps Work Items - -Once the integration is complete, follow these steps to link work items with your test cases or test runs: - -1. Head to the issues section in Test Manager, click on **Link Issue** and go to Azure DevOps. -> Note: Issue linking is supported at the Test Case, Test Run, Test Case instance, and step level. To link issues, go to the Issues tab within a Test Case or a Test Run. For linking at the instance and step level within a Test Run, see [Track Bugs and Issues in Test Runs](/support/docs/track-issues-in-test-runs/). - -2. Enter the Azure DevOps Work Item ID or URL in the input field and click on **Link Issue**. - -3. The Azure DevOps Work Item from your integrated Organization is successfully linked and listed. You can view the work item name, ID, type, state, priority, and creator. - -4. To **unlink an issue**, click on the unlink button located on the right side of the work item. diff --git a/static/docs/link-jira-issues-with-test-manager.md b/static/docs/link-jira-issues-with-test-manager.md deleted file mode 100644 index 264706050..000000000 --- a/static/docs/link-jira-issues-with-test-manager.md +++ /dev/null @@ -1,30 +0,0 @@ -# Link Jira Issues with Test Manager - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Test Manager enhances its functionality by seamlessly linking test cases with Jira issues. This integration ensures a reliable connection between your testing activities and Jira tasks, optimizing project management and collaboration across teams. - -### Initial Setup and Configuration - -> **Note:** If you have already integrated Jira with your TestMu AI account, you can skip this section and proceed directly to [Linking Jira Issues](#linking-jira-issues-in-test-manager) - -1. Navigate to the [Integration page](https://integrations.lambdatest.com/) to integrate your TestMu AI account with Jira. Search for Jira and click on **Connect**. - -2. A side drawer will open to initiate Jira integration. Click on **Install** to proceed. - -3. A prompt will appear requesting access to your Atlassian account. Click on **Accept** to proceed. - -4. Select your Jira instance site, click on **Select Site & Proceed**, then select your preferred project and click **Integrate Projects & Install**. - -### Linking Jira Issues in Test Manager - -Once the integration is complete, follow these steps to link issues with your test cases or test runs: - -1. Head to the issues section in Test Manager, click on **Link Issue** and go to Jira. -> Note: Issue linking is supported at the Test Case, Test Run, Test Case instance, and step level. To link issues, go to the Issues tab within a Test Case or a Test Run. For linking at the instance and step level within a Test Run, see [Track Bugs and Issues in Test Runs](/support/docs/track-issues-in-test-runs/). - -2. Enter the Jira Issue ID or URL in the input field and click on **Link Issue**. - -3. The Jira issue from your integrated Jira Project is successfully linked and listed. You can view the Issue summary, ID, type, status, priority, and creator. - -4. To **unlink an issue**, click on the unlink button located on the right side of the Jira Issue. diff --git a/static/docs/list-of-supported-locales.md b/static/docs/list-of-supported-locales.md deleted file mode 100644 index 1a6969e2a..000000000 --- a/static/docs/list-of-supported-locales.md +++ /dev/null @@ -1,227 +0,0 @@ -# Supported Locales And Languages - Appium - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Use the given below list of supported locale and language codes for app testing. - -## Android Locales and Language Codes - -To test localised strings in your android app, configure Appium's language capability with language code. - -Language to be passed in the capability - language:'fr' where fr is language code for french - -### Android Language and Language Codes - -| Language | Language Code | -| -----------------------| ---------------------------| -| Arabic | ar | -| Bulgarian | bg | -| Catalan | ca | -| Chinese | zh | -| Croatian | hr | -| Czech | cs | -| Danish | da | -| Dutch | nl | -| English | en | -| Finnish | fi | -| French | fr | -| German | de | -| Greek | el | -| Hebrew | he | -| Hindi | hi | -| Hungarian | hu | -| Indonesian | id | -| Italian | it | -| Japanese | ja | -| Korean | ko | -| Latvian | lv | -| Lithuanian | lt | -| Norwegian-Bokmol | nb | -| Polish | pl | -| Portuguese | pt | -| Romanian | ro | -| Russian | ru | -| Serbian | sr | -| Slovak | sk | -| Slovenian | sl | -| Spanish | es | -| Swedish | sv | -| Tagalog | tl | -| Thai | th | -| Turkish | tr | -| Ukrainian | uk | -| Vietnamese | vi | - -### Android Locale and Locale Codes -Set Appium's locale capability with an appropriate country code to display or format data such as dates, times, decimal separators, and calendars in accordance with the specified country's regional conventions. - -Locale to be passed in the capability - locale:'FR' where FR is the locale code for France - -| Locale | Locale Code | -| -----------------------| ---------------------------| -| Australia | AU | -| Austria | AT | -| Belgium | BE | -| Brazil | BR | -| Britain | GB | -| Bulgaria | BG | -| Canada | CA | -| Croatia | HR | -| Czech Republic | CZ | -| Denmark | DK | -| Egypt | EG | -| Finland | FI | -| France | FR | -| Germany | DE | -| Greece | GR | -| Hong-Kong | HK | -| Hungary | HU | -| India | IN | -| Indonesia | ID | -| Ireland | IE | -| Israel | IL | -| Italy | IT | -| Japan | JP | -| Korea | KR | -| Latvia | LV | -| Liechtenstein | LI | -| Lithuania | LT | -| Mexico | ES | -| Netherlands | NL | -| New Zealand | NZ | -| Norway | NO | -| Philippines | PH | -| Poland | PL | -| Portugal | PT | -| PRC | CN | -| Romania | RO | -| Russia | RU | -| Serbia | RS | -| Singapore | SG | -| Slovakia | SK | -| Slovenia | SI | -| Spain | ES | -| Sweden | SE | -| Switzerland | CH | -| Taiwan | TW | -| Thailand | TH | -| Turkey | TR | -| Ukraine | UA | -| US | US | -| USA | US | -| Vietnam | VN | -| Zimbabwe | ZA | - -## iOS Locales and Language Codes - -To test localised strings in your iOS app, configure Appium's language capability with language code. - -Language to be passed in the capability - language:'fr' where fr is language code for french - -### iOS Language and Language Codes - -| Language | Language Code | -| -----------------------| ---------------------------| -| Chinese | zh | -| Czech | cs | -| Dutch | nl | -| English | en | -| Finnish | fi | -| French | fr | -| German | de | -| Greek | el | -| Hebrew | he | -| Hindi | hi | -| Hungarian | hu | -| Indonesian | id | -| Italian | it | -| Japanese | ja | -| Korean | ko | -| Malay | ms | -| Norwegian (Bokmal) | nb | -| Polish | pl | -| Portuguese (Brazil) | pt | -| Romanian | ro | -| Russian | ru | -| Slovak | sk | -| Spanish | es | -| Swedish | sv | -| Tagalog | tl | -| Thai | th | -| Turkish | tr | -| Ukrainian | uk | -| Vietnamese | vi | - -Set Appium's locale capability with an appropriate country code to display or format data such as dates, times, decimal separators, and calendars in accordance with the specified country's regional conventions. - -Locale to be passed in the capability - locale: 'fr_FR' where fr is language code for french and FR is the locale code for France - -### iOS Locale and Locale Codes - -| Locale | Locale Code | -| -----------------------| ---------------------------| -| Australia  | en_AU | -| Belgium  | nl_BE | -| Belgium  | fr_BE | -| Brunei Darussalam | ms_BN | -| Canada  | en_CA | -| Canada | fr_CA | -| Czech Republic  | cs_CZ | -| Finland | fi_FI | -| Germany  | de_DE | -| Greece | el_GR | -| Hungary | hu_HU | -| India | hi_IN | -| Indonesia | id_ID | -| Israel | he_IL | -| Italy | it_IT | -| Japan | ja_JP | -| Malaysia | ms_MY | -| Netherlands  | nl_NL | -| New Zealand   | en_NZ | -| Norway | nb_NO | -| Philippines  | tl_PH | -| Poland | pl_PL | -| PRC  | zh_CN | -| Romania | ro_RO | -| Russia | ru_RU | -| Singapore | en_SG | -| Slovakia | sk_SK | -| Korea | ko_KR | -| Sweden  | sv_SE | -| Taiwan  | zh_TW | -| Thailand  | th_TH | -| Turkey  | tr_TR | -| UK  | en_GB | -| Ukraine  | uk_UA | -| US | es_US | -| USA | en_US | -| Vietnam  | vi_VN | -| Brazil | pt-BR | -| China (Simplified) | zh-Hans | -| China (Traditional) | zh-Hant | -| Hong Kong | zh-HK | -| India | en-IN | -| Ireland | en-IE | -| Latin America | es-419 | -| Mexico | es-MX | -| South Africa | en-ZA | - -### Enabling Locales and Language via Hook -To **dynamically change** the language and locale of a device during test execution use the `updateLanguageLocale` action inside `lambda_executor`.The example below demonstrates how to use the following. - -```python -driver.execute_script('lambda_executor: { -"action": "updateLanguageLocale", -"arguments": { -"language" : "" , -"locale" : "" -} -}') -``` - -- **iOS**: Supported on devices running iOS 14 and above. -- **Android**: Supported on devices running Android 11 and above. -- Both language and locale are required parameters. If either one is missing, the hook will return an error - -> That's all! In case you come across any doubts or have any questions for us, feel free to drop them at our **24x7 Chat Support** or you could also mail us directly at **[support@testmuai.com](mailto:support@testmuai.com)**. diff --git a/static/docs/live-debug-in-app-automation.md b/static/docs/live-debug-in-app-automation.md deleted file mode 100644 index f41b3c709..000000000 --- a/static/docs/live-debug-in-app-automation.md +++ /dev/null @@ -1,25 +0,0 @@ -# Live Inspect Your App Automation Tests - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Optimize your [app automation testing](https://www.testmuai.com/blog/mobile-app-testing-tools/) with TestMu AI's Real Devices UI Inspector. Now access unparalleled live debugging capabilities right from the app automation dashboard, enabling direct interaction with the test device, not just viewing the script's video. - -With features like real-time navigation, detailed UI element inspection, and the ability to capture screenshots, now significantly enhance both the efficiency and effectiveness of your [automated tests](https://www.testmuai.com/automation-testing). - -## How to use Live Debugger for App Automation - -**Step 1:** Run your app automation test on the TestMu AI platform. You can perform app automation testing across 3000+ devices, OS and browser combinations. Using debug mode to run the test is preferred as you can inspect applications at debug points. - -**Step 2:** After executing your test, head to the TestMu AI App Automation dashboard and open your recently executed app automation test. - -**Step 3:** While an app automation test runs on the dashboard, a **Live Debug** option is available above the live test video. - ->**Note:** Live interaction on iOS devices is currently under beta. Please reach out to [support@testmuai.com](mailto:support@testmuai.com) for any query. - -**Step 4:** You can simply click the Live Debug option to access the extended UI Inspector and inspect the frames. - -**Step 5:** Selecting the Live Debug option smoothly transitions to the enhanced UI inspector, which tailors its behavior to the user's script and allows commands to run according to the [Appium](https://www.testmuai.com/appium) session. - -**Step 6:** With Live Debugger, you can interact in real time with the real device selected for the automation test allowing you to navigate the app, inspect UI elements, and take screenshots, all without disrupting the ongoing automation script. - ->**Note:** The extended UI inspector is designed for the specific app in the script and does not support inspecting multiple apps, as each script is intended for testing a single app for focused efficiency. diff --git a/static/docs/live-interaction.md b/static/docs/live-interaction.md deleted file mode 100644 index 58cab60c9..000000000 --- a/static/docs/live-interaction.md +++ /dev/null @@ -1,41 +0,0 @@ -# Live Interaction - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## What Is Live Interaction? - -Live Interaction lets you take manual control of an active automation session. - -**Live Interaction** is a feature within automation testing that allows you to take control of the session **manually**. This helps abort the automated test session, **debug** any exception in **real time**, and report the test session. - -## Why Use Live Interaction? - -Debug exceptions, interact with OS-level prompts, and track who paused the test. - -- It allows you to partially interact by clicking a button or filling OS-specific credentials without breaking the currently running automated tests. -- It helps find the root cause of exceptions raised during automated tests and allows you to debug the exception during the session. -- It works smartly by letting stakeholders know who has stopped or paused the test, which helps report the issue. - -## Prerequisites and Limitations - -Review the supported platforms, browsers, and session time limits before using Live Interaction. - -- Live interaction is available on: - - **Windows**: 11, 10, 8, and 8.1 - - **macOS**: Big Sur, Monterey, Ventura - - All browsers except **Safari**. -- You can only interact with the browser for **15 minutes**. - -**note** -Safari browser will be added in upcoming releases. - -## How to Use the Live Interaction Feature - -Open the test summary page and click "Click to view live test" to take manual control. - -Follow the below steps to interact with the browser manually in an automation test: - -1. Open the test summary page in the **TestMu AI dashboard**. -2. Click on the **"Click to view live test"** button. -3. The status of the test changes to **idle_timeout** while using the Live Interaction feature. -4. Take control of the browser. diff --git a/static/docs/live-interactive-mobile-app-testing.md b/static/docs/live-interactive-mobile-app-testing.md deleted file mode 100644 index ce0c437cf..000000000 --- a/static/docs/live-interactive-mobile-app-testing.md +++ /dev/null @@ -1,185 +0,0 @@ -# Realtime Native Mobile App Testing - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Mobile app testing is the process of testing a mobile application to ensure that their functionality and usability is not comprised when installed across multiple Android and iOS devices. With TestMu AI, you can test mobile applications manually to ensure they perform seamlessly across all Android emulators and iOS simulators. - -## Uploading An App - -You can upload an app via a browser/finder or a public cloud link. If you don't already have an app, the TestMu AI sample app will help you easily get started with native app testing. - -### Uploading An App Via Browser - -**Step 1**: Login to your TestMu AI account. Visit **'Real Time Testing**' from the left panel and navigate to the **'App Testing'**. - -**Step 2**: Click on **'Upload'**. You can select an application either by drag and drop or browse for and select the application. The uploading process will take few moments depending upon the application'size. - -Your application will be uploaded successfully. Currently, you can upload `.apk` files up to 1GB. - -### Uploading An App Via Public Cloud Link - -**Step 1**: Login to your TestMu AI account. Visit **'Real Time Testing**' from the left panel and you'll be routed to the App testing console page. - -**Step 2**: Click on **'URL'**. Enter the App source URL and press Enter. - -Your application will be uploaded successfully. - -## Testing Mobile Applications On Android/iOS - -You can test your native mobile apps on Android emulators and iOS simulators. For demonstration, we will be picking up Android. - -**Step 1**: Select Real Time Testing from the left panel and navigate to **'App Testing'**. - -**Step 2**: Upload your Android app, select the device brand, choose the device and click on **'START'**. - -An Android emulator will fire up where you can perform native application testing of your mobile apps. - -> -**Note**: Similarly, you can perform mobile app testing on iOS simulators where all the steps and features will remain same. - -## Capturing Screenshots And Logging Bugs - -Once the virtual machine is launched, you'll notice the controller buttons on the left panel. - -**Step 1**: Click on the **'Mark As Bug'** icon - -**Step 2**: Once the screenshot has been captured, you can download the captured screenshot by **'Down Arrow'** icon or can use the built-in image editor tools to mark any bugs or tasks. After that click on **'Mark as Bug'**. - -**Step 3**: An integration specific ticket will appear. For example, if you have integration with YouTrack then fill up the details of your bugs in the YouTrack ticket and click on **'Create Issue'**. - - Your bugs will be marked successfully. - -## Recording Video Of A Test Session - -**Step 1**: From the left panel, click on the **'Video'** icon. - -**Step 2**: Once you click, the video recording of a current test session will start. To stop the recording, click again on the same video button. - -**Step 3**: To view your saved recording of test sessions, click on the **'Gallery'** icon from the left panel and navigate to Video. You can even download these recordings to your local system by clicking on the **'Down Arrow'** button. - -## DevTools For Live Debugging - -On clicking the Dev Tools option, you will come across two options i.e., Dev Tools and UI Inspector. - -Using developer tools, you can debug mobile apps on Android and iOS. Currently, you can view your Networks Logs, App Logs, and Device Logs. To view the logs, click on the 'DevTools' icon from the left and all the logs of the current device will be displayed. - -Upon selecting the UI Inspector feature, you can view all the XML details along with the properties table. By clicking on the arrow in the top right corner, UI Inspector allows you to download all the XML details. - -> **Note:** The window of the UI inspector is horizontally expandable if the XML tree is not visible (Only case when the XML tree is not visible properly). - -## Device Controls - -With the device controls, you can change the keyboard language, the device orientation, put device to sleep mode, increase and decrease volume. - -### Changing Keyboard Language - -From the left panel, select the **'Device Controls'** icon and from the dropdown menu, select your desired keyboard language. - -After a few seconds, your keyboard language will be changed. - -### Changing Device Orientation - -From the left panel, select the **'Device Controls'** icon and click on the **'Rotate'** button. - -The device will be rotated successfully. - -### Sleep Mode, Increase And Decrease Volume - -From the left panel, select the **'Device Controls'** icon. You'll notice three buttons as highlighted below. - -* To put the device on sleep mode, click on icon. -* To increase volume, click on icon. -* To decrease volume, click on icon. - -## Setting The Idle Time Out - -You can set the idle time out of your test session by clicking on the **'Settings'** icon from the left panel. Select the dropdown menu and choose the desired idle time out. - -## Geolocation Testing Of Mobile Apps - -You can test native mobile apps from different geolocations by clicking on **'Settings'** icon from the left panel. Select the dropdown menu and choose the desired country on which you wish to perform the test. - -You can perform Geolocation testing of mobile apps in the below countries. - -1. Argentina -2. Australia -3. Belgium -4. Brazil -5. Bulgaria -6. Canada -7. Chile -8. China -9. Croatia -10. Colombia -11. Czech Republic -12. Denmark -13. Finland -14. France -15. Georgia -16. Germany -17. Greece -18. Hong Kong -19. Hungary -20. India -21. Indonesia -22. Ireland -23. Israel -24. Italy -25. Japan -26. Kuwait -27. Malaysia -28. Mexico -29. Morocco -30. Netherlands -31. New Zealand -32. Nigeria -33. Norway -34. Philippines -35. Poland -36. Portugal -37. Qatar -38. Russia -39. Saudi Arabia -40. Singapore -41. South Africa -42. South Korea -43. Spain -44. Sweden -45. Switzerland -46. Taiwan -47. Thailand -48. Turkey -49. UAE -50. Ukraine -51. United Kingdom -52. United States -53. Vietnam - -## Installing Another App In A Running Test Session - -You can upload another app inside a running test session either by uninstalling the existing app or by installing the new app. You can also upload the apps from the already uploaded app list. - -**Step 1**: From the left panel, navigate to the Android icon and click on **'Install New'**. - -**Step 2**: Once you click on Install New, you will need to upload your app either via browser or a URL. - -Once your app is uploaded, you'll be guided to the VM where the application will be automatically downloaded and installed on the particular emulator/simulator. - -## Killing The Application - -You can kill/close the current running mobile application. Select the **'Android'** icon from the left panel and click on **'Kill App'**. - -## Uninstalling The Application - -You can uninstall the installed mobile application. Select the **'Android'** icon from the left panel and click on **'Uninstall App'**. - -## Ending The Test Session - -**Step 1**: From the left panel, click on the **'End Session'** icon. - -**Step 2**: A confirmation pop-up box will appear. Click on **'Yes, End Session'**. - -You will be logged out from the test session. - -> -That's all about live-interactive mobile application testing. In case you have any questions, please reach out to us at our **24*7 Chat Portal** or you could mail us at [**support@testmuai.com**](mailto:support@testmuai.com). diff --git a/static/docs/llms.txt b/static/docs/llms.txt deleted file mode 100644 index dfaf5b1e9..000000000 --- a/static/docs/llms.txt +++ /dev/null @@ -1,1328 +0,0 @@ -# TestMu AI (Formerly LambdaTest) Documentation - -> Documentation for TestMu AI (formerly LambdaTest) - cross-browser and real-device testing, automation, AI agents, accessibility, analytics, and integrations. - -Each link below points to the plain-Markdown copy at `/support/docs/.md`. - -Sitemap: https://www.testmuai.com/support/sitemap.xml -OpenAPI: https://www.testmuai.com/support/docs/OpenAPI.md - -## Agent skill - -- [TestMu AI Documentation Skill](https://www.testmuai.com/support/docs/SKILL.md): Product-specific operating guidance for AI agents (agentskills.io format) - how to discover docs, choose the right product area, and follow common testing workflows. - -## Docs - -- [A/B Testing and Baseline Variations in SmartUI](https://www.testmuai.com/support/docs/smartui-ab-testing-variations.md): Use SmartUI A/B testing on web screenshots and PDF pages to compare captures against multiple baseline variations (up to 20), see Matched with status, manage variations, and end an experiment by picking a winner. -- [Ability to Install and Uninstall Apps in App Automation Scripts](https://www.testmuai.com/support/docs/appium-install-uninstall-testmu-hooks.md): Now you can install or uninstall applications using hooks in your App Automation scripts that you run on TestMu AI. -- [ACCELQ Integration](https://www.testmuai.com/support/docs/accelq-integration.md): Accelerate your end-to-end digital testing journey with TestMu AI and ACCELQ. Automate your web and mobile apps across 3000+ real browsers, devices and operating system combinations. -- [Accelq Integration with HyperExecute](https://www.testmuai.com/support/docs/hyperexecute-accelq-integration.md): Leverage TestMu AI's cloud mobile devices to run automation testing using Accelq Studio. -- [Access and Change HyperExecute Org Product Preferences](https://www.testmuai.com/support/docs/hyperexecute-org-product-preferences.md): Find out how to access and change org product preferences for hyperexecute. -- [Access Management - KaneAI <> Test Manager](https://www.testmuai.com/support/docs/kaneai-test-manager-access-management.md): This documentation will help you to understand how to allocate licence to all the users present in your organization -- [Access your Contacts](https://www.testmuai.com/support/docs/contacts-on-real-devices.md): Now you can test your contacts in your app on TestMu AI Real Device Cloud Platform with 3000+ real mobile devices. -- [Access your Contacts on Real Devices (Browser)](https://www.testmuai.com/support/docs/contacts-on-real-devices-browser.md): Now you can test your contacts in your app on TestMu AI Real Device Cloud Platform with 10000+ real mobile devices. -- [Accessibility App Scanner (Overview)](https://www.testmuai.com/support/docs/accessibility-app-scanner.md): Manual mobile app accessibility testing on real devices: upload, session, scan screens, review issues, and export reports. -- [Accessibility Automation (Overview)](https://www.testmuai.com/support/docs/accessibility-automation.md): Overview of TestMu AI Accessibility Automation for web testing across supported frameworks and execution environments. -- [Accessibility Capture Screenshot](https://www.testmuai.com/support/docs/accessibility-capture-screenshot.md): Capture annotated screenshots with red bounding boxes highlighting accessibility violations during manual and automated testing. -- [Accessibility Compliance Guide (ADA / WCAG / EAA / 508)](https://www.testmuai.com/support/docs/accessibility-compliance-guide.md): High-level reference for common accessibility compliance frameworks and how Accessibility Testing supports those workflows. -- [Accessibility Crawler](https://www.testmuai.com/support/docs/accessibility-test-crawler.md): Learn how to use TestMu AI's Crawler to automate URL extraction for accessibility testing. Configure crawl settings, include/exclude URL patterns, and analyze scan results. -- [Accessibility Dashboard Widgets](https://www.testmuai.com/support/docs/analytics-modules-accessibility-widgets.md): Discover TestMu AI's Accessibility Dashboard widgets for comprehensive accessibility testing insights. Optimize your accessibility testing efforts today. -- [Accessibility DevTools (Overview)](https://www.testmuai.com/support/docs/accessibility-devtools.md): Overview of TestMu AI Accessibility DevTools for manual and assisted web accessibility testing. -- [Accessibility FAQs](https://www.testmuai.com/support/docs/accessibility-faq.md): This document consists of frequently asked questions related to the accessibility tool of TestMu AI. It will help you to answer some of your common questions. -- [Accessibility Issue Remediation Guide](https://www.testmuai.com/support/docs/accessibility-issue-remediation-guide.md): Use this guide to move from Accessibility findings to remediation planning, prioritization, and rule-level fixes. -- [Accessibility Label Not Punctuated](https://www.testmuai.com/support/docs/accessibility-ios-rule-accessibility-label-not-punctuated.md): Rule-level Accessibility guidance for Accessibility Label Not Punctuated on iOS. -- [Accessibility MCP Tool](https://www.testmuai.com/support/docs/accessibility-mcp-server.md): Run accessibility checks on hosted URLs or local React apps and get detailed remediation guidance using the Accessibility MCP Tool. -- [Accessibility Role Definition](https://www.testmuai.com/support/docs/accessibility-ios-rule-accessibility-role-definition.md): Rule-level Accessibility guidance for Accessibility Role Definition on iOS. -- [Accessibility Rules & Checklist](https://www.testmuai.com/support/docs/accessibility-rules-checklist.md): This page has moved. Use the Web, iOS, or Android accessibility checklists for supported rules and manual gaps, with links into each platform rule repository. -- [Accessibility Score](https://www.testmuai.com/support/docs/accessibility-web-score.md): How the Accessibility Score works across web and mobile accessibility testing-severity, issue density, scored elements, score bands, and how to use it without mistaking it for legal compliance. -- [Accessibility Testing on HyperExecute using Cypress v10](https://www.testmuai.com/support/docs/cypress-v10-hyprerexecute-accessibility-tests.md): Run Accessibility Automation with Cypress v10 and later on HyperExecute to detect and report accessibility issues during automated execution. -- [Accessibility Testing on HyperExecute using Cypress v9](https://www.testmuai.com/support/docs/cypress-v9-hyprerexecute-accessibility-tests.md): Run Accessibility Automation with Cypress v9 on HyperExecute to detect and report accessibility issues during automated execution. -- [Accessibility Testing on iOS Devices](https://www.testmuai.com/support/docs/accessibility-settings-ios.md): Ensure iOS app accessibility by testing text size, high-contrast, and reduce motion settings with real-time updates, notifications, and simplified animations. -- [Accessible Authentication (Minimum) (3.3.8)](https://www.testmuai.com/support/docs/accessibility-web-rule-3-3-8-accessible-authentication.md): Rule-level Accessibility guidance for Accessible Authentication (WCAG 3.3.8 Level AA). -- [Accessing Test Results From App Automation Dashboard](https://www.testmuai.com/support/docs/app-automation-dashboard.md): TestMu AI App Automation dashboard allows you to access test results, debug and fix failed test cases. This documentation guides you how to configure the desired capability for debugging test cases. -- [Account Management](https://www.testmuai.com/support/docs/account-management.md): Guide tpage to access the different components related to Account Management like Managing Subscriptions, Work on Service Accounts, Concurrency distribution in Sub-Organization, Support Access, Team Management, Multi Factor Authentication, hopw to Delete your Account, Group Management, Check Audit logs. -- [ADB Commands Support](https://www.testmuai.com/support/docs/adb-commands-support.md): TestMu AI provides support for limited commands which can be executed in your app automation test scripts via javascript executors. -- [ADB Shell](https://www.testmuai.com/support/docs/real-device-adb-shell.md): ADB shell support in TestMu AI for direct command execution on Android devices during manual testing. -- [Add Test Metadata Using the customData Capability](https://www.testmuai.com/support/docs/selenium-add-test-meta-data.md): Attach metadata like Jira tickets, GitHub PRs, and build info to your test runs using the customData capability. -- [Adding Custom Request Headers in SmartUI CLI](https://www.testmuai.com/support/docs/smartui-cli-request-headers.md): Learn how to configure the SmartUI CLI to send custom HTTP headers for fetching protected assets during visual regression testing. -- [Adding URLs for Web Scanner](https://www.testmuai.com/support/docs/web-scanner-adding-urls.md): A guide on the different methods for adding URLs to your scan, including manual input, CSV file upload, and importing from a sitemap XML. -- [Advanced Click Interactions in KaneAI](https://www.testmuai.com/support/docs/kane-ai-click-interactions.md): Author press and hold, multi-click, and right click interactions in KaneAI on Desktop Web, Android apps, iOS apps, and Mobile Web using natural language or manual recording. Covers supported patterns, replay behavior, and limitations. -- [Advanced Features in Web Scanner](https://www.testmuai.com/support/docs/web-scanner-advanced-features.md): Learn how to configure advanced features in Web Scanner including authentication (Basic Auth and Form Selectors), dedicated proxy, geolocation, and tunnel for testing locally hosted pages. -- [Advanced Settings in KaneAI Test Case](https://www.testmuai.com/support/docs/kaneai-advanced-settings.md): Learn how to edit advanced settings for your KaneAI test cases from the test summary page across desktop web, mobile app, and mobile web platforms. -- [Advanced TestMu AI Tunnel Features](https://www.testmuai.com/support/docs/advanced-tunnel-features.md): This provides you with all the Advanced features of the TestMu AI Tunnel. -- [Agent Features & Metrics - Customer Reference Guide](https://www.testmuai.com/support/docs/agent-features-and-metrics.md): A comprehensive overview of all customer-visible features and metrics for each agent type supported on the platform: Chat, Voice, Phone Caller Inbound, Phone Caller Outbound, and Image Analyzer. -- [Agent Mode](https://www.testmuai.com/support/docs/kane-cli-agent-mode.md): Use the --agent flag to get structured NDJSON output from Kane CLI: designed for AI coding agents like Claude Code, Codex CLI, and Gemini CLI. -- [Agent Mode Output](https://www.testmuai.com/support/docs/kane-cli-agent-output.md): Structured NDJSON output format for Kane CLI agent mode. -- [Agent Testing Platform FAQs: Setup, Security, Pricing](https://www.testmuai.com/support/docs/agent-testing-platform-faqs.md): Answers to common Agent Testing questions on the free tier, connecting an agent, firewall testing, HIPAA, phone testing, and evaluation accuracy. -- [AI Agent Skills - TestMu AI Browser Cloud](https://www.testmuai.com/support/docs/browser-cloud-skills.md): Give any AI agent a real browser with a ready-to-use skill for TestMu AI Browser Cloud. Works with Claude, Cursor, LangChain, CrewAI, OpenAI, and any LLM tool. -- [AI Issue Detection Agent](https://www.testmuai.com/support/docs/accessibility-ai-issue-detection-agent.md): Overview of AI-assisted issue interpretation and triage for Accessibility workflows where available. -- [AI Native Test Failure Analysis in HyperExecute](https://www.testmuai.com/support/docs/ai-powered-test-failure-analysis.md): This documentation will help you leverage Root Cause Analysis and Error Classification features in HyperExecute to view different kinds of errors and directly land on corrective measures or fixes. -- [AI Native Test Intelligence For Advance Test Analysis](https://www.testmuai.com/support/docs/test-intelligence-overview.md): Test Intelligence is an AI Native analytics module that helps you to identify the flaky tests in your test suite. It provides you with the list of tests that are flaky and the reason behind the flakiness of the test. -- [AI Root Cause Analysis (AI RCA) - Test Intelligence](https://www.testmuai.com/support/docs/analytics-ai-root-cause-analysis.md): Automatically analyze test failures with AI-powered Root Cause Analysis. Get instant insights into test failure patterns and accelerate debugging with actionable RCA results. -- [AI Test Generation: Requirements to Code](https://www.testmuai.com/support/docs/manual-to-automated-test-conversion.md): Convert manual test requirements into automated test scripts using AI. Generate test scenarios from text, documents, images, audio, Jira tickets, and more,then automate them with KaneAI and export code in Selenium, Playwright, Appium, and other frameworks. -- [Airbrake Integration](https://www.testmuai.com/support/docs/airbrake-integration.md): Streamline Your Error Monitoring with TestMu AI's Airbrake Integration Documentation. Discover how to seamlessly integrate Airbrake with TestMu AI for efficient bug management and error tracking. Get step-by-step instructions and insights for optimizing your development workflow. -- [algoQA Integration](https://www.testmuai.com/support/docs/algoqa-integration.md): Integrate TestMu AI with algoQA and automate your browser testing on an online browser farm of 3000+ real browsers and operating system combinations. -- [All Issues](https://www.testmuai.com/support/docs/accessibility-testing-dashboard-all-issues.md): Review the full list of Accessibility findings and filter the issue set for triage. -- [Allure Reports on HyperExecute](https://www.testmuai.com/support/docs/allure-reports.md): Learn how to generate Allure test report for HyperExecute on TestMu AI and download the reports from the dashboard -- [Allure Test Insights with HyperExecute](https://www.testmuai.com/support/docs/analytics-allure-api-widgets.md): Test Insights by executing test cases using the Allure report with HyperExecute -- [ALM Octane Integration with TestMu AI](https://www.testmuai.com/support/docs/alm-octane-integration.md): Learn how to integrate ALM Octane with TestMu AI for seamless test management and reporting while running your automated tests on the cloud. -- [Analytics](https://www.testmuai.com/support/docs/analytics-faqs.md): Analytics - FAQs -- [Analytics Dashboard AI CoPilot](https://www.testmuai.com/support/docs/analytics-dashboard-copilot.md): Analytics - Dashboard CoPilot to analyze your data and get insights faster then ever -- [Analytics Dashboard Features to manage your data](https://www.testmuai.com/support/docs/analytics-dashboard-features.md): Analytics - Dashboard Features for managing your data and setting up the dashboard -- [Analytics Dashboard Pre-built templates](https://www.testmuai.com/support/docs/analytics-dashboard-templates.md): Discover Dashboard Templates for streamlined test analysis. Save time with pre-built widgets tailored to specific testing needs. Learn more -- [Analytics Dashboard Settings](https://www.testmuai.com/support/docs/analytics-dashboard-settings.md): Customize your Analytics Dashboard settings for optimal test analysis. Learn how to adjust settings to fit your testing needs. -- [Analytics Drill-Down Widgets - Export Data Options](https://www.testmuai.com/support/docs/analytics-widget-drill-down-export.md): Export detailed testing analytics from TestMu AI's Drill-Down widgets to CSV, Excel, or PDF. Share insights, collaborate, and optimize testing processes effectively. -- [Analytics Modules - Jobs & Tests Overview](https://www.testmuai.com/support/docs/analytics-modules-hyperexecute.md): Analytics Modules - HyperExecute -- [Analytics Widgets - Filter by Regex](https://www.testmuai.com/support/docs/analytics-widget-filter-by-regex.md): Filter all the data with Regex patterns to get insights on your test runs. Learn how to use Regex in the analytics widget. -- [Android Accessibility Checklist](https://www.testmuai.com/support/docs/accessibility-android-what-we-cover.md): Supported Android accessibility rules with links to rule pages, plus a manual test checklist for human-led review. -- [Android Accessibility Manual Checklist](https://www.testmuai.com/support/docs/accessibility-android-what-we-do-not-cover.md): This page has moved. Android coverage and manual checklist are now on one page. -- [Android Accessibility Rules - Quick Reference](https://www.testmuai.com/support/docs/accessibility-android-rules.md): Use Android accessibility rule references to understand mobile-app findings from Accessibility App Scanner and Native App Automation. -- [Android Rule Repository](https://www.testmuai.com/support/docs/accessibility-android-rule-repository.md): Browse Android accessibility rule guidance, WCAG mappings, and remediation references. -- [Android TV Automation](https://www.testmuai.com/support/docs/appium-androidtv.md): Learn how to automate Android TV apps using Appium with a step-by-step tutorial and sample Python code. -- [Annotations in SmartUI Audit Logs](https://www.testmuai.com/support/docs/smartui-audit-logs-annotations.md): SmartUI audit logs now show the annotations applied to a screenshot, so you can see exactly which ignore regions were drawn, by whom, and when. -- [API Calls](https://www.testmuai.com/support/docs/kane-cli-api-calls.md): Have the Kane CLI agent make HTTP API calls directly inside an objective to seed data, hit a backend, then assert on or reuse the response. -- [API Reference - TestMu AI Browser Cloud](https://www.testmuai.com/support/docs/browser-cloud-api-reference.md): REST API reference for TestMu AI Browser Cloud. Planned endpoints for sessions, context, profiles, files, extensions, quick actions, and tunnels. -- [API Testing & Network Assertions](https://www.testmuai.com/support/docs/kaneai-kb-api-testing-and-network-assertions.md): Combine API testing with UI testing in KaneAI, make API calls, assert network responses, validate backend behavior alongside frontend interactions -- [App Automation Using Emulators and Simulators on TestMu AI](https://www.testmuai.com/support/docs/app-automation-on-emulators-simulators.md): Learn how to run app automated tests on using Emulators and Simulators on TestMu AI. -- [App Distribution on Real Devices](https://www.testmuai.com/support/docs/app-distribution.md): Upload, organize, and distribute Android and iOS app builds to internal QA and external beta testers using TestMu AI App Distribution. Share via authenticated download links, auto-notify testers on every new build, and launch builds on real devices in one click. -- [App Manager on Real Devices](https://www.testmuai.com/support/docs/app-manager.md): Manage and test multiple app versions on real devices using TestMu AI App Manager. Supports .apk, .aab, and .ipa uploads with version control and live testing. -- [App Profiling Insights](https://www.testmuai.com/support/docs/insights-app-profiling.md): Discover TestMu AI's App Profiling Insights for comprehensive test cases insights. Optimize your testing efforts today. -- [App Settings in Real Device Cloud](https://www.testmuai.com/support/docs/app-settings.md): Explore App Settings in TestMu AI's Real Device Cloud to enhance testing functionality and ensure comprehensive app validation. -- [Appium Flutter Integration Driver Testing](https://www.testmuai.com/support/docs/appium-flutter-integration.md): Guide to testing Flutter apps using Appium Flutter Integration Driver on TestMu AI real device cloud. -- [Appium Inspector Integration on TestMu AI](https://www.testmuai.com/support/docs/appium-inspector-integration.md): This documentation will help you integrate Appium Inspector with TestMu AI to interact with TestMu AI's Real Devices. With TestMu AI mobile testing cloud, you can test your mobile applications on wide range of real Android and iOS devices. -- [Appium Supported Languages and Frameworks](https://www.testmuai.com/support/docs/appium-languages-and-frameworks.md): Here is a list of languages and frameworks that are supported by the TestMu AI to run Appium automation tests on TestMu AI Real Device Cloud Platform. -- [Appium Testing - Real Device On HyperExecute](https://www.testmuai.com/support/docs/hyperexecute-appium-testing.md): Maximize Appium test execution with HyperExecute - Explore TestMu AI's support documentation for seamless automation testing. -- [Appium Testing - Virtual Device On HyperExecute](https://www.testmuai.com/support/docs/hyperexecute-appium-virtual-device.md): Execute Appium tests on virtual devices using HyperExecute with TestNG and YAML 0.2. Learn setup, configuration, and seamless test execution steps. -- [Appium Testing On HyperExecute - WebApp](https://www.testmuai.com/support/docs/hyperexecute-webapp-appium-testing.md): Now you can run your automation scripts using Selenium with Behave on TestMu AI online grid of 3000+ real desktop browsers and real operating systems. -- [Appium TestNG](https://www.testmuai.com/support/docs/accessibility-appium-testng.md): Appium TestNG with Accessibility: session capabilities, lambda-accessibility-scan checkpoints, and dashboard review. -- [Appium WebdriverIO](https://www.testmuai.com/support/docs/accessibility-appium-webdriverio.md): Appium WebdriverIO with Accessibility: WDIO config, async hooks, lambda-accessibility-scan, and reports. -- [Appium with Behat](https://www.testmuai.com/support/docs/appium-php-behat.md): Now you can run your automation scripts using Appium with PHP Behat on TestMu AI online grid of 3000+ real desktop browsers and real operating systems. -- [Appium with Behave](https://www.testmuai.com/support/docs/appium-python-behave.md): Now you can run your Appium automation scripts using with Behave on TestMu AI Real Device Cloud Platform of 3000+ real mobile devices. -- [Appium with C#](https://www.testmuai.com/support/docs/appium-csharp.md): Now you can run your automation scripts using Appium with CSharp on TestMu AI online grid of 3000+ real desktop browsers and real operating systems. -- [Appium With Cucumber for Java](https://www.testmuai.com/support/docs/appium-java-cucumber.md): Test your Appium automation scripts using Java Cucumber on TestMu AI Real Device Cloud Platform of 3000+ real mobile devices. -- [Appium with Cucumber for Ruby](https://www.testmuai.com/support/docs/appium-ruby-cucumber.md): Complete guide to running your first Cucumber Appium automated test script on TestMu AI Real Device Cloud Platform. Test on 3000+ Real Devices. -- [Appium with Gauge](https://www.testmuai.com/support/docs/appium-python-gauge.md): Complete guide to running your first Gauge Appium automated test script on TestMu AI Real Device Cloud Platform. Test on 3000+ Real Devices. -- [Appium with Java](https://www.testmuai.com/support/docs/appium-java.md): Automate Mobile Apps with ease using Java and Appium -- [Appium With JBehave](https://www.testmuai.com/support/docs/appium-java-jbehave.md): Run your Appium automation scripts using Java JBehave on TestMu AI Real Device Cloud Platform of 3000+ real mobile devices. -- [Appium With JUnit](https://www.testmuai.com/support/docs/appium-java-junit.md): Automate Mobile App using Java Appium Testing with JUnit Framework -- [Appium With Kotlin](https://www.testmuai.com/support/docs/appium-kotlin.md): Now you can run your Appium automation scripts using Kotlin on TestMu AI Real Device Cloud Platform of 3000+ real mobile devices. -- [Appium With Mocha](https://www.testmuai.com/support/docs/appium-nodejs-mocha.md): Now you can run your automation scripts using JavaScript Mocha on TestMu AI online grid of 3000+ real desktop browsers and real operating systems. -- [Appium with Nodejs](https://www.testmuai.com/support/docs/appium-nodejs.md): Optimize your Appium automation with Node.js using TestMu AI's comprehensive support documentation. -- [Appium with NUnit](https://www.testmuai.com/support/docs/appium-csharp-nunit.md): Now you can run your Appium automation scripts using NUnit on TestMu AI Real Device Cloud Platform of 3000+ real mobile devices. -- [Appium with PHP](https://www.testmuai.com/support/docs/appium-php.md): Now you can run your automation scripts using Appium with PHP on TestMu AI online grid of 3000+ real desktop browsers and real operating systems. -- [Appium with Pytest](https://www.testmuai.com/support/docs/appium-python-pytest.md): Complete guide to running your first Pytest Appium automated test script on TestMu AI Real Device Cloud Platform. Test on 3000+ Real Devices. -- [Appium with Python](https://www.testmuai.com/support/docs/appium-python.md): Complete guide to running your first Python Appium automated test script on TestMu AI Real Device Cloud Platform. Test on 3000+ Real Devices. -- [Appium with Robot](https://www.testmuai.com/support/docs/appium-python-robot.md): Now you can run your Appium automation scripts using with Robot on TestMu AI Real Device Cloud Platform of 3000+ real mobile devices. -- [Appium with Rspec](https://www.testmuai.com/support/docs/appium-ruby-rspec.md): Now you can run your Appium automation scripts using Rspec on TestMu AI Real Device Cloud Platform of 3000+ real mobile devices. -- [Appium with Ruby](https://www.testmuai.com/support/docs/appium-ruby.md): Complete guide to running your first Ruby Appium automated test script on TestMu AI Real Device Cloud Platform. Test on 3000+ Real Devices. -- [Appium With TestNG](https://www.testmuai.com/support/docs/appium-java-testng.md): Appium Testing with Java and TestNG at TestMu AI:Master Mobile Automation. -- [Appium With WebDriverIO](https://www.testmuai.com/support/docs/appium-nodejs-webdriverio.md): Now you can run your Appium automation scripts using with WebDriverIO on TestMu AI Real Device Cloud Platform of 3000+ real mobile devices. -- [Apple Pay Automation on Real Devices](https://www.testmuai.com/support/docs/apple-pay-auto.md): Learn how to automate Apple Pay flows on iOS public devices, including capabilities, payment injection, passcode entry, and limitations. -- [Apple Pay Manual Testing on Real Devices](https://www.testmuai.com/support/docs/apple-pay-manual.md): Learn how to manually test Apple Pay flows on iOS public devices in App Live, including enabling Apple Pay, configuring wallets, and completing transactions. -- [Apple TV With Appium](https://www.testmuai.com/support/docs/appium-appletv.md): Complete guide to running your first AppleTV Appium automated test script on TestMu AI Real Device Cloud Platform. Test on 3000+ Real Devices. -- [Application Performance Analytics](https://www.testmuai.com/support/docs/appium-app-performance-analytics.md): Learn more about the metrics that are crucial in tracking the performance of your app. -- [Applitools Integration With TestMu AI](https://www.testmuai.com/support/docs/applitools-integration-with-testmu.md): TestMu AI integration with Applitools will allow you perform automated cross browser testing on more than 2000 real browsers running on real operating systems, hosted on a cloud-based Selenium Grid. You can execute your automation test script on TestMu AI Selenium Grid directly from your Applitools instance. -- [Applitools Integration With TestMu AI For Cypress](https://www.testmuai.com/support/docs/applitools-integration-cypress.md): TestMu AI integration with Applitools will allow you perform automated cross browser testing on more than 2000 real browsers running on real operating systems, hosted on a cloud platform. You can execute your Cypress automation test scripts on TestMu AI directly from your Applitools instance. -- [Asana Integration For Easy Browser Testing | TestMu AI](https://www.testmuai.com/support/docs/asana-integration.md): With TestMu AI - Asana integration, now you can mark your bug to Asana dashboard directly from TestMu AI while performing testing. -- [Assembla Integration with TestMu AI](https://www.testmuai.com/support/docs/assembla-integration.md): TestMu AI helps you manage your bugs in Assembla by moving them to project in a single click. All the details you provide in TestMu AI like task list, assignee, title and description would automatically be presented in the project on Assembla. -- [Assertions & Validation](https://www.testmuai.com/support/docs/kaneai-kb-assertions-and-validation.md): Comprehensive guide to writing reliable assertions in KaneAI with real-world examples, best practices, and workarounds for common validation scenarios -- [Assistive Technology Access](https://www.testmuai.com/support/docs/accessibility-ios-rule-assistive-technology-access.md): Rule-level Accessibility guidance for Assistive Technology Access on iOS. -- [Assistive Touch on Real iOS Devices](https://www.testmuai.com/support/docs/assistive-touch-on-real-ios-devices.md): Simulate system-level actions like Home, Notification Center, Screenshot, and more during manual testing on real iOS devices. -- [Assistive Touch on Real iOS Devices (Browser)](https://www.testmuai.com/support/docs/assistive-touch-on-real-ios-browser.md): Trigger system actions like Home, Notification Center, and Screenshot while running manual browser testing sessions on real iOS devices. -- [Associate Test Cases](https://www.testmuai.com/support/docs/hyperexecute-azure-associate-test-case.md): Associate test cases with Test Plan via Visual Studio Code. Follow the instructions to integrate your test cases and enhance your testing process seamlessly -- [Assurance in CI and from Agents](https://www.testmuai.com/support/docs/kane-cli-assurance-automation.md): The headless contract for kane-cli assurance - the --mode agent|ci|override ask policy, exit codes, the NDJSON event streams for extract, design, and reconcile, and the pause -> answer -> resume loop. -- [Audio Control (1.4.2)](https://www.testmuai.com/support/docs/accessibility-web-rule-1-4-2-audio-control.md): Rule-level Accessibility guidance for Audio Control (WCAG 1.4.2 Level A). -- [Audio Injection Manual Testing on Real Device Browsers](https://www.testmuai.com/support/docs/audio-injection-manual-browser.md): Manually test microphone-driven web flows on real Android and iOS device browsers in Real Time by injecting pre-recorded audio files or streaming live input from your system mic. -- [Audio Injection Manual Testing on Real Devices](https://www.testmuai.com/support/docs/audio-injection-manual.md): Manually test microphone-driven flows on real Android and iOS devices in App Live by injecting pre-recorded audio files or streaming live input from your system mic. -- [Audio Injection/Input on Real Devices](https://www.testmuai.com/support/docs/audio-injection.md): Test audio-driven and microphone-dependent features on real Android and iOS devices by injecting pre-recorded audio files directly into the device microphone. -- [Audit Logs](https://www.testmuai.com/support/docs/audit-logs.md): Guide to Access, Filter and Download Audit Logs -- [Authentication](https://www.testmuai.com/support/docs/authentication.md): Explore comprehensive documentation on Cypress Automation and Testing for building robust, secure authentication systems. Understand authentication processes and bolster your testing efficiency. -- [Authentication & Session Management](https://www.testmuai.com/support/docs/kaneai-kb-authentication-and-session-management.md): Test login flows, TOTP/MFA, SSO, session persistence, and secure authentication patterns in KaneAI -- [Author Your First Desktop Browser Test](https://www.testmuai.com/support/docs/author-your-first-desktop-browser-test.md): In this detailed guide, you will learn how you can manually author your first desktop browser test with KaneAI. -- [Author Your First Mobile App Test](https://www.testmuai.com/support/docs/author-your-first-mobile-app-test.md): In this detailed guide, you will learn how you can manually author your first mobile app test with KaneAI. -- [Author Your First Mobile Browser Test](https://www.testmuai.com/support/docs/author-your-first-mobile-browser-test.md): In this detailed guide, you will learn how you can manually author your first mobile browser test with KaneAI. -- [Auto Healing in HyperExecute](https://www.testmuai.com/support/docs/hyperexecute-auto-healing.md): This documentation will help you to understand the concept of auto healing in hyperexecute -- [Auto-Heal for Automation Scripts in KaneAI with HyperExecute](https://www.testmuai.com/support/docs/kaneai-auto-heal.md): This documentation will help you to understand how auto-healing works on KaneAI -- [Auto-Wait and Retry for Screenshot Status (fetchScreenshotStatus timeout)](https://www.testmuai.com/support/docs/smartui-hooks-fetch-status-timeout.md): Use the timeout option on the smartui.fetchScreenshotStatus hook to make SmartUI poll for the comparison result automatically instead of writing your own retry loop. -- [Automate Unity Games With AltTester on TestMu AI Real Device Cloud](https://www.testmuai.com/support/docs/alttester-unity-game-automation.md): Run AltTester-instrumented Unity game tests on real Android and iOS devices with TestMu AI, using AltDriver, Appium, the LT tunnel, and pytest. -- [Automate your App Testing with KaneAI](https://www.testmuai.com/support/docs/kane-ai-app-test.md): Learn how to automate app testing with KaneAI. Generate and execute tests in plain English, save cases, and run them seamlessly -- [Automate your Web App Testing with KaneAI](https://www.testmuai.com/support/docs/kane-ai-web-test.md): Author your first test with KaneAI -- [Automated Screenshot API For Cross Browser Testing](https://www.testmuai.com/support/docs/automated-screenshot-api-for-cross-browser-testing.md): Automated Screenshot API will help you capture full-page screenshots of any URL by invoking an API call to TestMu AI cloud servers. You can ensure cross browser compatibility of your website stays in check by performing bulk screenshot testing on a regular basis. -- [Automating Accessibility Testing with Selenium](https://www.testmuai.com/support/docs/accessibility-automation-test.md): Identify and log accessibility issues with TestMu AI Accessibility Automation, ensuring automated compliance with WCAG accessibility standards. -- [Automating Test Runs Execution with CI/CD on KaneAI](https://www.testmuai.com/support/docs/kaneai-ci-cd-automation.md): Automate test run execution on KaneAI with CI/CD tools like GitHub Actions and Jenkins. Follow our step-by-step guide for seamless integration. -- [Automation Tests with Accessibility Tool using Cypress v10](https://www.testmuai.com/support/docs/cypress-v10-accessibility-test.md): Run Accessibility Automation in Cypress v10 and later to detect and report accessibility issues during automated web test execution. -- [Automation Tests with Accessibility Tool using Cypress v9](https://www.testmuai.com/support/docs/cypress-v9-accessibility-test.md): Run Accessibility Automation in Cypress v9 to detect and report accessibility issues during automated web test execution. -- [Automation Tests with Accessibility Tool using Playwright](https://www.testmuai.com/support/docs/playwright-accessibility-test.md): Use TestMu AI Accessibility Automation with Playwright to detect and report accessibility issues during automated test execution. -- [AutoSplit Strategy](https://www.testmuai.com/support/docs/hyperexecute-auto-split-strategy.md): Explore HyperExecute Auto Split Strategy | Optimize Testing Efficiency - Discover how HyperExecute revolutionizes testing with auto split strategy for enhanced efficiency. -- [Axosoft Integration](https://www.testmuai.com/support/docs/axosoft-integration.md): With TestMu AI - Axosoft integration, now you can mark your bug to Axosoft dashboard directly from TestMu AI while performing testing. -- [Azure AD Scim User Provisioning](https://www.testmuai.com/support/docs/azure.md): Integrating TestMu AI SCIM with Azure AD -- [Azure DevOps Integration](https://www.testmuai.com/support/docs/azure-with-hyperexecute.md): TestMu AI now integrates with Azure DevOps Pipeline to boost your go-to market delivery. Perform automated cross browser testing with TestMu AI to ensure your development code renders seamlessly through an online Selenium grid providing 3000+ real browsers running through machines. -- [Azure Pipeline Integration with SmartUI](https://www.testmuai.com/support/docs/smartui-with-azure.md): SmartUI now integrates with Azure Pipeline to boost pipeline delivery. Perform automated cross browser testing with SmartUI to seamlessly providing 3000+ real browsers running through machines. -- [Azure Test Plan](https://www.testmuai.com/support/docs/hyperexecute-azure-setup-test-plan.md): Learn how to set up your Azure Test Plan project for seamless integration with HyperExecute. -- [Azure Test Plan Integration with HyperExecute](https://www.testmuai.com/support/docs/hyperexecute-azure-test-plan.md): Seamless Azure Test Plan integration with HyperExecute for efficient test management and execution. Boost software quality effortlessly. -- [Background Services](https://www.testmuai.com/support/docs/hyperexecute-background-services.md): This documentation will help you -- [Bamboo CI](https://www.testmuai.com/support/docs/bamboo-integration-with-hyperexecute.md): TestMu AI now integrates with Bamboo CI Pipeline to boost your go-to market delivery. Perform automated cross browser testing with TestMu AI to ensure your development code renders seamlessly through an online Selenium grid providing 3000+ real browsers running through machines. -- [Bar and Line Chart Widgets](https://www.testmuai.com/support/docs/analytics-bar-line-widgets.md): Learn how to create and configure Bar and Line chart widgets to visualize test data trends, comparisons, and patterns over time or across categories. -- [Baseline Management in SmartUI](https://www.testmuai.com/support/docs/smartui-baseline-management.md): Learn how to manage baselines effectively in TestMu AI SmartUI. Understand baseline creation, updating, and versioning for reliable visual regression testing. -- [Basic Authentication for Safari Web Automation](https://www.testmuai.com/support/docs/basic-authentication-for-web-automation.md): Now you can Basic Authentication during your Safari web automation sessions on iOS real devices with TestMu AI Real Device Cloud Platform with 3000+ real mobile devices. -- [Basic Authentication in SmartUI CLI](https://www.testmuai.com/support/docs/smartui-cli-basic-auth.md): Learn how to capture screenshots of assets protected behind authentication using SmartUI CLI -- [Beta Testers and Teams Management](https://www.testmuai.com/support/docs/beta-testers-management.md): Add individual beta testers, group them into Beta Teams, and use them to control who can download builds and receive notifications in TestMu AI App Distribution. -- [BiDi Testing with Selenium WebDriver on TestMu AI](https://www.testmuai.com/support/docs/selenium-bidi-integration.md): Run BiDi tests with Selenium WebDriver on the cloud grid for event-driven, cross-browser automation. -- [Biometric Authentication](https://www.testmuai.com/support/docs/biometric-authentication.md): Test biometric authentication functionalities such as fingerprint or face recognition in your app on TestMu AI Real Device Cloud Platform with 3000+ real mobile devices. -- [Biometric Authentication on Real Devices](https://www.testmuai.com/support/docs/biometric-authentication-on-real-devices.md): Learn how to test biometric authentication in iOS and Android apps using TestMu AI's real device testing platform. Enhance app security and user experience. -- [Bitbucket Integration](https://www.testmuai.com/support/docs/bitbucket-integration.md): TestMu AI provides integrations with Bitbucket to help you log bugs directly from the middle of your test session on TestMu AI to your Bitbucket repository. -- [Bitbucket Pipeline Integration](https://www.testmuai.com/support/docs/bitbucket-pipeline-with-hyperexecute.md): HyperExecute now integrates with Bitbucket CI to boost your go-to market delivery. Perform automated cross browser testing with TestMu AI to ensure your development code renders seamlessly through an online Selenium grid providing 3000+ real browsers running through machines. -- [Bitbucket Pipeline Integration with SmartUI](https://www.testmuai.com/support/docs/smartui-with-bitbucket.md): SmartUI now integrates with Bitbucket Pipeline to boost pipeline delivery. Perform automated cross browser testing with SmartUI to seamlessly providing 3000+ real browsers running through machines. -- [Bitbucket Pipelines Integration With TestMu AI](https://www.testmuai.com/support/docs/bitbucket-pipelines-integration-with-testmu.md): TestMu AI integration with Bitbucket Pipelines will help you perform cross-browser testing on 3000+ real browsers & browser versions, through a Selenium Grid hosted on TestMu AI cloud servers. -- [Boost Your API Testing Efficiency with HyperExeucte](https://www.testmuai.com/support/docs/hyperexecute-api-testing.md): Master API Testing with HyperExecute! This technical doc explains API test automation using Hyperexecute. Learn how to ensure efficient API testing. -- [Branch-Level Merging in SmartUI](https://www.testmuai.com/support/docs/smartui-branch-merging.md): Learn how to effectively merge branches in SmartUI for visual regression testing workflow management. -- [Branching Strategy in SmartUI SDK](https://www.testmuai.com/support/docs/smartui-cli-git-branching-strategy.md): In this documentation, learn how the Git Branching Strategy is implemented for the SmartUI SDK -- [Breeze Integration](https://www.testmuai.com/support/docs/breeze-integration-with-testmu.md): TestMu AI integration with Breeze will help you log UI bugs or observations in your Breeze project as you perform cross browser testing on TestMu AI. -- [Browser State Actions](https://www.testmuai.com/support/docs/kane-cli-browser-state.md): Set cookies, localStorage, and the clipboard directly from a Kane CLI objective to seed state before a flow or test copy and paste behavior. -- [Bug Report](https://www.testmuai.com/support/docs/accessibility-report-bug.md): Create a bug from Accessibility report data and send it to your integrated issue workflow. -- [Bug Tracking Tools](https://www.testmuai.com/support/docs/bug-tracking-tools.md): TestMu AI offers integrations with numerous Bug Tracking tools to help you mark and track bugs for pacing up your debugging. -- [Bugasura Integration](https://www.testmuai.com/support/docs/bugasura-integration.md): Integrate TestMu AI with Bugasura for efficient one-click bug logging of your web applications and avoid the hassle of maintaining bugs and tasks manually. -- [BugHerd Integration with TestMu AI](https://www.testmuai.com/support/docs/bugherd-integration.md): TestMu AI helps you manage your bugs in BugHerd by moving them to project in a single click. All the details you provide in TestMu AI like task list, assignee, title and description would automatically be presented in the project on BugHerd. -- [Bugsnag Integration](https://www.testmuai.com/support/docs/bugsnag-integration.md): With TestMu AI and Bugsnag integration, you can easily mark your bugs, keep a track of them, share, and collaborate with your team on your Bugsnag projects. -- [Bugzilla Integration](https://www.testmuai.com/support/docs/bugzilla-integration.md): Perform one-click bug logging with TestMu AI and Bugzilla integration. Capture screenshots, annotate issues, and share them directly from the TestMu AI platform to your Bugzilla products. -- [Build Comparison - Compare Test Builds and Track Regressions](https://www.testmuai.com/support/docs/analytics-build-comparison.md): Compare two builds side by side to identify new failures, fixed tests, and stability changes across your test suite -- [Build Configuration and Options for Smart UI](https://www.testmuai.com/support/docs/smart-ui-build-options.md): Navigate through our support doc to adeptly build Configuration & Options for Smart UI using TestMu AI! -- [Build Inactivity Time](https://www.testmuai.com/support/docs/build-inactivity-time.md): Organize and manage automation builds by setting a build inactivity time interval for automatic build splitting. -- [Build Insights - Analyze your test builds and get build level insights](https://www.testmuai.com/support/docs/analytics-build-insights.md): Analytics - Builds Insights for analyzing test results and build health over time -- [Build-Level Merging in SmartUI](https://www.testmuai.com/support/docs/smartui-build-merging.md): Learn how to effectively merge builds in SmartUI for granular control over your visual regression testing workflow. -- [Building Mobile Applications Using Tools](https://www.testmuai.com/support/docs/hyperexecute-build-mobile-apps-using-tools.md): Building Mobile Applications Using Tools -- [Building the Context Graph](https://www.testmuai.com/support/docs/kane-cli-assurance-context.md): Snapshot requirement documents into a local, content-addressed store and extract use-cases with an AI agent using kane-cli context - ingest, extract, review, sessions, inspection, and housekeeping. -- [Building Your iOS App for Simulator Testing](https://www.testmuai.com/support/docs/build-ios-app-for-simulator-testing.md): Learn how to build your iOS app correctly for testing on iOS Simulators on TestMu AI Cloud. Covers Xcode UI, command line builds, and build verification. -- [Buildkite Integration With TestMu AI](https://www.testmuai.com/support/docs/buildkite-integration-with-testmu.md): TestMu AI offers continuous integration with Buildkite to ease your automated cross browser testing process through their Selenium grid consisting more than 3000+ browsers. -- [Buildkite Pipeline Integration with SmartUI](https://www.testmuai.com/support/docs/smartui-with-buildkite.md): SmartUI now integrates with Buildkite Pipeline to boost pipeline delivery. Perform automated cross browser testing with SmartUI to seamlessly providing 3000+ real browsers running through machines. -- [Bulk Module Update](https://www.testmuai.com/support/docs/kaneai-bulk-module-update.md): Learn how to update module versions across multiple KaneAI test cases in a single action using the Bulk Module Update feature. -- [Bulk Update Fields](https://www.testmuai.com/support/docs/bulk-update-fields.md): Learn how to Bulk update system & custom fields. -- [Button Click By Text (OCR)](https://www.testmuai.com/support/docs/button-text-click.md): Click a button by its visible text or icon description using OCR on LambdaTest Real Device Cloud. Works across Android & iOS real and virtual devices when native locators are unavailable. -- [Button Element Capitalisation Check](https://www.testmuai.com/support/docs/accessibility-ios-rule-button-element-capitalisation-check.md): Rule-level Accessibility guidance for Button Element Capitalisation Check on iOS. -- [Bypass Blocks (2.4.1)](https://www.testmuai.com/support/docs/accessibility-web-rule-2-4-1-bypass-blocks.md): Rule-level Accessibility guidance for Bypass Blocks (WCAG 2.4.1 Level A). -- [Camera Image Injection](https://www.testmuai.com/support/docs/camera-image-injection.md): Simulate camera input to test camera-based features like QR scanning and document capture without needing physical devices. -- [Capabilities for Playwright Android Testing](https://www.testmuai.com/support/docs/capabilities-for-playwright-android.md): Master Playwright test configuration including browser and OS selection, test organization, and desktop resolution adjustments for your tests. -- [Capabilities for Playwright Testing](https://www.testmuai.com/support/docs/capabilities-for-playwright.md): Master Playwright testing with our guide on configuring capabilities! Select browsers & OS, organize tests, adjust desktop resolution, and more for efficient and precise testing outcomes. -- [Capabilities for Puppeteer Testing](https://www.testmuai.com/support/docs/capabilities-for-puppeteer.md): Learn how to configure the capability for selecting browsers and OS, organzing tests, changing desktop resolution, and more for your Playwright tests. -- [Capture Screenshots by Freezing Animations](https://www.testmuai.com/support/docs/smartui-freeze-animations.md): SmartUI can now capture the screenshots of your animations running in the browser by freezing it. -- [Capture Static URLs via CLI](https://www.testmuai.com/support/docs/smartui-cli.md): In this documentation, learn how to perform Visual UI Testing using command line interface on the TestMu AI Automation Cloud across 40+ browser versions. -- [Capturing Full-Page Screenshots Of Your Websites](https://www.testmuai.com/support/docs/full-page-screenshots.md): LT Browser allows you to capture full-page screenshots of your pre-installed mobile devices, desktops and laptops and compare them across multiple viewports. -- [Capturing Multiple Assets Hosts](https://www.testmuai.com/support/docs/smartui-multiple-assets-hosts.md): In this documentation, learn about how to capture multiple assts hosts -- [Capturing Responsive DOM with SmartUI SDK](https://www.testmuai.com/support/docs/smartui-cli-responsive-dom.md): In this documentation, learn how to capture responsive DOM by capturing multiple screenshots of the same DOM -- [Change Browser Window Size](https://www.testmuai.com/support/docs/change-browser-window-size.md): Set browser resolution for Selenium tests on TestMu AI using the resolution capability in LT:Options. -- [Changelog](https://www.testmuai.com/support/docs/kane-cli-changelog.md): Kane CLI release history, version notes, and links to GitHub releases. -- [Changing Individual Test Details](https://www.testmuai.com/support/docs/change-individual-test-details.md): Mark test status as pass or fail and rename tests during execution on the Selenium grid. -- [Character Key Shortcuts (2.1.4)](https://www.testmuai.com/support/docs/accessibility-web-rule-2-1-4-character-key-shortcuts.md): Rule-level Accessibility guidance for Character Key Shortcuts (WCAG 2.1.4 Level A). -- [Chat Agent Testing With TestMu AI](https://www.testmuai.com/support/docs/chat-agent.md): Test chat agents on TestMu AI. How multi-turn conversations are simulated, which quality metrics are scored, and how to run an evaluation from UI, CLI, or API. -- [Check Responsiveness of Your Website | TestMu AI](https://www.testmuai.com/support/docs/responsive-testing.md): Check responsiveness of your website across multiple devices and screensize with TestMu AI with just one click. -- [Checkpoints](https://www.testmuai.com/support/docs/kane-cli-checkpoints.md): Checkpoints are verification points KaneAI evaluates during test execution: assert conditions, branch on results, or extract values for later use. -- [Choosing the Right Accessibility Tool](https://www.testmuai.com/support/docs/accessibility-choosing-the-right-tool.md): Pick DevTools, Automation, Test Scheduling, Web Scanner, mobile paths, reports, checklists, or MCP based on your workflow, with links to each doc. -- [Chrome Options](https://www.testmuai.com/support/docs/kaneai-chrome-options.md): This documentation will help you to understand how to use the Chrome Options feature while testing your test cases via KaneAI. -- [ChromeOS App Testing](https://www.testmuai.com/support/docs/chrome-os-app-testing.md): TestMu AI provides a powerful solution for testing apps designed for Chromebooks by offering live interactive testing capabilities, and device emulation for responsive design testing. -- [ChromeOS Testing](https://www.testmuai.com/support/docs/chrome-os-testing.md): TestMu AI provides a powerful solution for testing apps designed for Chromebooks by offering live interactive testing capabilities, and device emulation for responsive design testing.s -- [ChromeOS Web Browser Testing](https://www.testmuai.com/support/docs/chrome-os-web-browser-testing.md): LChromeOS web browser testing with features like bug marking, IP geolocation, device controls, session recording, network throttling, and more for seamless testing. -- [CI/CD Integration](https://www.testmuai.com/support/docs/kane-cli-cicd.md): Integrate Kane CLI into GitHub Actions, GitLab CI, Jenkins, Bitbucket Pipelines, and Docker for automated browser testing. -- [CI/CD Integration Guide](https://www.testmuai.com/support/docs/accessibility-cicd-integration-guide.md): Connect Accessibility Automation to CI/CD: secrets, matrix jobs, gating, and where to review reports after each build. -- [Circle CI Pipeline Integration with SmartUI](https://www.testmuai.com/support/docs/smartui-with-circle-ci.md): SmartUI now integrates with Circle CI Pipeline to boost pipeline delivery. Perform automated cross browser testing with SmartUI to seamlessly providing 3000+ real browsers running through machines. -- [CircleCI Integration with Hyperexecute](https://www.testmuai.com/support/docs/circle-ci-with-hyperexecute.md): HyperExecute integrates with CircleCI to boost your go-to market delivery. Perform automated cross browser testing with TestMu AI to ensure your development code renders seamlessly through an online Selenium grid providing 3000+ real browsers running through machines. -- [CLI FAQs](https://www.testmuai.com/support/docs/hyperexecute-cli-faqs.md): Explore HyperExecute CLI FAQs Downloading Test Artifacts, MacOS Verification, Test Run Steps & More! Get Quick Answers Here. -- [CLI Reference](https://www.testmuai.com/support/docs/kane-cli-cli-reference.md): Complete command and flag reference for Kane CLI: all commands, options, exit codes, TUI slash commands, keyboard shortcuts, and directory structure. -- [ClickUp](https://www.testmuai.com/support/docs/clickup-integration.md): Mark UI bugs to your ClickUp project directly while performing cross browser testing with TestMu AI. Capture screenshot, annotate and assign it to a colleague. -- [Clipboard Assertions](https://www.testmuai.com/support/docs/kane-cli-checkpoint-devtools-clipboard.md): Verify what a Copy button actually copied, extract copied values into variables, and confirm clipboard state, using an isolated test clipboard. -- [CodeceptJS With Selenium - Run Tests on TestMu AI](https://www.testmuai.com/support/docs/codeceptjs-with-selenium.md): Run CodeceptJS Selenium tests on TestMu AI's cloud grid across 3000+ browsers and OS. Use codeceptjs-lambdatest-service to auto-sync test names and statuses. -- [Codefresh CI Integration With TestMu AI](https://www.testmuai.com/support/docs/codefresh-ci-integration-with-testmu.md): TestMu AI offers continuous integration with Codefresh to ease your automated cross browser testing process through their Selenium grid consisting more than 3000+ browsers. -- [Codeship Integration With TestMu AI](https://www.testmuai.com/support/docs/codeship-integration-with-testmu.md): TestMu AI offers continuous integration with Codeship to ease your automated cross browser testing process through their Selenium grid consisting more than 3000+ browsers. -- [Color Contrast Issues](https://www.testmuai.com/support/docs/accessibility-ios-rule-color-contrast-issues.md): Rule-level Accessibility guidance for Color Contrast Issues on iOS. -- [Command Logs Analytics with Test Intelligence](https://www.testmuai.com/support/docs/analytics-modules-test-intelligence-command-logs-analytics.md): A Explore TestMu AI's powerful Analytics Modules for comprehensive test intelligence and command logs analytics. Elevate your testing efficiency today. -- [Commonly Used APIs For Mobile App Testing](https://www.testmuai.com/support/docs/app-testing-apis.md): Quick guide on commonly used APIs for Mobile App testing. -- [Comparison Settings for SmartUI - Pixel to Pixel Comparison](https://www.testmuai.com/support/docs/test-settings-options.md): In this documentation, you will learn about the following test settings options and their meanings. -- [Comprehensive Troubleshooting Guide for SmartUI](https://www.testmuai.com/support/docs/smartui-troubleshooting-guide.md): Comprehensive troubleshooting guide for common SmartUI issues, including solutions, best practices, and diagnostic steps -- [Concurrency Usage Insights](https://www.testmuai.com/support/docs/analytics-modules-resource-utilization.md): Optimize resource use with TestMu AI's Concurrency Usage Insights. Monitor concurrency trends at Org, Group, SubOrg levels and track custom concurrency with custom KPIs. -- [Conditional Logic (If / Else-If / Else) in KaneAI](https://www.testmuai.com/support/docs/kaneai-conditional-logic.md): Learn how to use If, Else-If, and Else conditions in KaneAI to build branching test flows with support for modules, JavaScript, API, and DB steps inside each branch. -- [Configuration](https://www.testmuai.com/support/docs/kane-cli-configuration.md): Configure Kane CLI settings including window size, Chrome profiles, Test Manager project and folder, run mode, and code export. -- [Configurations in KaneAI & Test Manager](https://www.testmuai.com/support/docs/test-runs-configurations.md): Learn how to create, manage, and apply reusable environment configurations for your test runs in LambdaTest Test Manager. -- [Configure Accessibility Automation](https://www.testmuai.com/support/docs/accessibility-automation-settings.md): Configure capabilities for TestMu AI Accessibility Automation including WCAG version, best practices, and needs review settings. -- [Configure Accessibility DevTools](https://www.testmuai.com/support/docs/accessibility-devtools-settings.md): Customize your testing experience with TestMu AI Accessibility DevTools' comprehensive settings to meet your specific needs and preferences. -- [Configure Date, Time & Hour Format on Real Devices Using Appium](https://www.testmuai.com/support/docs/appium-date-and-time.md): This document provides information about configuring date, time, and hour-format on Real Devices using Appium automation on TestMu AI. -- [Configure Multiple Projects per repo](https://www.testmuai.com/support/docs/smartui-multiple-projects-per-repo.md): Learn how to manage multiple SmartUI projects in a single repository with ease using this detailed documentation. -- [Configure Timezones for Your Tests](https://www.testmuai.com/support/docs/configure-timezones.md): Set the timezone on test VMs using the timezone capability in LT:Options for accurate locale-based testing on TestMu AI. -- [Configure YAML For Integrating GoCD Pipelines With TestMu AI](https://www.testmuai.com/support/docs/configure-yaml-for-integrating-gocd-pipelines-with-selenium-grid-online.md): This topic will teach you how you can configure your own YAML through GoCD and then integrate your pipeline with an online Selenium Grid. -- [Configuring App Center on TestMu AI](https://www.testmuai.com/support/docs/app-center.md): This article will guide you on how to configure App Center on TestMu AI platform so you can run tests directly without uploading apps manually. -- [Configuring Cypress Test Execution](https://www.testmuai.com/support/docs/run-settings.md): Explore TestMu AI's comprehensive guide on configuring run settings for seamless cross-browser testing. Optimize your testing workflow today. -- [Connect to a Session - TestMu AI Browser Cloud](https://www.testmuai.com/support/docs/connect-to-session.md): Drive TestMu AI Browser Cloud sessions using Puppeteer, Playwright, or Selenium. -- [Consistent Help (3.2.6)](https://www.testmuai.com/support/docs/accessibility-web-rule-3-2-6-consistent-help.md): Rule-level Accessibility guidance for Consistent Help (WCAG 3.2.6 Level A). -- [Console Assertions](https://www.testmuai.com/support/docs/kane-cli-checkpoint-devtools-console.md): Verify browser console output: errors, warnings, log messages, and JavaScript exceptions captured during test execution. -- [Consolidated Mochawesome Report for Cypress](https://www.testmuai.com/support/docs/cypress-mochaawesome-report.md): Discover how to create MochaAwesome test reports for Cypress framework testing on TestMu AI and how to download these reports from the dashboard -- [Consolidated Playwright HTML Report](https://www.testmuai.com/support/docs/playwright-html-report.md): Learn how to generate html test report for playwright framework testing on TestMu AI and download the reports from the dashboard -- [Construct URLs for your Artifacts](https://www.testmuai.com/support/docs/hyperexecute-artifacts-url.md): Efficiently manage and access your HyperExecute Artifacts with our detailed guide! Learn to navigate, construct URLs, and apply filters for streamlined artifact retrieval and viewing. -- [Contrast (Minimum) (1.4.3)](https://www.testmuai.com/support/docs/accessibility-web-rule-1-4-3-contrast-minimum.md): Rule-level Accessibility guidance for Contrast (Minimum) (1.4.3). -- [Cookies Assertions](https://www.testmuai.com/support/docs/kane-cli-checkpoint-devtools-cookies.md): Verify browser cookies (names, values, and flags such as httpOnly, secure, and sameSite) set during test execution. -- [Copy and Move Support for Test Cases](https://www.testmuai.com/support/docs/copy-and-move-support-for-test-cases.md): Learn how to efficiently copy and move test cases within TestMu AI, streamlining your testing process and enhancing organization across multiple projects. -- [Coverage: Proven vs Owed](https://www.testmuai.com/support/docs/kane-cli-assurance-coverage.md): Measure coverage on two axes with kane-cli cover - what a sealed evidence pack proved in execution, and what the design still owes - plus a ranked gaps worklist with ready-to-paste commands. -- [Create & Manage Test Cases](https://www.testmuai.com/support/docs/manual-test-case-creation.md): Learn efficient manual test case creation with TestMu AI's Test Manager, streamlining testing processes and enhancing test execution. -- [Create and Execute your Test Run for KaneAI Generated Tests on HyperExecute](https://www.testmuai.com/support/docs/kaneai-hyperexecute-test-run-execution.md): Learn how to execute KaneAI-generated test runs directly on HyperExecute with this step-by-step guide for efficient and rapid test execution. -- [Create Modules](https://www.testmuai.com/support/docs/create-modules.md): Guide for creating & editing Modules in Test Manager through Modules page. -- [Create Projects](https://www.testmuai.com/support/docs/create-projects.md): Create projects seamlessly with TestMu AI's Test Manager, streamlining your testing processes and ensuring efficient management of all your test activities. -- [Create Pull Requests from KaneAI Test Cases](https://www.testmuai.com/support/docs/kaneai-create-pr.md): Export KaneAI-generated test code directly to GitHub or GitLab as pull requests. Set up repository integration, configure PR settings, and track PR status from Test Manager. -- [Create SSO Connection With TestMu AI](https://www.testmuai.com/support/docs/testmu-sso-self-serve.md): TestMu AI allows you to integrate an external user directory with your TestMu AI Organization through an IDP so your team could leverage the Single Sign-on setup. This document will help you set up your TestMu AI SSO connection. -- [Creating iOS Applications](https://www.testmuai.com/support/docs/ios-ipa-files-xcui.md): Effortlessly test iOS IPA files with XCUi on TestMu AI. Ensure your apps run smoothly on all iOS devices with comprehensive testing support. -- [Creating your first Dashboard using TestMu AI Analytics](https://www.testmuai.com/support/docs/analytics-create-dashboard.md): Analytics - Overview -- [Credits Usage & Management in TestMu AI](https://www.testmuai.com/support/docs/credits-management.md): Guide for usage of credits for AI features in TestMu AI -- [Cucumber (Java)](https://www.testmuai.com/support/docs/accessibility-cucumber-java-test.md): Run Accessibility Automation with Java Cucumber and Selenium: hooks, step defs, shared driver, and Accessibility reports. -- [Cucumber Report](https://www.testmuai.com/support/docs/cucumber-report.md): Learn how to generate Cucumber Report on TestMu AI and download the reports from the dashboard -- [Custom CSS Injection in SmartUI](https://www.testmuai.com/support/docs/smartui-custom-css.md): Learn how to use SmartUI's customCSS feature to inject test-only CSS styles during snapshots without modifying your application code -- [Custom DNS Mapping for Selenium Tests](https://www.testmuai.com/support/docs/custom-dns-map.md): Define custom IP addresses for specific domains during Selenium tests using the customDnsMap capability. -- [Custom Headers in KaneAI](https://www.testmuai.com/support/docs/kaneai-custom-headers.md): Learn how to configure and use custom HTTP headers in KaneAI for Desktop Web and Mobile Browser test authoring, including support for secrets. -- [Custom Specflow Report](https://www.testmuai.com/support/docs/custom-speclow-report.md): Custom specflow report -- [Custom Viewport Capture in SmartUI Screenshots](https://www.testmuai.com/support/docs/smartui-customscroll.md): Learn how to use SmartUI CustomScroll screenshots to preserve page and element scroll positions for pages with nested scroll containers, PDF viewers, data grids, modals, and virtualized lists. -- [Custom Widgets in TestMu AI Analytics Dashboards](https://www.testmuai.com/support/docs/dashboards-custom-widgets.md): A powerful tool to visualize your test data and gain insights into your testing process. -- [Cypress Integration with HyperExecute](https://www.testmuai.com/support/docs/cypress-integration-with-hyperexecute.md): You can now integrate Cypress with HyperExecute. To do so, follow the steps listed in the document. -- [Daily Usage Limit Per Organization | HyperExecute](https://www.testmuai.com/support/docs/hyperexecute-shared-cloud-usage-limit.md): TestMu AI enforces a daily test execution time limit at the organization level, based on the number of concurrent sessions, to ensure fair usage and system stability. -- [Daily Usage Limit Per Organization | TestMu AI](https://www.testmuai.com/support/docs/shared-cloud-usage.md): To ensure fair usage and best practices, TestMu AI enforces a daily capacity limit on the total test time for each organization. -- [Dark Mode](https://www.testmuai.com/support/docs/dark-mode-on-browser.md): Toggle and test dark mode themes during manual testing sessions on real browsers. -- [Dark Mode on Real Devices](https://www.testmuai.com/support/docs/dark-mode-on-real-devices.md): Toggle and test dark mode themes during manual testing sessions on real iOS and Android devices. -- [Data Retention Policy](https://www.testmuai.com/support/docs/data-retention-policy.md): Understand how long your test data, artifacts, reports, and other assets are retained across all TestMu AI products. -- [Database Connection in KaneAI](https://www.testmuai.com/support/docs/kaneai-database.md): Guide for usage of Database connection in KaneAI -- [Datadog Integration](https://www.testmuai.com/support/docs/datadog-integration.md): With TestMu AI and Datadog, you can log bugs with a single click while performing cross-browser testing. Capture screenshots, highlight bugs, and send them directly from the TestMu AI platform to Datadog. -- [Debug your CLI Errors](https://www.testmuai.com/support/docs/hyperexecute-cli-error.md): This document will help you debug your failed job if you are facing any CLI issues. -- [Debug your Scenario Steps Errors](https://www.testmuai.com/support/docs/hyperexecute-scenario-steps-error.md): This document will help you debug your failed job if you are facing any errors in the Scenario stage. -- [Debugging & Observability - TestMu AI Browser Cloud](https://www.testmuai.com/support/docs/browser-cloud-debugging.md): Watch, replay, and debug your agent's browser sessions with video recordings, console logs, and network capture. -- [Debugging Options for Automation Testing on TestMu AI](https://www.testmuai.com/support/docs/debugging-options.md): Use logging options like command, visual, video, network, and console logs to debug failed automation tests. -- [Dedicated Proxy - IP Whitelisting](https://www.testmuai.com/support/docs/dedicated-proxy.md): This article will guide you on how does TestMu AI Cloud access network restricted resources using a dedicated proxy server. -- [Deep Dive into HyperExecute YAML](https://www.testmuai.com/support/docs/deep-dive-into-hyperexecute-yaml.md): This document delivers detailed elucidations for each and every YAML flags, offering an in-depth understanding of each configuration parameter -- [Deeplinks Support for Mobile App Authoring](https://www.testmuai.com/support/docs/kane-ai-deeplink-support.md): Learn how to use deeplinks in KaneAI to directly navigate to specific screens in your Android and iOS mobile applications during test authoring. -- [Default List Of Browsers For Screenshot Testing](https://www.testmuai.com/support/docs/default-list-of-mobile-and-desktop-browsers-for-automated-screenshot-testing.md): The default list of Mobile and Desktop browsers for Automated Screenshot Testing on TestMu AI platform. -- [Defect Analysis and Prediction](https://www.testmuai.com/support/docs/defect-analysis-prediction.md): Leverage AI-powered defect analysis and prediction capabilities on TestMu AI to detect flaky tests, classify failures, identify root causes, and proactively improve test suite reliability. -- [Deleting Your TestMu AI Account](https://www.testmuai.com/support/docs/delete-testmu-account.md): This documentation will help in deleting the TestMu AI account for freemium users, premium users, and team members. -- [Designing Tests from Use-Cases](https://www.testmuai.com/support/docs/kane-cli-assurance-design.md): Turn one committed use-case into acceptance criteria, scenarios, and exactly one runnable test per scenario with kane-cli design tests - reviewed, cited, and permanently linked to requirements via @verifies tags. -- [Desired Capabilities](https://www.testmuai.com/support/docs/desired-capabilities-in-appium.md): This document will teach you how to leverage TestMu AI Capability Generator to easily for declaring desired capabilities in Appium to run your App Automation leveraging Appium Desktop. -- [Desired Capabilities in Selenium](https://www.testmuai.com/support/docs/selenium-automation-capabilities.md): The complete desired capabilities reference for Selenium on the TestMu AI cloud grid, covering default, Selenium 3, Selenium 4, and advanced capabilities. -- [Detailed Command Logs for Cypress](https://www.testmuai.com/support/docs/cypress-detailed-command-logs.md): Learn how to generate detailed command logs for the test reports of cypress framework testing on TestMu AI and download the reports from the dashboard. -- [Detox Testing on HyperExecute](https://www.testmuai.com/support/docs/hyperexecute-detox-testing.md): Learn how to run Detox automation tests on HyperExecute for React Native apps -- [Device Passcode in App Automation on Real Devices](https://www.testmuai.com/support/docs/device-passcode-appautomation.md): Run automated tests on iOS apps that require a device passcode during execution on real devices. -- [Device Passcode on Real Devices](https://www.testmuai.com/support/docs/device-passcode-on-realdevice.md): Enable and test iOS app flows that require a device passcode during manual sessions on real iPhones and iPads. -- [Device Reservation on Private Cloud Devices](https://www.testmuai.com/support/docs/Device-reservation.md): Learn how to reserve private cloud devices in TestMu AI to ensure uninterrupted testing, fair access, and conflict-free usage across your team. -- [DevRev Integration](https://www.testmuai.com/support/docs/devrev-integration.md): Integrate TestMu AI with DevRev for efficient one-click bug logging of your web applications and eliminate the painpoint of maintaining bugs and tasks manually. -- [DevTools Assertions](https://www.testmuai.com/support/docs/kane-cli-checkpoint-devtools.md): Verify data that is not visible on the page: HTTP network traffic, console output, performance metrics, cookies, localStorage, and clipboard. -- [DevTools On Real Devices](https://www.testmuai.com/support/docs/devtools-on-real-devices-app.md): Inspect and debug embedded web views in real mobile devices using TestMu AI's DevTools. -- [Diff Highlighter & Navigation](https://www.testmuai.com/support/docs/smartui-diff-highlighter.md): Quickly discover and review visual differences with SmartUI's diff navigation and highlight. Use Previous, Next, and Highlight to find every change without manual scanning. -- [Disable Screenshot Block for Android](https://www.testmuai.com/support/docs/disable-screenshot-block.md): Discover how to disable screenshot blocking on TestMu AI Real Device Cloud Platform. Test biometric authentication functionalities without limitations on 3000+ real mobile devices. -- [Download and Process Job Artifacts in Global Post](https://www.testmuai.com/support/docs/hyperexecute-global-post-artifacts.md): Download every task's artifacts onto a VM after your HyperExecute job finishes, run custom shell commands to merge or transform them, upload the results to the dashboard, and optionally email them out. -- [Download Files Using TestMu AI Selenium Grid](https://www.testmuai.com/support/docs/download-files-using-testmu-selenium-grid.md): Download files inside test VMs using base64 encoding with Java, Python, Node.js, C#, and Ruby scripts. -- [Downloading Artefacts For A Cypress Project](https://www.testmuai.com/support/docs/download-artefacts-cypress.md): This article guides you on how to download the artefacts for a cypress project from TestMu AI Cloud machine. -- [Drag and Drop in KaneAI](https://www.testmuai.com/support/docs/kane-ai-drag-drop.md): Author drag and drop interactions in KaneAI on Desktop Web, Android apps, iOS apps, and Mobile Web using natural language or manual recording. Covers replay behavior, supported scenarios, and limitations. -- [Dragging Movements (2.5.7)](https://www.testmuai.com/support/docs/accessibility-web-rule-2-5-7-dragging-movements.md): Rule-level Accessibility guidance for Dragging Movements (WCAG 2.5.7 Level AA). -- [Duplicate Accessibility Label](https://www.testmuai.com/support/docs/accessibility-ios-rule-duplicate-accessibility-label.md): Rule-level Accessibility guidance for Duplicate Accessibility Label on iOS. -- [Dynamic Content, Waits & Page State](https://www.testmuai.com/support/docs/kaneai-kb-dynamic-content-waits-and-page-state.md): Handle loading spinners, async content, toast notifications, modals, infinite scroll, and timing-sensitive scenarios in KaneAI -- [Dynamic Type Support](https://www.testmuai.com/support/docs/accessibility-ios-rule-dynamic-type-support.md): Rule-level Accessibility guidance for Dynamic Type Support on iOS. -- [Dynamic URL Replacement](https://www.testmuai.com/support/docs/kaneai-dynamic-url-replacement.md): This documentation will help you to understand how to dynamically update your URL during the KaneAI testing -- [Easy Debug With TestMu AI Developer Tools](https://www.testmuai.com/support/docs/developer-tools.md): Now you can easily debug with developer tools while testing in mobile and desktop browsers at TestMu AI platform. -- [Edit an Accessibility Scan](https://www.testmuai.com/support/docs/accessibility-test-scheduling-edit.md): Easily edit accessibility scan settings in TestMu AI. Update scan names, recurrence schedules, and timings to keep your tests organized and efficient. -- [Edit and Update Test Steps Using KaneAI](https://www.testmuai.com/support/docs/kaneai-edit-test-steps.md): Learn how to KaneAI how to efficiently update and edit test steps using automation. -- [Enterprise FAQs](https://www.testmuai.com/support/docs/hyperexecute-enterprise-faqs.md): Get answers to all your queries about HyperExecute's enterprise-level features, including SSO, dedicated support, custom setups, and more, enhancing your testing experience. -- [Enterprise Readiness](https://www.testmuai.com/support/docs/enterprise-ready.md): A comprehensive overview of how TestMu AI meets enterprise requirements across security, compliance, access control, integrations, deployment, and more. -- [Environment Variables Support for Cypress Tests](https://www.testmuai.com/support/docs/cyp-environment.md): Learn how to specify Environment variables. -- [Environment Variables Supported By TestMu AI Tunnel](https://www.testmuai.com/support/docs/environment-variables-supported-by-lambda-tunnel.md): Now you can access the TestMu AI tunnel through a proxy tunnel and leverage the environment variables to define your proxy settings. If these proxy environment variables are set then the tunnel will automatically detect them. -- [Error Categorization Report](https://www.testmuai.com/support/docs/error-categorization-report.md): Streamline error analysis with the Error Categorization Report. Identify patterns in test failures to enhance quality and efficiency in your testing process. -- [Error Codes Reference](https://www.testmuai.com/support/docs/kane-cli-error-codes.md): Complete reference of Kane CLI result codes: what each code means, why it occurred, and what action to take. Designed for both human and AI agent consumption. -- [Error Handling in Authoring](https://www.testmuai.com/support/docs/error-handling-kaneai.md): Use TestMu AI Accessibility DevTools to automatically find and report accessibility issues as per WCAG guidelines. -- [Error Identification (3.3.1)](https://www.testmuai.com/support/docs/accessibility-web-rule-3-3-1-error-identification.md): Rule-level Accessibility guidance for Error Identification (WCAG 3.3.1 Level A). -- [Error Insights](https://www.testmuai.com/support/docs/analytics-modules-error-insights.md): Analytics Modules - Error Insights -- [Error Messages During Test Execution](https://www.testmuai.com/support/docs/error-messages.md): Identify and resolve common error messages that occur during Selenium test execution on the cloud grid. -- [Espresso Automation using Emulators and Simulators on TestMu AI](https://www.testmuai.com/support/docs/espresso-automation-on-emulators-simulators.md): Learn how to run espresso app automated tests on using Emulators and Simulators on TestMu AI. -- [Espresso JUnit XML Reports](https://www.testmuai.com/support/docs/espresso-junit-report.md): Espresso Junit Report -- [Espresso Supported Capabilities](https://www.testmuai.com/support/docs/espresso-supported-capabilities.md): Now you can run your Espresso framework on TestMu AI online grid of 3000+ real desktop browsers and real operating systems with its supported capabilities. -- [Espresso Testing On HyperExecute](https://www.testmuai.com/support/docs/hyperexecute-espresso-testing.md): Follow the instructions in this documentation, so that you can seamlessly execute Espresso tests on HyperExecute via TestMu AI. -- [Espresso Testing On TestMu AI](https://www.testmuai.com/support/docs/getting-started-with-espresso-testing.md): Now you can run your automation scripts using Espresso framework on TestMu AI online grid of 3000+ real desktop browsers and real operating systems. -- [Execute BiDi Framework Tests on HyperExecute](https://www.testmuai.com/support/docs/hyperexecute-bidi-testing.md): Follow this guide to run Selenium BiDi protocol tests seamlessly on HyperExecute and capture rich browser events with TestMu AI. -- [Execute Cypress Tests Including Private Dependencies](https://www.testmuai.com/support/docs/private-dependencies-cypress.md): This article guides you on how to execute your Cypress tests including the private dependencies on TestMu AI platform. -- [Execute k6 Tests on HyperExecute](https://www.testmuai.com/support/docs/hyperexecute-k6-testing.md): Prevent app launch disaster! Test app performance under load with HyperExecute and JMeter. Get real-time insights, identify weaknesses & ensure smooth launch. -- [Execute Maestro Framework Tests on HyperExecute](https://www.testmuai.com/support/docs/hyperexecute-maestro-testing.md): Follow the instructions in this documentation, so that you can seamlessly execute Maestro tests on HyperExecute via TestMu AI. -- [Execute XCUI Framework Tests on HyperExecute](https://www.testmuai.com/support/docs/hyperexecute-xcui-testing.md): Follow the instructions in this documentation, so that you can seamlessly execute XCUI tests on HyperExecute via TestMu AI. -- [Explore the Automation Dashboard](https://www.testmuai.com/support/docs/inside-testmu-platform.md): Navigate the TestMu AI Automation Dashboard. View test results, video recordings, command logs, network logs, and build timelines. -- [Explore Visual Regression tests with SmartUI](https://www.testmuai.com/support/docs/smart-visual-regression-testing.md): Experience Visual Regression Testing at its finest with SmartUI by TestMu AI. Automate UI changes tracking for precision, collaboration, and scalability on the cloud. Optimize your testing journey now! -- [Export SmartUI Build Data](https://www.testmuai.com/support/docs/smartui-export-build-data.md): Export a SmartUI build from the dashboard in PDF, CSV, or JSON using the build menu, after the UI finishes loading export data. -- [Export Test Cases](https://www.testmuai.com/support/docs/export-test-cases.md): Export multiple test cases with the export to csv option for easy download, tracking, and sharing. -- [Exporting & Sharing Reports](https://www.testmuai.com/support/docs/accessibility-exporting-sharing-reports.md): Export, share, and distribute Accessibility results for triage, audit, and remediation workflows. -- [Extended Debugging Options for Selenium Testing](https://www.testmuai.com/support/docs/extended-debugging-options.md): Intercept network requests, throttle CPU and network, and download HAR files for Selenium test debugging. -- [Extensions - TestMu AI Browser Cloud](https://www.testmuai.com/support/docs/browser-cloud-extensions.md): Load Chrome extensions into TestMu AI Browser Cloud sessions. -- [Extent Report](https://www.testmuai.com/support/docs/extent-report.md): Learn how to generate Extent Report on TestMu AI and download the reports from the dashboard -- [FailFast](https://www.testmuai.com/support/docs/hyperexecute-failfast.md): Learn more about how to use HyperExecute's FailFast feature. -- [Failure Categorization AI - Test Intelligence](https://www.testmuai.com/support/docs/analytics-test-failure-classification.md): TestMu AI Analytics uses Failure Categorization AI to classify test failures into different categories. This helps in identifying the root cause of failures and improving test automation efficiency. -- [Failure Conditions](https://www.testmuai.com/support/docs/kaneai-failure-conditions.md): Failure Conditions for test steps to decide on the behavior of test execution & status on failure of individual steps -- [FAQs](https://www.testmuai.com/support/docs/hyperexecute-general-faqs.md): Explore HyperExecute General FAQs Testing Frameworks, Configurations, and More! Get answers to your questions about HyperExecute's features and capabilities. -- [Features FAQs](https://www.testmuai.com/support/docs/hyperexecute-feature-faqs.md): Explore HyperExecute Feature FAQs | Get comprehensive answers and insights to enhance your understanding of TestMu AI's HyperExecute feature. Maximize efficiency and effectiveness in your testing processes. -- [Fetching Results through SmartUI SDK](https://www.testmuai.com/support/docs/smartui-sdk-fetch-results.md): In this documentation, learn how to fetch live results for SmartUI tests -- [Files - TestMu AI Browser Cloud](https://www.testmuai.com/support/docs/browser-cloud-files.md): Upload, download, and manage files within cloud browser sessions in TestMu AI Browser Cloud. -- [Files and Media](https://www.testmuai.com/support/docs/upload-files-media.md): Learn how to upload and download files and media on real Android and iOS devices during app and browser testing on TestMu AI. -- [Filter by Custom Data](https://www.testmuai.com/support/docs/analytics-filter-by-custom-data.md): Learn how to filter analytics data by custom attributes on TestMu AI to track and analyze your testing metrics more effectively. -- [Filters for Espresso Tests](https://www.testmuai.com/support/docs/speedup-espresso.md): This document helps you learn how to speed up your Espresso Tests. -- [Filters for XCUI Tests](https://www.testmuai.com/support/docs/speedup-xcui.md): Filter XCUI test cases with TestMu AI's REST API. Run specific tests or skip others using customizable parameters. Streamline your testing process effortlessly. -- [Finding & Interacting with Elements](https://www.testmuai.com/support/docs/kaneai-kb-finding-and-interacting-with-elements.md): Learn how to reliably target and interact with web and mobile elements using KaneAI's natural language instructions -- [Fire TV With Appium](https://www.testmuai.com/support/docs/appium-firetv.md): Complete guide to running your first FireTV Appium automated test script on TestMu AI Real Device Cloud Platform. Test on 3000+ Real Devices. -- [Fixed Orientation Lock](https://www.testmuai.com/support/docs/accessibility-android-rule-fixed-orientation-lock.md): Rule-level Accessibility guidance for Fixed Orientation Lock on Android. -- [Flaky Test Analytics with Test Intelligence](https://www.testmuai.com/support/docs/analytics-modules-test-intelligence-flaky-test-analytics.md): Analyze your flaky tests with Test Intelligence and get insights on the flaky tests that are causing your test runs to fail. -- [Flock Integration](https://www.testmuai.com/support/docs/flock-integration.md): TestMu AI's integration with Flock lets you log bugs directly into Flock team channels with just one click while running cross browser testing of websites across 3000+ browser and OS combinations. -- [Flutter Dart Testing On TestMu AI - Android](https://www.testmuai.com/support/docs/getting-started-with-flutter-dart-android-automation.md): This guide shows you how to run your first Flutter Dart test on 5000+ real Android devices using TestMu AI's Real Device Cloud. -- [Flutter Dart Testing On TestMu AI - iOS](https://www.testmuai.com/support/docs/getting-started-with-flutter-dart-ios-automation.md): This guide details running your first Flutter Dart test on real iPhones using TestMu AI's Real Device Cloud. -- [Flutter Supported Capabilities](https://www.testmuai.com/support/docs/flutter-supported-capabilities.md): Now you can run your Flutter framework on TestMu AI online grid of 3000+ real desktop browsers and real operating systems with its supported capabilities. -- [Focus Not Obscured (Minimum) (2.4.11)](https://www.testmuai.com/support/docs/accessibility-web-rule-2-4-11-focus-not-obscured.md): Rule-level Accessibility guidance for Focus Not Obscured (WCAG 2.4.11 Level AA). -- [Focus Order (2.4.3)](https://www.testmuai.com/support/docs/accessibility-web-rule-2-4-3-focus-order.md): Rule-level Accessibility guidance for Focus Order (WCAG 2.4.3 Level A). -- [Focus Visible (2.4.7)](https://www.testmuai.com/support/docs/accessibility-web-rule-2-4-7-focus-visible.md): Rule-level Accessibility guidance for Focus Visible (2.4.7). -- [FogBugz Integration](https://www.testmuai.com/support/docs/fogbugz-integration.md): TestMu AI helps you manage your bugs in FogBugz by moving them to project in a single click. All the details you provide in TestMu AI like task list, assignee, title and description would automatically be presented in the project in your FogBugz dashboard. -- [Forms, Inputs & Data Entry](https://www.testmuai.com/support/docs/kaneai-kb-forms-inputs-and-data-entry.md): Practical guide to handling forms, dropdowns, date pickers, file uploads, and complex input patterns in KaneAI -- [Fragment Identifier for Accessibility Testing](https://www.testmuai.com/support/docs/accessibility-fragment-identifier.md): Treat URLs with different hash fragments as separate entities for accessibility issue tracking, triage, and reporting when you need section-level or SPA state granularity. -- [Frequently Asked Questions](https://www.testmuai.com/support/docs/selenium-faq.md): Find answers to common questions about running Selenium tests on the cloud grid. -- [Full Page Scanner](https://www.testmuai.com/support/docs/accessibility-testing-full-page-scanner.md): Full Page Scanner scans multiple elements like HTML/CSS code, images, forms, and multimedia elements, ensuring WCAG compliance and automated issue detection. -- [Gatling Automation on HyperExecute](https://www.testmuai.com/support/docs/hyperexecute-gattling-testing.md): Learn how to run Selenium automation tests on HyperExecute using the Gatling framework -- [Generate Lighthouse Reports With Playwright Lighthouse Library](https://www.testmuai.com/support/docs/playwright-lighthouse-library.md): Learn how to generate multiple Lighthouse reports using Playwright Lighthouse Library. -- [Generate Test Cases with AI](https://www.testmuai.com/support/docs/generate-test-cases-with-ai.md): Learn efficient test case generation with in TestMu AI's Test Manager using AI. -- [Generate your HyperExecute YAML file](https://www.testmuai.com/support/docs/hyperexecute-generate-sample-yaml.md): This document helps you to understand how to generate the HyperExecute YAML files using the HyperExecute dashboard and then a sample yaml file for understanding the YAML -- [Generating Test Cases with AI](https://www.testmuai.com/support/docs/kane-cli-generate.md): Turn a plain-language description of what you want to test into structured test scenarios and test cases with kane-cli generate: refine in plain language, then save the functional cases as runnable _test.md files. -- [GenRocket Integration](https://www.testmuai.com/support/docs/genrocket-integration.md): Integrate TestMu AI with GenRocket to perform desktop web, mobile web and native app testing across 3000+ real browsers, devices, and operating systems. -- [Gesture Supported for iOS Devices](https://www.testmuai.com/support/docs/gesture-support-ios.md): Learn about various kind of gestures which we support for iOS devices on our real devices, and how to use the following. -- [Get Started With Selenium Testing](https://www.testmuai.com/support/docs/getting-started-with-testmu-automation.md): Run Selenium tests on TestMu AI's cloud grid. 3000+ browser and OS combinations, parallel execution, video recordings, and built-in debugging. -- [Getting Started (Quick Setup)](https://www.testmuai.com/support/docs/accessibility-getting-started-quick-setup.md): This page has moved. Use Getting Started with Accessibility Testing for quick setup and first-run guidance. -- [Getting Started with Accessibility Testing](https://www.testmuai.com/support/docs/accessibility-testing.md): Overview of TestMu AI Accessibility Testing across web, mobile apps, reports, checklists, and integrated product surfaces. Includes quick setup to get your first report. -- [Getting Started with Appium Testing](https://www.testmuai.com/support/docs/getting-started-with-appium-testing.md): Getting Started with Appium Testing on TestMu AI Upload, Script, Execute, View - Run your automation scripts seamlessly on 3000+ real browsers and operating systems. -- [Getting Started with Automation MCP Tool](https://www.testmuai.com/support/docs/automation-mcp-server.md): Triage test failures faster with the Automation MCP Tool by connecting AI assistants to your test execution data. -- [Getting Started With Cypress Testing On TestMu AI Platform](https://www.testmuai.com/support/docs/getting-started-with-cypress-testing.md): In this documentation, learn how to perform end-to-end Cypress testing on the TestMu AI Automation Cloud across 40+ browser versions. -- [Getting started with Desktop Web Browser Real Time Testing](https://www.testmuai.com/support/docs/getting-started-with-desktop-browser-real-time-testing.md): Test your websites on real desktop browsers with TestMu AI's Real-Time Desktop Web Browser Testing. Easily launch virtual machines and test across different OS, browsers, and versions. -- [Getting Started with HyperExecute](https://www.testmuai.com/support/docs/getting-started-with-hyperexecute.md): Get started with HyperExecute on TestMu AI for faster web automation tests using Matrix and Auto-Split strategies. Test with Selenium languages at high speeds. -- [Getting Started with Kane CLI](https://www.testmuai.com/support/docs/kane-cli-getting-started.md): Install Kane CLI, authenticate, and run your first browser test. -- [Getting started with Mobile App Real Time Testing](https://www.testmuai.com/support/docs/getting-started-with-mobile-app-real-time-testing.md): Test your mobile apps on real devices in real-time with TestMu AI. Easily launch virtual machines and test across various OS and versions. -- [Getting started with Mobile Web Browser Real Time Testing](https://www.testmuai.com/support/docs/getting-started-with-mobile-browser-real-time-testing.md): Test your websites on real mobile browsers with TestMu AI's Real-Time Mobile Web Browser Testing. Easily launch virtual machines and test across various OS, browsers, and versions. -- [Getting started with Real Device App Testing](https://www.testmuai.com/support/docs/app-testing-on-real-devices.md): Test mobile apps on 3000+ real devices with TestMu AI. Upload, choose devices, regions, and OS versions, and start seamless manual app testing today!! -- [Getting started with Real Device App Testing](https://www.testmuai.com/support/docs/real-device-app-testing.md): With TestMu AI, perform live interactive testing of mobile applications on Android emulators and iOS simulators and ensure your apps work seamlessly across multiple versions of Android emulators and iOS simulators. -- [Getting started with Real Device Browser Testing](https://www.testmuai.com/support/docs/browser-testing-on-real-devices.md): With TestMu AI, conduct live interactive testing of websites and web apps on real Android and iOS devices to ensure a seamless user experience. -- [Getting Started With Selenium 4 On TestMu AI Platform](https://www.testmuai.com/support/docs/getting-started-with-selenium-4.md): Now run automated tests with Selenium 4 using the TestMu AI Selenium Grid across 3000+ browsers and operating systems. -- [Getting Started With Selenium Debugging Using Telemetry Logs](https://www.testmuai.com/support/docs/selenium-telemetry-logs.md): Debug Selenium tests using telemetry logs with OpenTelemetry and Jaeger on TestMu AI. -- [Getting Started With Single Sign On (SSO)](https://www.testmuai.com/support/docs/single-sign-on.md): Single sign-on (SSO) is an authentication feature that allows users to authenticate and access to TestMu AI through an identity provider (IdP). -- [Getting started with SmartUI using Cypress On TestMu AI](https://www.testmuai.com/support/docs/smart-ui-cypress.md): Master Visual UI Testing with Cypress on TestMu AI Automation Cloud across 40+ browser versions through this comprehensive documentation. -- [Getting started with SmartUI using K6](https://www.testmuai.com/support/docs/smartui-k6-setup.md): In this documentation, learn how to perform Visual Regression testing with K6 on the TestMu AI Automation Cloud across 40+ browser versions. -- [Getting started with SmartUI using Playwright](https://www.testmuai.com/support/docs/playwright-visual-regression.md): Master Playwright Visual Regression with our step-by-step guide. Seamlessly integrate SmartUI, enhance your testing skills, and unlock a world of possibilities. -- [Getting started with SmartUI using Puppeteer](https://www.testmuai.com/support/docs/puppeteer-visual-regression.md): Learn Puppeteer Visual Regression Set up SmartUI, run tests on TestMu AI Cloud across 40+ browsers. Step-by-step guide for seamless testing. -- [Getting started with SmartUI using Storybook On TestMu AI](https://www.testmuai.com/support/docs/smart-ui-storybook.md): In this documentation, learn how to perform Visual UI Testing with Storybook on the TestMu AI Automation Cloud across 40+ browser versions. -- [Getting Started with TestMu AI Web Scanner](https://www.testmuai.com/support/docs/web-scanner-getting-started.md): A guide to accessing the Web Scanner dashboard and using the 4-step wizard to create and schedule your first Visual or Accessibility scan. -- [Getting started with TestMu AI-Tunnel](https://www.testmuai.com/support/docs/testmu-tunnel.md): TestMu AI Tunnel enables secure testing of locally hosted web apps on cloud-based real machines, providing access to various browsers and devices for seamless testing. -- [Getting started with TestMu AI's SmartUI Figma CLI](https://www.testmuai.com/support/docs/smartui-cli-figma.md): In this documentation, learn how to perform Visual Testing of your Figma files using command line interface using SmartUI CLI. -- [Getting Started with TestMu AI's SmartUI Figma-App CLI](https://www.testmuai.com/support/docs/smartui-cli-figma-app.md): Learn how to compare your Figma mobile designs with native app screenshots on real devices using SmartUI CLI. -- [Getting started with TestMu AI's SmartUI Figma-Web CLI](https://www.testmuai.com/support/docs/smartui-cli-figma-web.md): In this documentation, learn how to compare your figma designs to live production websites and web applications. -- [Getting Started With the Agent Testing Platform](https://www.testmuai.com/support/docs/getting-started-with-agent-testing-platform.md): Get started with the Agent Testing Platform to validate chat, voice, phone, and image AI agents against standardized quality metrics before you ship. -- [Getting Started With Uploading Screenshots through API for Visual Regression Testing](https://www.testmuai.com/support/docs/smartui-upload-api-v2.md): Discover how to leverage TestMu AI's SmartUI upload API v2 for seamless Visual Regression Testing. Learn the prerequisites, create projects. -- [Getting Started With Visual Regression Testing Using Appium On SmartUI Real Devices (NodeJS)](https://www.testmuai.com/support/docs/smartui-appium-hooks.md): Explore our Appium Visual Regression support documentation for step-by-step guidance! Conduct visual testing, manage apps, and ensure your mobile apps are flawless before launch. -- [Getting Started With Visual Regression Testing Using Espresso On SmartUI Real Devices](https://www.testmuai.com/support/docs/espresso-visual-regression.md): Dive into our detailed Appium Visual Regression support documentation for step-by-step guidance! Efficiently perform visual testing, manage applications, and ensure your mobile apps are visually perfect before launch. -- [Getting Started With Visual Regression Testing Using Selenium On SmartUI](https://www.testmuai.com/support/docs/selenium-visual-regression.md): Master Selenium Visual Regression testing with TestMu AI. Detect visual deviations early in your web applications to ensure a seamless user experience. -- [Getting Started With Visual Regression Testing Using XCUI On SmartUI Real Devices](https://www.testmuai.com/support/docs/xcui-visual-regression.md): Dive into our detailed XCUI Visual Regression support documentation for step-by-step guidance! Efficiently perform visual testing, manage applications, and ensure your mobile apps are visually perfect before launch. -- [Getting The JavaScript Error Report Of Your Website](https://www.testmuai.com/support/docs/error-reports.md): With LT Browser you can monitor web-page errors and services that fail to load. LT Browser alerts you when the issue occurs and generates an error report. -- [Ghost Inspector Integration With TestMu AI's Cloud](https://www.testmuai.com/support/docs/ghost-inspector-integration-with-testmu.md): TestMu AI now integrates with Ghost Inspector to boost your go-to market delivery. Perform automated cross browser testing with TestMu AI to ensure your development code renders seamlessly through an online Selenium grid providing 3000+ real browsers running through machines. -- [Git Branching Strategy in SmartUI Storybook Integration](https://www.testmuai.com/support/docs/smartui-git-branching-strategy.md): In this documentation, learn how the Git Branching Strategy is implemented in Visual Regression testing on the TestMu AI Automation Cloud across 40+ browser versions. -- [GitHub Actions Integration](https://www.testmuai.com/support/docs/github-actions-with-hyperexecute.md): TestMu AI now integrates with GitHub Actions Pipeline to boost your go-to market delivery. Perform automated cross browser testing with TestMu AI to ensure your development code renders seamlessly through an online Selenium grid providing 3000+ real browsers running through machines. -- [GitHub Actions Pipeline Integration with SmartUI](https://www.testmuai.com/support/docs/smartui-with-github-actions.md): SmartUI now integrates with GitHub Actions Pipeline to boost your go-to market delivery. Perform automated cross browser testing with SmartUI to ensure your development code renders seamlessly providing 3000+ real browsers running through machines. -- [GitHub App Integration Support for HyperExecute](https://www.testmuai.com/support/docs/hyperexecute-github-app-integration.md): GitHub App Integration Support for HyperExecute -- [GitHub App Integration with SmartUI](https://www.testmuai.com/support/docs/smartui-github-app-integration.md): Integrate GitHub with TestMu AI SmartUI for seamless CI and visual regression testing. Follow our guide to set up, configure, and run your first test suite. -- [GitHub CI Integration](https://www.testmuai.com/support/docs/github-with-hyperexecute.md): TestMu AI now integrates with GitLab CI to boost your go-to market delivery. Perform automated cross browser testing with TestMu AI to ensure your development code renders seamlessly through an online Selenium grid providing 3000+ real browsers running through machines. -- [GitLab CI Integration](https://www.testmuai.com/support/docs/gitlab-ci-integration-with-testmu.md): Integrate GitLab CI seamlessly with TestMu AI for efficient and automated testing, supported by step-by-step guides and detailed support doc. -- [GitLab Integration:HyperExecute](https://www.testmuai.com/support/docs/gitlab-integration-with-hyperexecute.md): TestMu AI integrates with GitLab Pipeline to boost your go-to market delivery. Perform automated cross browser testing with TestMu AI to ensure your development code renders seamlessly through an online Selenium grid providing 3000+ real browsers running through machines. -- [GitLab Integration:TestMu AI](https://www.testmuai.com/support/docs/gitlab-integration.md): TestMu AI provides integrations with GitLab to help you log bugs directly from the middle of your test session on TestMu AI to your GitLab repository. -- [GitLab Pipeline Integration with SmartUI](https://www.testmuai.com/support/docs/smartui-with-gitlab.md): SmartUI now integrates with GitLab Pipeline to boost your go-to market delivery. Perform automated cross browser testing with SmartUI to ensure your development code renders seamlessly providing 3000+ real browsers running through machines. -- [GitLab PR Checks with SmartUI Exec (SDK)](https://www.testmuai.com/support/docs/smartui-gitlab-pr-checks-exec.md): Integrate SmartUI visual regression testing with GitLab merge requests using SmartUI SDK/Exec method for web and mobile testing with Selenium, Playwright, Appium, and more. -- [GitLab PR Checks with SmartUI Hooks](https://www.testmuai.com/support/docs/smartui-gitlab-pr-checks-hooks.md): Integrate SmartUI visual regression testing with GitLab merge requests using SmartUI Hooks for web and mobile testing with Selenium, Playwright, WebdriverIO, Appium, and more. -- [GoodDay Integration](https://www.testmuai.com/support/docs/goodday-integration.md): TestMu AI helps you manage your bugs in GoodDay by moving them to project in a single click. All the details you provide in TestMu AI like task list, assignee, title and description would automatically be presented in the project on GoodDay. -- [Google Chat Integration](https://www.testmuai.com/support/docs/google-chat-integration.md): TestMu AI helps you manage your bugs in Google Chat by moving them to project in a single click. All the details you provide in TestMu AI like task list, assignee, title and description would automatically be presented in the project in your Google Chat Space. -- [GPS Location Mocking for Mobile App and Browser Authoring](https://www.testmuai.com/support/docs/kaneai-gps-location.md): Learn how to use GPS Location Override feature in KaneAI for testing location-based functionalities in Mobile App and Mobile Browser sessions -- [Gradle Plugin Integration With TestMu AI](https://www.testmuai.com/support/docs/gradle-integration-with-testmu.md): Automate testing at scale! Integrate TestMu AI with Gradle for seamless CI/CD & lightning-fast execution. -- [Group and Filter Your Test Builds Using Build Tags](https://www.testmuai.com/support/docs/group-and-filter-your-test-builds-using-build-tags.md): Create build tags in desired capabilities to group and filter test builds on the automation dashboard. -- [Group Folder Redirect for iOS Apps in App Live](https://www.testmuai.com/support/docs/group-folder-redirect-on-real-device.md): Enable Group Folder Redirect support for iOS apps on TestMu AI Real Devices. -- [Group Folder Redirect for iOS Apps in Appium Tests](https://www.testmuai.com/support/docs/group-folder-redirects.md): Learn how to enable Group Folder Redirect for iOS apps during Appium testing on TestMu AI Real Device Cloud. -- [Group Management](https://www.testmuai.com/support/docs/group-management.md): This documentation will help you in creating the different groups and allot concurrency to those groups to perform test automation easily. -- [Grouping Screenshots through Build Names](https://www.testmuai.com/support/docs/smartui-cli-build-name.md): In this documentation, learn how to group screenshots in a single build across multiple executions. -- [Grouping SmartUI Screenshots by Test Cases](https://www.testmuai.com/support/docs/smartui-group-by-test-cases.md): SmartUI grouping by test case using the Java Hooks flow (smartUI.project, smartui.takeScreenshot), one build with multiple captures, Automation SmartUI tab, and bulk actions; plus Web-only Node SDK alternatives. -- [Guide - Running Python Tests On TestMu AI Selenium Grid](https://www.testmuai.com/support/docs/run-python-tests-on-selenium-grid.md): Guide to running Python and Selenium test scripts on TestMu AI Selenium automation grid online. Automated cross browser testing online using Selenium and Python on 3000+ browsers on cloud -- [Guide to Run Selenium IDE Tests Using TestMu AI](https://www.testmuai.com/support/docs/run-selenium-ide-tests-on-testmu-selenium-cloud-grid.md): Guide on how to run Selenium IDE tests with TestMu AI online Selenium automation grid, for testing on 3000+ real browsers & browser versions hosted on TestMu AI cloud servers. -- [Guide:Running NodeJS Tests On Cloud Selenium Grid](https://www.testmuai.com/support/docs/quick-guide-to-run-node-js-tests-on-testmu-selenium-grid.md): Guide to running Node.js and Selenium test scripts on TestMu AI Selenium automation grid online. Automated cross browser testing online using Selenium and Node.js on 3000+ browsers on cloud. -- [Guided Walkthrough of HyperExecute](https://www.testmuai.com/support/docs/hyperexecute-guided-walkthrough.md): Learn how to manage Jobs, configure tunnels, upgrade subscriptions, and access detailed test and job information. Discover the power of custom report, analytic. -- [Guided Walkthrough of KaneAI](https://www.testmuai.com/support/docs/kane-ai-guided-walkthrough.md): Explore KaneAI's features and components to efficiently manage automated tests. Learn to navigate projects, edit test cases, and track test history. -- [Guided Walkthrough of SmartUI](https://www.testmuai.com/support/docs/smartui-guided-walkthrough.md): Learn how to manage tests, update the images, compare the images, perform testing using the dashboard. -- [Handle Dynamic Data in Visual Tests](https://www.testmuai.com/support/docs/smartui-handle-dynamic-data.md): Learn how to handle dynamic data in SmartUI visual regression tests by ignoring or selecting specific DOM elements that change between test runs. -- [Handle Pages with Lazy Loading](https://www.testmuai.com/support/docs/smartui-handle-lazy-loading.md): Learn how to handle lazy-loaded content in SmartUI visual regression tests using scrolling techniques and waitForTimeout configuration. -- [Handle Pages with Sticky Elements](https://www.testmuai.com/support/docs/smartui-handle-sticky-elements.md): Ensure accurate visual testing with TestMu AI SmartUI by effectively handling sticky elements. Prevent false positives and maintain consistency across visual snapshots. -- [Handle Pages with Videos](https://www.testmuai.com/support/docs/smartui-handle-videos.md): Learn how SmartUI automatically handles video content in visual regression tests by capturing the first frame, preventing false positives from dynamic video playback. -- [Handling Dynamic Data with DOM Configuration and Options for SmartUI](https://www.testmuai.com/support/docs/html-dom-smartui-options.md): In this documentation, learn how to handle dynamic content of your web pages for visual regression. -- [HAR Log Viewer With Waterfall](https://www.testmuai.com/support/docs/har-log-viewer-with-waterfall.md): Analyze web page performance with the HAR Log Viewer and Waterfall chart for Selenium tests on TestMu AI. -- [Heatmap Widgets](https://www.testmuai.com/support/docs/analytics-heatmap-widgets.md): Learn how to create and configure Heatmap widgets to visualize test data patterns and generate meaningful insights across browsers, resolutions, and other dimensions. -- [Hide and Restore Issues](https://www.testmuai.com/support/docs/accessibility-hide-restore-issues.md): Hide or restore accessibility issues at the URL, rule, element group, or element level, add reasons, and control whether the preference should also apply to future scheduled web scans. -- [Hive Integration](https://www.testmuai.com/support/docs/hive-integration.md): TestMu AI helps you to push bugs directly in Hive project with details like work space, assignee, title and description with a single click. -- [How the TestMu AI Agent Testing Platform Evaluates Agents](https://www.testmuai.com/support/docs/architecture-and-how-evaluation-works.md): See how the Agent Testing Platform works. Its layered architecture and the five-phase evaluation loop that turns uploaded context into a go-live verdict. -- [How to automate apps using App Center on TestMu AI](https://www.testmuai.com/support/docs/app-automation-using-app-center.md): Learn App Test Automation with App Center & how to install apps for testing in this guide. -- [How to build .app file for iOS App testing on Simulators](https://www.testmuai.com/support/docs/troubleshooting-ios-app-testing.md): Facing issues in uploading .app files on Simulators? Here is a quick guide to troubleshoot it. -- [How to Bypass Firewalls Using CustomHeaders Capability](https://www.testmuai.com/support/docs/custom-headers.md): Add custom headers to Selenium tests and bypass firewall restrictions using the customHeaders capability. -- [How to Bypass Proxy Domains](https://www.testmuai.com/support/docs/bypass-proxy-domains.md): Learn how to use TestMu AI bypassProxyDomains capability to specify domains that should bypass the dedicated proxy. -- [How to configure Operating System and Browser in your Test Suite](https://www.testmuai.com/support/docs/hyperexecute-how-to-configure-os-and-browser.md): Discover the correct way to configure the operating system and the browser capabilities in your codebase to execute in the HyperExecute platform. -- [How to configure sourcePayload](https://www.testmuai.com/support/docs/hyperexecute-how-to-configure-sourcePayload.md): Learn to configure sourcePayload in HyperExecute with ease, allowing secure and direct sourcing of test scripts from your Git provider for seamless testing integration. -- [How to Connect to TestMu AI Browser Cloud](https://www.testmuai.com/support/docs/browser-cloud-sdk-setup.md): Connect your agents to TestMu AI Browser Cloud using the TestMu AI Browser SDK. -- [How to Create and Manage Service Accounts](https://www.testmuai.com/support/docs/service-accounts.md): This documentation will help you create and manage service accounts on TestMu AI. -- [How to Create Custom Automation Tags](https://www.testmuai.com/support/docs/group-tests-using-custom-tags.md): Create custom tags in desired capabilities to group, view, and filter automation tests on the dashboard. -- [How to Debug a Failed Job](https://www.testmuai.com/support/docs/hyperexecute-how-to-debug-job.md): This document will help you debug failed tests on HyperExecute by finding error logs in different stages using the HyperExecute Dashboard. -- [How to debug your Espresso tests](https://www.testmuai.com/support/docs/debugging-espresso-tests.md): Learn how to debug Espresso tests for your mobile applications to resolve different kinds of bugs for your failed test builds. -- [How to Find the Correct Concurrency for my Job](https://www.testmuai.com/support/docs/hyperexecute-how-to-find-correct-concurrency.md): Discover the right concurrency levels with HyperExecute using TestMu AI's guide for effective automation testing. -- [How to Generate Multiple Lighthouse Reports in Selenium](https://www.testmuai.com/support/docs/generate-multiple-lighthouse-reports.md): Generate multiple Lighthouse reports in a single Selenium session using Lambda Hooks. -- [How to Get my Username and Access key](https://www.testmuai.com/support/docs/hyperexecute-how-to-get-my-username-and-access-key.md): Find out how to Get my Username and Access key -- [How to Integrate a Chat Agent API](https://www.testmuai.com/support/docs/chat-agent-api-integration.md): Connect a chat agent to TestMu AI over its API. Request format, three connection methods, response schema, and the fields the platform scores. -- [How to integrate AppVeyor CI with TestMu AI](https://www.testmuai.com/support/docs/appveyor-ci-integration.md): TestMu AI now integrates with AppVeyor CI to boost your go-to-market delivery. Perform automated cross-browser testing on 3000+ browser environment combinations. -- [How to integrate AWS CodePipeline with HyperExecute](https://www.testmuai.com/support/docs/aws-codepipeline-with-hyperexecute.md): Optimize AWS CodePipeline with HyperExecute for seamless, efficient testing. Streamline your CI/CD workflow with TestMu AI's comprehensive guide. -- [How to integrate AWS CodePipeline with TestMu AI](https://www.testmuai.com/support/docs/aws-codepipeline-integration-with-testmu.md): Rapidly commit changes in CI CD pipeline and validate them for UI/UX anomalies by integrating your AWS CodePipeline instance with TestMu AI Selenium Grid. -- [How to integrate Bamboo with TestMu AI](https://www.testmuai.com/support/docs/bamboo-integration-with-testmu.md): Deploy your code in a reliable manner at scale using Bamboo integration with TestMu AI, and ensure it looks robust across every browser with Selenium Grid on cloud. -- [How to integrate CircleCI with TestMu AI](https://www.testmuai.com/support/docs/circleci-integration-with-testmu.md): TestMu AI now integrates with CircleCI to boost your go-to market delivery. Perform automated cross browser testing with TestMu AI to ensure your development code renders seamlessly through an online Selenium grid providing 3000+ real browsers running through machines. -- [How to integrate Drone CI with TestMu AI](https://www.testmuai.com/support/docs/drone-ci-integration-with-testmu.md): This document will help you understand how you can integrate your Drone CI pipeline with cloud Selenium Grid for continuous testing. -- [How to integrate GoCD With HyperExecute](https://www.testmuai.com/support/docs/gocd-integration-with-hyperexecute.md): HyperExecute integration with GoCD will help your perform Test automation for your code changes on more than 3000+ real browsers for both mobile and desktop. You can trigger your testing scripts directly from GoCD instance and they would be executed over your HyperExecute Dashboard. -- [How to integrate GoCD With TestMu AI](https://www.testmuai.com/support/docs/gocd-integration-with-testmu.md): TestMu AI integration with GoCD will help your perform Selenium test automation for your code changes on more than 3000+ real browsers for both mobile and desktop. You can trigger your Selenium testing scripts directly from GoCD instance and they would be executed over your TestMu AI automation dashboard. -- [How to integrate qTest With TestMu AI](https://www.testmuai.com/support/docs/qtest-integration.md): Run your tests using qTest Launch on TestMu AI. -- [How to integrate selenified with TestMu AI](https://www.testmuai.com/support/docs/selenified-integration-with-testmu.md): With TestMu AI integration to Selenified, you can perform cross browser testing in parallel on more than 2000 real browsers available at an online Selenium Grid offered by TestMu AI. So you don't have to worry about configuring an in-house testing infrastructure as TestMu AI will take care of that for you. -- [How to integrate SimplyfyQA With TestMu AI](https://www.testmuai.com/support/docs/simplifyqa-integration.md): This guide walk you through how you can run your tests using simplifyqa after integrating on TestMu AI's real device. -- [How to integrate TeamCity with HyperExecute](https://www.testmuai.com/support/docs/teamcity-with-hyperexecute.md): Integrate TeamCity with HyperExecute effortlessly! Speed up your CI/CD pipeline while ensuring UI/UX consistency with TestMu AI's reliable testing platform -- [How to integrate test reporting & test management tools](https://www.testmuai.com/support/docs/integrate-test-reporting-test-management-tools.md): Using the integration with your favourite test reporting and test management tools, you can view and analyze the results of your automated browser testing on 3000+ real browsers and operating systems, executed over TestMu AI. -- [How to integrate TestMo With TestMu AI](https://www.testmuai.com/support/docs/testmo-integration.md): TestMu AI integration to TestMo will help you run your automation scripts from TestMo instance to our cloud-based Grid. -- [How To Integrate TestMu AI With Zebrunner Instance](https://www.testmuai.com/support/docs/zebrunner-integration.md): TestMu AI Selenium Grid integration to Zebrunner will help you run your Selenium automation scripts from Zebrunner instance to our cloud-based Selenium Grid. -- [How to integrate TestRail With TestMu AI Selenium Grid](https://www.testmuai.com/support/docs/testrail-integration-with-testmu-selenium-grid.md): TestMu AI Selenium Grid integration to TestRail will help you run your Selenium automation scripts from TestRail instance to our cloud-based Selenium Grid. -- [How to integrate Travis CI with TestMu AI](https://www.testmuai.com/support/docs/travis-ci-with-testmu.md): TestMu AI now integrates with Travis CI to boost go-to market delivery. Perform automated cross browser testing with TestMu AI to ensure your development code renders seamlessly through an online Selenium grid providing 3000+ real browsers running through machines, on cloud. Perform automation testing in parallel with TestMu AI's Selenium grid to drastically trim down test cycles. -- [How to integrate your Appium tests with Bitrise](https://www.testmuai.com/support/docs/bitrise-integration.md): Integrate TestMu AI with Bitrise CI to get started with Appium app automation across a wide range of real Android and iOS devices. -- [How to integrate your Espresso tests with Bitrise CI](https://www.testmuai.com/support/docs/run-espresso-tests-in-bitrise.md): Integrate TestMu AI with Bitrise CI to get started with Espresso test automation across a wide range of real Android devices. -- [How to List Supported Browser in HyperExecute via API](https://www.testmuai.com/support/docs/hyperexecute-browser-list-api.md): Find out how to list the browser used in hyperexecute using the BrowserList API -- [How To Mark As Bug In Responsive Testing | TestMu AI](https://www.testmuai.com/support/docs/mark-as-bug-in-responsive-testing.md): Perform responsive testing with TestMu AI and mark your UI bugs in just a single click. Share them with your colleagues on respective project management tool such as Jira, Trello, Paymo, and more. -- [How to migrate from BrowserStack to TestMu AI](https://www.testmuai.com/support/docs/browserstack-to-testmu-migration-guide.md): Using this guide, you can seamlessly migrate from BrowserStack to TestMu AI. It showcases a hands-on implementation using which you can run tests on TestMu AI (within few minutes) -- [How to Migrate From Legacy Test Execution Platform to TestMu AI](https://www.testmuai.com/support/docs/migrate-from-legacy-platform-to-testmu.md): Learn how to migrate from legacy test execution platform to TestMu AI cloud platform. -- [How to migrate from Sauce Labs to TestMu AI](https://www.testmuai.com/support/docs/saucelabs-to-testmu-migration-guide.md): Using this guide, you can seamlessly migrate from SauceLabs to TestMu AI. It showcases a hands-on implementation using which you can run tests on TestMu AI (within few minutes) -- [How to Organize Your Selenium Tests and Builds](https://www.testmuai.com/support/docs/filter-your-selenium-tests.md): Filter, tag, and group Selenium tests and builds, split builds, edit test details, and share results from the automation dashboard. -- [How to perform Accessibility testing using Deque's axe](https://www.testmuai.com/support/docs/deque-integration.md): A quick guide to perform Automated Accessibility testing using Deque's axe across 3000+ browsers and OS combinations. -- [How to Perform Automation Testing with Ranorex Using TestMu AI](https://www.testmuai.com/support/docs/ranorex-integration-with-testmu.md): TestMu AI offers integration with Ranorex to help you perform automated cross browser testing of your web app on over 3000+ real browsers & browser versions through an on-cloud Selenium Grid. You could perform tests on your locally hosted files and can execute parallel testing to trim down your test cycles drastically. -- [How to Perform Dependent Test Case Discovery](https://www.testmuai.com/support/docs/hyperexecute-how-to-perform-dependent-test-based-discovery.md): How to Perform Dependent Test Case Discovery -- [How to Perform Group-Based Test Discovery in TestNG](https://www.testmuai.com/support/docs/hyperexecute-how-to-perform-group-based-test-discovery-in-testng.md): Perform group-based test discovery in TestNG with @BeforeGroups and @AfterGroups annotations. Configure tests in HyperExecute and Maven for flexible test execution. -- [How To Perform Headless Browser Testing](https://www.testmuai.com/support/docs/perform-selenium-automation-on-headless-browsers.md): Run headless browser tests on the TestMu AI cloud Selenium Grid across 3000+ browser and OS combinations. -- [How to Perform Vividus Automation Tests Using TestMu AI?](https://www.testmuai.com/support/docs/vividus-integration.md): With this integration, run your tests from Vividus tool, directly on TestMu AI cloud-based Selenium Grid, on 3000+ desktop & mobile browser and OS combinations -- [How to Record and Replay Tests using LT Debug](https://www.testmuai.com/support/docs/record-and-replay-tests.md): Learn how to use LT Debug to replay or retest user flow as an automated test on 3000+ browser and OS combinations. -- [How to Resolve Version Conflict Issues](https://www.testmuai.com/support/docs/hyperexecute-how-to-resolve-version-conflict.md): Resolve version conflicts by ensuring consistency across YAML, package files, and environment variables for frameworks like Selenium, Playwright, and Cypress to avoid errors. -- [How to Run Groovy Script In Katalon With TestMu AI Selenium Grid?](https://www.testmuai.com/support/docs/running-groovy-script-in-katalon-with-testmu-selenium-grid.md): Run your Groovy script in Katalon with TestMu AI Selenium Grid. Perform automated cross browser testing with TestMu AI to ensure your development code renders seamlessly through an online Selenium grid providing 3000+ real browsers running through machines. -- [How To Run k6 Browser Tests On TestMu AI Platform](https://www.testmuai.com/support/docs/k6-browser-testing.md): Learn how to run your k6 Browser tests across 40+ browsers and operating systems on the TestMu AI automation platform -- [How to Run LEAPWORK Automation Tests on TestMu AI](https://www.testmuai.com/support/docs/leapwork-integration-with-testmu.md): Connect LEAPWORK with TestMu AI to run codeless automation flows on 3,000+ real browsers, devices, and OS, plus native mobile apps via a server connection. -- [How to Run Oxygen Automation Scripts Using TestMu AI?](https://www.testmuai.com/support/docs/run-oxygen-automation-scripts-on-testmu.md): Run Oxygen test automation scripts on the TestMu AI Selenium Grid, and perform cross browser testing on 3000+ real browser, on-cloud, for a seamless UI. -- [How To Run Playwright Tests In Parallel](https://www.testmuai.com/support/docs/parallel-testing-with-playwright.md): Here you can learn how to run parallel tests with Playwright across 40+ browser versions on the TestMu AI automation platform. -- [How To Run Playwright Tests On TestMu AI Android Devices](https://www.testmuai.com/support/docs/playwright-android.md): Here you can learn how to run Playwright tests on the TestMu AI android devices. -- [How To Run Playwright Tests On TestMu AI iOS Devices](https://www.testmuai.com/support/docs/playwright-ios-device.md): Here you can learn how to run Playwright tests on the TestMu AI iOS real devices. -- [How To Run Playwright Tests On TestMu AI Platform](https://www.testmuai.com/support/docs/playwright-testing.md): Here you can learn how to run Playwright tests on the TestMu AI test automation platform on over 40+ browsers and operating systems. -- [How To Run Puppeteer Tests On TestMu AI Platform](https://www.testmuai.com/support/docs/puppeteer-testing.md): Learn how to run your Puppeteer tests across 40+ browsers and operating systems on the TestMu AI automation platform -- [How to Run QMetry Automation Tests using TestMu AI?](https://www.testmuai.com/support/docs/qmetry-integration.md): With this integration, run your tests from QMetry app, directly on TestMu AI's cloud-based Selenium Grid, on 3000+ desktop & mobile browser and OS combinations -- [How To Run Ruby Tests On Online Selenium Grid](https://www.testmuai.com/support/docs/quick-guide-to-run-ruby-tests-on-testmu-selenium-grid.md): Guide to running Ruby and Selenium test scripts on TestMu AI Selenium automation grid online. Automated cross browser testing online using Selenium and Ruby on 3000+ browsers on cloud. -- [How to run selenium test with google cloud CI](https://www.testmuai.com/support/docs/google-cloud-ci-integration.md): This documentation will help you in integrating Selenium test suites with Google Cloud CI and perform automated testing on cloud Selenium grid. -- [How to setup HyperExecute Private Cloud with AWS](https://www.testmuai.com/support/docs/hyperexecute-private-cloud-setup-with-aws.md): This documentation will help you setup HyperExecute Private Cloud with AWS. -- [How to Test Amazon Connect Bots With TestMu AI](https://www.testmuai.com/support/docs/test-amazon-connect-bots.md): Automate phone and chat testing for Amazon Connect bots with TestMu AI, covering the Lex bot and the contact flow together across multi-turn scenarios. -- [How to Test Amazon Lex Bots With TestMu AI](https://www.testmuai.com/support/docs/test-amazon-lex-bots.md): Automated voice and chat testing for bots built on Amazon Lex. Drive multi-turn conversations across personas and edge cases and score every run with TestMu AI. -- [How to Test Apps Installed via App Center?](https://www.testmuai.com/support/docs/install-apps-from-app-center.md): Learn how to test apps installed from App Center on TestMu AI for optimal performance on real Android and iOS devices. -- [How to Test Apps Installed via App Store?](https://www.testmuai.com/support/docs/install-apps-from-app-store.md): Learn how to test apps installed from the App Store on TestMu AI for optimal performance on real iOS devices. -- [How to Test Apps Installed via Firebase?](https://www.testmuai.com/support/docs/install-apps-from-firebase.md): Learn how to test apps installed from the firebase on TestMu AI for optimal performance on real iOS devices. -- [How to Test Apps Installed via Play Store?](https://www.testmuai.com/support/docs/install-apps-from-play-store.md): Learn how to test apps installed via Play Store on TestMu AI, ensuring optimal performance on real Android devices. -- [How to Test Apps Installed via Test flight?](https://www.testmuai.com/support/docs/install-apps-from-test-flight.md): Learn how to test apps via Test Flight on TestMu AI, optimizing performance on real iOS devices before App Store release. -- [How to Test Bland AI Agents With TestMu AI](https://www.testmuai.com/support/docs/test-bland-agents.md): Automated phone, voice, and chat testing for Bland AI agents with TestMu AI. Drive multi-turn conversations across personas and edge cases, then score every run. -- [How to Test Chat and Phone Agents With TestMu AI Agent Testing CLI](https://www.testmuai.com/support/docs/testmu-a2a-cli.md): Install and use TestMu A2A CLI to test chat and phone AI agents from your terminal with scenario generation, red team security, and CI/CD support. -- [How to Test Copilot Studio Agents With TestMu AI](https://www.testmuai.com/support/docs/test-copilot-studio-agents.md): Automated phone and chat testing for agents built on Microsoft Copilot Studio. Drive multi-turn conversations across personas and edge cases with TestMu AI. -- [How to Test ElevenLabs Agents With TestMu AI](https://www.testmuai.com/support/docs/test-elevenlabs-agents.md): Automated phone, voice, and chat testing for agents built on the ElevenLabs Agents Platform. Drive multi-turn conversations across personas and score every run. -- [How to test flutter apps using Appium framework](https://www.testmuai.com/support/docs/testing-flutter-apps.md): Learn how to test flutter apps for Appium testing across 3000+ real Android and iOS devices. -- [How to test flutter apps using Appium framework on Virtual Devices](https://www.testmuai.com/support/docs/virtual-device-flutter-apps.md): Learn how to test flutter apps for Appium testing across 3000+ real Android and iOS devices for virtual devices -- [How to Test LangGraph Agents With TestMu AI](https://www.testmuai.com/support/docs/test-langgraph-agents.md): Automate chat testing for LangChain and LangGraph agents with TestMu AI, connecting to the deployed endpoint to drive multi-turn conversations and score runs. -- [How to Test LiveKit Agents With TestMu AI](https://www.testmuai.com/support/docs/test-livekit-agents.md): Automated phone and voice testing for agents built on LiveKit Agents. Drive full multi-turn conversations across personas and edge cases, and score every run. -- [How to Test Pipecat Agents With TestMu AI](https://www.testmuai.com/support/docs/test-pipecat-agents.md): Automated phone and voice testing for agents built on Pipecat. Drive full multi-turn conversations across personas and edge cases, and score every run with TestMu AI. -- [How to Test Retell Agents With TestMu AI](https://www.testmuai.com/support/docs/test-retell-agents.md): Automated phone, voice, and chat testing for Retell agents with TestMu AI. Drive multi-turn conversations across personas and edge cases, then score every run. -- [How to Test Synthflow Agents With TestMu AI](https://www.testmuai.com/support/docs/test-synthflow-agents.md): Automate phone, voice, and chat testing for Synthflow AI agents with TestMu AI. Drive multi-turn conversations across personas and score every run. -- [How to Test Twilio IVR Flows With TestMu AI](https://www.testmuai.com/support/docs/test-twilio-ivr.md): Automate phone testing for Twilio IVR flows with TestMu AI. Dial the number, walk every menu branch, cover DTMF and speech input, and score each run. -- [How to Test Vapi Agents With TestMu AI](https://www.testmuai.com/support/docs/test-vapi-agents.md): Automated phone, voice, and chat testing for Vapi AI agents with TestMu AI. Drive multi-turn conversations across personas and edge cases, then score every run. -- [How to Test Vertex AI Agent Builder Agents With TestMu AI](https://www.testmuai.com/support/docs/test-vertex-agent-builder-agents.md): Automate phone, voice, and chat testing for Vertex AI Agent Builder agents with TestMu AI, driving multi-turn conversations on every surface and scoring runs. -- [How to Test Voiceflow Agents With TestMu AI](https://www.testmuai.com/support/docs/test-voiceflow-agents.md): Automate chat and phone testing for Voiceflow AI agents with TestMu AI. Drive multi-turn conversations across personas and edge cases, scoring every run. -- [How to Test watsonx Assistant Bots With TestMu AI](https://www.testmuai.com/support/docs/test-watsonx-assistant-bots.md): Automated chat and phone testing for assistants built on IBM watsonx Assistant. Drive multi-turn conversations across personas and edge cases with TestMu AI. -- [How to Test Your First AI Agent With TestMu AI Agent Testing Platform](https://www.testmuai.com/support/docs/testing-your-first-ai-agent.md): Step-by-step guide to setting up and running your first AI agent test on TestMu AI: configure inputs, run validations, and review results. -- [How to troubleshoot TestMu AI Tunnel](https://www.testmuai.com/support/docs/troubleshooting-lambda-tunnel.md): This document will help you troubleshoot the most common challenges faced during local app testing using TestMu AI Tunnel. -- [How to Upload Custom Chrome Profile on TestMu AI](https://www.testmuai.com/support/docs/upload-custom-chrome-profile.md): Upload and manage custom Chrome profiles to run automated tests on TestMu AI with specific browser settings. -- [How to Use Auto Healing for Selenium Test Suites](https://www.testmuai.com/support/docs/auto-healing.md): Use Auto Healing in Selenium tests to reduce test flakiness and improve test suite reliability automatically. -- [How to use Auto Healing for your Playwright test suites](https://www.testmuai.com/support/docs/playwright-auto-healing.md): Learn how to use TestMu AI Auto Healing feature in your Playwright tests to avoid test flakiness and increase reliability of your test suites. -- [How to Use AutoHeal Using Hooks](https://www.testmuai.com/support/docs/autoheal-with-hooks.md): Enable and disable AutoHeal at any point in your Selenium test script using hooks for precise control. -- [How to Use CustomHeaders Capability](https://www.testmuai.com/support/docs/appium-custom-headers.md): Learn how to use TestMu AI CustomHeaders capability to add custom headers to your tests and bypass firewalls restrictions in app automation tests. -- [How to use in-session testing tools?](https://www.testmuai.com/support/docs/how-to-use-in-session-testing-tools.md): With TestMu AI, perform live interactive testing of websites and web applications on real Android and iOS devices to ensure seamless user experience. -- [How To Use Load Balancing In TestMu AI Tunnel](https://www.testmuai.com/support/docs/load-balancing-in-lambda-tunnel.md): Load Balancing feature allows you to perform parallel testing of your local or private website on 3000+ TestMu AI's real browsers through TestMu AI Tunnel. -- [How to use Testing Tools for Desktop Browsers?](https://www.testmuai.com/support/docs/real-time-desktop-browser-testing-tools.md): Enhance your web testing with TestMu AI's comprehensive tool suite. Capture screenshots, record sessions, manage files, and more. Test across devices, browsers, and locations for optimal results. -- [How to use Testing Tools for Mobile Applications?](https://www.testmuai.com/support/docs/real-time-mobile-app-testing-tools.md): Enhance mobile app testing with TestMu AI's comprehensive tools. Simulate actions, capture screenshots, record sessions, and more. Test across devices, locations, and network conditions for optimal results. -- [How to use Testing Tools for Mobile Browsers?](https://www.testmuai.com/support/docs/real-time-mobile-browser-testing-tools.md): Enhance your mobile app testing with TestMu AI's comprehensive tools. Simulate actions, capture screenshots, record sessions, and more. Test across devices, locations, and network conditions for optimal results. -- [How to use testing tools in-session?](https://www.testmuai.com/support/docs/how-to-use-testing-tools-in-session.md): Mobile app testing is the process of testing a mobile application to ensure that their functionality and usability is not compromised when installed across multiple Android and iOS devices. With TestMu AI, you can test mobile applications manually to ensure they perform seamlessly across all Android and iOS real devices. -- [How To View Lighthouse Performance Metrics](https://www.testmuai.com/support/docs/view-lighthouse-performance-metrics.md): View Lighthouse performance scores for your website using the TestMu AI cloud-based Selenium Grid. -- [HTTP Basic Authentication](https://www.testmuai.com/support/docs/http-basic-authentication.md): HTTP basic authentication represents a mechanism of challenge and response to help the server make a request authentication from a client, passing the User ID and password in the URL as the standard HTTP "Authorization" header. -- [http2-support](https://www.testmuai.com/support/docs/http2-support.md) -- [Hybrid Strategy in HyperExecute](https://www.testmuai.com/support/docs/hyperexecute-hybrid-strategy.md): Unlock efficient testing with HyperExecute's Hybrid Strategy! Our guide walks you through combining AutoSplit and Matrix Strategy for parallel test execution, saving time without sacrificing test coverage. -- [HyperExecute CLI - Run test on HyperExecute](https://www.testmuai.com/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid.md): Explains about HyperExecute CLI -- [HyperExecute CSharp Use Cases](https://www.testmuai.com/support/docs/hyperexecute-csharp-use-cases.md): Explore real-world C# test automation use cases on HyperExecute with NUnit, MSTest, and SpecFlow to speed up your daily testing workflows. -- [HyperExecute Emulator and Simulator Devices](https://www.testmuai.com/support/docs/hyperexecute-emu-simu-devices-list.md): List of devices available on HyperExecute for Emulator and Simulator -- [HyperExecute Environment Variable Setup](https://www.testmuai.com/support/docs/hyperexecute-environment-variable-setup.md): Find out how to get access to Hyperexecute authentication credentials, fetching them from the Automation tab in your TestMu AI profile, and configuring them on Mac OSX/Linux, Windows systems. -- [HyperExecute FAQs !!](https://www.testmuai.com/support/docs/hyperexecute-faqs.md): Get quick answers to all your HyperExecute queries with our comprehensive FAQs! -- [HyperExecute GUI App](https://www.testmuai.com/support/docs/hyperexecute-cli-gui.md): This documentation outlines the steps on how to trigger a job on hyperexecute using the underpass app - gui app for hyperexecute cli. -- [HyperExecute how to guide](https://www.testmuai.com/support/docs/hyperexecute-how-to-guide.md): Discover the HyperExecute cases -- [HyperExecute integration: Selenium accessibility automation](https://www.testmuai.com/support/docs/selenium-hyperexecute-accessibility-tests.md): HyperExecute integration guide: run Selenium-based Accessibility Automation on the HyperExecute grid, align YAML with accessibility capabilities, and open reports after the job completes. -- [HyperExecute Job Reports](https://www.testmuai.com/support/docs/hyperexecute-job-reports.md): Effortlessly download your HyperExecute job reports from the UI. Click the Report button on the top right-hand side of your tasks page. . -- [HyperExecute Job Widgets](https://www.testmuai.com/support/docs/hyperexecute-job-widgets.md): Quickly and easily set up Hyperexecute Widgets for efficient analysis of test results. -- [Hyperexecute Knowledge Base](https://www.testmuai.com/support/docs/hyperexecute-knowledge-base.md): Discover the HyperExecute Knowledge Base for insights on concepts, setup, configuration, and more. Get answers to your questions about HyperExecute. -- [HyperExecute Language-Specific Use Cases](https://www.testmuai.com/support/docs/hyperexecute-testng-use-cases.md): Practical TestNG automation use cases on HyperExecute: parallel execution, data-driven tests, and Java workflows to accelerate QA delivery. -- [HyperExecute Private Cloud](https://www.testmuai.com/support/docs/hyperexecute-private-cloud-setup.md): Optimize Your Test Execution with HyperExecute Private Cloud Setup | TestMu AI -- [HyperExecute Private Cloud Benefits](https://www.testmuai.com/support/docs/hyperexecute-private-cloud-benefits.md): Explore the benefits of HyperExecute on TestMu AI's private cloud for powerful automation testing. -- [HyperExecute Private Cloud on JumpHost](https://www.testmuai.com/support/docs/hyperexecute-private-cloud-on-jumphost.md): Deploy & run HyperExecute Private Cloud on Azure with JumpHost! Follow our guide for setup, test triggering, and dashboard viewing for a smooth, customizable testing experience. -- [HyperExecute Private Cloud Setup with Azure](https://www.testmuai.com/support/docs/hyperexecute-private-cloud-setup-with-azure.md): Discover HyperExecute Private Cloud Setup with Azure and understand how to inherit configurations for optimal performance. Explore the integration of hyperexecute inherit config. -- [HyperExecute Projects](https://www.testmuai.com/support/docs/hyperexecute-projects.md): Discover the power of HyperExecute Projects for agile test execution. Learn how to set up and run custom projects with ease, maximizing testing efficiency. -- [HyperExecute Python Use Cases](https://www.testmuai.com/support/docs/hyperexecute-python-use-cases.md): Explore Python test automation use cases on HyperExecute with PyTest, Behave, and Robot Framework to streamline your daily testing pipelines. -- [HyperExecute Release Notes](https://www.testmuai.com/support/docs/hyperexecute-release-notes.md): See the release notes for HyperExecute to stay informed about new features, fixes, and enhancements that improve your testing performance. -- [HyperExecute Release Notes 2023](https://www.testmuai.com/support/docs/hyperexecute-release-notes-2023.md): Consolidated HyperExecute release notes for 2023 -- [HyperExecute Release Notes 2024](https://www.testmuai.com/support/docs/hyperexecute-release-notes-2024.md): Consolidated HyperExecute release notes for 2024 -- [HyperExecute Release Notes 2025](https://www.testmuai.com/support/docs/hyperexecute-release-notes-2025.md): HyperExecute Release Notes 2025 -- [HyperExecute Resource Utilization Widgets](https://www.testmuai.com/support/docs/hyperexecute-resource-widgets.md): HyperExecute's Concurrency Trends widget tracks queued and running tests, offering insights into resource utilization. -- [HyperExecute Stage Widgets](https://www.testmuai.com/support/docs/hyperexecute-stage-widgets.md): HyperExecute Stage Widgets track and group stages by status (Completed, Skipped, Failed, Aborted, etc.) for efficient monitoring. -- [HyperExecute Status](https://www.testmuai.com/support/docs/hyperexecute-status.md): This documentation outlines the status list and the meaning of status used in job, test, task and scenario level at hyperexecute. -- [HyperExecute Task Widgets](https://www.testmuai.com/support/docs/hyperexecute-task-widgets.md): HyperExecute Task Widgets enable trend analysis of each job's tasks, categorized and filterable by specific dates and times. -- [HyperExecute Test Chains](https://www.testmuai.com/support/docs/hyperexecute-test-chains.md): Learn how HyperExecute Test Chains link dependent jobs into end-to-end pipelines, enabling smarter orchestration and faster automation runs. -- [HyperExecute Test Widgets](https://www.testmuai.com/support/docs/hyperexecute-test-widgets.md): HyperExecute Test Widgets allow trend analysis of tests over time, providing insights into platform performance. -- [HyperExecute Use Cases](https://www.testmuai.com/support/docs/hyperexecute-use-cases.md): Discover the HyperExecute Use cases to understand the real world implementations of it. -- [HyperExecute vs Traditional Test Grids](https://www.testmuai.com/support/docs/hyperexecute-vs-traditional-test-grids.md): Discover the nuances in testing efficiency with HyperExecute vs. Traditional Test Grids Explore the comparison and choose the right approach for optimized testing. -- [HyperExecute Widgets](https://www.testmuai.com/support/docs/hyperexecute-widgets.md): HyperExecute Widgets offer comprehensive insights into executed Jobs, all in one convenient location for easy exploration. -- [HyperExecute YAML](https://www.testmuai.com/support/docs/hyperexecute-yaml-parameters.md): This document list enumerates the diverse configuration options supported by HyperExecute YAML, providing you with the flexibility to tailor test execution according to the specific requirements of your project. -- [HyperExecute YAML Creation for Playwright](https://www.testmuai.com/support/docs/hyperexecute-yaml-creation-for-playwright.md): A step-by-step guide to creating a HyperExecute YAML configuration for Playwright tests: matrix strategy, dependencies, and parallel runs. -- [HyperExecute Yaml Version 0.2](https://www.testmuai.com/support/docs/hyperexecute-yaml-version0.2.md): Learn more about HyperExecute YAML 0.2 -- [HyperExecute- Monitoring CPU and Memory Resource Consumption](https://www.testmuai.com/support/docs/hyperexecute-task-metrics.md): HyperExecute Task Metrics helps you to understand your resource utilization for different stages of test execution. -- [Iframes and Embedded Content in SmartUI](https://www.testmuai.com/support/docs/smartui-iframes-and-embedded-content.md): How SmartUI treats same-origin and cross-origin iframes for screenshots and comparison, with practical tips for videos, widgets, and automation context. -- [Ignore and Select Regions on SmartUI Web Hooks (Coordinates and Selectors)](https://www.testmuai.com/support/docs/smartui-hooks-region-ignore.md): Define ignore and select regions on the SmartUI Web Hooks path using pixel coordinates in addition to DOM selectors, with ignoreDOM and selectDOM. -- [Ignore Color-Only Changes in a Region (ignoreColors)](https://www.testmuai.com/support/docs/smartui-ignore-colors.md): Mark regions where SmartUI ignores color-only changes during comparison while structural changes still flag, via the SDK and Lambda Hooks with the Smart Ignore strategy. -- [Ignore or Select Annotated Regions](https://www.testmuai.com/support/docs/smartui-draw-on-ui.md): In this documentation, learn how to ignore, select the annotated regions of the tests with TestMu AI's SmartUI, including ignore colors, floating regions, and select ignore features. -- [Ignore Unnecessary Files in your Payload](https://www.testmuai.com/support/docs/hyperexecute-gitignore.md): Optimize HyperExecute CLI with .gitignore & .hyperexecuteignore files. Exclude unneeded files for efficient test script execution. -- [Image Analyzer Testing With TestMu AI](https://www.testmuai.com/support/docs/image-analyzer.md): Test image generation and analysis agents on TestMu AI. Score images against a prompt with a 0 to 100 alignment score, brand rules, and technical specs. -- [Import Test Cases from CSV](https://www.testmuai.com/support/docs/csv-import.md): Simplify your csv import with TestMu AI's Test Manager. Easily upload and manage test cases to enhance testing efficiency and productivity. -- [Inaccessible Text Focus](https://www.testmuai.com/support/docs/accessibility-android-rule-inaccessible-text-focus.md): Rule-level Accessibility guidance for Inaccessible Text Focus on Android. -- [Inbound Phone Agent Testing With TestMu AI](https://www.testmuai.com/support/docs/inbound-phone-agent.md): Test an inbound phone agent with TestMu AI. Live test calls, production recording analysis, voice and noise simulation, and 30+ call quality metrics. -- [Info and Relationships (1.3.1)](https://www.testmuai.com/support/docs/accessibility-web-rule-1-3-1-info-and-relationships.md): Rule-level Accessibility guidance for Info and Relationships (WCAG 1.3.1 Level A). -- [Inherit Your YAML Configurations](https://www.testmuai.com/support/docs/hyperexecute-inherit-config.md): Discover the Power of HyperExecute Inherit Config | Learn how to optimize your testing workflow with HyperExecute and inherit configurations seamlessly.! -- [Insights Best Practices](https://www.testmuai.com/support/docs/insights-best-practices.md): Best practices for using TestMu AI Insights dashboards, widgets, and insights modules to maximize value and drive data-driven testing decisions. -- [Insights Dashboard](https://www.testmuai.com/support/docs/insights-dashboard.md): Manage system and custom fields in TestMu AI's Test Manager, enhancing test case organization and streamlining processes with detailed fields and values. -- [Install Custom Certificates on the VM Through Pre-run Executables](https://www.testmuai.com/support/docs/setup-pre-run-executable.md): Run custom scripts before Selenium tests start to install certificates required for test execution. -- [Install NPM Packages via Tunnel](https://www.testmuai.com/support/docs/npm-via-tunnel.md): Guide to install npm packages using a flag that enables users to route npm install traffic via a tunnel. -- [Install TestMu AI Accessibility Toolkit](https://www.testmuai.com/support/docs/accessibility-testing-install-devtools.md): Easily install the TestMu AI Accessibility Toolkit extension from the Chrome Web Store to improve website accessibility. -- [Installing Kane CLI](https://www.testmuai.com/support/docs/kane-cli-installation.md): Install Kane CLI using npm or Homebrew. Supports macOS (Apple Silicon and Intel), Linux (x64 and arm64), and Windows (x64). -- [Insufficient Color Ratio](https://www.testmuai.com/support/docs/accessibility-android-rule-insufficient-color-ratio.md): Rule-level Accessibility guidance for Insufficient Color Ratio on Android. -- [Insufficient Target Spacing](https://www.testmuai.com/support/docs/accessibility-android-rule-insufficient-target-spacing.md): Rule-level Accessibility guidance for Insufficient Target Spacing on Android. -- [Integrate algoQA with HyperExecute](https://www.testmuai.com/support/docs/hyperexecute-algoqa-integration.md): Unleash the power of algoQA & TestMu AI HyperExecute! Streamline test management & experience fast execution. -- [Integrate Azure Pipelines With TestMu AI](https://www.testmuai.com/support/docs/integrate-azure-pipelines-with-testmu.md): TestMu AI now integrates with Azure to boost your go-to-market delivery. Perform automated cross browser testing with TestMu AI to ensure your development code renders seamlessly through an online Selenium grid providing 3000+ real browsers running through machines. -- [Integrate Provar with HyperExecute](https://www.testmuai.com/support/docs/hyperexecute-provar-integration.md): Integrate Provar with HyperExecute for Salesforce test automation. Configure, generate build files, set up licenses, and execute tests seamlessly. -- [Integrate qTest with HyperExecute](https://www.testmuai.com/support/docs/hyperexecute-qtest-integration.md): Unleash the power of QTest & TestMu AI HyperExecute! Streamline test management & experience blazing-fast execution. -- [Integrate Sikuli with HyperExecute](https://www.testmuai.com/support/docs/hyperexecute-sikuli-integration.md): Automate the visual testing of any software by integrating Sikuli with HyperExecute. -- [Integrate SmartUI SDK with Cypress Tests](https://www.testmuai.com/support/docs/smartui-cypress-sdk.md): In this documentation, learn how integrate your Cypress automated tests with TestMu AI's SmartUI. -- [Integrate SmartUI SDK with Playwright - Java](https://www.testmuai.com/support/docs/smartui-playwright-java-sdk.md): In this documentation, learn how integrate your Playwright Java automated tests with TestMu AI's SmartUI. -- [Integrate SmartUI SDK with Playwright-JavaScript](https://www.testmuai.com/support/docs/smartui-playwright-sdk.md): In this documentation, learn how integrate your Playwright automated tests with TestMu AI's SmartUI. -- [Integrate SmartUI SDK with Playwright-Python](https://www.testmuai.com/support/docs/smartui-playwright-python-sdk.md): In this documentation, learn how integrate your Playwright Python automated tests with TestMu AI's SmartUI. -- [Integrate SmartUI SDK with Puppeteer Tests](https://www.testmuai.com/support/docs/smartui-puppeteer-sdk.md): In this documentation, learn how integrate your Puppeteer JavaScript automated tests with TestMu AI's SmartUI. -- [Integrate SmartUI SDK with Selenium-CSharp](https://www.testmuai.com/support/docs/smartui-selenium-csharp-sdk.md): In this documentation, learn how integrate your Selenium C# automated tests with TestMu AI's SmartUI. -- [Integrate SmartUI SDK with Selenium-Java](https://www.testmuai.com/support/docs/smartui-selenium-java-sdk.md): In this documentation, learn how integrate your Selenium Java automated tests with TestMu AI's SmartUI. -- [Integrate SmartUI SDK with Selenium-JavaScript](https://www.testmuai.com/support/docs/smartui-selenium-js-sdk.md): In this documentation, learn how integrate your Selenium JavaScript automated tests with TestMu AI's SmartUI. -- [Integrate SmartUI SDK with Selenium-Python](https://www.testmuai.com/support/docs/smartui-selenium-python-sdk.md): In this documentation, learn how integrate your Selenium Python automated tests with TestMu AI's SmartUI. -- [Integrate SmartUI SDK with Selenium-Ruby](https://www.testmuai.com/support/docs/smartui-selenium-ruby-sdk.md): In this documentation, learn how integrate your Selenium Ruby automated tests with TestMu AI's SmartUI. -- [Integrate SmartUI SDK with Testcafe Tests](https://www.testmuai.com/support/docs/smartui-testcafe-sdk.md): In this documentation, learn how integrate your Testcafe automated tests with TestMu AI's SmartUI. -- [Integrate SmartUI SDK with WebdriverIO Tests](https://www.testmuai.com/support/docs/smartui-wdio-sdk.md): In this documentation, learn how integrate your WebdriverIO automated tests with TestMu AI's SmartUI. -- [Integrate SmartUI with Appium Tests](https://www.testmuai.com/support/docs/smartui-appium-sdk.md): Learn how to integrate your Appium automated tests with TestMu AI's SmartUI. Java uses both SDK and hooks integration, while JavaScript, Python, Ruby, and C# use Hooks integration. -- [Integrate Test Cases](https://www.testmuai.com/support/docs/hyperexecute-integrate-azure-test-plan.md): To ensure effective test execution, integrate your test plan with HyperExecute by downloading the required executable files, configuring YAML files, and starting your test plan. -- [Integrate Testim with HyperExecute](https://www.testmuai.com/support/docs/hyperexecute-testim-integration.md): Testim, a test case management tool, and HyperExecute, a cloud-based test execution platform streamline your testing process by efficiently managing test cases. -- [Integrate TestMu AI Extension With Azure Pipelines](https://www.testmuai.com/support/docs/integrate-testmu-extension-with-azure-pipelines.md): Now integrate your Azure Pipelines with TestMu AI extension plugin to automate your test scripts. Perform live interactive and automated cross-browser testing on 3000+ real browsers and operating systems online using TestMu AI. -- [Integrate TestMu AI Report With Jenkins Pipeline](https://www.testmuai.com/support/docs/integrating-testmu-report-with-jenkins-pipeline.md): Learn how to integrate TestMu AI Report with Jenkins Pipeline. -- [Integrate TestMu AI with Azure DevOps | TestMu AI](https://www.testmuai.com/support/docs/vsts-integration.md): TestMu AI provides integrations with Azure DevOps to help you log bugs directly from the middle of your test session on TestMu AI to your Azure DevOps project. -- [Integrate TestMu AI with Cypress Dashboard](https://www.testmuai.com/support/docs/integrate-testmu-with-cypress.md): This article guides you on how to integrate TestMu AI platform with Cypress Dashboard. -- [Integrate TestMu AI with GitHub | TestMu AI](https://www.testmuai.com/support/docs/github-integration.md): TestMu AI provides integrations with GitHub to help you log bugs directly from the middle of your test session on TestMu AI to your GitHub repository. -- [Integrate TestMu AI With GitHub Using Repo-Based OAuth](https://www.testmuai.com/support/docs/github-repo-based-oauth-integration.md): Easily integrate TestMu AI with GitHub using repo-based OAuth to log bugs directly from the TestMu AI platform to your GitHub repositories. -- [Integrate TestMu AI With Jenkins Freestyle Project](https://www.testmuai.com/support/docs/integrating-testmu-with-jenkins-freestyle-project.md): This document will help you integrate your Jenkins freestyle project with TestMu AI to ensure your Selenium testing scripts are executed on a cloud-based Grid of 3000+ real browsers. -- [Integrate TestMu AI with Jira | TestMu AI](https://www.testmuai.com/support/docs/jira-integration.md): TestMu AI provides integrations with JIRA to help you log bugs directly from the middle of your test session on TestMu AI to JIRA platform. -- [Integrate TestMu AI with Trello | TestMu AI](https://www.testmuai.com/support/docs/trello-integration.md): TestMu AI provides integrations with Trello to help you log bugs directly from the middle of your test session on TestMu AI to your Trello card. -- [Integrate Zephyr Scale with HyperExecute](https://www.testmuai.com/support/docs/hyperexecute-zephyr-scale-integration.md): Zephyr, a test case management tool, and HyperExecute, a cloud-based test execution platform streamline your testing process by efficiently managing test cases. -- [Integrating Mattermost With TestMu AI](https://www.testmuai.com/support/docs/integrating-mattermost-with-testmu.md): The TestMu AI Mattermost Integration allows you to share or push a bug to your specified Mattermost channel from the TestMu AI platform in a single click. You can share any UI observations and input with your team on Mattermost anytime, by just capturing a screenshot in the middle of your test session through TestMu AI. You can also annotate the screenshot or highlight your issue or input. The fields populated by you while marking as a bug through TestMu AI gets displayed as information on Mattermost for that testing instance. -- [Integrating TestMu AI as a Bitrise CI/CD Step [XCUITest]](https://www.testmuai.com/support/docs/bitrise-integration-xcuitest.md): Integrate TestMu AI with Bitrise CI/CD Step [XCUITest] to run your XCUITest workflow successfully on TestMu AI. -- [Integrating Wrike With TestMu AI](https://www.testmuai.com/support/docs/integrating-wrike-with-testmu.md): With TestMu AI + Wrike integration, now you can mark your bug directly to Wrike workspace from TestMu AI even in middle of testing, and effectively track it. -- [Integrating Your Jenkins Pipeline With TestMu AI](https://www.testmuai.com/support/docs/integrating-your-jenkins-pipeline-with-testmu.md): TestMu AI now integrates with Jenkins pipeline to boost your go-to market delivery. Perform automated cross browser testing with TestMu AI to ensure your development code renders seamlessly through an online Selenium grid providing 3000+ real browsers running through machines. -- [Integration With CI/CD Tools](https://www.testmuai.com/support/docs/hyperexecute-integration-with-ci-cd-tools.md): HyperExecute offers integrations with numerous CI/CD tools to help you fast-track your robust delivery pipeline for pacing up your go-to-market launch. -- [Integration With fastlane Plugin](https://www.testmuai.com/support/docs/fastlane-with-testmu.md): Upload Android and iOS apps using the TestMu AI fastlane plugin and test your app on TestMu AI real devices. -- [Integration With Products](https://www.testmuai.com/support/docs/hyperexecute-integration-with-products.md): Boost your testing workflow with TestMu AI's seamless integrations. Achieve efficient cross-browser testing with popular tools for faster, more reliable results. -- [Integration With TestMu AI Products](https://www.testmuai.com/support/docs/he-integration-with-testmu-products.md): Learn to seamlessly integrate HyperExecute with TestMu AI products like Smart UI Testing for enhanced and efficient testing experiences! -- [Integrations](https://www.testmuai.com/support/docs/integration-with-hyperexecute.md): Learn more about the integrations HyperExecute offers. -- [Integrations (JIRA / Slack)](https://www.testmuai.com/support/docs/accessibility-report-integrations.md): Use integrations to route Accessibility results and issues into collaboration and issue management workflows. -- [Integrations For Codeless Automation Tools](https://www.testmuai.com/support/docs/integrations-with-codeless-automation-tools.md): TestMu AI integrations to your favorite codeless automation tool can help you perform automated browser testing on 3000+ real browsers and operating systems. -- [Intelligent Test Planner - Automate your Test Generation](https://www.testmuai.com/support/docs/kane-ai-test-plan.md): Automate test generation with KaneAI's Intelligent Test Planner. Define objectives, set parameters, and execute tests seamlessly for web apps. -- [Interactive Role Undefined](https://www.testmuai.com/support/docs/accessibility-android-rule-interactive-role-undefined.md): Rule-level Accessibility guidance for Interactive Role Undefined on Android. -- [Introducing HyperExecute MCP Tool](https://www.testmuai.com/support/docs/hyperexecute-mcp-server.md): HyperExecute MCP Tool lets you set up test jobs, generate YAML configurations, and monitor HyperExecute runs directly from your IDE using natural language prompts. -- [Introducing SmartUI MCP Tool](https://www.testmuai.com/support/docs/smartui-mcp-server.md): Debug visual regressions with SmartUI MCP Tool. Get natural-language summaries of pixel, layout, DOM, and perceptual differences from SmartUI comparison runs. -- [Introducing Test Manager MCP Tool](https://www.testmuai.com/support/docs/test-manager-mcp-server.md): Manage the complete test management lifecycle with Test Manager MCP Tool. Generate test cases with AI, build configured test runs, record results in bulk, and track milestones and coverage from your AI client. -- [Introducing TestMu AI MCP Server](https://www.testmuai.com/support/docs/testmu-mcp-server.md): With TestMu AI MCP Server, explore all TestMu AI MCP Server tools in one place, from setup to usage. -- [Introduction to KaneAI](https://www.testmuai.com/support/docs/getting-started-with-kane-ai.md): World's first GenAI-Native testing assistant that simplifies end-to-end automation testing for web, mobile, and cloud apps, using natural language. -- [iOS Accessibility Checklist](https://www.testmuai.com/support/docs/accessibility-ios-what-we-cover.md): Supported iOS accessibility rules with links to rule pages, plus a manual test checklist for human-led review. -- [iOS Accessibility Manual Checklist](https://www.testmuai.com/support/docs/accessibility-ios-what-we-do-not-cover.md): This page has moved. iOS coverage and manual checklist are now on one page. -- [iOS Accessibility Rules - Quick Reference](https://www.testmuai.com/support/docs/accessibility-ios-rules.md): Use iOS accessibility rule references to understand mobile-app findings from Accessibility App Scanner and Native App Automation. -- [iOS App Settings](https://www.testmuai.com/support/docs/appium-ios-app-settings.md): iOS App Settings are the permissions or preferences that can be enabled/disabled for an app through iOS settings. -- [iOS Entitlements](https://www.testmuai.com/support/docs/ios-entitlements.md): Quick guide on entitlements for iOS applications. -- [iOS Rule Repository](https://www.testmuai.com/support/docs/accessibility-ios-rule-repository.md): Browse iOS accessibility rule guidance, WCAG mappings, and remediation references. -- [IP Geolocation](https://www.testmuai.com/support/docs/appium-ip-geolocation.md): This document provides information about configuring IP Geolocation for tests on the TestMu AI platform and also provides a list of supported geolocations on Real Devices. -- [IP Geolocation on Real Devices App](https://www.testmuai.com/support/docs/ip-geolocation-on-real-devices-app.md): Test your app and website across real devices with TestMu AI. -- [IP Geolocation on Real Devices Browser](https://www.testmuai.com/support/docs/ip-geolocation-on-real-devices-browser.md): Test your app and website across real devices and browsers with TestMu AI. -- [IPsec Site-to-Site VPN Setup Guide](https://www.testmuai.com/support/docs/site-to-site-ipsec-vpn-setup.md): A comprehensive guide to setting up IPsec Site-to-Site VPN for testing privately hosted applications on TestMu AI cloud infrastructure. -- [Issue Summary](https://www.testmuai.com/support/docs/accessibility-testing-dashboard-issue-summary.md): Understand the summary view of Accessibility issues, affected elements, guidelines, and counts. -- [JAWS on Windows](https://www.testmuai.com/support/docs/accessibility-jaws-windows.md): JAWS on Windows for enterprise manual accessibility validation: sessions, web modes, and triage patterns. -- [Jenkins Integration](https://www.testmuai.com/support/docs/jenkins-with-hyperexecute.md): Streamline automation with Jenkins and HyperExecute by exploring TestMu AI's comprehensive support documentation. -- [Job Artifacts](https://www.testmuai.com/support/docs/hyperexecute-artifacts.md): Learn how to generate the Artifacts for your desired framework and seamlessly interact with artifacts through the HyperExecute UI. -- [Job Reports](https://www.testmuai.com/support/docs/hyperexecute-reports.md): Effortlessly explore HyperExecute Reports! Gain a deeper understanding of your test outcomes with the comprehensive insights and analytics offered by TestMu AI. -- [Jobs Archiving](https://www.testmuai.com/support/docs/hyperexecute-jobs-archiving.md): Optimize your workspace with HyperExecute job archiving to boost productivity. Keep only relevant jobs. Learn more! -- [JS Snippets & Workarounds](https://www.testmuai.com/support/docs/kaneai-kb-js-snippets-and-workarounds.md): Work around natural language limitations in KaneAI using JavaScript snippets for DOM manipulation, validation, date pickers, shadow DOM, and more -- [JumpCloud User Provisioning](https://www.testmuai.com/support/docs/jumpcloud-scim.md): Integrating TestMu AI SCIM with JumpCloud -- [JUnit 5](https://www.testmuai.com/support/docs/accessibility-junit5-test.md): Run Accessibility Automation with Selenium and JUnit 5: capabilities, lifecycle hooks, and dashboard reporting. -- [JUnit XML Report](https://www.testmuai.com/support/docs/junit-xml-report.md): Generate a consolidated JUnit XML report with HyperExecute. Track test results, including pass/fail statuses, with easy-to-follow configuration steps -- [Kane CLI Authentication](https://www.testmuai.com/support/docs/kane-cli-authentication.md): Authenticate Kane CLI with OAuth or basic auth, manage multiple profiles, and securely store credentials. -- [Kane CLI Documentation - Getting Started](https://www.testmuai.com/support/docs/kane-cli-introduction.md): Kane CLI is an AI-powered command-line tool that runs browser automation tests in plain English: from your terminal, IDE, or CI pipeline. -- [Kane CLI Skills for AI Agents](https://www.testmuai.com/support/docs/kane-cli-skills.md): Install the Kane CLI skill for Claude Code, Codex CLI, or Gemini CLI to enable browser automation directly from your AI coding agent. -- [KaneAI - API Testing](https://www.testmuai.com/support/docs/kane-ai-api-testing.md): Learn how to test api via kane ai -- [KaneAI - Command Guide](https://www.testmuai.com/support/docs/kane-ai-command-guide.md): Master KaneAI commands to automate web actions like navigation, clicking, typing, and more. Create dynamic, readable test cases with ease -- [KaneAI - Geolocation, Tunnel and Proxy Support](https://www.testmuai.com/support/docs/kane-ai-geolocation-tunnel-proxy.md): Learn how to run your web tests on KaneAI using geolocation, tunnel, and dedicated proxy configurations. -- [KaneAI - JavaScript Execution](https://www.testmuai.com/support/docs/kane-ai-javascript-execution.md): Learn how to execute your tests on Kane AI using custom javascript snippets -- [KaneAI - Jira Integration](https://www.testmuai.com/support/docs/kane-ai-jira-integration.md): Learn how to integrate your jira tickets with KaneAI and test planner using Atalssian Marketplace -- [KaneAI - Scroll in Element](https://www.testmuai.com/support/docs/kane-ai-scroll-in-feature.md): Learn how to run scroll commands during your web testing. -- [KaneAI - Using Datasets on Test Manager](https://www.testmuai.com/support/docs/kane-ai-using-datasets.md): Learn how to use datasets in the KaneAI test cases -- [KaneAI - Using Parameters](https://www.testmuai.com/support/docs/kane-ai-using-parameters.md): Learn how to use parameters in the KaneAI test cases -- [KaneAI - Utilizing JSON Variables for Assertions](https://www.testmuai.com/support/docs/kane-ai-using-json-variables.md): Learn how to use JSON variables in the KaneAI test cases for assertions -- [KaneAI App Agent - Guidelines for Writing Instructions](https://www.testmuai.com/support/docs/kane-ai-app-test-writing-guidelines.md): Learn how to write instructions for running the kane ai app agent smoothly and without any problem -- [KaneAI Automation Code Generation](https://www.testmuai.com/support/docs/kane-ai-automation-code-generation.md): Explore KaneAI's features and components to generate automation test code. -- [KaneAI FAQs](https://www.testmuai.com/support/docs/kaneai-faqs.md): Frequently asked questions about KaneAI, including supported execution, results, and running tests on sites protected by CAPTCHA or Cloudflare Turnstile. -- [KaneAI Free Trial is here!](https://www.testmuai.com/support/docs/kane-ai-free-trial-is-here.md): Here's what your KaneAI free trial plan includes. -- [KaneAI Knowledge Base](https://www.testmuai.com/support/docs/kaneai-kb-knowledge-base-index.md): Central hub for all KaneAI test authoring guides, best practices, workarounds, and reference documentation -- [KaneAI Manual Interaction Mode](https://www.testmuai.com/support/docs/kaneai-manual-interaction.md): Learn how to KaneAI simplifies test automation by allowing manual interaction to create prompts in natural language, analyzing test steps, generating code in various languages. -- [KaneAI Release Notes](https://www.testmuai.com/support/docs/kaneai-release-notes.md): Keep yourself informed about new releases from KaneAI. View our AI-powered testing assistant's detailed release notes, which include new features and enhancements. -- [KaneAI Web Agent - Guidelines for Writing Instructions](https://www.testmuai.com/support/docs/kane-ai-web-test-writing-guidelines.md): Learn how to write instructions for running the kane ai web agent smoothly and without any problem -- [Karate Automation on HyperExecute](https://www.testmuai.com/support/docs/hyperexecute-karate-testing.md): Learn how to run Selenium automation tests on HyperExecute using the Karate framework -- [Karate Report](https://www.testmuai.com/support/docs/karate-report.md): Learn how to configure and generate Karate test execution reports on HyperExecute with step-by-step guidance for seamless test insights and reporting. -- [Katalon](https://www.testmuai.com/support/docs/katalon-integration-with-hyperexecute.md): Integrate Katalon Studio with HyperExecute for cross-platform testing! Follow our guide to perform web & mobile automation testing on multiple OS with codeless UI validation. -- [Katalon Integration with TestMu AI](https://www.testmuai.com/support/docs/katalon-integration.md): Leverage TestMu AI's cloud mobile devices to run automation testing using Katalon Studio. -- [Katalon Reports on HyperExecute](https://www.testmuai.com/support/docs/hyperexecute-katalon-report.md): Learn how to generate Katalon test report for HyperExecute on TestMu AI and download the reports from the dashboard -- [Key Features of HyperExecute](https://www.testmuai.com/support/docs/key-features-of-hyperexecute.md): Various key features of HyperExecute is mentioned in this document -- [Keyboard (2.1.1)](https://www.testmuai.com/support/docs/accessibility-web-rule-2-1-1-keyboard.md): Rule-level Accessibility guidance for Keyboard (WCAG 2.1.1 Level A). -- [Keyboard Accessibility Scan (Beta)](https://www.testmuai.com/support/docs/accessibility-keyboard-scan.md): Keyboard Accessibility Scanning tools highlight tab stops sequentially, making tab order identification effortless. Users can review the results, mark any overlooked interactive elements, and generate detailed keyboard accessibility reports. -- [Label in Name (2.5.3)](https://www.testmuai.com/support/docs/accessibility-web-rule-2-5-3-label-in-name.md): Rule-level Accessibility guidance for Label in Name (WCAG 2.5.3 Level A). -- [Labels or Instructions (3.3.2)](https://www.testmuai.com/support/docs/accessibility-web-rule-3-3-2-labels-or-instructions.md): Rule-level Accessibility guidance for Labels or Instructions (WCAG 3.3.2 Level A). -- [Lambda Hooks For Appium Automation](https://www.testmuai.com/support/docs/appium-testmu-hooks.md): Now you can add custom status & remark to your App Automation scripts that you run on TestMu AI. -- [Lambda Hooks for Selenium Automation](https://www.testmuai.com/support/docs/lambda-hooks.md): Use Lambda Hooks to modify test status, download files, throttle networks, and manage sessions in Selenium scripts. -- [LambdaMax XCode Plugin](https://www.testmuai.com/support/docs/xcode-plugin-testmu.md): TestMu AI XCode plugin to run automated tests directly on the TestMu AI grid. -- [Languages and Frameworks supported on HyperExecute](https://www.testmuai.com/support/docs/hyperexecute-supported-languages-and-frameworks.md): HyperExecute supports languages and frameworks that are compatible with Selenium -- [Layout Comparison in SmartUI SDK](https://www.testmuai.com/support/docs/smartui-layout-testing.md): Learn how to use SmartUI SDK's layout testing feature to verify only the layout structure of your pages while ignoring content and style changes -- [Legacy Binary For Linux](https://www.testmuai.com/support/docs/local-testing-for-linux.md): This TestMu AI documentation will help you perform Localhost testing of your private web app or websites on your Linux machine through TestMu AI tunnel. -- [Legacy Binary For macOS](https://www.testmuai.com/support/docs/local-testing-for-macos.md): This document will help you modify your open file limit temporarily or permanently while using a proxy with TestMu AI for seamless cross browser testing. -- [Legacy Binary For Windows](https://www.testmuai.com/support/docs/local-testing-for-windows.md): TestMu AI Tunnel feature allows you to test your private server URLs or locally hosted web-apps or websites through TestMu AI cloud servers. -- [Leveraging Custom Libraries in SikuliX for Advanced Visual Verification](https://www.testmuai.com/support/docs/hyperexecute-utilizing-custom-libraries.md): Leveraging Custom Libraries in SikuliX for Advanced Visual Verification -- [Linear App Integration](https://www.testmuai.com/support/docs/linear-app-integration.md): With TestMu AI and Linear App integration, you can perform one-click bug logging to your Linear App projects and capture screenshots, annotate issues, and share them across your teammates and colleagues. -- [Link Azure DevOps Issues with Test Manager](https://www.testmuai.com/support/docs/link-ado-issues-with-test-manager.md): Effortlessly link Azure DevOps issues with Test Manager using TestMu AI to streamline workflows and enhance project management efficiency. -- [Link Jira Issues with Test Manager](https://www.testmuai.com/support/docs/link-jira-issues-with-test-manager.md): Effortlessly link Jira issues with Test Manager using TestMu AI to streamline workflows and enhance project management efficiency. -- [Link Purpose in Context (2.4.4)](https://www.testmuai.com/support/docs/accessibility-web-rule-2-4-4-link-purpose.md): Rule-level Accessibility guidance for Link Purpose in Context (WCAG 2.4.4 Level A). -- [Linking Automated Test Cases to Test Manager](https://www.testmuai.com/support/docs/automated-test-cases-linked-using-capability.md): Link automated tests to Test Manager with Test Case IDs for better traceability. Supported on HyperExecute and Web Automation. Streamline your workflows! -- [List of Supported Languages and Addons w.r.t. Operating System](https://www.testmuai.com/support/docs/hyperexecute-supported-languages-and-packages.md): This guide lists all the supported languages, tools, packages and frameworks along with their versions that are compatible with different supported operating system - windows, linux and macOS -- [List of TestMu AI Cypress CLI Commands](https://www.testmuai.com/support/docs/cypress-cli-commands.md): List of all the commands and arguments supported by TestMu AI-Cypress CLI. -- [Live Debug Your App Automation Tests](https://www.testmuai.com/support/docs/live-debug-in-app-automation.md): Optimize your app automation with TestMu AI's Live Debugger. Enjoy live debugging and direct interaction with real devices from the app automation dashboard. -- [Live Interaction](https://www.testmuai.com/support/docs/live-interaction.md): Take manual control of an automation session to debug exceptions and interact with the browser in real time. -- [Local Testing For Linux](https://www.testmuai.com/support/docs/local-testing-linux.md): This documentation will help you perform Localhost testing on Linux machine via TestMu AI tunnel .Test private server URLs, locally hosted web-apps and websites on 3000+ real browsers on secure TestMu AI cloud platform. -- [Local Testing For MacOS](https://www.testmuai.com/support/docs/local-testing-macos.md): This documentation will help you perform Localhost testing on macOS via TestMu AI tunnel .Test private server URLs, locally hosted web-apps and websites on 3000+ real browsers on secure TestMu AI cloud infrastructure. -- [Local Testing For Windows](https://www.testmuai.com/support/docs/local-testing-windows.md): This documentation will help you perform Localhost testing on Windows OS via TestMu AI tunnel .Test private server URLs, locally hosted web-apps and websites on 3000+ real browsers on secure TestMu AI cloud infrastructure. -- [Localhost Testing With Docker Tunnel](https://www.testmuai.com/support/docs/docker-tunnel.md): Now you can use Docker image to perform live interactive and automated cross-browser testing of your locally hosted webpage on the TestMu AI platform. -- [Localhost Testing With Shared Tunnel](https://www.testmuai.com/support/docs/sharing-lambda-tunnel.md): Share your SSH connection of TestMu AI Tunnel with teammates belonging to your organization and test your locally hosted web applications or web pages faster. -- [Locally Hosted Pages](https://www.testmuai.com/support/docs/real-time-locally-hosted-pages.md): Securely test your locally hosted websites and application using the real time testing feature of TestMu AI with over 3000+ browsers and different OS. -- [LocalStack Integration With TestMu AI](https://www.testmuai.com/support/docs/localstack-integration-with-testmu.md): LocalStack is a tool that acts as a drop-in replacement for AWS on your local machine, understand the how LocalStack integration with TestMu AI will help you. -- [localStorage Assertions](https://www.testmuai.com/support/docs/kane-cli-checkpoint-devtools-localstorage.md): Verify key-value pairs stored in the browser localStorage during test execution: auth tokens, feature flags, cached data. -- [Logging Contextual Information Using Command Annotations](https://www.testmuai.com/support/docs/command-annotations.md): Send custom logs from your test scripts to the TestMu AI dashboard using command annotations. -- [Login & Authentication for Scheduled Scans](https://www.testmuai.com/support/docs/accessibility-test-scheduling-login-authentication.md): Create reusable login configurations from the scheduler or dashboard. Choose Basic, Form, or Multi-page authentication and attach them to scheduled accessibility scans. -- [Login to Google account on Android](https://www.testmuai.com/support/docs/login-to-google-account-on-android.md): Learn how to login to Google Account on an Android device for App automation to test features or services that are linked to your Google Account. -- [LT Browser - Test Website For Responsiveness Easily](https://www.testmuai.com/support/docs/lt-browser.md): LT Browser allows you to ensure your website's responsiveness over 50+ device viewports. Open your website in the LT Browser and perform live testing now! -- [LT Browser 2.0 - Test Website For Responsiveness Easily](https://www.testmuai.com/support/docs/lt-browser-2.md): Built-on Chromium, LT Browser 2.0 enables you test website's responsiveness on over 50+ device viewports. Open your website in the LT Browser and perform live testing now! -- [Mainframe Testing with HyperExecute](https://www.testmuai.com/support/docs/mainframe-testing-with-hyperexecute.md): Run a Robot Framework mainframe 3270 suite on HyperExecute. Install s3270 and x3270 emulators on the VM, start mock hosts as background services, autosplit suites across parallel Linux machines, and record the x3270 green screen. -- [Maintaining the Suite as Sources Change](https://www.testmuai.com/support/docs/kane-cli-assurance-maintain.md): Keep tests aligned with changing requirements - kane-cli maintain reconcile triages one changed source into an ADD/MODIFY/ARCHIVE plan you approve card by card, and maintain evolve re-designs stale use-cases. -- [Manage AI Capabilities](https://www.testmuai.com/support/docs/manage-ai-capabilities.md): Learn how to manage, enable, or disable AI-powered features across your organization using the AI Capabilities toggle in TestMu AI Organization Settings. -- [Manage Project Level Secrets in HyperExecute](https://www.testmuai.com/support/docs/hyperexecute-how-to-manage-project-level-secrets.md): Learn how to manage project-level secrets in HyperExecute, scope credentials per project, restrict access, and keep sensitive data secure. -- [Manage SSO Connection](https://www.testmuai.com/support/docs/testmu-sso-manage-connection.md): This document will help you set up and manage your TestMu AI SSO connection. -- [Manage Subscriptions](https://www.testmuai.com/support/docs/manage-subscriptions.md): Learn how to manage your subscriptions by successfully upgrading to a new plan, switching between plans, and switching from Monthly to Annual plan on TestMu AI. -- [Manage User-Level Secrets in HyperExecute](https://www.testmuai.com/support/docs/hyperexecute-how-to-save-and-manage-secrets.md): Securely save and manage user-level secrets in HyperExecute. Encrypt credentials, rotate keys, and use them safely across automation runs. -- [Managing Automated Tests with Test Manager](https://www.testmuai.com/support/docs/automated-test-cases-with-ai.md): Streamline testing with Test Manager! Manage automated tests, auto-generate BDD scenarios, and link automation runs seamlessly for Selenium Web frameworks. -- [Mantis Integration](https://www.testmuai.com/support/docs/mantis-integration.md): TestMu AI helps you manage your bugs in Mantis by moving them to project in a single click. All the details you provide in TestMu AI like projects, severity, priority, summary and description would automatically be presented in the project on Mantis. -- [Manual App Profiling](https://www.testmuai.com/support/docs/manual-app-profiling.md): Profile your app's performance during live manual testing sessions on real devices. Monitor CPU, Memory, Disk, FPS, Network, Battery, ANR, and Startup Time in real-time. -- [Manual Interaction Using KaneAI](https://www.testmuai.com/support/docs/kaneai-manual-interaction-video.md): Discover how KaneAI streamlines test automation by enabling manual interactions to create natural language prompts, analyze test steps, and generate code in multiple programming languages. -- [Manual Linking of Automation Runs with Test Manager](https://www.testmuai.com/support/docs/automated-test-cases-linked-using-dashboard.md): Effortlessly link test cases manually with Test Manager. Create, link, or raise bugs directly, and sync issues with Jira for streamlined test management. -- [Manual Test Cases with AI](https://www.testmuai.com/support/docs/manual-test-cases-with-ai.md): Enhance your manual test cases with AI from TestMu AI's Test Manager, facilitating efficient management of all testing activities on a single platform. -- [Manual Tests Sessions Dashboard](https://www.testmuai.com/support/docs/manual-tests-dashboard.md): View, search, filter, and analyze all your manual testing sessions from Real Devices App and Browser testing in one unified dashboard. -- [Mark as Bug in Automation Testing](https://www.testmuai.com/support/docs/mark-as-bug-in-automation-testing.md): Log bugs from automation test sessions directly to Jira, Trello, and other project management tools. -- [Mark As Bug In Real Time Testing](https://www.testmuai.com/support/docs/mark-as-bug.md): Mark your bugs while performing Real Time testing on TestMu AI. Just a single click and you can share bugs to your favourite project management tools like Jira, Asana, Trello etc. -- [Mark As Bug In Screenshot Testing](https://www.testmuai.com/support/docs/mark-as-bug-in-screenshot-testing.md): Don't forget to mark your bugs while performing Screenshot testing at TestMu AI. Just a single click and you can share bugs to your favourite project management tools such as Asana, Trello, Jira etc. -- [Mask Your Sensitive Data in Selenium Tests](https://www.testmuai.com/support/docs/selenium-mask-your-data.md): Mask sensitive data like usernames and access keys in TestMu AI platform logs using the maskCommands capability. -- [Matrix Multiplexing Strategy](https://www.testmuai.com/support/docs/hyperexecute-matrix-multiplexing-strategy.md): Discover the power of HyperExecute's Matrix Multiplexing Strategy for efficient cross-browser testing. Optimize test combinations for faster results.. -- [Maven Tunnel For Locally Hosted Web Applications](https://www.testmuai.com/support/docs/maven-tunnel-for-locally-hosted-web-applications.md): Configure a secure TestMu AI Tunnel to execute your Selenium test automation scripts through your Maven project on our online Selenium Grid. -- [Maximize Test Execution Performance on Automation Platform](https://www.testmuai.com/support/docs/performance-tips.md): Speed up Selenium test execution on the cloud grid by disabling optional logging capabilities. -- [Media Injection on Real Devices](https://www.testmuai.com/support/docs/camera-image-injection-on-real-devices.md): Test camera features on real devices with TestMu AI's Image Injection. Easily inject custom images for QR codes, photos, and more. -- [Microsoft Teams App Integration](https://www.testmuai.com/support/docs/microsoft-teams-app.md): This documentation will help you integrate Microsoft Teams App with TestMu AI to run the real-time and screenshot tests directly from any personal chat or team channel. -- [Microsoft Teams Integration](https://www.testmuai.com/support/docs/microsoft-teams-integration.md): TestMu AI integration with Microsoft Teams allows you to push a bug directly to your specified Teams channel from TestMu AI platform. Share your UI observations and input with your teammates on any time, by capturing a screenshot in the middle of your test session through TestMu AI. You can annotate the screenshot & highlight your issue or input. The fields populated by you when marking as a bug through TestMu AI are displayed as information on the respective Teams channel for that testing instance. -- [Migrate Appium Tests](https://www.testmuai.com/support/docs/migrate-appium-tests-from-browserstack-and-saucelabs.md): This documentation will guide you through the process of migrating your Appium test suites from BrowserStack and SauceLabs to TestMu AI. -- [Migrate Appium Tests From Local Grid](https://www.testmuai.com/support/docs/migrate-existing-appium-test-suites.md): This documentation will guide you through the process of migrating your existing Appium test suites from your local grid setup or another cloud testing tools to run on TestMu AI. -- [Migrate Appium Tests From Sauce Labs](https://www.testmuai.com/support/docs/migrate-appium-tests-from-sauce-labs.md): This documentation will guide you through the process of migrating your Appium test suites from Sauce Labs to TestMu AI. -- [Migrate Existing Playwright Test Suites On TestMu AI](https://www.testmuai.com/support/docs/migrate-existing-playwright-tests.md): Learn how to run migrate existing Playwright test scripts or suitesfrom your local grid on the TestMu AI platform. -- [Migrate Your Selenium Test Suite](https://www.testmuai.com/support/docs/migrate-selenium-test-suite.md): Migrate your local Selenium tests to TestMu AI's cloud grid. Change the hub URL, add capabilities, and run your existing suite on 3000+ browsers. -- [Milestones - Creation and Management](https://www.testmuai.com/support/docs/milestone-creation-and-management.md): Check the milestones - datasets, test run and -- [Miro Integration with TestMu AI](https://www.testmuai.com/support/docs/miro-integration.md): TestMu AI's integration with Miro will help you to log bugs to boards on Miro. You can easily find and report any cross browser compatibility issues to your team instantly. -- [Mismatch Thresholds for SmartUI Visual Regression Testing](https://www.testmuai.com/support/docs/smartui-mismatch-thresholds.md): Learn how to configure project-level, build-level, and screenshot-level mismatch thresholds for SmartUI Visual Regression testing to control comparison sensitivity. -- [Mismatched Label Text](https://www.testmuai.com/support/docs/accessibility-android-rule-mismatched-label-text.md): Rule-level Accessibility guidance for Mismatched Label Text on Android. -- [Misplaced Field Label](https://www.testmuai.com/support/docs/accessibility-android-rule-misplaced-field-label.md): Rule-level Accessibility guidance for Misplaced Field Label on Android. -- [Missing Accessibility Labels](https://www.testmuai.com/support/docs/accessibility-ios-rule-missing-accessibility-labels.md): Rule-level Accessibility guidance for Missing Accessibility Labels. -- [Missing Button Element Label](https://www.testmuai.com/support/docs/accessibility-ios-rule-missing-button-element-label.md): Rule-level Accessibility guidance for Missing Button Element Label on iOS. -- [Missing Checkbox Element Label](https://www.testmuai.com/support/docs/accessibility-ios-rule-missing-checkbox-element-label.md): Rule-level Accessibility guidance for Missing Checkbox Element Label on iOS. -- [Missing Editable Element Label](https://www.testmuai.com/support/docs/accessibility-ios-rule-missing-editable-element-label.md): Rule-level Accessibility guidance for Missing Editable Element Label on iOS. -- [Missing Field Label](https://www.testmuai.com/support/docs/accessibility-android-rule-missing-field-label.md): Rule-level Accessibility guidance for Missing Field Label. -- [Missing Image Alt](https://www.testmuai.com/support/docs/accessibility-android-rule-missing-image-alt.md): Rule-level Accessibility guidance for Missing Image Alt. -- [Missing Image Element Label](https://www.testmuai.com/support/docs/accessibility-ios-rule-missing-image-element-label.md): Rule-level Accessibility guidance for Missing Image Element Label on iOS. -- [Missing Input Value](https://www.testmuai.com/support/docs/accessibility-android-rule-missing-input-value.md): Rule-level Accessibility guidance for Missing Input Value on Android. -- [Missing Screen Title](https://www.testmuai.com/support/docs/accessibility-android-rule-missing-screen-title.md): Rule-level Accessibility guidance for Missing Screen Title on Android. -- [Missing Switch Element Label](https://www.testmuai.com/support/docs/accessibility-ios-rule-missing-switch-element-label.md): Rule-level Accessibility guidance for Missing Switch Element Label on iOS. -- [Missing View Accessibility](https://www.testmuai.com/support/docs/accessibility-android-rule-missing-view-accessibility.md): Rule-level Accessibility guidance for Missing View Accessibility on Android. -- [Mobile App Accessibility Testing](https://www.testmuai.com/support/docs/kaneai-mobile-app-accessibility.md): Learn how to perform automated accessibility testing for native mobile applications using KaneAI's accessibility scan feature. -- [Mobile App advanced authoring capabilities](https://www.testmuai.com/support/docs/kane-ai-mobile-app-capabilities.md): Learn how to use advanced capabilities in the KaneAI authoring for mobile app tests -- [Mobile App Testing Patterns](https://www.testmuai.com/support/docs/kaneai-kb-mobile-app-testing-patterns.md): Best practices for authoring reliable native mobile app tests in KaneAI covering Android and iOS patterns, OTP fields, keyboards, pickers, and mobile-specific interactions -- [Mobile Web Automation Testing on Real Devices](https://www.testmuai.com/support/docs/mobile-web-automation-on-real-devices.md): Run mobile web automation tests via Appium on real Android and iOS devices from Samsung, Apple, and more. -- [Mock Video Injection on Web Automation (Selenium/Chromium)](https://www.testmuai.com/support/docs/mock-video-injection-web-automation-selenium-chromium.md): Inject a custom video file as a fake camera feed in Chromium-based Selenium tests for WebRTC and KYC workflows. -- [Modes of Operation](https://www.testmuai.com/support/docs/kane-cli-modes.md): Kane CLI has three modes: Interactive TUI for development, Headless CLI for scripts and CI/CD, and Agent Mode for AI coding agent integrations. -- [Module Versioning and Enhancements](https://www.testmuai.com/support/docs/kaneai-modules-versions-and-enhancement.md): Learn how module versioning works in KaneAI, track changes, compare versions, and revert to previous versions of your test modules. -- [Modules](https://www.testmuai.com/support/docs/kane-ai-modules.md): Learn how to create, manage, and reuse modules in KaneAI to streamline your test automation workflows across platforms. -- [Modules in Manual Testcases](https://www.testmuai.com/support/docs/modules-in-manual-testcases.md): Guide for linking Modules with Manual Testcases in Test Manager. -- [monday.com Integration](https://www.testmuai.com/support/docs/monday-com-integration.md): LTestMu AI helps you manage your bugs in monday.com by moving them to project in a single click. You'll get all the details like task list, assignee, and title in you monday project. -- [Motion Actuation (2.5.4)](https://www.testmuai.com/support/docs/accessibility-web-rule-2-5-4-motion-actuation.md): Rule-level Accessibility guidance for Motion Actuation (WCAG 2.5.4 Level A). -- [Multi Page Scanner](https://www.testmuai.com/support/docs/accessibility-testing-multi-page-scanner.md): Multi-Page Scanner scans across multiple web pages for a comprehensive overview of potential issues and prioritizes fixes impacting the user experience. -- [Multi Reporters Support](https://www.testmuai.com/support/docs/cyp-multi-reporters.md): Learn how to generate multiple reporters supported by Cypress. -- [Multi-Device Testing on Real Devices](https://www.testmuai.com/support/docs/multi-device-testing.md): Learn how to test multiple devices simultaneously in a single session on TestMu AI App Live, including setup, workflow, and best practices. -- [Multi-Factor Authentication](https://www.testmuai.com/support/docs/multi-factor-authentication.md): This documentation will help you enable and manage multi-factor authentication for your account. -- [n8n Integration With TestMu AI for Browser Automation](https://www.testmuai.com/support/docs/n8n-integration.md): Install the TestMu AI Agent node in n8n to automate browser actions across 3,000+ real environments. No driver setup or local infrastructure required. -- [Name, Role, Value (4.1.2)](https://www.testmuai.com/support/docs/accessibility-web-rule-4-1-2-name-role-value.md): Rule-level Accessibility guidance for Name, Role, Value (WCAG 4.1.2 Level A). -- [Native App Automation Appium (Overview)](https://www.testmuai.com/support/docs/accessibility-native-app-automation-test.md): Appium-based native app accessibility automation with lambda-accessibility-scan, capabilities, and dashboard reports. -- [Native App Automation with MCP](https://www.testmuai.com/support/docs/native-app-automation-mcp.md): Automate and debug native mobile app tests through MCP, using the TestMu AI MCP Server for cloud test triage and the Appium MCP server for live device control. -- [Native Extent Report](https://www.testmuai.com/support/docs/native-extent-report.md): Learn how to generate Native Extent Report on TestMu AI and download the reports from the dashboard -- [Native TestNG Reports on HyperExecute](https://www.testmuai.com/support/docs/hyperexecute-native-testng-report.md): Learn how to generate Native TestNG test report for HyperExecute on TestMu AI and download the reports from the dashboard -- [Navigating the Dashboard](https://www.testmuai.com/support/docs/accessibility-testing-navigating-dashboard.md): Learn how to open, filter, review, and act on Accessibility reports in the dashboard. -- [Nested Control Issues](https://www.testmuai.com/support/docs/accessibility-android-rule-nested-control-issues.md): Rule-level Accessibility guidance for Nested Control Issues on Android. -- [Netlify Integration With TestMu AI](https://www.testmuai.com/support/docs/netlify-integration-with-testmu.md): Simplify DevOps! Run blazing-fast Cypress tests on TestMu AI directly from your Netlify builds. -- [Network Assertions](https://www.testmuai.com/support/docs/kane-cli-checkpoint-devtools-network.md): Verify HTTP traffic (API responses, status codes, headers, response bodies, and request timing) captured by KaneAI in the background. -- [Network Configurations in App Live](https://www.testmuai.com/support/docs/network-configurations-in-real-devices.md): Learn how to configure and capture HTTP/S traffic on real devices during manual testing in App Live & Browser Live, with options for content capture and domain filtering. -- [Network Configurations in Automation Tests](https://www.testmuai.com/support/docs/network-configurations.md): Learn how to configure and capture HTTP/S traffic during automation testing on TestMu AI Real Devices. -- [Network Configurations in Browser Live](https://www.testmuai.com/support/docs/browser-network-configurations.md): Learn how to configure and capture HTTP/S traffic on real devices during manual testing in Browser Live, with options for content capture and domain filtering. -- [Network Data Masking for Selenium Tests](https://www.testmuai.com/support/docs/network-data-masking.md): Mask sensitive data like passwords, tokens, and API keys in network logs during automation testing sessions. -- [Network Logs Assertions](https://www.testmuai.com/support/docs/kane-ai-network-assertions.md): In this detailed guide, you will learn how you can add network logs assertions in your tests. -- [Network Throttling](https://www.testmuai.com/support/docs/kaneai-network-throttling.md): Guide for usage of Network Throttling feature. -- [Network Throttling for Appium tests](https://www.testmuai.com/support/docs/app-auto-network-throttling.md): Now validate your mobile applications over low latency networks (2G/3G/LTE) or in offline mode with varying upload and download speeds. TestMu AI empowers you to simulate these mobile network conditions through its desired capabilities, ensuring comprehensive testing of your mobile applications. -- [Network Throttling for Selenium Tests](https://www.testmuai.com/support/docs/network-throttling.md): Simulate low latency networks like 2G, 3G, LTE, or offline in Selenium tests using network throttling capabilities. -- [Network Throttling To Test Website Responsiveness](https://www.testmuai.com/support/docs/network-throttling-for-responsive-testing.md): LT Browser allows you to test your mobile website view over different networks such as fast 3G, slow 3G, online, or even offline with network throttling. -- [Network Whitelisting Guide](https://www.testmuai.com/support/docs/network-whitelisting-and-tunnel-guide.md): Follow the instructions in this documentation, to understand the difference between the network and the ip whitelisting. -- [New Relic Integration With TestMu AI](https://www.testmuai.com/support/docs/new-relic-integration.md): View your TestMu AI test results in a custom New Relic dashboard. Gain deep insights into your automation scripts and identify gaps to ship quality products at light speed. -- [No Keyboard Trap (2.1.2)](https://www.testmuai.com/support/docs/accessibility-web-rule-2-1-2-no-keyboard-trap.md): Rule-level Accessibility guidance for No Keyboard Trap (WCAG 2.1.2 Level A). -- [Non-accessible Interaction](https://www.testmuai.com/support/docs/accessibility-android-rule-non-accessible-interaction.md): Rule-level Accessibility guidance for Non-accessible Interaction on Android. -- [Non-text Content (1.1.1)](https://www.testmuai.com/support/docs/accessibility-web-rule-1-1-1-non-text-content.md): Rule-level Accessibility guidance for Non-text Content (1.1.1). -- [Non-text Contrast (1.4.11)](https://www.testmuai.com/support/docs/accessibility-web-rule-1-4-11-non-text-contrast.md): Rule-level Accessibility guidance for Non-text Contrast (WCAG 1.4.11 Level AA). -- [Notion Integration](https://www.testmuai.com/support/docs/notion-integration.md): Perform one-click bug logging while performing cross browser testing with TestMu AI and Notion integration. Capture screenshots, annotate issues, and share them directly from the TestMu AI platform to your Notion pages. -- [NUnit (C#)](https://www.testmuai.com/support/docs/accessibility-nunit-csharp-test.md): Run Accessibility Automation with Selenium C# and NUnit: RemoteWebDriver capabilities, test lifecycle, and reports. -- [NVDA on Windows](https://www.testmuai.com/support/docs/accessibility-nvda-windows.md): Platform-specific screen reader reference for NVDA on Windows: setup, navigation basics, and what to validate after automated scans. -- [Okta Scim User Provisioning](https://www.testmuai.com/support/docs/okta.md): Integrating TestMu AI SCIM with Okta -- [On Focus (3.2.1)](https://www.testmuai.com/support/docs/accessibility-web-rule-3-2-1-on-focus.md): Rule-level Accessibility guidance for On Focus (WCAG 3.2.1 Level A). -- [On Input (3.2.2)](https://www.testmuai.com/support/docs/accessibility-web-rule-3-2-2-on-input.md): Rule-level Accessibility guidance for On Input (WCAG 3.2.2 Level A). -- [One Click Migration from qTest](https://www.testmuai.com/support/docs/one-click-migration-from-qtest.md): Migrate your test cases, folder structures, custom fields, attachments, and linked Jira requirements from qTest to TestMu AI Test Manager in a single click. -- [One Click Migration from TestRail](https://www.testmuai.com/support/docs/one-click-migration-from-testrail.md): Effortlessly migrate from TestRail to TestMu AI with our one-click migration guide. Simplify the process and enhance your testing capabilities in minutes. -- [One Click Migration from X-Ray](https://www.testmuai.com/support/docs/one-click-migration-from-xray.md): Migrate your test cases, attachments, and custom fields from X-Ray (Jira Cloud) to TestMu AI Test Manager in a single click. -- [One Click Migration from Zephyr Scale](https://www.testmuai.com/support/docs/one-click-migration-from-zephyr-scale.md): Migrate your test cases, folders, embedded screenshots, and custom fields from Zephyr Scale (Jira Cloud) to TestMu AI Test Manager in a single click. -- [OpsGenie Integration](https://www.testmuai.com/support/docs/opsgenie-integration.md): You can log incidents with a single click when performing cross-browser testing using TestMu AI and OpsGenie integration. Capture screenshots, pinpoint errors, and send them straight to OpsGenie from the TestMu AI platform. -- [Orientation (1.3.4)](https://www.testmuai.com/support/docs/accessibility-web-rule-1-3-4-orientation.md): Rule-level Accessibility guidance for Orientation (WCAG 1.3.4 Level AA). -- [Outbound Phone Agent Testing With TestMu AI](https://www.testmuai.com/support/docs/outbound-phone-agent.md): Test an outbound phone agent with TestMu AI. Outbound number pools, passive-mode monitoring, and the same 30+ call quality metrics as inbound testing. -- [Page Titled (2.4.2)](https://www.testmuai.com/support/docs/accessibility-web-rule-2-4-2-page-titled.md): Rule-level Accessibility guidance for Page Titled (WCAG 2.4.2 Level A). -- [PagerDuty Integration](https://www.testmuai.com/support/docs/pagerduty-integration.md): You can log defects with a single click when performing cross-browser testing using TestMu AI and PagerDuty integration. Capture screenshots, pinpoint errors, and send them straight to PagerDuty from the TestMu AI platform. -- [Parallel Test Execution](https://www.testmuai.com/support/docs/kane-cli-parallel-execution.md): Run multiple independent Kane CLI browser tests in parallel using shell background processes or AI agent sub-tasks. -- [Parsing (4.1.1)](https://www.testmuai.com/support/docs/accessibility-web-rule-4-1-1-parsing.md): Rule-level Accessibility guidance for Parsing (WCAG 4.1.1 Level A). Note that this criterion is obsolete in WCAG 2.2 but may still appear in scans against earlier WCAG versions. -- [Partial Page Scanner](https://www.testmuai.com/support/docs/accessibility-testing-partial-page-scanner.md): Partial Page Scanner scans specific webpage elements, tests components, investigates reported issues and evaluates changes made to a specific area. -- [Passed Test Cases](https://www.testmuai.com/support/docs/accessibility-passed-test-cases.md): Show passing accessibility checks alongside violations so teams can track both achievements and remaining gaps. -- [Password and Access Key Expiration Policy](https://www.testmuai.com/support/docs/password-and-access-key-expiration-policy.md): Guide to Manage your Password and Access Key Expiration Policies. -- [Pause, Stop, Hide (2.2.2)](https://www.testmuai.com/support/docs/accessibility-web-rule-2-2-2-pause-stop-hide.md): Rule-level Accessibility guidance for Pause, Stop, Hide (WCAG 2.2.2 Level A). -- [Paymo Integration](https://www.testmuai.com/support/docs/paymo-integration.md): TestMu AI helps you manage your bugs in Paymo by moving them to project in a single click with details like task list, assignee, title and description in Paymo project. -- [PDF Accessibility Scanning](https://www.testmuai.com/support/docs/accessibility-pdf-accessibility-scanning.md): Overview of PDF accessibility scanning guidance and how to position it alongside other Accessibility workflows. -- [Perform Automated Screenshot Testing](https://www.testmuai.com/support/docs/automated-screenshot-testing.md): Effortlessly Master Automated Screenshot Testing with TestMu AI | Step-by-Step Instructions for Screenshot Testing, Downloads, and More! -- [Perform Operations on your Application via API](https://www.testmuai.com/support/docs/application-setup-via-api.md): This guide will explain how to perform operations with your applications via api for real and virtual devices. -- [Perform Operations on your Application via GUI](https://www.testmuai.com/support/docs/application-setup-via-gui.md): Explore how to perform operations with your applications like uploading, deleting via gui for real and virtual devices. -- [Performance Assertions](https://www.testmuai.com/support/docs/kane-cli-checkpoint-devtools-performance.md): Verify Core Web Vitals and other performance metrics (LCP, CLS, INP, FCP, TTFB) captured during test execution. -- [Performance Report](https://www.testmuai.com/support/docs/performance-report.md): LT Browser integrates with Google Lighthouse that allows you to view the performance report of your web-page instantly. Along with testing the website's responsiveness, you can now test your website performance statistics across 50+ pre-installed device viewports. -- [Performance Testing with HyperExecute](https://www.testmuai.com/support/docs/hyperexecute-performance-testing.md): Prevent app launch disaster! Test app performance under load with HyperExecute. Get real-time insights, identify weaknesses & ensure smooth launch. -- [Phone Agent Testing With TestMu AI](https://www.testmuai.com/support/docs/phone-agent.md): Test phone voice agents on TestMu AI with real calls. Live call monitoring, recording analysis, voice and noise simulation, and 30+ call quality metrics. -- [PingOne SCIM User & Group Provisioning](https://www.testmuai.com/support/docs/pingone-scim.md): Integrating TestMu AI SCIM with PingOne (PingIdentity) -- [Platform FAQs](https://www.testmuai.com/support/docs/hyperexecute-platform-faqs.md): Find answers to frequently asked questions about the HyperExecute platform, covering OS support, available browsers, customization options, and more for efficient testing. -- [Playwright Bundled Browser Support](https://www.testmuai.com/support/docs/playwright-bundled-browser-support.md): List of all the Playwright Bundled Browser Support -- [Playwright Integration with HyperExecute](https://www.testmuai.com/support/docs/playwright-integration-with-hyperexecute.md): You can now integrate playwright with HyperExecute. To do so, follow the steps listed in the document. -- [Playwright SDK](https://www.testmuai.com/support/docs/playwright-sdk.md): A guide to getting started with the Playwright-Node.js SDK by TestMu AI. -- [Playwright Test Execution Setup](https://www.testmuai.com/support/docs/playwright-test-execution-setup.md): Learn how to configure the desired capability for selecting browsers and OS, organzing tests, changing desktop resolution, and more for your Playwright tests. -- [Playwright Webview Testing](https://www.testmuai.com/support/docs/playwright-webview-test.md): Learn how to perform webview testing on android using playwright framework on TestMu AI platform with 3000+ browsers cross platform. -- [Plugins and Extensions](https://www.testmuai.com/support/docs/plugins-and-extensions.md): TestMu AI offers you plugins with various third-party application to help you perform cross browser testing on 3000+ real browsers with pace and convenience. -- [Pointer Cancellation (2.5.2)](https://www.testmuai.com/support/docs/accessibility-web-rule-2-5-2-pointer-cancellation.md): Rule-level Accessibility guidance for Pointer Cancellation (WCAG 2.5.2 Level A). -- [Pointer Gestures (2.5.1)](https://www.testmuai.com/support/docs/accessibility-web-rule-2-5-1-pointer-gestures.md): Rule-level Accessibility guidance for Pointer Gestures (WCAG 2.5.1 Level A). -- [PractiTest Integration with TestMu AI](https://www.testmuai.com/support/docs/practitest-integration.md): TestMu AI provides integrations with PractiTest to help you log bugs directly from the middle of your test session on TestMu AI to your PractiTest. -- [Pre Steps Errors](https://www.testmuai.com/support/docs/hyperexecute-pre-steps-error.md): This document will help you debug your failed job if you are facing any errors in the pre-command stage. -- [Pre-Loaded Extension for Chrome](https://www.testmuai.com/support/docs/pre-loaded-chrome-extension.md): Learn how to install and manage Chrome extensions during real-time testing on TestMu AI. Enhance your manual testing sessions with ad blockers, debugging tools, and more. -- [Pricing FAQs](https://www.testmuai.com/support/docs/hyperexecute-pricing-faqs.md): Explore answers to all your HyperExecute Pricing-related queries with our comprehensive FAQs, designed to clarify all your cost concerns! -- [Prioritize Your Tests](https://www.testmuai.com/support/docs/hyperexecute-prioritize-tests.md): Learn more about how to prioritize your tests with HyperExecute. -- [Private Desktop Insights](https://www.testmuai.com/support/docs/insights-private-desktop.md): Discover TestMu AI's Private Desktop Insights for comprehensive test cases insights. Optimize your testing efforts today. -- [Private Device Allocation to Teams](https://www.testmuai.com/support/docs/private-device-allocation-to-teams.md): Efficiently manage private cloud devices with TestMu AI's Teams feature. Allocate or deallocate devices to teams for secure and streamlined testing workflows. -- [Private Real Devices Insights](https://www.testmuai.com/support/docs/insights-private-real-devices.md): Discover TestMu AI's Private Real Devices Insights for comprehensive test cases insights. Optimize your testing efforts today. -- [Profiles - TestMu AI Browser Cloud](https://www.testmuai.com/support/docs/browser-cloud-profiles.md): Persist browser state across sessions and script runs. Reuse auth, cookies, and browser settings across sessions. -- [Project Management Tools](https://www.testmuai.com/support/docs/integrations-with-project-management-tools.md): TestMu AI offers integration with third-party tools for project management and bug tracking to help you collaborate with your colleagues as you encounter a bug while cross browser testing. -- [Project Settings](https://www.testmuai.com/support/docs/smartui-project-settings.md): In this documentation, learn about the project settings available for SmartUI Projects in order to customize your project, manage approvals, configure comparison parameters, and more. -- [Projects](https://www.testmuai.com/support/docs/analytics-project-report.md): In-depth guide to the Project Analytics Dashboard, including detailed feature descriptions, widgets, workflows, C-Suite reporting, and best practices for maximizing test insights. -- [Public Cloud VS Private Cloud](https://www.testmuai.com/support/docs/public-cloud-vs-private-cloud.md): Discover how TestMu AI's private real device cloud ensures seamless app and website functionality across devices with tailored, robust testing solutions for accuracy and flexibility. -- [Quality Dimensions and Go-Live Readiness for AI Agents](https://www.testmuai.com/support/docs/ai-agent-testing-platform-overview.md): How the Agent Testing Platform scores AI agent conversations across quality dimensions, applies pass/fail thresholds, and produces a Green, Yellow, or Red go-live verdict. -- [Queuing Appium Automated Tests](https://www.testmuai.com/support/docs/queuing-appium-tests.md): To avoid overuse of TestMu AI , we have put a capacity limit on the number of tests queued on our platform. Here's a brief guide on queuing your Appium automated tests. -- [Quick Actions - Scrape, Screenshot, PDF](https://www.testmuai.com/support/docs/browser-cloud-quick-actions.md): One-liner operations that handle browser setup, navigation, and cleanup for scraping, screenshots, and PDF generation. -- [Quick Start](https://www.testmuai.com/support/docs/kane-cli-quickstart.md): Authenticate Kane CLI and run your first browser automation test in under 5 minutes. -- [Quickstart - TestMu AI Browser Cloud](https://www.testmuai.com/support/docs/launch-first-session.md): Get up and running with your first TestMu AI Browser Cloud session in a few minutes using TypeScript and Puppeteer. -- [Rally Integration with TestMu AI](https://www.testmuai.com/support/docs/rally-integration.md): TestMu AI provides integration with Rally to help you log bugs directly from your test session on TestMu AI to Rally. -- [Re-Run Automation Tests](https://www.testmuai.com/support/docs/rerun-automation-tests.md): Re-run a Selenium test from the TestMu AI Automation Dashboard using Replay to run it again live, or Reproduce to replay it against captured network and DOM data. -- [Real Devices](https://www.testmuai.com/support/docs/real-devices-integration-with-hyperexecute.md): Maximize real device testing with HyperExecute! Accelerate test orchestration and intelligence on real devices with TestMu AI's support. -- [Real Time Testing Troubleshooting](https://www.testmuai.com/support/docs/real-time-testing-troubleshooting.md): Perform real time live interactive cross browser compatibility testing of your website or web application with TestMu AI. -- [Realtime & Real-Device Modules](https://www.testmuai.com/support/docs/analytics-modules-manual-test-overview.md): A complete guide to understanding TestMu AI Analytics! Optimize your testing process with widgets for Browser & App Testing - OS, Browser, Device, and Trends. -- [Realtime Native Mobile App Testing](https://www.testmuai.com/support/docs/live-interactive-mobile-app-testing.md): With TestMu AI, perform live interactive testing of mobile applications on Android emulators and iOS simulators and ensure your apps work seamlessly across multiple versions of Android emulators and iOS simulators. -- [Receive Instant Notification on your Jobs Directly in your Slack](https://www.testmuai.com/support/docs/hyperexecute-slack-integration.md): Streamline testing & communication! Integrate TestMu AI HyperExecute with Slack for real-time test notifications & updates. -- [Receive Job Reports and Artifacts via Email](https://www.testmuai.com/support/docs/hyperexecute-email-reports.md): Streamline testing with HyperExecute Email Reports! Learn how to effortlessly automate test runs and receive detailed reports. Boost efficiency today. -- [Recents and Favourites](https://www.testmuai.com/support/docs/real-time-recents-and-favourites.md): Discover how to use the Recents and Favourites feature under Quick Links in TestMu AI. Quickly access recently used items and mark important resources for faster navigation. -- [Redmine Integration](https://www.testmuai.com/support/docs/redmine-integration.md): Using TestMu AI integration with Redmine, you'll be able to log bugs in your Redmine project from TestMu AI in just one click. You can grab a screenshot of a problem, highlight an exception, annotate additional detail, select an assignee, include a fitting title and a summary to help convey further actions. You can do all of this directly from TestMu AI. -- [Redundant Entry (3.3.7)](https://www.testmuai.com/support/docs/accessibility-web-rule-3-3-7-redundant-entry.md): Rule-level Accessibility guidance for Redundant Entry (WCAG 3.3.7 Level A). -- [Reflow (1.4.10)](https://www.testmuai.com/support/docs/accessibility-web-rule-1-4-10-reflow.md): Rule-level Accessibility guidance for Reflow (WCAG 1.4.10 Level AA). -- [Regular Expression - Appium](https://www.testmuai.com/support/docs/regular-expression-appium.md): If your chosen device for testing on TestMu AI is unavailable, use REGEX to broaden the search request for alternative devices to run the test. -- [Regular Expression - Espresso](https://www.testmuai.com/support/docs/regular-expression-espresso.md): If your chosen device for testing on TestMu AI is unavailable, use REGEX to broaden the search for alternative devices to run espresso test. -- [Regular Expression - XCUI](https://www.testmuai.com/support/docs/regular-expression-xcui.md): There may be instances when the device you chose for a test on TestMu AI is not available. In these scenarios, REGEX will help you widen the search request for devices to run the test on. -- [Rename Slash Command Steps](https://www.testmuai.com/support/docs/kaneai-rename-instructions.md): Learn how to rename test steps in KaneAI for API, JS, Database, Network Assertion and Visual Comparison instructions -- [ReportPortal.io Integration With TestMu AI For Cypress](https://www.testmuai.com/support/docs/report-portal-cypress.md): This article will guide you on how to integrate the LambdatTest platform with Report Portal platform for running your Cypress automation tests. -- [Reports](https://www.testmuai.com/support/docs/tms-reports.md): View & Download the Execution & Traceability reports for Test Instances execution. -- [Rerun Failed Tests on HyperExecute](https://www.testmuai.com/support/docs/hyperexecute-rerun-failed-tests.md): Speed up debugging by rerunning only failed tests or entire jobs on HyperExecute using the rerun job feature. -- [Reusing Context & Auth - TestMu AI Browser Cloud](https://www.testmuai.com/support/docs/browser-cloud-context.md): Extract and inject browser state across sessions to preserve login and user data in TestMu AI Browser Cloud. -- [RKTracer Integration](https://www.testmuai.com/support/docs/rktracer-integration.md): TestMu AI and RKTracer integration allows you to generate Code coverage for your web and mobile apps across 3000+ real browsers, devices, and operating system combinations. -- [Robot Framework](https://www.testmuai.com/support/docs/accessibility-robot-framework-test.md): Run Accessibility Automation when Robot Framework drives Selenium: variables, Open Browser keywords, hooks, and reports. -- [Robot Report](https://www.testmuai.com/support/docs/robot-report.md): Learn how to configure and generate Robot test execution reports on HyperExecute with step-by-step guidance for seamless test insights and reporting. -- [Rocket.Chat Integration With TestMu AI](https://www.testmuai.com/support/docs/rocketchat-integration-with-testmu.md): The TestMu AI Rocket.Chat Integration allows you to share or push a bug to your specified Rocket.Chat channel from the TestMu AI platform in a single click. You can share any UI observations and input with your team on Rocket.Chat anytime, by just capturing a screenshot in the middle of your test session through TestMu AI. You can also annotate the screenshot or highlight your issue or input. The fields populated by you while marking as bug through TestMu AI gets displayed as information on Rocket.Chat for that testing instance. -- [Roku TV With Appium](https://www.testmuai.com/support/docs/appium-rokutv.md): Complete guide to running your first RokuTV Appium automated test script on TestMu AI Real Device Cloud Platform. Test on 3000+ Real Devices. -- [Roles and Permissions (RBAC)](https://www.testmuai.com/support/docs/rbac-roles-and-permissions.md): Learn how to use Custom Roles and Permissions (RBAC) on LambdaTest to control product-level access and granular permissions across your organization. -- [Run automation tests on HyperExecute using Behave](https://www.testmuai.com/support/docs/behave-on-hyperexecute-grid.md): Learn how to run Selenium automation tests on HyperExecute using the Behave framework -- [Run automation tests on HyperExecute using Capybara](https://www.testmuai.com/support/docs/capybara-on-hyperexecute-grid.md): Learn how to run Selenium automation tests on HyperExecute using Capybara -- [Run automation tests on HyperExecute using cucumber](https://www.testmuai.com/support/docs/cucumber-on-hyperexecute-grid.md): Learn how to run Selenium automation tests on HyperExecute using the Cucumber framework -- [Run Automation Tests on HyperExecute using Cypress v10](https://www.testmuai.com/support/docs/cypressv10-on-hyperexecute.md): Learn how to run playwright automation tests on HyperExecute using Cypress v10 framework -- [Run automation tests on HyperExecute using Cypress v9](https://www.testmuai.com/support/docs/cypressv9-on-hyperexecute.md): Learn how to run playwright automation tests on HyperExecute using Cypress v9 framework -- [Run automation tests on HyperExecute using JUnit](https://www.testmuai.com/support/docs/junit-on-hyperexecute-grid.md): Learn how to run Selenium automation tests on HyperExecute using the JUnit framework -- [Run automation tests on HyperExecute using Nightwatch](https://www.testmuai.com/support/docs/nightwatch-on-hyperexecute-grid.md): Learn how to run Selenium automation tests on HyperExecute using the Nightwatch framework -- [Run automation tests on HyperExecute using NUnit](https://www.testmuai.com/support/docs/nunit-on-hyperexecute-grid.md): Learn how to run Selenium automation tests on HyperExecute using the NUnit framework -- [Run automation tests on HyperExecute using Protractor](https://www.testmuai.com/support/docs/protractor-on-hyperexecute-grid.md): Learn how to run Selenium automation tests on HyperExecute using the Protractor framework -- [Run automation tests on HyperExecute using Puppeteer Codecept](https://www.testmuai.com/support/docs/codecept-on-hyperexecute.md): Learn how to run Selenium automation tests on HyperExecute using the Puppeteer Codecept framework -- [Run automation tests on HyperExecute using Puppeteer Jest](https://www.testmuai.com/support/docs/jest-on-hyperexecute.md): Learn how to run Selenium automation tests on HyperExecute using the Puppeteer Jest framework -- [Run automation tests on HyperExecute using Puppeteer Mocha](https://www.testmuai.com/support/docs/mocha-on-hyperexecute.md): Learn how to run Selenium automation tests on HyperExecute using the Puppeteer Mocha framework -- [Run automation tests on HyperExecute using PyTest](https://www.testmuai.com/support/docs/pytest-on-hyperexecute-grid.md): Learn how to run Selenium automation tests on HyperExecute using the PyTest framework -- [Run automation tests on HyperExecute using PyUnit](https://www.testmuai.com/support/docs/pyunit-on-hyperexecute-grid.md): Learn how to run Selenium automation tests on HyperExecute using the PyUnit (or unittest) framework -- [Run automation tests on HyperExecute using Reqnroll](https://www.testmuai.com/support/docs/reqnroll-on-hyperexecute-grid.md): Learn how to run Selenium automation tests on HyperExecute using the Reqnroll framework -- [Run automation tests on HyperExecute using Ruby](https://www.testmuai.com/support/docs/ruby-on-hyperexecute-grid.md): Learn how to run Selenium automation tests on HyperExecute using Ruby -- [Run automation tests on HyperExecute using SpecFlow](https://www.testmuai.com/support/docs/specflow-on-hyperexecute-grid.md): Learn how to run Selenium automation tests on HyperExecute using the SpecFlow framework -- [Run automation tests on HyperExecute using TestNG](https://www.testmuai.com/support/docs/testng-on-hyperexecute-grid.md): Learn how to run Selenium automation tests on HyperExecute using the TestNG framework -- [Run automation tests on HyperExecute using WebdriverIO](https://www.testmuai.com/support/docs/webdriverio-on-hyperexecute-grid.md): Learn how to run Selenium automation tests on HyperExecute using the WebdriverIO framework -- [Run Local Tests Using Playwright](https://www.testmuai.com/support/docs/local-testing-using-playwright.md): Learn how to perform local testing with Playwright across 40+ browser versions on the TestMu AI platform. -- [Run Local Tests Using Puppeteer](https://www.testmuai.com/support/docs/local-testing-with-puppeteer.md): Learn how to perform local testing with Puppeteer across 40+ browser versions on the TestMu AI platform. -- [Run Parallel Tests With Selenium](https://www.testmuai.com/support/docs/run-parallel-tests-with-selenium.md): Run Selenium tests in parallel on the TestMu AI cloud grid to cut build times across 3000+ browser and OS combinations. -- [Run playwright automation tests on HyperExecute using .Net](https://www.testmuai.com/support/docs/playwright-dotnet-on-hyperexecute.md): Learn how to run playwright automation tests on HyperExecute using .Net framework -- [Run playwright automation tests on HyperExecute using CodeceptJs](https://www.testmuai.com/support/docs/playwright-codeceptjs-on-hyperexecute.md): Learn how to run playwright automation tests on HyperExecute using CodeceptJs framework -- [Run playwright automation tests on HyperExecute using Jest](https://www.testmuai.com/support/docs/playwright-jest-on-hyperexecute.md): Learn how to run playwright automation tests on HyperExecute using Jest framework -- [Run playwright automation tests on HyperExecute using JUnit](https://www.testmuai.com/support/docs/playwright-junit-on-hyperexecute.md): Learn how to run playwright automation tests on HyperExecute using JUnit framework -- [Run playwright automation tests on HyperExecute using PyTest framework](https://www.testmuai.com/support/docs/pytest-on-hyperexecute.md): Learn how to run playwright automation tests on HyperExecute using PyTest framework -- [Run playwright automation tests on HyperExecute using Python](https://www.testmuai.com/support/docs/playwright-python-on-hyperexecute.md): Learn how to run playwright automation tests on HyperExecute using Python framework -- [Run playwright automation tests on HyperExecute using VanillaJs](https://www.testmuai.com/support/docs/playwright-vanillajs-on-hyperexecute.md): Learn how to run playwright automation tests on HyperExecute using Vanilla Javascript -- [Run Puppeteer Tests Using CodeceptJS](https://www.testmuai.com/support/docs/puppeteer-testing-with-codecept.md): Learn how to integrate and run your Puppeteer tests using CodeceptJS across 40+ browser versions on the TestMu AI platform. -- [Run Puppeteer Tests Using Mocha](https://www.testmuai.com/support/docs/puppeteer-testing-with-mocha.md): Learn how to integrate and run your Puppeteer tests using Mocha across 40+ browser versions on the TestMu AI platform. -- [Run Puppeteer Tests Using Pytest-Pyppeteer](https://www.testmuai.com/support/docs/puppeteer-testing-with-pytest-pyppeteer.md): Learn how to integrate and run your Puppeteer tests using Pytest-Pyppeteer across 40+ browser versions on the TestMu AI platform. -- [Run Puppeteer Tests With Jest](https://www.testmuai.com/support/docs/puppeteer-testing-with-jest.md): Learn how to integrate and run your Puppeteer tests with Jest across 40+ browser versions on the TestMu AI platform. -- [Run Selenium Tests Behind the Proxy](https://www.testmuai.com/support/docs/selenium-tests-behind-proxy.md): Define proxy settings in your Selenium C# scripts to run tests on the cloud grid behind a proxy server. -- [Run Selenium Tests Using Agent Skills](https://www.testmuai.com/support/docs/selenium-agent-skills.md): Use TestMu AI Agent Skills to generate production-ready Selenium test automation with AI coding assistants like Claude Code, GitHub Copilot, and Cursor. -- [Run your C# automation scripts with Playwright on TestMu AI](https://www.testmuai.com/support/docs/csharp-with-playwright.md): Run your C# automation scripts with Playwright on TestMu AI scalable cloud grid of 50+ real desktop browsers and operating systems. -- [Run Your Cypress Tests In Parallel On TestMu AI Platform](https://www.testmuai.com/support/docs/run-your-cypress-tests-in-parallel.md): Run multiple Cypress tests on the TestMu AI cloud simultaneously across 40+ browser versions. This documentation will help you in running Cypress tests in parallel. -- [Run Your Cypress Tests Using Agent Skills](https://www.testmuai.com/support/docs/cypress-agent-skills.md): Learn how to use TestMu AI Agent Skills to generate production-ready Cypress test automation using AI coding assistants like Claude Code, GitHub Copilot, and Cursor. -- [Run Your First Selenium Test](https://www.testmuai.com/support/docs/testmu-running-your-first-selenium-test.md): Run your first Selenium test on TestMu AI's cloud grid. Clone a sample project, set credentials, configure capabilities, and execute. -- [Run Your First Web Test Using KaneAI](https://www.testmuai.com/support/docs/kaneai-test.md): Learn how to run your first web test using KaneAI. -- [Run your Java automation scripts with Playwright on TestMu AI](https://www.testmuai.com/support/docs/java-with-playwright.md): Run your Java and JUnit automation scripts with Playwright on TestMu AI scalable cloud grid of 50+ real desktop browsers and operating systems. -- [Run your JavaScript and TypeScript automation scripts with Playwright on TestMu AI](https://www.testmuai.com/support/docs/javascript-with-playwright.md): Run your JavaScript, Jest, and TypeScript automation scripts with Playwright on TestMu AI scalable cloud grid of 50+ real desktop browsers and operating systems. -- [Run your Jest test scripts with Playwright on TestMu AI](https://www.testmuai.com/support/docs/jest-with-playwright.md): Run your Jest scripts with Playwright on TestMu AI scalable cloud grid of 50+ real desktop browsers and operating systems. -- [Run your JUnit automation scripts with Playwright on TestMu AI](https://www.testmuai.com/support/docs/junit-with-playwright.md): Run your JUnit automation scripts with Playwright on TestMu AI scalable cloud grid of 50+ real desktop browsers and operating systems. -- [Run Your Playwright Tests Using Agent Skills](https://www.testmuai.com/support/docs/playwright-agent-skills.md): Learn how to use TestMu AI Agent Skills to generate production-ready Playwright test automation using AI coding assistants like Claude Code, GitHub Copilot, and Cursor. -- [Run Your Puppeteer Tests Using Agent Skills](https://www.testmuai.com/support/docs/puppeteer-agent-skills.md): Learn how to use TestMu AI Agent Skills to generate production-ready Puppeteer test automation using AI coding assistants like Claude Code, GitHub Copilot, and Cursor. -- [Run your Python automation scripts with Playwright on TestMu AI](https://www.testmuai.com/support/docs/python-with-playwright.md): Run your Python automation scripts with Playwright on TestMu AI scalable cloud grid of 50+ real desktop browsers and operating systems. -- [Run Your Quick Scan](https://www.testmuai.com/support/docs/accessibility-testing-run-quick-scan.md): Improve your website's accessibility by running a quick scan with TestMu AI Accessibility DevTools and efficiently review and resolve any issues found. -- [Run Your SmartUI Visual Tests Using Agent Skills](https://www.testmuai.com/support/docs/smartui-agent-skills.md): Use TestMu AI Agent Skills to let AI coding agents generate and run SmartUI visual regression tests (screenshot comparison across browsers and viewports) directly from natural language prompts. -- [Run your TypeScript automation scripts with Playwright on TestMu AI](https://www.testmuai.com/support/docs/typescript-with-playwright.md): Run your TypeScript automation scripts with Playwright on TestMu AI scalable cloud grid of 50+ real desktop browsers and operating systems. -- [Running Automation Tests Using TestingWhiz TestMu AI](https://www.testmuai.com/support/docs/testingwhiz-integration.md): Now perform cross-browser testing of your website or web app on 3000+ desktop & mobile browser and OS combinations, using TestMu AI and TestingWhiz integration -- [Running Automation Tests Using Testsigma TestMu AI Integration](https://www.testmuai.com/support/docs/testsigma-integration.md): TestMu AI now offers integration with Testsigma to help you perform automated cross-browser testing on 3000+ real browsers of its cloud-based Selenium Grid. -- [Running Cerberus Automation Tests Using TestMu AI](https://www.testmuai.com/support/docs/cerberus-integration.md): Now perform cross-browser testing of your website or web app on 3000+ desktop & mobile browser and OS combinations, using TestMu AI and Cerberus integration -- [Running PHP Tests on Online Selenium Grid | TestMu AI](https://www.testmuai.com/support/docs/quick-guide-to-run-php-tests-on-testmu-selenium-grid.md): Guide to running PHP and Selenium test scripts on TestMu AI Selenium automation grid online. Automated cross browser testing online using Selenium and PHP on 3000+ browsers on cloud -- [Running Playwright Real Device Tests on HyperExecute](https://www.testmuai.com/support/docs/playwright-real-device-on-hyperexecute.md): Learn how to run playwright automation tests using Real Devices on HyperExecute -- [Running Playwright Tests In CI/CD](https://www.testmuai.com/support/docs/playwright-tests-in-ci-cd.md): Learn how to perform Playwright testing through CI/CD across 40+ browser versions on the TestMu AI platform. -- [Running Playwright Tests With Cucumber.js](https://www.testmuai.com/support/docs/playwright-with-cucumberjs.md): Learn how to perform Playwright testing using Cucumber.js across 40+ browser versions on the TestMu AI platform. -- [Running Playwright Tests With Playwright Test Runner](https://www.testmuai.com/support/docs/playwright-test-runner.md): Learn how to perform Playwright testing using Playwright test runner across 40+ browser versions on the TestMu AI platform. -- [Running Puppeteer Tests In CI/CD](https://www.testmuai.com/support/docs/puppeteer-tests-in-ci-cd.md): Learn how to perform Puppeteer testing through CI/CD across 40+ browser versions on the TestMu AI platform. -- [Running Robot Framework Tests on HyperExecute](https://www.testmuai.com/support/docs/robot-on-hyperexecute-grid.md): Now you can run your automation scripts using Selenium with Robot on TestMu AI online grid of 2000+ real desktop browsers and real operating systems. -- [Running your Cypress tests in WebKit](https://www.testmuai.com/support/docs/cypress-testing-using-webkit.md): This documentation explains how to run your Cypress tests using WebKit. -- [Running Your First Cypress Test On TestMu AI Platform](https://www.testmuai.com/support/docs/running-your-first-cypress-test.md): TestMu AI enables you to run Cypress automated tests across 40+ browser versions on cloud. This documentation will help you get started with your first test. -- [Running Your First Job on HyperExecute](https://www.testmuai.com/support/docs/hyperexecute-running-your-first-job.md): Run your first job on HyperExecute -- [Running Your First Project on SmartUI](https://www.testmuai.com/support/docs/smartui-running-your-first-project.md): Learn how to run your first project on smartui step by step guide for the first time users to help them understand how the platform works -- [SauceLabs to HyperExecute](https://www.testmuai.com/support/docs/saucelabs-to-hyperexecute-migrate.md): Understand how to migrate from SauceLabs to HyperExecute -- [Scan Configurations for Mobile App Accessibility (Manual)](https://www.testmuai.com/support/docs/accessibility-app-scanner-scan-configurations.md): Configure which accessibility rules run on a manual mobile app scan: pick a WCAG level, toggle rule groups, enable individual rules, and reuse last-used settings. -- [Scan Configurations via Capabilities (Automation)](https://www.testmuai.com/support/docs/accessibility-automation-scan-configurations.md): Configure mobile app accessibility scans in automation through Appium capabilities: master toggle, WCAG version, and Best Practice, Beta, and AI rule groups. -- [Schedule Analytics Widget Reports](https://www.testmuai.com/support/docs/analytics-report-scheduling.md): Customize the delivery time for your analytics widget reports. Schedule reports at your preferred time for timely business insights. -- [Scheduled Test Runs](https://www.testmuai.com/support/docs/kaneai-scheduled-test-runs.md): This documentation will help you to understand how to create a scheduled test runs via KaneAI -- [Scheduling Options in Web Scanner](https://www.testmuai.com/support/docs/web-scanner-scheduling-options.md): A guide to the scheduling options available in Web Scanner, including one-time Quick Scans, recurring scans, and how to manage scheduled jobs. -- [SCIM Provisioning: Users & Groups](https://www.testmuai.com/support/docs/scim.md): Automate user and group lifecycle management with SCIM 2.0. Provision users, sync groups, map to teams, concurrency groups, and sub-organizations, all from your Identity Provider. -- [Screen Reader](https://www.testmuai.com/support/docs/screen-reader-on-accessibility.md): Test your app's accessibility with TestMu AI's Screen Reader as per the WCAG standards. -- [Screen Reader (TalkBack) on Android Devices for App Testing](https://www.testmuai.com/support/docs/screen-reader-on-real-devices-app.md): Test your app's accessibility with TestMu AI's Screen Reader. Navigate through app elements with spoken descriptions on real Android devices, ensuring compliance with WCAG standards. -- [Screen Reader (TalkBack) on Android Devices for Web Testing](https://www.testmuai.com/support/docs/screen-reader-on-real-devices-browser.md): Test the accessibility of your website with the Screen Reader from TestMu AI. Using actual Android devices, navigate through elements with spoken descriptions to make sure they adhere to WCAG standards. -- [Screen Reader (VoiceOver) on iOS Devices for App Testing](https://www.testmuai.com/support/docs/screen-reader-voiceover-real-devices-app.md): Test your app's and website's accessibility with TestMu AI's Screen Reader (VoiceOver) on iOS devices across multiple browsers. Navigate through app elements with spoken descriptions, ensuring compliance with WCAG standards. -- [Screen Reader (VoiceOver) on iOS Devices for Web Testing](https://www.testmuai.com/support/docs/screen-reader-voiceover-real-devices-browser.md): Test your website's accessibility with TestMu AI's Screen Reader (VoiceOver) on iOS devices. Navigate through web elements with spoken descriptions, ensuring compliance with WCAG standards. -- [Scroll Until Element for Mobile Authoring](https://www.testmuai.com/support/docs/kaneai-scroll-until-mobile.md): Learn how to use Scroll Until Element feature in KaneAI to automate scrolling using natural language commands for mobile app testing -- [Seamless Integration for Enhanced Test Orchestration](https://www.testmuai.com/support/docs/hyperexecute-seamless-integration-with-tools.md): Learn how to integrate your tools with HyperExecute -- [Search Query in GUI](https://www.testmuai.com/support/docs/hyperexecute-gui-query.md): This documentation outlines the steps on how to use search query in hyperexecute gui app. -- [Security FAQs](https://www.testmuai.com/support/docs/hyperexecute-security-faqs.md): Explore answers to all your HyperExecute Security-related queries with our comprehensive FAQs, designed to clarify all your concerns! -- [Select Pre-Installed App in Session](https://www.testmuai.com/support/docs/select-pre-installed-apps.md): Accelerate testing with pre-installed apps on TestMu AI. Start sessions faster, reduce setup time, and enhance efficiency in private cloud environments. -- [Selenium Capabilities for Version 3 and Below](https://www.testmuai.com/support/docs/selenium-three-capabilities.md): Configure desired capabilities for Selenium version 3 and below on the TestMu AI cloud grid. -- [Selenium Capabilities for Version 4 and Above](https://www.testmuai.com/support/docs/selenium-four-capabilities.md): Configure desired capabilities for Selenium version 4 and above on the TestMu AI cloud grid. -- [Selenium Default Capabilities](https://www.testmuai.com/support/docs/selenium-default-capabilities.md): View the default Selenium capabilities supported in TestMu AI for browser configuration and session control. -- [Selenium Grid Configuration Guide](https://www.testmuai.com/support/docs/testmu-grid-configuration.md): View the current Selenium Grid configuration including driver versions, HUB URI, ports, and defaults. -- [Selenium Grid Supported Languages & Frameworks](https://www.testmuai.com/support/docs/supported-languages-and-frameworks.md): TestMu AI Selenium Grid supports all the programming languages & testing frameworks that are compatible with Selenium. -- [Selenium Supported Languages and Frameworks](https://www.testmuai.com/support/docs/selenium-supported-languages-and-frameworks.md): Run Selenium tests in Java, JavaScript, Python, C#, PHP, and Ruby on the TestMu AI cloud grid. Find your framework-specific guide. -- [Selenium Testing from Different Geolocations](https://www.testmuai.com/support/docs/selenium-geolocation-capabilities.md): Run Selenium tests from different countries and timezones using the geoLocation capability. -- [Selenium Testing With Concourse CI And TestMu AI](https://www.testmuai.com/support/docs/selenium-testing-with-concourse-ci-and-testmu.md): This document will help you understand how you can integrate your Concourse CI instance with cloud Selenium Grid for continuous testing. -- [Selenium With AngularJS and Karma](https://www.testmuai.com/support/docs/angularjs-with-karma-running-angular-tests-on-testmu-selenium-grid.md): Run AngularJS automated tests with Karma on TestMu AI cloud Selenium Grid with 3000+ real browsers and operating systems. -- [Selenium With Behat](https://www.testmuai.com/support/docs/behat-with-selenium-running-behat-automation-scripts-on-testmu-selenium-grid.md): Run Behat PHP Selenium automation tests on the TestMu AI cloud grid across 3000+ browser and OS combinations. -- [Selenium With Behave](https://www.testmuai.com/support/docs/behave-with-selenium-running-behave-automation-scripts-on-testmu-selenium-grid.md): Run Behave automation scripts on TestMu AI Selenium Grid with 3000+ browser and OS combinations. Setup, config, and execution steps included. -- [Selenium With C#](https://www.testmuai.com/support/docs/c-with-selenium-running-c-automation-scripts-on-testmu-selenium-grid.md): Run C# Selenium tests on TestMu AI cloud grid with 3000+ browsers. Covers NUnit, MSTest, SpecFlow, and xUnit. -- [Selenium With Capybara](https://www.testmuai.com/support/docs/ruby-with-capybara-running-automation-scripts-on-testmu-selenium-grid.md): Run Capybara Ruby Selenium automation tests on the TestMu AI cloud grid across 3000+ browser and OS combinations. -- [Selenium With Codeception](https://www.testmuai.com/support/docs/codeception-integration-with-testmu.md): Run Codeception PHP Selenium tests on the TestMu AI cloud grid across 3000+ browser and OS combinations. -- [Selenium With Cucumber](https://www.testmuai.com/support/docs/running-cucumber-scripts-with-testng-and-selenium.md): Run Cucumber TestNG Selenium tests on TestMu AI cloud grid across 3000+ browsers and devices. -- [Selenium With CucumberJS](https://www.testmuai.com/support/docs/cucumberjs-with-selenium-running-cucumberjs-automation-scripts-on-testmu-selenium-grid.md): Run Cucumber JS Selenium automation scripts on TestMu AI cloud grid with 3000+ real browsers and operating systems. -- [Selenium With Gauge](https://www.testmuai.com/support/docs/running-gauge-tests-on-testmu-selenium-grid.md): Run Gauge Selenium tests on TestMu AI cloud grid with parallel execution across 3000+ browsers. -- [Selenium With Geb](https://www.testmuai.com/support/docs/run-geb-tests-on-selenium-grid.md): Run Geb Groovy Selenium tests on TestMu AI cloud grid with parallel execution across 3000+ browsers. -- [Selenium With Jasmine and Karma](https://www.testmuai.com/support/docs/jasmine-with-karma-running-jasmine-tests-on-testmu-selenium-grid.md): Run Jasmine BDD tests with Karma on TestMu AI cloud Selenium Grid with 3000+ real browsers and operating systems. -- [Selenium With Java](https://www.testmuai.com/support/docs/java-with-selenium-running-java-automation-scripts-on-testmu-selenium-grid.md): Run Java Selenium tests on TestMu AI cloud grid across 3000+ browsers and OS combinations. Covers TestNG, JUnit, Cucumber, Selenide, Gauge, Geb, Serenity, Tesbo, and the Java SDK. -- [Selenium With JavaScript](https://www.testmuai.com/support/docs/javascript-with-selenium-running-javascript-automation-scripts-on-testmu-selenium-grid.md): Run JavaScript Selenium test scripts on TestMu AI cloud grid with 3000+ real browsers and operating systems. Covers Mocha, Nightwatch, WebdriverIO, Jest, CucumberJS, Jasmine, Karma, TestCafe, NemoJS, and CodeceptJS. -- [Selenium With Jest](https://www.testmuai.com/support/docs/automation-testing-with-selenium-and-jest.md): Run Jest Selenium automation tests on TestMu AI cloud grid with 3000+ real browsers and operating systems. -- [Selenium With JUnit](https://www.testmuai.com/support/docs/junit-with-selenium-running-junit-automation-scripts-on-testmu-selenium-grid.md): Run JUnit Selenium tests on TestMu AI cloud grid with parallel execution across 3000+ browsers. -- [Selenium With Karma](https://www.testmuai.com/support/docs/karma-integration-with-testmu.md): Integrate Karma test runner with TestMu AI to run automated cross browser tests on 3000+ real browsers in the cloud. -- [Selenium With Laravel Dusk](https://www.testmuai.com/support/docs/laravel-dusk-with-selenium-running-laravel-dusk-automation-scripts-on-testmu-selenium-grid.md): Run Laravel Dusk Selenium automation tests on the TestMu AI cloud grid across 3000+ browser and OS combinations. -- [Selenium With Lettuce](https://www.testmuai.com/support/docs/automation-testing-selenium-with-lettuce.md): Run Lettuce Selenium test scripts on TestMu AI cloud grid with 3000+ desktop and mobile browsers. Setup, config, and execution steps included. -- [Selenium With Mocha](https://www.testmuai.com/support/docs/automation-testing-with-mocha-and-selenium.md): Run Mocha Selenium automation tests on TestMu AI cloud grid with 3000+ real browsers. Use parallel testing to speed up builds. -- [Selenium With MSTest](https://www.testmuai.com/support/docs/mstest-with-selenium-running-mstest-automation-scripts-on-testmu-selenium-grid.md): Run MSTest Selenium tests on TestMu AI cloud grid with 3000+ browsers. Set up, configure, and execute C# tests. -- [Selenium With NemoJS](https://www.testmuai.com/support/docs/selenium-tests-with-nemojs.md): Run NemoJS Selenium automated tests on TestMu AI cloud grid with 3000+ real browsers and operating systems. -- [Selenium With Nightwatch.js](https://www.testmuai.com/support/docs/nightwatch-with-selenium-running-nightwatch-automation-scripts-on-testmu-selenium-grid.md): Run Nightwatch Selenium automation scripts on TestMu AI cloud grid with 3000+ real browsers and operating systems. -- [Selenium With NUnit](https://www.testmuai.com/support/docs/nunit-with-selenium-running-nunit-automation-scripts-on-testmu-selenium-grid.md): Run NUnit Selenium tests on TestMu AI cloud grid with 3000+ browsers. Set up, configure, and execute C# tests. -- [Selenium With PHP](https://www.testmuai.com/support/docs/php-with-selenium-running-php-automation-scripts-on-testmu-selenium-grid.md): Learn to run PHP Selenium automation tests on the TestMu AI cloud grid with 3000+ browser and OS combinations. Covers Behat, Laravel Dusk, Codeception, and PHPUnit. -- [Selenium With PHPUnit](https://www.testmuai.com/support/docs/phpunit-with-selenium-running-phpunit-automation-scripts-on-testmu-selenium-grid.md): Run PHPUnit Selenium automation tests on the TestMu AI cloud grid across 3000+ browser and OS combinations. -- [Selenium With Protractor](https://www.testmuai.com/support/docs/protractor-with-selenium-running-protractor-automation-scripts-on-testmu-selenium-grid.md): Run Protractor Selenium automation scripts on TestMu AI cloud grid with 3000+ real browsers and operating systems. -- [Selenium With PyTest](https://www.testmuai.com/support/docs/pytest-with-selenium-running-pytest-automation-script-on-testmu-selenium-grid.md): Run PyTest Selenium test scripts on TestMu AI cloud grid with 3000+ desktop and mobile browsers. Setup, config, and parallel execution guide. -- [Selenium With Python](https://www.testmuai.com/support/docs/python-with-selenium-running-python-automation-scripts-on-testmu-selenium-grid.md): Run Python Selenium test scripts on TestMu AI cloud grid with 3000+ browser and OS combinations. Covers unittest, pytest, Robot Framework, and Behave. -- [Selenium With Python Unittest](https://www.testmuai.com/support/docs/running-unit-testing-in-python-on-testmu-selenium-grid.md): Run Python unittest Selenium tests on TestMu AI cloud grid with 3000+ browser and OS combinations. Includes setup, config, and execution steps. -- [Selenium With Reqnroll](https://www.testmuai.com/support/docs/reqnroll-automation-with-selenium-appium-on-testmu.md): Run Reqnroll Selenium and Appium tests on TestMu AI cloud grid with 10000+ browsers and real devices. -- [Selenium With Robot Framework](https://www.testmuai.com/support/docs/robot-with-selenium-running-robot-automation-scripts-on-testmu-selenium-grid.md): Run Robot Framework Selenium tests on TestMu AI cloud grid with 3000+ browser and OS combinations. Setup, config, and parallel execution included. -- [Selenium With RSpec](https://www.testmuai.com/support/docs/rspec-with-selenium-running-rspec-automation-scripts-on-testmu-selenium-grid.md): Run RSpec Ruby Selenium automation tests on the TestMu AI cloud grid across 3000+ browser and OS combinations. -- [Selenium With Ruby](https://www.testmuai.com/support/docs/ruby-with-selenium-running-ruby-automation-scripts-on-testmu-selenium-grid.md): Run Ruby Selenium automation tests on the TestMu AI cloud grid across 3000+ browser and OS combinations. Covers Capybara, RSpec, and Test::Unit. -- [Selenium With Selenide](https://www.testmuai.com/support/docs/selenide-tests-with-testmu-online-selenium-grid-for-automated-cross-browser-testing.md): Run Selenide tests on TestMu AI cloud grid with parallel execution across 3000+ browsers. -- [Selenium With Serenity BDD](https://www.testmuai.com/support/docs/serenity-test-on-selenium-grid.md): Run Serenity BDD Selenium tests on TestMu AI cloud grid with parallel execution across 3000+ browsers. -- [Selenium With SpecFlow](https://www.testmuai.com/support/docs/specflow-with-selenium-running-specflow-automation-scripts-on-testmu-selenium-grid.md): Run SpecFlow Selenium tests on TestMu AI cloud grid with 3000+ browsers. Set up, configure, and execute C# tests. -- [Selenium With Tesbo](https://www.testmuai.com/support/docs/executing-tesbo-automation-script-over-online-selenium-grid.md): Run Tesbo keyword-driven automation scripts on the TestMu AI cloud Selenium grid across 3000+ browsers. -- [Selenium With Test::Unit](https://www.testmuai.com/support/docs/testunit-with-selenium-running-testunit-automation-scripts-on-testmu-selenium-grid.md): Run Test::Unit Ruby Selenium automation tests on the TestMu AI cloud grid across 3000+ browser and OS combinations. -- [Selenium With TestCafe](https://www.testmuai.com/support/docs/npm-plugin-for-testcafe-integration-with-testmu.md): Integrate TestCafe with TestMu AI using the npm plugin to run tests on 3000+ real browsers and operating systems. -- [Selenium With TestNG](https://www.testmuai.com/support/docs/testng-with-selenium-running-java-automation-scripts-on-testmu-selenium-grid.md): Run TestNG Selenium tests on TestMu AI cloud grid with parallel execution across 3000+ browsers. -- [Selenium With WD](https://www.testmuai.com/support/docs/wd-with-selenium-running-wd-automation-scripts-on-testmu-selenium-grid.md): Run WD Selenium automation scripts on TestMu AI cloud grid with 3000+ real browsers and operating systems. -- [Selenium With WebDriverIO](https://www.testmuai.com/support/docs/webdriverio-with-selenium-running-webdriverio-automation-scripts-on-testmu-selenium-grid.md): Run WebDriverIO Selenium automation scripts on TestMu AI cloud grid with 3000+ real browsers and operating systems. -- [Selenium With WebDriverIO 5.6.2](https://www.testmuai.com/support/docs/webdriverio-5-6-2-with-selenium-running-automation-scripts-on-testmu-selenium-grid.md): Run WebDriverIO 5.6.2 Selenium automation scripts on TestMu AI cloud grid with 3000+ real browsers and operating systems. -- [Selenium With xUnit](https://www.testmuai.com/support/docs/selenium-with-xunit.md): Run xUnit Selenium tests on TestMu AI cloud grid with 3000+ browsers. Set up, configure, and execute C# tests. -- [Self-Hosted Jira Integration | TestMu AI](https://www.testmuai.com/support/docs/jira-self-hosted-integration.md): TestMu AI and Self-hosted Jira integration allow you to integrate your self-hosted Jira instance and create the issues directly from the TestMu AI platform to your Jira dashboard. -- [Semaphore CI Integration With HyperExecute](https://www.testmuai.com/support/docs/semaphore-integration-with-hyperexecute.md): Semaphore CI/CD, a cloud-based platform, now seamlessly integrates with TestMu AI HyperExecute for enhanced testing performance. -- [Semaphore CI Integration With TestMu AI](https://www.testmuai.com/support/docs/semaphore-integration-with-testmu.md): Boost market delivery with TestMu AI's integration with Semaphore CI and automate tests on a cloud grid with 3000+ real browsers seamlessly. -- [Semaphore Pipeline Integration with SmartUI](https://www.testmuai.com/support/docs/smartui-with-semaphore.md): SmartUI now integrates with Semaphore Pipeline to boost pipeline delivery. Perform automated cross browser testing with SmartUI to seamlessly providing 3000+ real browsers running through machines. -- [Sequential Test Runs](https://www.testmuai.com/support/docs/kaneai-sequential-test-runs.md): Learn how to set up and execute sequential test runs in KaneAI to run dependent test cases in a defined order across single and multi-platform configurations. -- [ServiceNow Integration with TestMu AI](https://www.testmuai.com/support/docs/servicenow-integration.md): TestMu AI helps you manage your incidents in ServiceNow by moving them to your project in a single click. All the details you provide in TestMu AI like task list, assignee, title, and description would automatically be presented in the project on ServiceNow. -- [Session Configuration - TestMu AI Browser Cloud](https://www.testmuai.com/support/docs/browser-cloud-session-configuration.md): Complete reference for all options available when creating a session with the TestMu AI Browser SDK. -- [Session Lifecycle - TestMu AI Browser Cloud](https://www.testmuai.com/support/docs/browser-cloud-session-lifecycle.md): How sessions are created, managed, and released in TestMu AI Browser Cloud. -- [Sessions Overview - TestMu AI Browser Cloud](https://www.testmuai.com/support/docs/browser-cloud-sessions-overview.md): The Sessions API lets you create and control cloud-based browser sessions through simple SDK calls in TestMu AI Browser Cloud. -- [Set Custom Date, Time & Hour Format on Real Devices](https://www.testmuai.com/support/docs/set-date-time-hour-format-real-devices.md): Set custom date, time, and 12/24-hour format on real iOS and Android devices during manual app testing and Appium automation sessions on TestMu AI. -- [Set Custom Date, Time & Hour Format on Real Devices (Browser)](https://www.testmuai.com/support/docs/set-date-time-hour-format-real-devices-browser.md): Change the date, time, and hour format on real iOS and Android devices while running real-time browser tests on TestMu AI, across manual and automated runs. -- [Set Device Dark Mode](https://www.testmuai.com/support/docs/set-device-dark-mode.md): This documentation will help you change thed device level dark and light mode. -- [Set SmartUI Comparison Modes at the Session Level](https://www.testmuai.com/support/docs/smartui-comparison-capabilities.md): Enable SmartUI comparison modes such as layout and Smart Ignore once in LT:Options and apply them to every screenshot in the session, with per-screenshot overrides. -- [Setting Browser Options](https://www.testmuai.com/support/docs/selenium-set-browser-options.md): Set browser options for Chrome, Firefox, and Edge in Selenium tests on TestMu AI. -- [Setting The Open File Limit While Using TestMu AI Tunnel](https://www.testmuai.com/support/docs/how-to-set-the-open-file-limit-while-using-tunnel.md): This document will help you modify your open file limit temporarily or permanently while using a proxy with TestMu AI tunnel for seamless cross browser testing. -- [Setting Up Charles Proxy](https://www.testmuai.com/support/docs/charles-proxy.md): This article will guide you on how to setup Charles Proxy for the tests run on TestMu AI platform through TestMu AI Tunnel. -- [Setting Up Espresso Environment Variables](https://www.testmuai.com/support/docs/espresso-env-variables-settings.md): Now you can run your Espresso framework on TestMu AI and this particular feature allows users to pass and retrieve environment variables (like STAGE, PROD, or DEV) during automated Android tests. -- [Settings and Security](https://www.testmuai.com/support/docs/settings-and-security.md): Discover how to protect your data with TestMu AI- Manage security settings & ensure data confidentiality and integrity efficiently. -- [Shadow DOM](https://www.testmuai.com/support/docs/smartui-shadow-dom.md): In this documentation, learn about about the shadow dom feature of smartui -- [Sharding for Espresso](https://www.testmuai.com/support/docs/sharding-espresso-rd-hyperexecute.md): This document will cover how to execute Espresso Tests on real devices with HyperExecute. Before starting, please make sure you have App Automation plans on your account. -- [Sharding for XCUI](https://www.testmuai.com/support/docs/sharding-rd-hyperexec.md): This document will cover how to execute XCUI Tests on real devices with HyperExecute. Before starting, please make sure you have App Automation and HyperExecute Cloud plans on your account. -- [Sharing Test Results on TestMu AI](https://www.testmuai.com/support/docs/sharing-test-results.md): Share individual test results and execution videos with team members using Session ID and auth tokens. -- [Shortcut Integration](https://www.testmuai.com/support/docs/shortcut-integration.md): TestMu AI helps you manage your bugs in Shortcut by moving them to project in a single click. All the details you provide in TestMu AI like task list, assignee, title and description would automatically be presented in the project on Shortcut. -- [Siesta Integration](https://www.testmuai.com/support/docs/siesta-integration.md): Integrate TestMu AI with Siesta to perform automated browser testing of your locally hosted websites and web pages across 3000+ real browsers and operating systems. -- [SIM & eSIM Automation on Real Devices](https://www.testmuai.com/support/docs/sim-support.md): Learn how to automate SIM and eSIM scenarios on real mobile devices using Appium, including capabilities, SMS validation, and supported devices. -- [SIM & eSIM on Real Devices](https://www.testmuai.com/support/docs/sim-support-realdevices.md): Learn how to manually test SIM and eSIM scenarios on real mobile devices using TestMu AI, including configurations, SMS validation, and supported devices. -- [Sitemap Extraction & URL Import](https://www.testmuai.com/support/docs/accessibility-sitemap-extraction-url-import.md): Use sitemap extraction and URL import to seed scheduled Accessibility scans with the right page set. -- [Slack App Integration](https://www.testmuai.com/support/docs/slack-app.md): Integrate Slack App with TestMu AI and quickly launch the real-time and screenshot tests directly from Slack. Collaborate faster with your teammates on issues and fasten the feedback cycle. -- [Slack Integration with SmartUI](https://www.testmuai.com/support/docs/smartui-slack-integration.md): Integrate Slack with TestMu AI SmartUI for real-time notifications. Follow our guide to set up, configure, and receive SmartUI job updates directly in your Slack workspace. -- [Slack Integration With TestMu AI](https://www.testmuai.com/support/docs/slack-integration.md): The TestMu AI Integration with Slack allows you to push a bug directly to your specified Slack channel from TestMu AI platform. -- [Slack Notifications for App Automation](https://www.testmuai.com/support/docs/app-automation-slack-notifications.md): Get real-time Slack notifications for your test sessions and build summaries on TestMu AI. -- [Smart Git Strategy in SmartUI](https://www.testmuai.com/support/docs/smartui-smart-git-strategy.md): Learn about SmartUI's Smart Git feature that provides intelligent branch comparison and management capabilities for visual regression testing. -- [Smart Heal in Automation](https://www.testmuai.com/support/docs/smart-heal-appium.md): Learn how to enable Smart Heal, TestMu AI's Auto-Heal capability, for real device automation tests to reduce flakiness by automatically recovering from locator failures during execution. -- [Smart Ignore](https://www.testmuai.com/support/docs/smartui-smartignore.md): Learn how to use our new comparison engine Smart Ignore to efficiently ignore noise, layout shifts and displacement differences in your visual tests -- [Smart Locator Priority](https://www.testmuai.com/support/docs/kaneai-smart-locator-priority.md): Learn how to configure Smart Locator Priority in KaneAI Org Product Preferences so your organization can define which locator strategy should take precedence during Smart Locator resolution. -- [Smart Tags - Test Intelligence](https://www.testmuai.com/support/docs/analytics-smart-tags-test-intelligence.md): Analyze tests faster with Smart Tags in TestMu AI Analytics. Get insights on your test automation execution with Smart Tags. -- [Smart Visual Regression Testing Using TestMu AI](https://www.testmuai.com/support/docs/smart-visual-testing.md): Discover visual deviations and regression bugs with TestMu AI's Smart Visual Testing. Upload, compare, and ensure top-quality UI -- [SmartUI Analytics - Visual Regression Testing Insights](https://www.testmuai.com/support/docs/analytics-modules-smartui-analytics.md): Discover TestMu AI's SmartUI Analytics widgets for comprehensive visual regression testing insights. Optimize your visual testing efforts today. -- [SmartUI App SDK Integration Guide](https://www.testmuai.com/support/docs/smartui-appium-java-sdk.md): Learn how to integrate SmartUI App SDK with your existing mobile app testing framework to perform visual regression testing on any cloud provider. -- [SmartUI Approval, Reject, Move, and Merge Functions Guide](https://www.testmuai.com/support/docs/smartui-approval-workflow-guide.md): Comprehensive guide to understanding and using SmartUI approval, reject, move, and merge functions. Learn how baseline changes require explicit user updates across different workflows. -- [SmartUI Audit & Activity Logs](https://www.testmuai.com/support/docs/smartui-audit-logs.md): Track and audit all significant actions in your SmartUI projects. Use project-, build-, and screenshot-level logs for accountability and compliance. -- [SmartUI Capture Onboarding with HyperExecute](https://www.testmuai.com/support/docs/hyperexecute-smart-ui-capture-onboarding.md): Complete step-by-step guide to get started with SmartUI Capture on HyperExecute for visual regression testing. Learn setup, configuration, best practices, and troubleshooting. -- [SmartUI CLI Complete Reference Guide](https://www.testmuai.com/support/docs/smartui-cli-complete-reference.md): Comprehensive reference guide for SmartUI CLI and Storybook CLI commands, options, advanced features, and end-to-end workflows -- [SmartUI CLI Exec Command](https://www.testmuai.com/support/docs/smartui-cli-exec.md): Learn how to use SmartUI CLI exec commands to manage snapshot servers and execute tests -- [SmartUI Hooks - Layout, Full Page, and Smart Ignore](https://www.testmuai.com/support/docs/smartui-hooks-layout-fullpage-smartignore.md): SmartUI Hooks on LambdaTest covering layout comparison via screenshot hook options, Smart Ignore via smartUI.smartIgnore in LT:Options, and full-page capture. -- [SmartUI in the Automation Dashboard](https://www.testmuai.com/support/docs/smartui-automation-dashboard.md): View, group, and review SmartUI visual regression results directly inside the Automation Dashboard, without switching products. Every test run that captures SmartUI screenshots now has a dedicated SmartUI tab for in-context triage. -- [SmartUI Multiselect & Bulk Operations](https://www.testmuai.com/support/docs/smartui-multiselect-bulkops.md): Streamline your visual testing workflow with SmartUI Multiselect & Bulk Operations. Approve or reject multiple screenshots simultaneously. -- [SmartUI Netlify Integration With TestMu AI](https://www.testmuai.com/support/docs/smartui-with-netlify.md): Run automated SmartUI visual regression tests on TestMu AI directly from your Netlify build pipeline and catch UI changes before deploy. -- [SmartUI Plugin for Katalon Studio](https://www.testmuai.com/support/docs/smartui-katalon-plugin.md): Integrate TestMu AI SmartUI with Katalon Studio to perform visual regression testing directly in your test automation workflows. -- [SmartUI Root Cause Analysis (RCA)](https://www.testmuai.com/support/docs/smartui-root-cause-analysis.md): Identify and resolve visual regression issues quickly with SmartUI Root Cause Analysis (RCA). Understand the underlying causes of visual mismatches. -- [SmartUI SDK Advanced Configuration Options](https://www.testmuai.com/support/docs/smartui-sdk-config-options.md): In this documentation, learn about the options available in SmartUI SDK configuration -- [SmartUI SDK Capabilities](https://www.testmuai.com/support/docs/smartui-sdk-capabilities.md): Learn about the various capabilities available in SmartUI SDK for configuring your visual regression tests -- [SmartUI SDK Environment Variables](https://www.testmuai.com/support/docs/smartui-cli-env-variables.md): In this documentation, learn about the various environment variables available in SmartUI -- [SmartUI Smart Comments](https://www.testmuai.com/support/docs/smartui-smart-comments.md): Learn how to collaborate on visual test results using Smart Comments. Add comments directly on screenshots and builds, mention teammates, and manage threaded discussions for better team collaboration. -- [SmartUI Testing in HyperExecute using Playwright](https://www.testmuai.com/support/docs/hyperexecute-smart-ui-test-using-playwright.md): Boost bug detection speed using HyperExecute smart UI testing with Playwright. Perform Visual UI Regression Testing for perfect app delivery. -- [SmartUI Testing Using HyperExecute](https://www.testmuai.com/support/docs/smart-ui-testing.md): Improve bug discovery using TestMu AI's Advanced Visual UI Regression Testing. Effortlessly pinpoint visual discrepancies and regression issues in recent builds to ensure impeccable app deployment. -- [SmartUI Testing Using HyperExecute with Cypress](https://www.testmuai.com/support/docs/hyperexecute-smart-ui-testing-using-cypress.md): Boost bug detection speed using HyperExecute smart UI testing with Cypress. Perform Visual UI Regression Testing for perfect app delivery. -- [SmartUI Tests in HyperExecute using Selenium](https://www.testmuai.com/support/docs/hyperexecute-smart-ui-testing-using-selenium.md): Accelerate bug identification with HyperExecute's intelligent UI testing via Selenium. Conduct Visual UI Regression Testing for flawless application deployment. -- [SmartUI Visual Regression Testing for PDF Files](https://www.testmuai.com/support/docs/smartui-pdf-comparison.md): In this documentation, learn how to perform Visual Regression testing of PDFs using SmartUI. -- [SmartWait Functionality for Selenium Automation](https://www.testmuai.com/support/docs/smart-wait.md): Use SmartWait to run actionability checks before each Selenium action, reducing explicit waits and flaky tests. -- [SOCKS5 Proxy Support in TestMu AI Tunnel](https://www.testmuai.com/support/docs/socks5-proxy-support.md): Learn how to use TestMu AI Tunnel with SOCKS5 proxy for secure, private, and flexible local testing. -- [SpecFlow Report](https://www.testmuai.com/support/docs/specflow-report.md): Learn how to generate SpecFlow Report on TestMu AI and download the reports from the dashboard -- [SpiraTeam Integration](https://www.testmuai.com/support/docs/spirateam-integration.md): TestMu AI provides integrations with SpiraTeam to help you log bugs directly from the middle of your test session on TestMu AI to SpiraTeam platform. -- [Starting a Visual UI Scan with Web Scanner](https://www.testmuai.com/support/docs/web-scanner-visual-scan.md): Learn how to configure a Visual UI scan, including default settings for browsers and resolutions, and advanced options using a SmartUI JSON config. -- [Starting an Accessibility Scan with Web Scanner](https://www.testmuai.com/support/docs/web-scanner-accessibility-scan.md): Configure an Accessibility scan in Web Scanner, including WCAG version, review issues, best-practice checks, local testing, and login settings. -- [Status Messages (4.1.3)](https://www.testmuai.com/support/docs/accessibility-web-rule-4-1-3-status-messages.md): Rule-level Accessibility guidance for Status Messages (WCAG 4.1.3 Level AA). -- [Stealth Mode - TestMu AI Browser Cloud](https://www.testmuai.com/support/docs/browser-cloud-stealth.md): Make your agent's browser look like a real human user to bot-detection systems with TestMu AI Browser Cloud stealth mode. -- [Step By Step Guide to Integrate TestMu AI With Katalon](https://www.testmuai.com/support/docs/katalon-integration-with-testmu.md): TestMu AI offers integration with Katalon Studio to ease your automated cross browser testing process through their Selenium grid consisting more than 3000+ browsers. -- [Steps to Schedule an Accessibility Scan](https://www.testmuai.com/support/docs/accessibility-test-scheduling-scan.md): Learn how to schedule accessibility scans with TestMu AI. Configure scans, automate WCAG compliance checks, and analyze detailed reports with ease. -- [Sub Organizations](https://www.testmuai.com/support/docs/sub-organizations.md): This documentation will help you create and manage sub-organizations on TestMu AI. -- [Sub Organizations Insights](https://www.testmuai.com/support/docs/analytics-sub-organization-widgets.md): A comprehensive guide to understanding test health and trends module. Explore widgets for Test Status, Browser, OS Distribution and more. -- [Sumo Logic Integration With TestMu AI](https://www.testmuai.com/support/docs/sumo-logic-integration.md): View your TestMu AI test results in a custom Sumo Logic dashboard. Gain deep insights into your automation scripts and identify gaps to ship quality products at light speed. -- [Support Access](https://www.testmuai.com/support/docs/support-access.md): Grant backend access to TestMu AI for efficient debugging & delegated tasks, with the ability to revoke anytime. -- [Support for iOS keychain cleanup and access groups](https://www.testmuai.com/support/docs/ios-keychain-cleanup.md): Secure and clean iOS testing with TestMu AI's iOS Keychain cleanup and seamless access group handling support. -- [Support for Wildcard in --allowHosts Flag](https://www.testmuai.com/support/docs/tunnel-allowHost-wildcard-support.md): Now you can use wildcard values for the -allow-Host flag for setting up the tunnel at TestMu AI. -- [Supported Appium Plugins](https://www.testmuai.com/support/docs/supported-appium-plugins.md): This document provides information about configuring Appium plugins for tests on the TestMu AI platform and also provides a list of supported plugins. -- [Supported Appium Versions - Real Devices](https://www.testmuai.com/support/docs/supported-appium-versions.md): This document provides information about configuring Appium versions for tests on the TestMu AI platform and also provides list of supported versions. -- [Supported Browsers and Operating Systems](https://www.testmuai.com/support/docs/supported-browsers-and-operating-systems-for-the-web-interface.md): Check which browsers, versions, and operating systems are supported for Selenium and Appium testing. -- [Supported Browsers and Operating Systems for Cypress](https://www.testmuai.com/support/docs/supported-browsers-and-os.md): Check out the list of supported browsers and operating systems for Cypress testing on the TestMu AI platform. -- [Supported Cypress Versions](https://www.testmuai.com/support/docs/supported-cypress-versions.md): This documentation will help you learn about the Cypress versions that are supported by TestMu AI. -- [Supported Locales And Languages - Appium](https://www.testmuai.com/support/docs/list-of-supported-locales.md): Checkout the list of all supported locales by TestMu AI. -- [Supported Locales And Languages - Espresso](https://www.testmuai.com/support/docs/supported-locales-espresso.md): Checkout the list of all supported locales for espresso. -- [Supported Locales And Languages - XCUI](https://www.testmuai.com/support/docs/supported-locales-xcui.md): Checkout the list of all supported locales -- [Supported Timezones on TestMu AI](https://www.testmuai.com/support/docs/supported-timezone.md): This document provides information about configuring timezone for tests on the TestMu AI platform and also provides list of supported timezones. -- [Supported WCAG Versions & Browsers](https://www.testmuai.com/support/docs/accessibility-supported-wcag-browsers.md): Reference page for supported WCAG versions, browser scope, and high-level environment considerations in Accessibility Testing. -- [Sync Test Instances in Manual Test Runs](https://www.testmuai.com/support/docs/sync-test-instance.md): Learn how to sync test instances in manual test runs with the latest master test case version in TestMu AI Test Manager. -- [System and Custom Fields](https://www.testmuai.com/support/docs/system-and-custom-fields.md): Learn about TestMu AI's system and custom Fields feature for test project organization. Explore System Fields and create Custom Fields to enhance your testing workflows. -- [Table Widgets](https://www.testmuai.com/support/docs/analytics-table-widgets.md): Learn how to create and configure Table widgets to display structured test data in tabular format with multiple columns, aggregations, and grouping options. -- [Tag Support for Accessibility Scans](https://www.testmuai.com/support/docs/accessibility-tag-support.md): Use tags to organize accessibility scans on mobile devices by screen, flow, or activity from the dashboard or during automation. -- [Taiga Integration with TestMu AI](https://www.testmuai.com/support/docs/taiga-integration.md): TestMu AI helps you manage your bugs in Taiga by moving them to your project in a single click. All the details you provide in TestMu AI like task list, assignee, title, and description would automatically be presented in the project on Taiga. -- [Take a Screenshot of a Specific Element on a Webpage (SmartUI Hooks)](https://www.testmuai.com/support/docs/smartui-hooks-element-screenshot.md): Capture a visual baseline for a specific element on a webpage using SmartUI Hooks on LambdaTest. -- [Taking Screenshots of Pages Behind Login](https://www.testmuai.com/support/docs/taking-screenshots-of-pages-behind-login.md): With TestMu AI, now you can capture bulk screenshots of your web app or webpages, whether they behind a sign-in form or on a private staging server, for different browsers to measure the cross-browser compatibility. -- [TalkBack on Android](https://www.testmuai.com/support/docs/accessibility-talkback-android.md): TalkBack on Android devices: enablement, gestures, and what to validate for mobile accessibility. -- [Target Size (Minimum) (2.5.8)](https://www.testmuai.com/support/docs/accessibility-web-rule-2-5-8-target-size.md): Rule-level Accessibility guidance for Target Size Minimum (WCAG 2.5.8 Level AA). -- [TargetProcess Integration](https://www.testmuai.com/support/docs/target-process-integration.md): Using TestMu AI integration with TargetProcess, you'll be able to log bugs over your project in TargetProcess from TestMu AI in just a single click. -- [Team Communication Tools](https://www.testmuai.com/support/docs/team-communication-tools.md): Integrate your TestMu AI account with communication tools. That way you can instantly share any UI bug that you come across while cross browser testing over 3000+ real browsers hosted by TestMu AI cloud servers. Here are the instant messaging tools that you can integrate with TestMu AI. -- [Team Management](https://www.testmuai.com/support/docs/team-management.md): This documentation will help you create and manage teams on TestMu AI. -- [TeamCity Integration With TestMu AI](https://www.testmuai.com/support/docs/teamcity-integration-with-testmu.md): TestMu AI now integrates with TeamCity to boost your go-to market delivery. Perform automated cross browser testing with TestMu AI to ensure your development code renders seamlessly through an online Selenium grid providing 3000+ real browsers running through machines. -- [TeamCity Plugin Integration](https://www.testmuai.com/support/docs/teamcity-plugin.md): Learn how to execute tests directly on the TestMu AI platform, and test your local, dev, or stage environments securely using the TestMu AI TeamCity plugin. -- [Teamwork Integration With TestMu AI](https://www.testmuai.com/support/docs/teamwork-integration.md): TestMu AI helps you to push bugs directly in Teamwork project with details like task list, assignee, title and description with a single click. -- [Terminal Logs for Appium Tests](https://www.testmuai.com/support/docs/appium-terminal-logs.md): Upload your own Appium server, test-runner, or CI logs to any app automation session and view or download them from the TestMu AI dashboard under Logs > Terminal. -- [Test Case Insights](https://www.testmuai.com/support/docs/analytics-test-case-insights.md): Gain insights into test case execution and performance with TestMu AI's analytics to optimize your testing strategy and enhance software quality. -- [Test Case Repository](https://www.testmuai.com/support/docs/test-case-repository.md): Manage your test case repository with TestMu AI's Test Manager, ensuring organized, accessible, and efficient handling of all your test cases. -- [Test Case Versioning](https://www.testmuai.com/support/docs/test-case-versioning.md): Learn efficient manual test case versioning with TestMu AI's Test Manager. -- [Test Data Generation](https://www.testmuai.com/support/docs/test-data-generation.md): Generate synthetic test data, mask sensitive payloads in network logs and recordings, and parameterize tests across environments on TestMu AI. -- [Test Execution Setup For Running Puppeteer Tests](https://www.testmuai.com/support/docs/puppeteer-test-execution-setup.md): Learn how to configure the desired capability for selecting browsers and OS, organzing tests, changing desktop resolution, and more for your Puppeteer tests. -- [Test in IE Mode on Microsoft Edge](https://www.testmuai.com/support/docs/ie-mode-on-edge.md): Run Selenium tests in Internet Explorer compatibility mode on Microsoft Edge for legacy app testing on TestMu AI. -- [Test Insights - Advanced insights for analyzing test runs](https://www.testmuai.com/support/docs/analytics-test-insights.md): Analytics - Tests Insights for analyzing the past results with the latest test runs -- [Test Instance Audit Logs](https://www.testmuai.com/support/docs/test-instance-audit-logs.md): See every thing you update in a test instance through Audit Logs. -- [Test Intelligence - Command Error Logs Analytics](https://www.testmuai.com/support/docs/test-intelligence-command-error-logs-analytics.md): Analyze your failed tests with Test Intelligence and get insights on the errors that are causing your test runs to fail. -- [Test Intelligence - Flaky Test Detection](https://www.testmuai.com/support/docs/test-intelligence-flakiness-test-detection.md): Flaky Test Detection helps you to identify the flaky tests in your test suite. It provides you with the list of tests that are flaky and the insights over the flakiness of the test. -- [Test Locally Hosted Web Pages](https://www.testmuai.com/support/docs/testing-locally-hosted-pages.md): TestMu AI Tunnel securely connects your local machine to cloud-based real devices for testing locally hosted web apps, with multiple connectivity options like TCP or Websocket. -- [Test Locally Hosted Websites With Selenium](https://www.testmuai.com/support/docs/test-locally-hosted-websites-with-selenium.md): Test locally hosted or staging web apps on the TestMu AI cloud grid using TestMu AI Tunnel with Selenium. -- [Test Logs](https://www.testmuai.com/support/docs/test-logs.md): Test Logs is a library where you can find logs of all cross browser testing performed by you on TestMu AI. -- [Test Manager](https://www.testmuai.com/support/docs/test-manager.md): Experience Test Manager for streamlined test management and real-time testing. Elevate your testing efficiency with various features and insights. -- [Test Manager Dashboard Widgets](https://www.testmuai.com/support/docs/analytics-modules-test-manager-widgets.md): Discover TestMu AI's Test Manager Analytics widgets for comprehensive test cases insights. Optimize your testing efforts today. -- [Test Manager Integration](https://www.testmuai.com/support/docs/kane-cli-tms-integration.md): Kane CLI uploads every session to TestMu AI Test Manager as a test case. Learn about the upload pipeline, project and folder selection, code export, share links, and session history. -- [Test Muting](https://www.testmuai.com/support/docs/hyperexecute-test-muting.md): Understand how to mute tests on scenario level in HyperExecute. -- [Test On Different Screen Resolutions In Cypress](https://www.testmuai.com/support/docs/screen-resolution-testing-with-cypress.md): Run your Cypress automation scripts across different screen resolutions on TestMu AI. -- [Test Overview Widgets](https://www.testmuai.com/support/docs/analytics-widgets-tests-overview-landing.md): Unlock valuable insights with TestMu AI Analytics Overview, providing a detailed analysis of your testing data to optimize and improve your testing processes. -- [Test Run - Creation and Management](https://www.testmuai.com/support/docs/test-run-creation-and-management.md): Gain insights into effective Test Run Creation with TestMu AI, designed to streamline your workflow. -- [Test Run Instance View](https://www.testmuai.com/support/docs/kaneai-test-run-instance-view.md): Learn how to use the new Test Run Instance view in KaneAI to replay execution steps, compare screenshots, inspect command logs, and debug test failures with AI-powered root cause analysis. -- [Test Scheduling - Sitemap (Overview)](https://www.testmuai.com/support/docs/accessibility-test-scheduling.md): Overview of scheduled accessibility scans for websites, including sitemap-based and crawler-based workflows. -- [Test Setup And Execution](https://www.testmuai.com/support/docs/test-setup-and-execution.md): This documentation guides you how to configure the desired capability for app, device, group tests, debugging, and mark tests as passed or failed. -- [Test Splitting and Multiplexing](https://www.testmuai.com/support/docs/hyperexecute-test-splitting-and-multiplexing.md): Boost testing efficiency with HyperExecute's test splitting and multiplexing in TestMu AI's documentation. -- [Test.md](https://www.testmuai.com/support/docs/kane-cli-testmd.md): Write browser tests as Markdown files that replay from cache after the first run: no LLM cost, faster execution. Commit tests to git, share recordings, and run in CI. -- [TestComplete](https://www.testmuai.com/support/docs/testcomplete-integration.md): TestMu AI offers integration with TestComplete to help you perform web and app automation testing on 3000+ real browsers, devices, and operating systems combinations. -- [Testim Integration](https://www.testmuai.com/support/docs/testim-integration.md): TestMu AI offers integration with Testim to help you perform automated browser testing on 3000+ real browsers and OS. -- [Testing Locally Hosted Apps For App Automation](https://www.testmuai.com/support/docs/testing-locally-hosted-apps.md): TestMu AI tunnel allows you to test locally and privately hosted apps across various real Android and iOS devices. In this documentation, learn how to configure TestMu AI tunnel to test locally or privately hosted apps while running app automation. -- [Testing with MockWebServer & Localhost](https://www.testmuai.com/support/docs/espresso-mockwebserver-localhost.md): Test applications using MockWebServer or localhost-based mock servers with Espresso on TestMu AI real devices. -- [TestMu AI - Test Insights for visualizing your tests runs](https://www.testmuai.com/support/docs/analytics-overview.md): Optimize test management with Test Insights. Visualize test runs, track progress, and improve quality with actionable analytics. Enhance testing efficiency. -- [TestMu AI Advanced Capabilities](https://www.testmuai.com/support/docs/testmu-selenium-advance-capabilities.md): Configure advanced TestMu AI capabilities for video, network, console logging, tunnels, and more. -- [TestMu AI Agent Skills for Test Automation Frameworks](https://www.testmuai.com/support/docs/agent-skills.md): Install TestMu AI Agent Skills so your AI coding assistant writes and runs Selenium, Cypress, Playwright, Appium, and 70+ framework tests on the cloud. -- [TestMu AI Analytics - Customizing Your Dashboard with the New Widget Management Feature](https://www.testmuai.com/support/docs/analytics-dashboard-edit-layout.md): Enhance Your Analytics Dashboard Edit Layout with TestMu AI's Widget Management Feature Customizing Layout for Improved Insights -- [TestMu AI Azure DevOps App](https://www.testmuai.com/support/docs/testmu-azure-devops-app.md): Efficiently manage your tests with TestMu AI Azure DevOps App, seamlessly integrating Test Manager capabilities into your Azure DevOps environment to enhance collaboration. -- [TestMu AI Capability Map to Migrate from BrowserStack & SauceLabs](https://www.testmuai.com/support/docs/testmu-capability-map.md): A quick guide on TestMu AI Capability Map to Migrate from BrowserStack & SauceLabs. -- [TestMu AI Chrome Extension](https://www.testmuai.com/support/docs/chrome-extension.md): This documentation will help you in configuring TestMu AI Chrome Extension to perform live-interactive, screenshot tests, geolocation tests and even more. -- [TestMu AI Docker Tunnel Extension For Local Testing](https://www.testmuai.com/support/docs/docker-tunnel-extension.md): Avoid the hassle of launching tunnels via the CLI. TestMu AI Docker Tunnel Extension lets you test locally hosted web applications across 3000+ browsers, devices, and operating systems. -- [TestMu AI Integration With Airtable](https://www.testmuai.com/support/docs/airtable-integration.md): Using TestMu AI integration with Airtable, you'll be able to log bugs over your project in Airtable from TestMu AI in just a single click. You can capture a screenshot of the bug, highlight the anomaly, annotate extra information, choose an assignee, provide a suitable title and description to help relay further actions. -- [TestMu AI Integration With Backlog](https://www.testmuai.com/support/docs/backlog-integration-with-testmu.md): TestMu AI integration with Backlog will help you file any UI bug on your Backlog project instance, directly from TestMu AI. If you ever come across a browser compatibility bug while cross browser testing on TestMu AI, all you need to do is hit a single click and the bug will be logged into your Backlog project. -- [TestMu AI Integration with Favro](https://www.testmuai.com/support/docs/favro-integration.md): The TestMu AI Favro Integration allows you to create a task directly to your specified Favro dashboard from TestMu AI platform. Share your UI observations and input with your teammates anytime, by capturing a screenshot, even in the middle of your test session. -- [TestMu AI Integration With Jenkins](https://www.testmuai.com/support/docs/jenkins-with-testmu.md): TestMu AI now integrates with Jenkins to boost your go-to market delivery. Perform automated cross browser testing with TestMu AI to ensure your development code renders seamlessly through an online Selenium grid providing 3000+ real browsers running through machines. -- [TestMu AI Integration with Pivotal Tracker](https://www.testmuai.com/support/docs/pivotal-tracker-integration.md): TestMu AI provides integrations with Pivotal Tracker to help you log bugs directly from the middle of your test session on TestMu AI to your Pivotal Tracker. -- [TestMu AI Integrations With CI/CD Tools](https://www.testmuai.com/support/docs/integrations-with-ci-cd-tools.md): TestMu AI offers integrations with numerous CI/CD tools to help you fast-track your robust delivery pipeline for pacing up your go-to-market launch. -- [TestMu AI Jira App](https://www.testmuai.com/support/docs/testmu-jira-app.md): Efficiently manage your tests with TestMu AI Jira App, seamlessly integrating Test Manager capabilities into your Jira environment to enhance collaboration. -- [TestMu AI Public IP Ranges](https://www.testmuai.com/support/docs/testmu-public-ip.md): Learn which IP addresses needs to be whitelisted before running your tests on TestMu AI grid and its other products -- [TestMu AI Role-Based Access Control (RBAC)](https://www.testmuai.com/support/docs/role-based-access-control.md): This documentation will help you in creating the different teams and allot concurrency to those teams to perform test automation easily. -- [TestMu AI Selenium Java SDK - Zero Code Changes](https://www.testmuai.com/support/docs/selenium-java-sdk.md): Run Selenium TestNG tests on TestMu AI cloud grid with zero code changes using YAML config and Maven. -- [TestMu AI SSO & Azure AD Integration](https://www.testmuai.com/support/docs/sso-azure-integration.md): This document will help you integrate your TestMu AI organization with your Azure AD directory to configure a Single sign-on for your organization. -- [TestMu AI SSO & Google Custom Using SAML](https://www.testmuai.com/support/docs/testmu-sso-google.md): Now integrate your Google Workspace user directory with your TestMu AI organization subscription through an IDP, so that your team could leverage the SSO setup. -- [TestMu AI SSO & Okta Integration](https://www.testmuai.com/support/docs/testmu-sso-okta-integration.md): TestMu AI allows you to integrate an external user directory with your TestMu AI Organization through an IDP so your team could leverage the Single Sign-on setup. This document will help you integrate your TestMu AI organization with your Okta directory. -- [TestMu AI SSO & OneLogin Using SAML](https://www.testmuai.com/support/docs/testmu-sso-onelogin-using-saml.md): Now integrate your OneLogin user directory with your TestMu AI organization subscription through an IDP, so that your team could leverage the SSO setup. -- [TestMu AI SSO & PingIdentity Integration](https://www.testmuai.com/support/docs/testmu-sso-pingidentity-integration.md): Integrate TestMu AI with Ping Identity for secure SSO using SAML. Follow step-by-step instructions to configure SP-initiated, IdP-initiated SSO, and group management. -- [TestMu AI SSO Integration Using OIDC](https://www.testmuai.com/support/docs/testmu-sso-oidc.md): Integrate your identity provider with your TestMu AI organization through OIDC (OpenID Connect) for seamless single sign-on authentication. -- [TestMu AI Support and Knowledge Base](https://www.testmuai.com/support/docs/support.md): Discover answers, explore products, and connect with our team for assistance with the help of TestMu AI Support documentation. -- [TestMu AI Tunnel Modifiers](https://www.testmuai.com/support/docs/lambda-tunnel-modifiers.md): This document will help you understand all the modifiers/arguments used with TestMu AI Tunnel to help you test your locally hosted web pages. -- [TestMu AI VS Code Extension For Cross Browser Testing](https://www.testmuai.com/support/docs/vs-code-extension.md): With TestMu AI VS Code Extension, perform real-time cross browser testing of your public and locally hosted web applications across 3000+ real, browsers, devices and OS combinations right from the VS Code Editor. -- [TestMu AI WordPress Plugin For Browser Testing](https://www.testmuai.com/support/docs/wordpress-plugin.md): With TestMu AI plugin for WordPress, you can generate automated screenshots of your web pages and post directly from your WordPress platform. -- [TestMu Cloud GitHub App Integration](https://www.testmuai.com/support/docs/github-app-integration.md): Transform every pull request into a validation surface with AI-powered test generation, execution on HyperExecute, and Root Cause Analysis, all without leaving GitHub. -- [TestNG](https://www.testmuai.com/support/docs/accessibility-testng-test.md): Run Accessibility Automation with Selenium and TestNG: capabilities, hooks, TestNG suite layout, and how to review reports in the dashboard. -- [testRigor Integration](https://www.testmuai.com/support/docs/testrigor-integration.md): Integrate TestMu AI with testRigor to perform desktop web, mobile web and native app testing across 3000+ real browsers, devices, and operating systems. -- [Testsigma Integration With HyperExecute](https://www.testmuai.com/support/docs/hyperexecute-testsigma-integration.md): Explore seamless integrations with popular tools at TestMu AI. Enhance your testing workflow for faster, efficient cross-browser testing. -- [Text Spacing (1.4.12)](https://www.testmuai.com/support/docs/accessibility-web-rule-1-4-12-text-spacing.md): Rule-level Accessibility guidance for Text Spacing (WCAG 1.4.12 Level AA). -- [Text Truncation Issues](https://www.testmuai.com/support/docs/accessibility-ios-rule-text-truncation-issues.md): Rule-level Accessibility guidance for Text Truncation Issues on iOS. -- [Textual (DOM) Assertions](https://www.testmuai.com/support/docs/kane-cli-checkpoint-textual.md): Extract data from the DOM (element states, attributes, and computed styles) and assert on values that may not be visible in a screenshot. -- [The Assurance Lifecycle](https://www.testmuai.com/support/docs/kane-cli-assurance.md): Go from requirement documents to designed, runnable tests with the kane-cli assurance commands - every test permanently linked to the requirement it verifies, coverage measured from sealed evidence, and the suite reconciled as your product changes. -- [The Generate Workflow](https://www.testmuai.com/support/docs/kane-cli-generate-workflow.md): Walk the kane-cli generate loop end to end: generate, refine in plain language, save functional cases as _test.md files, and run them with testmd. Includes worked examples, agent/CI automation, and exit codes. -- [Three Flashes or Below Threshold (2.3.1)](https://www.testmuai.com/support/docs/accessibility-web-rule-2-3-1-three-flashes.md): Rule-level Accessibility guidance for Three Flashes or Below Threshold (WCAG 2.3.1 Level A). -- [Time Insights](https://www.testmuai.com/support/docs/analytics-test-time-trends.md): Monitor and optimize test environment setup times with TestMu AI's Setup Time Insights module. Analyze trends, compare across platforms, and receive AI-driven optimization recommendations -- [Timeouts - Errors and Resolutions](https://www.testmuai.com/support/docs/timeouts-issues-and-resolutions.md): Resolve timeout errors in Selenium tests caused by idle sessions, network latency, queuing, and firewalls. -- [Timing Adjustable (2.2.1)](https://www.testmuai.com/support/docs/accessibility-web-rule-2-2-1-timing-adjustable.md): Rule-level Accessibility guidance for Timing Adjustable (WCAG 2.2.1 Level A). -- [Title Assertions](https://www.testmuai.com/support/docs/kane-cli-checkpoint-title.md): Verify the browser tab document.title, useful for confirming navigation reached the expected page. -- [Tosca](https://www.testmuai.com/support/docs/tosca-integration-with-hyperexecute.md): Integrate Tricentis Tosca with TestMu AI HyperExecute for efficient, no-code testing. Seamlessly connect Tosca Commander, DEX, and SAP for an optimized testing workflow. -- [Tosca Integration for SAP](https://www.testmuai.com/support/docs/tosca-integration-with-hyperexecute-for-sap.md): Master seamless integration of Tosca with HyperExecute using our detailed guide. Enhance your testing strategy for faster, more efficient automation results. -- [Tosca Integration using Commander](https://www.testmuai.com/support/docs/tosca-integration-with-hyperexecute-using-commander.md): This documentation provides a detailed guide on seamlessly integrating Tosca with HyperExecute -- [Tosca Integration using DEX](https://www.testmuai.com/support/docs/tosca-integration-with-hyperexecute-using-dex.md): This documentation provides a detailed guide on seamlessly integrating Tosca with HyperExecute using DEX -- [TOTP Authentication in KaneAI](https://www.testmuai.com/support/docs/kaneai-totp.md): This documentation will help you to understand how to use the TOTP Authentication feature while authoring your test cases via KaneAI. -- [Touch Target Sizing](https://www.testmuai.com/support/docs/accessibility-ios-rule-touch-target-sizing.md): Rule-level Accessibility guidance for Touch Target Sizing. -- [Track Bugs and Issues in Test Runs](https://www.testmuai.com/support/docs/track-issues-in-test-runs.md): Connect failing tests to tracked defects by linking Jira and Azure DevOps issues to test instances in a Test Manager Test Run, and view every issue raised across the run. -- [Travis CI Integration](https://www.testmuai.com/support/docs/travis-ci-with-hyperexecute.md): HyperExecute integrates with Travis CI Pipeline to boost your go-to market delivery. Perform automated cross browser testing with TestMu AI to ensure your development code renders seamlessly through an online Selenium grid providing 3000+ real browsers running through machines. -- [Travis CI Pipeline Integration with SmartUI](https://www.testmuai.com/support/docs/smartui-with-travis-ci.md): SmartUI now integrates with Travis CI Pipeline to boost pipeline delivery. Perform automated cross browser testing with SmartUI to seamlessly providing 3000+ real browsers running through machines. -- [Tricentis Tosca Integration With TestMu AI's Cloud Based Selenium Grid](https://www.testmuai.com/support/docs/tricentis-tosca-integration-testmu-selenium-grid.md): Now you can run your Tosca automation tests on TestMu AI online grid of 3000+ real desktop browsers and real operating systems. -- [Trigger iOS Shake Gesture on Mobile Devices Using Appium](https://www.testmuai.com/support/docs/ios-shake-gesture-appautomation.md): Simulate a shake gesture on iOS real devices and simulators during Appium automation tests on TestMu AI using the lambda_executor hook. -- [Trigger Your Job via Local Selenium Web Driver](https://www.testmuai.com/support/docs/hyperexecute-how-to-run-tests-using-local-selenium-drivers.md): How to Run Tests on HyperExecute using Local Selenium Web Driver -- [Troubleshoot Your Espresso Tests](https://www.testmuai.com/support/docs/troubleshoot-espresso-tests.md): Learn how to troubleshoot Espresso tests for your mobile applications to resolve different kinds of bugs for your failed test builds. -- [Troubleshooting](https://www.testmuai.com/support/docs/kane-cli-troubleshooting.md): Fix common Kane CLI issues: Chrome launch failures, authentication errors, run timeouts, variables not resolving, upload failures, and Agent Mode output problems. -- [Tunnel - TestMu AI Browser Cloud](https://www.testmuai.com/support/docs/browser-cloud-tunnel.md): Access localhost and internal networks from cloud browsers in TestMu AI Browser Cloud. -- [Tunnel Client Release Notes](https://www.testmuai.com/support/docs/tunnel-release-notes.md): Get the latest release notes for TestMu AI Tunnel, detailing the latest features, fixes, and improvements for secure and seamless testing. -- [Tunnel Configuration in HyperExecute](https://www.testmuai.com/support/docs/hyperexecute-how-to-configure-tunnel.md): Learn to set up a Tunnel with HyperExecute effortlessly with our straightforward guide, simplifying the configuration process for you! -- [Tunnel Security](https://www.testmuai.com/support/docs/tunnel-security.md): Guide to manage your Tunnel Security -- [Turbo Boost Your Tests Execution with Smart Caching](https://www.testmuai.com/support/docs/hyperexecute-how-smart-caching-boosts-tests-speed.md): Learn how to turbo boost your tests execution using the smart caching concept in hyperexecute -- [UI Inspector](https://www.testmuai.com/support/docs/ui-inspector.md): Inspect and interact with mobile app elements using TestMu AI UI Inspector. Visualize the DOM, identify elements, and generate automation Xpaths for efficient testing. -- [UI Testing with C# using HyperExecute and SmartUI SDK](https://www.testmuai.com/support/docs/hyperexecute-smart-ui-sdk-selenium-csharp.md): Automate Hyperexecute tests seamlessly! Integrate SmartUI SDK with CSharp for efficient & reliable UI testing. -- [UI Testing with Cypress using HyperExecute and SmartUI SDK](https://www.testmuai.com/support/docs/hyperexecute-smart-ui-sdk-using-cypress.md): Automate Hyperexecute tests seamlessly! Integrate SmartUI SDK with Cypress for efficient & reliable UI testing. -- [UI Testing with Maestro using HyperExecute and SmartUI SDK](https://www.testmuai.com/support/docs/hyperexecute-smart-ui-sdk-maestro.md): Use the Maestro framework to run visual tests on Android apps via HyperExecute and SmartUI. Capture screenshots for visual comparison in mobile test automation. -- [UI Testing with Node.js using HyperExecute and SmartUI SDK](https://www.testmuai.com/support/docs/hyperexecute-smart-ui-sdk-selenium-javascript.md): Effortlessly automate Hyperexecute UIs with SmartUI SDK & Selenium JavaScript. Write modular, cross-browser tests. -- [UI Testing with Playwright using HyperExecute and SmartUI SDK](https://www.testmuai.com/support/docs/hyperexecute-smart-ui-sdk-using-playwright.md): Supercharge Hyperexecute UI automation with SmartUI SDK & Playwright. Simplify test creation for all browsers. -- [UI Testing with Puppeteer using HyperExecute and SmartUI SDK](https://www.testmuai.com/support/docs/hyperexecute-smart-ui-sdk-using-puppeteer.md): Streamline Hyperexecute UI testing with SmartUI SDK and Puppeteer. Write robust, end-to-end tests effortlessly. -- [UiPath Integration](https://www.testmuai.com/support/docs/uipath-integration.md): Discover how TestMu AI and UiPath seamlessly integrate to enhance mobile automation. Dive into our support doc to leverage the power of real device cloud and UiPath. -- [Undersized Touch Target](https://www.testmuai.com/support/docs/accessibility-android-rule-undersized-touch-target.md): Rule-level Accessibility guidance for Undersized Touch Target on Android. -- [Understanding Unique Test Instances and Retry Detection](https://www.testmuai.com/support/docs/analytics-unique-instances-retry-detection.md): Discover how LambdaTest Analytics intelligently detects unique test instances and consolidates retry attempts for precision test reporting -- [Understanding Your TestMu AI Concurrency](https://www.testmuai.com/support/docs/concurrency-widget.md): Learn how your TestMu AI concurrency works and how to understand the concurrency widget that shows your parallel test usage across different plans and test types. -- [Unlabeled Checkbox Element](https://www.testmuai.com/support/docs/accessibility-android-rule-unlabeled-checkbox-element.md): Rule-level Accessibility guidance for Unlabeled Checkbox Element on Android. -- [Unlabeled Toggle Control](https://www.testmuai.com/support/docs/accessibility-android-rule-unlabeled-toggle-control.md): Rule-level Accessibility guidance for Unlabeled Toggle Control on Android. -- [Unleash the Power of JMeter Performance Testing with HyperExecute](https://www.testmuai.com/support/docs/hyperexecute-run-jmeter-tests.md): Perform the steps to learn how to do the performance testing on HyperExecute using the jmeter tests plans -- [Unnamed Nested Element](https://www.testmuai.com/support/docs/accessibility-android-rule-unnamed-nested-element.md): Rule-level Accessibility guidance for Unnamed Nested Element on Android. -- [Update your Accessibility DevTools Chrome Extension](https://www.testmuai.com/support/docs/accessibility-update-devtools-extension.md): Update the TestMu AI Accessibility DevTools Chrome extension to get the latest features and fixes. -- [Upload and Download Files](https://www.testmuai.com/support/docs/real-time-upload-and-download-files.md): Easily upload and download files during remote testing with TestMu AI. Manage cross-platform file transfers for seamless workflows across desktop and mobile devices. -- [Upload and Download Files in KaneAI](https://www.testmuai.com/support/docs/kaneai-upload-and-download-files.md): This documentation will help you to understand how to Upload and Download Files -- [Upload Apps on TestMu AI's Real Device Cloud](https://www.testmuai.com/support/docs/upload-apps-on-real-device-cloud.md): Learn how to upload apps to TestMu AI's Real Device Cloud from your system or a URL for testing on Android and iOS devices. -- [Upload Contacts on Real Devices](https://www.testmuai.com/support/docs/upload-contacts.md): Upload custom .vcf contact files to real Android and iOS devices and inject them into the native Contacts app before your test session begins. -- [Upload File and Media](https://www.testmuai.com/support/docs/upload-media.md): Seamlessly upload media and files on Real Devices during app and browser automation to enhance your testing scenarios and ensure comprehensive validation of your application's functionalities. -- [Upload Files Using TestMu AI Selenium Grid](https://www.testmuai.com/support/docs/upload-files-using-testmu.md): Upload files to the TestMu AI Selenium Grid and use them in automation test scripts across 3000+ browsers. -- [Upload PDFs via API](https://www.testmuai.com/support/docs/smartui-pdf-api-upload.md): Learn how to upload PDF files to SmartUI using the REST API for visual regression testing. -- [Upload PDFs via CLI](https://www.testmuai.com/support/docs/smartui-pdf-cli-upload.md): Learn how to upload PDF files to SmartUI using the command-line interface for visual regression testing. -- [Upload PDFs via Java SDK](https://www.testmuai.com/support/docs/smartui-pdf-java-sdk.md): Learn how to upload PDF files to SmartUI using the Java SDK for programmatic visual regression testing. -- [Upload your Screenshots via CLI](https://www.testmuai.com/support/docs/smartui-cli-upload.md): In this documentation, learn how to upload and compare images (jpgs, jpegs, pngs) using the SmartUI CLI. -- [Uploading app via Firebase](https://www.testmuai.com/support/docs/appium-firebase-app-upload.md): Discover how to test an app uploaded on Firebase seamlessly through Appium at TestMu AI! Our guide simplifies your Appium Firebase app upload process for efficient testing! -- [Uploading/Retrieving files on Real Devices](https://www.testmuai.com/support/docs/appium-uploading-retrieving-files.md): You can use sample data provided by TestMu AI in case you need to check uploading files, contacts etc. to your application. This document also shows how to retrieve files from TestMu AI rela devices to local and vice-versa. -- [URL Assertions](https://www.testmuai.com/support/docs/kane-cli-checkpoint-url.md): Check values in the browser address bar: current URL path, query parameters, fragments, and redirect targets. -- [Usage insights for Groups](https://www.testmuai.com/support/docs/analytics-modules-groups.md): Explore TestMu AI's powerful Analytics Modules for comprehensive test intelligence and groups insights. Elevate your testing efficiency today. -- [Usage Report](https://www.testmuai.com/support/docs/insights-usage-report.md): Discover TestMu AI's Usage Report for comprehensive test cases insights. Optimize your testing efforts today. -- [Use Cases of KaneAI](https://www.testmuai.com/support/docs/use-cases-of-kane-ai.md): Discover the key use cases of KaneAI, an AI-powered testing tool that automates test creation and validation across various business domains for enhanced reliability. -- [Use of Color (1.4.1)](https://www.testmuai.com/support/docs/accessibility-web-rule-1-4-1-use-of-color.md): Rule-level Accessibility guidance for Use of Color (WCAG 1.4.1 Level A). -- [User Management](https://www.testmuai.com/support/docs/user-management.md): This documentation will help you create and manage users on TestMu AI. -- [Userback Integration](https://www.testmuai.com/support/docs/userback-integration.md): TestMu AI helps you manage your bugs in your Userback instance in a single click. All the details you provide in TestMu AI like task list, assignee, title and description would automatically be presented in the project in your Userback instance. -- [Using Environment Variables for Authentication Credentials](https://www.testmuai.com/support/docs/using-environment-variables-for-authentication-credentials.md): Set up environment variables for your username and access key on Mac, Linux, and Windows systems. -- [Using Snooper Command on Hyperexecute](https://www.testmuai.com/support/docs/hyperexecute-snooper.md): Learn more about Using Snooper on Hyperexecute -- [Using TestMu AI Tunnel with SmartUI SDK](https://www.testmuai.com/support/docs/smartui-sdk-tunnel.md): Learn how to use TestMu AI Tunnel with SmartUI SDK for testing internal, development, and staging environments -- [Using TestMu AI Underpass | A Desktop Application For Localhost Testing](https://www.testmuai.com/support/docs/underpass-tunnel-application.md): With TestMu AI Underpass, you can perform cross browser testing of locally hosted web applications on 3000+ browsers for both mobile & desktop, without executing any command from your terminal. -- [Utilizing KaneAI Secrets Feature: A Step-by-Step Guide](https://www.testmuai.com/support/docs/kane-ai-secrets.md): Learn how to create, update and use custom secrets from the test manager and while executing your tests using the Kane AI -- [Utilizing KaneAI Smart Variables Feature: A Step-by-Step Guide](https://www.testmuai.com/support/docs/kane-ai-smart-variables.md): Learn how to utilize Smart Variables to dynamically adapt to various environments and conditions -- [Variables & Context](https://www.testmuai.com/support/docs/kane-cli-variables-and-context.md): Use variables to parameterize objectives with secrets and reusable values. Use context files to give the agent project-specific knowledge. -- [Variables & Parameters in KaneAI](https://www.testmuai.com/support/docs/kaneai-variables-and-parameters.md): Details and information about variables & parameters in KaneAI -- [Variables in KaneAI](https://www.testmuai.com/support/docs/kane-ai-using-variables.md): Learn how to use variables in KaneAI test cases -- [Version 0.0.1 - Beta](https://www.testmuai.com/support/docs/kaneai-release-notes-0-0-1.md): Beta Version 0.0.1 -- [Version 0.0.2 - Beta](https://www.testmuai.com/support/docs/kaneai-release-notes-0-0-2.md): Beta Version 0.0.2 -- [Version 0.2.249](https://www.testmuai.com/support/docs/hyperexecute-cli-release-notes-0-2-249.md): Version 0.2.249 -- [Version 1.0.0](https://www.testmuai.com/support/docs/hyperexecute-mcp-server-release-notes-1-0-0.md): Check out the release notes for HyperExecute MCP Server 1.0.0, introducing new features, improvements, and bug fixes for streamlined testing. -- [Video Injection](https://www.testmuai.com/support/docs/video-injection.md): Inject video content in your tests with TestMu AI to simulate video inputs for automated testing, enhancing your app's multimedia capabilities. -- [View Assertion Errors On TestMu AI](https://www.testmuai.com/support/docs/lambda-exceptions.md): Manage and display GET request assertion errors in Selenium scripts using the Lambda Exceptions feature. -- [Virtual Devices](https://www.testmuai.com/support/docs/hyperexecute-integration-with-virtual-devices.md): Maximize virtual device testing with HyperExecute! Accelerate test orchestration and intelligence on real devices with TestMu AI's support. -- [Visual AI Agent for SmartUI](https://www.testmuai.com/support/docs/smartui-visual-ai.md): Revolutionize your visual testing with SmartUI's Visual AI Agent. Detect changes that matter and ignore false positives with intelligent analysis. -- [Visual Assertions](https://www.testmuai.com/support/docs/kane-cli-checkpoint-visual.md): Verify what is visible on the screen by analyzing the current screenshot. Visual is the default analyze method KaneAI uses for assertions. -- [Visual Test Creation with KaneAI](https://www.testmuai.com/support/docs/kaneai-smartui-visual-testing.md): A step-by-step guide on using KaneAI to automate visual regression tests. Learn to capture screenshots, establish baselines, compare visual changes with SmartUI, and schedule automated runs. -- [Visual UI Testing](https://www.testmuai.com/support/docs/visual-ui-testing.md): Discover seamless Visual UI Testing with TestMu AI's Screenshot Feature - automate and compare screenshots across 3000+ browsers for pixel-perfect web experiences. Elevate your testing strategy today!. -- [VNet Peering and VNet Links](https://www.testmuai.com/support/docs/hyperexecute-vnet-peering-for-private-cloud.md): Secure your HyperExecute Private setup with VNet Peering & Links! Our guide helps provision resources, deploy ARM templates, and connect VNets for optimal testing. -- [Voice Agent Testing With TestMu AI](https://www.testmuai.com/support/docs/voice-agent.md): Test voice agents on the TestMu AI Agent Testing Platform. Audio conversations are transcribed and scored across the same 9 quality metrics as chat agents. -- [VoiceOver on macOS](https://www.testmuai.com/support/docs/accessibility-voiceover-macos.md): VoiceOver on macOS for manual accessibility validation: turn on, rotor, web navigation, and checklist. -- [VPAT and ACR evidence (customer-owned templates)](https://www.testmuai.com/support/docs/accessibility-vpat-report-generation.md): TestMu AI does not provide VPAT or ACR reports. Use Accessibility Testing to gather evidence while your organization completes official ITI VPAT or ACR templates separately. -- [Web & App Automation Modules](https://www.testmuai.com/support/docs/analytics-modules-automation-test-overview.md): A detailed guide to understanding the test health and trends module. Explore widgets for Test Status, Browser, OS Distribution, and more. -- [Web Accessibility Checklist](https://www.testmuai.com/support/docs/accessibility-web-what-we-cover.md): Supported WCAG web rules with links to rule pages, plus a manual test checklist for areas that need human review. -- [Web Accessibility Manual Checklist](https://www.testmuai.com/support/docs/accessibility-web-what-we-do-not-cover.md): This page has moved. Web coverage and manual checklist are now on one page. -- [Web Rule Repository](https://www.testmuai.com/support/docs/accessibility-web-rule-repository.md): Browse rule-level web accessibility guidance, WCAG mappings, and remediation references. -- [Web Scanner Overview](https://www.testmuai.com/support/docs/web-scanner-overview.md): An introduction to TestMu AI Web Scanner, its purpose, core features, supported scan types (Visual UI and Accessibility), and ideal users. -- [Web Testing On Real Devices](https://www.testmuai.com/support/docs/web-testing-on-real-devices.md): TestMu AI mobile testing cloud offers a wide range of real Android and iOS devices to test on.This documentation will help you perform mobile web testing of your websites and web apps on TestMu AI real device cloud. -- [Webex Teams Integration](https://www.testmuai.com/support/docs/webex-teams-integration.md): TestMu AI helps you manage your bugs in Webex Teams by moving them to project in a single click. All the details you provide in TestMu AI like task list, assignee, title and description would automatically be presented in the project in your Webex Teams Space. -- [Webhooks Integration With TestMu AI](https://www.testmuai.com/support/docs/webhooks-integration.md): Integrate Webhooks with TestMu AI to receive real-time build data notifications on your specified endpoints. Configure webhook URLs to automate your testing workflows. -- [WebView Testing In Android Online In Real Time](https://www.testmuai.com/support/docs/webview-testing-in-android.md): TestMu AI now allows you to perform your WebView Testing in Real-time over 3000+ combinations of browsers, operating systems, and mobile devices. -- [Welcome to TestMu AI Browser Cloud](https://www.testmuai.com/support/docs/what-is-browser-cloud.md): TestMu AI Browser Cloud is cloud browser infrastructure purpose-built for AI agents, with built-in stealth, session persistence, and full observability. -- [What Is Scroll Sync In LT Browser?](https://www.testmuai.com/support/docs/scroll-sync.md): LT Browser sync scroll allows you to compare your website side-by-side over different devices through mirrored interactions. -- [While Loops in KaneAI](https://www.testmuai.com/support/docs/kaneai-while-loops.md): Learn how to use While Loops in KaneAI to repeat test steps based on dynamic conditions, iterate over collections with JavaScript, and use the built-in {{loop_counter}} variable in natural-language steps and element selectors. -- [Whitelisting Proxy with TestMu AI](https://www.testmuai.com/support/docs/white-listing-proxy-with-testmu.md): Whitelist the TestMu AI domain to connect from a firewall-protected network. -- [Why do we need KaneAI?](https://www.testmuai.com/support/docs/why-we-need-kane-ai.md): Learn why KaneAI is go-to AI testing agent to help you plan, author and evolve e2e automation tests using natural language. -- [Widgets](https://www.testmuai.com/support/docs/analytics-widgets.md): Unlock valuable insights with TestMu AI Analytics Overview, providing a detailed analysis of your testing data to optimize and improve your testing processes. -- [WinAppDriver](https://www.testmuai.com/support/docs/hyperexecute-winapp-integration.md): Unlock enhanced automation testing by seamlessly integrating Excel with HyperExecute using TestMu AI's support documentation. -- [Workflow Scanner](https://www.testmuai.com/support/docs/accessibility-testing-workflow-scanner.md): Enhance website accessibility with Workflow Scan, a tool that records and analyzes real-time interactions to ensure easy navigation for users with disabilities. -- [Workflows in HyperExecute](https://www.testmuai.com/support/docs/hyperexecute-workflows.md): Schedule and orchestrate HyperExecute jobs without a full CI/CD, chain workflows across projects, and parameterize runs with workflow variables. -- [Writing Objectives](https://www.testmuai.com/support/docs/kane-cli-writing-objectives.md): Learn the three objective patterns: actions, assertions, and extractions: and how to write natural language objectives that produce reliable test results. -- [XCResult](https://www.testmuai.com/support/docs/xcresult.md): Learn how to generate and download Xcode Result Bundles (.xcresult) for XCUI test executions on TestMu AI. Debug smarter with detailed reports directly in Xcode. -- [XCTestPlan](https://www.testmuai.com/support/docs/xctestplan.md): Manage iOS, iPadOS, macOS, watchOS, and tvOS tests with XCTestPlan on TestMu AI. Learn how to upload apps, configure tests, and view execution logs efficiently. -- [XCUI Automation using Emulators and Simulators on TestMu AI](https://www.testmuai.com/support/docs/xcui-automation-on-emulators-simulators.md): Learn how to run xcui app automated tests on using Emulators and Simulators on TestMu AI. -- [XCUI Supported Capabilities](https://www.testmuai.com/support/docs/xcui-supported-capabilities.md): Now you can run your XCUI framework on TestMu AI online grid of 3000+ real desktop browsers and real operating systems with its supported capabilities. -- [XCUI Testing on TestMu AI](https://www.testmuai.com/support/docs/getting-started-with-xcuitest.md): Master XCUITest automation with TestMu AI! Learn to run scripts, utilize capabilities, and execute tests on real iOS devices effortlessly. Get started now! -- [XCUI XML Reports](https://www.testmuai.com/support/docs/xcui-report.md): XCUI XML Report -- [YAML FAQs](https://www.testmuai.com/support/docs/hyperexecute-yaml-faqs.md): Get answers to your HyperExecute YAML FAQs. Learn about automation, configurations, integrations, and more for efficient test execution on TestMu AI.. -- [YAML-Based Capability Overrides for Selenium Tests on HyperExecute](https://www.testmuai.com/support/docs/hyperexecute-yaml-capability-overrides.md): Easily override test capabilities using YAML in HyperExecute. No script changes needed. Flexible, scalable, and ideal for dynamic test environments. -- [YouTrack Integration](https://www.testmuai.com/support/docs/youtrack-integration.md): TestMu AI provides YouTrack integration to help you perform one-click bug logging from the TestMu AI to your YouTrack dashboard right away. -- [Zalenium Integration With TestMu AI](https://www.testmuai.com/support/docs/zalenium-integration-with-testmu.md): TestMu AI integration with Zalenium will allow you perform automated cross browser testing for your locally hosted web pages for over 3000+ real browser and browser versions, installed and hosted on-cloud. -- [Zapier Integration With TestMu AI](https://www.testmuai.com/support/docs/zapier-integration-with-testmu.md): TestMu AI integration with Zapier enable you to create a smarter and more customized workflow, helping you save time to focus on the important things to ship quality products at light speed. -- [Zenkit Integration](https://www.testmuai.com/support/docs/zenkit-integration.md): Discover how TestMu AI and Zenkit seamlessly integrate to enhance your testing workflows. Dive into our support doc to leverage the power of both platforms for efficient bug tracking and collaboration. -- [Zephyr Scale Integration](https://www.testmuai.com/support/docs/zephyr-scale-integration.md): Integrate Zephyr Scale with TestMu AI to perform automated browser testing, and view test results directly from the TestMu AI platform to your Zephyr Scale instance. -- [zipBoard Integration](https://www.testmuai.com/support/docs/zipboard-integration.md): This document will help you integrate TestMu AI with zipBoard. That way, you can log bugs to your zipBoard project in a single click as you perform cross browser testing with TestMu AI. -- [Zoho BugTracker Integration](https://www.testmuai.com/support/docs/zoho-bugtracker-integration.md): Integrate TestMu AI with Zoho BugTracker and perform effortless one-click bug logging from TestMu AI platform to Zoho BugTracker projects. Capture screenshots, annotate bugs, and more. -- [Zoho Cliq Integration](https://www.testmuai.com/support/docs/zoho-cliq-integration.md): Seamless Zoho Cliq Integration with TestMu AI Streamline Bug Management Effortlessly. Get real-time bug notifications in your Zoho Cliq Channels. -- [Zoho Project Integration with TestMu AI](https://www.testmuai.com/support/docs/zohoproject-integration.md): TestMu AI provides integration with Zoho Project to help you log bugs directly from your test session on TestMu AI to Zoho Projects. diff --git a/static/docs/load-balancing-in-lambda-tunnel.md b/static/docs/load-balancing-in-lambda-tunnel.md deleted file mode 100644 index 4caf4479c..000000000 --- a/static/docs/load-balancing-in-lambda-tunnel.md +++ /dev/null @@ -1,44 +0,0 @@ -# Parallel Testing Of Locally Hosted Pages Using Load Balancing In TestMu AI Tunnel - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Load Balancing feature allows you to perform parallel testing of your locally hosted or privately hosted website or web app on 3000+ real browsers through TestMu AI Tunnel. The Tunnels created under this feature will act as a single pool for the tests and thereby will help in performing parallel testing on the TestMu AI platform. - -## How To Activate Load Balancing In TestMu AI Tunnel? - -The Load Balancing in [TestMu AI Tunnel](/docs/testing-locally-hosted-pages/) is achieved with the help of load balanced tunnel modifier, used as shown in the below syntax: - -`--load-balanced` - -Now to activate the Load Balancing feature in the TestMu AI Tunnel, create more than one tunnel with the same name and Load Balancing tunnel modifier. It means that, we need to open more than one tunnel binary and use the below command in each of them: - -`LT --user username --key access_key --tunnelName SampleTunnel --load-balanced` - -> -To get your TestMu AI Username and Access Key, visit your TestMu AI automation dashboard and click on the "key" icon, present on the left to the help button, on the top right corner. - -For example, in the below screenshot, you can see two tunnels activated in the Load Balanced mode, using the above syntax: - -Now when you will go to you TestMu AI dashboard, you will find both the tunnels active with same name, stating that the tunnels are load balanced. - -## Parallel Testing Using Load Balanced TestMu AI Tunnel - -After activating the load balanced tunnels, all you need to do is to pass the "tunnel" capability as true in your automation test script. This will automatically run your tests in parallel by taking advantage of the Load Balancing. - -> -Please note that you do not need to set the "tunnelName" capability as the tunnel name will be automatically managed from active tunnels in your profile. - -For example, here is the Automation Logs view of the test run using the Load Balancing feature of TestMu AI Tunnel. - -As you can see, 10 tests have been executed in Parallel in 8.33 minutes due to the advantage of Load Balanced TestMu AI Tunnel, which otherwise would have taken much longer. - -## How To Stop The Active Load Balanced Tunnels? - -You can stop the active load balanced TestMu AI Tunnels with the help of Tunnel sessions info. To do so, first you need to click on View All button in the Active Tunnel list, as shown below: - -Then in the Tunnel Sessions info, you can see all the active tunnels with their information, active state and a Stop button to stop the corresponding active tunnel. - -Just press on this Stop button against the tunnel you want to stop, and it will end the tunnel session right away. - -> -That’s all! You can now successfully activate and use the Load Balancing feature of TestMu AI Tunnel, to test your locally hosted website in parallel. If you still have any questions, feel free to share them with us through our **24*7 Chat Portal** or by mailing to us on [**support@testmuai.com**](mailto:support@testmuai.com). Happy testing! diff --git a/static/docs/local-testing-for-linux.md b/static/docs/local-testing-for-linux.md deleted file mode 100644 index 002ca40c5..000000000 --- a/static/docs/local-testing-for-linux.md +++ /dev/null @@ -1,158 +0,0 @@ -# Legacy Binary For Linux - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI tunnel feature allows you to test your private server URLs or locally hosted web-apps or websites through TestMu AI cloud servers. Through our secure shell tunneling features, you can also test plain HTML, CSS, PHP, Python or other similar web files saved on your local system, on combinations of operating systems, browsers, and screen resolutions that are available on TestMu AI. - -Feel more confident than ever, before deploying any UI specific change into Production. Test your development thoroughly with Secure TCP with TLS 1.2 network protocol hosted by TestMu AI tunnel. Experience the flexibility to securely test your website on 3000+ browsers and their assorted versions, running on real OS code for cross platform and [cross browser compatibility](https://www.testmuai.com/feature), even before it gets deployed over production environment. - -> Note: From November 2, 2020 onwards, we have deprecated our existing tunnel binary. In order to setup the new version, you can refer to [new tunnel binary for Linux](/docs/local-testing-linux/). - -You can download the binary file responsible for establishing a TCP with TLS 1.2 connection between your local device and our testing cloud by clicking on the downloadable link below. - -Download TestMu AI Tunnel For Linux - -## Configuring TCP with TLS 1.2 connection with TestMu AI tunnel - -**Step 1**: Click the button "**Configure Tunnel**". Download the binary zip file by clicking the "**Download Link**". Also, hit the 'Copy' button in the form that opens up to copy the complete string to your system's clipboard. - -**What is this zip file for?** - -This zip file will help in establishing a secure connection to TestMu AI cloud servers so you could test your locally hosted web pages. - -**Step 2**: Extract the downloaded zip file. Go to terminal and route your pointer to the same directory where you extracted the zip file. - -**Step 3**: Paste the copied string to execute the downloaded binary file. The command will look like: - -`LT --user {user's login email} --key {user's access key} --cui` - -So for example, if your user login email is example@lambdatest.com and your user key is 123asd123, the command would be: - -`LT --user example@lambdatest.com --key 123asd123 --cui` - ->**Note:** After you execute the command, you may be asked for user permission. Make sure you have administrator access to allow the file to run successfully. - -In a few seconds, after you execute the command, an TCP with TLS 1.2 connection will be established. You will receive a message inside your command prompt mentioning "***Secure connection established, you may start your tests now***". - -## Testing Locally Hosted Web Pages Using TestMu AI tunnel - -**Step 1**: Navigate to '**Real Time Testing**' menu and enter the local host URL you want to test in the text field provided to enter URL. - -**Step 2**: Select the tunnel via which you want to run the test. - -**Step 3**: Select the test configuration of your choice. You can select from various major browsers & their assorted versions to perform a test session. You may also choose from a variety of Operating System & screen resolutions to check the **RWD(Responsive Web Design)** of the website when rendering on different resolutions. - -**Step 4:** Make sure you turn on **XAMPP** or any other web hosting tool to provide you an Apache test environment. If you are using XAMPP, hit ‘**Start**‘ for Apache under ‘**Services**‘. If you also want to test the back-end functionality for your website then make sure you hit the ‘**Start**‘ button for **MySQL** as well. - -**Step 5:** Once you are done selecting your configurations and hosting a web server. Hit the ‘Start’ button in Real Time Test. Your test will start and you will be navigated to your localhost URL. - ->**Important Note:** Some Safari browsers, on both Mac machines and iOS browsers, doesn’t support automatic resolution of URL string "localhost". Therefore if you test on URLs like "`http://localhost/`" or "`http://localhost:8080`" etc, you would get an error in these browsers. A possible solution is to replace the string "`localhost`" with machine IP address. For example if you wanted to test "`http://localhost/dashboard`" or "`http://localhost/dashboard:8080`", and your machine IP is 192.168.2.6 you can instead test on "`http://192.168.2.6/dashboard`" or "`http://192.168.2.6/dashboard:8080`". - -## Testing Local Folder Files Using TestMu AI tunnel - ->**Note:** If you were already running a test session through tunnel then you will have to terminate the existing TCP with TLS 1.2 connection in cmd before you re-establish it for a new directory. - -**Step 1:** If you want to test the local folders and files stored in your system, pass an additional argument ‘dir’ with the directory/folder in double quotes name as you run the command on the terminal to establish a TCP with TLS 1.2 connection with TestMu AI tunnel. - -**Sample format:** - -`LT --user {user login email} --key {user key} --dir {"directory/folder name"}` - -**Example:** Let us consider your user login email to be "example@lambdatest.com" with your user key as ‘123asd123’ and you want to test files in the ‘Downloads’ folder, then the command would be: - -`LT --user example@lambdatest.com --key 123asd123 --dir "/home/lambda/Downloads"` - -**Step 2:** After you establish a TCP with TLS 1.2 connection, go to TestMu AI and navigate to **Real Time test**. Enter local-folder.lambdatest.com" in URL bar. The list of all files/folders present under the selected directory will be displayed in the browser. Select the test configuration(OS, Browser and screen resolution) of your choice. - -**Step 3:** Hit the ‘Start’ button. Once the VM(Virtual Machine) is launched, you will be able to access your local folders on a testing environment hosted by our cloud servers. - -> **Note:** Similarly, if you want to run any file in the specified directory, you need to give URL as local-folder.lambdatest.com/filename. - -For example: If you have a file ‘test.html’ in your folder. You need to give URL as "local-folder.lambdatest.com/test.html" to run the same file. In this way, you can run any HTML, PHP, Python or similar web files present in any folder of your system, directly in TestMu AI browser. - -## Terminating The TCP with TLS 1.2 Connection - -A secure tunnel connection is maintained unless you explicitly disconnect it. You can disconnect the tunnel in following ways: - -**Option 1:** You can kill the process by using Ctrl+C in the terminal. - -You will receive the below highlighted messages indicating the termination on TCP with TLS 1.2 Connection. - -**Option 2:** You can also disconnect it using the option provided in the user interface. In the top right corner of TestMu AI interface, you will find a button that says "**Tunnel active**:". This button displays the number of active tunnels. As you click on it, you will find your tunnel name along with an 'X'. Hit the 'x' to terminate the TCP with TLS 1.2 tunnel. - -## Geolocation Testing Through VPN with TestMu AI tunnel - -Test the performance of your web-app through different geographic locations using VPN in TestMu AI tunnel. Make sure your website delivers well to your targeted audience from all around the globe. - -Use the following arguments in your command line. - -| ARGUMENTS | EXAMPLES | DESCRIPTION | -|---------|-------|-------------| -| --key | 123asd123 | Helps Lambdatest to map a secure shell connection with the right user. | -| --proxy-host | abc.wonderproxy.com | Full hostname for the proxy you’d like to use. | -| --proxy-port | 12345 | The port declared by you for fetching the proxy. | -| --proxy-user | abcdefg | WonderProxy username of the user. | -| --proxy-pass | hijklmnop | WonderProxy password of the user. | - -Here is how the format of full command would look like: - -`LT --user {user login email} --key {user key} --proxy-host {full proxy hostname} --proxy-user {proxy username} --proxy-pass {proxy password} --proxy-port {port number}` - -Example of full command: - -`LT --user example@lambdatest.com --key 123asd123 --proxy-host abc.wonderproxy.com --proxy-user abcdefg --proxy-pass hijklmnop --proxy-port 12345` - -### How Can Geolocation Testing Help You? - -**1. SERPs**: Search Engine Result Pages: For a website that is targeting audience from different parts of globes. It is always recommended to have a constant check on SERPs to address you website according to the respective trends of different geographies. This is extremely necessary for clothing brands. Traditional wearing differs from one country to another. - -**2. GDPR compliance**: General Data Protection Rule Compliance: Payment Gateways may differ from one geography to another due to different GDPR compliance for different geographies. - -**3. Billing**: Your billing page should support currency conversion with respect to the IP. So if a user from US would show up at your website then he/she should be billed in USD, similarly if one is visiting from France then the bill should be presented in Euros. - -**4. Google Ads**: They also differ from one geographic region to another. You need to check through IPs of different geolocations regarding which ads are visible and in what manner? - -**5. Bandwidth**: 4G may or may not be available to every one of your targeted geography and even if it is available. The available bandwidth will usually be different. Meaning some countries would provide a faster download speed in 4G as compared to others. So you need to be mindful about the content optimization according to your targeted geolocations. Testing from IPs belonging to different geolocations will help you get a realistic view for page loading speed. A faster page loading will not only help in pleasing your customers, it would also help in ranking higher in SERPs. - -### How Can You Test From An IP Belonging To A Different Geolocation? - -This is where a VPN comes to rescue! VPN – Virtual Private Network – It helps in expanding a private network through a public network. Initially VPNs were introduced and used among large enterprises, defense organization, banking companies etc. to keep their data secure. A VPN created a pathway through which people working in large organizations could access the data they want, remotely, in a secured manner. The data is securely transmitted through the process of Cryptography. In Cryptography, the data is encrypted from the sender’s end and then it gets decrypted with the help of a unique key on the receivers end. - -Time have changed and with it the popularity of VPN too! These days VPNs are so widely available that they are used by almost everyone of us. Some use it to surf the internet without worrying about the restrictions applied for their GeoIP. Some use it for the purpose of maintaining anonymity. Some use it without even realizing the meaning of a VPN. - -Remember though, not all VPNs will work in the same manner. Few may provide you considerably more geolocations to choose from as compared to others. Few may provide less number of geolocations but more number of dynamic GeoIPs with respect to others. There are many more parameters related to comparison. - -## Command References - -Following is the list of command arguments which can be used as modifiers while executing the binary file: - -| COMMAND | DESCRIPTION | -|---------|-------------| -| --key | Access key of the user. | -| --port | (optional) Local port to connect tunnel. | -| --proxy-host | Full hostname for the proxy you’d like to use. | -| --proxy-pass | WonderProxy password of the user. | -| --proxy-port | The port declared by you for fetching the proxy. | -| --proxy-user | WonderProxy username of the user. | -| --tunnelName | (optional) Name of the tunnel. | -| --user | Registered email ID of the user. | - -## Changing Your Access Key - -**What is an Access Key?** - -An Access Key is a unique string used for the identification of your account. - -As we offer a cloud-based platform to host a testing ecosystem to everyone around the globe. Access key plays a crucial role in mapping a Secure Shell (SSH) environment for hosting your locally stored web-pages online. They work as your IDs as you gain access to our cloud servers. - -If you ever misplace your Access key into the hands of someone you don’t trust then you can always regenerate it by clicking the button "**Regenerate Access Key**". - -Access Key is pivotal for establishing a TCP with TLS 1.2 connection through TestMu AI tunnel to test your locally hosted web-pages or web-apps. - -Find your Access Key while performing any type of testing, from the left navigation menu. You will find a button that says "Configure Tunnel". Click on that button & a string containing your access key will be presented. - -For instance, if you go to **Real-Time Test**, you will observe a button that says "**Configure Tunnel**" on the top-right area of the page. Click on that button and a form would open up through it, providing you with a string containing your access key. You will find the access key in the area underlined below. - -You can also find your **Access Key** by going to ‘**Settings**’ in the left navigation menu and routing to ‘Profile’. - ->**TestMu AI tunnel Update:** Earlier, when our TestMu AI tunnel binary was updated with a new version, we requested our users to re-download the binary file for uninterrupted [cross browser testing](https://www.testmuai.com/) experience of locally hosted web pages. Not anymore! We have now updated the TestMu AI tunnel binary in a way that it auto-updates to the latest version every time you execute the TestMu AI tunnel from cmd/terminal. diff --git a/static/docs/local-testing-for-macos.md b/static/docs/local-testing-for-macos.md deleted file mode 100644 index 6578f1362..000000000 --- a/static/docs/local-testing-for-macos.md +++ /dev/null @@ -1,170 +0,0 @@ -# Legacy Binary For macOS - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI tunnel feature allows you to test your private server URLs or locally hosted web-apps or websites through TestMu AI cloud servers. Through our TCP with TLS 1.2 secure connection features, you can also test plain HTML, CSS, PHP, Python or other similar web files saved on your local system, on combinations of operating systems, browsers, and screen resolutions that are available on TestMu AI. - -Feel more confident than ever, before deploying any UI specific change into Production. Test your development thoroughly with TCP with TLS 1.2 secure connection hosted by TestMu AI tunnel. Experience the flexibility to securely test your website on 3000+ browsers and their assorted versions, running on real OS code for cross platform and [cross browser compatibility](https://www.testmuai.com/feature), even before it gets deployed over the production environment. - -> **Note**: From November 2, 2020 onwards, we have deprecated our existing tunnel binary. In order to setup the new version, you can refer to [new tunnel binary for macOS](/docs/local-testing-macos/). - -You can download the binary file responsible for establishing a TCP with TLS 1.2 secure connection between your local device and our testing cloud by clicking on the downloadable link below. - -Download TestMu AI Tunnel For macOS - -This file is supported for the below listed macOS: - -* macOS Big Sur -* macOS Catalina -* macOS Mojave -* macOS High Sierra -* macOS Sierra -* OS X El Capitan -* OS X Yosemite -* OS X Mavericks -* OS X Mountain Lion -* OS X Lion - -## Configuring TCP with TLS 1.2 Secure Connection with TestMu AI Tunnel - -**Step 1**: Click the button "Configure Tunnel". Download the binary zip file by clicking the "Download Link". Also, hit the ‘Copy’ button in the form that opens up to copy the complete string to your system’s clipboard. - -**What is this zip file for?** - -This zip file will help in establishing a secure connection to TestMu AI cloud servers so you can test your locally hosted web pages using a TCP with TLS 1.2 secure connection. - -**Step 2**: Extract the downloaded zip file. Go to terminal and route your pointer to the same directory where you extracted the zip file. - -**Step 3**: Paste the copied string to execute the downloaded binary file. The command will look like: - -`LT --user {user's login email} --key {user's access key} --cui` - -So for example, if your user login email is `example@lambdatest.com` and your user key is 123asd123, the command would be: - -`LT --user example@lambdatest.com --key 123asd123 --cui` - -**Note:** After you execute the command, you may be asked for user permission. Make sure you have administrator access to allow the file to run successfully. - -In a few seconds, after you execute the command, a TCP with TLS 1.2 secure connection will be established. You will receive a message inside your command prompt mentioning "Secure connection established, you may start your tests now". - -## Testing Locally Hosted Web Pages Using TestMu AI tunnel - -**Step 1**: Navigate to 'Real Time Testing' menu and enter the localhost URL you want to test in the text field provided to enter URL. - -**Step 2**: Select the tunnel via which you want to run the test. - -**Step 3**: Select the test configuration of your choice. You can select from various major browsers & their assorted versions to perform a test session. You may also choose from a variety of Operating System & screen resolutions to check the RWD(Responsive Web Design) of the website when rendering on different resolutions. - -**Step 4**: Make sure you turn on XAMPP or any other web hosting tool to provide you an Apache test environment. If you are using XAMPP, hit ‘Start’ for Apache under ‘Services’. If you also want to test the back-end functionality for your website then make sure you hit the ‘Start’ button for MySQL as well. - -**Step 5**: Once you are done selecting your configurations and hosting a web server. Hit the ‘Start’ button in Real Time Test. Your test will start and you will be navigated to your localhost URL. - ->**Important Note:** Some Safari browsers, on both Mac machines and iOS browsers, doesn’t support automatic resolution of URL string "localhost". Therefore if you test on URLs like "`http://localhost/`" or "`http://localhost:8080`" etc, you would get an error in these browsers. A possible solution is to replace the string "`localhost`" with machine IP address. For example if you wanted to test "`http://localhost/dashboard`" or "`http://localhost/dashboard:8080`", and your machine IP is 192.168.2.6 you can instead test on "`http://192.168.2.6/dashboard`" or "`http://192.168.2.6/dashboard:8080`". - -## Testing Local Folder Files Using TestMu AI tunnel - ->**Note:** If you were already running a test session through tunnel then you will have to terminate the existing TCP with TLS 1.2 secure connection in cmd before you re-establish it for a new directory. - -**Step 1**: If you want to test the local folders and files stored in your system, pass an additional argument 'dir' with the directory/folder in double quotes name as you run the command on the terminal to establish a TCP with TLS 1.2 secure connection with TestMu AI tunnel. - -Sample format: - -`LT --user {user login email} --key {user key} --dir {"directory/folder name"}` - -**Example:** Let us consider your user login email to be "example@lambdatest.com" with your user key as ‘123asd123’ and you want to test files in the ‘Downloads’ folder, then the command would be: - -`LT --user example@lambdatest.com --key 123asd123 --dir "/home/lambda/Downloads"` - -**Step 2**: After you establish a TCP with TLS 1.2 secure connection, go to TestMu AI and navigate to Real Time test. Enter "local-folder.lambdatest.com" in URL bar. The list of all files/folders present under the selected directory will be displayed in the browser. Select the test configuration(OS, Browser and screen resolution) of your choice. - -**Step 3**: Hit the ‘Start’ button. Once the VM(Virtual Machine) is launched, you will be able to access your local folders on a testing environment hosted by our cloud servers. - -**Note:** Similarly, if you want to run any file in the specified directory, you need to give URL as local-folder.lambdatest.com/filename. - -**For example**: If you have a file ‘test.html’ in your folder. You need to give URL as "local-folder.lambdatest.com/test.html" to run the same file. In this way, you can run any HTML, PHP, Python or similar web files present in any folder of your system, directly in TestMu AI browser. - -## Terminating The Secure Connection - -A secure tunnel connection is maintained unless you explicitly disconnect it. You can disconnect the tunnel in following ways: - -**Option 1**: You can kill the process by using Ctrl+C in the terminal. -You will receive the below highlighted messages indicating the termination of the TCP with TLS 1.2 secure connection. - -**Option 2**: You can also disconnect it using the option provided in the user interface. In the top right corner of TestMu AI interface, you will find a button that says "Tunnel active:". This button displays the number of active tunnels. As you click on it, you will find your tunnel name along with an ‘x’. Hit the ‘x’ to terminate the secure tunnel. - -## Geolocation Testing Through VPN with TestMu AI tunnel - -Test the performance of your web-app through different geographic locations using VPN in TestMu AI tunnel. Make sure your website delivers well to your targeted audience from all around the globe. - -Use the following arguments in your command line. - -| ARGUMENTS | EXAMPLES | DESCRIPTION | -|---------|-------|-------------| -| –proxy-host | abc.wonderproxy.com | Full hostname for the proxy you’d like to use. | -| –key | 123asd123 | Helps Lambdatest to map a TCP with TLS 1.2 secure connection with the right user. | -| –proxy-port | 12345 | The port declared by you for fetching the proxy. | -| –proxy-pass | hijklmnop | WonderProxy password of the user. | -| –proxy-user | abcdefg | WonderProxy username of the user. | - -Here is how the format of full command would look like: - -`LT --user {user login email} --key {user key} --proxy-host {full proxy hostname} --proxy-user {proxy username} --proxy-pass {proxy password} --proxy-port {port number}` - -Example of full command: - -`LT --user example@lambdatest.com --key 123asd123 --proxy-host abc.wonderproxy.com --proxy-user abcdefg --proxy-pass hijklmnop --proxy-port 12345` - -### How Can Geolocation Testing Help You? - -**1. SERPs** – Search Engine Result Pages: For a website that is targeting audience from different parts of globes. It is always recommended to have a constant check on SERPs to address you website according to the respective trends of different geographies. This is extremely necessary for clothing brands. Traditional wearing differs from one country to another. - -**2. GDPR compliance** – General Data Protection Rule Compliance: Payment Gateways may differ from one geography to another due to different GDPR compliance for different geographies. - -**3. Billing:** Your billing page should support currency conversion with respect to the IP. So if a user from US would show up at your website then he/she should be billed in USD, similarly if one is visiting from France then the bill should be presented in Euros. - -**4. Google Ads:** They also differ from one geographic region to another. You need to check through IPs of different geolocations regarding which ads are visible and in what manner? - -**5. Bandwidth:** 4G may or may not be available to every one of your targeted geography and even if it is available. The available bandwidth will usually be different. Meaning some countries would provide a faster download speed in 4G as compared to others. So you need to be mindful about the content optimization according to your targeted geolocations. Testing from IPs belonging to different geolocations will help you get a realistic view for page loading speed. A faster page loading will not only help in pleasing your customers, it would also help in ranking higher in SERPs. - -### How Can You Test From An IP Of A Different Geolocation? - -This is where a VPN comes to rescue! VPN – Virtual Private Network – It helps in expanding a private network through a public network. Initially VPNs were introduced and used among large enterprises, defense organization, banking companies etc. to keep their data secure. A VPN created a pathway through which people working in large organizations could access the data they want, remotely, in a secured manner. The data is securely transmitted through the process of Cryptography. In Cryptography, the data is encrypted from the sender’s end and then it gets decrypted with the help of a unique key on the receivers end. - -Time have changed and with it the popularity of VPN too! These days VPNs are so widely available that they are used by almost everyone of us. Some use it to surf the internet without worrying about the restrictions applied for their GeoIP. Some use it for the purpose of maintaining anonymity. Some use it without even realizing the meaning of a VPN. - -Remember though, not all VPNs will work in the same manner. Few may provide you considerably more geolocations to choose from as compared to others. Few may provide less number of geolocations but more number of dynamic GeoIPs with respect to others. There are many more parameters related to comparison. - -## Command References - -Following is the list of command arguments which can be used as modifiers while executing the binary file: - -| Command | Description | -|---------|-------------| -| –user | Registered email ID of the user. | -| –key | User access key for the user. | -| –tunnelName | (optional) Name of the tunnel. | -| –port | (optional) Local port to connect tunnel. | -| –proxy-host | Full hostname for the proxy you’d like to use. | -| –proxy-port | The port declared by you for fetching the proxy. | -| –proxy-user | WonderProxy username of the user. | -| –proxy-pass | WonderProxy username of the user. | - -## Changing Your Access Key - -**What is an Access Key?** - -An Access Key is a unique string used for the identification of your account. - -As we offer a cloud-based platform to host a testing ecosystem to everyone around the globe. Access key plays a crucial role in mapping a TCP with TLS 1.2 secure connection for hosting your locally stored web pages online. They work as your IDs when you gain access to our cloud servers. - -If you ever misplace your Access key into the hands of someone you don’t trust. You can always redefine it by clicking the button for "**Regenerate Access Key**". - -Access Key is pivotal for establishing a TCP with TLS 1.2 secure connection through TestMu AI tunnel to test your locally hosted web-pages or web-apps. - -Find your Access Key while performing any type of testing, from the left navigation menu. You will find a button that says "Configure Tunnel". Click on that button & a string containing your access key will be presented. - -For instance, if you go to Real-Time Test, you will observe a button that says "Configure Tunnel" on the top-right area of the page. Click on that button and a form would open up through it, providing you with a string containing your access key. You will find the access key in the area underlined below. - -You can also find your Access Key by going to ‘Settings’ in the left navigation menu and routing to ‘Profile’. - ->**TestMu AI tunnel Update:** Earlier, when our TestMu AI tunnel binary was updated with a new version, we requested our users to re-download the binary file for uninterrupted cross browser testing experience of locally hosted web pages. Not anymore! We have now updated the TestMu AI tunnel binary in a way that it auto-updates to the latest version every time you execute the TestMu AI tunnel from cmd/terminal. diff --git a/static/docs/local-testing-for-windows.md b/static/docs/local-testing-for-windows.md deleted file mode 100644 index 710bb147b..000000000 --- a/static/docs/local-testing-for-windows.md +++ /dev/null @@ -1,182 +0,0 @@ -# Legacy Binary For Windows - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI Tunnel feature allows you to test your private server URLs or locally hosted web-apps or websites through TestMu AI cloud servers. Through our TCP with TLS 1.2 secure connection tunneling features, you can also test plain HTML, CSS, PHP, Python or other similar web files saved on your local system, on combinations of operating systems, browsers, and screen resolutions that are available on TestMu AI. - -Feel more confident than ever, before deploying any UI specific change into Production. Test your development thoroughly with TCP with TLS 1.2 secure connection network protocol hosted by TestMu AI Tunnel. Experience the flexibility to securely test your website on 3000+ browsers and their assorted versions, running on real OS code for cross platform and [cross browser compatibility](https://www.testmuai.com/feature), even before it gets deployed over production environment. - -> **Note**: From November 2, 2020 onwards, we have deprecated our existing tunnel binary. In order to setup the new version, you can refer to [new tunnel binary for Windows](/docs/local-testing-windows/). - -You can download the binary file responsible for establishing a TCP with TLS 1.2 secure connection between your local device and our testing cloud by clicking on the downloadable link below. - -Download TestMu AI Tunnel For Windows - -This file is supported for the below listed windows: - -* Windows 10 -* Windows 8.1 -* Windows 8 -* Windows 7 -* Windows XP - -## Configuring TCP with TLS 1.2 secure connection with TestMu AI Tunnel - -**Step 1**: Click the button **"Configure Tunnel"**. Download the binary zip file by clicking the **"Download Link"**. Also, hit the ‘Copy’ button in the form that opens up to copy the complete string to your system’s clipboard. - -**What is this zip file for?** - -This zip file will help in establishing a secure connection to TestMu AI cloud servers so you could test your locally hosted web pages in a TCP with TLS 1.2 secure connection environment. - -**Step 2**: Extract the downloaded zip file. Go to **cmd(Command Prompt)** and route your pointer to the same directory where you extracted the zip file. - -**Step 3**: Paste the copied string to execute the downloaded binary file. The command will look like: - -`LT --user {user's login email} --key {user's access key} --cui` - -So for example, if your user login email is example@lambdatest.com and your user key is 123asd123, the command would be: - -`LT --user example@lambdatest.com --key 123asd123 --cui` - -**Note:** After you execute the command, Windows may ask for user permission. Make sure you have administrator access to allow the file to run successfully. - -In a few seconds, after you execute the command, a TCP with TLS 1.2 secure connection will be established. You will receive a message inside your command prompt mentioning "**Secure connection established, you may start your tests now**". - -## Testing locally hosted web pages using TestMu AI Tunnel - -**Step 1**: Navigate to **‘Real Time Testing’** menu and enter the localhost URL you want to test in the text field provided to enter URL. - -**Step 2**: Select the tunnel via which you want to run the test. - -**Step 3**: Select the test configuration of your choice. You can select from various major browsers & their assorted versions to perform a test session. You may also choose from a variety of Operating System & screen resolutions to check the RWD(Responsive Web Design) of the website when rendering on different resolutions. - -**Step 4**: Make sure you turn on XAMPP or any other web hosting tool to provide you an Apache test environment. If you are using XAMPP, hit **‘Start’** for Apache under **‘Actions’**. If you also want to test the back-end functionality for your website then make sure you hit the ‘Start’ button for MySQL as well. - -**Step 5**: Once you are done selecting your configurations and hosting a web server. Hit the ‘**Start**’ button in **Real Time Test**. Your test will start and you will be navigated to your localhost URL. - -> **IMPORTANT NOTE**: Some browsers doesn’t support automatic resolution of the URL string "localhost". Therefore if you test on URLs like `http://localhost/` or "`http://localhost:8080`" etc, you would get an error in these browsers. A possible solution is to use "localhost.lambdatest.com" or replace the string "localhost" with machine IP address. For example if you wanted to test `http://localhost/dashboard` or, and your machine IP is 192.168.2.6 you can instead test on "`http://192.168.2.6/dashboard`". - -## Testing Local Folder Files Using TestMu AI Tunnel - ->**Note:** If you were already running a test session through tunnel then you will have to terminate the existing TCP with TLS 1.2 secure connection in cmd before you re-establish it for a new directory. - -**Step 1:** If want to test the local folders and files stored in your system, pass an additional argument ‘dir’ with the directory/folder name in double quotes as you run the command on Windows cmd to establish a TCP with TLS 1.2 secure connection with TestMu AI Tunnel. - -**Sample format:** - -`LT --user {user login email} --key {user key} --dir {"directory/folder name"}` - -**Example:** Let us consider your user login email to be "example@lambdatest.com" with your user key as ‘123asd123’ and you want to test files in the ‘Downloads’ folder, then the command would be: - -`LT --user example@lambdatest.com --key 123asd123 --dir "/home/lambda/Downloads"` - -**Step 2:** After you establish a TCP with TLS 1.2 secure connection, go to TestMu AI and navigate to Real Time test. Enter "local-folder.lambdatest.com" in URL bar. The list of all files/folders present under the selected directory will be displayed in the browser. Select the test configuration(OS, Browser and screen resolution) of your choice. - -**Step 3:** Hit the **‘Start’** button. Once the VM(Virtual Machine) is launched, you will be able to access your local folders on a testing environment hosted by our cloud servers. - -**Note:** Similarly, if you want to run any file in the specified directory, you need to give URL as local-folder.lambdatest.com/filename. - -**For example:** If you have a file ‘test.html’ in your folder. You need to give URL as "local-folder.lambdatest.com/test.html" to run the same file. - -In this way, you can run any HTML, PHP, Python or similar web files present in any folder of your system, directly in TestMu AI browser. - -#### Testing Locally Hosted Web Pages On Android - -**Step 1:** Go to "**Real Time Test**" from the left navigation menu and click on the **mobile** icon. This will guide you to a screen where you can perform cross browser testing for your website on mobile OS. Select the icon for Android and choose from a variety of **Android** mobile devices to start testing. - -**Step 2:** After choosing your configurations, hit the ‘Start’ button to launch the VM. You will be able to access you locally hosted web pages. - -#### Testing Locally Hosted Web Pages On iOS - -**Step 1:** Go to **"Real Time Test"** from the left navigation menu and click on the mobile icon. This will guide you to a screen where you can perform cross browser testing for your website on mobile OS. Select the icon for Apple and choose from a variety of iOS devices to start testing. - -**Note:** To test your locally hosted web pages on iOS, you need to mention "`http://localhost.lambdatest.com`" or simply mention your machine IP as the URL in the below-highlighted box. For example if you wanted to test "`http://localhost/dashboard`" or "`http://localhost/dashboard:8080`", and your machine IP is 192.168.2.6 you can instead test on "`http://localhost.lambdatest.com/dashboard`" or "`http://192.168.2.6/dashboard`". - -**Step 2:** After choosing your configurations, hit the ‘**Start**’ button to launch the VM. You will be able to access you locally hosted web pages. - -_Notice how the font changed from Android to iOS or rather from Google Chrome to Mozilla Firefox, in the above images of local testing on Android & iOS. That right there is a cross browser compatibility issue!_ - -#### Terminating the TCP with TLS 1.2 secure connection - -A secure tunnel connection is maintained unless you explicitly disconnect it. You can disconnect the tunnel in following ways: - -**Option 1:** You can kill the process by using Ctrl+C in Windows command prompt. -You will receive the below highlighted messages indicating the termination on TCP with TLS 1.2 secure connection. - -**Option 2:** You can also disconnect it using the option provided in the user interface. In the top right corner of TestMu AI interface, you will find a button that's says **"Tunnel active"**. This button displays the number of active tunnels. As you click on it, you will find your tunnel name along with an 'x'. Hit the 'x' to terminate the TCP with TLS 1.2 secure connection. - -#### Geolocation Testing through VPN with TestMu AI Tunnel - -Test the performance of your web-app through different **geographic locations** using VPN in TestMu AI Tunnel. Make sure your website delivers well to your targeted audience from all around the globe. - -**How can Geolocation Testing help you?** - -1. **SERPs:** – Search Engine Result Pages: For a website that is targeting audience from different parts of globes. It is always recommended to have a constant check on SERPs to address you website according to the respective trends of different geographies. This is extremely necessary for clothing brands. Traditional wearing differs from one country to another. - -2. **GDPR compliance:** – General Data Protection Rule Compliance: Payment Gateways may differ from one geography to another due to different GDPR compliance for different geographies. - -3. **Billing:** Your billing page should support currency conversion with respect to the IP. So if a user from US would show up at your website then he/she should be billed in USD, similarly if one is visiting from France then the bill should be presented in Euros. - -4. **Google Ads:** They also differ from one geographic region to another. You need to check through IPs of different geolocations regarding which ads are visible and in what manner? - -5. **Bandwidth:** 4G may or may not be available to every one of your targeted geography and even if it is available. The available bandwidth will usually be different. Meaning some countries would provide a faster download speed in 4G as compared to others. So you need to be mindful about the content optimization according to your targeted geolocations. Testing from IPs belonging to different geolocations will help you get a realistic view for page loading speed. A faster page loading will not only help in pleasing your customers, it would also help in ranking higher in SERPs. - -**How can you test from an IP of a different Geolocation?** - -This is where a VPN comes to rescue! VPN – Virtual Private Network – It helps in expanding a private network through a public network. Initially VPNs were introduced and used among large enterprises, defense organization, banking companies etc. to keep their data secure. A VPN created a pathway through which people working in large organizations could access the data they want, remotely, in a secured manner. The data is securely transmitted through the process of Cryptography. In Cryptography, the data is encrypted from the sender’s end and then it gets decrypted with the help of a unique key on the receivers end. - -Times have changed and with it the popularity of VPN too! These days VPNs are so widely available that they are used by almost everyone of us. Some use it to surf the internet without worrying about the restrictions applied for their GeoIP. Some use it for the purpose of maintaining anonymity. Some use it without even realizing the meaning of a VPN. - -Remember though, not all VPNs will work in the same manner. Few may provide you considerably more geolocations to choose from as compared to others. Few may provide less number of geolocations but more number of dynamic GeoIPs with respect to others. There are many more parameters related to comparison. - -Use the following arguments in your command line. - -| Argument | Example | Description | -|----------|---------|-------------| -| --key | 123asd123 | Helps Lambdatest to map a TCP with TLS 1.2 secure connection with the right user. | -| --proxy-host | abc.wonderproxy.com | Full hostname for the proxy you’d like to use. | -| --proxy-port | 12345 | The port declared by you for fetching the proxy. | -| --proxy-user | abcdefg | WonderProxy username of the user. | -| --proxy-pass | hijklmnop | WonderProxy password of the user. | - -Here is how the format of full command would look like: - -`LT --user {user login email} --key {user key} --proxy-host {full proxy hostname} --proxy-user {proxy username} -proxy-pass {proxy password} -proxy-port {port number}` - -Example of full command: - -`LT --user example@lambdatest.com --key 123asd123 --proxy-host abc.wonderproxy.com --proxy-user abcdefg --proxy-pass hijklmnop --proxy-port 12345` - -#### Command References -Following is the list of command arguments which can be used as modifiers while executing the binary file: - -| Command | Description | -|----------|-------------| -| --user | Registered email ID of the user. | -| --key | User access key for the user. | -| --tunnelName | (optional) Name of the tunnel. | -| --port | (optional) Local port to connect tunnel. | -| --proxy-host | Full hostname for the proxy you’d like to use. | -| --proxy-port | The port declared by you for fetching the proxy. | -| --proxy-user | WonderProxy username of the user. | -| --proxy-pass | WonderProxy password of the user. | - -## Changing Your Access Key - -**What is an Access Key?** - -An Access Key is a unique string used for the identification of your account. - -As we offer a cloud-based platform to host a testing ecosystem to everyone around the globe. Access key plays a crucial role in mapping a TCP with TLS 1.2 secure connection environment for hosting your locally stored web pages online. They work as your IDs when you gain access to our cloud servers. - -If you ever misplace your Access key into the hands of someone you don’t trust. You can always redefine it by clicking the button for "**Regenerate Access Key**". - -**Access Key** is pivotal for establishing a TCP with TLS 1.2 secure connection through TestMu AI Tunnel to test your locally hosted web-pages or web-apps. - -Find your Access Key while performing any type of testing, from the left **navigation menu**. You will find a button that says "**Configure Tunnel**". Click on that button & a string containing your access key will be presented. - -For instance, if you go to **Real-Time Test**, you will observe a button that says "**Configure Tunnel**" on the top-right area of the page. Click on that button and a form would open up through it, providing you with a string containing your access key. You will find the access key in the area underlined below. - -You can also find your Access Key by going to ‘**Settings**’ in the left navigation menu and routing to ‘**Profile**’. - -**TestMu AI Tunnel Update:** Earlier, when our TestMu AI Tunnel binary was updated with a new version, we requested our users to re-download the binary file for uninterrupted cross browser testing experience of locally hosted web pages. Not anymore! We have now updated the TestMu AI Tunnel binary in a way that it auto-updates to the latest version every time you execute the TestMu AI Tunnel from cmd/terminal. diff --git a/static/docs/local-testing-linux.md b/static/docs/local-testing-linux.md deleted file mode 100644 index ec7507e64..000000000 --- a/static/docs/local-testing-linux.md +++ /dev/null @@ -1,148 +0,0 @@ -# Local Testing For Linux - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI tunnel feature allows you to test your private server URLs or locally hosted web-apps or websites on 3000+ real browsers through TestMu AI. You can test plain HTML, CSS, PHP, Python, or other similar web files saved on your local system, over combinations of operating systems, browsers, and screen resolutions that are available on TestMu AI. You can leverage the new TestMu AI tunnel binary for corporate firewalls and proxy settings without any restrictions. TestMu AI tunnel follows various protocols such as Web Socket, HTTPS, TCP with TLS 1.2 secure connection, etc. to help you establish a secure and unique tunnel connection through corporate firewalls between your system and TestMu AI cloud servers. - -Feel more confident than ever, before deploying any UI specific change into Production. Test your development thoroughly with TestMu AI tunnel and experience the flexibility to securely test your website on 3000+ browsers and their assorted versions, running on real OS code for cross-platform and [cross browser compatibility](https://www.testmuai.com/feature), even before it gets deployed over the Production environment. - -You can download the binary file responsible for establishing a tunnel connection between your local device and our testing cloud by clicking on the downloadable link below. - -| [DOWNLOAD TestMu AI tunnel ZIP FOR LINUX 64 BIT](https://downloads.lambdatest.com/tunnel/v3/linux/64bit/LT_Linux.zip) | -|:------------------------------------------------------------------------------------------------------------------:| - -## Configuring Connection With TestMu AI tunnel - -**Step 1**:  Click the button **‘Configure Tunnel’**. Download the binary zip file by clicking the **‘Download Link’**. Press the **‘Copy’**  button in the form that opens up to copy the complete string to your system’s clipboard. - ->**What is this zip file for?** -This zip file contains the tunnel binary which will help in establishing a secure tunnel connection to TestMu AI cloud servers so you could test your locally hosted web pages over thousands of browsers and operating systems for desktop and mobile using TestMu AI. - -**Step 2**:  Extract the downloaded zip file. Go to the terminal and route your pointer to the same directory where you extracted the zip file. - -**Step 3**:  Paste the copied string to execute the downloaded binary file. The command will look like: -``` -LT --user {user's login email} --key {user's access key} --tunnelName {user's tunnel name} -``` - -So for example, if your user login email is example@lambdatest.com, user key is 123456789abcdefghijklmnopqrstuv and tunnel name is TestMu AI, the command would be: -``` -LT --user example@lambdatest.com --key 123456789abcdefghijklmnopqrstuv --tunnelName LambdaTest -``` - ->**Note:**  After you execute the command, you may be asked for user permission. Make sure you have administrator access to allow the file to run successfully. - -In a few seconds, after you execute the command, a tunnel connection will be established. You will receive a message inside your command prompt mentioning **"You can start testing now"**. - -## Testing Locally Hosted Web Pages Using TestMu AI tunnel - -**Step 1**:  Make sure you turn on XAMPP or any other web hosting tool to provide you an Apache test environment. If you are using XAMPP, Press the **‘Start’**  button for **‘Apache’**  under **‘Actions’**. If you also want to test the back-end functionality for your website then Press the **‘Start’**  button for **‘MySQL’**  as well. - -**Step 2**:  Navigate to **‘Real Time Testing’**  menu. Enter the localhost URL you want to test in the text field provided and Select the tunnel via which you want to run the test. - -**Step 3**:  Select the test configuration of your choice. You can select from various major browsers & their assorted versions to perform a test session. After selecting the configuration, click on the **‘Start’**  button. - -Once you press the **‘Start’**  button, your test will start and you will be navigated to your localhost URL. - ->**Note:** If `http://localhost:{port}`  does not work then try with either `http://localhost.lambdatest.com:{port}`  or your machine’s private IP `http://{IP}:{port}`. - -## Testing Local Folder Files Using TestMu AI tunnel - ->**Note:**  If you were already running a test session through tunnel then you will have to terminate the existing tunnel connection in cmd before you re-establish it for a new directory. - -**Step 1:**  Navigate to **‘Configure Tunnel’**  button. - -**Step 2:**  Click on **‘Folder’**  button and copy the string on your system’s clipboard. - -**Step 3:**  After you establish a tunnel connection, go to TestMu AI and navigate to **‘Real Time Test’**. Enter `local-folder.lambdatest.com`  in URL bar. The list of all files/folders present under the selected directory will be displayed in the browser. Select the test configuration(OS, Browser and screen resolution) of your choice. Click on the **‘Start’**  button. - -Once the VM (Virtual Machine) is launched, you will be able to access your local folders on a testing environment hosted by our cloud servers. - ->**Note:**  Similarly, if you want to run any file in the specified directory, you need to give URL as `local-folder.lambdatest.com/filename`. -For example: If you have a file `test.html`  in your folder. You need to give URL as `local-folder.lambdatest.com/test.html`  to run the same file. -In this way, you can run any HTML, PHP, Python, or similar web files present in any folder of your system, directly in the TestMu AI browser. - -## Terminating The TestMu AI tunnel Connection - -A secure tunnel connection is maintained unless you explicitly disconnect it. You can disconnect the tunnel in the following ways: - -**Option 1:**  You can kill the process by using **‘Ctrl+C’**  in the terminal. You will receive the below-highlighted message **‘Tunnel stopped successfully’**  indicating the termination of Tunnel Connection. - -**Option 2:**  You can also disconnect it using the option provided in the user interface. In the top right corner of the TestMu AI interface, you will find a button that says **‘Tunnel active’**. This button displays the number of active tunnels. As you click on it, you will find your tunnel name along with an **‘x’**button. Press the **‘x’**  button to terminate the tunnel connection. - -## Geolocation Testing Through VPN with TestMu AI tunnel - -Test the performance of your web-app through different geographic locations using VPN in TestMu AI tunnel. Make sure your website delivers well to your targeted audience from all around the globe. - -Use the following arguments in your command line. - -|**ARGUMENTS**|**EXAMPLES**|**DESCRIPTION**| -| --- | --- | --- | -|`--proxy-host`|abc.wonderproxy.com|Full hostname for the proxy you’d like to use.| -|`--key`|123asd123|Helps Lambdatest to map a tunnel connection with the right user.| -|`--proxy-port`|12345|The port declared by you for fetching the proxy.| -|`--proxy-pass`|hijklmnop|WonderProxy password of the user.| -|`--proxy-user`|abcdefg|WonderProxy username of the user.| - -Here is how the format of full command would look like: -``` -LT --user {user login email} --key {user key} --proxy-host {full proxy hostname} --proxy-user {proxy username} --proxy-pass {proxy password} --proxy-port {port number} -``` - -Example of full command: -``` -LT --user example@lambdatest.com --key 123asd123 --proxy-host abc.wonderproxy.com --proxy-user abcdefg --proxy-pass hijklmnop --proxy-port 12345 -``` -  - -### How Can Geolocation Testing Help You? - -1. **SERPs**  – Search Engine Result Pages: For a website that is targeting an audience from different parts of the globe. It is always recommended to have a constant check on SERPs to address your website according to the respective trends of different geographies. This is extremely necessary for clothing brands. Traditional wearing differs from one country to another. - -2. **GDPR compliance**  – General Data Protection Rule Compliance: Payment Gateways may differ from one geography to another due to different GDPR compliance for different geographies. - -3. **Billing**  – Your billing page should support currency conversion with respect to the IP. So if a user from the US would show up at your website then he/she should be billed in USD, similarly if one is visiting from France then the bill should be presented in Euros. - -4. **Google Ads**  – They also differ from one geographic region to another. You need to check through IPs of different geolocations regarding which ads are visible and in what manner? - -5. **Bandwidth**  – 4G may or may not be available to every one of your targeted geography and even if it is available. The available bandwidth will usually be different. Meaning some countries would provide a faster download speed in 4G as compared to others. So you need to be mindful of the content optimization according to your targeted geolocations. Testing from IPs belonging to different geolocations will help you get a realistic view of page loading speed. A faster page loading will not only help in pleasing your customers, it would also help in ranking higher in SERPs. - -### How Can You Test From An IP Of A Different Geolocation? - -This is where a VPN comes to the rescue! VPN – Virtual Private Network – It helps in expanding a private network through a public network. Initially, VPNs were introduced and used among large enterprises, defense organizations, banking companies, etc. to keep their data secure. A VPN created a pathway through which people working in large organizations could access the data they want, remotely, in a secure manner. The data is securely transmitted through the process of Cryptography. In Cryptography, the data is encrypted from the sender’s end and then it gets decrypted with the help of a unique key on the receiver’s end. - -Time has changed and with it the popularity of VPN too! These days VPNs are so widely available that they are used by almost every one of us. Some use it to surf the internet without worrying about the restrictions applied to their GeoIP. Some use it for the purpose of maintaining anonymity. Some use it without even realizing the meaning of a VPN. - -Remember though, not all VPNs will work in the same manner. Few may provide you considerably more geolocations to choose from as compared to others. Few may provide less number of geolocations but more number of dynamic GeoIPs with respect to others. There are many more parameters related to the comparison. - -## Command References - -Following is the list of command arguments which can be used as modifiers while executing the binary file: - -|**COMMAND**|**DESCRIPTION**| -| --- | --- | -|`--user`|Registered email ID of the user.| -|`--key`|User access key for the user.| -|`--tunnelName`|(optional) Name of the tunnel.| -|`--port`|(optional) Local port to connect tunnel.| -|`--proxy-host`|Full hostname for the proxy you’d like to use.| -|`--proxy-port`|The port declared by you for fetching the proxy.| -|`--proxy-user`|WonderProxy username of the user.| -|`--proxy-pass`|WonderProxy username of the user.| - -## Regenerating Your Access Key or Access Token - ->**What is an Access Key?** ->An Access Key is a unique string used for the identification of your account. -As we offer a cloud-based platform to host a testing ecosystem to everyone around the globe. Access key plays a crucial role in authenticating your TestMu AI account while mapping a secure tunnel connection for hosting your locally stored web pages online. They work as your IDs when you gain access to our cloud servers. -If you ever misplace your Access key into the hands of someone you don’t trust. You can always regenerate it by clicking the tilt arrow button. - -  - -Find your Access Key while performing any type of testing, from the left navigation menu. You will find a button that says **‘Configure Tunnel’**. Click on that button & a string containing your access key will be presented. - -For instance, if you go to **‘Real Time Test’**, you will observe a button that says **‘Configure Tunnel’**  on the top-right area of the page. Click on that button and a form would open up through it, providing you with a string containing your access key. You will find the access key in the area underlined below. - -You can also find your Access Key by going to **‘Settings’**  in the left navigation menu and routing to **‘Profile’**. - ->**TestMu AI tunnel Update:**  Earlier, when our TestMu AI tunnel binary was updated with a new version, we requested our users to re-download the binary file for uninterrupted [cross browser testing](https://www.testmuai.com/) experience of locally hosted web pages. Not anymore! We have now updated the TestMu AI tunnel binary in a way that it auto-updates to the latest version every time you execute the TestMu AI tunnel from cmd/terminal. diff --git a/static/docs/local-testing-macos.md b/static/docs/local-testing-macos.md deleted file mode 100644 index 751565649..000000000 --- a/static/docs/local-testing-macos.md +++ /dev/null @@ -1,160 +0,0 @@ -# Local Testing For MacOS - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI tunnel feature allows you to test your private server URLs or locally hosted web-apps or websites on 3000+ real browsers through TestMu AI. You can test plain HTML, CSS, PHP, Python, or other similar web files saved on your local system, over combinations of operating systems, browsers, and screen resolutions that are available on TestMu AI. You can leverage the new TestMu AI tunnel binary for corporate firewalls and proxy settings without any restrictions. TestMu AI tunnel follows various protocols such as Web Socket, HTTPS, SSH(Secure Shell), etc. to help you establish a secure and unique tunnel connection through corporate firewalls between your system and TestMu AI cloud servers. - -Feel more confident than ever, before deploying any UI specific change into Production. Test your development thoroughly with TestMu AI tunnel and experience the flexibility to securely test your website on 3000+ browsers and their assorted versions, running on real OS code for cross-platform and [cross browser compatibility](https://www.testmuai.com/feature), even before it gets deployed over the Production environment. - -You can download the binary file responsible for establishing a tunnel connection between your local device and our testing cloud by clicking on the downloadable link below. - -| [DOWNLOAD TestMu AI tunnel ZIP FOR MAC 64 BIT](https://downloads.lambdatest.com/tunnel/v3/mac/64bit/LT_Mac.zip) | -|:------------------------------------------------------------------------------------------------------------:| - ->This file is supported for the below listed macOS: -> * macOS Sonoma -* macOS Ventura -* macOS Monterey -* macOS Big Sur -* macOS Catalina -* macOS Mojave -* macOS High Sierra -* macOS Sierra -* OS X El Capitan -* OS X Yosemite -* OS X Mavericks -* OS X Mountain Lion -* OS X Lion - -## Configuring Connection With TestMu AI tunnel - -**Step 1**:  Click the button **‘Configure Tunnel’**. Download the binary zip file by clicking the **‘Download Link’**. Press the **‘Copy’**  button in the form that opens up to copy the complete string to your system’s clipboard. - ->**What is this zip file for?** -This zip file contains the tunnel binary which will help in establishing a secure tunnel connection to TestMu AI cloud servers so you could test your locally hosted web pages over thousands of browsers and operating systems for desktop and mobile using TestMu AI. - -**Step 2**:  Extract the downloaded zip file. Go to the terminal and route your pointer to the same directory where you extracted the zip file. - -**Step 3**:  Paste the copied string to execute the downloaded binary file. The command will look like: -``` -LT --user {user's login email} --key {user's access key} --tunnelName {user's tunnel name} -``` -So for example, if your user login email is example@lambdatest.com, user key is 123456789abcdefghijklmnopqrstuv and tunnel name is TestMu AI, the command would be: -``` -LT --user example@lambdatest.com --key 123456789abcdefghijklmnopqrstuv --tunnelName LambdaTest -``` - ->**Note**:  After you execute the command, you may be asked for user permission. Make sure you have administrator access to allow the file to run successfully. -In a few seconds, after you execute the command, a tunnel connection will be established. You will receive a message inside your command prompt mentioning **"You can start testing now"**. - -## Testing Locally Hosted Web Pages Using TestMu AI tunnel - -**Step 1**:  Make sure you turn on XAMPP or any other web hosting tool to provide you an Apache test environment. If you are using XAMPP, Press the **‘Start’**  button for **‘Apache’**  under **‘Actions’**. If you also want to test the back-end functionality for your website then Press the **‘Start’**  button for **‘MySQL’**  as well. - -**Step 2**:  Navigate to **‘Real Time Testing’**  menu. Enter the localhost URL you want to test in the text field provided and Select the tunnel via which you want to run the test. - -**Step 3**:  Select the test configuration of your choice. You can select from various major browsers & their assorted versions to perform a test session. After selecting the configuration, click on the **‘Start’**  button. - -Once you press the **‘Start’**  button, your test will start and you will be navigated to your localhost URL. - ->**NOTE:** If `http://localhost:{port}`  does not work then try with either `http://localhost.lambdatest.com:{port}`  or your machine’s private IP `http://{IP}:{port}`. - -## Testing Local Folder Files Using TestMu AI tunnel - ->**Note:**  If you were already running a test session through tunnel then you will have to terminate the existing tunnel connection in cmd before you re-establish it for a new directory. - -**Step 1:**  Navigate to **‘Configure Tunnel’**  button. - -**Step 2:**  Click on **‘Folder’**  button and copy the string on your system’s clipboard. - -**Step 3:**  After you establish a tunnel connection, go to TestMu AI and navigate to **‘Real Time Test’**. Enter `local-folder.lambdatest.com`  in URL bar. The list of all files/folders present under the selected directory will be displayed in the browser. Select the test configuration(OS, Browser and screen resolution) of your choice. Click on the **‘Start’**  button. - -Once the VM (Virtual Machine) is launched, you will be able to access your local folders on a testing environment hosted by our cloud servers. - ->**Note:**  Similarly, if you want to run any file in the specified directory, you need to give URL as `local-folder.lambdatest.com/filename`. -For example: If you have a file `test.html`  in your folder. You need to give URL as `local-folder.lambdatest.com/test.html`  to run the same file. -In this way, you can run any HTML, PHP, Python, or similar web files present in any folder of your system, directly in the TestMu AI browser. - -## Terminating The TestMu AI tunnel Connection - -A secure tunnel connection is maintained unless you explicitly disconnect it. You can disconnect the tunnel in the following ways: - -**Option 1:**  You can kill the process by using **‘Ctrl+C’**  in the terminal. You will receive the below-highlighted message **‘Tunnel stopped successfully’**  indicating the termination of Tunnel Connection. - -**Option 2:**  You can also disconnect it using the option provided in the user interface. In the top right corner of the TestMu AI interface, you will find a button that says **‘Tunnel active’**. This button displays the number of active tunnels. As you click on it, you will find your tunnel name along with an **‘x’**button. Press the **‘x’**  button to terminate the tunnel connection. - -## Geolocation Testing Through VPN with TestMu AI tunnel - -Test the performance of your web-app through different geographic locations using VPN in TestMu AI tunnel. Make sure your website delivers well to your targeted audience from all around the globe. - -Use the following arguments in your command line. - -|**ARGUMENTS**|**EXAMPLES**|**DESCRIPTION**| -| --- | --- | --- | -|`--proxy-host`|abc.wonderproxy.com|Full hostname for the proxy you’d like to use.| -|`--key`|123asd123|Helps Lambdatest to map a tunnel connection with the right user.| -|`--proxy-port`|12345|The port declared by you for fetching the proxy.| -|`--proxy-pass`|hijklmnop|WonderProxy password of the user.| -|`--proxy-user`|abcdefg|WonderProxy username of the user.| - -Here is how the format of full command would look like: - -`LT --user {user login email} --key {user key} --proxy-host {full proxy hostname} --proxy-user {proxy username} --proxy-pass {proxy password} --proxy-port {port number}` - -Example of full command: - -`LT --user example@lambdatest.com --key 123asd123 --proxy-host abc.wonderproxy.com --proxy-user abcdefg --proxy-pass hijklmnop --proxy-port 12345` -  - -### How Can Geolocation Testing Help You? - -1. **SERPs**  – Search Engine Result Pages: For a website that is targeting an audience from different parts of the globe. It is always recommended to have a constant check on SERPs to address your website according to the respective trends of different geographies. This is extremely necessary for clothing brands. Traditional wearing differs from one country to another. - -2. **GDPR compliance**  – General Data Protection Rule Compliance: Payment Gateways may differ from one geography to another due to different GDPR compliance for different geographies. - -3. **Billing**  – Your billing page should support currency conversion concerning the IP. So if a user from the US would show up at your website, then he/she should be billed in USD. Similarly, if one is visiting from France, then the bill should be presented in Euros. - -4. **Google Ads**  – They also differ from one geographic region to another. You need to check through IPs of different geolocations regarding which ads are visible and in what manner? - -5. **Bandwidth**  – 4G may or may not be available to every one of your targeted geography and even if it is available. The available bandwidth will usually be different. Meaning some countries would provide a faster download speed in 4G as compared to others. So, you'll need to be mindful of the content optimization according to your targeted geolocations. Testing from IPs belonging to different geolocations will help you get a realistic view of page loading speed. A faster page loading will not only help in pleasing your customers. It would also help in ranking higher in SERPs. - -### How Can You Test From An IP Of A Different Geolocation? - -This is where a VPN comes to the rescue! VPN – Virtual Private Network – It helps in expanding a private network through a public network. Initially, VPNs were introduced and used among large enterprises, defense organizations, banking companies, etc. to keep their data secure. A VPN created a pathway through which people working in large organizations could access the data they want, remotely, in a secure manner. The data is securely transmitted through the process of Cryptography. In Cryptography, the data is encrypted from the sender’s end and then it gets decrypted with the help of a unique key on the receiver’s end. - -Time has changed and with it the popularity of VPN too! These days VPNs are so widely available that they are used by almost every one of us. Some use it to surf the internet without worrying about the restrictions applied to their GeoIP. Some use it to maintain anonymity. Some use it without even realizing the meaning of a VPN. - -Remember though, not all VPNs will work in the same manner. Few may provide you considerably more geolocations to choose from as compared to others. Few may provide less number of geolocations but more number of dynamic GeoIPs with respect to others. There are many more parameters related to the comparison. - -## Command References - -Following is the list of command arguments which can be used as modifiers while executing the binary file: - -|**COMMAND**|**DESCRIPTION**| -| --- | --- | -|`--user`|Registered email ID of the user.| -|`--key`|User access key for the user.| -|`--tunnelName`|(optional) Name of the tunnel.| -|`--port`|(optional) Local port to connect tunnel.| -|`--proxy-host`|Full hostname for the proxy you’d like to use.| -|`--proxy-port`|The port declared by you for fetching the proxy.| -|`--proxy-user`|WonderProxy username of the user.| -|`--proxy-pass`|WonderProxy username of the user.| - -## Regenerating Your Access Key or Access Token - -**What is an Access Key?** - -  - ->An Access Key is a unique string used for the identification of your account. ->As we offer a cloud-based platform to host a testing ecosystem to everyone around the globe. Access key plays a crucial role in authenticating your TestMu AI account while mapping a secure tunnel connection for hosting your locally stored web pages online. They work as your IDs when you gain access to our cloud servers. ->If you ever misplace your Access key in the hands of someone you don’t trust. You can always regenerate it by clicking the tilt arrow button. - -Find your Access Key while performing any type of testing, from the left navigation menu. You will find a button that says **‘Configure Tunnel’**. Click on that button & a string containing your access key will be presented. - -For instance, if you go to **‘Real Time Test’**, you will observe a button that says **‘Configure Tunnel’**  on the top-right area of the page. Click on that button and a form would open up through it, providing you with a string containing your access key. You will find the access key in the area underlined below. - -You can also find your Access Key by going to **‘Settings’**  in the left navigation menu and routing to **‘Profile’**. - ->**TestMu AI tunnel Update:**  Earlier, when our TestMu AI tunnel binary was updated with a new version, we requested our users to re-download the binary file for uninterrupted [cross browser testing](https://www.testmuai.com) experience of locally hosted web pages. Not anymore! We have now updated the TestMu AI tunnel binary in a way that it auto-updates to the latest version every time you execute the TestMu AI tunnel from cmd/terminal. diff --git a/static/docs/local-testing-using-playwright.md b/static/docs/local-testing-using-playwright.md deleted file mode 100644 index ee8060a02..000000000 --- a/static/docs/local-testing-using-playwright.md +++ /dev/null @@ -1,77 +0,0 @@ -# Local Testing Using Playwright - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI tunnel feature lets you test private server URLs, locally hosted web apps, and websites on 3000+ real browsers and operating systems. On TestMu AI, you can test plain HTML, CSS, PHP, Python, and other similar web files saved locally. When connecting to corporate firewalls or proxy settings, no restrictions apply to the new TestMu AI tunnel binary. To establish a secure and unique tunnel connection between your system and TestMu AI cloud servers, the TestMu AI tunnel utilizes various protocols like Web Sockets, HTTPS, SSH(Secure Shell), etc. - -Learn how to perform local testing using Playwright across 40+ real browsers and operating systems. - -## Playwright Testing Of Locally Hosted Websites - -You can run Playwright testing of locally hosted websites and web apps via TestMu AI tunnel binary. - -1. Clone the TestMu AI-Playwright repository on your system. - -2. Install the npm dependencies. - -``` -npm install -``` - -3. In order to run your Playwright tests, you will need to set your TestMu AI username and access key in the environment variables. - -**Windows** - -```js -set LT_USERNAME="YOUR_LAMBDATEST_USERNAME" -set LT_ACCESS_KEY="YOUR_LAMBDATEST_ACCESS_KEY" -``` - -**macOS/Linux** - -```js -export LT_USERNAME="YOUR_LAMBDATEST_USERNAME" -export LT_ACCESS_KEY="YOUR_LAMBDATEST_ACCESS_KEY" -``` - -4. To establish a tunnel connection between your local device and TestMu AI, download the binary file based on your OS. - -- Windows **[64 Bit](https://downloads.lambdatest.com/tunnel/v3/windows/64bit/LT_Windows.zip) | [32 Bit](https://downloads.lambdatest.com/tunnel/v3/windows/32bit/LT_Windows.zip)** -- macOS **[64 Bit](https://downloads.lambdatest.com/tunnel/v3/mac/64bit/LT_Mac.zip) | [32 Bit](https://downloads.lambdatest.com/tunnel/v3/mac/32bit/LT_Mac.zip)** -- Linux **[64 Bit](https://downloads.lambdatest.com/tunnel/v3/linux/64bit/LT_Linux.zip) | [32 Bit](https://downloads.lambdatest.com/tunnel/v3/linux/32bit/LT_Linux.zip)** - -5. Extract the downloaded binary file. - -6. Navigate to the directory or folder where you extracted the binary file in the Command Prompt. - -7. Run the below command in the terminal. - -```js -./LT --user {user's login email} --key {user's access key} --tunnelName {user's tunnel name} -``` - -8. In desired capability, add the capability `tunnel: true`. - -If multiple tunnels are running, you can add `tunnel` and `tunnel ID` capabilities. - -```js -const { chromium } = require('playwright') -const { expect } = require('@playwright/test'); - -(async () => { -const capabilities = { -'browserName': 'Chrome', // Browsers allowed: `Chrome`, `MicrosoftEdge`, `pw-chromium`, `pw-firefox` and `pw-webkit` -'browserVersion': 'latest', -'LT:Options': { -'platform': 'Windows 10', -'build': 'Playwright Sample Build', -'name': 'Playwright Sample Test', -'user': process.env.LT_USERNAME, -'accessKey': process.env.LT_ACCESS_KEY, -'tunnel': false, // Add tunnel configuration if testing locally hosted webpage -'tunnelName': '' // Optional -} -} -``` - -You can view test reports for your local tests on the TestMu AI automation dashboard. diff --git a/static/docs/local-testing-windows.md b/static/docs/local-testing-windows.md deleted file mode 100644 index 52d067a75..000000000 --- a/static/docs/local-testing-windows.md +++ /dev/null @@ -1,164 +0,0 @@ -# Local Testing For Windows - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI tunnel feature allows you to test your private server URLs or locally hosted web-apps or websites on 3000+ real browsers through TestMu AI. You can test plain HTML, CSS, PHP, Python, or other similar web files saved on your local system, over combinations of operating systems, browsers, and screen resolutions that are available on TestMu AI. You can leverage the new TestMu AI tunnel binary for corporate firewalls and proxy settings without any restrictions. TestMu AI tunnel follows various protocols such as Web Socket, HTTPS, SSH(Secure Shell), etc. to help you establish a secure and unique tunnel connection through corporate firewalls between your system and TestMu AI cloud servers. - -Feel more confident than ever, before deploying any UI specific change into Production. Test your development thoroughly with TestMu AI tunnel and experience the flexibility to securely test your website on 3000+ browsers and their assorted versions, running on real OS code for cross-platform and [cross browser compatibility](https://www.testmuai.com/feature), even before it gets deployed over the Production environment. - -> Here is all you need to know about TestMu AI tunnel from configuring a secure connection to executing test cases on your locally hosted web-apps. - -You can download the binary file responsible for establishing a tunnel connection between your local device and our testing cloud by clicking on the downloadable link below. - -| [DOWNLOAD TestMu AI tunnel ZIP FOR WINDOWS 64 BIT](https://downloads.lambdatest.com/tunnel/v3/windows/64bit/LT_Windows.zip) | -|--------------------------------------------------------------------------------------------------------------------------| - -This file is supported for the below listed windows: - -* Windows 11 -* Windows 10 -* Windows 8.1 -* Windows 8 -* Windows 7 -* Windows XP - -## Configuring Connection With TestMu AI tunnel - -**Step 1**:  Click the button **‘Configure Tunnel’**. Download the binary zip file by clicking the **‘Download Link’**. Press the **‘Copy’**  button in the form that opens up to copy the complete string to your system’s clipboard. - ->**What is this zip file for?** -This zip file contains the tunnel binary which will help in establishing a secure tunnel connection to TestMu AI cloud servers so you could test your locally hosted web pages over thousands of browsers and operating systems for desktop and mobile using TestMu AI. - -**Step 2**:  Extract the downloaded zip file. Go to **‘Command Prompt’**  and route your pointer to the same directory where you extracted the zip file. - -**Step 3**:  Paste the copied string to execute the downloaded binary file. The command will look like: -``` -LT --user {user's login email} --key {user's access key} --tunnelName {user's tunnel name} -``` - -So for example, if your user login email is example@lambdatest.com, user key is 123456789abcdefghijklmnopqrstuv and tunnel name is TestMu AI, the command would be: - -``` -LT --user example@lambdatest.com --key 123456789abcdefghijklmnopqrstuv --tunnelName LambdaTest -``` - ->**Note:**  After you execute the command, Windows may ask for user permission. Make sure you have administrator access to **‘Allow’** the file to run successfully. -In a few seconds, after you execute the command, a tunnel connection will be established. You will receive a message inside your command prompt mentioning **‘You can start testing now’**. - -## Testing Locally Hosted Web Pages Using TestMu AI tunnel - -**Step 1**:  Make sure you turn on XAMPP or any other web hosting tool to provide you an Apache test environment. If you are using XAMPP, Press the **‘Start’**  button for **Apache**  under **‘Actions’**. If you also want to test the back-end functionality for your website then Press the **‘Start’**  button for **‘MySQL’**  as well. - -**Step 2**:  Navigate to **‘Real Time Testing’**  menu. Enter the localhost URL you want to test in the text field provided and Select the tunnel via which you want to run the test. - -**Step 3**:  Select the test configuration of your choice. You can select from various major browsers & their assorted versions to perform a test session. After selecting the configuration, click on the **‘Start’**  button. - -Once you press the **‘Start’**  button, your test will start and you will be navigated to your localhost URL. - ->**NOTE:**  If `http://localhost:{port}`  does not work then try with either `http://localhost.lambdatest.com:{port}`  or your machine’s private IP `http://{IP}:{port}`. - -## Testing Locally Hosted Web Pages On Android - -**Step 1:**  Go to **‘Real Time Test’**  from the left navigation menu and click on the **‘Mobile’**  icon. This will guide you to a screen where you can perform cross browser testing for your website on mobile OS. Select the icon for **‘Android’**  and choose from a variety of Android mobile devices to start testing. - -**Step 2:**  After choosing your configurations, click on the **‘Start’**  button to launch the VM (Virtual Machine). You will be able to access you locally hosted web pages. - -## Testing Locally Hosted Web Pages On iOS - -**Step 1:**  Go to **‘Real Time Test’**  from the left navigation menu and click on the **‘Mobile’**  icon. This will guide you to a screen where you can perform cross browser testing for your website on mobile OS. Select the icon for **‘Apple’**  and choose from a variety of iOS devices to start testing. - -**Step 2:**  After choosing your configurations, press the **‘Start’**  button to launch the VM. You will be able to access you locally hosted web pages. - ->**NOTE:** If `http://localhost:{port}`  does not work then try with either `http://localhost.lambdatest.com:{port}`  or your machine’s private IP `http://{IP}:{port}`. - ->Notice how the font changed from Android to iOS or rather from Google Chrome to Mozilla Firefox, in the above images of local testing on Android & iOS. That right there is a cross browser compatibility issue! - -## Testing Local Folder Files Using TestMu AI tunnel - ->**Note:**  If you were already running a test session through tunnel then you will have to terminate the existing tunnel connection in cmd before you re-establish it for a new directory. - -**Step 1:**  Navigate to **‘Configure Tunnel’**  button. - -**Step 2:**  Click on **‘Folder’**  button and copy the string on your system’s clipboard. - -**Step 3:**  After you establish a tunnel connection, go to TestMu AI and navigate to **Real Time test**. Enter `local-folder.lambdatest.com`  in URL bar. The list of all files/folders present under the selected directory will be displayed in the browser. Select the test configuration(OS, Browser and screen resolution) of your choice. Click on the **‘Start’**  button. - -Once the VM (Virtual Machine) is launched, you will be able to access your local folders on a testing environment hosted by our cloud servers. - ->**Note:**  Similarly, if you want to run any file in the specified directory, you need to give URL as `local-folder.lambdatest.com/filename`. -For example: If you have a file `'test.html'`  in your folder. You need to give URL as `local-folder.lambdatest.com/test.html`  to run the same file. -In this way, you can run any HTML, PHP, Python or similar web files present in any folder of your system, directly in TestMu AI browser. - -## Terminating the TestMu AI tunnel Connection - -A secure tunnel connection is maintained unless you explicitly disconnect it. You can disconnect the tunnel in following ways: - -**Option 1:**  You can kill the process by using **‘Ctrl+C’**  in Windows command prompt. -You will receive the below highlighted message **‘Tunnel stopped successfully’**  indicating the termination on Tunnel Connection. - -**Option 2:**  You can also disconnect it using the option provided in the user interface. In the top right corner of TestMu AI interface, you will find a tunnel icon. Click on this icon and then click **x**  to terminate the TestMu AI tunnel. - -## Geolocation Testing through VPN with TestMu AI tunnel - -Test the performance of your web-app through different geographic locations using VPN in TestMu AI tunnel. Make sure your website delivers well to your targeted audience from all around the globe. - ->**How can Geolocation Testing help you?** -**1. SERPs – Search Engine Result Pages:**  For a website that is targeting an audience from different parts of the globe. It is always recommended to have a constant check on SERPs to address your website according to the respective trends of different geographies. This is extremely necessary for clothing brands. Traditional wearing differs from one country to another. -**2. GDPR compliance – General Data Protection Rule Compliance:**  Payment Gateways may differ from one geography to another due to different GDPR compliance for different geographies. -**3. Billing:**  Your billing page should support currency conversion with respect to the IP. So if a user from the US would show up at your website then he/she should be billed in USD, similarly if one is visiting from France then the bill should be presented in Euros. -**4. Google Ads:**  They also differ from one geographic region to another. You need to check through IPs of different geolocations regarding which ads are visible and in what manner? -**5. Bandwidth:**  4G may or may not be available to every one of your targeted geography and even if it is available. The available bandwidth will usually be different. Meaning some countries would provide a faster download speed in 4G as compared to others. So you need to be mindful of the content optimization according to your targeted geolocations. Testing from IPs belonging to different geolocations will help you get a realistic view of page loading speed. A faster page loading will not only help in pleasing your customers, but it would also help in ranking higher in SERPs. -**How can you test from an IP of a different Geolocation?** -This is where a VPN comes to rescue! VPN – Virtual Private Network – It helps in expanding a private network through a public network. Initially, VPNs were introduced and used among large enterprises, defense organizations, banking companies, etc. to keep their data secure. A VPN created a pathway through which people working in large organizations could access the data they want, remotely, in a secure manner. The data is securely transmitted through the process of Cryptography. In Cryptography, the data is encrypted from the sender’s end and then it gets decrypted with the help of a unique key on the receiver’s end. -Times have changed and with it the popularity of VPN too! These days VPNs are so widely available that they are used by almost every one of us. Some use it to surf the internet without worrying about the restrictions applied to their GeoIP. Some use it for the purpose of maintaining anonymity. Some use it without even realizing the meaning of a VPN. -Remember though, not all VPNs will work in the same manner. Few may provide you considerably more geolocations to choose from as compared to others. Few may provide less number of geolocations but more number of dynamic GeoIPs with respect to others. There are many more parameters related to the comparison. - -Use the following arguments in your command line. - -|ARGUMENTS|EXAMPLE|DESCRIPTION| -| --- | --- | --- | -|`--key`|123asd123|Helps Lambdatest to map a tunnel connection with the right user.| -|`--proxy-host`|abc.wonderproxy.com|Full hostname for the proxy you’d like to use.| -|`--proxy-port`|12345|The port declared by you for fetching the proxy.| -|`--proxy-user`|abcdefg|WonderProxy username of the user.| -|`--proxy-pass`|hijklmnop|WonderProxy password of the user.| - -Here is how the format of full command would look like: - -``` -LT --user {user login email} --key {user key} --proxy-host {full proxy hostname} --proxy-user {proxy username} --proxy-pass {proxy password} --proxy-port {port number} -``` - -Example of full command: -``` -LT --user example@lambdatest.com --key 123asd123 --proxy-host abc.wonderproxy.com --proxy-user abcdefg --proxy-pass hijklmnop --proxy-port 12345 -``` -## Command References - -Following is the list of command arguments which can be used as modifiers while executing the binary file: - -|COMMAND|DESCRIPTION| -| --- | --- | -|`--user`|Registered email ID of the user.| -|`--key`|User access key for the user.| -|`--tunnelName`|(optional) Name of the tunnel.| -|`--port`|(optional) Local port to connect tunnel.| -|`--proxy-host`|Full hostname for the proxy you’d like to use.| -|`--proxy-port`|The port declared by you for fetching the proxy.| -|`--proxy-user`|WonderProxy username of the user.| -|`--proxy-pass`|WonderProxy password of the user.| - -## Regenerating Your Access Key Or Access Token - ->**What is an Access Key?** ->An Access Key is a unique string used for the identification of your account. -As we offer a cloud-based platform to host a testing ecosystem to everyone around the globe. Access key plays a crucial role in authenticating your TestMu AI account while mapping a secure tunnel connection for hosting your locally stored web pages online. They work as your IDs when you gain access to our cloud servers. -If you ever misplace your Access key into the hands of someone you don’t trust. You can always regenerate it by navigating to your TestMu AI Profile > Account Settings > [Password & Security](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/security/username-accesskey). - -You can find your Access Key while performing any type of testing. Go to **Credentials** in the left sidebar and click the copy icon next to Access Key to retrieve it. - -For instance, if you go to **‘Real Time Test’**, you will observe a button that says **‘Configure Tunnel’**  on the top-right area of the page. Click on that button and a form would open up through it, providing you with a string containing your access key. You will find the access key in the area underlined below. - -You can also find your **‘Access Key’**  by going to **‘Settings’**  in the left navigation menu and routing to **‘Profile’**. Also, you can regenerate your **‘Access Key’**  by clicking on tilt arrow button. - -**TestMu AI tunnel Update:**  Earlier, when our TestMu AI tunnel binary was updated with a new version, we requested our users to re-download the binary file for uninterrupted [cross browser testing](https://www.testmuai.com) experience of locally hosted web pages. Not anymore! We have now updated the TestMu AI tunnel binary in a way that it auto-updates to the latest version every time you execute the TestMu AI tunnel from cmd/terminal. diff --git a/static/docs/local-testing-with-puppeteer.md b/static/docs/local-testing-with-puppeteer.md deleted file mode 100644 index 0e1c89c36..000000000 --- a/static/docs/local-testing-with-puppeteer.md +++ /dev/null @@ -1,80 +0,0 @@ -# Local Testing Using Puppeteer - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI tunnel feature lets you test private server URLs, locally hosted web apps, and websites on 3000+ real browsers and operating systems. On TestMu AI, you can test plain HTML, CSS, PHP, Python, and other similar web files saved locally. When connecting to corporate firewalls or proxy settings, no restrictions apply to the new TestMu AI tunnel binary. To establish a secure and unique tunnel connection between your system and TestMu AI cloud servers, the TestMu AI tunnel utilizes various protocols like Web Sockets, HTTPS, SSH(Secure Shell), etc. - -Learn how to perform local testing using Puppeteer across 40+ real browsers and operating systems. - -## Puppeteer Testing Of Locally Hosted Websites - -You can run Puppeteer testing of locally hosted websites and web apps via TestMu AI tunnel binary. - -1. Clone the TestMu AI-Puppeteer repository on your system. - -2. Install the npm dependencies. - -``` -npm install -``` - -3. In order to run your Puppeteer tests, you will need to set your TestMu AI username and access key in the environment variables. - -**Windows** - -```js -set LT_USERNAME="YOUR_LAMBDATEST_USERNAME" -set LT_ACCESS_KEY="YOUR_LAMBDATEST_ACCESS_KEY" -``` - -**macOS/Linux** - -```js -export LT_USERNAME="YOUR_LAMBDATEST_USERNAME" -export LT_ACCESS_KEY="YOUR_LAMBDATEST_ACCESS_KEY" -``` - -4. To establish a tunnel connection between your local device and TestMu AI, download the binary file based on your OS. - -- Windows **[64 Bit](https://downloads.lambdatest.com/tunnel/v3/windows/64bit/LT_Windows.zip) | [32 Bit](https://downloads.lambdatest.com/tunnel/v3/windows/32bit/LT_Windows.zip)** -- macOS **[64 Bit](https://downloads.lambdatest.com/tunnel/v3/mac/64bit/LT_Mac.zip) | [32 Bit](https://downloads.lambdatest.com/tunnel/v3/mac/32bit/LT_Mac.zip)** -- Linux **[64 Bit](https://downloads.lambdatest.com/tunnel/v3/linux/64bit/LT_Linux.zip) | [32 Bit](https://downloads.lambdatest.com/tunnel/v3/linux/32bit/LT_Linux.zip)** - -5. Extract the downloaded binary file. - -6. Navigate to the directory or folder where you extracted the binary file in the Command Prompt. - -7. Run the below command in the terminal. - -```js -./LT --user {user's login email} --key {user's access key} --tunnelName {user's tunnel name} -``` - -8. In desired capability, add the capability `tunnel: true`. - -If multiple tunnels are running, you can add `tunnel` and `tunnel ID` capabilities. - -```js -'use strict'; -const { strict } = require('once'); -const puppeteer = require('puppeteer'); -const expect = require('chai').expect; - -(async () => { -const capabilities = { -'browserName': 'Chrome', -'browserVersion': 'latest', -'LT:Options': { -'platform': 'Windows 10', -'build': 'puppeteer-build-1', -'name': 'My first Puppeteer test', -'resolution':'1366x768', -'user': process.env.LT_USERNAME || "Your Username", -'accessKey': process.env.LT_ACCESS_KEY || "Your Access Key", -'tunnel': false, // Add tunnel configuration if testing locally hosted webpage -'tunnelName': '' // Optional -} -}; -``` - -You can view test reports for your local tests on the TestMu AI automation dashboard. diff --git a/static/docs/localstack-integration-with-testmu.md b/static/docs/localstack-integration-with-testmu.md deleted file mode 100644 index 812090f3f..000000000 --- a/static/docs/localstack-integration-with-testmu.md +++ /dev/null @@ -1,165 +0,0 @@ -# LocalStack Integrates With TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -LocalStack is a tool that acts as a drop-in replacement for AWS on your local machine. It works on your laptop or in your CI environment. With LocalStack, you can run your AWS applications or Lambdas on your own computer without needing to connect to a remote cloud provider! - -LocalStack and TestMu AI have partnered to create a special combination that makes your tests run faster. Using LocalStack's local cloud emulation abilities and TestMu AI's smart AI solutions, developers and testers can speed up their work and get products out quickly. This collaboration helps you test things faster and bring your products to market sooner. - -With this integration, teams can perform tests at an impressive speed without constant internet connectivity. By leveraging the advanced AI solutions from TestMu AI, teams can enjoy an unmatched experience when it comes to executing tests. This results in enhanced productivity and efficiency throughout the development and testing lifecycle. - -## Prerequisites - -1. A TestMu AI account. Don't have an account, register for free. -2. A LocalStack account. Don’t have an account, you can create one [here](https://app.localstack.cloud/sign-in). - -## Installing LocalStack CLI - -The easiest way to begin using LocalStack is by using the `LocalStack` CLI. This handy tool allows you to start LocalStack directly from your command line. However, before you start, ensure your machine has Docker pre-installed, and you can start your Docker engine. - -You can refer to the [LocalStack support documentation for installing LocalStack CLI](https://docs.localstack.cloud/getting-started/installation/) per your system requirements. - -## Generating LocalStack API Key - -In this section, we will set up out LocalStack account to generate our API keys. - -**Step 1:** To get generate your API key, head to **Account**. - -**Step 2:** Select the **API Keys** tab. - -**Step 3:** Generate and **copy your API Key**. - -## Starting LocalStack CLI - -To ensure LocalStack functions properly, your API key must be available in the environment variable called `LOCALSTACK_API_KEY`. You have two options for setting this variable: define it beforehand in your environment or set it while starting LocalStack using the LocalStack CLI. - -### **For macOS/Linux:** - -```bash -export LOCALSTACK_API_KEY= -localstack start -d -``` - -### **For Windows:** - -```bash -$env:LOCALSTACK_API_KEY= localstack start -d -``` -## Running LocalStack Test Cases on Your Local Machine - -To showcase how you can run LocalStack on your local machine and execute your tests, we will showcase a sample solution to resize images uploaded to a local S3 through a serverless application. The solution will feature a simple web user interface that uses local Lambda functions to generate S3 pre-signed URLs allowing the S3 bucket notifications to trigger a Python Lambda that runs image resizing. - -> **Note:** The code for the solution in this post is located in this [repository on GitHub](https://github.com/localstack-samples/sample-serverless-image-resizer-s3-lambda). You can clone this repository and install its local dependencies by executing the following commands in your terminal: - -```bash -git clone -git@github.com:localstack-samples/sample-serverless-image-resizer-s3-lambda.git -cd sample-serverless-image-resizer-s3-lambda -python -m venv .venv -source .venv/bin/activate -pip install -r requirements-dev.txt -``` - -The above commands will create a Python virtual environment inside the project directory, activate the virtual environment, and install the Python dependencies. To mock the `AWS` commands against LocalStack, you can use `awslocal`, our wrapper around the `AWS CLI`. - -**Step 1:** Install it using `pip`. - -```bash -pip install awscli-local -``` - -**Step 2:** After the installation of `awslocal`, you can create the AWS infrastructure on LocalStack by running the following command: - -```bash -bin/deploy.sh -``` - -The above command will create the **S3 buckets**, put the bucket names into the parameter store, create the `DLQ Topic` for failed Lambda invokes, create the Lambda functions, connect the S3 bucket to the resizer lambda, and create the web application. - -> **Note:** You can visit **webapp.s3-website.localhost.localstack.cloud:4566** on your web browser and paste the function URL of the pre sign Lambda function you created earlier. - -**Step 3:** You can fetch the function URL by running the following commands: - -```bash -awslocal lambda list-function-url-configs --function-name presign -awslocal lambda list-function-url-configs --function-name list -``` - -**Step 4:** You can now upload an image file and download the resized file from the `localstack-thumbnails-app-resized` bucket. - -**Step 5:** To run the automated integration tests, use `pytest`. Run the following command to execute the tests for your application: - -```bash -pytest tests/ -``` - -Now that you have successfully set up an AWS Serverless application on our local machine, executed integration tests, and created local AWS resources, you can look into how to accelerate your testing over HyperExecute to achieve agile & nimble test-driven development. - -## Running LocalStack Test Cases On Your HyperExecute - -One way to run LocalStack test cases on HyperExecute is by leveraging GitHub Actions, the in-built continuous integration, and continuous delivery (CI/CD) platform provided by GitHub. To setup HyperExecute on GitHub Actions, we will create two files: - -* A GitHub Action workflow located at .github/workflows/main.yml downloads the HyperExecute CLI binary and sets it up. -* A HyperExecute file located at he.yml declares the workflow of the pipeline execution via a YAML configuration. - -**Step 1:** Navigate to the root directory and click on **Add File** button. Name the file `he.yml` and add the following configuration: - -```bash -version: "0.1" -runson: linux -autosplit: true -parallelism: 2 -concurrency: 2 -scenarioCommandStatusOnly: true -runtime: -language: python -version: 3.9.16 -pre: -- pip install -r requirements-dev.txt -- LOCALSTACK_API_KEY=${{ .secrets.LOCALSTACK_API_KEY }} localstack start -d -- localstack wait -t 60 -- bin/deploy.sh -testDiscovery: -type: raw -mode: remote -command: pytest --co -q tests | sed '$d' -testRunnerCommand: pytest $test -sourcePayload: -platform: git -link: https://github.com/macnev2013/sample-serverless-image-resizer-s3-lambda -ref: main -accessToken: ${{ .secrets.PAT }} -``` - -**Step 2:** Commit the he.yml file on your repository. Ensure that the `LOCALSTACK_API_KEY` secret has been configured in your GitHub repository as a secret. - -**Step 3:** Navigate to the **Settings** page of the repository, and from the sidebar, click **Secrets and Variables**. Click **Actions** and click the **New Repository Secret** button to add the `HE. LOCALSTACK_API_KEY`. This key is added to the HyperExecute Portal. - -**Step 4:** To create the GitHub Actions pipeline `YAML` file, navigate to the repository where your code is available. In this example, you can assume that we are adding this to the previous code sample that we demonstrated. - -**Step 5:** Under your repository name, click **Actions**. In the left sidebar, click the **New Workflow** button. Add the following `YAML` configuration to the workflow: - -```bash -name: HyperExecute -on: -push: -branches: -- main -jobs: -HE: -runs-on: ubuntu-latest -steps: -- uses: actions/checkout@v2 -- run: | - -wget https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute -chmod +x hyperexecute -./hyperexecute --user nevil.macwan --key ${{ secrets.HE }} ---config he.yaml -``` - -**Step 6:** Save the configuration in your GitHub Actions pipeline. Below is an example of how a test that is run on the GitHub Actions pipeline gets executed: - -Below is an example of a **HyperExecute** job that was triggered through the above pipeline: - -> That was all you need to know for TestMu AI + LocalStack Integration. Increase your productivity with our integrations. If you still have any questions for us, please feel free to let us know. Our experts are always available on **chat** to help you out with any roadblock regarding our product. Happy testing! diff --git a/static/docs/login-to-google-account-on-android.md b/static/docs/login-to-google-account-on-android.md deleted file mode 100644 index 2198e3e63..000000000 --- a/static/docs/login-to-google-account-on-android.md +++ /dev/null @@ -1,104 +0,0 @@ -# Login to Google account on Android - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This documentation will guide you to log in to your Google Account on an Android Device on TestMu AI so you can test features/services that are linked to your Google Account. - -## Objective - -To log in to Google Account on Real Devices [Android], and perform testing using Google account-related tasks. - -## Prerequisites - -1. TestMu AI App Automation Account. -2. Automation script for testing. - -## Logging into Google Account - -Add the following HashMap structure to store the credentials: - -```java -HashMap playstorelogin = new HashMap<>(); -playstorelogin.put("email", "EMAIL_ID@gmail.com"); //Add in your Google account ID -playstorelogin.put("password", "GOOGLE_PASSWORD"); //Add in your Google account Password -``` - -Once we have added this HashMap, we can now add the capability linking this HashMap to the capability. - -```java -capabilities.setCapability("playStoreLogin", playstorelogin); -``` - -As shown, please add your Google Account ID and password in the fields that correspond to the same. - -Once you have done that, run the App Automation test and the test will start with the device already logged into the account with the mentioned credentials. - -Once the test has started, you can also verify the same by opening Google Play Store. - -A sample code showing the addition of Google Account is shown below: - -```java -import io.appium.java_client.AppiumDriver; -import org.openqa.selenium.remote.DesiredCapabilities; - -import java.net.MalformedURLException; -import java.net.URL; -import java.util.HashMap; - -public class gmailLoginCap { -//highlight-next-line -public static String userName = System.getenv("LT_USERNAME") == null ? "LT_USERNAME" //Add LambdaTest username here -: System.getenv("LT_USERNAME"); -//highlight-next-line -public static String accessKey = System.getenv("LT_ACCESS_KEY") == null ? "LT_ACCESS_KEY" //Add LambdaTest accessKey here -: System.getenv("LT_ACCESS_KEY"); - -private static AppiumDriver driver; - -public static void main(String[] args) throws MalformedURLException, InterruptedException { -DesiredCapabilities capabilities = new DesiredCapabilities(); - -capabilities.setCapability("deviceName", "Pixel 5"); -capabilities.setCapability("platformVersion", "11"); -capabilities.setCapability("platformName", "android"); -capabilities.setCapability("isRealMobile", true); -capabilities.setCapability("video", true); -capabilities.setCapability("visual", true); -capabilities.setCapability("network", false); -capabilities.setCapability("devicelog", true); -capabilities.setCapability("console", true); -capabilities.setCapability("project", "labs"); -capabilities.setCapability("build", "gmailLoginCap"); -capabilities.setCapability("unicodeKeyboard", true); -capabilities.setCapability("autoGrantPermissions", true); -capabilities.setCapability("app", "APP_URL"); //Add in your app ID -capabilities.setCapability("autoGrantPermissions", true); -HashMap playstorelogin = new HashMap<>(); -playstorelogin.put("email", "EMAIL_ID@gmail.com"); //Add in your Google account ID -playstorelogin.put("password", "GOOGLE_PASSWORD"); //Add in your Google account Password -capabilities.setCapability("playStoreLogin", playstorelogin); - -AppiumDriver appiumDriver = new AppiumDriver( -new URL("https://" +userName + ":" + accessKey + "@mobile-hub.lambdatest.com/wd/hub"), capabilities); - -try { -Thread.sleep(5000); -appiumDriver.findElementById("google").click(); -Thread.sleep(15000); -appiumDriver.quit(); - -} catch (Exception e) { -appiumDriver.quit(); -} -} -} -``` - -**Important**: - -1. The account should not be using any kind of 2-factor authentication, otherwise, the test can fail. -2. The account should not require you to allow login using a trusted device using a pop-up or OTP. - -**Note** - -Since we clean the Device after each session, ensure you are logging in to the account during each session. diff --git a/static/docs/lt-browser-2.md b/static/docs/lt-browser-2.md deleted file mode 100644 index e854e3bd3..000000000 --- a/static/docs/lt-browser-2.md +++ /dev/null @@ -1,297 +0,0 @@ -# Getting Started With LT Browser 2.0 - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -> **Note:** This product is deprecated and as a result, no further support will be provided for this feature by the TestMu AI Organization. - -[LT Browser 2.0](https://www.testmuai.com/lt-browser) is a Chromium-based mobile-friendly browser that enables web developers to build, test and debug their mobile websites for responsiveness across a wide range of 50+ pre-installed device viewports, including mobile, tablet, desktop and laptop. - -The new LT Browser 2.0 boasts enhanced performance and several features that make it easier for you to create responsive websites for device viewports of all sizes. - -The following new features are available in LT Browser 2.0: - -* Based on latest version of Chromium. -* More clear and attractive UI. -* Interact and test with 6 devices simultaneously. -* Dedicated DevTools for different viewports. -* Test with unlimited custom device viewports. -* Generate and share multiple error reports. -* Use Chromium settings. -* Install Chrome extensions. -* Unlimited browser tabs to launch device viewports. -* Record with entire screen or browser tab. - -To get started with LT Browser 2.0, download the below executable file based on your operating system. - -Download LT Browser 2.0 For Windows - -Download LT Browser 2.0 For macOS - -## Running Your First Responsive Test - -1. Open the LT Browser 2.0 and select your preferred device from the left device panel. *By default, LT Browser 2.0 opens up with two viewports.* - -2. Provide the testing URL in the address bar, and then press **Enter**. - -3. It will launch the website on the selected device viewport. - -## Adding More Devices - -You can monitor different devices side by side by adding a device on the view screen. With LT Browser 2.0, you can interact with 6 device viewports at a time. - -1. To add more viewports, click **+ Add Devices**. - -2. From the pop-up box, select your desired viewport that you want to test side-by-side and Click **ADD DEVICES** - -You can now find a new device viewport on the view screen. This will allow you to test responsive web design on four different viewport sizes. - -## Rotating Device - -You can rotate devices from Portrait To Landscape view or vice versa. To rotate device, click on the rotate icon. - -## Adding A Custom Device - -If a device does not match the viewport or device pixel ratio of your requirement, you can add a custom device viewport. With LT Browser 2.0, you can add unlimited custom viewports. - -1. To add custom viewport, click **+ Add Devices** . - -2. Select the **Custom** tab and click **+ New Device**. - -3. A pop-up box will appear where you need to specify the following viewport details. - -* Device Name -* View Port Specifications (Width and Height) -* Device Pixel Ratio -* User Agent String -* Device Platform - -After entering the viewport details, click **ADD DEVICE** - -It will add your custom device based on the provided viewport details. - -## Searching For A Specific Device - -LT Browser 2.0 includes a search bar at the top-left of the screen to enable you to instantly locate any device from the device panel. Enter the brand of the device, select a device from the list and it will appear instantly on-screen. - -If you search for a device viewport which is not yet listed in the LT Browser 2.0 device panel, you will see **Create your own device** option that allows you to add your own custom device. - -## Generating Performance Report - -With LT Browser 2.0, you can view the performance report of your page instantly powered by Google Lighthouse. The Performance Report allows you to get a holistic view of your website or web app and gain actionable insights to improve your mobile web experience. - -With Performance Report you can view: - -* Desktop and Mobile Score - -* Performance Metrics - -* Runtime Settings - -1. To generate a performance report, enter the desired URL in the address bar and press Enter. - -2. Click on **Performance** tab in the header. - -It will take a few seconds for the Performance Report to generate. - -### Desktop and Mobile Score - -The Desktop and Mobile Score contains the universal score of the desktop and mobile scores. The score has three fields - Poor, Average, and Optimum. - -### Performance Metrics - -You can view Desktop and Mobile Score based on different parameters like Accessibility, Best Practices, SEO, and Progressive Web App (PWA). The Opportunities and Diagnostics section help you with guidelines to load your page faster. - -Towards the bottom, you can find the device resolution used to generate performance report along with the loading time. Also, view the DOM and Browser processing duration of your web page. These timestamps will change as per your choice of network simulation. - -### Runtime Settings - -In Runtime Settings, you'll find the URL of the entered website, time, and date. - -You can also share your performance report with your team members via e-mail or a shareable link. To share you report, click the Share icon. - -## Network Throttling - -LT Browser 2.0 allows you to test your mobile website view over different types of network connections, including fast 3G, slow 3G, online and even offline. You can change your network profile to test its functionality when exposed to different network environments. - -1. To test your website's performance over different network environments, click the Network Throttling icon. - -2. Select the network environment from the network list. - -Once you are done with selecting the network, you can test your mobile view of your website over the selected network. - -## Sync Devices - -Sync Devices synchronize your scrolls across multiple devices. It lets you scroll through different devices so that scrolling on one device will scroll the same amount on another device. Using Sync Devices, you can compare your website side-by-side over other devices via mirrored interactions. - -Using LT Browser 2.0, you can compare your website view on multiple device viewports. - -Navigate to the Settings icon and click **Sync Devices**. - -You can now compare your website view side-by-side on different viewports. - -## Capturing Full-Page Screenshots Of Websites - -With Full Page Screenshot feature, capture full-page screenshots of your website's long content and compare it across multiple devices. - -1. Navigate to the Settings icon and click **Full Page Screenshot**. - -2. Click on the Camera icon. - -Upon clicking on the Screenshot icon, you will receive a prompt message stating **Full Page screenshot is in progress**. - -3. Once the screenshots are captured, it will guide you to the page where you can download your full-page screenshots into the LT Browser 2.0 by pressing the 'Save' button or you can download your screenshots into your local machine by clicking on the 'Arrow' button. - -*To save screenshots on local system, enable **Save Media on Cloud** option from the Settings icon on the top-right.* - -## Viewing Error Reports - -Error report lets you check for web-page errors, such as services that fail to load, and alerts you when the issue occurs. With LT Browser 2.0, you can view multiple errors across different device viewports. - -1. To view error reports for websites, launch your website on your preferred viewports. - -2. If your website has errors, you will see a red error icon below the viewport of your device. Click on the error icon to view the error reports. - -LT Browser 2.0 allows you to mark errors in an error report as bugs. To do this, select the error that you want to mark and click **Mark as Bug**. - -You can also share error reports with your teammates and colleagues by clicking on Share icon. - -## Installing Chrome Extensions - -LT Browser 2.0 enables you to install your favorite Chrome extensions directly from the Chrome web store. - -1. Navigate to **More tools > Extensions** from the top-right of the LT Browser 2.0. - -It will guide you to the below screen. - -2. You can either search for your favorite Chrome extensions or drag and drop the CRX file from your local system to the Extensions page. - -3. Once you drop the file, click **Add extension** to confirm the installation. - -The Chrome extension you just installed will now appear in the LT Browser 2.0. - -## Marking And Sharing Bugs - -*To mark bugs, enable **Enable Mark as Bug** option from the Settings icon on the top-right.* - -While performing responsive testing, you can mark any website issues as bugs. - -1. If you come across an issue, click on the Camera icon to capture the screenshot of your mobile website. - -*In case you want to capture full-page screenshots, enable **Full Page Screenshot** option from the Settings icon on the top-right.* - -2. Use the in-built annotation tool bar to mark any issues, bugs and tasks on a web page. Then, click **Mark as Bug.** - -When you click the Mark as Bug, you'll be presented with a list of TestMu AI integrations. If your favorite integration isn't presented, then you can click **More Integrations** to view all available TestMu AI integrations. - -Once you integrate the LT Browser 2.0 with a third-party tools like Jira, GitLab, GitHub, and more, you can log bugs directly from the LT Browser 2.0 to your project. - -### Sharing Bugs - -To make it easier to share bugs across your teams, LT Browser 2.0 offer multiple ways to share a screenshot of the logged issues. - -#### Downloading The Screenshot - -You can download the highlighted screenshot in your system, then share it with your team through email or instant messaging. - -#### Saving The Screenshot - -After you capture a screenshot, you can save it directly on the LT Browser 2.0 by clicking **Save**. - -This will save your screenshot in the Media gallery. You can access this gallery by clicking the **Media** tab in the header. - -#### Sharing The Screenshot To Multiple Email IDs - -Click on the Share icon. - -You can share a bug directly to your teammate's email ID. You can include an optional note in order to help your colleague understand the nature of the bug. - -You can also copy the shareable link for sharing your bugs by clicking on **Get Shareable Link**. - -## Recording Video Of A Test Session - -1. Open your website in any device viewport and click the Video recorder icon. - -2. It will ask you to select how you want to record from two options: **Entire Screen** or **LT Browser Tab**. Select either of the option and click **Share**. - -A timer will appear at the top of the screen when you begin a live testing session. The timer continues to run unless you stop the recording by clicking the same Video Recording icon or **Stop sharing**. - -## Accessing Media - -To view your screen captures and recordings, visit the **Media** tab. In the Media section, you can find all your screenshots and video recordings. - -Media has two tabs: **Images** and **Videos** where you can download, share, rename or delete any image or video from the Media section. - -The **Videos** tab shows full recordings of your live interactions from different test sessions. - -## Testing Locally Hosted Websites Or Web Apps - -To test your locally hosted web apps or websites, you need to provide the localhost URL in the address bar of LT Browser 2.0. - -The following screenshot shows Apache server running on a system, hosting an image located at XAMPP's htdocs folder. - -Now open the localhost in the LT Browser 2.0 and add the file path you want to test. In this case, the localhost URL will be: `http://localhost/home_banner.webp`. - -The locally-hosted image will be rendered on your selected device viewports. - -## Testing Web Pages Behind Login - -LT Browser 2.0 aims to provide responsive testing and a live-interactive testing experience. You can load a website, login to the LT Browser 2.0 web application, and start testing your pages. - -Following screenshot shows TestMu AI web-application that is captured after logging in from the TestMu AI website. - -## Debugging Your Websites - -With LT Browser 2.0, you can see how your responsive design looks over 50+ device viewports and debug web applications with dedicated DevTools for different device viewports to determine the root cause of a browser or UI bug. - -To debug your website, click on the debug icon of the respective viewport. - -Once you click on the debug icon, you can access the developer tools for mobile view debugging. - -## Keyboard Shortcuts - -Here are the keyboard shortcuts for LT Browser 2.0. - -**macOS** - -| SHORTCUT | PURPOSE | -|-------------------|-------------------------| -| Command+Z | Undo | -| Shift+Command+Z | Redo | -| Command+X | Cut | -| Command+C | Copy | -| Command+V | Paste | -| Command+A | Select All | -| Ctrl+Command+F | Toggle Full Screen | -| Command+M | Minimize | -| Command+Q | Quit | -| Command+T | Opens a new tab. | -| Command+W | Close the active tab. | -| Command+Shift+M | Navigate to Media page. | -| Command+Shift+H | Navigate to Home page | -| [ | Expand and collapse the sidebar | -| Shift+P | Viewing the performance report | -| Command+Shift+R | Replacing or adding a right device | -| Command+Shift+D | Removing the right device | - -**Windows & Linux** - -| SHORTCUT | PURPOSE | -|-------------|---------------------| -| Ctrl+R | Reload | -| Ctrl+Z | Undo | -| F11 | Toggle Full Screen | -| Alt+F4 | Close | -| Ctrl+X | Cut | -| Ctrl+C | Copy | -| Ctrl+V | Paste | -| Ctrl+T | Opens a new tab. | -| Ctrl+W | Close the active tab. | -| Ctrl+M | Minimize. | -| Ctrl+Shift+M | Navigate to the Media page.| -| Ctrl+Shift+H | Navigate to the Home page. | -| Shift+Ctrl+Z | Redo | -| Ctrl+Shift+D | Removing the right device | -| Ctrl+A | Select All | -| [ | Expand and collapse the sidebar | -| Alt+P | Viewing the performance report | -| Ctrl+Shift+R | Replacing or adding a right device | diff --git a/static/docs/lt-browser.md b/static/docs/lt-browser.md deleted file mode 100644 index 3f538f28a..000000000 --- a/static/docs/lt-browser.md +++ /dev/null @@ -1,253 +0,0 @@ -# LT Browser - Test Website For Responsiveness Easily - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -LT Browser allows you to ensure your website's responsiveness over a variety of major devices and view ports. You can open a website in the [LT Browser](https://www.testmuai.com/lt-browser) and perform live testing across 50+ pre-installed device viewports. - -To get started, you need to download the executable file from here. - -Download LT Browser For Windows - -Download LT Browser For macOS - -Download LT Browser For Linux - -**Note:** After you are done with the installation, you will need to login to the LT Browser through your TestMu AI login credentials. - ->This document will help you: -> ->* Run First Responsive Test With LT Browser ->* Add A Custom Device To LT Browser ->* Search Devices In The LT Browser ->* Capture Screenshot & Highlight Bugs ->* Share Highlighted Bugs From The LT Browser ->* Record Video Of A Test Session ->* Find Recorded Videos & Highlighted Images In Media ->* Test Locally Hosted Web-Apps or Websites ->* Test Web Pages Behind Login ->* Debug Your Website Using LT Browser ->* [View Performance Report Of Your Website Using LT Browser](/support/docs/performance-report/) ->* [Network Throttling To Test Website Responsiveness](/support/docs/network-throttling-for-responsive-testing/) ->* [What Is Scroll Sync In LT Browser?](/support/docs/scroll-sync/) ->* [Capturing Full-Page Screenshots Of Your Websites](/support/docs/full-page-screenshots/) ->* Using Mouse Pointer For Touch Mode ->* [Getting The JavaScript Error Report Of Your Website](/support/docs/error-reports/) ->* What Is LT Browser Feedback Board? ->* Keyboard Shortcuts - -## Running Your First Responsive Test With LT Browser - -**Step 1:** Open the LT Browser. Provide the testing URL in the address bar and hit **Enter**. - -**Step 2:** Once you hit Enter, the website would be loaded on 50+ devices. You can select any device to start [responsive testing](https://www.testmuai.com/responsive-test-online) from the sidebar on the left. You can also monitor two different devices side by side by adding a device on the view screen. - -**Step 3:** After you click on the + button to add a new device, you will be prompted with a screen from which you can select the specific device you want to test side by side. Select the device and hit the button to **add device**. - -**Step 4:** You will now find the added device on the view area inside LT Browser. That way you can compare responsive web design on 2 different device through a side by side view. - -### Rotating Device From Portrait To Landscape View Or Vice Versa - -You can rotate any or both devices by clicking on the highlighted button with the **rotate icon**. - -### Couldn't Find Your Favourite Device? Add A Custom Device - -Although, we try our best to deliver the most majorly used devices to our user as per the global market share. However, in case you are unable to find a device that matches the view port or device pixel ratio of your requirement then you can even add a custom device to your LT Browser. To do so, you need to click on the tab which says **add a custom device**. - -You will now be routed to the below screen where you can specify device details. You can specify: - -* Device Name -* View Port Specifications -* Device Pixel Ratio -* User Agent String -* Device Platform - -Let's try adding OnePlus 5 to the LT Browser. We will provide the mandatory details to add OnePlus 5 as a custom device. - -As you click on add a device, you will find the custom device **One Plus 5** added to your [mobile view](https://www.testmuai.com/mobile-view-website) area on the LT Browser. You will also find it added over the top of sidebar on the left with a bin icon to delete the custom device, if needed. - -## How To Search For A Specific Device In The LT Browser? - -If you open the LT Browser, you will notice a search bar displayed over the top-left. You can leverage the search bar to quickly find your favourite devices. After you search a string, you can select the particular device & the selected device will be rendered over the browser instantly. - ->**Fun Fact:** If you search for a device name which isn't added to your LT Browser yet, you will find a button that will help you instantly add a custom device. -> -> - -## How To Capture A Screenshot & Highlight The Bugs? - -**Step 1:** Click on the **camera icon** to [capture screenshot](https://www.testmuai.com/full-page-screen-capture) of your website inside a mobile device screen of your choice. - -**Step 2:** Once you click on the camera icon, a built-in image editor would open up to present the captured screenshot of the selected device's screen. - ->**Note:** The camera button won't capture a full-page screenshot but will capture screenshot of the view in the screen of a particular device. You can leverage our **[Automated Screenshot Testing](/docs/automated-screenshot-testing/)** to capture full-page screenshots of your website over multiple browsers + OS combinations in a single test session. - -### How To Edit Captured Screenshot? Use Built-In Image Editor - -TestMu AI offers a built-in image editor to help you easily mark the bugs over your website and share it across your teammates. You can highlight the bugs in various shapes with different colors. Let us have a look at everything that you can do with the image editor. - -#### Draw A Line - -#### Highlight Bug With A Circle - -#### Draw A Rectangle - -#### Highlight The Bug From Multiple Colors - -#### Add Text In The Captured Screenshot - -#### Freestyle Highlighting With Marker - -#### Removing Your Edits With Eraser - -#### Undo Your Edits - -## How To Share The Highlighted Bugs With The Team? - -We understand that one of the most streneous task for any tester is to collect & share the bugs across your teams. Keeping that in mind, we offer multiple ways for you to share the highlighted screenshot with others through LT Browser. - -### 1\. Download The Screenshot In Your System - -You can download the highlighted screenshot in your system and share it with your team as you like through emails, instant messaging or more. - -### 2\. Saving The Screenshot Over LT Browser - -After capturing the screenshot, you can save the screenshot over the LT Browser by clicking the **Save** button. - -This will save your screenshot inside the **Media** over your LT Browser. You can find these saved screenshots by clicking **Media** in the header of the LT browser. - -### 3\. Sharing The Screenshot To Multiple Email IDs - -**Step 1:** After highlighting the screenshot, click on the share icon. - -**Step 2:** You will be prompted with the below screen where you can enter the note or description around your observation of the screenshot. You can also feed multiple Email IDs and as you hit the **Send** button an email is sent to the specified Email IDs from the LT Browser. This E-mail would contain your highlighted screenshot as well. - -### 4\. Get Shareable Link - -You may have noticed the option to **get shareable link** after you hit the share icon. - -If you click on it, you will generate a shareable link to access the highlighted image. You can copy the generated link and share it with your teammates. - ->**Note:** You can set a validity to the shared screenshots and shareable link. -> -> - -### 5\. Mark As Bug Directly To Third-Party Tools - -TestMu AI offers numerous integrations to third-party tools for project management, bug tracking, instant messaging, CI/CD and more. Once you integrate your TestMu AI account with your desired third-party integration. You can leverage the mark as bug button to push the bugs directly to your projects over third-party platforms. Meaning, you can share your highlighted screenshots across Jira, asana, Trello, slack and more. - -As you click on the mark as bug button, you will be presented with the screen which will present you few of TestMu AI integrations. If your favorite integration isn't presented then you can click on **more integrations** to view all of the [TestMu AI integrations](https://www.testmuai.com/integrations). - -Once you integrate the LT Browser with your favourite third-party integrations then you can log bugs directly from the LT Browser to your project over a third-party tool such as GitLab, GitHub, and more. - -## How To Record Video Of A Test Session? - -**Step 1:** Open you website in any device and click on the **video recording button**, you will be able to record your interaction of the website within the device. - -**Step 2:** Perform live-interaction with your website. You will find a timer on top of the device. This timer indicates the length of the video recording of your live testing session. The timer will continue to run unless you stop the recording by clicking over the same **video recording button**. - -**Step 3:** After you stop the recording, you will find a message on top of the LT Browser indicating that your video of the test session has been successfully saved. - -Now, to access this video you will need to go to Media. - -## What Is Media? - -To access any of your captured screenshots or recorded video, visit the Media. Consider Media as a Gallery which will comprise all your images and video that you may capture while testing your website through the LT Browser. - -In Media, you'll find tabs for Images & Videos. You can download, share, and delete any image or video from the Media tab. And similar to in-built image editor, you can mark a bug from an image stored in Media as well. - -If you switch to Video tab, you will find full recordings of your live interaction from different test sessions. - ->**Quick Access to Media**: You can also use **keyboard shortcut** to visit Media instantly. -> ->* Windows: `Ctrl+M` ->* macOS: `Command+M` ->* Linus: `Ctrl+M` - -## How To Test Locally Hosted Web-Apps or Websites? - -All you need to do for testing your locally hosted web-apps or websites, is to provide the localhost string as the URL in the header of your **LT browser** and you'll be able to access your localhost server. - -For example, if you are running localhost with Apache through XAMPP and have your website in XAMPP's **htdocs** folder. - -All you have to do is open the localhost and append the URL with the webpage or website-folder you want to test. - -It is as simple as that! - -## Can I Test My Web Pages Behind Login? - -Yes, you can! LT Browser aims at providing responsive testing along with live-interactive testing experience. You can simply load a website, login to the web-application and start testing your web pages that come after login. - -Here is a screenshot of the TestMu AI web-application which is captured after logging in from the TestMu AI website. - -## How To Debug My Website In LT Browser? - -With LT Browser, not only can you evaluate how your [responsive web design](https://www.testmuai.com/blog/responsive-web-design-all-you-need-to-know/) might look over 50+ device viewports but you can also debug your web-application to figure out the root cause behind a browser or UI bug. - -To debug your website, click on the highlighted button with the debug icon. - -As you click on the debug icon, you will be able to access the developer tools. - -Ta da! You can now perform responsive testing and debug any issues on the go. Cheers! - -## What Is LT Browser Feedback Board? - -A feedback board is an ideal way to acquire immediate feedback from the users. You can use the LT Browser feedback board to leave your suggestions/feedback on LT Browser. - -You can leverage feedback board to give feedback or suggestions about the next features to develop, integrations to implement, issues and enhancements on LT Browser. - -**Step 1**: To submit the feedback,click on the **Feedback** icon. - -**Step 2**: Click on **ADD icon** that says *Give Feedback*. - -**Step 3**: A Feedback form will open up where you will ask to enter your feedback details like title, description, category and images. After furnishing the required details, click on the **Submit button**. - -## Keyboard Shortcuts - -You can't really work with a browser which doesn't have any keyboard shortcuts, right? Well, don't worry! LT Browser has got your back on that! Here are the **keyboard shortcuts for LT browser**. - -### For macOS - -| KEYBOARD SHORTCUT | WHAT DOES IT DO? | -|-------------------|-------------------------| -| Command+Z | Undo | -| Shift+Command+Z | Redo | -| Command+X | Cut | -| Command+C | Copy | -| Command+V | Paste | -| Command+A | Select All | -| Ctrl+Command+F | Toggle Full Screen | -| Command+M | Minimize | -| Command+Q | Quit | -| Command+T | Opens a new tab. | -| Command+W | Close the active tab. | -| Command+Shift+M | Navigate to Media page. | -| Command+Shift+H | Navigate to Home page | -| [ | Expand and collapse the sidebar | -| Shift+P | Viewing the performance report | -| Command+Shift+R | Replacing or adding a right device | -| Command+Shift+D | Removing the right device | - -### For Windows & Linux Operating Systems - -| KEYBOARD SHORTCUT | WHAT DOES IT DO? | -|-------------------|-------------------------| -| Ctrl+R | Reload | -| Ctrl+Z | Undo | -| F11 | Toggle Full Screen | -| Alt+F4 | Close | -| Ctrl+X | Cut | -| Ctrl+C | Copy | -| Ctrl+V | Paste | -| Ctrl+T | Opens a new tab. | -| Ctrl+W | Close the active tab. | -| Ctrl+M | Minimize. | -| Ctrl+Shift+M | Navigate to the Media page.| -| Ctrl+Shift+H | Navigate to the Home page. | -| Shift+Ctrl+Z | Redo | -| Ctrl+Shift+D | Removing the right device | -| Ctrl+A | Select All | -| [ | Expand and collapse the sidebar | -| Alt+P | Viewing the performance report | -| Ctrl+Shift+R | Replacing or adding a right device | - -That is all about the LT browser. Don't forget to share you experience or questions with us. You can drop us an email through [support@testmuai.com](mailto:support@testmuai.com) or you could wave us a hello over our **24/7 customer chat support**. Happy testing! :) diff --git a/static/docs/mainframe-testing-with-hyperexecute.md b/static/docs/mainframe-testing-with-hyperexecute.md deleted file mode 100644 index 027399703..000000000 --- a/static/docs/mainframe-testing-with-hyperexecute.md +++ /dev/null @@ -1,227 +0,0 @@ -# Mainframe Testing with HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Run a Robot Framework suite that drives a mainframe 3270 terminal on TestMu AI HyperExecute. This guide uses the [FNB mainframe sample repo](https://github.com/alif-laam-Meem/mainframe-robot), which exercises a mocked bank host over the 3270 protocol. - -Mainframe suites depend on the `s3270` and `x3270` 3270 emulator binaries. Those do not install cleanly on Windows or macOS. HyperExecute provisions Ubuntu VMs where they do, installs them from one YAML file, starts the mock hosts, autosplits the suites across parallel machines, and merges the Robot Framework report. - -The sample suite runs on two tracks. The ASCII suites (`tests/01`-`05`) talk to the mock host through a custom socket client (`libs/fnb_client.py`), because Python removed the built-in `telnetlib` in 3.13 under PEP 594. - -The TN3270 suites (`tests/06`-`10`) use the standard [`robotframework-mainframe3270`](https://pypi.org/project/robotframework-mainframe3270/) library driving `s3270`, and the final suite drives the `x3270` GUI. - -**What a mainframe suite needs on the grid** -Three things a standard cloud run does not, all handled in the HyperExecute YAML: - -- The native emulator binaries (`s3270`, `x3270`) and X core fonts, installed on the VM by the [`pre`](/support/docs/deep-dive-into-hyperexecute-yaml/#pre) steps. -- The mock hosts, kept alive for the whole job as [background services](/support/docs/hyperexecute-background-services/). -- A rendered display for the GUI terminal, provided by [`captureScreenRecordingForScenarios`](/support/docs/deep-dive-into-hyperexecute-yaml/#capturescreenrecordingforscenarios), which `x3270` draws to. Do not add your own `DISPLAY` or `Xvfb`. - -## Prerequisites - -Everything the tests need at runtime (`s3270`, `x3270`, X fonts, Python packages, the mock hosts) is installed on the VM by the YAML, not on your machine. - -- Your TestMu AI Username and Access Key, with HyperExecute enabled on your plan. -- The HyperExecute CLI binary in the root of your suite. The sample repo already bundles the Windows build. -- The sample repo cloned locally (Step 1), or your own suite with a `tests/` folder of `.robot` files. -- Outbound HTTPS to `*.lambdatest.com`. Behind a corporate proxy, open an exception for it. - -## Step 1: Clone the Sample Repo - -Clone the FNB mainframe suite. It ships the `hyperexecute.yaml`, the Robot suites under `tests/`, and the mock hosts under `mock_server/`. - -```bash -git clone https://github.com/alif-laam-Meem/mainframe-robot -cd mainframe-robot -``` - -## Step 2: Set Your Credentials - -The CLI reads your Username and Access Key from environment variables or from `--user` / `--key` flags. Environment variables keep the key out of your shell history. - -Get them from your [Account Settings](https://accounts.lambdatest.com/security/username-accesskey) page, then export them in the terminal you will run from. - - {`$env:LT_USERNAME = "${ YOUR_LAMBDATEST_USERNAME()}" -$env:LT_ACCESS_KEY = "${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -Environment variables are session-scoped. Set them again in each new terminal, or add them to your PowerShell `$PROFILE` or shell rc file to persist them. - -## Step 3: Download the HyperExecute CLI - -The CLI triggers the job. Keep the binary in the root directory of your suite. The sample repo already bundles `hyperexecute.exe` for Windows. For macOS or Linux, download the matching build. - -| Platform | HyperExecute CLI | -| --- | --- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| macOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -On macOS, if you get a permission-denied warning, run `chmod u+x ./hyperexecute`. If a security popup blocks it, allow it under **System Settings** → **Privacy & Security**. On Windows, if the download is quarantined, run `Unblock-File .\hyperexecute.exe`. - -## Step 4: Understand the HyperExecute YAML - -The `hyperexecute.yaml` in the repo root is where the mainframe setup lives. This is the full file. The `runson`, `background`, `pre`, `testDiscovery`, and `testRunnerCommand` keys are required for this suite; the timeouts, retries, and caching are optional tuning. - -```yaml title="hyperexecute.yaml" ---- -version: 0.1 - -# Run all tests on Ubuntu Linux VMs (required for s3270 install) -runson: linux -# Automatically split discovered test files across parallel machines -autosplit: true -# Number of parallel VMs to spin up -concurrency: 5 -# Retry each failed test suite once before marking it as failed -retryOnFailure: true -maxRetries: 1 - -# Record a video for each test scenario on HyperExecute -captureScreenRecordingForScenarios: true - -# Timeout values in minutes -globalTimeout: 30 -testSuiteTimeout: 30 -testSuiteStep: 10 - -# Environment variables available to both mock servers and Robot Framework suites -env: -FNB_MOCK_PORT: "3271" # ASCII mock server port -MAINFRAME_HOST: "127.0.0.1" -MAINFRAME_PORT: "3270" # TN3270 mock server port (headless s3270 track) -MAINFRAME_GUI_PORT: "3272" # TN3270 mock server port (GUI x3270 track) -MAINFRAME_USER: "MOCKUSER" -MAINFRAME_PASS: "MOCKPASS" -DEBIAN_FRONTEND: "noninteractive" # Prevents apt-get from prompting during install - -# Cache pip packages across runs using requirements.txt as the cache key -cacheKey: '{{ checksum "requirements.txt" }}' -cacheDirectories: -- pip_cache - -# Start both mock servers in the background before tests run. -# HyperExecute manages these processes for the lifetime of the job. -background: -- nohup python3 mock_server/fnb_mock_server.py # ASCII mock on port 3271 -- nohup python3 mock_server/fnb_mock_tn3270.py # TN3270 mock on port 3270 (headless s3270) -- nohup python3 mock_server/fnb_mock_tnX3270.py # TN3270 mock on port 3272 (GUI x3270) - -# Steps run once on each VM before test execution begins -pre: -- sudo apt-get update -y -- sudo apt-get install -y s3270 # Headless 3270 emulator needed by the headless TN3270 suites -# GUI 3270 emulator, plus the X core-font packages it needs to render. -- sudo apt-get install -y x3270 x11-utils x11-xserver-utils xfonts-base xfonts-100dpi xfonts-75dpi fonts-dejavu-core -- pip3 install -r requirements.txt --cache-dir pip_cache - -# Dynamically discover all Robot Framework test files under tests/ -testDiscovery: -type: raw -mode: remote -command: find tests -name "*.robot" | sort - -# Register the X core-font dirs onto the recorded display so x3270 can render, -# wait for the mock servers to bind, then run each discovered suite. -testRunnerCommand: > -for d in /usr/share/fonts/X11/misc /usr/share/fonts/X11/100dpi /usr/share/fonts/X11/75dpi /usr/share/fonts/X11/Type1; -do [ -d "$d" ] && xset +fp "$d" 2>/dev/null || true; done; -xset fp rehash 2>/dev/null || true; -sleep 3 && robot --outputdir results --output output_$(basename "$test" .robot).xml --log log_$(basename "$test" .robot).html --report NONE $test - -# Determine pass/fail from the robot command exit code only -scenarioCommandStatusOnly: true - -# Merge artifacts from all parallel VMs into a single results folder -mergeArtifacts: true -uploadArtefacts: -- name: FNB_Reports -path: -- results/** - -# Render a combined Robot Framework report on the HyperExecute dashboard -report: true -partialReports: -type: json -location: results -frameworkName: robot - -jobLabel: [fnb-mainframe-mock, robotframework, linux, autosplit, x3270-gui] -``` - -Two lines are easy to get wrong: - -- **`captureScreenRecordingForScenarios`** is not only for the video. It provides the display `x3270` renders to. Do not add your own `DISPLAY` or `Xvfb`, which would send `x3270` to an unrecorded display and blank the video. -- **`sleep 3`** in `testRunnerCommand` gives the background mocks time to bind their ports before tests connect. Raise it if your mocks start slowly. - -## Step 5: Run the Suite - -Run the CLI from the repo root, pointing at the YAML. - -```powershell -.\hyperexecute.exe --config hyperexecute.yaml -``` - -The `.\` prefix is required in PowerShell. If you did not set the environment variables, pass the credentials inline: - -```powershell -.\hyperexecute.exe --config hyperexecute.yaml --user "" --key "" -``` - -```bash -./hyperexecute --config hyperexecute.yaml -``` - -If you did not set the environment variables, pass the credentials inline: - -```bash -./hyperexecute --config hyperexecute.yaml --user "" --key "" -``` - -That single command uploads the repo (respecting `.gitignore`), provisions the VMs, runs the `pre` steps, starts the background mocks, autosplits the `.robot` suites, records a video per scenario, and merges the Robot Framework report. The terminal prints live progress and ends with a job link to the dashboard. The exit code is `0` when all suites pass and non-zero otherwise, so it drops straight into CI. - -### What Runs Where - -| Suites | Mock host | Robot library | Emulator | Needs display | -| --- | --- | --- | --- | --- | -| `tests/01`-`05` (ASCII) | `fnb_mock_server.py` on 3271 | `libs/fnb_client.py` (raw socket) | none | no | -| `tests/06`-`09` (TN3270 headless) | `fnb_mock_tn3270.py` on 3270 | robotframework-mainframe3270 | `s3270` | no | -| `tests/10` (TN3270 GUI) | `fnb_mock_tnX3270.py` on 3272 | robotframework-mainframe3270 | `x3270` | yes, from `captureScreenRecordingForScenarios` | - -## Step 6: Read the Results - -Open the job link printed at the end of the run, or go to the [HyperExecute Dashboard](https://hyperexecute.lambdatest.com/hyperexecute). - -- **Job summary** shows pass and fail counts, per-VM timing, and the autosplit distribution. -- **Reports tab** holds the merged Robot Framework `report.html` and `log.html`. -- **Artifacts** lets you download the `FNB_Reports` archive with everything under `results/**`. See [HyperExecute Job Artifacts](/support/docs/hyperexecute-artifacts/) and [HyperExecute Job Reports](/support/docs/hyperexecute-reports/). -- **Scenario video** plays the per-scenario recording. For `tests/10` this shows the real `x3270` green screen being driven. - -Locally, the CLI also writes `hyperexecute-cli.log` (the full job log, the first place to look on a failure) and a `.hyperexecute/` run directory. - -## Common Tweaks - -Edit `hyperexecute.yaml` to change how the job runs. - -| Goal | Change | -| --- | --- | -| More or fewer parallel VMs | Set `concurrency` to the number you want, up to your plan's parallel limit. | -| Disable video for a faster run | Set `captureScreenRecordingForScenarios: false`. This removes the display `tests/10` needs, so exclude that suite if you do this. | -| Retry flaky suites | Raise `maxRetries` above `1`. | -| Allow longer-running suites | Raise `globalTimeout` and `testSuiteTimeout` (minutes). | -| Change mock ports or credentials | Edit the `env` block (`MAINFRAME_PORT`, `MAINFRAME_GUI_PORT`, `FNB_MOCK_PORT`, `MAINFRAME_USER`, `MAINFRAME_PASS`). | - -To run a single suite without editing the whole flow, narrow the discovery command: - -```yaml title="hyperexecute.yaml" -testDiscovery: -type: raw -mode: remote -command: find tests -name "06_tn3270_logon.robot" -``` - -All suites run in parallel on HyperExecute's Linux VMs, and the merged Robot Framework report lands on the dashboard under `FNB_Reports`. - -> For any query, contact us via **24x7 chat support** or email **support@testmuai.com**. diff --git a/static/docs/manage-ai-capabilities.md b/static/docs/manage-ai-capabilities.md deleted file mode 100644 index 31d6347d2..000000000 --- a/static/docs/manage-ai-capabilities.md +++ /dev/null @@ -1,62 +0,0 @@ -# Manage AI Capabilities - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI provides AI-powered capabilities across multiple products, including Test Manager, HyperExecute, SmartUI, Analytics, Web Automation, App Automation, Accessibility, and KaneAI. The **AI Capabilities** setting allows Org Admins to enable or disable all AI features for their entire organization with a single toggle. - -This is useful for enterprise customers who need to comply with client or corporate policies that prohibit the use of AI in testing workflows. - -Only **Org Admins** can access and modify AI Capabilities settings. This option is not available to other user roles. - -## Accessing AI Capabilities Settings - -**Step 1:** Log in to your TestMu AI account and navigate to **Settings** from the left sidebar. - -**Step 2:** Click on **Organization Settings**. - -**Step 3:** Navigate to the **Org Product Preferences** tab. - -**Step 4:** Select **AI Capabilities** from the left sidebar. - -You will see the **Toggle AI Features** option, which controls AI features for your entire organization. By default, this toggle is set to **ON** (enabled). - -## Disabling AI Features - -**Step 1:** On the **AI Capabilities** page, click the **Toggle AI Features** switch to turn it **OFF**. - -**Step 2:** Click **Update Settings** in the top right corner. A confirmation dialog will appear. - -**Step 3:** Click **Update Settings** to confirm, or **Cancel** to keep AI features enabled. - -Once disabled, a success message confirms that AI features have been turned off for your organization. - -## Enabling AI Features - -**Step 1:** On the **AI Capabilities** page, click the **Toggle AI Features** switch to turn it **ON**. - -**Step 2:** A confirmation dialog will appear asking you to confirm the change. - -**Step 3:** Click **Update Settings** to confirm. All AI features will be restored across all products for all users. - -## AI Features Controlled by the Toggle - -When the toggle is set to **OFF**, the following AI features are hidden across all TestMu AI products: - -| Product | AI Features | -|---------|------------| -| **Test Manager** | AI Scenario Generation, AI Test Case Creation, AI Test Step Generation, AI Autofill for Datasets, AI RCA in Test Runs, KaneAI Automation Features | -| **HyperExecute** | AI RCA Generation | -| **SmartUI** | AI Screenshot Comparison (Visual AI) | -| **Analytics** | Insights Copilot | -| **Web Automation** | AI RCA Generation | -| **App Automation** | AI RCA Generation, Smart Heal | -| **Accessibility** | AI-driven Rules | -| **KaneAI** | Entire Product | - -## Important Notes - -- **Existing AI-generated content is preserved.** When AI is disabled, all previously AI-generated content (test cases, test steps, RCA reports, autofill values, and scenario tags) remains visible and fully editable. Only new AI invocations are blocked. -- **Only Org Admins** can modify this setting. Non-admin users cannot access the Organization Settings page. -- **The setting applies organization-wide.** All users in the organization are affected when the toggle is changed. - -> In case you have any questions, please feel free to reach out to us via the **24/7 chat support** or email us over [support@testmuai.com](mailto:support@testmuai.com). diff --git a/static/docs/manage-subscriptions.md b/static/docs/manage-subscriptions.md deleted file mode 100644 index 6a69ebcca..000000000 --- a/static/docs/manage-subscriptions.md +++ /dev/null @@ -1,84 +0,0 @@ -# Manage Subscriptions - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -It's easy to manage your subscriptions by upgrading, removing, and switching the current plan. For example - upgrading from a free plan to a premium plan, removing an existing plan or switch from a monthly plan to an annual plan. - -## Upgrading From Free Plan - -1. Login to your TestMu AI account. - -2. Click **Upgrade** on the top right corner of the dashboard. - -3. From the left-flexbox, choose the plan as per your choice and requirement. Then click **GET STARTED**. - -4. It will redirect you to the checkout page. Verify the details and then select **PROCEED TO CHECKOUT**. - ->*In case you wish add more products, click **Add More Products**. Each product has a dropdown below QTY category through which you can customize your parallel test plans.* - -5. Enter the payment details, billing address, and info. Click **MAKE PAYMENT**. - -After few moments, a confirmation screen will appear stating the final status of your opted plan. - -## Customizing Plans - -You can customize your plans by adding new plans or remove existing ones. Choose any of them as per your choice. - -### Adding A New Plan - -1. Login to your TestMu AI account. - -2. Click **Upgrade** on the top right corner of the dashboard. - -3. Select **View Subscription**. - -4. Click **Go to Subscriptions & Billings** . - -5. Under the Subscriptions section, click **Edit**. - -6. It will route you to the checkout page. From the **Select Product** category, expand the dropdown and choose a new plan. - -7. After selecting the plan, verify the details and click **PROCEED TO CHECKOUT**. - -8. The price summary of the selected plan will show up. Click **MAKE PAYMENT**. - -After few moments, a confirmation screen will appear stating the final status of your additional plan. - -### Removing An Existing Plan - -Follow the steps 1 to 5 mentioned for *Adding New Plan*. - -6. Once you are in checkout page, Click 'x' on the plan that you wish to remove. - -7. After removing the plan, verify the details and click **PROCEED TO CHECKOUT**. - -8. The price summary of the removed plan will show up. Click **CONFIRM**. - -After few moments, a confirmation screen will appear stating the final status of your removed plan. - -### Switching From Monthly To Annual Plan - -If you are using a Monthly plan, you can always switch to Annual Plan and save up to 20%. - -1. Login to your TestMu AI account. - -2. Click **Upgrade** on the top right corner of the dashboard. - -3. Now enable the toggle button from Monthly to Yearly at the top-right. - -## Canceling The Current Plan Subscription - -**Step 1**: Login to your TestMu AI account. - -**Step 2**: Navigate to the profile avatar at the top-right corner. Click on **‘Billing’**. - -**Step 3**: Scroll down and click on **‘CANCEL SUBSCRIPTION’**. - -**Step 4**: Select the reason for canceling subscription from the given options. You can also mention the reason for canceling the subscription in the the description box. After that select **‘Complete Cancellation’**. - -You’ll be redirected to a page where you’ll notice a message as highlighted below. - -In case you wish to resume your current plan, you do so by clicking on **‘Resume your subscription’**. - -> -That's it! You can now successfully upgrade to a new plan, switch between plans, and switch from Monthly to Annual plans. If you still have any doubt, please feel free to reach out to us via **24*7 Chat Portal** or by mailing to us on [**support@testmuai.com**](mailto:support@testmuai.com). Happy testing! diff --git a/static/docs/mantis-integration.md b/static/docs/mantis-integration.md deleted file mode 100644 index e74a080fe..000000000 --- a/static/docs/mantis-integration.md +++ /dev/null @@ -1,75 +0,0 @@ -# Mantis Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Mantis is a free and an open source bug tracker. Mantis supports multiple OS such as Windows, Linux, and macOS, from the server side. Along with platform compatibility, It also offers cross browser compatibility on Google Chrome, Mozilla Firefox, Opera, Safari and IE10+. Mantis is highly customizable, you can customize your issue fields, workflows and notifications. You can also declare access as per role based in your organization. All these features along with email notifications to keep you updated on everything regarding the project makes Mantis a great choice as a project management tool. - -TestMu AI integration with Mantis helps you in filing issues to your project in Mantis directly from TestMu AI platform. With one-click integration you can push annotated issues to the project of your choice, assign it to the required team mate, and attach necessary screenshots. You can do all that while in the middle of a test session in TestMu AI platform. The fields populated by you when marking as bug through TestMu AI are displayed as information on the Mantis for a testing instance. - -> Mantis Integration with TestMu AI, like all of the integrations to 3rd party applications, is available for freemium as well as premium plan. - -## How To Establish Integration With Mantis From Your TestMu AI Account? - -**Step 1:** Login to your TestMu AI account. You would need to have Admin or User level access to see and install integrations. - -**Step 2:** Select ‘Integrations’ from the left navigation menu bar. This will guide you to a screen where you will find a list of 3rd party applications, available to integrate with your TestMu AI account. - -**Step 3:** Click on ‘ADD’ under the block that says ‘Mantis’. - -**Step 4:** Once you click on install, you will be redirected to the below screen. You will be asked to provide your Mantis API Token & Mantis Site Address in the highlighted fields to establish TestMu AI Integration with Mantis. - -> **What is an API token?** -When two different applications interact with each other, a server-to-server communication is established. API token is needed for authenticating user identity from one server to another, by fetching user-owned resources available on the server from where the communication is initiated. API token has been a necessity for maintaining security and data integrity of any application. They are just as vital to an end user as they are to an application. API tokens are relevant if an end user logs in an application through a 2-step verification process. - -> **What is Mantis Site Address?** -Your "Mantis Site Address" would be the URL for the instance of your project on Mantishub. The part highlighted with blue in the below image will be your Mantis Site Address. - -**Step 5:** To find the API token, login to your MantisHub account. Go to the ‘administrator’ drop-down on the top-right corner and click on "My Account". - -**Step 6:** As you click on "My Account", you will find a tab for "API Tokens". Enter a value in the provided field. It could be any value you like and then click on the button that says "Create API Token". - -**Step 7:** As you click on "Create API Token", an API token would be generated. Keep the API token stored with you safely as you won’t be able to see it again. - -> Note: These API tokens should be treated as safely as you treat your own passwords, as they are responsible for successful authentication between one application to another. However, in case your API token gets misplaced or is shared with someone whom you don’t trust, then you can always revoke access through that API token through your MantisHub instance. You can also generate multiple API tokens. - -Paste this API token into the field provided at TestMu AI & hit the ‘Next’ button. You are all set to experience one-click bug logging to share your issues directly from your TestMu AI account to your project on Mantis. - -That’s it! Go to Integrations again and you will be able to notice a green tick indicating that Mantis is successfully installed. - -## How To Log Your First Bug Through Mantis Integration? - -**Step 1:** Go for any of the test from the left navigation menu. For demo, we will be taking "Real Time Test" option. - -**Step 2:** Present a URL of the web-app you need to test in the dialog box. After that, select any configuration for browser and operating system of your choice & hit ‘Start‘. - -**Step 3:** After the VM is launched and operable. You can find bugs on you website by performing live, interactive testing. If a bug gets revealed, then you need to click on the bug icon from the left panel for capturing a screenshot of the same. We have highlighted that controller button in the below image. - -**Step 4:** After a screenshot is captured, you can annotate any issue or a task with an in-built image editor. Once you are done highlighting the bug, click on the button that says "Mark as Bug". - -**Step 5:** As you click on "Mark as Bug" button, a Mantis specific form would open up. Fill the fields as per your requirement. - -* You can select which project should the ticket go under. -* You can set the category for your UI observation. -* You can also select a value to signify the reproducibility. -* You can set the severity as well as the priority of the bug. -* You can mention the summary as well for better visibility on dashboard. -* You also get to post a description to help relate the cause of the issue or the task. -* If you have got a lot to convey then you can do so by populating the "Additional Information". - -**Step 6:** Click on "Create Bug" & observe it being successfully marked through a single click effort. Once you hit the button, you will get prompt messages on top of your Virtual Machine indicating the progress of bug logging. You will get a prompt message in few seconds mentioning "Bug successfully marked" indicating that the issue has been successfully pushed to the respective workspace. - -**Step 7:** Visit your Mantis dashboard view. You will be able to notice the logged issue right away! - -**Step 8:** As you click on the task you will notice all the data you provided in TestMu AI already presented in the task under task details. TestMu AI automatically includes test environment details and related screenshots in the ticket as attachments. - -## How To Uninstall Mantis Integration? - -> You can work with one integration at a time. So if you would want to integrate to a similar 3rd party application, then you would have to `uninstall` your current integration. Here is how you can do that. - -**Step 1:** Login to your TestMu AI account. - -**Step 2:** Select ‘Integrations’ from the left navigation menu bar. This will guide you to a screen where you will find 3rd party applications, available to integrate with your TestMu AI account. - -**Step 3:** Click on ‘REMOVE’. You can find the remove button right under the ‘Mantis’ block which would be highlighted with a green tick. - -> That was all you need to know for TestMu AI + Mantis Integration. Increase your productivity with our integrations. If you still have any questions for us, please feel free to let us know. Our experts are always available on **chat** to help you out with any roadblock regarding our product. Happy testing! diff --git a/static/docs/manual-app-profiling.md b/static/docs/manual-app-profiling.md deleted file mode 100644 index 07d54744d..000000000 --- a/static/docs/manual-app-profiling.md +++ /dev/null @@ -1,166 +0,0 @@ -# Manual App Profiling - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -App performance is a critical factor for user satisfaction and adoption. While automated profiling through [Appium tests](/support/docs/appium-app-performance-analytics/) provides insights during CI/CD pipelines, there are times when you need to manually interact with your app and observe its performance in real-time. TestMu AI's Manual App Profiling feature enables you to profile your app's performance during live manual testing sessions on real devices, giving you instant visibility into CPU, memory, disk, network, and more as you interact with the app. - -You can enable App Profiling from the App Live session toolbar before starting your session. Simply toggle the **Advance App Profiling** option, and profiling will begin capturing metrics as soon as your session starts. - -> This feature is available for **App Live** sessions on **iOS and Android** real devices. It must be enabled **before** starting the session and cannot be toggled mid-session. - -**Beta Feature** -Manual App Profiling is currently in **Beta**. To use this feature, get it enabled for your organization by reaching out to our **24x7 Chat Support** or mailing us at [support@testmuai.com](https://support.lambdatest.com/). - -## Getting Started - -**Step 1:** Navigate to **Real Devices > App Testing** and select your app, device, and OS version. - -**Step 2:** Before launching the session, enable the **Advance App Profiling** toggle in the session configuration. - -**Step 3:** Start your session, then start profiling from the toolbar. The profiling toolbar appears with options to monitor performance, capture network logs, and capture device logs. - -**Step 4:** During your session, you can: -- **Start/Stop Profiling**: Create multiple profiling segments within a single session. -- **Restart App Performance**: Reset profiling data and start a fresh capture from that point. -- **View Live Metrics**: CPU, Memory, Disk, FPS, Network, and Battery (iOS only) graphs update in real-time. - -**Step 5:** When done, click **Stop Profiling** to see a summary of captured metrics. Use **View Detailed Report** to navigate to the dashboard for in-depth analysis. - -**Step 6:** End the session through the **Session Details** modal (set the test name, status, remark, and tags). Once the session is created, it appears in the **[Manual Tests Sessions Dashboard](/support/docs/manual-tests-dashboard/)** under the **Real Devices** product, where you and your team can locate it using the dashboard filters. See [Ending a Session](/support/docs/manual-tests-dashboard/#ending-a-session) for details. - -## Basic Summary - -Here is a description of each metric from the App Profiling summary: - -- **Cold Startup Time :** Time taken by the app to launch from a complete shutdown or a non-running state. A lower cold startup time improves user experience, as it reduces wait time when the app is opened for the first time. -- **Hot Startup Time :** Time taken by the app to resume from a background state. A fast hot startup ensures that users can quickly switch between apps without delays. -- **Maximum CPU Utilisation :** It indicates the peak percentage of CPU consumed by the app during its execution. High CPU utilization can affect other processes and contribute to battery drain, so it's important to keep it optimized. -- **Average CPU :** Average CPU utilization gives a more balanced view of the app's typical CPU usage during a session. This value helps assess the app's efficiency in resource usage over time. -- **Maximum Memory Usage :** It refers to the highest amount of RAM consumed by the app during the session. Monitoring this metric helps developers ensure that their app doesn't consume excessive memory, which can lead to slower performance or crashes on low-memory devices. -- **Average Memory Usage :** Average memory usage shows the typical amount of memory used by the app. This value is useful for identifying memory leaks and optimizing the overall resource consumption of the application. -- **Maximum Disk Usage :** Highest amount of storage consumed by the app on the device. Keeping disk usage minimal helps improve app performance and ensures better storage management for end users. -- **Average Disk Usage :** It reflects the typical amount of storage space used by the app over time. This metric is important for identifying opportunities to reduce the app's footprint on the device's storage. -- **Average Frame Rate :** The frame rate measures the smoothness of the app's graphical rendering. A higher frame rate (measured in frames per second or fps) indicates smoother animations and a better visual experience. A rate close to 60 fps or higher is typically desired for smooth app performance. -- **Network Download/Upload :** This metric tracks the amount of data downloaded and uploaded by the app during the session. Monitoring network usage helps developers optimize how the app handles data transmission to ensure it uses bandwidth efficiently. -- **Application Not Responding :** ANR events occur when the app becomes unresponsive and prompts the user to wait or close the app. A value of zero indicates the app has not faced any ANR issues, which is crucial for maintaining a stable and responsive app. You can also view ANR Dumps and Data logs. -- **App Crashes :** This metric tracks the number of times the app has crashed during the session. Zero crashes indicate that the app is stable, ensuring a reliable user experience. -- **Battery Energy (iOS only) :** Energy consumed by the app, measured in Joules. This metric helps identify features that are energy hogs and optimize them to extend battery life. - -## Key Performance Metrics - -### 1. CPU Consumption -The CPU handles all operations within a mobile device. High CPU usage impacts overall system performance and drains the battery faster. By monitoring CPU consumption during manual testing, you can pinpoint which user interactions are resource-heavy and optimize them for better performance. TestMu AI provides two CPU metrics in a time-series format: - -- **System CPU Usage (%):** Total CPU usage across the system. -- **App CPU Usage (%):** CPU usage specific to your app. - -Using these real-time insights, you can identify CPU-intensive interactions as you manually navigate through your app and minimize CPU-heavy operations. - -### 2. Memory Consumption -RAM is a critical resource for mobile apps. Excessive memory consumption can lead to memory leaks, degraded performance, and application crashes. - -TestMu AI offers three memory metrics: -- **System Memory Usage (MB):** Overall memory consumption during the session. -- **App Memory Usage (MB):** Memory consumed by your application. -- **Available Memory (MB):** Remaining memory available for use. - -By monitoring memory during manual testing, you can identify specific user flows that trigger memory spikes or leaks. - -### 3. Disk Usage -The Disk Usage metric provides insights into how much disk space is being utilized by both the system and the application during a session. Monitoring disk usage is crucial for optimizing storage consumption and ensuring efficient app performance, especially on devices with limited storage capacity. - -- **System Disk (MB):** Indicates the total amount of disk space used by the entire system, including the operating system and other background processes, displayed in megabytes (MB). -- **App Disk (MB):** Shows the disk space specifically consumed by your application (App Under Test) during the session, helping you track how much space your app is utilizing and whether it might cause storage-related issues. - -### 4. Rendering (Frame Rate) -Smooth rendering directly affects how responsive your app feels to users. Rendering performance is a typical performance issue that any app faces. It is measured at different stages in the app's lifecycle to ensure that users do not have a bad experience with the app. - -TestMu AI measures your app's Frames Per Second (FPS) across its lifecycle, which helps you identify performance bottlenecks in rendering and improve user experience. - -- **Frames Per Second (FPS):** Real-time measurement of rendering performance throughout the session. - -Low FPS during specific interactions helps pinpoint UI rendering bottlenecks. - -### 5. Network Usage -Understanding how your app utilizes network resources is essential for optimizing data transfer and ensuring efficient performance. - -TestMu AI tracks: -- **Download Size (MB):** Amount of data downloaded during the session. -- **Upload Size (MB):** Amount of data uploaded during the session. - -This helps you to monitor: - -- **Data Efficiency:** Optimize data usage to enhance performance and reduce costs. -- **Performance Optimization:** Identify and mitigate high network usage areas that may affect app responsiveness. - -When network log capture is enabled, a detailed HAR file is also available on the dashboard containing all HTTP/HTTPS requests made by the app. - -### 6. Battery Consumption -Apps that consume excessive battery negatively affect user experience. With TestMu AI, you can monitor how your app impacts battery drain, identify features that are energy hogs, and optimize them. This will help extend battery life for devices running your app. - -- **Battery Energy (J):** Energy consumed by the app, measured in Joules. - -Battery consumption data is only displayed for **iOS** applications. This feature is not available for Android applications during manual profiling. - -### 7. ANR (Application Not Responding) - -Sometimes, your application stops responding, and you receive a pop-up to wait or close the app. Application freezes and crashes are frustrating for users. TestMu AI provides detailed logs on ANR events, that helps you diagnose the root causes and eliminate them. - -> Currently, ANR monitoring is available for **Android devices** only. - -Metrics Provided: - -- **ANR Count:** Number of times the app encountered ANR during the session. -- **ANR Dumps:** Detailed logs for each ANR event to aid in troubleshooting. - -### 8. Application Start-up Time -Application start-up time significantly impacts the first impression and overall user experience. Slow startup times create a poor first impression. TestMu AI tracks both cold and hot startup times, giving you the data needed to reduce delays and deliver a faster app launch. - -- **Cold Startup Time:** Time from app launch (from scratch) to the first frame rendered. -- **Hot Startup Time:** Time from app resume (from background) to interactive state. - -## Profiling Segments - -Manual App Profiling uses a **parent-child hierarchy** for organizing your profiling data: - -- **Parent Session:** Your App Live manual testing session. -- **Profiling Segments (Children):** Each time you start and stop profiling within a session, a new profiling segment is created. - -You can create **multiple profiling segments** within a single session to isolate performance data for different user flows or app features. Each segment has its own: -- Performance summary and graphs -- Video recording -- Network logs (if enabled) -- Device logs (if enabled) - -## Capturing Network and Device Logs - -When starting profiling, you can optionally enable: - -- **Network Logs:** Captures all HTTP/HTTPS traffic in HAR format. Available on the dashboard under the **Network Logs** tab. -- **Device Logs:** Captures system-level device logs. Available on the dashboard under the **Device Logs** tab. - -> These tabs only appear on the dashboard if they were enabled at the time of starting profiling. If not enabled, only the **App Performance** and **Meta Data** tabs will be visible. - -## Viewing Profiling Results - -After profiling is complete and you end the session, the session is created and stored on the **[Manual Tests Sessions Dashboard](/support/docs/manual-tests-dashboard/)** under the **Real Devices** product. Navigate there to: - -- Locate your session using the dashboard filters: **Date**, **OS**, **Status**, **Session Type**, **Users**, and **Tags** -- View profiling summary cards with all captured metrics -- Explore time-series performance graphs synchronized with session video -- Analyze network logs (HAR viewer) and device logs -- Compare profiling segments across different user flows - -For more details on navigating the dashboard, see the [Manual Tests Sessions Dashboard](/support/docs/manual-tests-dashboard/) documentation. - -## Platform Availability - -| Capability | Android | iOS | -|------------|---------|-----| -| Core metrics (CPU, Memory, Disk, FPS, Network, Startup Time) | Android 9+ | All versions | -| Network Logs (HAR) and Device Logs | Android 9+ | All versions | -| Battery Energy | Not supported | All versions | -| ANR Monitoring | Android 9+ | Not supported | - -> Got any questions? -> Please reach out at our **24x7 Chat Support** or you could also mail us at [support@testmuai.com](https://support.lambdatest.com/). diff --git a/static/docs/manual-test-case-creation.md b/static/docs/manual-test-case-creation.md deleted file mode 100644 index 097d441ab..000000000 --- a/static/docs/manual-test-case-creation.md +++ /dev/null @@ -1,76 +0,0 @@ -# Create & Manage Test Cases - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Test Manager streamlines the process of creating, managing, and editing test cases. It enables users to develop manual test cases, allowing efficient management of all testing activities from a single platform. - -## Creating a Test Case - -After selecting your project, type your test case **title** to create a new test case. You can create a **folders** to organize and store the test cases. - -To add a new test case, click on the **Add Test Case** button on the right side or use `Command + \` for Mac and `Control + \` for Windows shortcut to create a new test case. - -Press **shift + Enter** to create multiple test cases simultaneously. - -After test case is created you will be redirected to a screen listing all the test cases. - - By default test cases having no parent folder are stored under **Untitled** folder. - -## Manage Test Case - -### Test Case Details - -When you select a test case, you’ll be redirected to the Test Case Details page, where you can view Test Case details such as: **Test Case Title**, **Description**, **Precondition** , **Attachments**, **System Fields**: **Type**, **Status**, **Priority**, **Tag(s)** & **Custom Fields** - -### Test Case Steps - -Test Steps can be accessed from the `Test Steps` section, present in the Summary Page of a Test Case. - -The Test Manager offers a comprehensive Steps section with a rich text editor and these features: - -**1. Add Step level Attachments** - -**2. Add Steps and Modules in between Steps** - -**3. Duplicate a Step:** Click the three-dot menu on any created step and select `Duplicate Step` to create a copy of that step immediately below it. This option is available for individual steps only and is not available for Modules. - -**4. Importing or Creating a Module:** - -- Select Test Steps & click on the `Create Module` option to make a sharable module out of them. - -- Import existing Modules by clicking on the Modules button. - -To know more about how to use Modules refer the [Modules doc](/support/docs/create-modules/). - -#### Add New Step: - -To add a new step click on on the `Add Test Step` button. - -You can use the **Generate with AI** option to ask agent to analyse your test case details & generate the next step of action for you. - -#### Save Changes: - -To save the changes made in the Test Case click on the `Save Changes` button OR use the `Command + /` for Mac & `Ctrl + /` shortcut. - - Any change in the Test Case, on **saving** will create a new version for that Test Case. You can add a commit message to note the reason for changes. To know more about how versioning works checkout the [Versioning doc](/support/docs/test-case-versioning/). - -### Test Case Runs Section - -**Test Case Runs Section** provides an overview of a test case's executions & plannings, whether are manual or automated. It includes detailed information of Test Instances of the Test case such as **applied configuration**, **executed by**, **Test Run name**, **Test Instance Execution Status**, **Execution time** and filters based on available fields. - -#### Executed Test Instances of Test Case - -- This section displays test instances that have been executed. You can filter the test instances based on the **executor** and the **current status**. - -#### Planned Test Instances of Test Case -- This section shows Planned Instances that are included in a Test Run but their execution is not yet started. You can filter Test Instances based on the **Assignee** of the Test Instance. - -- On clicking on that Test Case's execution you will be redirected to `Automation Dashboard` if its a Automation execution, the `Test Instance Page` if its Manual Instance or `Test Runs Page` if its a unexecuted Automation Instance. - -### Issues section - -This section lists the linked Jira/ADO Issues with the Test Case. For more information refer [this page](/support/docs/link-jira-issues-with-test-manager/) - -### Version History - -This section allows you to manage the Version History of the test caase by comparing Versions, Viewing or Reverting to them. For more information refer [this page](/support/docs/test-case-versioning). diff --git a/static/docs/manual-test-cases-with-ai.md b/static/docs/manual-test-cases-with-ai.md deleted file mode 100644 index 9f63c1db3..000000000 --- a/static/docs/manual-test-cases-with-ai.md +++ /dev/null @@ -1,17 +0,0 @@ -# Manual Test Cases with AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Test Manager enhances the process of manual test case creation by integrating AI capabilities. This feature enables users to efficiently generate and manage test cases, utilizing AI to automatically create titles and test steps, enhancing productivity and accuracy. - -**Steps for Creating Manual Test Cases with AI** - -1. After selecting your project and clicking to create a new test case, press Tab to generate the title using AI, in order to create a new test case. - -2. The test case is then created, redirecting to a screen listing all the test cases created. - -3. To add a new test case, click on the **Add Test Case** button on the right side and press Tab to generate the title using AI. - -4. Add steps to your test cases by clicking **Generate** or **Press Tab** allowing AI to create the test step. - -5. Click on **Save Test Case** to create the test steps and update your AI generated test cases. diff --git a/static/docs/manual-tests-dashboard.md b/static/docs/manual-tests-dashboard.md deleted file mode 100644 index 25be5c547..000000000 --- a/static/docs/manual-tests-dashboard.md +++ /dev/null @@ -1,226 +0,0 @@ -# Manual Tests Sessions Dashboard - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The Manual Tests Sessions Dashboard provides a unified view of all your manual testing sessions across TestMu AI Real Devices products. It converts ephemeral manual sessions into structured historical records, helping you track, analyze, and report on your manual testing activity. - -All your **Real Devices App Testing** (App Live) and **Real Devices Browser Testing** (Browser Live) sessions are automatically stored on this dashboard. - -**New Dashboard** -The Manual Tests Sessions Dashboard is a newly launched feature. To get it enabled for your organization, please reach out to our **24x7 Chat Support** or mail us at [support@testmuai.com](https://support.lambdatest.com/). - -## Supported Session Types - -Currently, the dashboard supports manual testing sessions run on **Real Devices**. - -| Product | Platform | Session Type | -|---------|----------|-------------| -| Real Devices App Testing (App Live) | Android, iOS | App Testing | -| Real Devices Browser Testing (Browser Live) | Android, iOS | Browser Testing | - -> Support for **Virtual Devices** sessions is coming soon. - -## Accessing the Dashboard - -Navigate to **Real Devices > Sessions** from the sidebar to open the Manual Tests Sessions Dashboard. - -## Dashboard List View - -The dashboard displays all completed manual testing sessions in a list format. Each session row shows: - -- **Test Name**: Session name, set when ending the session or edited later via rename. -- **Test ID**: Unique session identifier (for example, `RMAL-AND-164519-...`). -- **Session Type**: App Testing or Browser Testing. -- **Platform & Device**: OS icon, device name, and OS version. -- **Duration**: Total session time. -- **Status**: Session outcome (Passed, Failed, or Completed). -- **Performance Tag**: Indicates the session has profiling data. Click the chevron to expand and view its profiling tests. -- **Tags**: Custom tags assigned to the session (up to 10 per session). -- **Remark**: User-added notes (up to 256 characters). - -### Filters - -You can filter sessions using the filter dropdown. Click the **Filter** button at the top of the dashboard to open the filter panel. - -| Filter | Options | Description | -|--------|---------|-------------| -| **Date** | Start and end date picker | Filter sessions within a specific time period | -| **OS** | iOS, Android | Filter by device operating system | -| **Status** | Passed, Failed, Completed | Filter by session outcome | -| **Session Type** | App Testing, Browser Testing | Filter by the type of manual test | -| **Users** | Team member names | Filter by the user who ran the session | -| **Tags** | Custom tag names | Filter by tags (AND logic, session must have **all** selected tags) | - -When filters are active, a **Clear All** bar appears at the top. Clicking it removes all active filters but does **not** clear search text. - -> Filter state is **URL-encoded**, so your filter selections are preserved in the URL, making filtered views shareable and bookmarkable. - -### Search - -Use the search bar to find sessions by **test name** or **test ID**. Search works alongside active filters, so results are filtered by both search text and selected filters at the same time. - -Click the **X** button in the search bar to clear the search and restore the full list. - -### Sorting - -Sessions can be sorted by: -- **Created At** (default): newest first -- **Duration**: longest or shortest first - -## Ending a Session - -When you stop a manual testing session, a **Session Details** modal appears. Use it to set session metadata before the session is saved to the dashboard. - -In the **Session Details** modal, you can: - -- **Test Name**: Rename the test to a descriptive name for easy identification. Click the **X** in the field to clear the current name. -- **Status**: Defaults to **Completed**. Use the dropdown to update it to **Passed** or **Failed** based on your test results. -- **Remark**: Add notes about the session (up to **256 characters**, with a live counter showing usage). -- **Tag(s)**: Add custom tags for later filtering. Enter multiple tags separated by a comma (up to **10 tags** per session). - -Once the details are set, click **Save & End Session** to save the session. Click **Cancel** to dismiss the modal without ending the session. - -> The test name, remark, and tags can only be set when ending the session through the Session Details modal. Remarks and tags cannot be edited after the session ends. The test name can be renamed later from the dashboard. - -After you click **Save & End Session**, the session is created and appears on the Manual Tests Sessions Dashboard, under the **Real Devices** product, with all the metadata you provided. You can then locate it on the dashboard using the search bar or filters. - -## Performance Sessions View - -Sessions with [Manual App Profiling](/support/docs/manual-app-profiling/) enabled use a **parent-child hierarchy** on the dashboard: - -- **Parent Session**: Your App Live manual testing session row (for example, `RMAL-AND-164519-...`). -- **Profiling Tests (Children)**: Each time you start and stop profiling within a session, a new child profiling test is created (for example, `PROF_ANDROID_...` or `PROF_IOS_...`). - -Click the **expand chevron** on a parent session to reveal the profiling tests as child rows. Each child has its own name, duration, and platform icon. Click a child profiling test to navigate directly to its **App Performance** view. - -## Session Detail Page - -Click on any session row to open its detail page. - -### Tabs - -The tabs available on the detail page depend on whether the session has profiling data: - -**Without profiling (2 tabs):** - -| Tab | Content | -|-----|---------| -| **Meta Data** | Session info, configuration, timing, user details | -| **Media** | Video recording and screenshots | - -**With profiling (up to 5 tabs):** - -| Tab | Content | -|-----|---------| -| **App Performance** | Profiling summary cards and time-series performance graphs synced with video | -| **Meta Data** | Session info, configuration, timing, user details | -| **Media** | Video recording and screenshots | -| **Network Logs** | HAR viewer with HTTP/HTTPS request details (only if network log capture was enabled) | -| **Device Logs** | System-level device logs (only if device log capture was enabled) | - -### Meta Data Tab - -The Meta Data tab displays session information in a two-column layout: - -- **Session Info**: Test ID, Status, Session Type -- **Configuration**: Device, Platform, OS Version -- **Timing**: Started At, Ended At, Duration, Updated At -- **User**: Username who ran the session -- **Remark**: Full remark text (if set) - -### Media Tab - -The Media tab shows all artifacts captured during the session: - -- **Video Recording**: Full session video playback -- **Screenshots**: Any screenshots captured during the session - -Toggle between **list view** and **grid view** for browsing artifacts. Each artifact has a **download** button. - -If no artifacts were captured, an empty state message is displayed. - -### App Performance Tab - -Available only for sessions with [App Profiling](/support/docs/manual-app-profiling/) enabled. This is where you view the detailed profiling results for your session. - -This tab displays: - -- **Profiling Summary Card**: All key metrics at a glance, including CPU (max and average), Memory (max and average), Disk (max and average), Average FPS, Network (download/upload), Battery Energy (iOS only), Cold Startup Time, Hot Startup Time, ANR Count (Android only), and App Crashes. -- **Time-Series Graphs**: Interactive performance graphs for each metric category (CPU, Memory, Disk, FPS, Network, Battery). -- **Video-Graph Sync**: Session video playback is synchronized with the performance graphs. As the video plays, a timeline marker moves across the graphs in real time, making it easy to correlate user interactions with performance changes. - -#### Navigating Between Tests - -A session and its profiling tests are viewed individually. To move between them, use the **Select Test** option in the toolbar. It opens a panel with: - -- A **Search test** bar to find a specific test by name. -- The **parent session** listed at the top. The currently open test is marked with a check. -- A **Performance Test** group listing the profiling tests captured during the session (for example, `Perf-app-debug(5)-...`). - -Click any test to switch to its view. You can also use the **previous and next arrows** in the toolbar (for example, "1 of 2") to move between tests one at a time. - -### Network Logs Tab - -Available when profiling was started with network log capture enabled. Displays a **HAR viewer** showing: - -- All HTTP/HTTPS requests made by the app -- Request method, URL, status code, and duration -- Request and response headers and body (expandable) -- Filter by URL, method, or status code - -### Device Logs Tab - -Available when profiling was started with device log capture enabled. Displays a **log viewer** with: - -- System-level device logs captured during the profiling test -- Searchable and filterable log entries -- **Download** button to export logs - -## Session Actions - -### Rename - -Rename a session from the **kebab menu** (three dots) on the list row or the detail page: - -**Step 1:** Click the kebab menu icon. - -**Step 2:** Select **Edit name**. - -**Step 3:** Enter the new name in the modal. - -**Step 4:** Click **Save**. - -The updated name is reflected immediately in both the list view and detail page, and persists after refresh. - -### Delete - -Delete a session from the **kebab menu**: - -**Step 1:** Click the kebab menu icon. - -**Step 2:** Select **Delete**. - -**Step 3:** Confirm in the warning dialog. - -> Deleted sessions are **soft-deleted**: they are removed from the dashboard list, but the data is retained in the backend. - -## Tags - -- Add up to **10 custom tags** per session. -- Set tags in the **Session Details** modal when ending a session, entering multiple tags separated by a comma. -- Tags are displayed as label chips on the session row. -- Filter by tags using the dashboard filter (AND logic, session must match all selected tags). -- Spaces are trimmed from tag names. - -> Tags can only be set when ending a session through the Session Details modal. They cannot be edited after the session ends. - -## Data Retention - -All manual testing sessions from Real Devices App Testing and Browser Testing are automatically stored on this dashboard. Session data is retained in line with your organization's data retention policy. - -## Related Documentation - -- [Manual App Profiling](/support/docs/manual-app-profiling/): Enable and use real-time performance profiling during App Live sessions - -> Got any questions? -> Please reach out at our **24x7 Chat Support** or you could also mail us at [support@testmuai.com](https://support.lambdatest.com/). diff --git a/static/docs/manual-to-automated-test-conversion.md b/static/docs/manual-to-automated-test-conversion.md deleted file mode 100644 index 68fa433de..000000000 --- a/static/docs/manual-to-automated-test-conversion.md +++ /dev/null @@ -1,140 +0,0 @@ -# AI Test Generation: Requirements to Code - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI provides a complete workflow to convert your manual test requirements into fully automated test scripts,from requirement input to executable code. This guide walks through the end-to-end flow using **Test Manager** and **KaneAI**. - -## Overview - -The conversion flow has four stages: - -1. **Set up your project** in Test Manager -2. **Provide requirements**,text, documents, images, audio, Jira/ADO tickets, or record user interactions -3. **Generate test scenarios**,AI creates structured test cases from your input -4. **Automate and export code**,convert generated test cases into automation scripts across frameworks - -## Step 1: Create a Project - -Start by creating a project in TestMu AI Test Manager. A project is the container for all your test cases, test runs, and configurations. - -1. Navigate to [Test Manager](https://www.testmuai.com/login/?redirectTo=https://test-manager.lambdatest.com). -2. Click **Create New Project**. -3. Enter the project name and description. -4. Save the project. - -For detailed steps, see [Create Projects](/support/docs/create-projects/). - -## Step 2: Generate Test Scenarios with AI - -Navigate to the **Test Case Listing** page within your project and click **Generate With AI**. - -You can also access this from KaneAI by clicking **Generate Scenarios** on the Agent page. - -### Supported Input Formats - -Provide your requirements in any of the following formats: - -| Input Type | How to Add | -|-----------|------------| -| **Text** | Type your requirement directly in the input box | -| **Documents** | Upload PDFs, spreadsheets (CSV/XLSX), JSON, XML | -| **Images** | Upload screenshots, wireframes, or UI mockups | -| **Audio** | Record audio or upload audio files describing the feature | -| **Videos** | Upload walkthrough or demo videos | -| **Jira Tickets** | Link Jira issues (epics, stories, tasks) via the pin icon | -| **Azure DevOps** | Link ADO work items via the pin icon | - -Before generating, configure **Max Scenarios**, **Max Test Cases per Scenario**, and toggle **Memory Enhancement** to leverage your existing test case repository and avoid duplicates. - -Press **Enter** to generate. The AI analyzes your input and produces structured test scenarios with test cases. - -Each scenario is tagged with priority (`Must have`, `Should have`, `Could have`) and each test case is categorized as **Positive**, **Negative**, or **Edge**. - -### Refine with Conversation Layer - -After generation, use the **Conversation Layer** to refine your test cases through natural language,add coverage, remove duplicates, reorganize scenarios, or update details without manual editing. - -For the full guide on conversational refinement, see [AI Test Case Generator](/support/docs/generate-test-cases-with-ai/#conversation-layer-refine-your-test-cases). - -# Telemetry- and trace-driven test generation - -TestMu AI can ingest production telemetry, application logs and runtime execution traces to generate and maintain tests. Real user interactions and session traces are converted into structured, editable test scenarios that reflect how the application is actually used, and telemetry signals feed Test Intelligence to prioritise coverage where failures are most likely. - -## Step 3: Save or Automate - -Once your test scenarios are ready, select the test cases you want to keep and choose one of two paths: - -### Option A: Create Test Cases - -Click **Create** to save the test cases to your **Test Case Repository** in Test Manager. These can be used for manual test planning, assigned to test runs, or shared with your team. - -### Option B: Create and Automate with KaneAI - -Click **Create and Automate** to save the test cases and automatically send them to **KaneAI** for automation. KaneAI executes each test case using natural language understanding and generates automation scripts. - -## Step 4: Generate Automation Code - -Once KaneAI has automated your test cases, you can generate code in multiple frameworks and languages. - -1. Open the automated test case in KaneAI. -2. Click **Generate New Code**. -3. Select your desired framework and language. - -### Supported Frameworks - -| Platform | Framework | Languages | -|----------|-----------|-----------| -| **Web** | Selenium | Python | -| **Web** | Playwright | JavaScript, Python | -| **Web** | Cypress | JavaScript | -| **Web** | WebdriverIO | JavaScript | -| **Mobile** | Appium | Python, Java, JavaScript | - -The generated code is fully accessible,download it, customize it, and integrate it into your CI/CD pipeline or execute it on TestMu AI [HyperExecute](/support/docs/getting-started-with-hyperexecute/). - -For the full code generation guide, see [KaneAI Code Generation](/support/docs/kane-ai-automation-code-generation/). - -## Alternative Path: Record User Interactions - -Instead of providing written requirements, you can use KaneAI's **Manual Interaction Mode** to record real user actions and convert them into automated tests. - -1. Start a new test in KaneAI. -2. Switch to **Manual Interaction** mode. -3. Perform actions on your application,clicks, form fills, navigation. -4. KaneAI captures each action and converts it into test steps. -5. Generate automation code from the recorded steps. - -This is useful when requirements are easier to demonstrate than describe, or when you want to capture existing user workflows as automated tests. - -For the full guide, see [KaneAI Manual Interaction Mode](/support/docs/kaneai-manual-interaction/). - -## End-to-End Flow Summary - -``` -Requirements (text, docs, images, audio, Jira, ADO) -│ -▼ -AI Test Case Generator (Test Manager / KaneAI) -│ -├── Create ──► Test Case Repository (manual testing) -│ -└── Create and Automate ──► KaneAI Automation -│ -▼ -Code Generation -(Selenium, Playwright, -Cypress, Appium, etc.) -│ -▼ -Download / CI/CD / HyperExecute -``` - -## Related Docs - -- [Create Projects](/support/docs/create-projects/): Set up your Test Manager project -- [AI Test Case Generator](/support/docs/generate-test-cases-with-ai/): Full guide for AI-powered test case generation -- [KaneAI Manual Interaction](/support/docs/kaneai-manual-interaction/): Record user actions to create tests -- [KaneAI Code Generation](/support/docs/kane-ai-automation-code-generation/): Export automation scripts in multiple frameworks -- [KaneAI Test Planner](/support/docs/kane-ai-test-plan/): Generate tests from high-level objectives -- [KaneAI Jira Integration](/support/docs/kane-ai-jira-integration/): Generate tests from Jira tickets -- [Getting Started with HyperExecute](/support/docs/getting-started-with-hyperexecute/): Execute automated tests at scale diff --git a/static/docs/mark-as-bug-in-automation-testing.md b/static/docs/mark-as-bug-in-automation-testing.md deleted file mode 100644 index 6e30b8e66..000000000 --- a/static/docs/mark-as-bug-in-automation-testing.md +++ /dev/null @@ -1,41 +0,0 @@ -# Mark as Bug in Automation Testing - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - ->You can skip the steps of capturing a screenshot, annotating it, and logging onto a separate tool to create a task. TestMu AI does it all for you. ->**Mark as Bug** is a feature that lets you highlight any UI observation from your test sessions on TestMu AI and share it with your colleagues on their favorite project management or bug tracking tools such as Jira, Trello, Hive, Paymo, and more. You can choose the assignee who handles your reported task or bug. You can also mention a description to help them understand the issue. You can do all of this while testing on TestMu AI, without hopping to any other platform or tool. - -**Note:** Mark as Bug is available across every type of testing on TestMu AI. Below are the various types of testing we offer: -* **Real Time Testing:** Test your web-app with live interaction by running it on VMs hosted by TestMu AI cloud server. -* **Screenshot Testing:** Automated bulk screenshot testing on 25 (browsers + OS) combinations in a single test session. -* **Responsive Testing:** Perform cross-browser testing across a variety of mobile and desktop devices to ensure your website renders well across different viewports. Run responsive testing on 46 devices in a single test session. -* **Automation Testing:** Perform automated cross-browser testing with our on-cloud Selenium Grid. Execute parallel testing to reduce your test cycles. Our Selenium Grid is compatible with every [test automation](https://www.testmuai.com/automation-testing/) framework that supports Selenium. -* **Smart UI Testing:** Automated visual regression testing that performs a pixel-to-pixel comparison between multiple images and highlights any UI deviations such as padding, icon, text, and more with respect to the baseline images. - -**Did You Know?** - -TestMu AI offers more than 2000 real browsers and browser versions on every type of testing so you can maximize test coverage of your cross-browser testing. - -## How to Mark as Bug in Automation Testing - -Log a bug directly from the automation test session to your integrated project management tool. - -1. Visit the **Automation** tab from the left navigation menu to go to the automation console. Go to **AUTOMATION LOGS**. - -2. In the automation logs, you can access various logs of your test script execution such as network logs, command logs, and more. By default, you are routed to **SUMMARY**. In summary, you will find a button to **Create Issue**. - -As you click on the **bug** icon, an integration-specific form opens up. Fill the fields as needed. For instance, if you have installed the TestMu AI + JIRA integration, you can fill the below fields: ->* Select which **project** should the bug go under. ->* Select the **assignee** for that particular task. ->* Select the **issue type** of the bug. ->* Post a **summary** to help relate the cause of the bug. - -All the fields you populate reflect directly in your respective project based on the integration you have configured with your TestMu AI account. In the above case, the changes reflect on the respective JIRA instance. - -Similarly, you can find the mark as bug button in **EXCEPTION** and **COMMAND** logs. -* Exception logs - -* Command Logs - -**Got any questions?** -Share them with our **24/7 chat support** or drop a mail at support@testmuai.com. diff --git a/static/docs/mark-as-bug-in-responsive-testing.md b/static/docs/mark-as-bug-in-responsive-testing.md deleted file mode 100644 index abebed8f3..000000000 --- a/static/docs/mark-as-bug-in-responsive-testing.md +++ /dev/null @@ -1,89 +0,0 @@ -# Mark As Bug In Responsive Testing - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -You can now avoid the hassle of capturing a screenshot, annotating the same before you finally share it with a teammate by logging onto a respective tool for creating a task. TestMu AI does it all for you. -**Mark as Bug** is a feature that would help you highlight any UI observation from your test sessions on TestMu AI. And it let you share the same with your colleagues on their favorite project management/ bug tracking tools such as Jira and Trello, Hive, Paymo, etc. You can choose the assignee who handles your reported task or bug. You can even mention the description to help them understand your point better. You can do all that while testing on TestMu AI, without having to hop on to any other platform or tool. - ->**Note:** Mark as Bug feature is not restricted to any particular feature at TestMu AI. In other words, every type of testing that you perform on TestMu AI will have a mark as a bug embedded in it. Below are various types of testing that we offer at TestMu AI. - -* **Real Time Testing:** To help you test your web-app with live interaction by running it on VMs hosted by TestMu AI cloud server. - -* **Screenshot Testing:** Automated bulk screenshot testing on 25(browsers + OS) combinations in a single test session. - -* **Responsive Testing:** Perform cross browser testing across various mobile & desktop devices to ensure your website renders well across different view-ports. Run [responsive testing](https://www.testmuai.com/responsive-test-online) on 45+ devices in a single test session. - -* **Automation Testing:** Perform automated cross browser testing with our on-cloud Selenium Grid. You can execute parallel testing to trim down your test cycles drastically. Our Selenium Grid support is compatible with every test automation framework that supports Selenium. - -* **Smart UI Testing:** Automated visual regression testing that performs a pixel-to-pixel comparison between multiple images & highlights any UI deviations such as padding, icon, text, etc., with respect to the baseline images. - -> -**Did You Know?** -TestMu AI offers more than 2000 real browsers and browser versions on every type of testing so you could maximize test coverage of your cross browser testing. - -## How Can I Mark As Bug While Responsive Testing? - -**Note:** Below is a demonstration of marking a bug from Responsive testing. - -**Step 1:** Go to Responsive testing from the left navigation menu, provide the testing URL, and hit the **Generate** button to start your responsive test session. - -**Step 2:** Once you hit the **Generate** button. You will find a loader on devices mentioned on the right side of the screen, an indication that the screenshot test has begun. A counter will represent the number of devices for which the screenshot has been successfully generated. You can **stop** the test anytime by using the Stop button. - -**Step 3:** Once your responsive test session is complete, you can perform a vertical-scroll to find your desired device for ensuring your RWD(Responsive Web Design). After selecting the device, double-click on the camera icon on top for marking a bug. - -**Step 4:** As you double-click on any screenshot, an image editor will pop up with various buttons to help you highlight and annotate your UI observation before sharing the screenshot with your colleagues. - - - You can highlight the bug through different shapes like square, rectangular, circular, or you can even underline. Environment details of the test will be presented at the bottom-left of the image editor. - - - You can also scribble the bug using the pencil icon & add text for annotating the screenshot with the text button's help. - - - Choose from a variety of colors to make the bug more noticeable. - - - The eraser icon highlighted below will help you to erase any unwanted highlights. - - - If you have got too much to erase, then the last button at the bottom will do the job for you. It will help you undo any unwanted change in the screenshot. - -**Step 4:** Once you are done annotating the screenshot, you need to click on the **Mark as Bug** button at the bottom-right of the image editor. - -## Pushing Bugs To Third Party Tools - -TestMu AI provides integration with various 3rd party applications to help you collaborate with your teammates easily as you perform cross browser testing on our cloud-based platform. The **"Mark as Bug"** button will help you to push your edited screenshot to your integrated workspace instantly. You can integrate with the below mentioned 3rd party tools. - -- [JIRA](/docs/jira-integration/) -- [Slack](/docs/slack-integration/) -- [Asana](/docs/asana-integration/) -- [Trello](/docs/trello-integration/) -- [GitHub](/docs/github-integration/) -- [GitLab](/docs/gitlab-integration/) -- [Bitbucket](/docs/bitbucket-integration/) -- [Azure DevOps previously known as VSTS](/docs/vsts-integration/) -- [Paymo](/docs/paymo-integration/) -- [Teamwork](/docs/teamwork-integration/) -- [Hive](/docs/hive-integration/) -- [Shortcut](/docs/shortcut-integration/) -- [BugHerd](/docs/bugherd-integration/) -- [Mantis](/docs/mantis-integration/) -- [Monday.com](/docs/monday-com-integration/) -- [Breeze](/docs/breeze-integration-with-testmu/) -- [Clickup](/docs/clickup-integration/) -- [Zipboard](/docs/zipboard-integration/) -- [Backlog](/docs/backlog-integration-with-testmu/) -- [Pivotal](/docs/pivotal-tracker-integration/) -- [Practitest](/docs/practitest-integration/) -- [Miro](/docs/miro-integration/) - -We are continually adding more integration to our platform. If you use some other tool, don’t hesitate to share it with us! Just drop a message through **chat** & we’ll bring it here on our platform for you. - -As you click on the **"Mark as Bug"** button, an integration specific form would open up. Fill in the fields as per your requirement. For instance, if you have installed TestMu AI + Asana integration, you can fill the below represented fields. - -- You can select which workspace the should the task go under. -- You can specify which project the task should be placed under. -- You can select the assignee for that particular task. -- You can also provide a summary to help you organize & easily identify your tasks. -- You also get to post a description in the description field to help relate the cause of the issue or the card. - -All the fields you populate will be directly reflected in your respective project based upon the integration you have configured with your TestMu AI account. In the above case, the changes would be reflecting on the respective Asana instance. - -**Got any questions?** - -Feel free to share them with our **24/7 chat support**, or you can drop us a mail at support@testmuai.com. Happy testing! diff --git a/static/docs/mark-as-bug-in-screenshot-testing.md b/static/docs/mark-as-bug-in-screenshot-testing.md deleted file mode 100644 index d38da3eb3..000000000 --- a/static/docs/mark-as-bug-in-screenshot-testing.md +++ /dev/null @@ -1,90 +0,0 @@ -# Mark As Bug In Screenshot Testing - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -> You can now avoid the hassle of capturing a screenshot, annotating the same before you finally share it with a teammate by logging onto a respective tool for creating a task. TestMu AI does it all for you. **Mark as Bug** is a feature that would help you highlight any UI observation from your test sessions on TestMu AI. And you can share the same with your colleagues on their favorite project management/ bug tracking tools such as Jira and Trello, Hive, Paymo, etc. You can choose the assignee who handles your reported task, or Bug. You can even mention the description to help them understand your point better. You can do all that while testing on TestMu AI, without having to hop on to any other platform or tool. - -**Note:** Mark as Bug feature is not restricted to any particular feature at TestMu AI. In other words, every type of testing that you perform on TestMu AI will have a mark as bug embedded in it. Below are various types of testing that we offer at TestMu AI. - -- **Real Time Testing:** To help you test your web-app with live interaction by running it on VMs hosted by TestMu AI cloud server. - -- **Screenshot Testing:** Automated bulk [screenshot testing](/support/docs/automated-screenshot-testing/) on 25(browsers + OS) combinations in a single test session. - -- **Responsive Testing:** Perform cross browser testing across various mobile & desktop devices to ensure your website renders well across different view-ports. Run responsive testing on 46 devices in a single test session. - -- **Automation Testing:** Perform automated cross browser testing with our on-cloud Selenium Grid. You can execute parallel testing to trim down your test cycles drastically. Our [Selenium Grid](https://www.testmuai.com/blog/why-selenium-grid-is-ideal-for-automated-browser-testing/) is compatible with every test automation framework that supports Selenium. - -- **Smart UI Testing:** Automated visual regression testing that performs a pixel-to-pixel comparison between multiple images & highlights any UI deviations such as padding, icon, text, etc., concerning the baseline images. - -> **Did You Know?** -TestMu AI offers more than 3000+ real browsers and browser versions on every type of testing so you could maximize test coverage of your cross browser testing. - -## How Can I Mark As Bug While Screenshot Testing? - -**Note:** Below is a demonstration of marking a bug from Screenshot testing. - -**Step 1:** Go to Screenshot testing from the left navigation menu, select your desired browser to test from, and hit the **Capture** button. - -**Step 2:** Once you hit the **Capture** button, you will find a message on the top indicating that the screenshot test has begun. - -**Step 3:** Once the screenshot testing is complete, you will have all the screenshots categorized by browsers selected from your input. Double-click on any screenshot you may want to validate. - -**Step 4:** As you double-click on any screenshot, an image editor will pop up with various buttons to help you highlight and annotate your UI observation before sharing the screenshot with your colleagues. - -- You can highlight the bug through different shapes like square, rectangular, circular, or you can even underline. Environment details of the test will be presented at the bottom-left of the image editor. - -- You can also scribble the bug using the pencil icon & add text for annotating the screenshot with the text button's help. - -- Choose from a variety of colors available to make the bug more noticeable. - -- The eraser icon highlighted below will help you to erase any unwanted highlights. - -- If you have got too much to erase, then the last button at the bottom will do the job for you. It will help you undo any unwanted change in the screenshot. - -**Step 4:** Once you are done annotating the screenshot, you need to click on the **"Mark as Bug"** button at the bottom-right of the image editor. - -## Pushing Bugs To Third Party Tools - -TestMu AI provides integration with various 3rd party applications to help you collaborate with your teammates easily as you perform cross browser testing on our cloud-based platform. The "Mark as Bug" button will allow you instantly push your edited screenshot into your integrated workspace. You can integrate with the below mentioned 3rd party tools. - -- [JIRA](/docs/jira-integration/) -- [Slack](/docs/slack-integration/) -- [Asana](/docs/asana-integration/) -- [Trello](/docs/trello-integration/) -- [GitHub](/docs/github-integration/) -- [GitLab](/docs/gitlab-integration/) -- [Bitbucket](/docs/bitbucket-integration/) -- [Azure DevOps previously known as VSTS](/docs/vsts-integration/) -- [Paymo](/docs/paymo-integration/) -- [Teamwork](/docs/teamwork-integration/) -- [Hive](/docs/hive-integration/) -- [Shortcut](/docs/shortcut-integration/) -- [BugHerd](/docs/bugherd-integration/) -- [Mantis](/docs/mantis-integration/) -- [Monday.com](/docs/monday-com-integration/) -- [Breeze](/docs/breeze-integration-with-testmu/) -- [Clickup](/docs/clickup-integration/) -- [Zipboard](/docs/zipboard-integration/) -- [Backlog](/docs/backlog-integration-with-testmu/) -- [Pivotal](/docs/pivotal-tracker-integration/) -- [Practitest](/docs/practitest-integration/) -- [Miro](/docs/miro-integration/) - -We are continually adding more integration to our platform. If you use some other tool, don’t hesitate to share it with us! Just drop a message through **chat** & we’ll bring it here on our platform for you. - -As you click on the "**Mark as Bug**" button, an integration specific form would open up. Fill in the fields as per your requirement. For instance, if you have installed TestMu AI + Asana integration, you can fill the below-represented fields. - -- You can select which workspace the task should go under. - -- You can specify which project the task should be placed under. - -- You can choose the assignee for that particular task. - -- You can also provide a summary to help you organize & easily identify your tasks. - -- You also get to post a description in the description field to help relate the cause of the issue or the card. - -All the fields you populate will be directly reflected in your respective project based upon the integration you have configured with your TestMu AI account. In the above case, the changes would reflect on the respective Asana instance. - ->**Got any questions?** -Feel free to share them with our **24/7 chat support**, or you can drop a mail on support@testmuai.com. Happy testing! diff --git a/static/docs/mark-as-bug.md b/static/docs/mark-as-bug.md deleted file mode 100644 index 8d5505f6d..000000000 --- a/static/docs/mark-as-bug.md +++ /dev/null @@ -1,85 +0,0 @@ -# Mark As Bug In Real Time Testing - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -> -Now, you can avoid the hassle of capturing a screenshot, annotating the same before you finally share it with a teammate by logging onto a respective tool for creating a task. TestMu AI does it all for you. ->**Mark as Bug** is a feature that would help you highlight any UI observation from your test sessions on TestMu AI and share the same with your colleagues on their favourite project management/ bug tracking tools like Jira, Trello, Hive, Paymo, etc. You can choose the assignee who handles your reported task or bug. You can even mention the description to help them understand your point better. You can do all that while testing on TestMu AI, without having to hop on to any other platform or tool. - -**Note:** Mark as Bug feature is not restricted to any particular feature at TestMu AI. In other words, every type of testing that you perform on TestMu AI will have **mark as bug** embedded in it. Below are various types of testing that we offer at TestMu AI. -* **Real Time Testing:** To help you test your web-app with live interaction by running it on VMs hosted by TestMu AI cloud server. -* **Screenshot Testing:** Automated bulk screenshot testing on 25(browsers + OS) combination in a single test session. -* **Responsive Testing:** Perform cross browser testing across a variety of mobile & desktop devices to ensure your website renders well across different view-ports. Run responsive testing on 46 devices in a single test session. -* **Automation Testing:** Perform automated cross browser testing with our on-cloud Selenium Grid. You can execute parallel testing to trim down your test cycles drastically. Our Selenium Grid supports is compatible with every test automation framework that supports Selenium. -* **Smart UI Testing:** Automated visual regression testing that performs a pixel-to-pixel comparison between multiple images & highlights any UI deviations such as padding, icon, text, etc with respect to the baseline images. - -> -**Did You Know?** -TestMu AI offers more than 2000 real browsers and browser versions on every type of testing so you could maximize test coverage of your cross browser testing. - -## How Can I Mark As Bug ? - -**Step 1**: Visit Real Time Testing from the left navigation menu to fire up a VM of your favourite browser + OS configuration by hitting the Start button. - -**Step 2**: Once the desired VM is launched and operable, you will notice a controller buttons on the left-side of the screen. Click the button with bug icon to generate a screenshot for the visible part of screen you are currently on inside the VM. - -**Step 3**: As you click on the bug icon, an image editor will pop up with various buttons to help you highlight and annotate your UI observation before sharing the screenshot with your colleagues. - - * You can highlight the bug through different shapes like square, rectangular, circular or you can even underline. - - * You can also scribble the bug using the pencil icon & can add text for annotating the screenshot with the help of text button. - - * Choose from a variety of colors for to make the bug more noticeable. - - * The erasor icon highlighted below will help you to erase any unwanted highlights. - - * If you have got too much to erase, then the last button at the bottom will do the job for you. It will help you undo any unwanted change in the screenshot. - -**Step 4**: Once you are done annotating the screenshot, you need to click on the Mark as Bug button at the bottom-right of the image editor. - -You can also find the **Mark as Bug** button in your gallery. That way you can reflect on your saved screenshots and log an issue from your already captured screenshots. - -You can access the saved screenshot from the below-highlighted gallery button on the left-side, inside your running VM. You will find the **download button** and **mark as bug button** in the gallery as well. - -## Pushing Bugs To Third Party Tools - -TestMu AI provides integration with various 3rd party applications to help you collaborate with your teammates easily as you perform cross browser testing on our cloud-based platform. **“Mark as Bug”** button will help you to instantly push your edited screenshot to your integrated workspace. You can integrate with the below mentioned 3rd party tools. - -* [**JIRA**](/support/docs/jira-integration/) -* [**Slack**](/support/docs/slack-integration/) -* [**Asana**](/support/docs/asana-integration/) -* [**Trello**](/support/docs/trello-integration/) -* [**GitHub**](/support/docs/github-integration/) -* [**GitLab**](/support/docs/gitlab-integration/) -* [**Bitbucket**](/support/docs/bitbucket-integration/) -* [**Azure DevOps previously known as VSTS**](/support/docs/vsts-integration/) -* [**Paymo**](/support/docs/paymo-integration/) -* [**Teamwork**](/support/docs/teamwork-integration/) -* [**Hive**](/support/docs/hive-integration/) -* [**Clubhouse**](/support/docs/shortcut-integration/) -* [**BugHerd**](/support/docs/bugherd-integration/) -* [**Mantis**](/support/docs/mantis-integration/) -* [**Monday.com**](/support/docs/monday-com-integration/) -* [**Breeze**](/support/docs/breeze-integration-with-testmu/) -* [**Clickup**](/support/docs/clickup-integration/) -* [**Zipboard**](/support/docs/zipboard-integration/) -* [**Backlog**](/support/docs/backlog-integration-with-testmu/) -* [**Pivotal**](/support/docs/pivotal-tracker-integration/) -* [**Practitest**](/support/docs/practitest-integration/) -* [**Miro**](/support/docs/miro-integration/) - -We are constantly adding more integration to our platform. If you use some other tool, don’t hesitate in sharing it with us! Just drop a message through **Chat** & we’ll bring it here on our platform for you. - -As you click on **“Mark as Bug”** button an integration specific form would open up. Fill the fields as per your requirement. For instance, if you have installed TestMu AI + monday.com integration then you can fill the below represented fields. - -* You can select which **board** should the task go under. -* You can specify which **group** should the task be placed under. -* You can select the **status** for that particular task. -* You can also provide a **title** to help you organize & easily identify your tasks. -* You also get to post a description in the **text** field to help relate the cause of the issue or the card. - -All the fields you populate, will be directly reflected in your respective project based upon the integration you have configured with your TestMu AI account. In above case, the changes would be reflect on the respective monday.com instance. - -> -**Got any questions?** -Feel free to share them with our **24/7 chat support** or you can drop a mail on support@testmuai.com. Happy testing! diff --git a/static/docs/maven-tunnel-for-locally-hosted-web-applications.md b/static/docs/maven-tunnel-for-locally-hosted-web-applications.md deleted file mode 100644 index 93a4ec748..000000000 --- a/static/docs/maven-tunnel-for-locally-hosted-web-applications.md +++ /dev/null @@ -1,295 +0,0 @@ -# Maven Tunnel For Locally Hosted Web Applications - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Maven is a renowned build automation tool for orchestrating project builds by automatically handling Selenium dependencies, compiling source code to binary & then packaging of the binary. TestMu AI offers an SSH (Secure Shell) tunnel which allows you to test your locally hosted websites over 3000+ browsers for both mobile and desktop through TestMu AI Selenium Grid. - -This document will help you configure a secure TestMu AI Tunnel to execute your [Selenium test automation](https://www.testmuai.com/selenium-automation) scripts through your Maven project on our online Selenium Grid. - -> TestMu AI Maven Tunnel is now compatible with both **Java 17** and **Java 20**. Additionally, we also rolled out the latest **[Maven Tunnel](https://mvnrepository.com/artifact/com.github.lambdatest/lambdatest-tunnel-binary/4.0.2)**, with various enhancements and optimizations. - -## Prerequisites - -**TestMu AI Authentication Credentials:** -Be aware of your TestMu AI authentication credentials i.e. your TestMu AI username, access key and HubURL. You need to set them up as your environment variables. You can retrieve them from your TestMu AI automation dashboard by clicking on the key icon near the help button. - -**For Linux/Mac:** - -```javascript -$ export LT_USERNAME= {YOUR_LAMBDATEST_USERNAME} -$ export LT_ACCESS_KEY= {YOUR_LAMBDATEST_ACCESS_KEY} -``` - -**For Windows:** -```javascript -$ set LT_USERNAME= {YOUR_LAMBDATEST_USERNAME} -$ set LT_ACCESS_KEY= {YOUR_LAMBDATEST_ACCESS_KEY} -``` - -* Make sure you have [Maven installed](https://maven.apache.org/install.html). -* Install [Java 8](https://www.oracle.com/in/java/technologies/javase/javase-jdk8-downloads.html). - -## Set Up Maven Sample Project & Dependencies - -> Note: Make sure you clone our [GitHub repository for Maven tunnel](https://github.com/LambdaTest/maven-tunnel-sample). - -Navigate to the cloned directory & compile maven dependencies using the command: - -`$ mvn compile` - -Ensure that you have the Maven Tunnel dependency configured in your `pom.xml`. - -```js - -com.github.lambdatest -lambdatest-tunnel-binary -4.0.2 - -``` - -## Running Maven Tunnel - -Here is the code that is present in the cloned [GitHub repository for Maven Tunnel](https://github.com/LambdaTest/maven-tunnel-sample). - -```js -package com.lambdatest.tunnel; - -import java.net.URL; -import java.util.HashMap; -import org.openqa.selenium.JavascriptExecutor; -import org.openqa.selenium.WebDriver; -import org.openqa.selenium.remote.DesiredCapabilities; -import org.openqa.selenium.remote.RemoteWebDriver; -import org.testng.annotations.AfterTest; -import org.testng.annotations.BeforeTest; -import org.testng.annotations.Test; -import com.lambdatest.tunnel.Tunnel; - -public class MavenSingle { -Tunnel t; - -WebDriver driver = null; -public static String status = "passed"; - -String username = System.getenv("LT_USERNAME"); -String access_key = System.getenv("LT_ACCESS_KEY"); - -@BeforeTest -public void setUp() throws Exception { - -DesiredCapabilities capabilities = new DesiredCapabilities(); -capabilities.setCapability("build", "Single Maven Tunnel"); -capabilities.setCapability("name", "Maven Tunnel"); -capabilities.setCapability("platform", "Windows 10"); -capabilities.setCapability("browserName", "Chrome"); -capabilities.setCapability("version","latest"); -capabilities.setCapability("tunnel",true); -capabilities.setCapability("network",true); -capabilities.setCapability("console",true); -capabilities.setCapability("visual",true); - -//create tunnel instance -t = new Tunnel(); -HashMap options = new HashMap(); -options.put("user", username); -options.put("key", access_key); - -//start tunnel -t.start(options); -driver = new RemoteWebDriver(new URL("http://" + username + ":" + access_key + "@hub.lambdatest.com/wd/hub"), capabilities); -System.out.println("Started session"); -} - -@Test() -public void testTunnel() throws Exception { -//Check LocalHost on XAMPP -driver.get("http://localhost.lambdatest.com"); -// Let's check that the item we added is added in the list. -driver.get("https://google.com"); -} - -@AfterTest -public void tearDown() throws Exception { -((JavascriptExecutor) driver).executeScript("lambda-status=" + status); -driver.quit(); -//close tunnel -t.stop(); -} -} -``` - -## Parallel Testing With Maven Tunnel - -In order to trigger parallel test execution you will need to specify the parameters inside an XML file, in our GitHub repository you will find the XML file named as `MavenParallel.xml` file. - -```js - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -``` - -Here is the final code to execute parallel testing with Maven tunnel. - -```js -package com.lambdatest.tunnel; - -import java.net.MalformedURLException; -import java.net.URL; -import java.util.HashMap; -import org.openqa.selenium.JavascriptExecutor; -import org.openqa.selenium.remote.DesiredCapabilities; -import org.openqa.selenium.remote.RemoteWebDriver; -import org.testng.IExecutionListener; -import org.testng.annotations.AfterTest; -import org.testng.annotations.BeforeTest; -import org.testng.annotations.Test; -import com.lambdatest.tunnel.Tunnel; - -public class MavenParallel implements IExecutionListener{ - -public RemoteWebDriver driver = null; -String status = "passed"; -String username = System.getenv("LT_USERNAME"); -String accessKey = System.getenv("LT_ACCESS_KEY"); -Tunnel t; - -@Override -public void onExecutionStart() { -try { -//start the tunnel -t = new Tunnel(); -HashMap options = new HashMap(); -options.put("user", username); -options.put("key", accessKey); -t.start(options); -} catch (Exception e) { -e.printStackTrace(); -} -} - -@BeforeTest -@org.testng.annotations.Parameters(value={"browser","version","platform", "resolution"}) -public void setUp(String browser, String version, String platform, String resolution) throws Exception { -DesiredCapabilities capabilities = new DesiredCapabilities(); - -capabilities.setCapability("build", "Parallel Maven Tunnel"); -capabilities.setCapability("name", "Maven Tunnel"); -capabilities.setCapability("browserName", browser); -capabilities.setCapability("version", version); -capabilities.setCapability("platform", platform); -capabilities.setCapability("tunnel",true); -capabilities.setCapability("network",true); -capabilities.setCapability("console",true); -capabilities.setCapability("visual",true); - -try { -driver= new RemoteWebDriver(new URL("https://"+username+":"+accessKey+"@hub.lambdatest.com/wd/hub"), capabilities); -} catch (MalformedURLException e) { -System.out.println("Invalid grid URL"); -} -} - -@Test() -public void testTunnel() throws Exception { -//Check LocalHost on XAMPP -driver.get("http://localhost.lambdatest.com"); -// Let's check that the item we added is added in the list. -driver.get("https://google.com"); -} - -@AfterTest -public void tearDown() throws Exception { -if (driver != null) { -((JavascriptExecutor) driver).executeScript("lambda-status=" + status); -driver.quit(); -} -} - -@Override -public void onExecutionFinish() { -try { -//stop the Tunnel; -t.stop(); -} catch (Exception e) { -e.printStackTrace(); -} -} -} -``` -## Enable Parallel Testing With Maven Tunnel using Serentiy BDD - -In order to trigger parallel test execution with Maven Tunnel using Serenity BDD, you will need to specify the dependency of `lambdatest-tunnel-binary` with the exclusion of TestNG dependency in `MavenParallel.xml` file. - -```js - -com.github.lambdatest -lambdatest-tunnel-binary -4.0.2 - - -org.testng -testng - - - -``` - -## Optional Arguments For TestMu AI Tunnel - -| ARGUMENT | SYNTAX | DESCRIPTION | -|---|---|---| -| tunnelName | `tunnelArgs.put("tunnelName","YourName");` | To change the name of the TestMu AI Tunnel. | -| pidFile | `tunnelArgs.put("pidFile","Your/pid/path");` | To change the PID path. | -| dir | `tunnelArgs.put("dir","give/lambda/directory/path");` | To change the directory path. | -| logFile | `tunnelArgs.put("logFile","give/tunnel/log/directory/path");` | To change the path for Tunnel logs. | -| loadBalanced | `tunnelArgs.put("load-balanced","true");` | To start the tunnels in load balanced mode. | -| customBinaryPath | `tunnelArgs.put("binary", "")` | To give custom binary path. | -| basicAuth | `tunnelArgs.put("basicAuth", "")` | Basic Auth. | -| mitm | `tunnelArgs.put("mitm", "true")` | MITM only if basicAuth is provided. | - -> For more arguments, refer to our document of [TestMu AI Tunnel Modifiers](/support/docs/lambda-tunnel-modifiers/). diff --git a/static/docs/microsoft-teams-app.md b/static/docs/microsoft-teams-app.md deleted file mode 100644 index f690f10a7..000000000 --- a/static/docs/microsoft-teams-app.md +++ /dev/null @@ -1,97 +0,0 @@ -# Microsoft Teams App Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Microsoft Teams or MS Teams is a cloud-based collaboration application that helps with the handy features for document sharing, online meetings, and marketing communications. It helps your team to stay organized and have interactions in one place. - -The TestMu AI-Microsoft Teams App integration helps you in, - -* Quickly launch your real-time tests and screenshot tests directly inside from any conversation or channel in a single click. -* Performing the real-time tests and screenshot tests across 3000+ browser versions on TestMu AI cloud. -* Reuse the test cards and launch the test again with the same configurations. - -## How To Add TestMu AI From Microsoft Teams App? - -Here is how to set up the integrations to integrate the TestMu AI from the Microsoft Teams app store. - -**Step 1**: Open the Microsoft Teams App. - -**Step 2**: Navigate to the ‘Apps’ tab from the left menu bar. - -**Step 3**: Search for ‘TestMu AI’. - -**Step 4**: Click on the ‘Add’ button. - -**Step 5**: Upon successful integration, you’ll receive the welcome message confirming the same. The TestMu AI app will be reflected in your left navigation menu as well as on your chatbot. - -You can also pin the TestMu AI app to your left navigation menu by right-clicking on the TestMu AI App icon. - -> Note: To initiate the Real-time testing and Screenshot testing, you would need to login with your TestMu AI Account. If you don’t have a TestMu AI account, you can register yourself by clicking at the ‘Register for TestMu AI account‘ link as shown below: - -## How To Perform Real-Time Testing From Microsoft Teams App? - -With TestMu AI+Microsoft Teams App integration, you can now perform Real-time testing of your websites over 3000+ browsers directly from inside your any personal chat or team channel in a single click. You can share the UI bugs, screenshots, and videos directly to your personal chat or team channel. - -**Step 1**: Click on the ‘TestMu AI bot‘ from the chat option and select ‘Real-time testing‘. - -**Step 2**: A pop-up box will appear, click on ‘Sign in‘. - -**Step 3**: A prompt box will appear where you need to log in with your TestMu AI credentials. You can also log in by clicking on the ‘Sign in with Google‘. - -**Step 4**: After logging in, you’ll be taken to the Real-time test prompt box. Enter your test URL. Choose the desktop or mobile combination. Select the browser, browser version, OS, and OS resolution, and click on the ‘Create Launcher‘. - -Step 5: A Real-time test card will be generated consisting of the test configuration details that you just selected. To launch the test, click on the ‘Run Session‘. - -> Note:You can also send the particular Realtime test card to the chats or share it with your team channels by pressing the send arrow icon. - -**Step 6**: You will be routed to a TestMu AI cloud-based virtual machine where you can perform real-time testing of your websites and web apps. You can perform one-click bug logging, capture screenshots, record videos, perform geolocation testing, and much more. - -For further information on the Real-time Testing, refer to our documentation [Real Time Browser Testing](/docs/getting-started-with-desktop-browser-real-time-testing/). - -> Note: You can create the real-time test from inside your team conversation and direct conversation. You and your team members can directly launch the session from the respective card with a click of a button, resulting in better collaboration and an easier way to find UI bugs. - -* You can create the test from inside your team conversation and share it within your team channel or groups. - -* You can also create the test from inside your direct conversation and share it with your team members. - -## How To Perform Screenshot Testing From Microsoft Teams App? - -> Note: If you already logged in and executed the Realtime tests, you won’t be asked to login again. - -With TestMu AI+Microsoft Teams App integration, you can now perform Screenshot testing of your websites over 3000+ browsers directly from inside your any personal chat or team channel in a single click. You can view and share your screenshots directly to your personal chat or team channel. - -**Step 1**: Click on the ‘TestMu AI bot‘ from the chat option and select ‘Screenshot testing‘. - -**Step 2**: A pop-up box will appear, click on ‘Sign in‘. - -**Step 3**: A prompt box will appear where you need to log in with your TestMu AI credentials. You can also log in by clicking on the ‘Sign in with Google‘. - -**Step 4**: After logging in, you’ll be taken to the Screenshot test prompt box. Enter your test URL. Choose the desktop or mobile combination. Select the browsers on which you wish to test on, select the respective operating system, browser versions and click on the ‘CAPTURE‘. - -**Step 5**: A Screenshot test card will be generated consisting of the test configuration details that you just selected. To view the screenshots, click on the ‘View Test‘. - -> Note:You can also send the generated Screenshot test card to the personal chats or share it with your team channels by pressing the send arrow icon. - -**Step 6**: You will be routed to a TestMu AI Screenshot console page where you can view and download the automated screenshots of your websites and web apps. - -You can also view and download your automated screenshots through the Screenshot test cards by clicking on the ‘View Shareable Link‘. - -You will be guided to a page where you can view the individual screenshot and download it by clicking on the download arrow icon or download all the screenshots as a zipped file by clicking on ‘Download All‘. - -For further information about Screenshot testing, refer to our documentation [Automated Screenshot Testing](/docs/automated-screenshot-testing/). - -> Note: You can create the Screenshot test from inside your team conversation and direct conversation. You and your team members can directly view the test and screenshot. You can also view the screenshot via a shareable link from the respective card with a click of a button, resulting in better collaboration and an easier way to identify browser compatibility issues. - -* You can create the test from inside your team conversation and share it within your team channel or groups. - -* You can also create the test from inside your direct conversation and share it with your team members. - -## How To Logout From The TestMu AI App? - -**Step 1**: Navigate to 'Chat' option from the left menu and open the TestMu AI chat. - -**Step 2**: In the conversation box, type the command `logout` and press the Send icon. - -After few seconds, you’ll receive the confirmation message stating- *'You have successfully logged out'*. - -> That’s all about TestMu AI+Microsoft Teams App integration. If you have any questions or require any further information, you can reach out to us at our **24/7 chat portal** or drop us a mail at support@testmuai.com. diff --git a/static/docs/microsoft-teams-integration.md b/static/docs/microsoft-teams-integration.md deleted file mode 100644 index a1dd7cd9d..000000000 --- a/static/docs/microsoft-teams-integration.md +++ /dev/null @@ -1,58 +0,0 @@ -# Microsoft Teams Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Microsoft Teams is a leading communication and collaboration platform that combines workplace chat, video conferencing, file storage, and app integration. Designed for teamwork, it enables seamless interaction and productivity within organizations, making it an ideal tool for managing projects and workflows. - -The Microsoft Teams integration with TestMu AI enables seamless collaboration and real-time updates on your test automation workflows. With this integration, you can: -- Receive instant notifications on test status and results directly in your Microsoft Teams channels. -- Collaborate efficiently by sharing test execution logs and reports with your team. -- Stay informed about your test runs without leaving your Teams environment. - -## Generate your Webhook URL - -**Step 1:** Visit your Teams account -> click on the **+** button and create a channel in which you want to recieve all your notifications. - -**Step 2:** Click on the **...** button in the top right section of your channel and select workflows. - -**Step 3:** Provide the description and create a new workflow by selecting the channels - -**Step 4:** Copy your generated workflow, you will need it in the next step. - -## Integrate Teams with your TestMu AI Account - -**Step 1:** Login to your TestMu AI account. You should have Admin or User level access to see and install integrations. - -**Step 2:** Click on Settings -> Integrations -> Communication. - -**Step 3:** Click on **Connect** button of `Microsoft Teams` block. - -**Step 4:** Now, provide your Microsoft Teams Workflow URL to establish integration with TestMu AI and click on **install** button. - -## Lodge your First Bug - -> Note: If you are using Rocket.Chat for the first time, then make sure to create a project for yourself. It is a pre-requisite in order to push screenshots from your TestMu AI account. - -**Step 1:** Start with any type of testing, for the demo purpose we are going with the **Real Time Testing**. - -**Step 2:** Enter your Project URL and configure for browser and operating system of your choice & hit **Start**. - -**Step 3:** After the VM is launched and operable. You can perform testing on your web-app for finding bugs. If a bug gets revealed, then you need to click on the Bug icon from the left panel for capturing a screenshot of the same. - -**Step 4:** After clicking on "**Mark as Bug**" button your respective **Bug Tracker** tool or **Project Management** tool specific form would open up. Fill the fields as per your requirement. - -> Make sure you mark on the check at the last of the form to send the notification on your integrated Teams channel. - -**Step 5:** Click on **Mark as Bug** button. Now go to your dashboard and check a ticket will be created for the same. - -**Step 6:** Go to your Teams channel and you can check a notification is sent for the same. - -## Uninstall Microsoft Teams Integration - -**Step 1:** Login to your TestMu AI account. You should have Admin or User level access to see and install integrations. - -**Step 2:** Click on Settings -> Integrations -> Communication. - -**Step 3:** Click on the **Remove** button. - -> That was all you need to know for TestMu AI + Teams Integration. Increase your productivity multifold with our integrations. If you still have any questions for us, please feel free to let us know. Our experts are always **available on chat** to help you out with any roadblock regarding our product. Happy testing! diff --git a/static/docs/migrate-appium-tests-from-browserstack-and-saucelabs.md b/static/docs/migrate-appium-tests-from-browserstack-and-saucelabs.md deleted file mode 100644 index 102459ff1..000000000 --- a/static/docs/migrate-appium-tests-from-browserstack-and-saucelabs.md +++ /dev/null @@ -1,542 +0,0 @@ -# Migrate Appium Tests - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## From Local Grid, BrowserStack Or SauceLabs To TestMu AI - -TestMu AI offers an online Appium automation grid to perform App automation. The online Appium Grid is available on local grid, BrowserStack, SauceLabs and TestMu AI. Therefore, you can effortlessly migrate your current Appium automation scripts (or suites) from local grid, Sauce Labs or BrowserStack to TestMu AI. - -In this documentation, we look at how to leverage TestMu AI cloud for App automation and migrate your test scripts (or test suites) from your local grid, Sauce labs or BrowserStack. You can use TestMu AI's desired capabilities in your tests, authenticate your test session, and execute tests on the cloud. - -## Introduction - -Migrating your current local grid, BroweseStack or Sauce Labs tests to TestMu AI requires a few tweaks in your code. In this guide, we'll look at how to leverage TestMu AI's desired capabilities in your tests, authenticate your test session, and execute tests on our cloud browsers. - -## Migration From BrowserStack And SauceLabs - -## Changes In The Test Script - -To move from the local grid, BroweseStack or Sauce Labs to TestMu AI, you need to make some changes to your test suites such as authentication, desired capabilities etc. - -### Authentication - -Firstly, you need to change the authentication in the configuration settings of your test suite. For running tests on TestMu AI Appium Grid, you need to have a valid `user_name` and `access_key` to perform tests on our cloud Grid. In case you don’t have an account on TestMu AI, visit the TestMu AI **signup** page and create a new account. - -The following are the changes in the parameters: - -- Username -- Access Key - -You can find the `Username` and `Access Key` in the **TestMu AI Profile Section** of the **Automation Dashboard**. - -When migrating from BrowserStack or SauceLabs to TestMu AI, you need to make the following changes in the existing code: - -1. UserName -2. AccessKey -3. Hub URL -4. Desired Capabilities - -Here is a side-by-side comparison of each of the fields that we have highlighted above: - -| Property | Type | BrowserStack | SauceLabs | TestMu AI | -| --------- | ------ | ----------------------------------------------- | --------------------------------------------- | --------------------------------------------- | -| UserName | String | UserName to access Appium Grid on BrowserStack | UserName to access Appium Grid on Sauce Labs | UserName to access Appium Grid on TestMu AI | -| AccessKey | String | AccessKey to access Appium Grid on BrowserStack | AccessKey to access Appium Grid on Sauce Labs | AccessKey to access Appium Grid on TestMu AI | -| Hub URL | String | @hub-cloud.browserstack.com/wd/hub | ondemand.us-west-1.saucelabs.com/wd/hub | @hub.lambdatest.com/wd/hub | - -For a Python-based implementation, here are the changes in the script for the authentication process. - -**BrowserStack** - -```python -userName = "BrowserStack_UserName" -accessKey = "BrowserStack_AccessKey" -``` - -**SauceLabs** - -```python -userName = "SAUCE_USERNAME" -accessKey = "SAUCE_ACCESS_KEY" -``` - -**TestMu AI** - -```python -userName = "LambdaTest_UserName" -accessKey = "LambdaTest_AccessKey" -``` - -### Changes To The Hub URL - -Now you have to modify the hub URL in your test suite's configuration settings. The Hub URL is of the String type and specifies the Hub location to which the Appium tests will be routed for execution. - -For a Python-based implementation, here are the changes in the script for Hub URL. - -**BrowserStack** - -``` -@hub-cloud.browserstack.com/wd/hub -``` - -**SauceLabs** - -``` -@ondemand.us-west-1.saucelabs.com/wd/hub -``` - -**TestMu AI** - -``` -@mobile-hub.lambdatest.com/wd/hub -``` - -### Desired Capability Generator - -Capabilities generator allows you to specify the desired capabilities (or capabilities), which are configuration options that allow you to specify the following: - -1. Device -2. Operating system - -You can also select other advanced options available in the TestMu AI Capabilities Generator. - -For the migration, we have taken Java-based Appium tests. Below are the screenshots of the capability generator of BrowserStack and TestMu AI. - -#### **BrowserStack** - -#### **Sauce Labs** - -#### **TestMu AI** - -The comparison of the capabilities generated by BrowserStack and TestMu AI capabilities generator: - -| Capabilities | BrowserStack | SauceLabs | TestMu AI | -| ---------------- | ------------ | --------------- | --------------- | -| Device | device | deviceName | deviceName | -| Operating System | os_version | platformVersion | platformVersion | - -The following is an overview of the comparison of Desired Capabilities for the Java language: - -**BrowserStack** - -```js -//demo.java -DesiredCapabilities capabilities = new DesiredCapabilities(); -capabilities.setCapability("os_version", "9.0"); -capabilities.setCapability("device", "Google Pixel 3"); -capabilities.setCapability("browserstack.appium_version", "1.21.0"); -``` - -**SauceLabs** - -```java -//demo.java -MutableCapabilities caps = new MutableCapabilities(); -caps.setCapability("platformName", "Android"); -caps.setCapability("browserName", "Chrome"); -caps.setCapability("appium:deviceName", "Google Pixel 3 GoogleAPI Emulator"); -caps.setCapability("appium:platformVersion", "12.0"); -MutableCapabilities sauceOptions = new MutableCapabilities(); -sauceOptions.setCapability("appiumVersion", "1.21.0"); -caps.setCapability("sauce:options", sauceOptions); -``` - -**TestMu AI** - -```js -//demo.java -DesiredCapabilities capabilities = new DesiredCapabilities(); -capabilities.setCapability("build", "your build name"); -capabilities.setCapability("name", "your test name"); -capabilities.setCapability("platformName", "Android"); -capabilities.setCapability("deviceName", "Google Pixel 3"); -capabilities.setCapability("isRealMobile", true); -capabilities.setCapability("platformVersion","9"); -``` - -### Example: Migration To TestMu AI - -Let's look an example that shows the entire migration process. The test scenario is to open a Wikipedia app that search the term ‘lambdatest’. The following test runs on Google Pixel 3 running Android 11. - -#### **BrowserStack** - -```python - -from appium import webdriver -from appium.webdriver.common.mobileby import MobileBy -from selenium.webdriver.support.ui import WebDriverWait -from selenium.webdriver.support import expected_conditions as EC -import time - -userName = "BrowserStack_UserName" -accessKey = "BrowserStack_AccessKey" - -desired_caps = { -"build": "Python Android", -"device": "Google Pixel 3", -"app": "" -} - -driver = webdriver.Remote("https://" + userName + ":" + accessKey + "@hub-cloud.browserstack.com/wd/hub", desired_caps) - -search_element = WebDriverWait(driver, 30).until( -EC.element_to_be_clickable((MobileBy.ACCESSIBILITY_ID, "Search Wikipedia")) -) -search_element.click() - -search_input = WebDriverWait(driver, 30).until( -EC.element_to_be_clickable((MobileBy.ID, "org.wikipedia.alpha:id/search_src_text")) -) -search_input.send_keys("BrowserStack") -time.sleep(5) - -search_results = driver.find_elements_by_class_name("android.widget.TextView") -assert(len(search_results) > 0) - -driver.quit() -``` - -#### **SauceLabs** - -```python -#samplewikipedia.py -import ssl - -try: -_create_unverified_https_context = ssl._create_unverified_context -except AttributeError: -# Legacy Python that doesn't verify HTTPS certificates by default -pass -else: -# Handle target environment that doesn't support HTTPS verification -ssl._create_default_https_context = _create_unverified_https_context - -from threading import Thread -import time -from selenium import webdriver -from selenium.webdriver.common.keys import Keys -from selenium.webdriver.common.desired_capabilities import DesiredCapabilities -from selenium.common.exceptions import TimeoutException -from selenium.webdriver.support.ui import WebDriverWait -from selenium.webdriver.support import expected_conditions as EC -from appium.webdriver.common.mobileby import MobileBy -from appium import webdriver - -# This array 'caps' defines the capabilities browser, device and OS combinations where the test will run - -caps = { -caps['platformName'] = 'Android' -caps['browserName'] = 'Chrome' -caps['appium:deviceName'] = 'Google Pixel 3a GoogleAPI Emulator' -caps['appium:platformVersion'] = '11.0' -caps['sauce:options'] = {} -caps['sauce:options']['appiumVersion'] = '1.20.2' -} - -# run_session function searches for 'saucelabs' on google.com - -def run_session(desired_cap): -driver = webdriver.Remote( - -command_executor="https://SAUCE_USERNAME:SAUCE_ACCESS_KEY@ondemand.us-west-1.saucelabs.com/wd/hub", -desired_capabilities=desired_cap) - -# driver.get("https://www.ifconfig.me") -# time.sleep(10) -# Test case for the saucelabs sample Android app. -# If you have uploaded your app, update the test case here. -search_element = WebDriverWait(driver, 30).until( -EC.element_to_be_clickable( -(MobileBy.ACCESSIBILITY_ID, "Search Wikipedia")) -) -search_element.click() -search_input = WebDriverWait(driver, 30).until( -EC.element_to_be_clickable( -(MobileBy.ID, "org.wikipedia.alpha:id/search_src_text")) -) -search_input.send_keys("saucelabs") -time.sleep(5) -search_results = driver.find_elements_by_class_name( -"android.widget.TextView") -assert(len(search_results) > 0) - -# Invoke driver.quit() after the test is done to indicate that the test is completed. -driver.quit() - -# The Thread function takes run_session function and each set of capability from the caps array as an argument to run each session in parallel -for cap in caps: -Thread(target=run_session, args=(cap,)).start() -``` - -#### **TestMu AI** - -```python -#samplewikipedia.py - -import ssl - -try: -_create_unverified_https_context = ssl._create_unverified_context -except AttributeError: -# Legacy Python that doesn't verify HTTPS certificates by default -pass -else: -# Handle target environment that doesn't support HTTPS verification -ssl._create_default_https_context = _create_unverified_https_context - -from threading import Thread -import time -from selenium import webdriver -from selenium.webdriver.common.keys import Keys -from selenium.webdriver.common.desired_capabilities import DesiredCapabilities -from selenium.common.exceptions import TimeoutException -from selenium.webdriver.support.ui import WebDriverWait -from selenium.webdriver.support import expected_conditions as EC -from appium.webdriver.common.mobileby import MobileBy -from appium import webdriver - -# This array 'caps' defines the capabilities browser, device and OS combinations where the test will run -caps = [ - -{ -"deviceName": "Google Pixel 3", -"platformName": "Android", -"platformVersion": "11", -"app": "", -"isRealMobile": True, -"deviceOrientation": "PORTRAIT", -"build": "Demo", -}, -] -# run_session function searches for 'lambtest' on google.com - -def run_session(desired_cap): -driver = webdriver.Remote( -# hub.mobile-dev-1.dev.lambdatest.io/wd/hub", -command_executor="https://LT_USERNAME:LT_ACCESS_KEY@mobile-hub.lambdatest.com/wd/hub", -desired_capabilities=desired_cap) - -# driver.get("https://www.ifconfig.me") -# time.sleep(10) -# Test case for the lambdatest sample Android app. -# If you have uploaded your app, update the test case here. -search_element = WebDriverWait(driver, 30).until( -EC.element_to_be_clickable( -(MobileBy.ACCESSIBILITY_ID, "Search Wikipedia")) -) -search_element.click() -search_input = WebDriverWait(driver, 30).until( -EC.element_to_be_clickable( -(MobileBy.ID, "org.wikipedia.alpha:id/search_src_text")) -) -search_input.send_keys("lambdatest") -time.sleep(5) -search_results = driver.find_elements_by_class_name( -"android.widget.TextView") -assert(len(search_results) > 0) - -# Invoke driver.quit() after the test is done to indicate that the test is completed. -driver.quit() - -# The Thread function takes run_session function and each set of capability from the caps array as an argument to run each session in parallel -for cap in caps: -Thread(target=run_session, args=(cap,)).start() -``` - -The majority of the implementation, as shown above, remains unchanged. Only changes to the infrastructure are made (i.e. instead of BrowserStack, the app automation tests would be run on TestMu AI). - -Let's analyze what has changed from the implementation point of view. - -**BrowserStack** - -```python - -from appium import webdriver -from appium.webdriver.common.mobileby import MobileBy -from selenium.webdriver.support.ui import WebDriverWait -from selenium.webdriver.support import expected_conditions as EC -import time - -userName = "BrowserStack_UserName" -accessKey = "BrowserStack_AccessKey" - -desired_caps = { -"build": "Python Android", -"device": "Google Pixel 3", -"app": "" -} - -driver = webdriver.Remote("https://" + userName + ":" + accessKey + "@hub-cloud.browserstack.com/wd/hub", desired_caps) -``` - -**SauceLabs** - -```python - -from threading import Thread -import time -from selenium import webdriver -from selenium.webdriver.common.keys import Keys -from selenium.webdriver.common.desired_capabilities import DesiredCapabilities -from selenium.common.exceptions import TimeoutException -from selenium.webdriver.support.ui import WebDriverWait -from selenium.webdriver.support import expected_conditions as EC -from appium.webdriver.common.mobileby import MobileBy -from appium import webdriver - -# This array 'caps' defines the capabilities browser, device and OS combinations where the test will run - -caps = { -caps['platformName'] = 'Android' -caps['browserName'] = 'Chrome' -caps['appium:deviceName'] = 'Google Pixel 3a GoogleAPI Emulator' -caps['appium:platformVersion'] = '11.0' -caps['sauce:options'] = {} -caps['sauce:options']['appiumVersion'] = '1.20.2' -} - -# run_session function searches for 'saucelabs' on google.com - -def run_session(desired_cap): -driver = webdriver.Remote( - -command_executor="https://SAUCE_USERNAME:SAUCE_ACCESS_KEY@ondemand.us-west-1.saucelabs.com/wd/hub", -desired_capabilities=desired_cap) - -``` - -**TestMu AI** - -```python -from threading import Thread -import time -from selenium import webdriver -from selenium.webdriver.common.keys import Keys -from selenium.webdriver.common.desired_capabilities import DesiredCapabilities -from selenium.common.exceptions import TimeoutException -from selenium.webdriver.support.ui import WebDriverWait -from selenium.webdriver.support import expected_conditions as EC -from appium.webdriver.common.mobileby import MobileBy -from appium import webdriver - -caps = [ - -{ -"deviceName": "Google Pixel 3", -"platformName": "Android", -"platformVersion": "11", -"app": "", -"build": "Demo", -}, -] - -def run_session(desired_cap): -driver = webdriver.Remote(command_executor="https://LT_USERNAME:LT_ACCESS_KEY@mobile-hub.lambdatest.com/wd/hub", desired_capabilities=desired_cap) - -``` - -We have discussed how to migrate from Sauce Labs or BrowserStack to TestMu AI. Let’s explore how to migrate from the local grid to the cloud-based Appium grid. - -## Migration From Local Grid - -### Desired Capabilities In Appium - -Appium's Desired Capabilities are a collection of key-value pairs wrapped inside a JSON object. These key-value pairs request the Appium server for the required test automation session. - -Let’s say you want to run an app test in Python on SAMSUNG GALAXY TAB S4 running ANDROID 10. You can define the same in the form of capability as given below. - -```python -caps = [ - -{ -"deviceName": "Galaxy Tab S4", -"platformName": "Android", -"platformVersion": "10", -"build": "Demo", -}, -] -``` - -### Running Tests On TestMu AI Appium Grid - -To begin, change the authentication in your test suite's configuration settings. To run the tests on TestMu AI Appium Grid, you need a valid user name and access key. If you were already performing tests on your local grid, you will need to modify your test script to initialize an Appium driver along with your desired capabilities. - -Pass the capabilities to `@hub.lambdatest.com/wd/hub` with your TestMu AI authentication details, and you are done. Here is the sample Python test script. - -```python -#samplewikipedia.py - -import ssl - -try: -_create_unverified_https_context = ssl._create_unverified_context -except AttributeError: -# Legacy Python that doesn't verify HTTPS certificates by default -pass -else: -# Handle target environment that doesn't support HTTPS verification -ssl._create_default_https_context = _create_unverified_https_context - -from threading import Thread -import time -from selenium import webdriver -from selenium.webdriver.common.keys import Keys -from selenium.webdriver.common.desired_capabilities import DesiredCapabilities -from selenium.common.exceptions import TimeoutException -from selenium.webdriver.support.ui import WebDriverWait -from selenium.webdriver.support import expected_conditions as EC -from appium.webdriver.common.mobileby import MobileBy -from appium import webdriver - -# This array 'caps' defines the capabilities of the browser, device, and OS combinations where the test will run -caps = [ - -{ -"deviceName": "Galaxy Tab S4", -"platformName": "Android", -"platformVersion": "10", -"app": "lt://APP10051525539885437397", -"isRealMobile": True, -"deviceOrientation": "PORTRAIT", -"visual": True, -"console": True, -"build": "Demo", -}, -] -# run_session function searches for 'lambdatest' on google.com - -def run_session(desired_cap): -driver = webdriver.Remote( -# hub.mobile-dev-1.dev.lambdatest.io/wd/hub", -command_executor="https://LT_USERNAME:LT_ACCESS_KEY@mobile-hub.lambdatest.com/wd/hub", -desired_capabilities=desired_cap) - -# driver.get("https://www.ifconfig.me") -# time.sleep(10) -# Test case for the lambdatest sample Android app. -# If you have uploaded your app, update the test case here. -search_element = WebDriverWait(driver, 30).until( -EC.element_to_be_clickable( -(MobileBy.ACCESSIBILITY_ID, "Search Wikipedia")) -) -search_element.click() -search_input = WebDriverWait(driver, 30).until( -EC.element_to_be_clickable( -(MobileBy.ID, "org.wikipedia.alpha:id/search_src_text")) -) -search_input.send_keys("lambdatest") -time.sleep(5) -search_results = driver.find_elements_by_class_name( -"android.widget.TextView") -assert(len(search_results) > 0) - -# Invoke driver.quit() after the test is done to indicate that the test is completed. -driver.quit() - -# The Thread function takes run_session function and each set of capability from the caps array as an argument to run each session in parallel -for cap in caps: -Thread(target=run_session, args=(cap,)).start() - -``` - -> That’s all! In case you have any questions or need any additional information, you could reach out at our **24X7 Chat Support** or mail us directly at support@testmuai.com. diff --git a/static/docs/migrate-appium-tests-from-sauce-labs.md b/static/docs/migrate-appium-tests-from-sauce-labs.md deleted file mode 100644 index 54c9e6a2d..000000000 --- a/static/docs/migrate-appium-tests-from-sauce-labs.md +++ /dev/null @@ -1,347 +0,0 @@ -# Migrate Appium Tests From Sauce Labs To TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The online Appium Grid is available on both TestMu AI and Sauce Labs. Therefore, you can effortlessly migrate your current Appium automation scripts (or suites) from Sauce Labs to TestMu AI. - -## Introduction - -Migrating your current Sauce Labs tests to TestMu AI requires a few tweaks in your code. In this guide, we'll look at how to leverage TestMu AI's desired capabilities in your tests, authenticate your test session, and execute tests on our cloud browsers. - -### Changes In The Test Script - -To move from the Sauce Labs to TestMu AI, you need to make some changes to your test suites such as authentication, desired capabilities etc. - -### Authentication - -Firstly, you need to change the authentication in the configuration settings of your test suite. For running tests on TestMu AI Appium Grid, you need to have a valid user_name and access_key to perform tests on our cloud Grid. In case you don’t have an account on TestMu AI, visit the TestMu AI signup page and create a new account. - -The following are the changes in the parameters: - -* Username -* Access Key - -You can find the Username and Access Key in the TestMu AI Profile Section of the Automation Dashboard. - -When migrating from Sauce Labs to TestMu AI, you need to make the following changes in the existing code: - -1. UserName -2. AccessKey -3. Hub URL -4. Desired Capabilities - -Here is a side-by-side comparison of each of the fields that we have highlighted above: - -| Property | Type | Sauce Labs | TestMu AI | -| -------- | -----| ------------ | ---------- | -| UserName | String | UserName to access Appium Grid on Sauce Labs | UserName to access Appium Grid on TestMu AI | -| AccessKey | String | AccessKey to access Appium Grid on Sauce Labs | AccessKey to access Appium Grid on TestMu AI | -| Hub URL | String | ondemand.us-west-1.saucelabs.com/wd/hub | @hub.lambdatest.com/wd/hub | - -For a Python-based implementation, here are the changes in the script for the authentication process. - -**Sauce Labs** - -```python -userName = "SAUCE_USERNAME" -accessKey = "SAUCE_ACCESS_KEY" -``` -**TestMu AI** - -```python -userName = "LambdaTest_UserName" -accessKey = "LambdaTest_AccessKey" -``` -### Changes To The Hub URL - -Now you have to modify the hub URL in your test suite's configuration settings. The Hub URL is of the String type and specifies the Hub location to which the Appium tests will be routed for execution. - -For a Python-based implementation, here are the changes in the script for Hub URL. - -**Sauce Labs** - -``` -@ondemand.us-west-1.saucelabs.com/wd/hub -``` - -**TestMu AI** - -``` -@mobile-hub.lambdatest.com/wd/hub -``` - -### Desired Capability Generator - -Capabilities generator allows you to specify the desired capabilities (or capabilities), which are configuration options that allow you to specify the following: - -1. Device -2. Operating system - -You can also select other advanced options available in the TestMu AI Capabilities Generator. - -For the migration, we have taken Java-based Appium tests. Below are the screenshots of the capability generator of Sauce Labs and TestMu AI. - -**Sauce Labs** - -**TestMu AI** - -The comparison of the capabilities generated by Sauce Labs and TestMu AI capabilities generator: - -| Capabilities | Sauce Labs | TestMu AI | -| -------- | -----| ------------ | -| Device | deviceName | deviceName | -| Operating System | platformVersion | platformVersion | - -The following is an overview of the comparison of Desired Capabilities for the Java language: - -**Sauce Labs** - -```py -# demo.py -caps = { -caps['platformName'] = 'Android' -caps['browserName'] = 'Chrome' -caps['appium:deviceName'] = 'Google Pixel 3a GoogleAPI Emulator' -caps['appium:platformVersion'] = '11.0' -caps['sauce:options'] = {} -caps['sauce:options']['appiumVersion'] = '1.20.2' -} -``` - -**TestMu AI** - -```py -# demo.py -caps = [ -{ -"deviceName": "Google Pixel 3", -"platformName": "Android", -"platformVersion": "11", -"app": "", -"isRealMobile": True, -"deviceOrientation": "PORTRAIT", -"build": "Demo", -}, -] -``` - -## Example: Sauce Labs To TestMu AI Migration - -Let's look an example that shows the entire migration process. The test scenario is to open a Wikipedia app that search the term ‘lambdatest’. The following test runs on Google Pixel 3 running Android 11. - -**Sauce Labs** - -```python -#samplewikipedia.py -import ssl - -try: -_create_unverified_https_context = ssl._create_unverified_context -except AttributeError: -# Legacy Python that doesn't verify HTTPS certificates by default -pass -else: -# Handle target environment that doesn't support HTTPS verification -ssl._create_default_https_context = _create_unverified_https_context - -from threading import Thread -import time -from selenium import webdriver -from selenium.webdriver.common.keys import Keys -from selenium.webdriver.common.desired_capabilities import DesiredCapabilities -from selenium.common.exceptions import TimeoutException -from selenium.webdriver.support.ui import WebDriverWait -from selenium.webdriver.support import expected_conditions as EC -from appium.webdriver.common.mobileby import MobileBy -from appium import webdriver - -# This array 'caps' defines the capabilities browser, device and OS combinations where the test will run - -caps = { -caps['platformName'] = 'Android' -caps['browserName'] = 'Chrome' -caps['appium:deviceName'] = 'Google Pixel 3a GoogleAPI Emulator' -caps['appium:platformVersion'] = '11.0' -caps['sauce:options'] = {} -caps['sauce:options']['appiumVersion'] = '1.20.2' -} - -# run_session function searches for 'saucelabs' on google.com - -def run_session(desired_cap): -driver = webdriver.Remote( - -command_executor="https://SAUCE_USERNAME:SAUCE_ACCESS_KEY@ondemand.us-west-1.saucelabs.com/wd/hub", -desired_capabilities=desired_cap) - -# driver.get("https://www.ifconfig.me") -# time.sleep(10) -# Test case for the saucelabs sample Android app. -# If you have uploaded your app, update the test case here. -search_element = WebDriverWait(driver, 30).until( -EC.element_to_be_clickable( -(MobileBy.ACCESSIBILITY_ID, "Search Wikipedia")) -) -search_element.click() -search_input = WebDriverWait(driver, 30).until( -EC.element_to_be_clickable( -(MobileBy.ID, "org.wikipedia.alpha:id/search_src_text")) -) -search_input.send_keys("saucelabs") -time.sleep(5) -search_results = driver.find_elements_by_class_name( -"android.widget.TextView") -assert(len(search_results) > 0) - -# Invoke driver.quit() after the test is done to indicate that the test is completed. -driver.quit() - -# The Thread function takes run_session function and each set of capability from the caps array as an argument to run each session in parallel -for cap in caps: -Thread(target=run_session, args=(cap,)).start() -``` - -**TestMu AI** - -```python -#samplewikipedia.py -import ssl - -try: -_create_unverified_https_context = ssl._create_unverified_context -except AttributeError: -# Legacy Python that doesn't verify HTTPS certificates by default -pass -else: -# Handle target environment that doesn't support HTTPS verification -ssl._create_default_https_context = _create_unverified_https_context - -from threading import Thread -import time -from selenium import webdriver -from selenium.webdriver.common.keys import Keys -from selenium.webdriver.common.desired_capabilities import DesiredCapabilities -from selenium.common.exceptions import TimeoutException -from selenium.webdriver.support.ui import WebDriverWait -from selenium.webdriver.support import expected_conditions as EC -from appium.webdriver.common.mobileby import MobileBy -from appium import webdriver - -# This array 'caps' defines the capabilities browser, device and OS combinations where the test will run -caps = [ - -{ -"deviceName": "Google Pixel 3", -"platformName": "Android", -"platformVersion": "11", -"app": "", -"isRealMobile": True, -"deviceOrientation": "PORTRAIT", -"build": "Demo", -}, -] -# run_session function searches for 'lambtest' on google.com - -def run_session(desired_cap): -driver = webdriver.Remote( -# hub.mobile-dev-1.dev.lambdatest.io/wd/hub", -command_executor="https://LT_USERNAME:LT_ACCESS_KEY@mobile-hub.lambdatest.com/wd/hub", -desired_capabilities=desired_cap) - -# driver.get("https://www.ifconfig.me") -# time.sleep(10) -# Test case for the lambdatest sample Android app. -# If you have uploaded your app, update the test case here. -search_element = WebDriverWait(driver, 30).until( -EC.element_to_be_clickable( -(MobileBy.ACCESSIBILITY_ID, "Search Wikipedia")) -) -search_element.click() -search_input = WebDriverWait(driver, 30).until( -EC.element_to_be_clickable( -(MobileBy.ID, "org.wikipedia.alpha:id/search_src_text")) -) -search_input.send_keys("lambdatest") -time.sleep(5) -search_results = driver.find_elements_by_class_name( -"android.widget.TextView") -assert(len(search_results) > 0) - -# Invoke driver.quit() after the test is done to indicate that the test is completed. -driver.quit() - -# The Thread function takes run_session function and each set of capability from the caps array as an argument to run each session in parallel - -for cap in caps: -Thread(target=run_session, args=(cap,)).start() -``` -The majority of the implementation, as shown above, remains unchanged. Only changes to the infrastructure are made (i.e. instead of Sauce Labs, the app automation tests would be run on TestMu AI). - -Let's analyze what has changed from the implementation point of view. - -**Sauce Labs** - -```python - -from threading import Thread -import time -from selenium import webdriver -from selenium.webdriver.common.keys import Keys -from selenium.webdriver.common.desired_capabilities import DesiredCapabilities -from selenium.common.exceptions import TimeoutException -from selenium.webdriver.support.ui import WebDriverWait -from selenium.webdriver.support import expected_conditions as EC -from appium.webdriver.common.mobileby import MobileBy -from appium import webdriver - -# This array 'caps' defines the capabilities browser, device and OS combinations where the test will run - -caps = { -caps['platformName'] = 'Android' -caps['browserName'] = 'Chrome' -caps['appium:deviceName'] = 'Google Pixel 3a GoogleAPI Emulator' -caps['appium:platformVersion'] = '11.0' -caps['sauce:options'] = {} -caps['sauce:options']['appiumVersion'] = '1.20.2' -} - -# run_session function searches for 'saucelabs' on google.com - -def run_session(desired_cap): -driver = webdriver.Remote( - -command_executor="https://SAUCE_USERNAME:SAUCE_ACCESS_KEY@ondemand.us-west-1.saucelabs.com/wd/hub", -desired_capabilities=desired_cap) - -``` - -**TestMu AI** - -```python -from threading import Thread -import time -from selenium import webdriver -from selenium.webdriver.common.keys import Keys -from selenium.webdriver.common.desired_capabilities import DesiredCapabilities -from selenium.common.exceptions import TimeoutException -from selenium.webdriver.support.ui import WebDriverWait -from selenium.webdriver.support import expected_conditions as EC -from appium.webdriver.common.mobileby import MobileBy -from appium import webdriver - -caps = [ - -{ -"deviceName": "Google Pixel 3", -"platformName": "Android", -"platformVersion": "11", -"app": "", -"build": "Demo", -}, -] - -def run_session(desired_cap): -driver = webdriver.Remote(command_executor="https://LT_USERNAME:LT_ACCESS_KEY@mobile-hub.lambdatest.com/wd/hub", desired_capabilities=desired_cap) - -``` ->That’s all about Sauce Labs to TestMu AI migration. In case you have any questions or need any additional information, you could reach out at our **24X7 Chat Support** or mail us directly at support@testmuai.com. diff --git a/static/docs/migrate-existing-appium-test-suites.md b/static/docs/migrate-existing-appium-test-suites.md deleted file mode 100644 index cadc01931..000000000 --- a/static/docs/migrate-existing-appium-test-suites.md +++ /dev/null @@ -1,123 +0,0 @@ -# Migrate Appium Tests From Local Grid - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI offers an online Appium automation grid to perform App automation. Therefore you can easily migrate Appium tests from your local grid to the TestMu AI platform. - -In this documentation, we look at how to leverage TestMu AI cloud for App automation and migrate your test scripts (or test suites) from your local grid. You can use TestMu AI's desired capabilities in your tests, authenticate your test session, and execute tests on the cloud. - -## Changes In The Test Script - -To move from the local grid to the cloud-based Appium grid, you need to make some changes to your automation scripts like authentication, desired capabilities etc. - -## Authentication - -To run tests on TestMu AI, you will need TestMu AI authentication credentials: TestMu AI Username and a TestMu AI Access Key. If you do not have a TestMu AI account, visit the TestMu AI register page and create one. - -You can get the TestMu AI Username and TestMu AI Access Key from your TestMu AI Profile. - -## Desired Capabilities In Appium - -Appium's Desired Capabilities are a collection of key-value pairs wrapped inside a JSON object. These key-value pairs request the Appium server for the required test automation session. - -Let’s say you want to run an app test in Python on SAMSUNG GALAXY TAB S4 running ANDROID 10. You can define the same in the form of capability as given below. - -```python -caps = [ - -{ -"deviceName": "Galaxy Tab S4", -"platformName": "Android", -"platformVersion": "10", -"build": "Demo", -}, -] -``` - -## Running Tests On TestMu AI Appium Grid - -To begin, change the authentication in your test suite's configuration settings. To run the tests on TestMu AI Appium Grid, you need a valid user name and access key. If you were already performing tests on your local grid, you will need to modify your test script to initialize an Appium driver along with your desired capabilities. - -Pass the capabilities to `@hub.lambdatest.com/wd/hub` with your TestMu AI authentication details, and you are done. Here is the sample Python test script. - -```python -#samplewikipedia.py - -import ssl - -try: -_create_unverified_https_context = ssl._create_unverified_context -except AttributeError: -# Legacy Python that doesn't verify HTTPS certificates by default -pass -else: -# Handle target environment that doesn't support HTTPS verification -ssl._create_default_https_context = _create_unverified_https_context - -from threading import Thread -import time -from selenium import webdriver -from selenium.webdriver.common.keys import Keys -from selenium.webdriver.common.desired_capabilities import DesiredCapabilities -from selenium.common.exceptions import TimeoutException -from selenium.webdriver.support.ui import WebDriverWait -from selenium.webdriver.support import expected_conditions as EC -from appium.webdriver.common.mobileby import MobileBy -from appium import webdriver - -# This array 'caps' defines the capabilities of the browser, device, and OS combinations where the test will run -caps = [ - -{ -"deviceName": "Galaxy Tab S4", -"platformName": "Android", -"platformVersion": "10", -"app": "lt://APP10051525539885437397", -"isRealMobile": True, -"deviceOrientation": "PORTRAIT", -"visual": True, -"console": True, -"build": "Demo", -}, -] -# run_session function searches for 'lambdatest' on google.com - -def run_session(desired_cap): -driver = webdriver.Remote( -# hub.mobile-dev-1.dev.lambdatest.io/wd/hub", -command_executor="https://LT_USERNAME:LT_ACCESS_KEY@mobile-hub.lambdatest.com/wd/hub", -desired_capabilities=desired_cap) - -# driver.get("https://www.ifconfig.me") -# time.sleep(10) -# Test case for the lambdatest sample Android app. -# If you have uploaded your app, update the test case here. -search_element = WebDriverWait(driver, 30).until( -EC.element_to_be_clickable( -(MobileBy.ACCESSIBILITY_ID, "Search Wikipedia")) -) -search_element.click() -search_input = WebDriverWait(driver, 30).until( -EC.element_to_be_clickable( -(MobileBy.ID, "org.wikipedia.alpha:id/search_src_text")) -) -search_input.send_keys("lambdatest") -time.sleep(5) -search_results = driver.find_elements_by_class_name( -"android.widget.TextView") -assert(len(search_results) > 0) - -# Invoke driver.quit() after the test is done to indicate that the test is completed. -driver.quit() - -# The Thread function takes run_session function and each set of capability from the caps array as an argument to run each session in parallel -for cap in caps: -Thread(target=run_session, args=(cap,)).start() - -``` - -## Testing Locally Hosted Apps - -## Supported Languages And Frameworks - ->That’s all! In case you have any questions or need any additional information, you could reach out at our **24X7 Chat Support** or mail us directly at support@testmuai.com. diff --git a/static/docs/migrate-existing-playwright-tests.md b/static/docs/migrate-existing-playwright-tests.md deleted file mode 100644 index 8f8e9c630..000000000 --- a/static/docs/migrate-existing-playwright-tests.md +++ /dev/null @@ -1,73 +0,0 @@ -# Migrate Your Existing Playwright Tests - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI offers an online automation platform for test automation. Therefore you can easily migrate Playwright tests from your local grid to the TestMu AI platform. - -In this documentation, we look at how to migrate your existing Playwright test scripts (or test suites) from your local grid to TestMu AI. This lets you automate Playwright scripts across 40+ browsers and operating systems on TestMu AI cloud platform. - -## Sample Playwright Script Running On A Local Machine - -With just a few lines of code tweaks in your test script, you can migrate your Playwright tests running on a local grid to TestMu AI. A sample Playwright script below launches a browser on your local machine and runs the script. - -```js -const { chromium } = require('playwright') -const { expect } = require('@playwright/test'); - -(async () => { -const browser = await chromium.launch({ -headless: false -}); - -const page = await browser.newPage() -await page.goto('https://www.bing.com') -const element = await page.$('[aria-label="Enter your search term"]') -await element.click() -await element.type('LambdaTest') -await element.press('Enter') -const title = await page.title() - -expect(title).toEqual('LambdaTest - Search') - -await browser.close() -})() -``` - -## Changes In Scripts To Run Playwright Tests On TestMu AI - -The above script shows that `playwright['chromium'].launch` runs in a Chromium browser window. To run the test scripts on TestMu AI platform, you will need to add the following instead of `playwright['chromium'].launch`. - -```js -const { chromium } = require('playwright') -const { expect } = require('@playwright/test'); - -(async () => { -const capabilities = { -'browserName': 'Chrome', // Browsers allowed: `Chrome`, `MicrosoftEdge`, `pw-chromium`, `pw-firefox` and `pw-webkit` -'browserVersion': 'latest', -'LT:Options': { -'platform': 'Windows 10', -'build': 'Playwright Sample Build', -'name': 'Playwright Sample Test', -'user': process.env.LT_USERNAME, -'accessKey': process.env.LT_ACCESS_KEY, -} -} - -const browser = await chromium.connect({ -wsEndpoint: `wss://cdp.lambdatest.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` -}) - -// Test code ... - -await browser.close() -})() -``` - -In the above code snippet, you need to connect to the CDP endpoint at TestMu AI using `chromium.connect` method. The `capabilities` variable contains additional parameters that enable a specific browser and OS combination to be assigned to your test on TestMu AI. - -## Selecting Browser-OS Combinations For Test Runs - -To run your script, you can choose any of the browsers and OS combinations. Just specify the browserName, browserVersion, platform in the capabilities JSON before calling the CDP endpoint. - -To generate Playwright tests with AI coding assistants, see [Run Tests With Agent Skills](/support/docs/playwright-agent-skills/). diff --git a/static/docs/migrate-from-legacy-platform-to-testmu.md b/static/docs/migrate-from-legacy-platform-to-testmu.md deleted file mode 100644 index b6785cf5d..000000000 --- a/static/docs/migrate-from-legacy-platform-to-testmu.md +++ /dev/null @@ -1,214 +0,0 @@ -# How to Migrate From Legacy Test Execution Platform to TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Legacy test execution platforms present various challenges including limited browser and device support, slow test execution, lack of built-in test orchestration, and others; that can impact software quality and increase maintenance efforts and costs. Migrating to a cloud-based test execution platform like TestMu AI is a smart move for any organization that wants to improve software quality, reduce maintenance efforts, and cut costs. - -In this guide, we will look at how to migrate from legacy test execution platform to TestMu AI. - -## How to Migrate From Local Grid to TestMu AI - -The major difference between a local Selenium Grid and a cloud-based Selenium Grid like TestMu AI is the point of execution. - -On a local grid, tests run directly on the system where browser drivers (ChromeDriver, FirefoxDriver, etc.) are installed. -When using TestMu AI, tests execute remotely on cloud infrastructure, all you need is a valid TestMu AI account, and no driver installation is required. - -TestMu AI provides scalability, parallel execution, and increased reliability, which are difficult to maintain with a local grid setup. - -Migration is simple, tests running on a local grid can be executed on TestMu AI with minimal modifications, typically requiring no changes to your automation logic. Only the execution endpoint and capabilities need to be configured. - -## Run Your Script Locally - -You can run your script locally by executing it directly on your machine with your preferred browser setup. This allows you to quickly test functionality, debug issues, and verify results without relying on external environments. It's an efficient way to validate tests during development. - -## Connect Your Local Script to TestMu AI - -To migrate your existing local script to TestMu AI, you only need to update your WebDriver configuration with cloud capabilities and the TestMu AI Hub URL. Once the credentials and capabilities are added, the same test can run on remote browsers without code logic changes. - -### Authentication -Firstly, you need to change the authentication in your configuration settings of your test suite. For running tests on TestMu AI Selenium Grid, you need to have a valid user_name and access_key to perform tests on our Grid. In case you do not have an account on TestMu AI, visit the TestMu AI signup page and create a new account. - -When migrating your Selenium 4 tests from BrowserStack to TestMu AI, the following updates are required in your existing code: - -1. Get TestMu AI Credentials: You can find these credentials under Account Settings > [Password & Security](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/security/username-accesskey) and copy your Username and Access Key, then add them to the .env file to keep them safe from public exposure. - -2. Create .env file: Securely store your TestMu AI credentials, create a .env file in the root of your project and add the following values: - -``` -LT_USERNAME="" -LT_ACCESS_KEY="" -``` - -Once the .env file is set up, ensure your test framework correctly reads these variables at runtime. This helps keep your authentication secure and avoids hard-coding credentials within your scripts. With the credentials in place, you’re now ready to update your Hub URL for TestMu AI execution. - -### Add TestMu AI Hub URL -You need to now add the hub URL in the configuration settings of your test suite. Hub URL is of type String and it defines the Hub location to which the Selenium tests would be submitted for execution. - -```js -@hub.lambdatest.com/wd/hub -``` - -### TestMu AI Automation Capabilities -Add your capabilities using the [TestMu AI Capabilities Generator](https://www.testmuai.com/capabilities-generator/), where you can quickly generate the required browser, OS, and platform configurations for your test script. Select the environment you want, copy the capabilities, and paste them directly into your script to run on TestMu AI. - -```js -SafariOptions browserOptions = new SafariOptions(); -browserOptions.setPlatformName("MacOS Tahoe"); -browserOptions.setBrowserVersion("26"); -HashMap ltOptions = new HashMap(); -ltOptions.put("username", ""); -ltOptions.put("accessKey", ""); -ltOptions.put("w3c", true); -browserOptions.setCapability("LT:Options", ltOptions); -``` - -```js -DesiredCapabilities capabilities = new DesiredCapabilities(); -capabilities.setCapability("browserName", "Safari"); -capabilities.setCapability("browserVersion", "26"); -HashMap ltOptions = new HashMap(); -ltOptions.put("username", ""); -ltOptions.put("accessKey", ""); -ltOptions.put("platformName", "MacOS Tahoe"); -ltOptions.put("visual", true); -ltOptions.put("video", true); -capabilities.setCapability("LT:Options", ltOptions); -``` - -## Hands On Guide - TestMu AI Migration - -You can execute the same test that previously ran on a local environment by connecting it to the TestMu AI cloud grid. With minor configuration changes and added capabilities, your script can run cross-browser on TestMu AI without altering the core test logic. - -**Test Scenario**: - -This test script performs a basic text validation on the website [TestMu AI eCommerce Playground](https://ecommerce-playground.lambdatest.io/) and shows the expected execution results when running the test in the TestMu AI cloud. - -```java -// TextValidationTest.java -import org.openqa.selenium.WebDriver; -import org.openqa.selenium.remote.RemoteWebDriver; -import org.openqa.selenium.JavascriptExecutor; -import org.openqa.selenium.safari.SafariOptions; - -import java.net.URL; -import java.util.HashMap; - -public class TextValidationTest { - -public static void main(String[] args) throws Exception { - -String username = System.getenv("LT_USERNAME") == null ? -"Your LT Username" : System.getenv("LT_USERNAME"); - -String authkey = System.getenv("LT_ACCESS_KEY") == null ? -"Your LT AccessKey\n" : System.getenv("LT_ACCESS_KEY"); - -String GRID_URL = "https://" + username + ":" + authkey + "@hub.lambdatest.com/wd/hub"; - -SafariOptions browserOptions = new SafariOptions(); -browserOptions.setPlatformName("MacOS Tahoe"); -browserOptions.setBrowserVersion("26"); -HashMap ltOptions = new HashMap(); -ltOptions.put("username", ""); -ltOptions.put("accessKey", ""); -ltOptions.put("project", "Text Validation Test"); -ltOptions.put("build", "Text Validation Test Build"); -ltOptions.put("w3c", true); -browserOptions.setCapability("LT:Options", ltOptions); - -WebDriver driver = new RemoteWebDriver(new URL(GRID_URL), browserOptions); - -try { - -driver.get("https://ecommerce-playground.lambdatest.io/"); - -String expectedText = "This is a dummy website for Web Automation Testing"; -boolean isTextPresent = driver.getPageSource().contains(expectedText); - -if (isTextPresent) { -((JavascriptExecutor) driver).executeScript("lambda-status=passed"); -System.out.println("✔ Text validation PASSED"); -} else { -((JavascriptExecutor) driver).executeScript("lambda-status=failed"); -System.out.println("✘ Text validation FAILED"); -} - -} catch (Exception e) { -((JavascriptExecutor) driver).executeScript("lambda-status=pass"); -e.printStackTrace(); -} finally { -driver.quit(); // 🔹 Correctly placed – runs even if test fails -} -} -} -``` - -```java -// TextValidationTest.java – Selenium 3 Configuration -import org.openqa.selenium.WebDriver; -import org.openqa.selenium.remote.DesiredCapabilities; -import org.openqa.selenium.remote.RemoteWebDriver; -import org.openqa.selenium.JavascriptExecutor; - -import java.net.URL; -import java.util.HashMap; - -public class TextValidationTest { - -public static void main(String[] args) throws Exception { - -String username = System.getenv("LT_USERNAME") == null ? -"Your LT Username" : System.getenv("LT_USERNAME"); - -String authkey = System.getenv("LT_ACCESS_KEY") == null ? -"Your LT AccessKey" : System.getenv("LT_ACCESS_KEY"); - -String GRID_URL = "https://" + username + ":" + authkey + "@hub.lambdatest.com/wd/hub"; - -DesiredCapabilities capabilities = new DesiredCapabilities(); -capabilities.setCapability("browserName", "Safari"); -capabilities.setCapability("browserVersion", "26"); -HashMap ltOptions = new HashMap(); -ltOptions.put("username", ""); -ltOptions.put("accessKey", ""); -ltOptions.put("platformName", "MacOS Tahoe"); -ltOptions.put("visual", true); -ltOptions.put("video", true); -capabilities.setCapability("LT:Options", ltOptions); - -WebDriver driver = new RemoteWebDriver(new URL(GRID_URL), capabilities); - -try { - -driver.get("https://ecommerce-playground.lambdatest.io/"); - -String expectedText = "This is a dummy website for Web Automation Testing"; -boolean isTextPresent = driver.getPageSource().contains(expectedText); - -if (isTextPresent) { -((JavascriptExecutor) driver).executeScript("lambda-status=passed"); -System.out.println("✔ Text validation PASSED"); -} else { -((JavascriptExecutor) driver).executeScript("lambda-status=failed"); -System.out.println("✘ Text validation FAILED"); -} - -} catch (Exception e) { -((JavascriptExecutor) driver).executeScript("lambda-status=pass"); -e.printStackTrace(); -} finally { -driver.quit(); // 🔹 Correctly placed – runs even if test fails -} -} -} -``` - -**Result** - -Visit TestMu AI Web Automation dashboard to view your test execution result. - -## Contact Us for Support - -If you come across any challenges while migrating or need help at any stage, feel free to reach out to our support team. We are dedicated to ensuring a seamless transition to TestMu AI and are available around the clock to help you with any queries or issues. - -Get in touch with us through our support portal 💬 or community forums 👥. diff --git a/static/docs/migrate-selenium-test-suite.md b/static/docs/migrate-selenium-test-suite.md deleted file mode 100644 index 7feb1c414..000000000 --- a/static/docs/migrate-selenium-test-suite.md +++ /dev/null @@ -1,240 +0,0 @@ -# Migrate Your Selenium Test Suite - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Already have Selenium tests running locally? You can run them on the TestMu AI cloud grid with three changes: swap the driver URL, add your credentials, and set the desired capabilities. Your test logic stays the same. - -If you are moving from BrowserStack or Sauce Labs, use these dedicated migration guides with capability mapping tables. - -- [Migrate from BrowserStack to TestMu AI](/support/docs/browserstack-to-testmu-migration-guide/) - hub URL, capability mapping, and feature equivalents -- [Migrate from Sauce Labs to TestMu AI](/support/docs/saucelabs-to-testmu-migration-guide/) - hub URL, capability mapping, and feature equivalents - -## Prerequisites - -Make sure you have the following set up before you start. - -1. [Create a TestMu AI account](https://www.testmuai.com/register/) if you don't have one. -2. Get your **Username** and **Access Key** from the [TestMu AI Dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard). -3. An existing Selenium test suite in any supported language (Java, JavaScript, Python, C#, PHP, or Ruby). - -## Step 1: Set Your Credentials - -Add your TestMu AI credentials as environment variables so your tests can authenticate with the grid. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -## Step 2: Replace Your Local Driver With RemoteWebDriver - -Point your tests to the TestMu AI hub instead of launching a local browser. - -Find where your test creates the WebDriver instance and replace it with a `RemoteWebDriver` pointing to the TestMu AI hub URL: - -``` -https://{YOUR_USERNAME}:{YOUR_ACCESS_KEY}@hub.lambdatest.com/wd/hub -``` - -Here is what the change looks like in each language: - -**Before (local):** -```java -WebDriver driver = new ChromeDriver(); -``` - -**After (cloud):** -```java -String username = System.getenv("LT_USERNAME"); -String accessKey = System.getenv("LT_ACCESS_KEY"); -String hubURL = "https://" + username + ":" + accessKey + "@hub.lambdatest.com/wd/hub"; - -ChromeOptions browserOptions = new ChromeOptions(); -browserOptions.setPlatformName("Windows 10"); -browserOptions.setBrowserVersion("latest"); - -HashMap ltOptions = new HashMap(); -ltOptions.put("build", "My First Cloud Build"); -ltOptions.put("name", "Sample Test"); -ltOptions.put("w3c", true); -browserOptions.setCapability("LT:Options", ltOptions); - -WebDriver driver = new RemoteWebDriver(new URL(hubURL), browserOptions); -``` - -**Before (local):** -```javascript -const driver = new Builder().forBrowser('chrome').build(); -``` - -**After (cloud):** -```javascript -const username = process.env.LT_USERNAME; -const accessKey = process.env.LT_ACCESS_KEY; -const hubURL = `https://${username}:${accessKey}@hub.lambdatest.com/wd/hub`; - -const capabilities = { -browserName: 'Chrome', -browserVersion: 'latest', -'LT:Options': { -platformName: 'Windows 10', -build: 'My First Cloud Build', -name: 'Sample Test', -w3c: true, -} -}; - -const driver = new Builder() -.usingServer(hubURL) -.withCapabilities(capabilities) -.build(); -``` - -**Before (local):** -```python -driver = webdriver.Chrome() -``` - -**After (cloud):** -```python -import os -from selenium import webdriver - -username = os.getenv("LT_USERNAME") -access_key = os.getenv("LT_ACCESS_KEY") -hub_url = f"https://{username}:{access_key}@hub.lambdatest.com/wd/hub" - -options = webdriver.ChromeOptions() -options.platform_name = "Windows 10" -options.browser_version = "latest" - -lt_options = { -"build": "My First Cloud Build", -"name": "Sample Test", -"w3c": True, -} -options.set_capability("LT:Options", lt_options) - -driver = webdriver.Remote(command_executor=hub_url, options=options) -``` - -**Before (local):** -```csharp -IWebDriver driver = new ChromeDriver(); -``` - -**After (cloud):** -```csharp -string username = Environment.GetEnvironmentVariable("LT_USERNAME"); -string accessKey = Environment.GetEnvironmentVariable("LT_ACCESS_KEY"); -string hubURL = $"https://{username}:{accessKey}@hub.lambdatest.com/wd/hub"; - -var options = new ChromeOptions(); -options.PlatformName = "Windows 10"; -options.BrowserVersion = "latest"; - -var ltOptions = new Dictionary -{ -{ "build", "My First Cloud Build" }, -{ "name", "Sample Test" }, -{ "w3c", true } -}; -options.AddAdditionalOption("LT:Options", ltOptions); - -IWebDriver driver = new RemoteWebDriver(new Uri(hubURL), options); -``` - -**Before (local):** -```php -$driver = ChromeDriver::start(); -``` - -**After (cloud):** -```php -$username = getenv("LT_USERNAME"); -$accessKey = getenv("LT_ACCESS_KEY"); -$hubURL = "https://{$username}:{$accessKey}@hub.lambdatest.com/wd/hub"; - -$capabilities = [ -"browserName" => "Chrome", -"browserVersion" => "latest", -"LT:Options" => [ -"platformName" => "Windows 10", -"build" => "My First Cloud Build", -"name" => "Sample Test", -"w3c" => true, -] -]; - -$driver = RemoteWebDriver::create($hubURL, $capabilities); -``` - -**Before (local):** -```ruby -driver = Selenium::WebDriver.for :chrome -``` - -**After (cloud):** -```ruby -username = ENV["LT_USERNAME"] -access_key = ENV["LT_ACCESS_KEY"] -hub_url = "https://#{username}:#{access_key}@hub.lambdatest.com/wd/hub" - -options = Selenium::WebDriver::Options.chrome( -platform_name: "Windows 10", -browser_version: "latest", -"LT:Options": { -build: "My First Cloud Build", -name: "Sample Test", -w3c: true, -} -) - -driver = Selenium::WebDriver.for :remote, url: hub_url, capabilities: options -``` - -Use the [Capabilities Generator](https://www.testmuai.com/capabilities-generator/) to auto-generate the capabilities code for any browser, version, and OS combination. - -## Step 3: Run Your Tests - -Execute your tests the same way you normally would. The only difference is they now run on the cloud. - -```bash -# Java (Maven) -mvn test - -# JavaScript (npm) -npm test - -# Python (pytest) -pytest - -# C# (dotnet) -dotnet test - -# PHP (PHPUnit) -vendor/bin/phpunit - -# Ruby (RSpec) -bundle exec rspec -``` - -## Step 4: View Your Results - -Check the Automation Dashboard to see exactly what happened during your test. - -Visit the [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) to see your results. Each session captures video playback, screenshots, console logs, network logs, and Selenium command logs. - -## What Stays the Same - -Everything except the driver setup. Here is what does not change when you move to the cloud. - -| What | Changes? | -|------|----------| -| Test logic (assertions, flows, waits) | No | -| Page Object Models | No | -| Test framework config (TestNG XML, pytest.ini, etc.) | No | -| CI/CD pipeline commands | No | -| Driver setup (URL + capabilities) | **Yes** | -| Local browser install requirement | **Removed** | diff --git a/static/docs/milestone-creation-and-management.md b/static/docs/milestone-creation-and-management.md deleted file mode 100644 index 1154c10bd..000000000 --- a/static/docs/milestone-creation-and-management.md +++ /dev/null @@ -1,45 +0,0 @@ -# Milestones - Creation and Management - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Milestones in TestMu AI serve as a central organizational tool for grouping test runs. They are particularly useful for tracking progress related to feature releases, ensuring comprehensive oversight of associated testing activities. - -This guide provides a comprehensive walkthrough for creating and managing milestones within TestMu AI, enabling efficient organization of test runs and tracking feature releases. - -## Creating a Milestone -### Step 1: Access the Milestone Section -- Navigate to the Milestone section in the TestMu AI dashboard. -- Click **"Create Milestone**" and locate the Create Milestone button and click it to start the creation process. - -### Step 2: Provide the required configurations -- Enter the following mandatory fields: - - **Milestone Name :** Provide a clear and descriptive name for the milestone. - - **Associated Test Runs:** Select or specify the test runs you want to include. - - Optional fields: - - **Description :** Add additional information about the milestone’s purpose or scope. - - **Tags :** Assign relevant tags for categorization. - - **Owner :** Designate a team member responsible for the milestone. - - **Duration :** Set the expected duration for completing the milestone. - - **Attachments :** Upload any relevant files or documentation. - -### Step 3: Finalize the Milestone -- Review all entered details for accuracy. -- Click **Create Milestone** to finalize the creation. - -## Managing Milestones -### Viewing Milestone Details -Once created, all milestones are listed for easy tracking. You can: -- Select a specific milestone to view its detailed information. -- Access associated test runs and other milestone attributes. - -### Marking Milestones as Completed -When a milestone is finished, you can mark it as Completed: -- Navigate to the desired milestone. -- Use the Mark as Completed option to update its status. - -### Viewing Completed Milestones -To review completed milestones: -- Navigate to the View Completed section in the milestone management area. -- This section provides a history of all milestones marked as complete. - -## Video Explanation diff --git a/static/docs/miro-integration.md b/static/docs/miro-integration.md deleted file mode 100644 index a32d4b30a..000000000 --- a/static/docs/miro-integration.md +++ /dev/null @@ -1,57 +0,0 @@ -# Miro Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Miro is a cloud-based collaborative whiteboard platform helping teams to manage their projects effectively. Miro gives the team access to digital whiteboards and sticky notes to enable them to brainstorm and do many other collaborative activities such as research, planning, managing workflows and more. - -TestMu AI's integration with Miro will help you to log bugs to boards on Miro. You can easily find and report any cross browser compatibility issues to your team instantly. - -## How To Integrate Miro With Your TestMu AI Account? - -**Step 1:** Login to your TestMu AI account. Select 'integration' from the left navigation bar. This page includes all the integrations provided with 3rd party tools for TestMu AI. You'll find Miro under the Project Management category. Hover on it and you'll see a plus icon, click on it go further with integration - -**Step 2:** If you've already logged into your Miro account, you'll be taken to another screen asking to install 'TestMu AI' in your team. You can click on the 'install' icon next to 'test'. - -**Step 3:** After you click on the install button, the integration of TestMu AI will be completed. You can verify the integration by selecting 'integrations' from the left navigation bar, here under the 'My Integration' section you can see 'Miro'. - -**Step 4:** If you are not already logged in, then you will be redirected to a page where you can log in with your Miro account by providing login credentials. You can even use your google account to login to Miro. - -## Log Your First Bug From TestMu AI Integration With Miro - -**Step 1:** Choose the type of test from the left navigation menu. For now, let's go with the "Real-Time" Test" - -**Step 2:** Enter your URL in the "Place your URL" dialog box. Choose the browser and OS Configuration you want to test on and click on 'Start'. - -**Step 3:** After the Virtual Machine is created and launched, you can perform manual tests on your Web-Application in the live environment to find bugs. When a bug is identified, click on the 'Mark as Bug' icon on the left panel to capture the screenshot. - -**Step 4:** After capturing the screenshot, you can mark the issues identified on the page with the help of the in-built image editor. After highlighting the bug, click on the "Mark as Bug" button. - -**Step 5:** A Miro tracker form would pop up to log the issue, you can fill the fields as per the case and click on 'create issue'. - -**Step 6:** The logged task can now be seen as a part of your board in your Miro Project. You can access your board in the dashboard under 'All boards'. - -**Step 7:** After logging into your board, you can see your task created on the board. - -**Step 8:** Upon clicking on the task, you will be able to view all the details you entered while logging the bug on TestMu AI. You'll be able to see the screenshot of the bug along with the configuration you tested on - -## Remove TestMu AI Integration With Miro - -### On TestMu AI's Platform - -In case you want to remove your Miro integration with TestMu AI, Here is how you can do that. - -**Step 1:** Log in to your TestMu AI account. - -**Step 2:** From the left navigation menu click on Integration. Under the My Integration section, you can see a remove button next to your Miro integration. By clicking on the remove button you can remove the integration. - ->**Note**: While removing the integration make sure you do the same from Miro so that you'll be able to integrate TestMu AI with Miro again. - -### On Miro's Platform - -**Step 1:** Log in to your Miro account. Click on the profile icon on the top left corner and click on settings - -**Step 2:** Click on Select 'Apps and Integrations' from the left navigation bar. Further, scroll down to "Installed Apps" and click on the TestMu AI icon. - -**Step 3:** Under Apps & Integrations, scroll down and you'll see the user history for the installation. Next to it you'll see the option 'uninstall for me'. - -That was all you need to know for integrating TestMu AI with Miro. In case you have any doubts or questions, feel free to share them with our product experts on **24/7 chat support**. Happy testing! 🙂 diff --git a/static/docs/mobile-web-automation-on-real-devices.md b/static/docs/mobile-web-automation-on-real-devices.md deleted file mode 100644 index 0bb2ccd82..000000000 --- a/static/docs/mobile-web-automation-on-real-devices.md +++ /dev/null @@ -1,106 +0,0 @@ -# Mobile Web Automation Testing on Real Devices - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Mobile web automation involves testing websites or web apps on real mobile devices like Android and iOS. Testing appearance and performance on a real device ensures that the test runs on real-world conditions, delivers better performance, and improves test coverage across devices. - - TestMu AI allows you to perform mobile web automation via Appium on a wide range of real Android and iOS devices. You can [test on mobile devices](https://www.testmuai.com/test-on-mobile-devices/) from OEMs like Samsung, Apple, Xiaomi, and more. - -## Prerequisites - -Install Python and the Appium client library before running mobile web tests. - -1. Download the latest version of [Python](https://www.python.org/downloads/). -2. Install the Appium dependency by running the following command from the CMD terminal. - -``` -pip install Appium-Python-Client -``` - -The above code installs the latest Python client library for Appium which only supports `w3c : True` capability. - -## Performing Mobile Web Automation on Real Devices - -Pass the isRealMobile capability in your test scripts to run on a real device. - -To test on a real mobile device, pass the below capability in your test scripts. - -`'isRealMobile': True` - -You can also add the capability using the [Desired Capabilities Generator](https://www.testmuai.com/capabilities-generator/). - -In the below Python test script, we take an example of our sample ToDo app. We check two checkboxes of the list and add another item to the list. The test runs on iPhone 11 with platform version iOS 13. - -```python -//todo.py - -#!/usr/local/bin/python -# -*- coding: utf-8 -*- -import time -from appium import webdriver - -desired_cap = { -'w3c': True, -'platformName': 'iOS', -'platformVersion': '13', -'deviceName':"iPhone 11", -'nativeWebScreenshot':True, -'build': 'Demo', -'isRealMobile' : True,// real mobile device capability -'console': True, -'visual': True, -'network': True, -'tunnel' : False, -"newCommandTimeout": 300 -} - -url = "https://LT_USERNAME:LT_ACCESS_KEY@mobile-hub.lambdatest.com/wd/hub" -driver = webdriver.Remote(desired_capabilities = desired_cap, command_executor = url) - -driver.get("https://lambdatest.github.io/sample-todo-app/") -time.sleep(2) - -list1 = driver.find_element_by_xpath("//input[@name='li1']") -list1.click() -time.sleep(1) - -list2 = driver.find_element_by_xpath("//input[@name='li2']") -list2.click() -time.sleep(1) - -inputfield = driver.find_element_by_id("sampletodotext") -inputfield.click() -inputfield.send_keys("item 1") -driver.hide_keyboard() -time.sleep(1) - -driver.execute_script("document.getElementById('addbutton').click();") -time.sleep(1) - -inputfield = driver.find_element_by_id("sampletodotext") -inputfield.click() -inputfield.send_keys("item 2") -driver.hide_keyboard() -time.sleep(1) - -driver.execute_script("document.getElementById('addbutton').click();") -time.sleep(5) - -driver.quit() -``` - -In the below test snippet, add your TestMu AI Username and TestMu AI Access Key. Get them from the [TestMu AI dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard). In the Hub URL, we added a beta prefix (`mobile-hub.lambdatest.com/wd/hub`). - -``` -url = "https://LT_USERNAME:LT_ACCESS_KEY@mobile-hub.lambdatest.com/wd/hub" -``` - -Execute the below command in the terminal to run the test. - -``` -python todo.py -``` - -Visit the [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/) to find the mobile web automation test details along with the `REAL DEVICE (BETA)` label as highlighted below. - ->That’s all! you have successfully executed the mobile web automation test on real devices. In case you have any questions or need any additional information, you could reach out at our **24X7 Chat Support** or mail us directly at support@testmuai.com. diff --git a/static/docs/mocha-on-hyperexecute.md b/static/docs/mocha-on-hyperexecute.md deleted file mode 100644 index 0939f5423..000000000 --- a/static/docs/mocha-on-hyperexecute.md +++ /dev/null @@ -1,219 +0,0 @@ -# Running Mocha Framework Tests on HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Mocha is a versatile JavaScript testing framework often used in conjunction with Puppeteer for testing web applications. With support for asynchronous testing and various reporting options, Mocha enables the creation of robust test suites. When combined with Puppeteer, Mocha facilitates end-to-end testing of web pages by allowing developers to control browser interactions. - -HyperExecute is an AI Native Test Orchestration Cloud Platform that empowers you to run **end-to-end** tests **quickly** and **efficiently**. It provides Just-in-Time (JIT) testing infrastructure with fast execution **speeds**, **smart orchestration**, and **detailed logs**. - -This guide details how to execute your **Mocha** framework tests on **HyperExecute** via two different methods: - -- [**Using Local System**](/support/docs/mocha-on-hyperexecute/#1-testing-using-local-system) - You can use your own local machine to execute tests. -- [**Using Gitpod Platform**](/support/docs/mocha-on-hyperexecute/#2-testing-using-gitpod) - Execute tests using GitPod. (Requires a [Gitpod](https://gitpod.io/login/) account) - -## 1. Testing Using Local System - -Follow the step-by-step guide to execute your test on HyperExecute. - -### Prerequisites - -To run the Tests on HyperExecute from your Local System, you are required: - -- Your TestMu AI [Username and Access key](/support/docs/hyperexecute-how-to-get-my-username-and-access-key/) -- [HyperExecute YAML](/support/docs/hyperexecute-yaml-version0.2/) file which contains all the necessary instructions. -- [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) in order to initiate a test execution Job . -- Setup the [Environmental Variable](/support/docs/hyperexecute-environment-variable-setup/) - -### Step 1: Download the Sample Repository - -**Sample repo** - -Download or Clone the code sample for the JUnit from the TestMu AI GitHub repository to run the tests on the HyperExecute. - - View on GitHub - -If you are using your own project, make sure you update the **Hub endpoint** in your tests file. - -By setting up the Hub endpoint, you establish the communication channel between your tests and the browser nodes, enabling effective test distribution and execution. - -Configure the desired capabilities based on your test requirements. For example: - -```bash -exports.capabilities = { -"browserName": "Chrome", -"browserVersion": "122.0", -"LT:Options": { -"username": "YOUR_USERNAME", -"accessKey": "YOUR_ACCESS_KEY", -"platformName": "Windows 10", -"project": "Untitled", -"w3c": true, -"plugin": "node_js-mocha" -} -}; -``` -> You can also use **dev** and **beta** browser versions. To generate capabilities for your test requirements, you can use our inbuilt 🔗 Capabilities Generator. - -### Step 2: Setup the CLI in your Test Suite - -After cloning / downloading the sample repo, you need to setup the CLI and the environment variables. - -#### Download the HyperExecute CLI - -The CLI is used for triggering the tests on HyperExecute. It is recommend to download the CLI binary on the host system and keep it in the root directory of the suite to perform the tests on HyperExecute. - -You can download the CLI for your desired platform from the below mentioned links: - -| Platform | HyperExecute CLI | -| ---------| ---------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| MacOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -#### Setup Environment Variable - -Now, you need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in the [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). - -Run the below mentioned commands in your terminal to setup the CLI and the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -### Step 3: Configure YAML in your Test Suite - -Configure your YAML file as per your use cases using **key value** pairs. - -In this sample YAML file, we have mentioned: - -- **version** of the YAML file -- **Timeouts** for executing your project -- **Mode of execution** is [Autosplit](/support/docs/hyperexecute-auto-split-strategy/). You can also opt for [Matrix](/support/docs/hyperexecute-matrix-multiplexing-strategy/) or [Hybrid](/support/docs/hyperexecute-hybrid-strategy/) mode. -- **Pre and Post** commands -- **Reports and Artefacts** that will be generated after the completion of tests -- and other necessary YAML Parameters - -```yaml ---- -version: 0.1 - -runson: linux - -globalTimeout: 90 -testSuiteTimeout: 90 -testSuiteStep: 90 - -autosplit: true - -retryOnFailure: true -maxRetries: 1 - -concurrency: 2 - -pre: -- npm install - -cacheKey: '{{ checksum "package.json" }}' -cacheDirectories: -- node_modules - -testDiscovery: -type: raw -mode: remote -command: grep -lr 'describe' specs/*.spec.js - -testRunnerCommand: npm test $test - -jobLabel: [puppeteer-Mocha, linux, autosplit] -``` - -### Step 4: Execute your Test Suite - -> **NOTE :** In case of MacOS, if you get a permission denied warning while executing CLI, simply run **`chmod u+x ./hyperexecute`** to allow permission. In case you get a security popup, allow it from your **System Preferences** → **Security & Privacy** → **General tab**. - -Run the below command in your terminal at the root folder of the project: - -```bash -./hyperexecute --config RELATIVE_PATH_OF_YOUR_YAML_FILE -``` - -OR use this command if you have not exported your username and access key in the step 2. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE `} - -### Step 5: Monitor the Test Execution - -Visit the [HyperExecute Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) and check your Job status. - -### Step 6: Download Artifacts and Reports - -HyperExecute also facilitates the provision to download the [Artifacts](/support/docs/hyperexecute-artifacts/) and [Reports](/support/docs/hyperexecute-reports/) on your local machine. Click on the corresponding button to download your generated artifacts and reports. - -## 2. Testing Using Gitpod - -You can also use the Gitpod platform to execute our sample repository. It will fetch all the sample codebases and trigger the CLI to execute the tests. - -Follow the below steps to run Test using Gitpod: - -**Step 1:** Click '**Open in Gitpod**' button. You will be redirected to Login/Signup page. This button is configured to redirect you to the Gitpod platform where you will be able to execute our sample repository. - -**Step 2:** Login with TestMu AI credentials. Once logged in, a pop-up confirmation will appear, asking you to **'Proceed'** to the Gitpod editor in a new tab. The current tab will display the HyperExecute Dashboard. - -**Step 3:** Choose your preferred editor (we recommend VS Code Editor) - -**Step 4:** As you are running a sample project, Fetching of the Test Scripts, [HyperExecute YAML](/support/docs/deep-dive-into-hyperexecute-yaml/), [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) and Triggering your tests using the `Execution Command` will be automated. - -**Step 5:** Once you see the `Job Link` in the logs, you can visit the [HyperExecute dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) to see the tests getting executed. - -You can also implement [Secret Keys](/support/docs/hyperexecute-how-to-save-and-manage-secrets/) in your YAML file. - -## Navigation in Automation Dashboard - -Every test run on the HyperExecute has a unique *jobId* associated with it. Each *jobId* can in turn constitute single (or multiple) *groupId*(s). You can visit [HyperExecute Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) for checking the status of the test execution. - -You can seamlessly navigate between JobId's and taskId's. You need to click on the *testID* to navigate from the HyperExecute logs to the Automation Dashboard. - -The snapshot below shows the videos, logs and other meta data for that specific *test_ID* - -> For any query or doubt, please feel free to contact us via **24×7 chat support** or you can also drop a mail to **support@testmuai.com**. -Happy testing! - -## Using the Mocha Agent Skill with TestMu AI - -The [mocha-skill](https://github.com/LambdaTest/agent-skills/tree/main/mocha-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The mocha-skill package includes: - -``` -mocha-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing Mocha Agent Skill - -Install a Mocha Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/mocha-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/mocha-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only mocha-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). diff --git a/static/docs/mock-video-injection-web-automation-selenium-chromium.md b/static/docs/mock-video-injection-web-automation-selenium-chromium.md deleted file mode 100644 index baf5cf4bf..000000000 --- a/static/docs/mock-video-injection-web-automation-selenium-chromium.md +++ /dev/null @@ -1,203 +0,0 @@ -# Mock Video Injection on Web Automation (Selenium/Chromium) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Inject a custom video file as a fake camera feed in Chromium-based Selenium tests on TestMu AI. This is useful for testing WebRTC, video conferencing, KYC or identity verification, and any flow that calls `getUserMedia()`. - -## How It Works - -Upload a video file, pre-load it onto the test VM, and pass Chrome flags to use it as a fake camera device. - -1. Upload your `.mjpeg` or `.y4m` video file to TestMu AI using the web automation user-files API. -2. Pre-load the file onto the test VM using the `lambda:userFiles` capability. -3. Pass Chrome flags to use the uploaded file as a fake camera device. - -Chrome treats the file as a looping camera feed, so any `getUserMedia({ video: true })` call receives frames from your file instead of a real camera. - -## Prerequisites - -Verify the following before configuring mock video injection. - -1. Get a TestMu AI account with Web Automation access from [your dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard). -2. Prepare a `.mjpeg` video file such as `sample_640x360.mjpeg` (recommended resolution: `640x360` or `1280x720`; Chrome also supports `.y4m` format). -3. Set TestMu AI credentials as `LT_USERNAME` and `LT_ACCESS_KEY`. - -## Step 1: Upload the Video File - -Upload your `.mjpeg` file using the web automation user-files API. - -### cURL - -```bash -curl -X POST \ -"https://api.lambdatest.com/automation/api/v1/user-files" \ --u "$LT_USERNAME:$LT_ACCESS_KEY" \ --F "files=@/path/to/sample_640x360.mjpeg" -``` - -### Response - -```json -{ -"status": "success", -"data": [ -{ -"error": "", -"message": "File have been uploaded successfully to our lambda storage", -"key": "sample_640x360.mjpeg" -} -] -} -``` - -**Limits**: maximum 20 MB per upload and maximum 150 files per organization. - -The `/mfs/v1.0/media/upload` endpoint is for mobile and app automation. For web automation, use `/automation/api/v1/user-files`. - -## Step 2: Configure Selenium Capabilities - -Add the video file and Chrome flags to your Selenium capabilities configuration. - -### Java (Selenium 4, W3C) - -```java -import org.openqa.selenium.chrome.ChromeOptions; -import org.openqa.selenium.remote.RemoteWebDriver; -import java.net.URL; -import java.util.*; - -ChromeOptions options = new ChromeOptions(); - -options.addArguments( -"--use-fake-ui-for-media-stream", -"--use-fake-device-for-media-stream", -"--use-file-for-fake-video-capture=/home/ltuser/Downloads/sample_640x360.mjpeg" -); - -HashSet userFiles = new HashSet<>(); -userFiles.add("sample_640x360.mjpeg"); - -HashMap ltOptions = new HashMap<>(); -ltOptions.put("platform", "Linux"); -ltOptions.put("build", "Mock Video Injection Test"); -ltOptions.put("name", "Fake Camera Feed Test"); -ltOptions.put("video", true); -ltOptions.put("w3c", true); - -options.setCapability("LT:Options", ltOptions); -options.setCapability("lambda:userFiles", userFiles); - -RemoteWebDriver driver = new RemoteWebDriver( -new URL("https://" + LT_USERNAME + ":" + LT_ACCESS_KEY + "@hub.lambdatest.com/wd/hub"), -options -); -``` - -### Python - -```python -from selenium import webdriver - -options = webdriver.ChromeOptions() -options.add_argument("--use-fake-ui-for-media-stream") -options.add_argument("--use-fake-device-for-media-stream") -options.add_argument("--use-file-for-fake-video-capture=/home/ltuser/Downloads/sample_640x360.mjpeg") - -lt_options = { -"platform": "Linux", -"build": "Mock Video Injection Test", -"name": "Fake Camera Feed Test", -"video": True, -"w3c": True, -} - -options.set_capability("LT:Options", lt_options) -options.set_capability("lambda:userFiles", ["sample_640x360.mjpeg"]) - -driver = webdriver.Remote( -command_executor=f"https://{LT_USERNAME}:{LT_ACCESS_KEY}@hub.lambdatest.com/wd/hub", -options=options, -) -``` - -### JavaScript (WebDriverIO or raw WebDriver) - -```javascript -const capabilities = { -browserName: "Chrome", -browserVersion: "latest", -platformName: "Linux", -"LT:Options": { -platform: "Linux", -build: "Mock Video Injection Test", -name: "Fake Camera Feed Test", -video: true, -w3c: true, -}, -"lambda:userFiles": ["sample_640x360.mjpeg"], -"goog:chromeOptions": { -args: [ -"--use-fake-ui-for-media-stream", -"--use-fake-device-for-media-stream", -"--use-file-for-fake-video-capture=/home/ltuser/Downloads/sample_640x360.mjpeg", -"--no-sandbox", -"--disable-gpu", -], -}, -}; -``` - -## Step 3: Verify the Video Feed in Your Test - -Navigate to a page that requests camera access and confirm the fake stream is active. - -After the session starts, navigate to a page that requests camera access and confirm the fake stream is being used. - -```java -driver.get("https://www.lambdatest.com/selenium-playground/webrtc-video"); - -// Or open your application's camera page -driver.get("https://your-app.com/video-call"); - -Boolean isPlaying = (Boolean) driver.executeScript( -"const video = document.querySelector('video');" + -"return video && !video.paused && video.readyState >= 2;" -); -``` - -## File Paths by OS - -The `lambda:userFiles` capability places files in these directories. - -| Platform | File Path | -|---|---| -| Linux | `/home/ltuser/Downloads/sample_640x360.mjpeg` | -| Windows | `C:\\Users\\ltuser\\Downloads\\sample_640x360.mjpeg` | -| macOS | `/Users/ltuser/Downloads/sample_640x360.mjpeg` | - -Adjust the `--use-file-for-fake-video-capture` path to match your target platform. - -## Chrome Flags Reference - -The following Chrome flags control fake media device behavior. - -| Flag | Purpose | -|---|---| -| `--use-fake-ui-for-media-stream` | Auto-grants `getUserMedia()` permission without a popup | -| `--use-fake-device-for-media-stream` | Replaces real camera and microphone hardware with fake devices | -| `--use-file-for-fake-video-capture=` | Uses the specified file as the fake camera feed | -| `--use-file-for-fake-audio-capture=` | Uses the specified file as fake microphone input (`.wav` format) | - -## Troubleshooting - -Refer to the following table for common issues and solutions. - -| Issue | Solution | -|---|---| -| Black or empty video feed | Verify the file name in `lambda:userFiles` matches exactly, including case | -| Permission popup still appears | Confirm `--use-fake-ui-for-media-stream` is set | -| File not found on VM | Check the OS-specific path; if needed, use `driver.executeScript("lambda-files-download=sample_640x360.mjpeg")` as a fallback | -| Video not looping | Chrome loops `.mjpeg` files automatically; `.y4m` files also loop by default | -| Wrong resolution | Use a file resolution that matches your app expectations; `640x360` is a safe default | -| Windows path errors | Use escaped backslashes: `C:\\Users\\ltuser\\Downloads\\sample_640x360.mjpeg` | -| macOS video does not render | This is a known Chrome limitation; use Linux when visual verification is required | diff --git a/static/docs/modules-in-manual-testcases.md b/static/docs/modules-in-manual-testcases.md deleted file mode 100644 index dff5e7fa7..000000000 --- a/static/docs/modules-in-manual-testcases.md +++ /dev/null @@ -1,15 +0,0 @@ -# Importing Modules into Test Cases - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -To incorporate existing modules into your test steps, simply click on the Modules Icon. - -From there, you'll be able to select and import the specific module you need. - - Keep in mind that only modules already linked with your project will be available for import. - -Notice that imported modules will appear visually distinct from other individual steps within your test case, making them easy to identify. - -You also have the flexibility to edit or delete these modules directly from this view. However, be aware that making changes will create a new version of that module. - -Please note that when a module is updated to a new version, its existing occurrences within your test cases will not be affected. They will remain linked to the previous version. To utilize the latest version of a module, you'll need to manually sync it by clicking the `Sync to latest` button. This allows you to review and confirm changes before they impact your test cases. diff --git a/static/docs/monday-com-integration.md b/static/docs/monday-com-integration.md deleted file mode 100644 index ed9518083..000000000 --- a/static/docs/monday-com-integration.md +++ /dev/null @@ -1,88 +0,0 @@ -# monday.com Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -monday.com is an online project management tool that helps your team to collaborate better by getting everyone on the same page. monday.com provides an intuitive timeline to keep all your deadlines in check. Monitor all the tasks of your colleagues and subordinates through dashboard representation to know who is working on what, and when? You find a collective space for all the communication, so you don't get stuck in finding important emails from your inbox. Develop your projects in a flexible manner using monday.com's intuitive platform and pace up your go-to-market launch with enhanced collaboration and transparency among the team members. - -The TestMu AI + monday.com integration helps you in filing issues to your project in monday.com directly from the TestMu AI platform. With one-click integration, you can push annotated issues to the project board of your choice, assign it to the required teammate, and attach necessary screenshots. You can do all that while in the middle of a test session on the TestMu AI platform. The fields populated by you when marking as a bug through TestMu AI are displayed as information on the respective monday.com testing instance. - -**monday.com Integration with TestMu AI, like all of the integrations to 3rd party applications, is available for freemium as well as premium plan.** - -> -**Note:** Before proceeding with Integration, make sure your monday.com project board has a **text field** on it. If the text field is not available in your monday.com project board then the environment details that auto-populate by TestMu AI will be reflected in the title field on your monday.com project. - -## How To Establish Integration With monday.com From Your TestMu AI Account? - -**Step 1:** Login to your TestMu AI account. You would need to have Admin or User level access to see and install integrations. - -**Step 2:** Select **'Integrations'** from the left navigation menu bar. This will guide you to a screen where you will find a list of 3rd party applications, available to integrate with your TestMu AI account. - -**Step 3:** Click on **‘ADD’** icon under the block that says **'MONDAY'**. - -> -**Note:** If you already have your TestMu AI account integrated with some other project management tool then you would receive a message asking you to uninstall your current integration. Clicking on **'OK'** would automatically uninstall your current integration to establish a new one with monday.com. - -**Step 4:** Once you click on install, you will be redirected to the below screen. You will be asked to enter your **'Site Address'** and **'Personal API Token'** in the highlighted fields to establish integration with TestMu AI. - -> -* **What is the Site Address?** - -When you open any web page, the URL showing in the address window of your web browser is termed as site address. The site address consists of **https://** followed by the domain name of your website. - -> -* **What is your Personal API token?** - -When two different applications interact with each other, server-to-server communication is established. The API token is needed for authenticating user identity from one server to another, by fetching user-owned resources available on the server from where the communication is initiated. API token has been a necessity for maintaining the security and data integrity of any application. They are just as vital to an end-user as they are to an application. API tokens are relevant if an end-user logs in an application through a two-step verification process. - -**Step 5:** You can find your API token from your respective **monday.com** instance. If you have not generated one before, you can generate an API token by going to the settings from the bottom left corner, near your **avtar image**. After you click on your avatar image, a menu would open up, navigate to **'Admin'** section. - -**Step 6:** You will be directed to a Admin page where you need to click on **'API'** from the left. - -**Step 7:**Now click on **'Generate'** button to generate your Personal API Token. - -**Step 8:**Once your Personal API token is generated, click on **'Copy'** button to copy the same. - -> -**Note:**You can keep a note of this API token with you for future authentication. Although, in case you forget to do so, or you end up neglecting it due to some reason. You can create a new token by pressing the **'Regenerate'** button. - -**Step 8:** Paste the **'Site Address'** and **'Personal API Token'** into the field provided at TestMu AI & press the **'Install'** button. You are all set to experience one-click bug logging to share your issues directly from your TestMu AI account to your project board on monday.com. - -That's it! Go to **'Integrations'** again and you will be able to notice a **'green tick'** indicating that monday.com is successfully installed. - -## How To Log Your First Bug Through monday.com Integration? - -**Step 1:** Go for any of the test from the TestMu AI’s left navigation menu. For demo, we will be taking **‘Real Time Testing’** option. After that, enter a URL of the web-app you need to test in the dialog box. Then, select any configuration for browser and operating system of your choice & click on **‘Start’** button. - -**Step 2:** After the VM is launched and operable. You can perform testing on your web-app for finding bugs. If a bug gets revealed, then you need to click on the **'Bug icon'** from the left panel for capturing a screenshot of the same. We have highlighted that controller button in the below image. - -**Step 3:** After a screenshot is captured, you can annotate any issue or a task with an **in-built image editor**. Once you are done highlighting the bug, click on the button that says **'Mark as Bug'**. - -**Step 4:** After clicking on **'Mark as Bug'** button a monday.com specific form would open up. Fill the fields as per your requirement. - -> -* You can select which **'Board'** should the task go under. -* You can specify the sprint number for that task by filling the **'Group'** field. -* You can choose the **‘Person’** to whom you would assign your task. -* You can mention the **'Title'** as well to easily organize your bugs. -* You can also select the **'Status'** of the task you log. -* You can relate the issue's cause in the **'Text'** field. - -**Step 5:** Click on **'Create Issue'** & observe it being successfully marked through a single click effort. Once you click the button, you will get **prompt messages** on top of your Virtual Machine indicating the progress of bug logging. You will get a prompt message in few seconds mentioning **'Bug Marked Successfully'** indicating that the issue has been successfully pushed to the respective workspace. - -**Step 6:** Visit your **monday.com** dashboard. You will be able to notice the logged issue right away by going to the particular project board! All the data you provided through TestMu AI would already be presented in it. TestMu AI automatically includes test environment details and related screenshots in the custom **text column** that you have on your monday.com project board. If you don't have a text column in your project board then these details would be merged with your title. - -## How To Remove monday.com Integration? - -> -You can work with one integration at a time. So if you would want to integrate to a similar 3rd party application, then you would have to **remove** your current integration. Here is how you can do that. - -**Step 1:** Login to your TestMu AI account. - -**Step 2:** Select **‘Settings’** from the left navigation menu bar & click on **'Integrations'**. This will guide you to a screen where you will find 3rd party applications, available to integrate with your TestMu AI account. - -**Step 3:** Click on **‘REMOVE’**. You can find the remove button right under the **'MONDAY'** block which would be highlighted with a green tick. - -You will get a prompt message in few seconds mentioning **'You have successfully uninstalled Monday'** indicating that the monday.com has been successfully removed from your TestMu AI integrations. - -> -That was all you need to know for TestMu AI + monday.com Integration. Increase your productivity with our integrations. If you still have any questions for us, please feel free to let us know. Our experts are always **available on chat** to help you out with any roadblock regarding our product. Happy testing! diff --git a/static/docs/mstest-with-selenium-running-mstest-automation-scripts-on-testmu-selenium-grid.md b/static/docs/mstest-with-selenium-running-mstest-automation-scripts-on-testmu-selenium-grid.md deleted file mode 100644 index dcddb1ac6..000000000 --- a/static/docs/mstest-with-selenium-running-mstest-automation-scripts-on-testmu-selenium-grid.md +++ /dev/null @@ -1,109 +0,0 @@ -# Selenium With MSTest - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Run MSTest tests on the TestMu AI cloud grid. This guide covers setup, running a sample test, configuring capabilities, and testing locally hosted pages. - -**Sample repo** -All the code used in this guide is available in the sample repository. - - View on GitHub - -## Prerequisites - -Complete the following steps before running MSTest Selenium tests. - -1. Download and install **Selenium WebDriver** from the [official website](https://www.selenium.dev/downloads/). -2. Install the latest version of C#. -3. Install the **.Net** framework for developing applications using C#. -4. Download [Selenium WebDriver Language Binding](https://www.selenium.dev/downloads/) for C# and extract them to the appropriate folder. -5. Install a [.NET Core SDK](https://dotnet.microsoft.com/en-us/download) of 3.0.0. -6. Get your TestMu AI Username and Access Key from the [TestMu AI Dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard). - -## Step 1: Clone the Sample Project - -Clone the repository and navigate to the project directory. - -```bash -git clone https://github.com/LambdaTest/MSTest-Selenium-Sample -cd MSTest-Selenium-Sample -``` - -## Step 2: Set Your Credentials - -Configure your credentials to connect to the TestMu AI Selenium Grid. - -Set TestMu AI Username and Access Key in environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -## Step 3: Configure Your Test Capabilities - -Define browser, version, and OS settings for your test run. - -In the test script, update your test capabilities. This code passes browser, browser version, and operating system information, along with TestMu AI Selenium grid capabilities via the capabilities object. - -```csharp -var browserOptions = new ChromeOptions(); -browserOptions.PlatformName = "Windows 10"; -browserOptions.BrowserVersion = "latest"; - -var ltOptions = new Dictionary -{ -{ "build", "MSTest Build" }, -{ "name", "MSTest Test" }, -{ "w3c", true } -}; -browserOptions.AddAdditionalOption("LT:Options", ltOptions); -``` - -**Capabilities Generator** -Use the TestMu AI [Capabilities Generator](https://www.testmuai.com/capabilities-generator/) to auto-generate the capabilities class for your test requirements. - -## Step 4: Run the Test - -Execute the MSTest test from Visual Studio or the command line. - -**In Visual Studio (Windows):** - -1. Build the solution by clicking **Build > Build Solution**. -2. Open **Test Explorer** in Visual Studio. - -3. Click **Run** from the Test Explorer to run the sample test. - -**In Linux/macOS:** -```csharp -dotnet test MS-Test-Cross-Browser.csproj -``` - -## Step 5: View Your Results - -Check the test output on the console and the TestMu AI dashboard. - -Visit the [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/) to view your test results. The dashboard provides: - -- Text logs for each test step -- Screenshots captured during execution -- Video recordings of the full test session - -## Run MSTest Tests Using Agent Skills - -Use AI coding assistants to generate and run MSTest tests with the TestMu AI Agent Skill. - -The [mstest-skill](https://github.com/LambdaTest/agent-skills/tree/main/mstest-skill) is part of [TestMu AI Agent Skills](https://github.com/LambdaTest/agent-skills/) - structured packages that teach AI coding assistants how to write production-grade test automation. - -Install the skill: - -```bash -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/mstest-skill .claude/skills/ - -# For Cursor / Copilot -cp -r agent-skills/mstest-skill .cursor/skills/ -``` - -Install all available framework skills at once by cloning the repository directly into your tool's skills directory (e.g., `.claude/skills/`, `.cursor/skills/`). diff --git a/static/docs/multi-device-testing.md b/static/docs/multi-device-testing.md deleted file mode 100644 index 3c6de3a10..000000000 --- a/static/docs/multi-device-testing.md +++ /dev/null @@ -1,42 +0,0 @@ -# Multi-Device Testing on Real Devices - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -**Multi-Device Testing** enables you to run multiple devices simultaneously in a single App Live session, allowing you to validate your application across different devices and OS versions faster and more efficiently. This approach simplifies **cross-device testing** by bringing multiple devices into a single session. - -TestMu AI allows you to run **up to 6 devices at a time** within a single session, depending on your plan’s parallel limit. You can choose different applications for each device or test multiple versions of the same app concurrently, all from a single interface, without the need to start separate sessions. - -## Use Cases - -- **Cross-Device Functional Testing**: Validate app functionality across multiple devices simultaneously. -- **UI Consistency Check**: Ensure your app displays correctly on different screen sizes and resolutions. -- **Multi-App Workflow Testing**: Test interactions between multiple apps in a single session. -- **OS Version Verification**: Verify app behavior across different OS versions. -- **Faster Regression Testing**: Reduce testing time by running multiple devices together. - -## Multi-Device Testing Workflow - -### Step 1: Open a Real Device Session - -Log in to your [App Live Dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard), upload your app, and select a **real device** (Android or iOS). - -### Step 2: Add Devices Using Multi-Device Option - -Once the session is open, click on the **add device** option to add more devices. You can set **up to 6 devices** per session and select the **device and application** for each. - -### Step 3: Apply Multi Device Control Settings - -Once devices have been added to the session, you will see an option called **Multi-Device Control**. This feature allows you to apply common actions and settings across all connected devices simultaneously, reducing repetitive steps and ensuring consistent configurations. - -**Supported Multi-Device Controls** - - | **Feature** | **Description** | - | -----------------------| ---------------------------| - |Home | Instantly navigate all devices to the Home screen with a single click | - |Record Session| Start recording on all devices together, instead of initiating recordings individually| - |IP Geolocation| Apply a unified IP geolocation to every device with one click| - |GPS Location| Set a common GPS location across all devices simultaneously | - |Network Throttling|Apply the same network throttling profile to all devices without configuring each device separately| - |Settings|Update global settings such as idle timeout, language and timezone for all devices at once| - -- Currently, **Record Session** downloads are not compatible with QuickTime Player. If you are using macOS, you may need an alternative media player to view the recordings. diff --git a/static/docs/multi-factor-authentication.md b/static/docs/multi-factor-authentication.md deleted file mode 100644 index 6c0173f14..000000000 --- a/static/docs/multi-factor-authentication.md +++ /dev/null @@ -1,73 +0,0 @@ -# Multi-Factor Authentication (MFA) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -**Multi-Factor Authentication (MFA)** adds an extra verification step at login on top of your password. After entering your password, you also provide a code from an authenticator app (such as Authy, Google Authenticator, or Microsoft Authenticator) or a biometric factor. This significantly reduces the risk of unauthorized access, even if your password is compromised. - -All steps below assume you are signed in to your TestMu AI account. If you don't have an account, register for free. - -## Enable User MFA - -**Step 1:** From the **Profile** menu, select **Account Settings**. - -**Step 2:** Open the **Passwords & Security** tab and click **Setup** next to **Authenticator App**. - -**Step 3:** Scan the generated **QR code** with your preferred authenticator app (such as Authy, Google Authenticator, or Microsoft Authenticator), and enter the code shown in the app. - -**Step 4:** Copy or download the **recovery code**, select **I have written down this security code**, and click **Submit**. - -You will need the recovery code if you lose access to your authenticator app or delete it by mistake. Save it somewhere secure. - -You have successfully enabled User MFA on your TestMu AI account. - -## Disable User MFA - -**Step 1:** From the **Profile** menu, select **Account Settings**. - -**Step 2:** Open the **Passwords & Security** tab and click **Disable** next to **Authenticator App**. - -**Step 3:** Enter your account password to verify your identity. - -**Step 4:** Enter the **six-digit authentication code** generated by your authenticator app. - -MFA is now disabled on your account, and you will be redirected to the login page. - -## Enforce TestMu AI MFA for Your Organization - -**Step 1:** Navigate to **Settings** and select **Organization Settings** from the dropdown. - -**Step 2:** Open the **Authentication and SSO** tab and **toggle on** the **Require MFA** option. - -Users who have not set up MFA will be logged out immediately and asked to set up MFA the next time they sign in. - -**Step 3:** Log in again to set up MFA. - -**Step 4:** Click **Set Up MFA**. - -**Step 5:** Scan the generated **QR code** with your preferred authenticator app, enter the verification code shown in the app, and click **Submit**. - -**Step 6:** Copy or download the **recovery code**, select **I have written down this security code**, and click **Submit**. - -You will need the recovery code if you lose access to your authenticator app or delete it by mistake. Save it somewhere secure. - -You have successfully set up MFA for your TestMu AI account. - -**Step 7:** Sign in again. Enter the newly generated **authentication code** from your authenticator app and click **Verify**. - -## Disable Enforced MFA for Your Organization - -**Step 1:** Navigate to **Settings** and select **Organization Settings** from the dropdown. - -**Step 2:** Open the **Authentication and SSO** tab and **toggle off** the **Require MFA** option. - -## Disable TestMu AI MFA for a User - -**Step 1:** Navigate to **Settings** and select **Organization Settings** from the dropdown. - -**Step 2:** Open the **Members** tab. - -**Step 3:** Click the **three-dot (⋮) menu** next to the user's name and select **Disable MFA**. - -You have successfully disabled MFA for the user. - -> If you have any questions, please reach out via our **24/7 chat support** or email us at [support@testmuai.com](mailto:support@testmuai.com). diff --git a/static/docs/n8n-integration.md b/static/docs/n8n-integration.md deleted file mode 100644 index 68c1899b9..000000000 --- a/static/docs/n8n-integration.md +++ /dev/null @@ -1,81 +0,0 @@ -# How to Integrate TestMu AI With n8n - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -If you use n8n to automate workflows between apps, you can give those workflows access to a real, cloud-hosted browser by adding the TestMu AI Agent node. The node connects your workflows to 3,000+ browser and OS environments, with no local driver setup or infrastructure to manage. It is available in n8n's built-in verified nodes panel and works in any workflow alongside n8n's 1,000+ built-in integrations. - -## Prerequisites - -Before you install the node, confirm you have the following in place. - -1. A TestMu AI account. [Sign up for a free TestMu AI account](https://www.testmuai.com/register/) if you do not have one. -2. An n8n account on n8n Cloud or a self-hosted n8n instance running n8n 1.0 or later. -3. Admin access on the n8n instance, or permission from your instance admin to install nodes. -4. Before installing, check the **TestMu AI Agent** listing in the [n8n integrations directory](https://www.n8n.io/integrations/) for the current version number and any reported compatibility notes. - -## How to Add TestMu AI Credentials to n8n - -The TestMu AI Agent node uses your TestMu AI username and access key to authenticate each browser session. Retrieve both values from your account before you configure the node in n8n. - -**Step 1:** Sign into your TestMu AI account. Open the **Profile** dropdown in the top-right corner and click **Account Settings** > **Password & Security**. Your **Username** and **Access Key** are listed on that page. Copy both values. - -**Step 2:** In n8n, open the **TestMu AI Agent** node settings. Click the **Credential** dropdown and select **Create credential**. A credential modal opens. - -**Step 3:** Paste your **Username** into the **Username** field and your **Access Key** into the **Access Key** field and click **Save** - -n8n displays a green checkmark next to the credential name if the values are valid. - -Keep your Access Key private. Anyone who has it can run billable browser sessions on your TestMu AI account. If you suspect the key has been exposed, regenerate it from **Account Settings** immediately. - -## How to Install the TestMu AI Agent Node in n8n - -Installing the node once makes it available to every workflow on the instance. You do not need to reinstall it per workflow. - -**Step 1:** Sign in to n8n on n8n Cloud or your self-hosted instance. - -**Step 2:** Open the workflow editor and click **+** in the top-left corner to open the **Nodes panel**. - -**Step 3:** Search for **TestMu AI** in the **Nodes panel**. Select **TestMu AI Agent** from the results. - -If the node does not appear in search results, confirm you are searching in the built-in verified nodes panel. The TestMu AI Agent node is a verified node and does not require community node access to install. - -**Step 4:** Click **Install node** on the confirmation prompt. n8n downloads the node and makes it available to all workflows on the instance. - -**Step 5:** The **TestMu AI Agent** node now appears in the **Nodes panel**, ready to add to any workflow on the instance. - -## How to Build Your First Workflow With TestMu AI and n8n - -**Step 1:** Open the workflow editor. Click **+** in the **Nodes panel** and add a **Manual Trigger** node to the canvas. - -**Step 2:** Click **+** after the trigger node, search for **TestMu AI**, and add the **TestMu AI Agent** node to the canvas. - - When added, it will look like this: - -**Step 3**: Click the **TestMu AI Agent** node to open its settings panel. Fill in the following fields: - -- **Action:** Select `navigate` to open a URL in the browser. -- **URL:** Enter the absolute URL to navigate to, for example `https://www.testmuai.com/`. -- **Browser Name:** Select `Chrome`. -- **Platform Name:** Select `Windows 11`. -- **Browser Version:** Enter `latest` or specify a version number. -- **Region:** Select `us` or your preferred region. - -Now click **Execute step** to run the workflow. TestMu AI opens a real browser in the cloud and navigates to the specified URL. Check the node output to see the session ID, dashboard URL, and page snapshot. - -**Step 4**: Visit the TestMu AI Web Automation dashboard to view your test results. - -Running n8n on a local or self-hosted instance? The [`n8n-nodes-browsercloud`](https://www.npmjs.com/package/n8n-nodes-browsercloud) community package gives you two additional nodes: a browser automation node and a script runner that runs test scripts directly on TestMu AI infrastructure. - -Both use the browsercloud SDK, which executes faster and makes fewer server calls than the verified agent node. Install it via **Settings > Community Nodes** by entering the package name `n8n-nodes-browsercloud`. These nodes are not available on n8n Cloud. - -## How to Uninstall the TestMu AI Agent Node - -Uninstalling removes the node from the entire n8n instance, not just the current workflow. Every workflow that uses it will fail until the node is reinstalled. - -**Step 1:** In n8n, open **Settings** from the left sidebar and click **Community Nodes**. - -**Step 2:** Locate **TestMu AI Agent** in the installed node list and click **Uninstall package**. - -**Step 3:** Click **Confirm uninstall** next to the node. n8n removes it from the instance and disables it in all existing workflows immediately. - -Pause or deactivate any active workflows that depend on the TestMu AI Agent node before uninstalling. The uninstall takes effect immediately across the entire instance. diff --git a/static/docs/native-app-automation-mcp.md b/static/docs/native-app-automation-mcp.md deleted file mode 100644 index d4d6ca016..000000000 --- a/static/docs/native-app-automation-mcp.md +++ /dev/null @@ -1,275 +0,0 @@ -# Native App Automation with MCP - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Two different MCP servers cover native mobile app workflows on TestMu AI, and they do very different jobs. Pick the right one before you start, because the setup, transport, and tools are not interchangeable. - -| You want to... | Use this server | What it does | -| -------------------------------------------------------------------------------------------------- | ---------------------------------- | ------------------------------------------------------------------- | -| Triage or debug an app-automation test that **already ran** on TestMu AI (pull failure reason, command logs) | **TestMu AI MCP Server** (Automation tool) | Reads test execution data from the cloud. No live device control. | -| **Drive a device or simulator live**: open apps, tap, type, screenshot, read on-screen text, generate tests | **Appium MCP server** | Controls a real or virtual device through Appium in real time. | - -If you are not sure which one you need, use this rule of thumb: if the test has a **TestID** on TestMu AI and you want to know why it failed, that is the TestMu AI MCP Server. If you want the agent to physically interact with an app on a device right now, that is the Appium MCP server. - -The rest of this page covers each server in turn, then a shared troubleshooting section drawn from real customer setups. - -## Part 1: Debug App Automation Tests with the TestMu AI MCP Server - -The [TestMu AI MCP Server](/support/docs/testmu-mcp-server/) connects your AI client to test execution data on TestMu AI. Its [Automation tool](/support/docs/automation-mcp-server/) now supports **app automation** tests in addition to web and desktop, so you can triage a failed mobile app run without leaving your IDE. - -### What you can do - -Point the Automation tool at a mobile app-automation TestID and ask it to: - -- Fetch the failure reason and root cause of a specific app-automation test. -- Retrieve command logs for a mobile session to see the exact step where the run failed. -- Retrieve network logs for a mobile session to inspect traffic during the run. -- Generate a fix for your test code based on the identified cause. - -Browser console logs apply to web tests only and are not available for mobile app automation tests. - -This is a **read and analyze** capability. The Automation tool does not open apps, tap buttons, or take live screenshots. For that, see [Part 2](#part-2-drive-a-live-device-with-the-appium-mcp-server). - -### Setup - -The Automation tool is part of the TestMu AI MCP Server, so setup is identical to any other TestMu AI MCP tool. Follow the client configuration steps in [Set Up TestMu AI MCP Server](/support/docs/testmu-mcp-server/), then confirm the OAuth flow completes. - -- **Server URL:** `https://mcp.lambdatest.com/mcp` -- **Authentication:** OAuth (you are redirected to testmuai.com on first use) - -No extra configuration is required to enable app-automation support. The same tools now recognize mobile app tests. - -### Specifying a mobile app test - -Web and mobile tests live behind different data endpoints on TestMu AI. To route your request correctly, the Automation tool needs to know the test type. Every Automation tool call carries a required `platform` argument, and your AI client must set it to `"app"` for a mobile app-automation TestID so the tool targets the mobile test-detail path rather than the web one. - -Most clients handle this for you: your AI client will either ask you to confirm whether the test is desktop or mobile, or infer it from the TestID and set `platform` on the tool call. When the tool runs, you will see arguments like this: - -```json -{ -"testId": "RMAA-IOS-585984-1767376885020815122FLT", -"platform": "app" -} -``` - -Command log and network log retrieval for app tests additionally takes the mobile `sessionId`, and command logs are paginated through a `pageNumber` argument. Your client fills these in from the test details. - -If your client does not prompt you, state the test type explicitly in your prompt, for example: "This is a mobile app automation test. Pull the failure reason for TestID ``." This avoids the tool defaulting to the web endpoint, which returns a **404 (Unable to find test details)** for a mobile TestID. See [Troubleshooting](#404-unable-to-find-test-details) if you hit this. - -### Example prompts - -- "This is a mobile app automation test. What is the failure reason for TestID ``?" -- "Pull the command logs for app-automation TestID `` and tell me which step failed." -- "Analyze the result for TestID `` and highlight the root cause." - -Your client routes the request to the Automation tool, pulls the relevant data from TestMu AI, and returns the analysis inline. A typical result looks like a root-cause summary, for example a `NoSuchElementException` where the test could not locate a UI element on the device screen, along with the point in the command log where it failed. - -**A note on large logs** -App-automation command logs can be very large. Asking the agent to dump every page of logs into its context can exceed the model's context window and cause the run to stall or error out. Ask for the **failure reason** or a **specific page** of command logs rather than the entire log history. See [Requests stall or hit a context-length error](#requests-stall-or-hit-a-context-length-error) for details and settings. - -## Part 2: Drive a Live Device with the Appium MCP Server - -The **Appium MCP server** (the open-source [`appium/appium-mcp`](https://github.com/appium/appium-mcp) project) gives an AI assistant real-time control of a mobile device or simulator through Appium. This is the server to use when you want the agent to open apps, tap buttons, enter text, take screenshots, read on-screen text, and generate test code from natural language. - -Unlike the TestMu AI MCP Server, the Appium MCP server runs **locally over stdio** and drives devices through your own Appium setup. It can target a local device or simulator by default, or a remote Appium endpoint such as a device farm (see [Targeting remote devices](#targeting-remote-devices-and-device-farms)). - -### What you can do - -Through natural language, the agent can: - -- Launch, install, terminate, and query native apps by bundle ID (iOS) or package name (Android). -- Find elements using accessibility id, resource-id, platform-native predicates, or xpath. -- Perform gestures: tap, double tap, long press, scroll, swipe, and drag and drop. -- Enter text, read element text, and capture the page source. -- Take screenshots and record the screen. -- Read device state: orientation, geolocation, battery, clipboard, and device info. -- Generate Java/TestNG test code from a described scenario. - -### Prerequisites - -Because the Appium MCP server drives devices through your local toolchain, you need a working mobile automation environment: - -- **Node.js** v22 or higher, and npm or yarn. -- **Java Development Kit (JDK)** 8 or higher. -- **Android SDK** with `ANDROID_HOME` set (for Android testing). -- **Xcode** and command line tools (for iOS testing, macOS only). - -For Android, enable USB debugging and install the UiAutomator2 driver dependencies. For iOS, install simulators through Xcode. For real device testing, enable Developer Mode and sign in to your Apple ID in Xcode. - -After the server is configured, you can ask your AI assistant to set up the remaining Appium driver prerequisites for you using the built-in `appium_skills` tool. - -### Installation - -The standard configuration works in most MCP clients. Add the following to your client's MCP configuration, updating `ANDROID_HOME` to match your Android SDK path: - -```json -{ -"mcpServers": { -"appium-mcp": { -"disabled": false, -"timeout": 100, -"type": "stdio", -"command": "npx", -"args": ["appium-mcp@latest"], -"env": { -"ANDROID_HOME": "/path/to/android/sdk", -"CAPABILITIES_CONFIG": "/path/to/your/capabilities.json" -} -} -} -} -``` - -**Claude Code CLI:** - -```bash -claude mcp add appium-mcp -- npx -y appium-mcp@latest -``` - -**Gemini CLI:** - -```bash -gemini mcp add appium-mcp npx -y appium-mcp@latest -``` - -Remember to set `ANDROID_HOME` in the resulting configuration for Android testing. - -### Capabilities file - -Create a `capabilities.json` file to define per-platform device capabilities, and point the `CAPABILITIES_CONFIG` environment variable at it: - -```json -{ -"android": { -"appium:app": "/path/to/your/android/app.apk", -"appium:deviceName": "Android Device", -"appium:platformVersion": "11.0", -"appium:automationName": "UiAutomator2", -"appium:udid": "your-device-udid" -}, -"ios": { -"appium:app": "/path/to/your/ios/app.ipa", -"appium:deviceName": "iPhone 15 Pro", -"appium:platformVersion": "17.0", -"appium:automationName": "XCUITest", -"appium:udid": "your-device-udid" -} -} -``` - -### Core tools - -The server exposes a broad tool set. The ones you will use most for native app automation: - -| Tool | Purpose | -| ----------------------------- | ------------------------------------------------------------------------------------------- | -| `select_device` | **Required first.** Discover available devices and select one (auto-selects if only one). | -| `appium_session_management` | Create, attach, list, select, or delete sessions. Supports remote Appium via `remoteServerUrl`. | -| `appium_app_lifecycle` | Launch, install, terminate, uninstall, background, deep-link, or query the state of an app. | -| `appium_find_element` | Find an element using accessibility id, id, platform-native predicates, or xpath. | -| `appium_gesture` | Tap, double tap, long press, scroll, swipe, or scroll-to-element. | -| `appium_set_value` | Enter text into an input field. | -| `appium_get_text` | Read text content from an element. | -| `appium_screenshot` | Capture a screenshot (full screen or a specific element). | -| `appium_get_page_source` | Retrieve the current screen's XML source. | -| `appium_generate_tests` | Generate automated test code from a natural-language scenario. | - -For the full catalog, including orientation, geolocation, clipboard, permissions, alerts, and screen recording, see the [`appium/appium-mcp` README](https://github.com/appium/appium-mcp/blob/main/README.md). - -### Element finding: prefer stable locators - -For reliable, fast automation, ask the agent to locate elements by stable identifier in priority order: `accessibility id` first, then `id`, then platform-native predicates (`-ios predicate string` or `-ios class chain` on iOS, `-android uiautomator` on Android). Use `xpath` only as a last resort, since it is slower and more brittle. - -Vision-based element finding (locating an element from a natural-language visual description) is available as an opt-in tool, `appium_ai`, gated behind the `AI_VISION_ENABLED` environment variable. Leave it off unless you specifically need to locate elements by visual cue, as it is slower and calls a paid vision model. - -### Targeting remote devices and device farms - -By default the Appium MCP server drives a **local** device or simulator. To run against a remote Appium endpoint, pass `remoteServerUrl` plus explicit `capabilities` (device udid, app path or id, platform version) on `appium_session_management` with `action=create`. Prefer this over local device discovery for grids, cloud labs, and CI, where the target device is allocated rather than plugged in locally. - -For security, you can restrict which remote servers are allowed with the `REMOTE_SERVER_URL_ALLOW_REGEX` environment variable. - -### Example prompt - -``` -Open the Amazon app, search for "iPhone 15 Pro", select the first result, -add it to the cart, and take a screenshot of the cart screen. -``` - -The agent selects a device, launches the app, finds and interacts with elements, and captures the screenshot, without you writing any Appium code. - -### Performance: NO_UI mode - -Screenshots and page-source dumps embedded in tool responses can be large and expensive in tokens. For CI, batch runs, or any headless scenario, set the `NO_UI` environment variable to `true` to return lightweight text-only responses. Screenshots are still saved to disk. The base64 payload is simply omitted from the response. Keep the UI enabled for interactive debugging. - -## Troubleshooting - -### 404: Unable to find test details - -**Symptom:** The Automation tool returns `Error fetching automation test details: Unable to find test details (Status: 404)` for a TestID you know is valid. - -**Cause:** The request is being routed to the web test-detail endpoint, but the TestID belongs to a **mobile app** test, which lives behind a different endpoint. - -**Fix:** Make the test type explicit. Tell your client the test is a mobile app automation test, or confirm the desktop-vs-mobile prompt when your client asks. The tool then targets the correct mobile path. The `platform` argument on the tool call should read `"app"` for mobile app tests. - -Also confirm that the authenticated MCP user actually has access to that TestID. A 404 is also returned when the TestID belongs to a different user or organization than the one authenticated on the MCP server. - -### The MCP tool times out - -**Symptom:** Tool calls time out repeatedly, often on `getAutomationTestDetails`, while other MCP servers work fine. - -**Causes and fixes:** - -- **Cline client issue.** Some Cline updates broke direct connections. Use the `mcp-remote` transport and raise the timeout in your Cline configuration: - -```json -{ -"mcp-lambdatest": { -"autoApprove": [], -"disabled": false, -"timeout": 120, -"command": "npx", -"args": ["mcp-remote@latest", "https://mcp.lambdatest.com/mcp"], -"alwaysAllow": [] -} -} -``` - -- **Large test payload.** A TestID with a very large volume of data can take longer than the default timeout to retrieve. Increasing the client timeout to `120` resolves most of these cases. - -- **Confirm scope.** Check whether the timeout happens on **every** tool or only one. A timeout isolated to a single tool points to payload size or that specific endpoint, not a connectivity or auth problem. - -### Requests stall or hit a context-length error - -**Symptom:** A run gets stuck partway, often on the last page of a paginated log retrieval, or the client shows an error such as: input tokens longer than the model's context length. - -**Cause:** App-automation command logs and device logs can be enormous. When the agent tries to pull every page of logs into a single model context, it exceeds the context window (for example, a 315k-token log against a ~197k-token limit) and the run fails or stalls. - -**Fixes:** - -- **Ask for the failure reason, not the full log.** Prompt specifically: "Highlight the failure reason for TestID ``," rather than "analyze all logs." This lets the tool return the relevant slice instead of the entire history. -- **Request a specific page** of command logs rather than iterating through all of them. -- **Use a model with a larger context window** in your client if you must analyze large logs. -- **For the Appium MCP server**, enable `NO_UI` mode to strip large base64 and XML payloads out of responses, which cuts per-request token usage substantially. - -### OAuth flow does not open or fails - -- Confirm pop-ups are not blocked in your default browser. -- If a previous connection has expired, clear the authentication in your client's MCP settings and reconnect. -- Verify your TestMu AI account has permissions for the workspace you are targeting. - -### Tools not appearing after configuration - -- Fully restart your client (quit and reopen, not just a window reload). -- For Continue, confirm you are in Agent Mode. -- Validate your configuration file for syntax errors, especially trailing commas and unescaped paths. - -### Command not found: npx - -Your MCP client cannot find Node.js on its PATH. Find the full path with `which npx` (macOS/Linux) or `where npx` (Windows), then replace `"command": "npx"` in your configuration with that full path. - -## Which server, again? - -- **TestMu AI MCP Server (Automation tool):** cloud test triage and RCA for app-automation runs that already executed. Remote, OAuth, no local setup. Reads data. Does not touch a device. -- **Appium MCP server:** live control of a real or virtual device. Local stdio, needs an Appium toolchain. Opens apps, taps, types, screenshots, and reads the screen. - -Use them together: drive an exploratory flow with the Appium MCP server, and when a scheduled TestID fails on TestMu AI, triage it with the Automation tool. diff --git a/static/docs/native-extent-report.md b/static/docs/native-extent-report.md deleted file mode 100644 index a9a23de45..000000000 --- a/static/docs/native-extent-report.md +++ /dev/null @@ -1,67 +0,0 @@ -# Native Extent Report - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The Extent Native Reports offer a standardized and easily accessible summary of information extracted from raw Extent reports per Virtual Machine (VM) at the end of a HyperExecute job. - -### Prerequisites - -Before implementing Extent Native Reports, ensure the following prerequisites are met: - -1. Upgrade to extent reporting version 5 in the `pom.xml` file. -2. Update import statements in the codebase from `com.relevantcodes` (version 2) to `com.aventstack` (version 5). - -## Implementation Steps - -Follow these steps to enable Extent Native Reports for your HyperExecute job: - -### 1. Upgrade Extent Reporting Version - -Update the `pom.xml` file to include the latest version of the Extent Reporting library (version 5). Ensure that the necessary dependencies are correctly configured. - -```xml - -com.aventstack -extentreports -5.0.0 - -``` - -### 2. Modify Import Statements - -Update import statements in your codebase to reflect the new package structure in Extent Reporting version 5. Replace `com.relevantcodes` with `com.aventstack`. - -```java -// Before -import com.relevantcodes.extentreports.ExtentReports; -import com.relevantcodes.extentreports.ExtentTest; - -// After -import com.aventstack.extentreports.ExtentReports; -import com.aventstack.extentreports.ExtentTest; -``` - -### 3. Generate JSON Reports - -Make changes in your codebase to generate individual JSON reports. These reports will serve as the source for the Extent Native Reports. - -### 4. Update HyperExecute YAML Configuration - -In the HyperExecute YAML configuration, add the following section to instruct the HyperExecute systems to generate Extent Native Reports: - -```yaml -report: true -partialReports: -type: json -location: reports/json -frameworkName: extent-native -``` - -- Set `report: true` to enable report generation. -- Specify the `type` as `json` in the `partialReports` section. -- Define the `location` where the JSON reports will be stored. -- Provide a meaningful `frameworkName` for the Extent Native Reports. - -## Conclusion - -By following these steps, your HyperExecute job will generate Extent Native Reports, providing a consolidated HTML report derived from individual JSON reports. This enhancement allows customers to access comprehensive and standardized reports conveniently at the conclusion of their HyperExecute jobs. diff --git a/static/docs/netlify-integration-with-testmu.md b/static/docs/netlify-integration-with-testmu.md deleted file mode 100644 index f7bf9ccf1..000000000 --- a/static/docs/netlify-integration-with-testmu.md +++ /dev/null @@ -1,49 +0,0 @@ -# Netlify Integration with TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Netlify is a cloud computing company that offers hosting and serverless backend services for web applications and static websites. It allows developers to deploy their websites and applications quickly and easily, with features such as continuous deployment, serverless functions, and global CDN. - -## Prerequisite - -1. A TestMu AI account. If you don't have an account, [sign up for free](https://www.testmuai.com/register/?redirectTo=https://accounts.lambdatest.com/dashboard). - -2. You need to have a site deployed in your Netlify account. - -## Steps to Setup the Netlify Plugin - -### Step 1: Configure your TestMu AI Integration on Netlify Dashboard - -- Go to your Netlify Dashboard. - -- Click on the Integrations tab, search for **TestMu AI** and click on the **Enable** button. - -- You need to provide your configuration details - - - Enter your **Username** and **Access Key** from the **TestMu AI Dashboard**. - - - Enter your **Smart-UI Project Name** - -For the Smart-UI Project Name, you can enter any project name. Even if you have not set up any projects in the Smart UI Dashboard. - -You can also enter your already-existing smart-UI project name. - -- Click on the **Save** button - -### Step 2: Deploy the Site for which you want to compare the results - -- Go to the **Deploys** section, click on **Deploy site** button and deploy your project. - -- This deployment is before any changes you have made and will be your **Baseline** image. - -- Now you need to make the required changes in your project and push the changes. - -- Visit the Netlify Dashboard again, and re-deploy your project. - -Your changes will be triggered and deployed. - -### Step 4: Compare the Changes Smart-UI Dashboard - -- Go to your Smart-UI Dashbard. You will notice there is already a project with the same name you entered while configuring in **Step 1** - -- Go on that project. Click on the latest build. You can now compare the changes done to your project with-respect-to the provious build. diff --git a/static/docs/network-configurations-in-real-devices.md b/static/docs/network-configurations-in-real-devices.md deleted file mode 100644 index 958b05f20..000000000 --- a/static/docs/network-configurations-in-real-devices.md +++ /dev/null @@ -1,61 +0,0 @@ -# Network Logs and Configurations in real devices - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Network logs record every network interaction during your test session, from API calls and page requests to server responses and load times. These logs are stored in HAR format, giving you a complete snapshot of **network performance** for each run. By reviewing this data, you can identify slow endpoints, troubleshoot failed requests, and validate that your app communicates with the right services, all without leaving your testing workflow. - -On TestMu AI, you can enable **network configurations** to capture and analyze this traffic in real time on real devices. With flexible options such as content capture and domain-based filtering, you can focus on the most relevant network interactions while reducing noise from unrelated requests. This helps ensure faster debugging, clearer insights, and more efficient test runs. - -> To enable it for your organization, please contact us via **24×7 chat support** or you can also drop a mail to **support@testmuai.com**. - -### Use cases - -- **Debug API calls** by viewing HTTP/S request and response data directly from real device sessions. -- **Verify backend integration** by checking if calls are made to the correct endpoints. -- **Filter noise** by including/excluding specific hosts. -- **Reproduce production issues** that depend on specific network conditions or API behaviors. - -## Enabling Network Configuration - -### Step 1: Access App Testing - -Log into your TestMu AI dashboard and navigate to Real Devices > App Testing. - -### Step 2: Select your device or browser - -Choose a real Android or iOS device, upload your app (APK/IPA), or open a browser URL. - -### Step 3: Configure network log options - -Run your app. In the network logs for your device, open the **Configuration Options** slider and select the options you want to apply for your current session. - -### Configuration Options - -| Setting | Type | Default | Description | -|-----------------------------|------------------------------|----------|--------------------------------------------------------------------------------------------------------------| -| Capture Content | Toggle | Enabled | Captures the full response body of network requests in the DevTools Response tab. Disable if only headers or metadata are needed. | -| Include specific domains | Text input (comma-separated domains) | Empty | Capture network requests only for the specified domains. | -| Exclude specific domains | Text input (comma-separated domains) | Empty | Exclude network requests to the specified domains. | - -## Available configuration options - -### **Include specific domains**: - Use this to capture logs only for network requests to specified domains. Enable the toggle and add the domains separated by commas, as shown below. - -### **Capture content**: -Use this option to capture or exclude response content. This is off by default but can be enabled if needed, as shown below. - -### **Exclude specific domains**: -Use this to exclude certain domains from network logs. This works similarly to **Include specific domains**. If both options are enabled, **Exclude specific domains** takes precedence. - -## Best practices & tips - -- Exclude noisy domains to reduce clutter in your logs. -- Use the include list to focus on relevant domains. -- Avoid broad patterns that may capture excessive traffic. -- Save your settings for reuse in future sessions. - -- Available only on **Plus Plans** for manual testing. -- Apps from the App Store or Play Store may not support network capture. -- Certificate-pinned apps may fail unless pinned hosts are excluded. -:: diff --git a/static/docs/network-configurations.md b/static/docs/network-configurations.md deleted file mode 100644 index 927da9d1d..000000000 --- a/static/docs/network-configurations.md +++ /dev/null @@ -1,99 +0,0 @@ -# Network Logs and Configurations - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Network logs record every network interaction during your test session, from API calls and page requests to server responses and load times. These logs are stored in HAR format, giving you a complete snapshot of **network performance** for each run. By reviewing this data, you can identify slow endpoints, troubleshoot failed requests, and validate that your app communicates with the right services, all without leaving your testing workflow. - -On TestMu AI, you can enable **network configurations** to capture and analyze this traffic in real time on real devices. With flexible options such as content capture and domain-based filtering, you can focus on the most relevant network interactions while reducing noise from unrelated requests. This helps ensure faster debugging, clearer insights, and more efficient test runs. - -> To enable it for your organization, please contact us via **24×7 chat support** or you can also drop a mail to **support@testmuai.com**. - -## Use Cases - -- **Debug API calls** by viewing HTTP/S request and response data directly from real device sessions. -- **Verify backend integration** by checking if calls are made to the correct endpoints. -- **Filter noise** by including/excluding specific hosts. -- **Reproduce production issues** that depend on specific network conditions or API behaviors. - -## Capabilities - -| Capability | Type | Default | Description | -|-----------------------------|------------------------|---------|--------------------------------------------------------------| -| **network** | Boolean | true | Enables network logs for the session. | - -## Network Configuration Settings -To enable the following you would need to pass them under `networkLogsOptions` Capability. - -| Network Configurations Capability | Type | Default | Description | -|-----------------------------|------------------------|---------|--------------------------------------------------------------| -| **captureContent** | Boolean | true | Captures the response body in network logs. | -| **excludeHosts** | List | None | Hosts to exclude from network logs. | -| **includeHosts** | List | None | Only capture network logs for these hosts; all others will be excluded. | - -- `networkLogsExcludeHosts` and `networkLogsIncludeHosts` are mutually exclusive. If both are set, **IncludeHosts** takes precedence. - -## Managing Certificate Pinning in Network Logs - -Some mobile apps add an extra layer of security by using **certificate pinning**, a process where the app is hard-coded to trust only a specific certificate or public key for certain hosts. This means that whenever the app communicates with those hosts, it **verifies the server’s certificate** against its pinned copy before allowing the connection. - -When you enable **network logging** on TestMu AI, traffic passes through a **secure proxy** so requests and responses can be captured. For **certificate-pinned hosts**, this interception may cause the connection to fail because the certificate no longer matches exactly. To avoid such issues, you can exclude pinned hosts from being proxied by adding them to the `networkLogsExcludeHosts` capability. This ensures your tests run smoothly without breaking **secure connections**. - -## Network Configurations in Automation Test - -### Step 1: Upload Your App to TestMu AI - -Before enabling network configurations, ensure your app is uploaded to TestMu AI. - -1. **Uploading Your App** – Follow the detailed steps in our [Upload Your Application](/support/docs/upload-apps-on-real-device-cloud/) guide. -2. Once uploaded, **note the App ID** returned by the API or dashboard. -3. Use this **App ID** in the `"app"` capability in your automation script. - -### Step 2: Add Desired Capabilities - -{`desired_caps = { - "deviceName": "Galaxy S20", - "platformName": "Android", - "platformVersion": "15", - "isRealMobile": True, - "app": "YOUR_APP_ID", - "build": "Sample Build", - "name": "Sample Test", - "network": True, - #highlight-next-line - "networkLogsOptions": { - "captureContent": False, - "excludeHosts": ["lambdatest"], - "includeHosts": ["youtube", "facebook"] - }, -}`} - -{`desired_caps = { - "deviceName": "iPhone 16", - "platformName": "ios", - "platformVersion": "18", - "isRealMobile": True, - "app": "YOUR_APP_ID", - "build": "Sample Build", - "name": "Sample Test", - "network": True, - #highlight-next-line - "networkLogsOptions": { - "captureContent": False, - "excludeHosts": ["lambdatest"], - "includeHosts": ["youtube", "facebook"] - }, -}`} - -### Step 3: Run Your Test -Execute your test script using your preferred automation framework with the above capabilities. - -## Best Practices - -- **Exclude analytics/tracking domains** to reduce log noise. -- For **certificate-pinned apps**, always add pinned hosts to `networkLogsExcludeHosts`. -- Use **networkLogsIncludeHosts** for focused debugging when investigating specific endpoints. -- Keep `captureContent: false` if you only need headers for performance gains. - -- **Certificate-pinned apps** may fail if pinned hosts are not excluded. -- Available only on **Plus Plan**. -- Certain HTTPS connections (e.g., using custom certificate pinning) cannot be intercepted even with MITM. diff --git a/static/docs/network-data-masking.md b/static/docs/network-data-masking.md deleted file mode 100644 index 1a61a84d2..000000000 --- a/static/docs/network-data-masking.md +++ /dev/null @@ -1,134 +0,0 @@ -# Network Data Masking - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI provides network data masking capability to protect sensitive information during automation testing sessions. This feature automatically masks sensitive data in network logs on the TestMu AI platform, ensuring that confidential information like passwords, tokens, and authentication keys are not exposed in test logs and network traffic reports. - -By the end of this document, you will be able to: - -- Enable automatic masking of sensitive data in network logs -- Protect confidential information - -## What Is Network Data Masking? - -Network data masking automatically identifies and masks sensitive information in HTTP network logs on the TestMu AI platform during test execution. - -When enabled, the system scans for predefined sensitive keywords in network requests and responses, replacing their values with masked placeholders to prevent data exposure. - -## Sensitive Keywords Detected - -The system automatically masks values for keys containing the following keywords. - -| Keyword | Description | -|---------|-------------| -| key | API keys, encryption keys | -| password | User passwords, system passwords | -| token | Authentication tokens, access tokens | -| auth | Authorization headers, auth credentials | -| email | Email addresses in headers/cookies | -| cipher | Cipher keys, encrypted data | -| secret | Secret keys, client secrets | -| nonce | Cryptographic nonces | -| salt | Password salts, hash salts | - -## Configuration - -Set the `network.mask` capability to `true` to enable network data masking. - -### Java Implementation - -```java -// Java code for configuring Network Masking in tests on LambdaTest -DesiredCapabilities capabilities = new DesiredCapabilities(); -capabilities.setCapability("network.mask", "true"); -// Add other capabilities as needed -``` - -### Alternative Language Examples - -**Python:** - -```python -capabilities = { -"network.mask": "true" -# Add other capabilities as needed -} -``` - -**Node.js:** - -```javascript -const capabilities = { -"network.mask": "true" -// Add other capabilities as needed -}; -``` - -## Supported Values - -The capability accepts the following values. - -| Value | Description | -|-------|-------------| -| true | Enables network data masking for sensitive information | -| false (default) | Disables network data masking (default behavior) | - -## Masking Coverage - -Network data masking covers the following areas of HTTP traffic. - -### Request Headers - -- Authorization headers containing tokens or keys -- Custom headers with sensitive authentication data -- API key headers - -### Response Headers - -- Set-Cookie headers with sensitive session data -- Authentication response headers -- Custom security headers - -### Request/Response Cookies - -- Session cookies containing tokens -- Authentication cookies -- Security cookies with sensitive identifiers - -## Example Scenarios - -The following examples show how network logs appear before and after masking. - -### Before Masking (network.mask: false) - -``` -Request Headers: -Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9... -X-API-Key: sk_live_abcd1234567890 -User-Email: user@example.com - -Response Cookies: -auth_token=abc123xyz789; session_key=def456uvw012 -``` - -### After Masking (network.mask: true) - -``` -Request Headers: -Authorization: Bearer ******** -X-API-Key: ******** -User-Email: ******** - -Response Cookies: -auth_token=********; session_key=******** -``` - -## Use Cases - -Network data masking supports several testing and security scenarios. - -- **Security Compliance**: Ensure sensitive data does not appear in test logs for compliance with security standards and regulations. -- **Team Collaboration**: Share test results and network logs safely without exposing production credentials or sensitive user data. -- **CI/CD Pipeline Security**: Protect sensitive information in automated test reports that may be stored or shared across development teams. -- **Third-party Integration Testing**: Mask API keys and authentication tokens when testing integrations with external services. -- **Debug and Troubleshooting**: Review network traffic for debugging while keeping sensitive authentication data protected. diff --git a/static/docs/network-throttling-for-responsive-testing.md b/static/docs/network-throttling-for-responsive-testing.md deleted file mode 100644 index b1e650700..000000000 --- a/static/docs/network-throttling-for-responsive-testing.md +++ /dev/null @@ -1,22 +0,0 @@ -# Network Throttling To Test Website Responsiveness - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -[LT Browser](https://www.testmuai.com/lt-browser/) allows you to test your [mobile website view](https://www.testmuai.com/mobile-view-website) over different networks such as fast 3G, slow 3G, online, or even offline. With network throttling, you can change your network profile to test its functionality when exposed to different network environments. - -To get started, you need to download the executable file from here. - -Download LT Browser For Windows - -Download LT Browser For macOS - -Download LT Browser For Linux - -**Step 1:** To test your website over different network environments, click on the **'Network Throttling'** button. - -**Step 2:** For the demo, we will test the website over the slow 3G network. Select the Slow 3G option from the network list. - -Once you are done with selecting the network, you can test your responsive mobile view of your website over a slow 3G network. - -> -That's all folks!. In case you come across any doubts, feel free to reach out through our **24*7 Chat Support** or you can also drop a mail to [support@testmuai.com](mailto:support@testmuai.com).Happy Testing! diff --git a/static/docs/network-throttling.md b/static/docs/network-throttling.md deleted file mode 100644 index 06036827a..000000000 --- a/static/docs/network-throttling.md +++ /dev/null @@ -1,201 +0,0 @@ -# Network Throttling - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Test your website's functionality on low latency networks (2G/3G/LTE) or offline conditions. These networks have variable upload and download speeds that can alter how your website performs on different browsers. - -To validate your website on such network profiles, simulate these network conditions using capabilities. If you want the test suite to start with the default network, use the preset capabilities. The device will have unobstructed internet connectivity. - -| KEY | VALUES |CAPABILITY | -|-----|--------|-----------| -| networkThrottling | Regular 4G, Regular 3G, Regular 2G, Good 3G, Good 2G, Offline, Reset, GPRS, DSL | Based on the user-provided input, this capability starts the test suite with the specified default network. For example, if the user selects **Regular 4G**, the capability looks like this: ```capabilities.setCapability("networkThrottling", "Regular 4G");``` | - -```yaml -capabilities.setCapability("networkThrottling", "Regular 4G"); -``` - -## List of Network Profiles - -The following table lists all available preset network profiles with their speed and latency values. - -| CONDITION | MAX DOWNLOAD SPEED (KBPS) | MAX UPLOAD SPEED (KBPS) | LATENCY (MS) | -| ------------ | --------------------------- | ------------------------- | ------------------ | -| Offline | 0 | 0 | 0 | -| Reset | Reset to default | Reset to default | Reset to default | -| GPRS | 50 | 20 | 500 | -| Regular 2G | 250 | 50 | 300 | -| Good 2G | 450 | 150 | 150 | -| Regular 3G | 750 | 250 | 100 | -| Good 3G | 1 Mbps | 750 | 20 | -| Regular 4G | 4 Mbps | 3 Mbps | 20 | -| DSL | 2 Mbps | 1 Mbps | 5 | - -**Custom Network Profile**: Create custom network conditions using objects. Define the upload speed, max download speed, and latency for the custom condition, as shown in the table above. - -## Configuring Network Profile - -Use the Selenium JavaScript Executor to apply custom network throttling during tests. - -```java -// Using executeScript to apply custom network throttling -Map throttleParams = Map.of( -"download", 500, // Maximum download speed in kbps -"upload", 100, // Maximum upload speed in kbps -"latency", 30 // Latency in ms -); - -driver.executeScript("lambda-throttle-network", throttleParams); - -``` -TestMu AI allows you to select a network profile before running automation tests. This lets you conduct functional tests on low/high latency networks and offline. Use the networkProfile capability as shown below to simulate the network conditions. - -| JAVASCRIPT EXECUTOR COMMAND | REQUEST PARAMETERS | EXAMPLE | -| ----------------------- | ------------------------------------------ | -------------------- | -| networkProfile | condition: a string or object representing browser network conditions | driver.execute_script("networkProfile", {"condition": {"download": 500,"upload": 100,"latency": 30}}) | - -## Configuring Network Throttling in Test Automation - -Define network throttle capabilities in your automation scripts to configure network throttling. - -To configure network throttling in automation, use the [TestMu AI TestNG GitHub repository](https://github.com/LambdaTest/Java-TestNG-Selenium) to run automation tests. - -### Configuring Capabilities for Pre-defined Network Settings -```java -DesiredCapabilities caps = new DesiredCapabilities(); -caps.setCapability("browserName", "Chrome"); -caps.setCapability("build", "Demo-TestNG"); -caps.setCapability("name", "TestNG-Todo-Script-1"); -caps.setCapability("networkThrottling", "Regular 4G"); //Set Network Speed to Regular 4G -``` - -The following TestNG code validates your TestMu AI credentials for authentication. The code selects basic capabilities such as OS, browser, browser version, and network. - -### Configuring Custom Network Settings -```java -package com.lambdatest; - -import java.net.MalformedURLException; -import java.net.URL; - -import org.openqa.selenium.By; -import org.openqa.selenium.remote.DesiredCapabilities; -import org.openqa.selenium.remote.RemoteWebDriver; -import org.testng.Assert; -import org.testng.annotations.AfterSuite; -import org.testng.annotations.BeforeSuite; -import org.testng.annotations.Test; - -public class TestNGTodo1 { - -private RemoteWebDriver driver; -private String Status="failed"; - -@BeforeSuite -public void setup() throws MalformedURLException { -String username = System.getenv("LT_USERNAME"); -String authkey = System.getenv("LT_ACCESS_KEY"); -String hub = "@hub.lambdatest.com/wd/hub"; - -DesiredCapabilities caps = new DesiredCapabilities(); -caps.setCapability("browserName", "Chrome"); -caps.setCapability("build", "Demo-TestNG"); -caps.setCapability("name", "TestNG-Todo-Script-1"); -caps.setCapability("networkThrottling", true); //To enable network throttling - -driver = new RemoteWebDriver(new URL("https://" + username + ":" + authkey + hub), caps); - -// Custom network throttling using executeScript -Map throttleParams = new HashMap<>(); -throttleParams.put("download", 500); // Maximum download speed in kbps -throttleParams.put("upload", 100); // Maximum upload speed in kbps -throttleParams.put("latency", 30); // Latency in ms - -// Use executeScript with the provided payload -driver.executeScript("lambda-throttle-network", throttleParams); - -} - -@Test -public void basicTest() throws InterruptedException { -String spanText; -System.out.println("Loading Url"); -Thread.sleep(100); -driver.get("https://4dvanceboy.github.io/lambdatest/todo.html"); -Thread.sleep(100); - -System.out.println("Checking Box"); -driver.findElement(By.name("todo-1")).click(); -Thread.sleep(400); - -System.out.println("Checking Another Box"); -driver.findElement(By.name("todo-2")).click(); -Thread.sleep(400); - -System.out.println("Checking Box"); -driver.findElement(By.name("todo-3")).click(); -Thread.sleep(400); - -System.out.println("Checking Another Box"); -driver.findElement(By.name("todo-4")).click(); -Thread.sleep(400); - -driver.findElement(By.id("todotext")).sendKeys(" List Item 6"); -driver.findElement(By.id("addbutton")).click(); -Thread.sleep(200); - -driver.findElement(By.id("todotext")).sendKeys(" List Item 7"); -driver.findElement(By.id("addbutton")).click(); -Thread.sleep(200); - -driver.findElement(By.id("todotext")).sendKeys(" List Item 8"); -driver.findElement(By.id("addbutton")).click(); -Thread.sleep(200); - -System.out.println("Checking Another Box"); -driver.findElement(By.name("todo-1")).click(); -Thread.sleep(300); - -System.out.println("Checking Another Box"); -driver.findElement(By.name("todo-3")).click(); -Thread.sleep(300); - -System.out.println("Checking Another Box"); -driver.findElement(By.name("todo-7")).click(); -Thread.sleep(300); - -System.out.println("Checking Another Box"); -driver.findElement(By.name("todo-8")).click(); -Thread.sleep(300); - -System.out.println("Entering Text"); -driver.findElement(By.id("todotext")).sendKeys("Get Taste of Lambda and Stick to It"); -Thread.sleep(300); - -driver.findElement(By.id("addbutton")).click(); - -System.out.println("Checking Another Box"); -driver.findElement(By.name("todo-9")).click(); -Thread.sleep(300); -// Let's also assert that the todo we added is present in the list. - -spanText = driver.findElementByXPath("/html/body/div/div/div/ul/li[9]/span").getText(); -Assert.assertEquals("Get Taste of Lambda and Stick to It", spanText); -Status="passed"; -Thread.sleep(150); - -System.out.println("TestFinished"); - -} - -@AfterSuite -public void tearDown() { -driver.executeScript("lambda-status=" + Status); -driver.quit(); -} - -} -``` - - Find your defined network capabilities under the section 'Input Config' by navigating to the 'METADATA' section of your automation build-logs. - -> In case you have any questions, feel free to share them with us.Our experts are available on **24/7 Customer chat support**. You can also drop us a mail at support@testmuai.com. Happy testing! 🙂 diff --git a/static/docs/network-whitelisting-and-tunnel-guide.md b/static/docs/network-whitelisting-and-tunnel-guide.md deleted file mode 100644 index 7d5d19386..000000000 --- a/static/docs/network-whitelisting-and-tunnel-guide.md +++ /dev/null @@ -1,110 +0,0 @@ -# Network Whitelisting Guide - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -When testing private or restricted environments on TestMu AI, your team may need to configure network access controls. Depending on how your applications are hosted and secured, TestMu AI supports multiple approaches **Inbound Whitelisting, Dedicated Proxy, Outbound Whitelisting, STUN/TURN Whitelisting, and Tunnel configurations (including Site-to-Site tunnels)**. - -> These are Inbound and Outbound with respect to user's network. - -This guide helps you decide which option to use for your infrastructure setup. - -## 1. When Inbound Whitelisting Has to Be Done (Shared IP) -Inbound IP whitelisting allows TestMu AI cloud browsers and devices to directly access your IP-restricted web applications without using a tunnel. - -#### When to Use -- Your resources (e.g., staging sites, dashboards, APIs) are publicly accessible on the internet, but locked down to requests from trusted IPs. -- Your firewall or access control list (ACL) supports adding static IPs. -- You want to avoid maintaining tunnels for each test run. - -#### Example -- A staging environment at staging.company.com is accessible only from a set of static IPs. -- By whitelisting TestMu AI’s shared public IPs, your QA team can test directly on the cloud grid. - -> Refer to [TestMu AI Public IP](/support/docs/testmu-public-ip/) for list for the IP addresses. - -## 2. When Dedicated Proxy Whitelisting Has to Be Done -Dedicated Proxy Whitelisting provides an extra layer of security by assigning a unique static IP exclusively for your organization. - -#### When to Use -- You want isolation from other TestMu AI customers (no shared IPs). -- Your compliance or security policies mandate that only your company’s traffic be routed through a controlled IP. -- You want easier log tracking, auditing, or firewall rules with a single source IP. - -#### Example -- A banking app testing environment requires that only a customer-specific IP be allowed, not shared infrastructure IPs. -- You request a dedicated proxy IP from TestMu AI and whitelist only that. - -## 3. When Outbound Whitelisting Is Required - -Outbound whitelisting applies when your corporate network restricts users from accessing external domains. - -#### When to Use -- Your testers cannot reach lambdatest.com or related subdomains because of firewall/proxy rules. -- You need to ensure your employees’ browsers and TestMu AI integrations can connect to the platform. - -#### What to Whitelist -- `*.lambdatest.com` (all subdomains) -- Standard ports `80` and `443` - -#### Example -A QA engineer inside a corporate VPN is unable to resolve `hub.lambdatest.com`. Once `*.lambdatest.com` is whitelisted, they can execute tests without connection issues. - -## 4. When STUN/TURN Server Whitelisting Is Required -For real-time testing sessions (browser, VM, or real device), TestMu AI uses the WebRTC protocol to stream video. This requires access to STUN/TURN servers. - -#### When to Use -- You notice slowness in live sessions. -- VM/real device sessions are not launching. -- The video stream falls back to Guacamole (slower protocol) instead of WebRTC. - -#### What to Whitelist -Allow TLS traffic on ports 80 and 443 for the following servers: - -```bash -stun-us-1.lambdatest.com -stun-us-2.lambdatest.com -stun-eu-1.lambdatest.com -stun-eu-2.lambdatest.com -stun-as-1.lambdatest.com -stun-ap-2.lambdatest.com -``` - -#### Example -If a tester in Europe is experiencing lag in live testing sessions, whitelisting `stun-eu-1.lambdatest.com` and `stun-eu-2.lambdatest.com` resolves it. - -## 5. When Whitelisting Will Not Help (Tunnel Required) - -Whitelisting only works if your application is publicly discoverable on the internet (with IP restrictions). If your resources are inside a private network, on localhost, or on a non-routable subnet, whitelisting will not work. - -#### Use TestMu AI Tunnel Instead -The Tunnel binary (or Underpass app) creates a secure outbound connection from your machine/network to TestMu AI, enabling the cloud to access: -- Localhost apps (http://localhost:3000) -- Internal apps behind VPN or private subnets (10.x.x.x, 192.168.x.x) -- Apps accessible only within your corporate LAN - -#### Example -- A developer wants to test a React app running on `http://localhost:3000`. Whitelisting won’t work because it’s not internet-exposed. -- Solution: Start a TestMu AI Tunnel session and run tests. - -## 6. When Site-to-Site (S2S) Tunnel Should Be Used -For enterprise-grade setups, a Site-to-Site (S2S) tunnel provides a persistent, secure connection between your corporate network and TestMu AI. - -#### When to Use -- You want to avoid starting a tunnel for each session. -- Your entire QA team needs always-on secure connectivity. -- You have strict compliance requirements (no public exposure, no shared infrastructure). -- You’re testing large-scale or highly regulated environments (finance, healthcare, government). - -#### Example -A bank sets up a permanent S2S tunnel so all their staging/test environments can be accessed by TestMu AI securely, without exposing them on the public internet or managing tunnels manually. - -## Quick Decision Guide - -| Scenario | Recommended Setup | -| ------------------------------------------------ | ------------------------------------------ | -| Public site with IP restrictions | Inbound Whitelisting (Shared IPs) | -| Public site with strict compliance | Dedicated Proxy Whitelisting | -| Corporate firewall blocks TestMu AI access | Outbound Whitelisting (`*.lambdatest.com`) | -| Slowness in live sessions / video issues | STUN/TURN Server Whitelisting | -| Localhost or private subnet apps | Tunnel (Binary/Underpass) | -| Always-on, enterprise-grade private connectivity | Site-to-Site Tunnel | diff --git a/static/docs/new-relic-integration.md b/static/docs/new-relic-integration.md deleted file mode 100644 index b05831d0d..000000000 --- a/static/docs/new-relic-integration.md +++ /dev/null @@ -1,58 +0,0 @@ -# New Relic Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -New Relic is a comprehensive observability platform that helps you monitor, troubleshoot, and optimize your software systems. It collects data from all your applications, infrastructure, and logs and provides insights into how your system is performing and where potential problems lie. - -TestMu AI and New Relic enable you to view TestMu AI results in a custom New Relic dashboard and gain deeper insight into your automation scripts to ship quality products at light speed. - -## Prerequisites - -- A TestMu AI account. If you don't have an account, sign up for free. -- A New Relic account, If you don't have an account, [sign up for free](https://newrelic.com/signup) - -## Integrating New Relic with TestMu AI - -**Step 1:** Visit the New Relic UI. Click on [**Add Data**](https://one.newrelic.com/marketplace) button on the left navigation. - -**Step 2:** Search **TestMu AI**, select the tile, and click on: - 1. **See installation docs** if you have not configured your TestMu AI account. - 2. **Skip this step** if you have already instrumented TestMu AI (from Step 3 to Step 5). - -You can copy the Licence key details from the side bar which will be required in the **Step 5**. - -This will automatically deploy a pre-built dashboard for you in New Relic. Click on **See your data** button to check the TestMu AI Dashboards. - -**Step 3:** Visit the TestMu AI Dashboard. Go to the **Settings** page. - -**Step 4:** Click on the **Integrations** tab. Select **New Relic** from the **Analytics** section. - -**Step 5:** Now enter the **License Key/API Key** and select your **Region** - -- In order to get the New Relic **License/API** Key, refer to the New Relic UI from Step 2. - -Alternatively, you can go to your **New Relic** account > **Profile** > [**API Keys**](https://one.newrelic.com/api-keys). - -> You can either use the existing API keys or you can create a new one. - -- Paste the License/API Key and Region in the required field. - -- Click on **Install** button. - -The New Relic will be integrated with your TestMu AI account. - -- After successful integration, New Relic will trigger the log ingestion that will now create the TestMu AI dashboards (TestMu AI - Test Overview and TestMu AI - Test Error Overview) to your directory. - -## TestMu AI Dashboards - Test Overview and Test Error Overview - -- Click on the **TestMu AI Test Overview** dashboard to get an overall insight about your testing trends. - -- Click on the **TestMu AI Test Error Overview** dashboard to understand your test error trends. - -## Removing The New Relic Integration - -**Step 1:** Login to your TestMu AI account and visit **Integrations** from the left sidebar. - -**Step 2:** From the **MY INTEGRATIONS** category, navigate to **NEW RELIC** and click REMOVE. - -The New Relic integration will be uninstalled. diff --git a/static/docs/nightwatch-on-hyperexecute-grid.md b/static/docs/nightwatch-on-hyperexecute-grid.md deleted file mode 100644 index 708cb5049..000000000 --- a/static/docs/nightwatch-on-hyperexecute-grid.md +++ /dev/null @@ -1,238 +0,0 @@ -# Running Nightwatch Framework Tests on HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Nightwatch.js is an automated testing framework built on top of Selenium and Node.js. It simplifies end-to-end testing, offering a simple syntax and a powerful set of built-in commands. Nightwatch.js supports the Selenium WebDriver protocol and allows efficient browser automation, making it a popular choice for web application testing. - -HyperExecute is an AI Native Test Orchestration Cloud Platform that empowers you to run **end-to-end** tests **quickly** and **efficiently**. It provides Just-in-Time (JIT) testing infrastructure with fast execution **speeds**, **smart orchestration**, and **detailed logs**. - -This guide details how to execute your **Nightwatch** framework tests on **HyperExecute** via two different methods: - -- [**Using Local System**](/support/docs/nightwatch-on-hyperexecute-grid/#1-testing-using-local-system) - You can use your own local machine to execute tests. -- [**Using Gitpod Platform**](/support/docs/nightwatch-on-hyperexecute-grid/#2-testing-using-gitpod) - Execute tests using GitPod. (Requires a [Gitpod](https://gitpod.io/login/) account) - -## 1. Testing Using Local System - -Follow the step-by-step guide to execute your test on HyperExecute. - -### Prerequisites - -To run the Tests on HyperExecute from your Local System, you are required: - -- Your TestMu AI [Username and Access key](/support/docs/hyperexecute-how-to-get-my-username-and-access-key/) -- [HyperExecute YAML](/support/docs/hyperexecute-yaml-version0.2/) file which contains all the necessary instructions. -- [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) in order to initiate a test execution Job . -- Setup the [Environmental Variable](/support/docs/hyperexecute-environment-variable-setup/) - -### Step 1: Configure Your Test Suite - -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** - -Download or Clone the code sample for the JUnit from the TestMu AI GitHub repository to run the tests on the HyperExecute. - - View on GitHub - -If you are using your own project, make sure you update the **Hub endpoint** in your tests file. - -By setting up the Hub endpoint, you establish the communication channel between your tests and the browser nodes, enabling effective test distribution and execution. - -Configure the desired capabilities based on your test requirements. For example: - -```bash -# nightwatch.conf.js - -module.exports = (function(settings) { -console.log(settings["test_settings"]["default"]["username"]) -if (process.env.LT_USERNAME) { -settings["test_settings"]["default"]["username"] = process.env.LT_USERNAME; -} -if (process.env.LT_ACCESS_KEY) { -settings["test_settings"]["default"]["access_key"] = process.env.LT_ACCESS_KEY; -} -if (process.env.SELENIUM_HOST) { -settings.selenium.host = process.env.SELENIUM_HOST; -} -if (process.env.SELENIUM_PORT) { -settings.selenium.host = process.env.SELENIUM_PORT; -} -return settings; -})(require('./nightwatch.json')); -``` - -> You can generate capabilities for your test requirements with the help of our inbuilt 🔗 Capabilities Generator. - -### Step 2: Setup the CLI in your Test Suite - -After cloning / downloading the sample repo, you need to setup the CLI and the environment variables. - -#### Download the HyperExecute CLI - -The CLI is used for triggering the tests on HyperExecute. It is recommend to download the CLI binary on the host system and keep it in the root directory of the suite to perform the tests on HyperExecute. - -You can download the CLI for your desired platform from the below mentioned links: - -| Platform | HyperExecute CLI | -| ---------| ---------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| MacOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -#### Setup Environment Variable - -Now, you need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in the [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). - -Run the below mentioned commands in your terminal to setup the CLI and the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -### Step 3: Configure YAML in your Test Suite - -Configure your YAML file as per your use cases using **key value** pairs. - -In this sample YAML file, we have mentioned: - -- **version** of the YAML file -- **Timeouts** for executing your project -- **Mode of execution** is [Autosplit](/support/docs/hyperexecute-auto-split-strategy/). You can also opt for [Matrix](/support/docs/hyperexecute-matrix-multiplexing-strategy/) or [Hybrid](/support/docs/hyperexecute-hybrid-strategy/) mode. -- **Pre and Post** commands -- **Reports and Artefacts** that will be generated after the completion of tests -- and other necessary YAML Parameters - -```yaml ---- -version: 0.1 -globalTimeout: 90 -testSuiteTimeout: 90 -testSuiteStep: 90 - -runson: linux -autosplit: true -retryOnFailure: true - -maxRetries: 1 -concurrency: 3 -parallelism: 1 - -cacheKey: '{{ checksum "package-lock.json" }}' -cacheDirectories: -- node_modules - -pre: -- npm install - -mergeArtifacts: true -uploadArtifacts: -- name: Reports -path: -- reports/ - -report: true -partialReports: -type: json -location: reports/ -frameworkName: extent - -testDiscovery: -type: raw -mode: remote -command: grep -B1 'desiredCapabilities' nightwatch.json | sed 's/-//g' | grep -vE 'desiredCapabilities' | grep -vE 'skip_testcases_on_fail' | awk '{print$1}' | sed 's/://g' | sed 's/"//g' - -testRunnerCommand: ./node_modules/.bin/nightwatch -e $test - -jobLabel: [selenium-Nightwatch, linux, autosplit] -``` - -### Step 4: Execute your Test Suite - -> **NOTE :** In case of MacOS, if you get a permission denied warning while executing CLI, simply run **`chmod u+x ./hyperexecute`** to allow permission. In case you get a security popup, allow it from your **System Preferences** → **Security & Privacy** → **General tab**. - -Run the below command in your terminal at the root folder of the project: - -```bash -./hyperexecute --config RELATIVE_PATH_OF_YOUR_YAML_FILE -``` - -OR use this command if you have not exported your username and access key in the step 2. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE `} - -### Step 5: Monitor the Test Execution - -Visit the [HyperExecute Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) and check your Job status. - -### Step 6: Download Artifacts and Reports - -HyperExecute also facilitates the provision to download the [Artifacts](/support/docs/hyperexecute-artifacts/) and [Reports](/support/docs/hyperexecute-reports/) on your local machine. Click on the corresponding button to download your generated artifacts and reports. - -## 2. Testing Using Gitpod - -You can also use the Gitpod platform to execute our sample repository. It will fetch all the sample codebases and trigger the CLI to execute the tests. - -Follow the below steps to run Test using Gitpod: - -**Step 1:** Click '**Open in Gitpod**' button. You will be redirected to Login/Signup page. This button is configured to redirect you to the Gitpod platform where you will be able to execute our sample repository. - -**Step 2:** Login with TestMu AI credentials. Once logged in, a pop-up confirmation will appear, asking you to **'Proceed'** to the Gitpod editor in a new tab. The current tab will display the HyperExecute Dashboard. - -**Step 3:** Choose your preferred editor (we recommend VS Code Editor) - -**Step 4:** As you are running a sample project, Fetching of the Test Scripts, [HyperExecute YAML](/support/docs/deep-dive-into-hyperexecute-yaml/), [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) and Triggering your tests using the `Execution Command` will be automated. - -**Step 5:** Once you see the `Job Link` in the logs, you can visit the [HyperExecute dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) to see the tests getting executed. - -You can also implement [Secret Keys](/support/docs/hyperexecute-how-to-save-and-manage-secrets/) in your YAML file. - -## Navigation in Automation Dashboard - -Every test run on the HyperExecute has a unique *jobId* associated with it. Each *jobId* can in turn constitute single (or multiple) *groupId*(s). You can visit [HyperExecute Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) for checking the status of the test execution. - -You can seamlessly navigate between JobId's and taskId's. You need to click on the *testID* to navigate from the HyperExecute logs to the Automation Dashboard. - -The snapshot below shows the videos, logs and other meta data for that specific *test_ID* - -> For any query or doubt, please feel free to contact us via **24×7 chat support** or you can also drop a mail to **support@testmuai.com**. -Happy testing! - -## Using the NightwatchJS Agent Skill with TestMu AI - -The [nightwatchjs-skill](https://github.com/LambdaTest/agent-skills/tree/main/nightwatchjs-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The nightwatchjs-skill package includes: - -``` -nightwatchjs-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing NightwatchJS Agent Skill - -Install a NightwatchJS Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/nightwatchjs-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/nightwatchjs-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only nightwatchjs-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). diff --git a/static/docs/nightwatch-with-selenium-running-nightwatch-automation-scripts-on-testmu-selenium-grid.md b/static/docs/nightwatch-with-selenium-running-nightwatch-automation-scripts-on-testmu-selenium-grid.md deleted file mode 100644 index 6ead00d8c..000000000 --- a/static/docs/nightwatch-with-selenium-running-nightwatch-automation-scripts-on-testmu-selenium-grid.md +++ /dev/null @@ -1,96 +0,0 @@ -# Selenium With Nightwatch.js - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Run Nightwatch.js tests on the TestMu AI cloud grid. This guide covers setup, running a sample test, configuring capabilities, and testing locally hosted pages. - -**Sample repo** -All the code used in this guide is available in the sample repository. - - View on GitHub - -## Prerequisites - -Complete these steps before running Nightwatch Selenium tests on TestMu AI. - -1. Create a [TestMu AI account](https://www.testmuai.com/register/?redirectTo=https://accounts.lambdatest.com/dashboard) and get your username and access key from the dashboard. -2. Install **NodeJS** v6 or newer from [nodejs.org](https://nodejs.org/en/). -3. Install **npm** from the [official npm website](https://www.npmjs.com/). -4. Download [Selenium JavaScript bindings](https://www.selenium.dev/downloads/) from the official website. - -## Step 1: Clone the Sample Project - -Clone the TestMu AI Nightwatch Selenium sample repository to your local machine. - -```bash -git clone https://github.com/LambdaTest/nightwatch-selenium-sample -cd nightwatch-selenium-sample -``` - -Install the required dependencies: -```bash -npm i -``` - -## Step 2: Set Your Credentials - -Set your TestMu AI username and access key as environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -## Step 3: Configure Your Test Capabilities - -Update the capabilities in `nightwatch.conf.js` to define the browser and platform settings. - -```javascript title="nightwatch.conf.js" reference -https://github.com/LambdaTest/nightwatch-selenium-sample/blob/master/nightwatch.conf.js -``` - -Generate capabilities for your test requirements with the [Capabilities Generator](https://www.testmuai.com/capabilities-generator/). - -## Step 4: Run the Test - -Execute the test using the following command. - -**Single test:** -```bash -npm run single -``` - -**Parallel tests:** -```bash -npm run parallel -``` - -## Step 5: View Your Results - -After running the test, view your results on the [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build). - -The dashboard provides: -- Video recordings of each test session -- Screenshots captured at each step -- Console logs from the browser -- Network logs for debugging -- Detailed command logs - -## Run Nightwatch.js Tests Using Agent Skills - -Use AI coding assistants to generate and run Nightwatch.js tests with the TestMu AI Agent Skill. - -The [nightwatchjs-skill](https://github.com/LambdaTest/agent-skills/tree/main/nightwatchjs-skill) is part of [TestMu AI Agent Skills](https://github.com/LambdaTest/agent-skills/) - structured packages that teach AI coding assistants how to write production-grade test automation. - -Install the skill: - -```bash -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/nightwatchjs-skill .claude/skills/ - -# For Cursor / Copilot -cp -r agent-skills/nightwatchjs-skill .cursor/skills/ -``` - -Install all available framework skills at once by cloning the repository directly into your tool's skills directory (e.g., `.claude/skills/`, `.cursor/skills/`). diff --git a/static/docs/notion-integration.md b/static/docs/notion-integration.md deleted file mode 100644 index 8fd8012ad..000000000 --- a/static/docs/notion-integration.md +++ /dev/null @@ -1,80 +0,0 @@ -# Notion Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Notion is an all-in-one workspace for the teams where you can write, plan, collaborate, and get organized. You can take notes, assign tasks, manage projects, and much more. Notion has customized markdown support that integrates kanban boards, tasks, wikis, and databases. - -The TestMu AI and Notion integration allows you to perform one-click bug logging while perform [cross browser testing](https://www.testmuai.com/) of your websites (and web apps). You can mark and share bugs directly from the TestMu AI to your Notion pages. - -## Integrating Notion With TestMu AI Account - -### For existing users: Re-Sync your Integration - -If you are already an existing user and have integrated Notion with TestMu AI earlier, then you have to re-sync the integration to update the changes. - -Follow the below-mentioned method to re-sync the changes: - -**Step 1:** Go to the Integrations page. - -**Step 2:** Click on the **Resync** button to re-sync the changes made. - -### For new users: Install the Integration - -> If you already have a TestMu AI integration with any third-party tool then you will be asked to uninstall the existing TestMu AI integration. - -1. Login to your TestMu AI account. To install integrations, ensure you have admin or user level access. - -2. Visit 'Integrations' from the left menu and select **'Notion'** from the **'BUG TRACKER'** section. - ->**Note**: Make sure you have created an account on Notion. - -3. You will be taken to OAuth Page. Click on **Select Pages**. - -4. Choose the page on which you want to give an access. For example, in the below screenshot, we select *DemoTest1* page. - -5. After selecting the pages, click on **Allow Access**. - -The Notion will be successfully integrated with your TestMu AI account. You'll also receive the confirmation message and an email of the same. - -## Creating A Kanban Board - -1. Open Notion dashboard. From the left menu, click **Add a page**. - -2. Select **Board**. - -3. Give a relevant title and add your desired properties in the board. - -Your Kanban board is now configured to store bugs. - -## One-Click Bug Logging Through Notion - -We will perform one-click bug looging while perform Real Time Testing. - -1. Select **Real Time Testing** from the left menu. - -2. Enter the test URL, select the test environment details like Browser, Browser versions, OS and Resolution. Then click on **'START'**. - -3. A cloud-based virtual machine will launch where you can perform real time testing of websites and web apps. If you found a bug, click on the **'Mark as Bug'** icon from the left tool bar to grab the screenshot. - -4. Once the screenshot is captured, annotate issues or tasks that you have encountered using in-built image editor tools. After annotating the bugs, click on **'Mark As Bug'**. - -5. A Notion ticket will open up. Fill the details like Board, Status, Assignee, Summary and Description. After providing the details in the ticket, click on **'Create Issue"**. - -Once you click on Create Issue, your bugs will be marked successfully. - -6. To view your logged issues, visit **Issue Tracker** from the TestMu AI left menu. You'll notice the marked bugs along with the Test environment details on the right. - -7. To view your marked bugs in Notion, click on the right diagonal arrow icon. - -You'll be routed to the Notion dashboard where you can find your marked issues along with the test environment details. - -## Uninstalling The Notion Integration - -1. Login to your TestMu AI account and visit Integrations from the left menu. - -2. Navigate to 'My Integrations'and click on **'REMOVE'** under the Notion block. - -The Notion integration will be uninstalled. - -> -In case you have any doubts or questions, feel free to drop them at our **24x7 Chat Support** or you could also mail us directly at **[support@testmuai.com](mailto:support@testmuai.com)**. diff --git a/static/docs/npm-plugin-for-testcafe-integration-with-testmu.md b/static/docs/npm-plugin-for-testcafe-integration-with-testmu.md deleted file mode 100644 index 2b03c451e..000000000 --- a/static/docs/npm-plugin-for-testcafe-integration-with-testmu.md +++ /dev/null @@ -1,119 +0,0 @@ -# Selenium With TestCafe - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Run TestCafe tests on the TestMu AI cloud grid. This guide covers setup, running a sample test, configuring capabilities, and testing locally hosted pages. - -**npm package** -Visit the TestMu AI [npm package for TestCafe](https://www.npmjs.com/package/testcafe-browser-provider-lambdatest) for full details. - -## Prerequisites - -Complete these steps before integrating TestCafe with TestMu AI. - -1. Create a [TestMu AI account](https://www.testmuai.com/register/?redirectTo=https://accounts.lambdatest.com/dashboard) and get your username and access key from the dashboard. -2. Add the dependency `"testcafe": ""` in your `package.json` file. - -## Step 1: Clone the Sample Project - -Install the TestMu AI npm plugin for TestCafe. - -``` js -npm install testcafe-browser-provider-lambdatest -``` - -## Step 2: Set Your Credentials - -Set your TestMu AI username and access key as environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -## Step 3: Configure Your Test Capabilities - -Configure your tests using environment variables for resolution, build name, and other settings. - -| VARIABLE | DESCRIPTION | -|---------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| LT_TEST_NAME | Specify a name for your test on TestMu AI Selenium Grid. | -| LT_BUILD | Specify a name for your build on TestMu AI Selenium Grid. | -| LT_CAPABILITY_PATH | Specify the path of a JSON file that declares additional Desired Capabilities. ` "Chrome@63.0:Windows 8.1" : {` `"network" : true,` `"visual" : true,` `"timezone" : "UTC+11:00"` `}` Here, ``Chrome@63.0:Windows 8.1`` represents browser alias. | -| LT_RESOLUTION | Perform cross browser testing on a specific screen resolution in the ${width}x${height} format. | -| LT_LOGFILE | View detailed logs of your automation script execution. Provide a specific path to this file. If you don't provide a path, the logs save in your present working directory by the filename: tunnel.log. | -| LT_VERBOSE | Set flag for verbose to either true or false. | -| LT_PROXY_HOST | A required variable if you perform cross browser testing through proxy. Declare the hostname/IP of proxy. | -| LT_PROXY_PORT | Port of the proxy. Default value for proxy port is 3128. | -| LT_PROXY_USER | Username for connecting to proxy. Required for using 'proxypass'. | -| LT_PROXY_PASS | Password for the USERNAME option. | -| LT_TUNNEL_NAME | Human readable tunnel identifier (Name of the tunnel). | -| LT_DIR | Path of the local folder you want to test. | -| LT_SELENIUM_VERSION | Browser specific capability. | -| LT_CONSOLE | Set flag as true or false. | -| LT_NETWORK | Set flag as true or false. | -| LT_VIDEO | Set flag as true or false. | -| LT_SCREENSHOT | Set flag as true or false. | -| LT_TIMEZONE | Configure tests to run on a custom time zone. | -| LT_TUNNEL_NUMBER | Number of tunnels to spawn at a time. | -| LOAD_BALANCED_MODE | Load Balanced Mode in TestCafe. | - -Generate capabilities for your test requirements with the [Capabilities Generator](https://www.testmuai.com/capabilities-generator/). - -## Step 4: Run the Test - -Execute the test using the following command. - -**Single test:** -``` js -$ testcafe "lambdatest:Chrome@74.0:Windows 8" 'path/to/test/file.js' -``` - -**Parallel tests:** -``` js -$ testcafe "lambdatest:Chrome@74.0:Windows 8","lambdatest:Chrome@75.0:Windows 10" "path/to/test/file.js" -``` - -**Using the API:** -``` js -testCafe -.createRunner() -.src('path/to/test/file.js') -.browsers('lambdatest:Chrome@74.0:Windows 8') -.run(); -``` - -To list all available browser aliases: -``` js -testcafe -b lambdatest -``` - -## Step 5: View Your Results - -After running the test, view your results on the [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build). - -The dashboard provides: -- Video recordings of each test session -- Screenshots captured at each step -- Console logs from the browser -- Network logs for debugging -- Detailed command logs - -## Run TestCafe Tests Using Agent Skills - -Use AI coding assistants to generate and run TestCafe tests with the TestMu AI Agent Skill. - -The [testcafe-skill](https://github.com/LambdaTest/agent-skills/tree/main/testcafe-skill) is part of [TestMu AI Agent Skills](https://github.com/LambdaTest/agent-skills/) - structured packages that teach AI coding assistants how to write production-grade test automation. - -Install the skill: - -```bash -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/testcafe-skill .claude/skills/ - -# For Cursor / Copilot -cp -r agent-skills/testcafe-skill .cursor/skills/ -``` - -Install all available framework skills at once by cloning the repository directly into your tool's skills directory (e.g., `.claude/skills/`, `.cursor/skills/`). diff --git a/static/docs/npm-via-tunnel.md b/static/docs/npm-via-tunnel.md deleted file mode 100644 index e0bffddf9..000000000 --- a/static/docs/npm-via-tunnel.md +++ /dev/null @@ -1,51 +0,0 @@ -# Install NPM Packages via Tunnel - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## What is `npm_via_tunnel`? - -`npm_via_tunnel` is a flag that you can use to enable downloading npm dependencies through a tunnel. This can be useful if you need to install npm packages from a private registry or if you are working in a restricted environment where you cannot access the public npm registry directly. - -## Why use `npm_via_tunnel`? - -There are a few reasons why you might want to use `npm_via_tunnel`: - -- **Security:** If you are installing npm packages from a private registry, you may want to use a tunnel to protect your traffic from being intercepted. - -- **Reliability:** If you are working in a restricted environment, a tunnel can help to ensure that npm can still access the registry and download the packages you need. - -- **Flexibility:** You can use a tunnel to access npm packages from any source, including public registries, private registries, and even local repositories. - -## How to use npm_via_tunnel - -To use `npm_via_tunnel`, simply add the following line to your run settings: - -```bash -"npm_via_tunnel": true -``` - -Once you have enabled this flag, npm will automatically route all of its traffic through the tunnel. - -## For first-time users: - -If you are using `npm_via_tunnel` for the first time, here are a few things you should keep in mind: - -- You will need to have a tunnel already configured and running. There are a number of different ways to create a tunnel, so you will need to choose the method that is best for your environment. - -- Once you have followed these steps, npm will automatically route all of its traffic through the tunnel and you will be able to install npm packages from the specified registry. - -> To download private dependencies follow this [documentation](/support/docs/private-dependencies-cypress/). - -## Troubleshooting - -If you are having problems with npm_via_tunnel, you can try the following troubleshooting tips: - -- Make sure that the tunnel is properly configured and running. - -- Verify that the npm registry is accessible from within the tunnel. - -- Try disabling any proxy servers or firewalls that may be interfering with the tunnel connection. - -## Conclusion - -`npm_via_tunnel` is a powerful tool that can help you to install npm packages from a variety of sources, including private registries and restricted environments. By using this flag, you can improve the security, reliability, and flexibility of your npm installations. diff --git a/static/docs/nunit-on-hyperexecute-grid.md b/static/docs/nunit-on-hyperexecute-grid.md deleted file mode 100644 index 2ea9cf4ad..000000000 --- a/static/docs/nunit-on-hyperexecute-grid.md +++ /dev/null @@ -1,234 +0,0 @@ -# Running NUnit Framework Tests on HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -NUnit is a testing framework for .NET languages, commonly used with Selenium for automated testing. It provides a structure for organizing and executing tests, supporting features like test fixtures, assertions, and setup/teardown methods. - -HyperExecute is an AI Native Test Orchestration Cloud Platform that empowers you to run **end-to-end** tests **quickly** and **efficiently**. It provides Just-in-Time (JIT) testing infrastructure with fast execution **speeds**, **smart orchestration**, and **detailed logs**. - -This guide details how to execute your **NUnit** framework tests on **HyperExecute** via two different methods: - -- [**Using Local System**](/support/docs/nunit-on-hyperexecute-grid/#1-testing-using-local-system) - You can use your own local machine to execute tests. -- [**Using Gitpod Platform**](/support/docs/nunit-on-hyperexecute-grid/#2-testing-using-gitpod) - Execute tests using GitPod. (Requires a [Gitpod](https://gitpod.io/login/) account) - -## 1. Testing Using Local System - -Follow the step-by-step guide to execute your test on HyperExecute. - -### Prerequisites - -To run the Tests on HyperExecute from your Local System, you are required: - -- Your TestMu AI [Username and Access key](/support/docs/hyperexecute-how-to-get-my-username-and-access-key/) -- [HyperExecute YAML](/support/docs/hyperexecute-yaml-version0.2/) file which contains all the necessary instructions. -- [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) in order to initiate a test execution Job . -- Setup the [Environmental Variable](/support/docs/hyperexecute-environment-variable-setup/) - -### Step 1: Configure Your Test Suite - -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** - -Download or Clone the code sample for the JUnit from the TestMu AI GitHub repository to run the tests on the HyperExecute. - - View on GitHub - -If you are using your own project, make sure you update the **Hub endpoint** in your tests file. - -By setting up the Hub endpoint, you establish the communication channel between your tests and the browser nodes, enabling effective test distribution and execution. - -Configure the desired capabilities based on your test requirements. For example: - -```bash -DesiredCapabilities capabilities = new DesiredCapabilities(); -capabilities.SetCapability(CapabilityType.BrowserName, "Chrome"); -capabilities.SetCapability(CapabilityType.Version, "96"); -capabilities.SetCapability(CapabilityType.Platform, "Windows 10"); -``` - -> You can also use **dev** and **beta** browser versions. To generate capabilities for your test requirements, you can use our inbuilt 🔗 Capabilities Generator. - -### Step 2: Setup the CLI in your Test Suite - -After cloning / downloading the sample repo, you need to setup the CLI and the environment variables. - -#### Download the HyperExecute CLI - -The CLI is used for triggering the tests on HyperExecute. It is recommend to download the CLI binary on the host system and keep it in the root directory of the suite to perform the tests on HyperExecute. - -You can download the CLI for your desired platform from the below mentioned links: - -| Platform | HyperExecute CLI | -| ---------| ---------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| MacOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -#### Setup Environment Variable - -Now, you need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in the [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). - -Run the below mentioned commands in your terminal to setup the CLI and the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -### Step 3: Configure YAML in your Test Suite - -Configure your YAML file as per your use cases using **key value** pairs. - -In this sample YAML file, we have mentioned: - -- **version** of the YAML file -- **Timeouts** for executing your project -- **Mode of execution** is [Autosplit](/support/docs/hyperexecute-auto-split-strategy/). You can also opt for [Matrix](/support/docs/hyperexecute-matrix-multiplexing-strategy/) or [Hybrid](/support/docs/hyperexecute-hybrid-strategy/) mode. -- **Pre and Post** commands -- **Reports and Artefacts** that will be generated after the completion of tests -- and other necessary YAML Parameters - -```yaml ---- -version: 0.1 -globalTimeout: 150 -testSuiteTimeout: 150 -testSuiteStep: 150 - -runson: linux - -autosplit: true -retryOnFailure: true - -maxRetries: 1 -concurrency: 2 - -project: NUnitHyperTestDemo/NUnitHyperTestDemo.csproj -solution: NUnitHyperTestDemo.sln - -env: -NUGET_PACKAGES: '/home/ltuser/.nuget/packages/' -NUGET_HTTP_CACHE_PATH: '/home/ltuser/.local/share/NuGet/v3-cache' -NUGET_PLUGINS_CACHE_PATH: '/home/ltuser/.local/share/NuGet/plugins-cache' - -pre: -- dotnet restore -- dotnet list $project package > packages.txt -- dotnet build -c Release - -post: -- cat yaml/linux/nunit_hyperexecute_autosplit_sample.yaml - -mergeArtifacts: true - -uploadArtefacts: -- name: ExecutionSnapshots -path: -- NUnitHyperTestDemo/Reports/** - -report: true -partialReports: -type: json -location: NUnitHyperTestDemo/ -frameworkName: extent - -testDiscovery: -type: raw -mode: remote -command: grep 'Category' NUnitHyperTestDemo -ir --include=\*.cs --exclude=DriverFactory.cs --exclude=HyperTestDemo.AssemblyInfo.cs | awk '{print$2}' | grep -o '".*"' - -testRunnerCommand: dotnet test $solution --filter TestCategory=$test - -jobLabel: [selenium-Nunit, linux, autosplit] -``` - -### Step 4: Execute your Test Suite - -> **NOTE :** In case of MacOS, if you get a permission denied warning while executing CLI, simply run **`chmod u+x ./hyperexecute`** to allow permission. In case you get a security popup, allow it from your **System Preferences** → **Security & Privacy** → **General tab**. - -Run the below command in your terminal at the root folder of the project: - -```bash -./hyperexecute --config RELATIVE_PATH_OF_YOUR_YAML_FILE -``` - -OR use this command if you have not exported your username and access key in the step 2. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE `} - -### Step 5: Monitor the Test Execution - -Visit the [HyperExecute Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) and check your Job status. - -### Step 6: Download Artifacts and Reports - -HyperExecute also facilitates the provision to download the [Artifacts](/support/docs/hyperexecute-artifacts/) and [Reports](/support/docs/hyperexecute-reports/) on your local machine. Click on the corresponding button to download your generated artifacts and reports. - -## 2. Testing Using Gitpod - -You can also use the Gitpod platform to execute our sample repository. It will fetch all the sample codebases and trigger the CLI to execute the tests. - -Follow the below steps to run Test using Gitpod: - -**Step 1:** Click '**Open in Gitpod**' button. You will be redirected to Login/Signup page. This button is configured to redirect you to the Gitpod platform where you will be able to execute our sample repository. - -**Step 2:** Login with TestMu AI credentials. Once logged in, a pop-up confirmation will appear, asking you to **'Proceed'** to the Gitpod editor in a new tab. The current tab will display the HyperExecute Dashboard. - -**Step 3:** Choose your preferred editor (we recommend VS Code Editor) - -**Step 4:** As you are running a sample project, Fetching of the Test Scripts, [HyperExecute YAML](/support/docs/deep-dive-into-hyperexecute-yaml/), [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) and Triggering your tests using the `Execution Command` will be automated. - -**Step 5:** Once you see the `Job Link` in the logs, you can visit the [HyperExecute dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) to see the tests getting executed. - -You can also implement [Secret Keys](/support/docs/hyperexecute-how-to-save-and-manage-secrets/) in your YAML file. - -## Navigation in Automation Dashboard - -Every test run on the HyperExecute has a unique *jobId* associated with it. Each *jobId* can in turn constitute single (or multiple) *groupId*(s). You can visit [HyperExecute Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) for checking the status of the test execution. - -You can seamlessly navigate between JobId's and taskId's. You need to click on the *testID* to navigate from the HyperExecute logs to the Automation Dashboard. - -The snapshot below shows the videos, logs and other meta data for that specific *test_ID* - -> For any query or doubt, please feel free to contact us via **24×7 chat support** or you can also drop a mail to **support@testmuai.com**. -Happy testing! - -## Using the NUnit Agent Skill with TestMu AI - -The [nunit-skill](https://github.com/LambdaTest/agent-skills/tree/main/nunit-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The nunit-skill package includes: - -``` -nunit-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing NUnit Agent Skill - -Install a NUnit Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/nunit-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/nunit-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only nunit-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). diff --git a/static/docs/nunit-with-selenium-running-nunit-automation-scripts-on-testmu-selenium-grid.md b/static/docs/nunit-with-selenium-running-nunit-automation-scripts-on-testmu-selenium-grid.md deleted file mode 100644 index af556974c..000000000 --- a/static/docs/nunit-with-selenium-running-nunit-automation-scripts-on-testmu-selenium-grid.md +++ /dev/null @@ -1,115 +0,0 @@ -# Selenium With NUnit - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Run NUnit tests on the TestMu AI cloud grid. This guide covers setup, running a sample test, configuring capabilities, and testing locally hosted pages. - -**Sample repo** -All the code used in this guide is available in the sample repository. - - View on GitHub - -## Prerequisites - -Complete the following steps before running NUnit Selenium tests. - -1. Download and install **Selenium WebDriver** from the [official website](https://www.selenium.dev/downloads/). -2. Install the latest version of C#. -3. Install the **.Net** framework for developing applications using C#. -4. Download [Selenium WebDriver Language Binding](https://www.selenium.dev/downloads/) for C# and extract them to the appropriate folder. -5. Install a [.NET Core SDK](https://dotnet.microsoft.com/en-us/download) of 2.1 or greater version. -6. Get your TestMu AI Username and Access Key from the [TestMu AI Dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard). - -## Step 1: Clone the Sample Project - -Clone the repository and navigate to the project directory. - -```bash -git clone https://github.com/LambdaTest/CSharp-NUnit-Selenium -cd CSharp-NUnit-Selenium -``` - -## Step 2: Set Your Credentials - -Configure your credentials to connect to the TestMu AI Selenium Grid. - -Set TestMu AI Username and Access Key in environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -## Step 3: Configure Your Test Capabilities - -Define browser, version, and OS settings for your test run. - -In the test script, update your test capabilities. This code passes browser, browser version, and operating system information, along with TestMu AI Selenium grid capabilities via the capabilities object. - -```csharp -var browserOptions = new ChromeOptions(); -browserOptions.PlatformName = "Windows 10"; -browserOptions.BrowserVersion = "latest"; - -var ltOptions = new Dictionary -{ -{ "build", "NUnit Build" }, -{ "name", "NUnit Test" }, -{ "w3c", true } -}; -browserOptions.AddAdditionalOption("LT:Options", ltOptions); -``` - -**Capabilities Generator** -Use the TestMu AI [Capabilities Generator](https://www.testmuai.com/capabilities-generator/) to auto-generate the capabilities class for your test requirements. - -## Step 4: Run the Test - -Execute the NUnit test from Visual Studio or the command line. - -**In Visual Studio (Windows):** - -1. Build the solution by clicking **Build > Build Solution**. -2. Open **Test Explorer** in Visual Studio. - -3. Click **Run** from the Test Explorer to run the sample test. - -**In Linux/macOS:** - -Install NuGet packages and build the project: -```csharp -cd Packages -nuget.exe install ..\NUnitSelenium\packages.config -``` -```csharp -nmake clean build -``` - -## Step 5: View Your Results - -Check the test output on the console and the TestMu AI dashboard. - -Visit the [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/) to view your test results. The dashboard provides: - -- Text logs for each test step -- Screenshots captured during execution -- Video recordings of the full test session - -## Run NUnit Tests Using Agent Skills - -Use AI coding assistants to generate and run NUnit tests with the TestMu AI Agent Skill. - -The [nunit-skill](https://github.com/LambdaTest/agent-skills/tree/main/nunit-skill) is part of [TestMu AI Agent Skills](https://github.com/LambdaTest/agent-skills/) - structured packages that teach AI coding assistants how to write production-grade test automation. - -Install the skill: - -```bash -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/nunit-skill .claude/skills/ - -# For Cursor / Copilot -cp -r agent-skills/nunit-skill .cursor/skills/ -``` - -Install all available framework skills at once by cloning the repository directly into your tool's skills directory (e.g., `.claude/skills/`, `.cursor/skills/`). diff --git a/static/docs/okta.md b/static/docs/okta.md deleted file mode 100644 index 7d284a350..000000000 --- a/static/docs/okta.md +++ /dev/null @@ -1,93 +0,0 @@ -# Okta Scim User Provisioning - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Prerequisites -Integrate SCIM With TestMu AI: - -- You will need an Enterprise plan with TestMu AI. -- SSO must be already integrated. Please complete [TestMu AI SSO & Okta Integration](/support/docs/testmu-sso-okta-integration/) - -## Integrating SCIM with Okta -**Step 1:** Sign in to your TestMu AI account. Don't have an account, register for free. - -**Step 2:** Head to **Settings** and select **Organization Settings** from the dropdown. - -**Step 3:** Head to the **Authentication and SSO** tab and click and copy the **SCIM Base URL and Bearer Token** option. - -**Step 4:** Sign in to the [Okta](https://developer.okta.com). Select Enterprise Applications, then select Applications tab. - -**Step 5:** In the applications list, select application used for TestMu AI SSO setup - -**Step 6:** Select the Provisioning tab. - -**Step 7:** Under the Integration section, input your TestMu AI SCIM Base URL-`https://auth.lambdatest.com/api/scim` and Bearer Token retrieved earlier in Step 3. - -**Unique identifier field for users** field must be set to **userName** -Click Test Connection to ensure Okta can connect to TestMu AI. If the connection fails, ensure your TestMu AI account has Admin permissions and try again. - -**Step 8:** Now navigate to App tab - -**Step 9:** Review the User Attribute mappings: - -userName, Active, name.givenName, name.familyName are required attributes - -**Creating Custom Attributes** - -For creating custom attribute in Okta, go to Directory > Profile Editor > Add Attribute > [Create Custom Attribute](https://help.okta.com/en-us/content/topics/users-groups-profiles/usgp-add-custom-user-attributes.htm) - -**`urn:ietf:params:scim:schemas:extension:LambdaTest:2.0:User:OrganizationRole`**: - -**`urn:ietf:params:scim:schemas:extension:LambdaTest:2.0:User:LambdatestGroup`** : Applicable only if organisation has group support active) - -Enter Display Name as per your choice -Enter Variable name as **OrganizationRole** -Enter External name as **OrganizationRole** -Enter External namespace as **urn:ietf:params:scim:schemas:extension:TestMu AI:2.0:User** - -Set the data type as **String** -Select Enum as **Yes** -Enter the values as **Admin, Guest, User** - -Also if you want to assign this attribute at a okta group level choose AttributeType as Group - -**Step 10:** To enable the Okta provisioning service for TestMu AI, set Create Users, Update User Attributes and Deactivate Users to enabled - -## Pushing Groups from Okta - -Once user provisioning is working, you can push Okta groups to TestMu AI for automatic team, concurrency group, or sub-org assignment. - -**Prerequisites** -Group Provisioning must be enabled for your org. Contact **24/7 chat support** to activate it. - -**Step 1:** In Okta, go to your TestMu AI application > **Push Groups** tab. - -**Step 2:** Click **Push Groups** > choose **Find groups by name** or **Find groups by rule**. - -**Step 3:** Search for or select the groups you want to push, then click **Save**. - -**Step 4:** Okta will immediately push the group and its members to TestMu AI via SCIM. - -**Step 5:** In TestMu AI, go to **Settings** > **Organization Settings** > **SCIM Group Provisioning** to view the pushed groups and configure mappings. - -### Setting Roles on Okta Groups - -To assign TestMu AI roles (Admin, User, Guest) to all members of an Okta group: - -1. In your Okta app's **Push Groups** settings, the `LambdatestRoles` attribute is sent automatically if configured -2. Alternatively, set `LambdatestRoles` in the SCIM group extension: `urn:ietf:params:scim:schemas:extension:LambdaTest:2.0:Group` -3. The role applies to **all** members of the group. When a user is in multiple groups, the **highest** role wins (Admin > User > Guest) - -### What Happens After Pushing - -| Okta Action | TestMu AI Effect | -|---|---| -| Push a group | Group created, mapping rules evaluated, members synced | -| Add member to group | Member added to all mapped TestMu AI entities | -| Remove member from group | Member removed (if no other group maps them there), role recomputed | -| Rename group in Okta | Group renamed, mapped entity renamed to match, rules re-evaluated | -| Unlink/delete pushed group | Group soft-deleted, members safely unassigned, roles recomputed | - -> For details on mapping, conflicts, and rules, see the [SCIM Provisioning guide](/support/docs/scim/#group-provisioning). - -> That's all you need to know about TestMu AI SCIM Auto User Provisioning with Okta. In case you have any questions please feel free to reach out to us via the **24/7 chat support** or email us over [support@testmuai.com](mailto:support@testmuai.com). diff --git a/static/docs/one-click-migration-from-qtest.md b/static/docs/one-click-migration-from-qtest.md deleted file mode 100644 index dbbb3a9ab..000000000 --- a/static/docs/one-click-migration-from-qtest.md +++ /dev/null @@ -1,113 +0,0 @@ -# One Click Migration from qTest - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI Test Manager supports one-click migration from qTest, allowing you to import your entire test library — including projects, test cases, folder structures, custom fields, attachments, and linked Jira requirements — directly into Test Manager without manual data entry. - -## Key Benefits - -- **Zero manual effort** — Projects, test cases, steps, and folder structures are automatically migrated. -- **Folder hierarchy preserved** — qTest module and folder ordering is carried over into Test Manager. -- **Custom fields carried over** — Custom fields are detected and created automatically, including dropdowns, multi-selects, user fields, checkboxes, and more. -- **Attachments migrated** — File attachments at the test case and step level are downloaded from qTest and stored in LambdaTest. Inline base64 images in step content are uploaded and served with auto-refreshed links. -- **Jira requirement links preserved** — Linked Jira issues on test cases are migrated (requires Jira integration to be configured in TestMu AI). - -## Prerequisites - -#### TestMu AI Account -An active TestMu AI account with access to Test Manager. - -#### qTest Account -An active qTest account with read access to the projects you want to migrate. - -#### qTest Instance URL -Your qTest instance URL (e.g., `https://yourorg.qtestnet.com`). - -#### qTest API Token -To obtain your API token: -1. Log in to your qTest instance. -2. Navigate to your profile or account settings. -3. Generate a new API Bearer token, or copy an existing one. - -#### Jira Integration in TestMu AI (Optional) -To migrate linked Jira requirements, the corresponding Jira instance must be integrated with TestMu AI. Without this, Jira links will not be carried over. See [TestMu AI Jira Integration](/support/docs/jira-integration/#how-to-establish-integration-with-jira-from-your-lambdatest-account) for setup steps. - -#### Permissions -- Your qTest account must have read access to the projects being migrated. -- The API token must have permission to access test cases, modules, and attachments. - -## Step-by-Step Migration Guide - -### Step 1: Open the Migration Tool -1. Log in to your TestMu AI account. -2. Navigate to Test Manager Projects from the left sidebar. -3. Click the **Import data** option on the Project Listing page. -4. Select **qTest** as the migration source from the Import from dropdown. - -### Step 2: Enter Your Credentials -Enter your **qTest Instance URL** and **qTest API Token**, then click **Continue**. The system validates your credentials. If either is invalid, an error message specifies what needs to be corrected. - -### Step 3: Select Projects -After successful validation, a list of your accessible qTest projects is displayed. Select the project(s) you want to migrate — multiple projects can be selected at once — and click **Start Migration**. - -### Step 4: Monitor Progress -A progress bar shows real-time migration status. You can navigate away at any time — the migration continues in the background and survives page reloads. An email notification is sent when the migration completes or fails. - -### Step 5: Review the Results -1. Migrated projects appear on the Projects listing page in Test Manager, tagged **"qTest Migrated"**. -2. Navigate to Test Cases in any migrated project. -3. Verify that test cases, folder structure, steps, custom fields, attachments, and linked issues are imported correctly. - -## What Gets Migrated - -| qTest | Test Manager | -|---|---| -| Project | Project (name deduplicated with a numeric suffix if a project with the same name already exists in the org) | -| Modules / folder tree | Folder hierarchy, preserving qTest's folder ordering | -| Test cases | Test cases — title, description, preconditions, priority, type, and status | -| Test steps | Steps with description and expected outcome | -| Called / shared test cases | Expanded inline into the calling test case's steps | -| qTest ID (e.g. `TC-15`) | Stored as the test case's external ID and added as a tag | -| Custom fields | Org-level custom fields, created automatically if absent (see field type mapping below) | -| Attachments (test case and step level) | Downloaded from qTest and stored in LambdaTest | -| Jira requirement links | Linked issues on migrated test cases (requires Jira integration in TestMu AI) | -| Required-field flags | Preserved on created custom fields | - -## What Does Not Get Migrated - -| Entity | Reason | -|---|---| -| Test runs, executions, results, and test plans | Execution-layer data; only test-design data is migrated | -| Defects and defect-scoped fields | Belong to execution data, which is out of scope | - -## Troubleshooting - -| Symptom | Likely cause and fix | -|---|---| -| Progress stuck at 99% for a while | Final bookkeeping is in progress, or the source project is still growing. Wait for the completion email. | -| Folder named `X 1` instead of `X` | Duplicate folder names exist in the source project. Test Manager requires unique names; duplicates are automatically renamed with a numeric suffix. | -| Test case shows raw "call step" text | The referenced shared or called test case could not be fetched, or is called more than once by the same test case. The fallback text from the original step is used. | -| Custom fields not appearing | Custom fields are created during migration. Navigate to Test Manager Settings > Custom Fields to verify they exist and are enabled for the project. | - -## FAQ - -**Can I migrate multiple qTest projects at once?** Yes. Select multiple projects in Step 3. Each project is migrated into its own separate Test Manager project. - -**Will migration affect my qTest data?** No. The migration is read-only on the qTest side. Your projects, test cases, and all other data remain untouched. - -**What happens if a project with the same name already exists in Test Manager?** The migrated project name is automatically suffixed with a number (e.g., `My Project 1`) to avoid conflicts. - -**Can I run multiple migrations simultaneously?** Each user can run one active qTest migration at a time. Start a new migration only after the current one completes. - -## Known Limitations - -| Limitation | Details | -|---|---| -| Test-design data only | Test runs, executions, results, cycles, and test plans are not migrated | -| Duplicate custom-field values not supported | Field values are get-or-created; case-insensitive duplicates (e.g. `High` vs `high`) reuse the first created value | -| Duplicate names renamed | Folders and projects with duplicate names are suffixed with a number (`X`, `X 1`, …) | -| Field matching by name | Custom fields are matched to existing org fields by name; if a field with the same name already exists, it is reused as-is and its type is not changed | -| Default custom-field values not applied | Org-level default values are not applied to migrated test cases; qTest values are used as the source of truth | -| One import per user at a time | A new migration can only be started after the current one completes | -| Progress caps at 99% | The progress bar stays at 99% during final bookkeeping; this is expected | -| Freeze the source during migration | Avoid modifying the qTest project while the migration is running to ensure a consistent import | diff --git a/static/docs/one-click-migration-from-testrail.md b/static/docs/one-click-migration-from-testrail.md deleted file mode 100644 index 8f559e6e1..000000000 --- a/static/docs/one-click-migration-from-testrail.md +++ /dev/null @@ -1,17 +0,0 @@ -# One Click Migration from TestRail - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Efficiently importing data from TestRail streamlines your test management process, allowing for seamless project integration. With the Quick Import feature, you can easily transfer your existing TestRail projects from Test Management in just a few steps. This guide will walk you through the process, ensuring a smooth and hassle-free import experience. - -1. To perform a Quick Import from TestRail, navigate to the **Projects** section and select the **Import Data** option. - -2. Fill in the necessary details such as **TestRail Email Address**, **TestRail Host Name**, and **TestRail API Key**, then click **Continue**. - -3. Choose single or multiple projects and click **Import Project**. Once completed, your selected TestRail projects will be imported into Test Management. - -4. You can check the import progress on the All Projects page, and you'll receive an email notification once the process is finished. - -5. The email will contain a **GO TO TEST MANAGER** button, which redirects you to the project section. - -6. The imported project(s) will appear at the top, where you can click on it for a detailed project analysis. diff --git a/static/docs/one-click-migration-from-xray.md b/static/docs/one-click-migration-from-xray.md deleted file mode 100644 index 976cf13d7..000000000 --- a/static/docs/one-click-migration-from-xray.md +++ /dev/null @@ -1,184 +0,0 @@ -# One Click Migration from X-Ray - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI Test Manager supports one-click migration from **X-Ray (Jira Cloud)**, allowing you to import your entire test library (including projects, test cases, folder structures, attachments, custom fields, and linked requirements) directly into Test Manager without manual data entry. - -## Key Benefits - -- **Zero manual effort**: All test cases, steps, attachments, and folder structures are automatically migrated. -- **Handles both Manual and BDD tests**: Manual step definitions and Gherkin/BDD scenarios are both fully supported. -- **Custom fields carried over**: Custom fields in your X-Ray tests are automatically detected and created in Test Manager. -- **Attachments included**: File attachments are downloaded from Jira and re-uploaded to Test Manager, including step-level references. -- **Real-time progress tracking**: A progress bar shows migration status so you know exactly where things stand. - -This migration supports **X-Ray Cloud (Jira Cloud)** only. X-Ray Server and Data Center are not supported at this time. - -## Prerequisites - -Before starting the migration, ensure you have the following ready. - -#### TestMu AI Account -- An active TestMu AI account with access to **Test Manager**. - -#### X-Ray Cloud Credentials - -You need your X-Ray Cloud API credentials (Client ID and Client Secret). Refer to the [X-Ray Global Settings - API Keys](https://docs.getxray.app/space/XRAYCLOUD/44568019/Global+Settings+-+API+Keys) documentation for detailed instructions. To obtain these: - -1. Log in to your Jira Cloud instance. -2. Navigate to **Apps > Manage your apps > X-Ray** (or go to X-Ray Settings). -3. Under **API Keys**, generate a new API key pair if you do not already have one. -4. Copy the **Client ID** and **Client Secret**. - -The Client Secret is shown only once. Store it securely before proceeding. - -#### Jira Cloud Credentials - -You also need Jira Cloud credentials to access project metadata and attachments: - -| Credential | Where to Find It | -|---|---| -| **Jira Host URL** | Your Jira instance URL, e.g., `https://yourcompany.atlassian.net` | -| **Email Address** | The email associated with your Atlassian account | -| **Jira API Token** | Generate from [Atlassian API Tokens](https://id.atlassian.com/manage-profile/security/api-tokens) | - -#### Jira Integration in TestMu AI - -To migrate linked Jira issues and requirements, you must have the corresponding Jira instance integrated with TestMu AI. Without this integration, linked issues will not be carried over during migration. Refer to the [Link Jira Issues with Test Manager](https://www.testmuai.com/support/docs/link-jira-issues-with-test-manager/) guide to set up the integration before starting the migration. - -#### Permissions -- Your Jira account must have **read access** to the project you are migrating. -- Your X-Ray API key must have permission to access test cases and test sets in the target project. - -## Step-by-Step Migration Guide - -### Step 1: Open the Migration Tool - -1. Log in to your **TestMu AI** account. -2. Navigate to **Test Manager Projects** from the left sidebar. -3. Click the **Import data** option on the Project Listing page. -4. Select **X-Ray Cloud** as the migration source from the **Import from** dropdown. - -### Step 2: Enter Your Credentials - -Fill in the following fields: - -| Field | Description | -|---|---| -| **Jira Host** | Your Jira Cloud instance URL (e.g., `https://yourcompany.atlassian.net`) | -| **Jira Email Address** | Your Atlassian account email | -| **Jira API Token** | API token generated from Atlassian | -| **X-Ray Client ID** | Client ID from X-Ray API Keys. Refer to the [X-Ray Global Settings - API Keys](https://docs.getxray.app/space/XRAYCLOUD/44568019/Global+Settings+-+API+Keys) documentation. | -| **X-Ray Client Secret** | Client Secret from X-Ray API Keys. Refer to the [X-Ray Global Settings - API Keys](https://docs.getxray.app/space/XRAYCLOUD/44568019/Global+Settings+-+API+Keys) documentation. | - -Click **Continue**. The system validates both your Jira and X-Ray credentials in parallel. If either set is invalid, an error message indicates which credentials need to be corrected. - -### Step 3: Select the Jira Project - -1. After successful validation, a list of your Jira projects containing X-Ray test cases is displayed. -2. Select the **project** you want to migrate from. You can select multiple projects at once. -3. Click **Start Migration**. - -### Step 4: Monitor Progress - -- A **progress bar** shows real-time migration status. -- The migration processes test cases in batches. Depending on the size of your test library, the complete process may take some time. - -- You can navigate away from the page and continue other work. The migration continues in the background. Once the migration is complete, you will receive an email notification. - -### Step 5: Review the Results - -Once migration completes: - -1. The migrated X-Ray projects will appear on the **Projects** listing page in Test Manager. -2. Navigate to the **Test Cases** section in any of your migrated projects. -3. Verify that your test cases, folder structure, and attachments have been imported correctly. -4. Check a few test cases to confirm that steps, custom fields, and linked requirements are intact. - -## What Gets Migrated - -The following entities and data are transferred from X-Ray to Test Manager during migration: - -| Entity | Details | -|---|---| -| **Test Cases** | All test cases from the selected project are imported with their complete definitions. | -| **Test Case Metadata** | Summary, description, priority, labels, and preconditions are mapped to their corresponding Test Manager fields. | -| **Manual Test Steps** | Each step's **Step Description** and **Expected Result** are migrated directly. Step ordering is preserved. | -| **BDD/Gherkin Scenarios** | Full Gherkin scenario text is extracted and stored under the BDD Scenarios template. | -| **Attachments** | Both test case-level and test step-level attachments are downloaded and re-uploaded to Test Manager. | -| **Rich Text Fields** | Rich text content (including embedded screenshots, tables, lists, and other supported formatting) is preserved. | -| **Custom Fields** | Custom fields are automatically detected in X-Ray and created in Test Manager with matching names. | -| **Linked Issues / Requirements** | Jira issue links and requirement associations are preserved. Requires [Jira integration](https://www.testmuai.com/support/docs/link-jira-issues-with-test-manager/) to be configured. | -| **X-Ray Test Key** | The original X-Ray test key is imported within tags, enabling quick reference and filtering in Test Manager. | - -X-Ray fields that cannot be directly mapped to system fields in Test Manager are automatically imported as custom fields. These may include Reporter, Assignee, Environment, Fix Version, Components, etc. - -**Test Data Migration** -Test Data from X-Ray steps is migrated as a single textarea custom field at the test case level, with textual data consolidated together. This is because Test Manager does not currently support a step-level Test Data field. - -## What Does Not Get Migrated - -The migration focuses exclusively on your **test library**: definitions, structure, and metadata. The following entities are not included: - -| Entity | Reason | -|---|---| -| **Test Runs** | Execution-layer entity that is not part of the test case definition. | -| **Test Execution Logs** | Execution history and pass/fail results are excluded by design. | -| **Datasets** | Datasets and their corresponding parameters are not imported. | -| **Standalone Preconditions** | Precondition text is captured within individual test cases, but standalone precondition objects are not migrated as independent reusable entities. | - -## Test Steps Handling - -### Manual Tests -Manual test cases are migrated with their full step definitions. Each step's **Step Description** and **Expected Result** fields are mapped directly. Step ordering is preserved, and step-level attachments (e.g., screenshots embedded in steps) are migrated as well. All X-Ray test step templates, except Gherkin, are mapped to the **Manual Test Steps** template in Test Steps of a Test Case. - -### BDD / Gherkin Tests -For BDD-type test cases in X-Ray, the full **Gherkin scenario** text (Given/When/Then) is extracted and stored in the test case as structured BDD content under the **BDD Scenarios** template in Test Steps. - -## Troubleshooting - -| Problem | Solution | -|---|---| -| **Invalid X-Ray credentials** | Verify your X-Ray Client ID and Client Secret. Re-generate them from X-Ray Settings if needed. Ensure you are using **X-Ray Cloud** credentials, not X-Ray Server/Data Center. | -| **Invalid Jira credentials** | Confirm you are using a **Jira API Token** (not your Jira password). Generate one at [Atlassian API Tokens](https://id.atlassian.com/manage-profile/security/api-tokens). Verify the email matches the account that owns the token. | -| **No projects found** | Verify your Jira account has access to at least one project containing X-Ray test cases. Check that X-Ray is installed and active on your Jira Cloud instance. | -| **Migration stalls or takes too long** | Large projects (thousands of test cases with attachments) may take 10–30 minutes. If it does not resume after 30 minutes, re-initiate the migration. | -| **Missing attachments** | Verify that the Jira API token has permission to download attachments from the source project. | -| **Custom fields not appearing** | Custom fields are auto-created during migration. Navigate to **Project Settings > Custom Fields** to verify they exist and are enabled. | - -## FAQ - -**Can I migrate multiple Jira projects?** - -Yes. Run the migration once per Jira project. Each migration imports test cases from one X-Ray project into one Test Manager project. Repeat the process for additional projects. - -**Will migration affect my X-Ray data?** - -No. The migration is **read-only** on the X-Ray side. Your X-Ray test cases, test sets, and all other data remain untouched. - -**Can I run the migration again if something goes wrong?** - -Yes. You can re-run the migration for the same project(s). Review the results after each run. - -**Are Gherkin/BDD scenarios preserved exactly as written?** - -Yes. The full Gherkin text (Feature, Scenario, Given/When/Then blocks) is extracted and stored in the Test Manager test case with formatting and step structure preserved. - -**What if I have test cases with no steps?** - -Test cases without steps are still migrated. The test case metadata (title, description, priority, labels, etc.) is imported, and the steps section will be empty. - -**Is X-Ray Server (Data Center) supported?** - -No. This migration currently supports **X-Ray Cloud** (on Jira Cloud) only. - -## Known Limitations - -| Limitation | Details | -|---|---| -| **X-Ray Cloud only** | X-Ray Server and Data Center are not supported | -| **No execution data** | Test Plans, Test Runs, and execution logs are excluded by design | -| **No incremental sync** | Migration is a one-time import, not a continuous sync | -| **Standalone Preconditions** | Precondition text is captured within test cases, but standalone Precondition objects are not migrated as independent reusable entities | -| **Test case comments** | Jira comments on test case issues are not migrated | -| **Execution environments** | X-Ray environment configurations are not migrated | diff --git a/static/docs/one-click-migration-from-zephyr-scale.md b/static/docs/one-click-migration-from-zephyr-scale.md deleted file mode 100644 index d20f8c325..000000000 --- a/static/docs/one-click-migration-from-zephyr-scale.md +++ /dev/null @@ -1,190 +0,0 @@ -# One Click Migration from Zephyr Scale - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI Test Manager supports one-click migration from **Zephyr Scale (Jira Cloud)**, allowing you to import your entire test library (including projects, test cases, folder structures, custom fields, and linked requirements) directly into Test Manager without manual data entry. - -## Key Benefits - -- **Zero manual effort**: All test cases, steps, and folder structures are automatically migrated. -- **Handles both Manual and BDD tests**: Manual step definitions and Gherkin/BDD scenarios are both fully supported. -- **Custom fields carried over**: Custom fields in your Zephyr Scale tests are automatically detected and created in Test Manager. -- **Rich text content preserved**: Embedded screenshots within rich text fields (description, preconditions, step descriptions, and expected results) are migrated. -- **Real-time progress tracking**: A progress bar shows migration status so you know exactly where things stand. - -This migration supports **Zephyr Scale (Jira Cloud)** only. Zephyr Enterprise and Data Center are not supported at this time. - -## Prerequisites - -Before starting the migration, ensure you have the following ready. - -#### TestMu AI Account -- An active TestMu AI account with access to **Test Manager**. - -#### Zephyr Scale Cloud Credentials - -You need your Zephyr Scale Cloud API access token. Refer to the [Zephyr Scale API Access Tokens Management](https://support.smartbear.com/zephyr/docs/en/rest-api/api-access-tokens-management.html) documentation for detailed instructions. To obtain this: - -1. Log in to your Jira Cloud instance. -2. Navigate to **Zephyr Scale** settings within your Jira instance. -3. Under **API Access Tokens**, generate a new token if you do not already have one. -4. Copy the **Access Token**. - -The Access Token may be shown only once. Store it securely before proceeding. - -#### Jira Cloud Credentials - -You also need Jira Cloud credentials to access project metadata: - -| Credential | Where to Find It | -|---|---| -| **Jira Host URL** | Your Jira instance URL, e.g., `https://yourcompany.atlassian.net` | -| **Email Address** | The email associated with your Atlassian account | -| **Jira API Token** | Generate from [Atlassian API Tokens](https://id.atlassian.com/manage-profile/security/api-tokens) | - -#### Jira Integration in TestMu AI - -To migrate linked Jira issues and requirements, you must have the corresponding Jira instance integrated with TestMu AI. Without this integration, linked issues will not be carried over during migration. Refer to the [Link Jira Issues with Test Manager](https://www.testmuai.com/support/docs/link-jira-issues-with-test-manager/) guide to set up the integration before starting the migration. - -#### Permissions -- Your Jira account must have **read access** to the project you are migrating. -- Your Zephyr Scale API token must have permission to access test cases and folders in the target project. - -## Step-by-Step Migration Guide - -### Step 1: Open the Migration Tool - -1. Log in to your **TestMu AI** account. -2. Navigate to **Test Manager Projects** from the left sidebar. -3. Click the **Import data** option on the Project Listing page. -4. Select **Zephyr Scale** as the migration source from the **Import from** dropdown. - -### Step 2: Enter Your Credentials - -Fill in the following fields: - -| Field | Description | -|---|---| -| **Jira Host** | Your Jira Cloud instance URL (e.g., `https://yourcompany.atlassian.net`) | -| **Jira Email Address** | Your Atlassian account email | -| **Jira API Token** | API token generated from Atlassian | -| **Zephyr Scale API Access Token** | API Access Token generated from Zephyr Scale settings. Refer to the [API Access Tokens Management](https://support.smartbear.com/zephyr/docs/en/rest-api/api-access-tokens-management.html) documentation. | - -Click **Continue**. The system validates both your Jira and Zephyr Scale credentials in parallel. If either set is invalid, an error message indicates which credentials need to be corrected. - -### Step 3: Select the Jira Project - -1. After successful validation, a list of your Jira projects containing Zephyr Scale test cases is displayed. -2. Select the **project** you want to migrate from. You can select multiple projects at once. -3. Click **Start Migration**. - -### Step 4: Monitor Progress - -- A **progress bar** shows real-time migration status. -- The migration processes test cases in batches. Depending on the size of your test library, the complete process may take some time. - -- You can navigate away from the page and continue other work. The migration continues in the background. Once the migration is complete, you will receive an email notification. - -### Step 5: Review the Results - -Once migration completes: - -1. The migrated Zephyr Scale projects will appear on the **Projects** listing page in Test Manager. -2. Navigate to the **Test Cases** section in any of your migrated projects. -3. Verify that your test cases and folder structure have been imported correctly. -4. Check a few test cases to confirm that steps, custom fields, embedded screenshots, and linked requirements are intact. - -## What Gets Migrated - -The following entities and data are transferred from Zephyr Scale to Test Manager during migration: - -| Entity | Details | -|---|---| -| **Test Cases** | All test cases from the selected project are imported with their complete definitions. | -| **Test Case Metadata** | Summary, description, priority, labels, and preconditions are mapped to their corresponding Test Manager fields. | -| **Manual Test Steps** | Each step's **Step Description** and **Expected Result** are migrated directly. Step ordering is preserved. | -| **BDD/Gherkin Scenarios** | Full Gherkin scenario text is extracted and stored under the BDD Scenarios template. | -| **Folders** | Zephyr Scale folder hierarchy is preserved and mapped to Test Manager folders. | -| **Rich Text Fields** | Rich text content (including embedded screenshots, tables, lists, and other supported formatting within description, preconditions, step descriptions, and expected results) is preserved. | -| **Custom Fields** | Custom fields are automatically detected in Zephyr Scale and created in Test Manager with matching names. | -| **Linked Issues / Requirements** | Jira issue links and requirement associations are preserved. Requires [Jira integration](https://www.testmuai.com/support/docs/link-jira-issues-with-test-manager/) to be configured. | -| **Zephyr Scale Test Key** | The original Zephyr Scale test key is imported within tags, enabling quick reference and filtering in Test Manager. | - -Zephyr Scale fields that cannot be directly mapped to system fields in Test Manager are automatically imported as custom fields. These may include Reporter, Assignee, Environment, Fix Version, Components, etc. - -**Test Data Migration** -Test Data from Zephyr Scale steps is migrated as a single textarea custom field at the test case level, with textual data consolidated together. This is because Test Manager does not currently support a step-level Test Data field. - -## What Does Not Get Migrated - -The migration focuses exclusively on your **test library**: definitions, structure, and metadata. The following entities are not included: - -| Entity | Reason | -|---|---| -| **Entity-Level Attachments** | General file attachments linked to the test case and steps are not imported via the API migration. Only inline screenshots within rich text fields descriptions, preconditions, and steps are supported. | -| **Test Case Owner** | The original "Owner" of the test case in Zephyr Scale is not imported. All migrated test cases will show the user performing the migration as the creator. | -| **Test Cycles / Test Runs** | Execution-layer entity that is not part of the test case definition. | -| **Test Execution Logs** | Execution history and pass/fail results are excluded by design. | -| **Datasets** | Datasets and their corresponding parameters are not imported. | - -## Test Steps Handling - -The migration supports all three Zephyr Scale test script types: **Step by Step**, **BDD-Gherkin**, and **Plain Text**. - -### Step by Step and Plain Text -Manual test cases are migrated with their full step definitions. Each step's **Step Description** and **Expected Result** fields are mapped directly. Step ordering is preserved, and any embedded screenshots within rich text fields are carried over. All Zephyr Scale **Step by Step** and **Plain text** test scripts are mapped to the **Manual Test Steps** template in Test Steps of a Test Case. - -### BDD / Gherkin -For BDD-type test cases in Zephyr Scale, the full **Gherkin scenario** text (Given/When/Then) is extracted and stored in the test case as structured BDD content under the **BDD Scenarios** template in Test Steps. - -### Call to Test Case (Recursive Steps) -If your Zephyr Scale steps reference other test cases using the **"Call to Test Case"** step type, the migrator recursively fetches and includes the steps from the referenced test case. This ensures that all linked step definitions are fully resolved and available in the migrated test case. - -## Troubleshooting - -| Problem | Solution | -|---|---| -| **Invalid Zephyr Scale credentials** | Verify your Zephyr Scale Access Token. Re-generate it from Zephyr Scale settings if needed. Ensure you are using a **Zephyr Scale Cloud** token, not a Zephyr Enterprise or Jira Data Center token. | -| **Invalid Jira credentials** | Confirm you are using a **Jira API Token** (not your Jira password). Generate one at [Atlassian API Tokens](https://id.atlassian.com/manage-profile/security/api-tokens). Verify the email matches the account that owns the token. | -| **No projects found** | Verify your Jira account has access to at least one project containing Zephyr Scale test cases. Check that Zephyr Scale is installed and active on your Jira Cloud instance. | -| **Migration stalls or takes too long** | Large projects (thousands of test cases) may take few hours depending on the size of embedded screenshots. If it does not gets stuck, re-initiate the migration. | -| **Custom fields not appearing** | Custom fields are auto-created during migration. Navigate to **Project Settings > Custom Fields** to verify they exist and are enabled. | - -## FAQ - -**Can I migrate multiple Jira projects?** - -Yes. Run the migration once per Jira project. Each migration imports test cases from one Zephyr Scale project into one Test Manager project. Repeat the process for additional projects. - -**Will migration affect my Zephyr Scale data?** - -No. The migration is **read-only** on the Zephyr Scale side. Your test cases, folders, and all other data remain untouched. - -**Can I run the migration again if something goes wrong?** - -Yes. You can re-run the migration for the same project(s). Review the results after each run. - -**Are Gherkin/BDD scenarios preserved exactly as written?** - -Yes. The full Gherkin scenario is extracted and stored in the Test Manager test case with formatting and step structure preserved. - -**What if I have test cases with no steps?** - -Test cases without steps are still migrated. The test case metadata (title, description, priority, labels, etc.) is imported, and the steps section will be empty. - -**Are Zephyr Enterprise and Zephyr on Jira Data Center supported?** - -No. This migration currently supports **Zephyr Scale** (on Jira Cloud) only. Zephyr Enterprise and Zephyr on Jira Data Center are not supported. - -## Known Limitations - -| Limitation | Details | -|---|---| -| **Zephyr Scale Cloud only** | Zephyr Enterprise and Data Center are not supported | -| **No execution data** | Test Cycles, Test Runs, and execution logs are excluded by design | -| **No incremental sync** | Migration is a one-time import, not a continuous sync | -| **Tag size** | Tags/Labels exceeding **48 characters** will be truncated | -| **Folder name length** | Folder names are limited to **100 characters**. Names exceeding this limit are automatically truncated | -| **Owner not imported** | The original test case owner in Zephyr Scale is not imported. All migrated test cases show the migration user as the creator | -| **Entity-level attachments** | General file attachments from the test case "Attachments" tab are not imported. Only inline attachments within rich text fields are supported | -| **Execution environments** | Zephyr Scale environment configurations are not migrated | diff --git a/static/docs/opsgenie-integration.md b/static/docs/opsgenie-integration.md deleted file mode 100644 index ab11a6fac..000000000 --- a/static/docs/opsgenie-integration.md +++ /dev/null @@ -1,99 +0,0 @@ -# OpsGenie Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -OpsGenie is a modern incident management platform by Atlassian that ensures critical alerts are never missed. It offers intelligent alerting, on-call scheduling, and escalation policies to help teams respond to incidents faster. OpsGenie aggregates alerts from monitoring tools and custom applications, routing them to the right people through multiple notification channels. - -With TestMu AI and OpsGenie integration, you can log incidents directly while performing [cross browser testing](https://www.testmuai.com/) of your websites (and web apps) from the TestMu AI platform to OpsGenie. - -## Integrating OpsGenie With TestMu AI - -1. Login to your TestMu AI account. To setup integrations, make sure you have admin or user level access. - -2. From the left menu, click **Integrations** then search for **OpsGenie** under Communication section and click on **Connect**. - -3. Select your **Region** (US or EU) based on your OpsGenie account location and enter your **OpsGenie API Key**. - -> -* **What is an OpsGenie API Key?** -The OpsGenie API Key is used to authenticate API requests to OpsGenie. API keys can be created from the OpsGenie Settings page. The API key grants access to create alerts, manage incidents, and interact with OpsGenie programmatically. - -> **Important:** The API key must have **Create and update** alert permissions enabled to allow TestMu AI to send alerts to OpsGenie. - -### Getting the OpsGenie API Key - -1. Sign in to your OpsGenie account and navigate to **Settings** from the top navigation bar. - -2. Click on **API key management** under the Settings section. - -3. Click on **Add new API key** button. - -4. Provide a name for your API key (e.g., "TestMu AI") and configure the access rights: - - Enable **Read** access - - Enable **Create and update** access - -5. Click **Add API key** and copy the generated API Key. - -6. Paste the generated OpsGenie API Key in the provided field on TestMu AI and click **Install**. - -The OpsGenie integration will be successfully connected with your TestMu AI account. - -### Editing the OpsGenie API Key - -If you need to update your API key, click on the **Edit** button in the Setup OpsGenie dialog. - -## Configuring OpsGenie Integration - -After connecting OpsGenie, you can configure notification preferences to customize which alerts are sent. - -1. Click on the three-dot menu next to the OpsGenie integration and select **Settings**. - -2. In the Configure OpsGenie panel, you can set up the following notification preferences: - - **Screenshot Test Completion Messages** - Get notified when screenshot tests complete - - **Build completion messages** - Receive alerts when builds finish - - **Test Fail messages** - Get alerted on test failures - - **Test Pass messages** - Get notified on test passes - - **Notification Time** - Set the notification timing - - **Linked Service Accounts** - Link specific service accounts for routing alerts - -3. To link service accounts, click on **Linked Service Accounts** and select the accounts you want to associate with OpsGenie alerts. - -4. Click **Confirm** to save your service account links. - -## Sharing OpsGenie Integration - -You can share the OpsGenie integration with other team members in your organization. - -> **Note:** You must have **Admin** access to your organization to share integrations. - -1. Click on the three-dot menu next to OpsGenie and select **Share**. - -2. Once shared, other team members will see the OpsGenie integration under **Shared Integrations**. - -## Viewing Alerts in OpsGenie - -Once the integration is set up, alerts from TestMu AI will automatically appear in your OpsGenie dashboard. - -1. Navigate to the **Alerts** section in your OpsGenie account to view all incoming alerts. - -2. Click on any alert to view detailed information including: - - Source IP and Integration details - - Responders and Owner Team - - Description with ticket information - - Priority level - -3. For test-related alerts, you'll find comprehensive test execution details including: - - Test ID and Test name - - Status (passed/failed) - - OS, Browser, and Version information - - Resolution details - - Direct link to view the test on TestMu AI - -## Uninstalling The OpsGenie Integration - -1. Login to your TestMu AI account and visit **Integrations** from the left menu. - -2. Find the OpsGenie integration, click on the three-dot menu and select **Remove**. - -> -For any queries, feel free to drop them at our **24x7 Chat Support** or you could also mail us directly at **[support@testmuai.com](mailto:support@testmuai.com)**. diff --git a/static/docs/outbound-phone-agent.md b/static/docs/outbound-phone-agent.md deleted file mode 100644 index 59f90bcde..000000000 --- a/static/docs/outbound-phone-agent.md +++ /dev/null @@ -1,39 +0,0 @@ -# Outbound Phone Agent Testing With TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -An outbound phone agent places calls. To test one, the Agent Testing Platform provisions a recipient number from an outbound pool, the agent under test calls it, and the platform answers and scores the conversation. Typical use cases are sales dialers, lead qualification, collections, and appointment reminders. - -Outbound testing shares the same two modes as inbound, pre-evaluation and post-evaluation, and the same features, with a few outbound-specific differences in pre-evaluation. - -## What Differs From Inbound Testing - -Outbound pre-evaluation changes five things compared to the inbound flow. - -- **The agent speaks first.** The platform answers the call, so the default first speaker is the agent, matching real outbound behavior. -- **Up to 7 scenarios per generation.** Outbound scenario generation produces up to 7 scenarios, versus 20 for inbound. -- **Caller profile selection.** Select an outbound caller profile when generating scenarios. -- **Outbound number pool.** The platform reserves recipient numbers from an outbound pool for the test calls. -- **Passive mode.** Listen to a live outbound call and score it without injecting into the conversation, for QA monitoring. - -## Outbound Number Pool - -Outbound test calls originate against numbers reserved from a dedicated pool. You can view the pool status and available numbers, view and manage the reservations held per suite, and clear reservations to release numbers when a run is done. - -## Everything Else Mirrors Inbound Testing - -Aside from the differences above, outbound testing is identical to inbound. Phone number management, voice and background-noise configuration, agent profiles, test suites, call execution, post-evaluation recording analysis, the go-live assessment, and scheduling all work the same way. - -For the full detail on each, see [inbound phone agent testing](/support/docs/inbound-phone-agent/). - -## Metrics - -Outbound agents are evaluated across the same 8 metric categories and 30+ individual metrics as inbound: conversation flow and interaction dynamics, accuracy and effectiveness, user experience and satisfaction, business operational metrics, audio voice quality, speech-to-text evaluation, validation results, and automated issue tags. - -For the full metric tables and thresholds, see the [inbound phone agent metrics](/support/docs/inbound-phone-agent/#metrics). - -## Related TestMu AI Guides - -- See how to [test an inbound phone agent](/support/docs/inbound-phone-agent/) for the shared features and full metrics. -- See the [phone agent testing overview](/support/docs/phone-agent/) for both testing modes. -- See how to [run these tests from the terminal](/support/docs/testmu-a2a-cli/) with the A2A CLI. diff --git a/static/docs/pagerduty-integration.md b/static/docs/pagerduty-integration.md deleted file mode 100644 index 67fd4e18b..000000000 --- a/static/docs/pagerduty-integration.md +++ /dev/null @@ -1,64 +0,0 @@ -# PagerDuty Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -PagerDuty is a bug tracking platform that uses machine learning and automation to notify clients of interrupts and failures. These flaws are discovered in real-time, informing the appropriate people to react to any issue as quickly as possible to avoid them in the future. - -With TestMu AI and PagerDuty integration, you can log issues directly while perform [cross browser testing](https://www.testmuai.com/) of your websites (and web apps) from the TestMu AI platform to PagerDuty. - -## Integrating PagerDuty With TestMu AI - -1. Login to your TestMu AI account. To setup integrations, make sure you have admin or user level access. - -2. From the left menu, click **Integrations** then **PagerDuty**. - -3. Enter your **PagerDuty Token**. - -> -* **What is a PagerDuty Token?** -The PagerDuty REST API supports authentication through an account or user API token. Account API tokens have read-only access to all data on an account and can be granted full read, write, update, and delete access. User API tokens for PagerDuty accounts with Advanced Permissions have access to all the linked user account's data. - -### Getting the PagerDuty Token - -1. Sign in to your PagerDuty account and navigate to the Profile avatar. - -2. Select the **User Settings** tab. - -3. Click on **Create API User Token**. - -4. Provide the API token name and press **Create Token**. - -5. Copy the generated API User Token. - -Paste the generated PagerDuty API User Token in the provided field and click **Install**. - -The PagerDuty will be successfully integrated with your TestMu AI account. - -## One-Click Bug Logging With PagerDuty - -1. Select **Real Time Testing** section from the left menu. - -2. Enter the test URL, select browser, browser versions, operating system and resolution and click **START**. - -A cloud-based virtual machine will launch, allowing you to perform real-time browser testing of websites and web applications. - -3. While testing the websites, if you find an issue, click on Bug icon to capture the screenshot. - -4. You can use the in-built image editor tools to annotate issues or tasks you've identified. After annotating the bugs, click on **Mark As Bug**. - -5. A PagerDuty ticket will open up. Enter the details like Service, Assignee, Urgency, Summary and Description. Now click on **Create Issue**. - -Your bugs will be marked right away. - -6. Visit the PagerDuty dashboard, you'll find the logged issues under the Incidents section. - -7. Scroll down and you will find the data you submitted via TestMu AI would already be displayed in it. TestMu AI automatically attaches test environment data and relevant screenshots to tickets. - -## Uninstalling The PagerDuty Integration - -1. Login to your TestMu AI account and visit Integrations from the left menu. - -2. Visit 'My Integrations'and click on **REMOVE** adjacent to the PagerDuty. - -> -For any queries, feel free to drop them at our **24x7 Chat Support** or you could also mail us directly at **[support@testmuai.com](mailto:support@testmuai.com)**. diff --git a/static/docs/parallel-testing-with-playwright.md b/static/docs/parallel-testing-with-playwright.md deleted file mode 100644 index 3ca51f667..000000000 --- a/static/docs/parallel-testing-with-playwright.md +++ /dev/null @@ -1,119 +0,0 @@ -# Parallel Testing With Playwright - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI allows you to perform parallel testing with Playwright across 40+ browsers and OS to automate your several test cases simultaneously. You can run one test case across various browsers, or you can run multiple test case scenarios in the same browser with different browser versions. - -Shown below are the steps on running parallel tests with Playwright on the TestMu AI platform. - -1. Clone the [TestMu AI-Playwright GitHub repository](https://github.com/LambdaTest/playwright-sample) and switch to the cloned directory. - -```js -git clone https://github.com/LambdaTest/playwright-sample.git -cd playwright-sample-main -``` - -2. Ensure you have npm dependencies installed. - -3. Configure your TestMu AI authentication credentials. - -Once you are done with the above-mentioned steps, you can run your parallel tests with Playwright on TestMu AI. - -The below test script searches the term 'TestMu AI' on Bing. - -```js -const { chromium } = require('playwright') -const { expect } = require('@playwright/test') - -const parallelTests = async (capability) => { -console.log('Initialising test:: ', capability['LT:Options']['name']) - -const browser = await chromium.connect({ -wsEndpoint: `wss://cdp.lambdatest.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(capability))}` -}) - -const page = await browser.newPage() - -await page.goto('https://www.bing.com') - -const element = await page.$('[aria-label="Enter your search term"]') -await element.click() -await element.type('LambdaTest') -await element.press('Enter') -const title = await page.title() - -try { -expect(title).toEqual('LambdaTest - Search') -// Mark the test as completed or failed -await page.evaluate(_ => {}, `lambdatest_action: ${JSON.stringify({ action: 'setTestStatus', arguments: { status: 'passed', remark: 'Title matched' } })}`) -} catch { -await page.evaluate(_ => {}, `lambdatest_action: ${JSON.stringify({ action: 'setTestStatus', arguments: { status: 'failed', remark: 'Title not matched' } })}`) -} - -await browser.close() -} - -// Capabilities array for with the respective configuration for the parallel tests -const capabilities = [ -{ -'browserName': 'Chrome', // Browsers allowed: `Chrome`, `MicrosoftEdge`, `pw-chromium`, `pw-firefox` and `pw-webkit` -'browserVersion': 'latest', -'LT:Options': { -'platform': 'Windows 10', -'build': 'Playwright Sample Build', -'name': 'Playwright Sample Test on Windows 10 - Chrome', -'user': process.env.LT_USERNAME, -'accessKey': process.env.LT_ACCESS_KEY, -'network': true, -'video': true, -'console': true -} -}, -{ -'browserName': 'MicrosoftEdge', -'browserVersion': 'latest', -'LT:Options': { -'platform': 'Windows 8', -'build': 'Playwright Sample Build', -'name': 'Playwright Sample Test on Windows 8 - MicrosoftEdge', -'user': process.env.LT_USERNAME, -'accessKey': process.env.LT_ACCESS_KEY, -'network': true, -'video': true, -'console': true -} -}, -{ -'browserName': 'Chrome', -'browserVersion': 'latest', -'LT:Options': { -'platform': 'MacOS Big sur', -'build': 'Playwright Sample Build', -'name': 'Playwright Sample Test on MacOS Big sur - Chrome', -'user': process.env.LT_USERNAME, -'accessKey': process.env.LT_ACCESS_KEY, -'network': true, -'video': true, -'console': true -} -}] - -capabilities.forEach(async (capability) => { -await parallelTests(capability) -}) - -``` - -4. Pass the below command to run the test. - -``` -node playwright-parallel.js -``` - -## View your Playwright test results - -The TestMu AI Automation Dashboard is where you can see the results of your Playwright tests after running them on the TestMu AI platform. - -The below screenshot of TestMu AI Automation Dashboard shows the Playwright build on the left and the build sessions associated with the selected build on the right. - -On clicking the session name of the respective test, you can view the details of Playwright test session that you just executed. For example, the below screenshot shows a test execution details of Playwright test like Test Name, Test ID, selected configurations, test logs, basic info, input config, and test session video. diff --git a/static/docs/password-and-access-key-expiration-policy.md b/static/docs/password-and-access-key-expiration-policy.md deleted file mode 100644 index be2e34e5c..000000000 --- a/static/docs/password-and-access-key-expiration-policy.md +++ /dev/null @@ -1,53 +0,0 @@ -# Password and Access Key Expiration Policy - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI allows you to set an expiry policy to passwords and access keys for users in your organisation. This means that your password and access keys will expire after a certain number of days, and you will be required to change them or update them, respectively. - -> **Note:** Only organization administrators are able to view and manage the Password and Access Key Expiration Policy. - -## Why are Password and Access Key Expiry Important? - -Password expiration and access key expiration are important security features that can help protect your account from unauthorized access. If your password or access key expires, it will prevent unauthorized users from accessing your account, even if they have your username and old password. - -## Setup Password Expiry Policy - -By default, password expiration policy is set to **Never**. To change this, follow the below mentioned steps: - -**Step 1:** Go to the [LambdaTest](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard) Dashboard. - -**Step 2:** Click on the Settings -> **Organization Settings**, navigate to **Security** tab and click on the **Password Expiry** button - -**Step 2:** Click on the Edit Button to edit your Password Expiry duration. - -**Step 3:** Select the number of days after which your password will expire. Click on the **Save** button. - -**Step 4:** If you have selected any number of days, and if there are users who have already passed the timeline, they will be immediately logged out. But they will be able to reset their password. - -You can also view the password expiration policy for **All the Users** and when their password will expire. You can also add users to the **Excluded Users** list, whose password will never expire (For example, an admin user). You can filter the data based on their specific **Role** as well by clicking on the drop-down menu. - -You will receive a mail 3 days before the expiration of your password to set a new password. If this action is not performed before the expiration date, your password will expire. After expiration, you will receive a mail that your password has expired with a link to set your new password. Until this action is performed, you will not be able to access your account. - -## Setup Access Key Expiry Policy - -By default, access key expiration policy is set to Never. To change this, follow the below mentioned steps: - -**Step 1:** Go to the [LambdaTest](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard) Dashboard. - -**Step 2:** Click on the Settings -> **Organization Settings**, navigate to **Security** tab and click on the **Access Key Expiry** button - -**Step 2:** Click on the Edit Button to edit your Access Key Expiry duration. - -**Step 3:** Select the number of days after which your Access Key will expire. Click on the **Save** button. - -**Step 4:** If you have selected any number of days, and if there are users who have already passed the timeline, they will be immediately get unauthorized. - -> After the expiration of the access key, it will be regenerated automatically. - -**Step 5:** Select the user for whom you want to regenerate the access key, click on the three dots and then select the **Regenerate Key** button. - -You can also check the access key expiration policy for **All the Users** of the organization. You can filter the data based on the **User Type** as well by clicking on the drop-down menu. - -> **Excluded User** are those users for whom there are no aceess key expiration policy. - -You will receive a mail 3 days before the expiration of your access key. You can regenerate this key on your account dashboard. If this action is not performed before the expiration date, your access key will expire. After expiration, your access key will be regenerated automatically, and you will receive an email to get your new access key from the dashboard. For service accounts, admins will receive the mail regarding expiry, and the access keys for service accounts can be found on the members page in organization settings. diff --git a/static/docs/paymo-integration.md b/static/docs/paymo-integration.md deleted file mode 100644 index dc225cca0..000000000 --- a/static/docs/paymo-integration.md +++ /dev/null @@ -1,71 +0,0 @@ -# Paymo Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -> Paymo is an online project management application that helps your team work better by getting everyone on the same page. Preferred by freelancers, small and medium businesses across globe, Paymo is dedicated to helping business in their planning, task management, and integrated time tracking. TestMu AI integration with Paymo helps to manage your bugs in Paymo by moving them into a project through a single click effort. All the details you provide in TestMu AI like task list, assignee, title and description would automatically be presented in the project on Paymo. - -The TestMu AI Paymo integration helps you in filing issues to your project in Paymo directly from TestMu AI platform. With one-click integration you can push annotated issues to the project of your choice, assign it to the required team mate, and attach necessary screenshots. You can do all that while in the middle of a test session in TestMu AI platform. The fields populated by you when marking as bug through TestMu AI are displayed as information on the Paymo for a testing instance. - -**Paymo Integration with TestMu AI, like all of the integrations to 3rd party applications, is available for freemium as well as premium plan.** - -## How To Establish Integration With Paymo From Your TestMu AI Account? - -**Step 1:** Login to your TestMu AI account. You would need to have Admin or User level access to see and install integrations. - -**Step 2:** Select **‘Integrations’** from the left navigation menu bar. This will guide you to a screen where you will find a list of 3rd party applications, available to integrate with your TestMu AI account. - -**Step 3:** Click on **‘ADD’** under the block that says **‘Paymo’.** - -**Step 4:** Once you click on install, you will be redirected to the below screen. You will be asked to provide your **Paymo API Token** credentials in the highlighted fields to establish integration with TestMu AI. - -> **What is an API token?** -> When two different applications interact with each other, a server-to-server communication is established. API token is needed for authenticating user identity from one server to another, by fetching user-owned resources available on the server from where the communication is initiated. API token has been a necessity for maintaining security and data integrity of any application. They are just as vital to an end user as they are to an application. API tokens are relevant if an end user logs in an application through a 2-step verification process. - -**Step 5:** You can find your API token from your **Paymoapp** instance. If you have not generated one before, you can generate an API token by going to the **settings.** The settings icon will be visible once you hover the mouse on the bottom left side where your name and display image is placed. We have highlighted the part you need to hover in red. Click on the settings icon. - -**Step 6:** As you click on settings icon, go to **"Your Settings".** - -**Step 7:** Scroll down to the bottom of the page and you will find a section for **"APP PASSWORDS(API KEYS)".** Mention any input you like in the box & click on **‘Generate’.** - -**Step 8:** As you click on **‘Generate’,** your API key will be pop-up on your screen. Copy the API key now as it won’t be visible once you click **OK.** - -You can keep a note of this API token with you for future authentication. Although, in case you forget to do so, or you end up neglecting it due to some reason. Then there is no need to panic! You can create a new token by defining a new label. You can also **delete** a previously declared API Key, in case you feel that your API Key has been placed in some wrong hands. You can delete an API Key by hovering over the label and clicking on the **bin** icon. - -**Step 9:** Paste the API token into the field provided at TestMu AI & hit the **‘Next’** button. You are all set to experience **one-click bug** logging to share your issues directly from your TestMu AI account to your project on Paymo. - -That’s it! Go to Integrations again and you will be able to notice a **green tick** indicating that Paymo is successfully **installed.** - -## How To Log Your First Bug Through Paymo Integration? - -**Step 1:** Go for any of the test from the left navigation menu. For demo, we will be taking **"Real Time Test"** option. - -**Step 2:** Present a URL of the web-app you need to test in the dialog box. After that, select any configuration for browser and operating system of your choice & click **‘Start‘.** - -**Step 3:** After the VM is launched and operable. You can perform testing on your web-app for finding bugs. If a bug gets revealed, then you need to click on the **camera icon** from the left panel for [capturing a screenshot](https://www.testmuai.com/full-page-screen-capture) of the same. We have highlighted that controller button in the below image. - -**Step 4:** After a screenshot is captured, you can annotate any issue or a task with an **in-built image editor.** Once you are done highlighting the bug, click on the button that says **"Mark as Bug".** - -**Step 5:** After clicking on **"Mark as Bug"** button a Paymo specific form would open up. Fill the fields as per your requirement. - -> - You can select which **project** should the ticket go under, along with the **Issue type.** - -- You can set which **Task List** should the bug go under. -- You can assign the bug to a colleague by populating the field **‘Member‘.** -- You can mention the **Title** as well to easily organize your bugs. -- You also get to post a **description** to help relate the cause of the issue or the task. - -**Step 6:** Click on **"Create Bug"** & observe it being successfully marked through a single click effort. Once you hit the button, you will get **prompt messages** on top of your Virtual Machine indicating the progress of bug logging. You will get a prompt message in few seconds mentioning **"Bug successfully marked"** indicating that the issue has been successfully pushed to the respective workspace. - -**Step 7:** Visit your Paymo dashboard. You will be able to notice the logged issue right away by going to the particular project and task list you assigned it to! All the data you provided through TestMu AI would already be presented in it. TestMu AI automatically includes test environment details and related screenshots in the ticket as attachments. - -## How To REMOVE Paymo Integration? - -> You can work with one integration at a time. So if you would want to integrate to a similar 3rd party application, then you would have to **remove** your current integration. Here is how you can do that. - -**Step 1:** Login to your TestMu AI account. - -**Step 2:** Select **‘Settings’** from the left navigation menu bar & click on **‘Integrations’.** This will guide you to a screen where you will find 3rd party applications, available to integrate with your TestMu AI account. - -**Step 3:** Click on **‘REMOVE’.** You can find the **remove** button right under the **‘Paymo’** block which would be highlighted with a **green tick.** - -> That was all you need to know for TestMu AI + Paymo Integration. Increase your productivity with our integrations. If you still have any questions for us, please feel free to let us know. Our experts are always available on **chat** to help you out with any roadblock regarding our product. Happy testing! diff --git a/static/docs/perform-selenium-automation-on-headless-browsers.md b/static/docs/perform-selenium-automation-on-headless-browsers.md deleted file mode 100644 index f27170d95..000000000 --- a/static/docs/perform-selenium-automation-on-headless-browsers.md +++ /dev/null @@ -1,202 +0,0 @@ -# Perform Headless Browser Testing on Cloud Selenium Grid - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI allows you to run headless browser tests on its cloud-based Selenium Grid. This guide shows you how to enable and run headless browser testing on the TestMu AI platform. - -## Enabling Headless Browser Testing on TestMu AI - -Set the headless desired capability to run tests without a graphical user interface. - -For headless browser testing on TestMu AI, enable the desired capability for headless browsing. You can enable this capability using either of the two methods below: - -### Enable Headless Browser Testing on Desired Capabilities Generator - -Visit the [Desired Capabilities Generator](https://www.testmuai.com/capabilities-generator/). Under the **Browser Specific Capabilities**, find the toggle button to enable **Headless** browsing. Switch the button to true. - -### Enable Headless Browser Testing via Code - -Add the headless capability directly in your test script code. - -While writing the desired capabilities in your code, enable headless browser testing by adding the below syntax: - -`capabilities.setCapability("headless",true);` - -Below is the complete code for sample desired capabilities in various languages: - -**Java** - -```java -DesiredCapabilities capabilities = new DesiredCapabilities(); -capabilities.setCapability("browser", "Chrome"); -capabilities.setCapability("version", "86"); -capabilities.setCapability("platform", "MacOS Catalina"); -capabilities.setCapability("build", "your build name"); -capabilities.setCapability("name", "your test name"); - -// Capability setting to enable Headless browser testing -caps.setCapability("headless",true); -``` - -**C#** - -```csharp -DesiredCapabilities capabilities = new DesiredCapabilities(); -capabilities.SetCapability("user","username") -capabilities.SetCapability("accessKey","access_key") -capabilities.SetCapability("build", "your build name"); -capabilities.SetCapability("name", "your test name"); -capabilities.SetCapability("platform", "MacOS Catalina"); -capabilities.SetCapability("browserName", "Chrome"); -capabilities.SetCapability("version","86.0"); - -// Capability setting to enable Headless browser testing -capabilities.setCapability("headless",true); -``` - -**PHP** - -```php -$capabilities = array( -"build" => "your build name", -"name" => "your test name", -"platform" => "MacOS Catalina", -"browserName" => "Chrome", -"version" => "86.0", - -"headless" => true -) -``` - -**Ruby** - -```ruby -capabilities = Selenium::WebDriver::Remote::Capabilities.new -capabilities["build"] = "your build name" -capabilities["name"] = "your test name" -capabilities["platform"] = "MacOS Catalina" -capabilities["browserName"] = "Chrome" -capabilities["version"] = "86.0", - -capabilities["headless"] = true -``` - -**JavaScript** - -```javascript -var capabilities = { -"build" : "your build name", -"name" : "your test name", -"platform" : "MacOS Catalina", -"browserName" : "Chrome", -"version" : "86.0", - -"headless" : true -} -``` - -**Python** - -```py -capabilities = { -"build" : "your build name", -"name" : "your test name", -"platform" : "MacOS Catalina", -"browserName" : "Chrome", -"version" : "86.0", - -"headless" : True -} -``` - -## Perform Headless Browser Testing on TestMu AI - -Run a headless browser test using the TestNG framework in Java. - -Below is an example to perform headless browser testing. For demo purposes, this uses the TestNG framework with Java. - -```java -package com.lambdatest; - -import java.net.MalformedURLException; -import java.net.URL; -import org.openqa.selenium.By; -import org.openqa.selenium.remote.DesiredCapabilities; -import org.openqa.selenium.remote.RemoteWebDriver; -import org.testng.Assert; -import org.testng.annotations.AfterSuite; -import org.testng.annotations.BeforeSuite; -import org.testng.annotations.Test; - -public class HeadlessAutomation { - -private static RemoteWebDriver driver; -private static String Status="failed"; - -@BeforeSuite -public void setup() throws MalformedURLException { - -try { -String username = System.getenv("LT_USERNAME"); -String authkey = System.getenv("LT_ACCESS_KEY"); -String hub = "@hub.lambdatest.com/wd/hub"; - -DesiredCapabilities caps = new DesiredCapabilities(); -caps.setCapability("browser", "Chrome"); -caps.setCapability("version", "86"); -caps.setCapability("platform", "MacOS Catalina"); -caps.setCapability("build", "Headless Automation"); -caps.setCapability("name", "Headless Automation"); -caps.setCapability("network", true); -caps.setCapability("visual", true); -caps.setCapability("video", true); -caps.setCapability("console", true); - -// Capability setting to enable Headless browsing -caps.setCapability("headless",true); - -System.out.println("Desired Caps: " + caps); -driver = new RemoteWebDriver(new URL("https://" + username + ":" + authkey + hub), caps); -} -catch(Exception e) -{ -System.out.println(e); -} -} - -@Test -public static void testAssertion() { - -try { -driver.get("https://opensource-demo.orangehrmlive.com/"); //define the url - -String pageTitle = driver.getTitle(); //get the title of the webpage -System.out.println("The title of this page is ===> " +pageTitle); -Assert.assertEquals("OrangeHRM", pageTitle); //verify the title of the webpage - -driver.findElement(By.id("txtUsername")).clear();//clear the input field before entering any value -driver.findElement(By.id("txtUsername")).sendKeys("Admin");//enter the value of username -driver.findElement(By.id("txtPassword")).clear(); -driver.findElement(By.id("txtPassword")).sendKeys("admin123");//enter the value of password -driver.findElement(By.id("btnLogin")).click(); //click Login button -System.out.println("Successfully logged in"); -Status = "passed"; -} -catch(Exception e) -{ -Status = "failed"; -} -} - -@AfterSuite -public void tearDown() { -driver.executeScript("lambda-status=" + Status); -driver.quit(); -} - -} -``` - -After you execute the above code, the test runs on TestMu AI successfully. View the status of the test on your [Automation dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/timeline). - -That's it folks! If you have any doubt or questions, feel free to contact our experts at **24/7 Customer chat support** or mail us at [support@testmuai.com](mailto:support@testmuai.com). Happy testing! 🙂 diff --git a/static/docs/performance-report.md b/static/docs/performance-report.md deleted file mode 100644 index 4eb6cb190..000000000 --- a/static/docs/performance-report.md +++ /dev/null @@ -1,43 +0,0 @@ -# Performance Report - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -[LT Browser](https://www.testmuai.com/lt-browser) now integrates with Google Lighthouse that allows you to view the performance report of your web-page instantly. Along with testing the website's responsiveness, you can now test your website performance statistics across 50+ pre-installed device viewports. The Performance Report allows you to view the score of: - -* Desktop and Mobile -* Performance, Accessibility, Best Practices, SEO, and Progressive Web App -* Page Load Time and Browser Processing Time - -To get started, you need to download the executable file from here. - -Download LT Browser For Windows - -Download LT Browser For macOS - -Download LT Browser For Linux - -## How To View Performance Report On LT Browser? - -Make sure you've installed the LT Browser, you will need to login to the LT Browser through your TestMu AI login credentials. - -**Step 1:** Open the LT Browser. Select a device from the device List. For the demo, we are selecting **'Galaxy Note 9'** and **'13-inch Macbook Air'**. - -**Step 2:** Enter the **'URL'** in the address bar and press the **'Enter'** button. - -**Step 3:** Once the website is loaded over the selected device, click on **'Performance Report'** button on the top-right. - -Once you click on the Performance Report button, a new tab of Performance Report would open up. You'll find all the performance statistics of the entered website and selected device. You can view: - -#### 1. Desktop Score And Mobile Score - -Based on the performance of your website over different browsers, the desktop and mobile scores are generated. The score has three fields **Poor**, **Average**, and **Medium**. - -#### 2. Performance Metrics - -You can view the performance metrics of desktop and mobile by navigating to the desktop icon and mobile icon under the section Performance Metrics from the left. - -#### 3. Runtime Settings - -In runtime settings, you'll find the URL of the entered website, time, and date. - -That is all about the LT browser Performance Report. Don't forget to share you experience or questions with us. You can drop us an email through [support@testmuai.com](mailto:support@testmuai.com) or you could wave us a hello over our **24/7 chat support** . Happy testing! diff --git a/static/docs/performance-tips.md b/static/docs/performance-tips.md deleted file mode 100644 index ad3ed0b0c..000000000 --- a/static/docs/performance-tips.md +++ /dev/null @@ -1,115 +0,0 @@ -# Performance Tips - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Running Selenium tests on a cloud grid introduces network latency compared to local execution. These tips help you reduce total execution time on TestMu AI. - -## Disable Unnecessary Logging - -Turn off visual, network, and console logging to reduce per-command overhead. - -TestMu AI runs complementary services that capture screenshots after every command, record video, and collect network and console logs. Each service adds processing time. If you do not need these logs, disable them through capabilities: - -```java -ChromeOptions browserOptions = new ChromeOptions(); -browserOptions.setPlatformName("Windows 10"); -browserOptions.setBrowserVersion("latest"); -HashMap ltOptions = new HashMap(); -ltOptions.put("build", "your build name"); -ltOptions.put("name", "your test name"); -ltOptions.put("console", false); // Disable console logs -ltOptions.put("network", false); // Disable network logs -ltOptions.put("visual", false); // Disable step-by-step screenshots -browserOptions.setCapability("LT:Options", ltOptions); -``` - -Console, network, and visual capabilities are `false` by default. Only set them explicitly if a previous configuration enabled them. - -## Run Tests in Parallel - -Execute multiple tests simultaneously to reduce total build time. - -Sequential test execution multiplies wait time across every test. Use your framework's parallel execution feature (TestNG `parallel="methods"`, pytest-xdist, Mocha `--parallel`) and set the desired concurrency in your TestMu AI plan. This is the single most impactful change for reducing build duration. - -## Use the Latest Browser Version - -Set `browserVersion` to `latest` instead of pinning a specific version number. - -Pinning old browser versions can cause delays if TestMu AI needs to spin up a less common environment. Using `latest` ensures your tests run on the most readily available browser instances: - -```java -browserOptions.setBrowserVersion("latest"); -``` - -## Set an Appropriate Idle Timeout - -Configure `idleTimeout` to balance between premature timeouts and wasted session time. - -The `idleTimeout` capability controls how long TestMu AI waits before closing an idle session. Setting it too high wastes concurrency slots. Setting it too low causes premature session termination during long page loads or waits. - -```java -ltOptions.put("idleTimeout", 120); // seconds - adjust based on your longest expected wait -``` - -A value between 60 and 150 seconds works well for most test suites. - -## Use a Smaller Resolution - -Set a lower screen resolution if your tests do not require visual validation. - -Higher resolutions increase rendering time. If you are not running visual or screenshot-based tests, use a standard resolution: - -```java -ltOptions.put("resolution", "1024x768"); -``` - -## Minimize Page Loads - -Reduce the number of `driver.get()` calls to cut down on navigation overhead. - -Each `driver.get()` call triggers a full page load over the network. Combine test steps that can share a page state, and avoid redundant navigations between steps. - -## Use Explicit Waits Instead of Thread.sleep() - -Replace fixed sleep calls with explicit waits to proceed as soon as the condition is met. - -`Thread.sleep()` always waits the full duration, even if the element is ready in milliseconds. Explicit waits poll for the expected condition and continue immediately: - -```java -// Avoid this -Thread.sleep(5000); - -// Use this instead -WebDriverWait wait = new WebDriverWait(driver, Duration.ofSeconds(10)); -wait.until(ExpectedConditions.visibilityOfElementLocated(By.id("element-id"))); -``` - -## Close Sessions Properly - -Always call `driver.quit()` to release the session and free up concurrency slots. - -If a test exits without calling `driver.quit()`, the session stays open until the idle timeout expires. This wastes concurrency and slows down queued tests. - -```java -@AfterMethod -public void tearDown() { -if (driver != null) { -driver.quit(); -} -} -``` - -## Troubleshoot Latency - -Run a traceroute if performance remains slow after applying the above tips. - -If you still experience latency after applying these optimizations, reach out to us on our **in-app chat support** or email [automationsupport@testmuai.com](mailto:automationsupport@testmuai.com). Include a screenshot of the traceroute output: - -For Windows: -``` -tracert hub.lambdatest.com -``` -For Linux/Mac: -``` -traceroute hub.lambdatest.com -``` diff --git a/static/docs/phone-agent.md b/static/docs/phone-agent.md deleted file mode 100644 index 4d5ad7faf..000000000 --- a/static/docs/phone-agent.md +++ /dev/null @@ -1,42 +0,0 @@ -# Phone Agent Testing With TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The Agent Testing Platform tests phone voice agents by placing real telephone calls, not simulations. An AI-powered simulated caller follows a scenario, the platform records the full conversation, and it scores the call across 30+ call quality metrics. It covers inbound support lines, IVR flows, and outbound dialers. - -Phone agents come in two directions, inbound and outbound, and each has its own workflow. This page covers what is shared across both. See the dedicated guides for [inbound phone agent testing](/support/docs/inbound-phone-agent/) and [outbound phone agent testing](/support/docs/outbound-phone-agent/). - -## Features - -Phone testing runs in two modes, both scored with the same 30+ metrics. - -**Live test calls.** The platform places a real call and drives the conversation with a simulated caller. During the call it tracks duration in real time, produces a speaker-identified transcript, and detects DTMF tones for menu navigation. - -**Recording analysis.** Upload batches of recorded production calls (MP3 or WAV) and the platform scores them with the same metrics, so you can monitor real production quality without placing new calls. - -**Voice and noise simulation.** To match production conditions, the platform simulates the voice and acoustic environment per scenario: - -- **Voice profiles:** 200+ voices across providers, with accents and speech speeds. -- **Background noise:** 15 presets such as café, street, call center, and poor cellular connection. -- **Response timing:** a configurable delay from 0.5 to 5.0 seconds, and a maximum call duration from 60 to 1800 seconds. - -**Shared capabilities.** Phone agents also support test suites, agent profiles, a Green, Yellow, or Red go-live assessment, and cron-based scheduled runs. See the [inbound](/support/docs/inbound-phone-agent/) and [outbound](/support/docs/outbound-phone-agent/) guides for the direction-specific workflow. - -## Metrics - -Every call is scored across 8 metric categories with 30+ individual metrics: conversation flow and interaction dynamics, accuracy and effectiveness, user experience and satisfaction, business operational metrics, audio voice quality, speech-to-text evaluation, validation results, and automated issue tags. Key metrics include First Call Resolution, CSAT, containment rate, intent recognition accuracy, and speech-to-text accuracy across accents and noise. - -For the full metric tables and thresholds, see the [inbound phone agent metrics](/support/docs/inbound-phone-agent/#metrics). - -## Run a Phone Agent Test - -Phone tests run from the dashboard or the CLI, driven by scenarios grouped into suites. Pick the direction that matches your agent. - -- To test an agent that answers calls, see [inbound phone agent testing](/support/docs/inbound-phone-agent/). -- To test an agent that places calls, see [outbound phone agent testing](/support/docs/outbound-phone-agent/). -- To run calls from the terminal, see how to [test phone agents with the CLI](/support/docs/testmu-a2a-cli/). - -## Related TestMu AI Guides - -- See how to [test an inbound phone agent](/support/docs/inbound-phone-agent/) step by step. -- See how to [test an outbound phone agent](/support/docs/outbound-phone-agent/) step by step. diff --git a/static/docs/php-with-selenium-running-php-automation-scripts-on-testmu-selenium-grid.md b/static/docs/php-with-selenium-running-php-automation-scripts-on-testmu-selenium-grid.md deleted file mode 100644 index 46f47c5a3..000000000 --- a/static/docs/php-with-selenium-running-php-automation-scripts-on-testmu-selenium-grid.md +++ /dev/null @@ -1,172 +0,0 @@ -# Selenium With PHP - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Run your PHP Selenium tests on the TestMu AI cloud grid across 3000+ browser and OS combinations. The setup is the same for every framework: you connect to the grid and pass your capabilities. This guide covers that shared flow once, then gives you a per-framework quickstart in the tabs below. - -## Prerequisites - -1. [Create a TestMu AI account](https://www.testmuai.com/register/) if you don't have one. -2. Get your **Username** and **Access Key** from the [TestMu AI Dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard). -3. Install the latest [PHP](https://www.php.net/downloads) and [Composer](https://getcomposer.org/). -4. Install the Selenium WebDriver for PHP (pulled in by the sample projects via Composer). - -## Set your credentials - -Every framework authenticates the same way: your Username and Access Key are read from environment variables. Set them once. Pick your operating system: - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME=${ YOUR_LAMBDATEST_USERNAME()} -set LT_ACCESS_KEY=${ YOUR_LAMBDATEST_ACCESS_KEY()}`} - -## How the sample test works - -Every framework below connects to the grid and passes your browser and OS choices through a capabilities array: - -```php -$capabilities = array( -"build" => "your build name", -"name" => "your test name", -"platform" => "Windows 10", -"browserName" => "Chrome", -"version" => "latest" -); -``` - -What changes between frameworks is only how those capabilities are supplied: a YAML config, a suite file, or an inline setup class. That is what each tab covers. - -Use the [Capabilities Generator](https://www.testmuai.com/capabilities-generator/) to build a capabilities block for any browser, version, and OS combination. - -## Run a test in your framework - -Each tab lists the framework-specific pieces. Clone the matching repo (it contains the full, ready-to-run project), then run. - -Behat is BDD for PHP: feature files plus step definitions, with the grid config in a Behat YAML file. - -1. Clone the [sample GitHub project](https://github.com/LambdaTest/behat-selenium-sample): - -```bash -git clone https://github.com/LambdaTest/behat-selenium-sample -cd behat-selenium-sample -``` - -2. Set your browser and OS in the Behat config: - -```yaml title="behat.yml" -default: -context: -parameters: -lambdatest: -server: "hub.lambdatest.com" -user: "YOUR_LAMBDATEST_USERNAME" -key: "YOUR_LAMBDATEST_ACCESS_KEY" -capabilities: -build: "behat-selenium-sample" -name: "single-behat-test" -environments: -- browserName: chrome -version: 71.0 -platform: Win10 -``` - -3. Run a single test, or in parallel: - -```bash -composer single -composer parallel -``` - -Laravel Dusk provides a fluent browser-automation API. Credentials and the grid live in the project's `.env` and `tests/DuskTestCase.php`. - -1. Clone the [sample GitHub project](https://github.com/LambdaTest/php-laravel-dusk-todo): - -```bash -git clone https://github.com/LambdaTest/php-laravel-dusk-todo -cd php-laravel-dusk-todo -``` - -2. Set your browser and OS in the `$capabilities` array: - -```php -$capabilities = array( -"build" => "LaravelDusk Build", -"name" => "LaravelDusk Build", -"platform" => "Windows 10", -"browserName" => "Chrome", -"version" => "latest" -); -``` - -3. Run the test: - -```bash -php artisan dusk -``` - -Codeception configures the grid in its acceptance suite's WebDriver module, with credentials in the host URL. - -1. Clone the [sample GitHub project](https://github.com/LambdaTest/codeception-selenium-sample): - -```bash -git clone https://github.com/LambdaTest/codeception-selenium-sample -cd codeception-selenium-sample -``` - -2. Set your browser and OS in the acceptance suite config: - -```yaml title="acceptance.suite.yml" -modules: -enabled: -- WebDriver: -url: 'https://lambdatest.github.io/sample-todo-app/' -host: '{username}:{token}@hub.lambdatest.com' -port: 80 -browser: chrome -capabilities: -name: 'Codeception Example' -build: '1.0' -browserName: 'Chrome' -platform: 'Windows 10' -version: '71.0' -``` - -3. Run the test: - -```bash -./vendor/bin/codecept run --steps -``` - -PHPUnit keeps capabilities inline in `LambdaTestSetup.php`, with Composer scripts for single and parallel runs. - -1. Clone the [sample GitHub project](https://github.com/LambdaTest/Php-PhpUnit-Selenium): - -```bash -git clone https://github.com/LambdaTest/Php-PhpUnit-Selenium -cd Php-PhpUnit-Selenium -``` - -2. Set your browser and OS in the `$capabilities` array (`LambdaTestSetup.php`): - -```php -$capabilities = array( -"build" => "Sample PHPUnit Build", -"name" => "Sample PHPUnit Test", -"platform" => "Windows 10", -"browserName" => "Chrome", -"version" => "latest" -); -``` - -3. Run a single test, in parallel, or both: - -```bash -composer single -composer parallel -composer test -``` - -## View your results - -Your test results, including video, network logs, and command-by-command execution, appear on the [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build). diff --git a/static/docs/phpunit-with-selenium-running-phpunit-automation-scripts-on-testmu-selenium-grid.md b/static/docs/phpunit-with-selenium-running-phpunit-automation-scripts-on-testmu-selenium-grid.md deleted file mode 100644 index fdec8d977..000000000 --- a/static/docs/phpunit-with-selenium-running-phpunit-automation-scripts-on-testmu-selenium-grid.md +++ /dev/null @@ -1,121 +0,0 @@ -# Selenium With PHPUnit - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Run PHPUnit tests on the TestMu AI cloud grid. This guide covers setup, running a sample test, configuring capabilities, and testing locally hosted pages. - -**Sample repo** -All the code used in this guide is available in the sample repository. - - View on GitHub - -## Prerequisites - -Complete the following steps before you begin automation testing with Selenium and PHPUnit. - -1. Install the latest **PHP** on your system. Use the following commands in the terminal: - - * **MacOS:** Previous versions of **MacOS** have **PHP** installed by default. For the latest **MacOS** versions starting with **Monterey**, download and install **PHP** manually: -```bash -/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" -brew install php -``` - * **Windows:** Download **PHP** from [PHP for Windows](http://windows.php.net/download/) and add it to your system PATH. Refer to the [PHP Windows installation guide](http://php.net/manual/en/install.windows.php) to ensure PHP is accessible through Command Prompt (cmd). - -2. Download **composer** in the project directory ([Linux/MacOS](https://getcomposer.org/download/), [Windows](https://getcomposer.org/doc/00-intro.md#installation-windows)). - - **Note:** To use the **composer** command directly, it either should have been downloaded in the project directory or should be accessible globally which can be done by the command below: -```bash -mv composer.phar /usr/local/bin/composer -``` - -## Step 1: Clone the Sample Project - -Clone the repository and install dependencies. - -```bash -git clone https://github.com/LambdaTest/Php-PhpUnit-Selenium -cd Php-PhpUnit-Selenium -``` - -Install the composer dependencies: -```bash -composer install -``` - -## Step 2: Set Your Credentials - -Configure your credentials to connect to the TestMu AI Selenium Grid. - -Set TestMu AI `Username` and `Access Key` in environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -## Step 3: Configure Your Test Capabilities - -Define browser, version, and OS settings for your test run. - -In `LambdaTestSetup.php` file, update your test capabilities to select OS, browser, browser version, and other settings. - -```php -$capabilities = array( -"build" => "Sample PHPUnit Build", -"name" => "Sample PHPUnit Test", -"platform" => "Windows 10", -"browserName" => "Chrome", -"version" => "latest" -); -``` - -**Capabilities Generator** -Use the TestMu AI [Capabilities Generator](https://www.testmuai.com/capabilities-generator/) to auto-generate the capabilities class for your test requirements. - -## Step 4: Run the Test - -Execute the PHPUnit test from the command line. - -```bash -composer single -``` - -To run parallel tests: -```bash -composer parallel -``` - -To run both single and parallel tests at once: -```bash -composer test -``` - -## Step 5: View Your Results - -Check the test output on the console and the TestMu AI dashboard. - -Visit the [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) to view your test results. The dashboard provides: - -- Text logs for each test step -- Screenshots captured during execution -- Video recordings of the full test session - -## Run PHPUnit Tests Using Agent Skills - -Use AI coding assistants to generate and run PHPUnit tests with the TestMu AI Agent Skill. - -The [phpunit-skill](https://github.com/LambdaTest/agent-skills/tree/main/phpunit-skill) is part of [TestMu AI Agent Skills](https://github.com/LambdaTest/agent-skills/) - structured packages that teach AI coding assistants how to write production-grade test automation. - -Install the skill: - -```bash -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/phpunit-skill .claude/skills/ - -# For Cursor / Copilot -cp -r agent-skills/phpunit-skill .cursor/skills/ -``` - -Install all available framework skills at once by cloning the repository directly into your tool's skills directory (e.g., `.claude/skills/`, `.cursor/skills/`). diff --git a/static/docs/pingone-scim.md b/static/docs/pingone-scim.md deleted file mode 100644 index 9db93d538..000000000 --- a/static/docs/pingone-scim.md +++ /dev/null @@ -1,153 +0,0 @@ -# PingOne SCIM User & Group Provisioning - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -PingOne (by PingIdentity) is a cloud-based identity platform that provides SSO, MFA, and directory services. It supports SCIM 2.0 Outbound provisioning to automatically sync users and groups to external applications. - -In this guide, we will walk through integrating PingOne SCIM provisioning with TestMu AI. - -## Prerequisites - -- You will need an Enterprise plan with TestMu AI. -- The SSO connection must be already established with TestMu AI. Learn how to [create a SSO Connection with TestMu AI](/support/docs/testmu-sso-self-serve/). - -## Integrating SCIM User Provisioning with PingOne - -### Step 1: Copy SCIM Credentials from TestMu AI - -Sign in to your TestMu AI account. Don't have an account, [register for free](https://www.testmuai.com/register/). - -Head to **Settings** > **Organization Settings** > **Security** > **SCIM Provisioning** tab. Copy the **SCIM Base URL** and **Bearer Token**. - -### Step 2: Create a SAML Application in PingOne (if not already done) - -If you already have a PingOne SAML application configured for TestMu AI SSO, skip to [Step 3](#step-3). - -Log in to the [PingOne Admin Console](https://admin.pingone.com). Go to **Applications** > **Applications** > click **+** (Add Application). - -Enter a name (e.g., your org name), select **SAML Application**, and click **Save**. - -### Step 3: Create SCIM Provisioning Connection {#step-3} - -In PingOne, go to **Integrations** > **Provisioning** > **Connections** tab. - -Click **+ (New Connection)**. - -Select **Identity Store** (SCIM) and click **Next**. - -Search for **SCIM** and select **SCIM** (Outbound). Click **Next**. - -Enter a **Name** for the connection (e.g., your org name) and click **Next**. - -### Step 4: Configure Authentication - -| Field | Value | -|---|---| -| **SCIM Base URL** | Paste the Base URL copied from TestMu AI | -| **Users Resource** | `/Users` | -| **SCIM Version** | `2.0` | -| **Groups Resource** | `/Groups` | -| **Authentication Method** | `OAuth 2 Bearer Token` | -| **OAuth Access Token** | Paste the Bearer Token from TestMu AI | -| **Auth Type Header** | `Bearer` | - -Click **Test Connection** to verify, then click **Next**. - -### Step 5: Configure Preferences - -| Field | Value | -|---|---| -| **User Filter Expression** | `userName eq "%s"` | -| **User Identifier** | `workEmail` | -| **Custom Attribute Schema URNs** | `urn:ietf:params:scim:schemas:extension:LambdaTest:2.0:User` | -| **Group Membership Handling** | `Merge` (recommended) | - -Enable the following user actions: -- **Create Users**: checked -- **Update Users**: checked -- **Disable Users**: checked - -Click **Save**. - -Add `urn:ietf:params:scim:schemas:extension:LambdaTest:2.0:User` to the **Custom Attribute Schema URNs** field to enable sending `OrganizationRole` and `LambdatestGroup` from PingOne. - -### Step 6: Create a User Provisioning Rule - -Go to the **Rules** tab in your provisioning connection and click **Add Rule**. - -Enter a **Rule Name** (e.g., "User Sync Rule") and click **Next**. - -Configure the **Directory** settings: -- Set a **User Filter** (e.g., `enabled Equals true` to only sync active users) - -Review the **Attribute Mapping**. PingOne maps the following attributes by default: - -| PingOne Directory | SCIM Attribute | -|---|---| -| Enabled | `active` | -| Family Name | `familyName` | -| Given Name | `givenName` | -| **Email Address** | **`userName`** | -| Email Address | `workEmail` | -| Formatted | `formattedName` | - -**Important** -Make sure **Email Address** is mapped to **`userName`**. This is required. TestMu AI uses `userName` as the unique identifier for SCIM users, and it must be a valid email address. - -Click **Save** to create the rule. - -Ensure the rule is **enabled** (toggle ON). - -Users assigned to the provisioning rule will now be automatically synced to TestMu AI. - -## Provisioning Groups from PingOne - -**Enable Group Provisioning First** -Group provisioning is **not enabled by default**. Before proceeding, reach out to our **24/7 chat support** or email [support@testmuai.com](mailto:support@testmuai.com) to get it activated for your organization. - -Once group provisioning is enabled and user provisioning is working, follow these steps to push PingOne groups to TestMu AI. - -### Step 1: Add Groups to Your Provisioning Rule in PingOne - -Go back to your SCIM provisioning connection in PingOne > **Rules** tab > edit your provisioning rule. - -Under **Directory** settings, click **Add Groups** to select which PingOne groups to provision to TestMu AI. - -Select the groups you want to push and click **Save**. - -### Step 2: Configure Mapping Rules in TestMu AI - -In TestMu AI, go to **Settings** > **Organization Settings** > **SCIM Group Provisioning** > **Mapping Rules** tab. - -Click **Add Mapping Rule** to create a rule that determines how incoming groups are mapped. - -Configure the rule: -- **Pattern**: match group names by prefix, regex, or match all -- **Target Entity Type**: Team, Concurrency Group, or Sub-Organization -- **Auto Approve**: toggle ON to automatically approve matching groups - -Set a **Match All → Team** rule with auto-approve enabled to automatically map all incoming groups to teams without manual intervention. - -### Step 3: Verify Groups in TestMu AI - -Once PingOne pushes the groups, go to **SCIM Group Provisioning** > **SCIM Groups** tab to see the synced groups. - -If you configured an auto-approve mapping rule, the groups will be automatically approved and members synced. - -Verify that the teams (or concurrency groups / sub-orgs) were created: - -Once the provisioning rule runs, group members will appear in the mapped entities. - -### What Happens After Group Provisioning - -| PingOne Action | TestMu AI Effect | -|---|---| -| Group included in provisioning rule | Group created, mapping rules evaluated, members synced | -| User added to group | Member added to all mapped TestMu AI entities | -| User removed from group | Member removed (if no other group maps them there), role recomputed | -| Group renamed | Group renamed, mapped entity renamed to match, rules re-evaluated | -| Group removed from rule / deleted | Group soft-deleted, members safely unassigned, roles recomputed | - -> For details on mapping, conflicts, and rules, see the [SCIM Provisioning guide](/support/docs/scim/#group-provisioning). - -> That's all you need to know about TestMu AI SCIM Provisioning with PingOne. In case you have any questions please feel free to reach out to us via the **24/7 chat support** or email us over [support@testmuai.com](mailto:support@testmuai.com). diff --git a/static/docs/pivotal-tracker-integration.md b/static/docs/pivotal-tracker-integration.md deleted file mode 100644 index b2641556b..000000000 --- a/static/docs/pivotal-tracker-integration.md +++ /dev/null @@ -1,57 +0,0 @@ -# Pivotal Tracker Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Pivotal Tracker offers a project management platform on cloud. Designed for Agile teams, the tracker helps you organize your release cycle through an intuitive dashboard where everyone can monitor who is working on what, along with their upcoming tasks! Also, a shared backlog is there to help the team prioritize tasks which require more attention. Tracker automatically calculates your team's velocity by analyzing the past iterations, to help you calculate ETAs on your future sprints. - -TestMu AI integration with Pivotal Tracker will help you instantly log UI bugs from TestMu AI to your projects on the Pivotal Tracker. That way, you can report any cross browser compatibility issues with your team as soon as you encounter them while cross browser testing with TestMu AI. - -## How To Integrate Pivotal Tracker With Your TestMu AI Account? - -**Step 1:** Login to your TestMu AI account. You would need to have Admin or User level access to see and install integrations. - -**Step 2:** Select 'Integration' from the left navigation menu bar. This will guide you to a screen where you will find a list of 3rd party applications, available to integrate with your TestMu AI account. - -**Step 3:** Under the category of **Project Management**, click on the block that says **'Pivotal Tracker'**. - -**Step 4:** You will be routed to the below screen where you are required to fill your **Pivotal Tracker API token**. - -**Step 5:** Log into your Pivotal Tracker account and visit your Profile. - -**Step 6:** Scroll down on your Profile page and you will find a section where you have your API token generated. This API token will help you authenticate your access as you integrate your Pivotal tracker with any third-party applications. Please handle the API token with care. Treat them as your passwords as somebody else can authenticate a third-party application on your behalf. In case you feel your API token has been revealed to someone you don't trust then you can create a new API token in the Pivotal tracker. You can also remove an older API token if you like. - -**Step 7:** Copy the API token from your Pivotal Tracker profile and paste it on TestMu AI authentication and hit the **Install** button. - -**Step 8:** Once you hit the Install button, you will notice that you have successfully integrated Pivotal Tracker with your TestMu AI account. You can find it under the section which says "MY INTEGRATIONS". - ->**Note:** You can leverage the Resync button to synchronize your latest projects of Pivotal Tracker into your TestMu AI accounts. - -## Log Your First Bug From TestMu AI Integration With Pivotal Tracker - -**Step 1:** Go for any of the test from the left navigation menu. For demo, we will be taking **"Real Time Test"** option. - -**Step 2:** Present a URL of the web-app you need to test in the dialog box. After that, select any configuration for browser and operating system of your choice & hit **'Start'**. - -**Step 3:** After the VM is launched and operable. You can perform testing on your web-app for finding bugs. If a bug gets revealed, then you need to click on the **Bug icon** from the left panel for capturing a screenshot of the same. We have highlighted that option with yellow in the below image. - -**Step 4:** After a screenshot is captured, you can annotate any issue or a task with an in-built image editor. Once you are done highlighting the bug, click on the **Bug** icon beside Pivotal Tracker under Integrations. - -**Step 5:** After clicking on **Bug** icon, a form specific to Pivotal Tracker would open up. Fill the fields as per your requirement, and click on **Mark As Bug** button. - -**Note:** After you click on **Mark As Bug**, you will be able to observe it being successfully marked through a single click effort. You will get prompt messages on top of your Virtual Machine indicating the progress of bug logging. After few seconds you will be notified with a **prompt message** *"Your bug has been logged successfully"* indicating that the screenshot has been pushed to your **Pivotal Tracker project**. - -**Step 6:** Now, go to your Pivotal Tracker project and you will notice your logged task under the Icebox of your project as a story. - -**Step 7:** Click on the task under Icebox and you will find all the details that you shared while logging the bug from TestMu AI. - -## Remove TestMu AI Integration With Pivotal Tracker - ->You can work with one integration at a time. So if you would want to integrate to a similar 3rd party application, then you would have to **remove** your current integration. Here is how you can do that. - -**Step 1:** Login to your TestMu AI account. - -**Step 2:** Select **'Integrations'** from the left-navigation menu. This will guide you to a screen where you will find 3rd party applications, available to integrate with your TestMu AI account. - -**Step 3:** Under the section of **MY INTEGRATIONS**, you will find the remove button. Click on it to remove TestMu AI integration with Pivotal Tracker. - -That was all you need to know for integrating TestMu AI with Pivotal Tracker. In case you have any questions, feel free to share them with out product experts on **24/7 chat support**. Happy testing! 🙂 diff --git a/static/docs/playwright-accessibility-test.md b/static/docs/playwright-accessibility-test.md deleted file mode 100644 index e6ff733c8..000000000 --- a/static/docs/playwright-accessibility-test.md +++ /dev/null @@ -1,145 +0,0 @@ -# Automation Tests with Accessibility Tool using Playwright - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This document walks you through the process of evaluating the accessibility of your website through the execution of automated tests using TestMu AI's Accessibility Tool. - -> **Note:** Accessibility Testing for Playwright is currently supported on the **Chrome browser**. It is **not supported on `pw-chromium`**. - -**Current limitation for `pw-chromium`** -Accessibility report generation in Playwright depends on a Chrome extension required by the platform being loaded during the session. - -With **Playwright's bundled Chromium (`pw-chromium`)**, the required extension is not loaded reliably, so accessibility reports may not be generated even when `accessibility: true` is enabled and the scan hook runs correctly. - -**Current recommendation:** run your Playwright accessibility tests on **Chrome** instead of `pw-chromium`. - -## Prerequisites - -- Your [TestMu AI Username and Access key](/support/docs/using-environment-variables-for-authentication-credentials/) -- Setup your local machine as per your testing framework. - -## Step-by-Step Guide to Trigger Your Test - -### Step 1: Setup Your Test Suite -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**sample repo** -Download or Clone the code sample from the TestMu AI GitHub repository to run your tests. - - View on GitHub - -If you are using your own project, make sure you update the **Hub endpoint** in your tests file. By setting up the Hub endpoint, you establish the communication channel between your tests and the browser nodes, enabling effective test distribution and execution. - -Configure the desired capabilities based on your test requirements. For example: - -```javascript -const capabilities = { -'browserName': 'Chrome', -'browserVersion': 'latest', -'LT:Options': { -'platform': 'Windows 10', -'build': 'Playwright Accessibility', -'name': 'Playwright Accessibility', -'user': process.env.LT_USERNAME, -'accessKey': process.env.LT_ACCESS_KEY, -..// -} -} -``` - -> You can generate capabilities for your test requirements with the help of our inbuilt 🔗 Capabilities Generator. - -### Step 2: Establish User Authentication - -Now, you need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in the [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). - -Run the below mentioned commands in your terminal to setup the CLI and the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -### Step 3: Configure the Necessary Capabilities - -To enable the accessibility testing within your automated test suite, set `accessibility: true` in your configuration file. - -There are two primary ways to run accessibility tests: - -#### 1. On-Demand Scans (via Hooks) -For precise control over which pages are scanned, you can trigger scans manually at specific points in your test execution. This is the recommended approach to reduce test execution time and focus only on relevant pages. - -To use this, simply enable accessibility in your capabilities: - -```javascript -capabilities['accessibility'] = true; // Enable accessibility testing -``` - -Then, trigger the scan directly within your test script when the desired page is fully loaded: - -```javascript -// Execute the LambdaTest accessibility scan hook -await page.evaluate('lambda-accessibility-scan'); -``` - -*Note: If you do not execute the hook in your script when using this method, no accessibility reports will be generated.* - -#### 2. Continuous Auto-Scanning -If you want the accessibility scanner to run automatically on every single page navigation throughout the entire test session without writing manual hooks, you can pass the `accessibility.autoscan` capability: - -```javascript -capabilities['accessibility'] = true; // Enable accessibility testing -capabilities['accessibility.autoscan'] = true; // Automatically scan all pages -``` - -#### Advanced Capabilities -You can also define other settings capabilities to refine your scan rules as described below: - -```javascript -const capabilities = { -"accessibility": true, -"accessibility.wcagVersion": "wcag21aa", // Specify WCAG version (e.g., WCAG 2.1 Level AA) -"accessibility.bestPractice": false, // Exclude best practice issues from results -"accessibility.needsReview": true // Include issues that need review -}; -``` - -**Browser choice for Playwright accessibility** -If your Playwright project is not browser-specific, use **Chrome** for accessibility automation until `pw-chromium` extension loading is supported reliably. - -This is the safest workaround when reports are not being generated for `pw-chromium`. - -### Step 4: Add the following add-on Script -TestMu AI uses a Chrome extension for accessibility scans and report generation. In your `lambdatest-setup.js` file add these three lines after your page creation command as shown below: - -```javascript -// Load the extension for report generation of the accessibility tests -await ltPage.goto("chrome://extensions/?id=johgkfjmgfeapgnbkmfkfkaholjbcnah"); -const secondToggleButton = ltPage.locator('#crToggle').nth(0); -await secondToggleButton.click(); -``` - -**Why this fails on `pw-chromium`** -If you run the same setup on **Playwright bundled Chromium (`pw-chromium`)**, the required accessibility extension may not stay loaded, which prevents accessibility report generation. - -At the moment, there is no confirmed public workaround for this behavior. Use **Chrome** for accessibility automation on Playwright. - -### Step 5: Execute and Monitor your Test - -Now execute your tests and visit the [Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard). Click on the Accessibility tab and check the report generated. - -```bash -npx playwright test --config=./playwright.config.js -``` - -You can access the detailed accessibility report from the [Accessibility Automation Reports Dashboard](https://www.testmuai.com/login/?redirectTo=https://accessibility.lambdatest.com/automation) - -## Troubleshooting - -| Issue | What it means | Recommended action | -|---|---|---| -| Accessibility report is not generated on `pw-chromium` | The required accessibility extension is not loading reliably in Playwright bundled Chromium | Run the same test on **Chrome** instead of `pw-chromium` | -| Hook executes but no report appears | `lambda-accessibility-scan` ran, but the accessibility extension was not active in the session | Use **Chrome**, then rerun the test | -| Unsure whether this is a product bug or setup issue | The test may be correct, but the browser target is unsupported for accessibility automation | Verify the browser is **Chrome**, not `pw-chromium` | -| Customer needs immediate unblock | Browser is not central to the use case | Ask the customer to run functional + accessibility automation on **Chrome** until the limitation is resolved | diff --git a/static/docs/playwright-agent-skills.md b/static/docs/playwright-agent-skills.md deleted file mode 100644 index d0e7d8b0e..000000000 --- a/static/docs/playwright-agent-skills.md +++ /dev/null @@ -1,128 +0,0 @@ -# Run Your Playwright Tests Using Agent Skills - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Instead of manually setting up your Playwright project, you can use **Agent Skills** to let your AI coding assistant generate production-ready Playwright test automation for you. - -The [playwright-skill](https://github.com/LambdaTest/agent-skills/tree/main/playwright-skill) is part of [TestMu AI Agent Skills](https://github.com/LambdaTest/agent-skills/) - structured packages that teach AI coding assistants how to write production-grade test automation code. - -## What Are Agent Skills? - -Agent Skills are self-contained packages of instructions, code patterns, debugging guides, and CI/CD configurations for specific testing frameworks. Once installed, your AI assistant works like a **Senior QA automation architect** with knowledge of: - -- Proper project structure for each framework -- Correct dependency versions and configurations -- Both local and TestMu AI cloud execution patterns -- Common pitfalls and debugging approaches -- CI/CD integration with GitHub Actions - -## Prerequisites - -- A TestMu AI account. If you don't have one, sign up for free. -- Your TestMu AI **Username** and **Access Key** from the [Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/). -- An AI coding assistant: [Claude Code](https://claude.ai/code), [GitHub Copilot](https://github.com/features/copilot), [Cursor](https://cursor.sh/), [Gemini CLI](https://github.com/google-gemini/gemini-cli), or any compatible tool. - -## Installing the Playwright Agent Skill - -Clone the repository and copy the Playwright skill into your AI tool's skills directory: - -```bash -git clone https://github.com/LambdaTest/agent-skills.git - -# For Claude Code -cp -r agent-skills/playwright-skill .claude/skills/ - -# For Cursor -cp -r agent-skills/playwright-skill .cursor/skills/ - -# For GitHub Copilot -cp -r agent-skills/playwright-skill .github/skills/ - -# For Gemini CLI -cp -r agent-skills/playwright-skill .gemini/skills/ -``` - -If you prefer installing **all available framework skills**, clone the repository directly into your tool's skills directory (e.g., `.claude/skills/`, `.cursor/skills/`). - -## Set Up Your Authentication - -Configure your TestMu AI credentials as environment variables: - -```bash -export LT_USERNAME="YOUR_LAMBDATEST_USERNAME" -export LT_ACCESS_KEY="YOUR_LAMBDATEST_ACCESS_KEY" -``` - -## Running Your First Test Using Agent Skills - -Once installed, simply ask your AI assistant to write tests in natural language: - -``` -"Write Playwright tests for the login page and run them on TestMu AI cloud using Chrome and Firefox" -``` - -``` -"Set up a Playwright TypeScript project with Page Object Model and run tests on TestMu AI" -``` - -``` -"Run Playwright tests on real mobile devices on TestMu AI cloud" -``` - -The Agent Skill automatically handles: - -1. **Project setup** - Creates the right folder structure, dependencies, and config files -2. **Language detection** - Maps your request to the correct language (TypeScript, JavaScript, Python, Java, or C#) -3. **Execution target** - Configures local or TestMu AI cloud execution via CDP WebSocket -4. **Best practices** - Uses web-first assertions, accessible selectors (`getByRole`, `getByLabel`), and Page Object Model - -## What's Included in the Playwright Skill - -``` -playwright-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -| Section | Details | -|---------|---------| -| **Project Setup** | Dependencies, versions, configuration files, project structure | -| **Core Patterns** | Complete, runnable code examples with web-first assertions | -| **Cloud Integration** | TestMu AI CDP WebSocket connection and capabilities configuration | -| **CI/CD Integration** | GitHub Actions workflows with reporting and parallel execution | -| **Debugging Guide** | Common issues mapped to solutions | -| **Best Practices** | Actionable recommendations for production code | - -## Supported Languages and Frameworks - -The Playwright Agent Skill supports 5 languages. The following framework docs also reference their own dedicated Agent Skills: - -### Languages -| Language | Agent Skill | Documentation | -|----------|------------|---------------| -| TypeScript (default) | [playwright-skill](https://github.com/LambdaTest/agent-skills/tree/main/playwright-skill) | [TypeScript with Playwright](/docs/typescript-with-playwright/) | -| JavaScript | [playwright-skill](https://github.com/LambdaTest/agent-skills/tree/main/playwright-skill) | [JavaScript with Playwright](/docs/javascript-with-playwright/) | -| Java | [playwright-skill](https://github.com/LambdaTest/agent-skills/tree/main/playwright-skill) | [Java with Playwright](/docs/java-with-playwright/) | -| Python | [playwright-skill](https://github.com/LambdaTest/agent-skills/tree/main/playwright-skill) | [Python with Playwright](/docs/python-with-playwright/) | -| C# | [playwright-skill](https://github.com/LambdaTest/agent-skills/tree/main/playwright-skill) | [C# with Playwright](/docs/csharp-with-playwright/) | - -### Frameworks with Dedicated Agent Skills -| Framework | Agent Skill | Documentation | -|-----------|------------|---------------| -| Jest | [jest-skill](https://github.com/LambdaTest/agent-skills/tree/main/jest-skill) | [Jest with Playwright](/docs/jest-with-playwright/) | -| JUnit 5 | [junit-5-skill](https://github.com/LambdaTest/agent-skills/tree/main/junit-5-skill) | [JUnit with Playwright](/docs/junit-with-playwright/) | -| CucumberJS | [cucumber-skill](https://github.com/LambdaTest/agent-skills/tree/main/cucumber-skill) | [CucumberJS with Playwright](/docs/playwright-with-cucumberjs/) | - -## Supported AI Tools - -| Tool | Type | Installation Path | -|------|------|-------------------| -| Claude Code | CLI | `.claude/skills/` | -| GitHub Copilot | Extension | `.github/skills/` | -| Cursor | IDE | `.cursor/skills/` | -| Gemini CLI | CLI | `.gemini/skills/` | -| Codex CLI | CLI | `.codex/skills/` | -| OpenCode | CLI | `.opencode/skills/` | -| Claude.ai | Web | Settings > Features > Skills | diff --git a/static/docs/playwright-android.md b/static/docs/playwright-android.md deleted file mode 100644 index b84b5c6d6..000000000 --- a/static/docs/playwright-android.md +++ /dev/null @@ -1,462 +0,0 @@ -# Getting Started With Playwright Testing on Android Real Devices - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Playwright Android automation is supported on TestMu AI across **Node.js, Java, C#, and Python**. Run Playwright tests on Chrome for Android across 100+ real Android devices. This guide covers getting started with Playwright testing on Android devices on the TestMu AI platform. - -**Supported Versions** -- Playwright versions **v1.20.0** to **v1.59.0** are supported for Android real device testing (excluding `v1.54.0`). -- **Java, C#, and Python** use the `chromium.connect()` API. **Node.js** supports both `chromium.connect()` and the Android-native `_android.connect()` API. All use stock Playwright packages, no custom forks required. -- Playwright v1.53.0 is currently supported for Playwright C# (for Android & iOS). - -## Prerequisites - -Set your TestMu AI username and access key in the environment variables. You can get your TestMu AI username and access key from your TestMu AI Profile > Account Settings > Password & Security. - -**Windows** - -```bash -set LT_USERNAME="YOUR_LAMBDATEST_USERNAME" -set LT_ACCESS_KEY="YOUR_LAMBDATEST_ACCESS_KEY" -``` - -**macOS/Linux** - -```bash -export LT_USERNAME="YOUR_LAMBDATEST_USERNAME" -export LT_ACCESS_KEY="YOUR_LAMBDATEST_ACCESS_KEY" -``` - -### Language-Specific Setup - -| Language | Supported Playwright Versions | -|----------|-------------------------------| -| **JavaScript (Node.js)** | Up to **v1.59.0** | -| **Java, Python, C#** | Up to **v1.53.0** | - -Install the Playwright package: - -```bash -npm install playwright -``` - -Install the Playwright package: - -```bash -pip install playwright -``` - -Add the Playwright dependency to your `pom.xml`: - -```xml - -com.microsoft.playwright -playwright -1.54.0 - -``` - -Add the Playwright NuGet package: - -```bash -dotnet add package Microsoft.Playwright -``` - -## Run Your First Test - -Node.js supports both the Chromium API (`chromium.connect()`) and the Android-native API (`_android.connect()`). - -**Using `chromium.connect()`** - -```javascript title="playwright-android-test.js" -const { chromium } = require("playwright"); - -(async () => { -const capabilities = { -"LT:Options": { -platformName: "android", -deviceName: ".*", -platformVersion: ".*", -isRealMobile: true, -build: "Playwright Android Build", -name: "Playwright Android Test", -user: process.env.LT_USERNAME, -accessKey: process.env.LT_ACCESS_KEY, -network: true, -video: true, -console: true, -playwrightClientVersion: "1.53.0", -}, -}; - -const cdpUrl = `wss://cdp.lambdatest.com/playwright?capabilities=${encodeURIComponent( -JSON.stringify(capabilities) -)}`; - -const browser = await chromium.connect(cdpUrl); -const context = browser.contexts()[0] || (await browser.newContext()); -const page = context.pages()[0] || (await context.newPage()); - -await page.goto("https://duckduckgo.com", { timeout: 30000 }); -await page.locator('[name="q"]').fill("LambdaTest"); -await page.locator('[name="q"]').press("Enter"); -await page.waitForTimeout(3000); - -const title = await page.title(); -console.log("Page title:", title); - -try { -if (title.includes("LambdaTest")) { -await page.evaluate( -(_) => {}, -`lambdatest_action: ${JSON.stringify({ -action: "setTestStatus", -arguments: { status: "passed", remark: "Title verified" }, -})}` -); -} -} catch (e) { -await page.evaluate( -(_) => {}, -`lambdatest_action: ${JSON.stringify({ -action: "setTestStatus", -arguments: { status: "failed", remark: e.message }, -})}` -); -} - -await page.close(); -await browser.close(); -})(); -``` - -**Using `_android.connect()`** - -```javascript title="playwright-android-test.js" -const { _android } = require("playwright"); - -(async () => { -const capabilities = { -"LT:Options": { -platformName: "android", -deviceName: ".*", -platformVersion: ".*", -isRealMobile: true, -build: "Playwright Android Build", -name: "Playwright Android Test", -user: process.env.LT_USERNAME, -accessKey: process.env.LT_ACCESS_KEY, -network: true, -video: true, -console: true, -playwrightClientVersion: "1.53.0", -}, -}; - -const cdpUrl = `wss://cdp.lambdatest.com/playwright?capabilities=${encodeURIComponent( -JSON.stringify(capabilities) -)}`; - -const device = await _android.connect(cdpUrl); -console.log(`Model: ${device.model()}, Serial: ${device.serial()}`); -await device.shell("am force-stop com.android.chrome"); - -const context = await device.launchBrowser(); -context.setDefaultTimeout(120000); -const page = await context.newPage(); - -await page.goto("https://duckduckgo.com"); -await page.locator('[name="q"]').fill("LambdaTest"); -await page.locator('[name="q"]').press("Enter"); -await page.waitForTimeout(3000); - -const title = await page.title(); -console.log("Page title:", title); - -try { -if (title.includes("LambdaTest")) { -await page.evaluate( -(_) => {}, -`lambdatest_action: ${JSON.stringify({ -action: "setTestStatus", -arguments: { status: "passed", remark: "Title verified" }, -})}` -); -} -} catch (e) { -await page.evaluate( -(_) => {}, -`lambdatest_action: ${JSON.stringify({ -action: "setTestStatus", -arguments: { status: "failed", remark: e.message }, -})}` -); -} - -await page.close(); -await device.close(); -})(); -``` - -The timeout value specified in the Playwright configuration may default to 30 seconds on real devices. To set a custom timeout, add: - -```javascript -context.setDefaultTimeout(120000); // Set your desired timeout value. -``` - -Run the test: - -```bash -node playwright-android-test.js -``` - -```python title="playwright_android_test.py" -import os, json, urllib.parse -from playwright.sync_api import sync_playwright - -def main(): -capabilities = { -"LT:Options": { -"platformName": "android", -"deviceName": ".*", -"platformVersion": ".*", -"isRealMobile": True, -"build": "Playwright Android Build", -"name": "Playwright Android Test", -"user": os.environ["LT_USERNAME"], -"accessKey": os.environ["LT_ACCESS_KEY"], -"network": True, -"video": True, -"console": True, -} -} - -cdp_url = ( -f"wss://cdp.lambdatest.com/playwright?capabilities=" -f"{urllib.parse.quote(json.dumps(capabilities))}" -) - -with sync_playwright() as p: -browser = p.chromium.connect(cdp_url) -context = browser.contexts[0] if browser.contexts else browser.new_context() -page = context.pages[0] if context.pages else context.new_page() - -page.goto("https://duckduckgo.com", timeout=30000) -page.locator('[name="q"]').fill("LambdaTest") -page.locator('[name="q"]').press("Enter") -page.wait_for_timeout(3000) - -title = page.title() -print(f"Page title: {title}") - -try: -if "LambdaTest" in title: -page.evaluate( -"_ => {}", -'lambdatest_action: {"action": "setTestStatus", "arguments": {"status": "passed", "remark": "Title verified"}}', -) -except Exception as e: -page.evaluate( -"_ => {}", -f'lambdatest_action: {json.dumps({"action": "setTestStatus", "arguments": {"status": "failed", "remark": str(e)}})}', -) - -page.close() -browser.close() - -if __name__ == "__main__": -main() -``` - -Run the test: - -```bash -python playwright_android_test.py -``` - -```java title="PlaywrightAndroidTest.java" -package com.lambdatest; - -import com.microsoft.playwright.*; -import com.google.gson.Gson; -import java.net.URLEncoder; -import java.nio.charset.StandardCharsets; -import java.util.LinkedHashMap; -import java.util.Map; - -public class PlaywrightAndroidTest { -public static void main(String[] args) { -Map ltOptions = new LinkedHashMap<>(); -ltOptions.put("platformName", "android"); -ltOptions.put("deviceName", ".*"); -ltOptions.put("platformVersion", ".*"); -ltOptions.put("isRealMobile", true); -ltOptions.put("build", "Playwright Android Build"); -ltOptions.put("name", "Playwright Android Test"); -ltOptions.put("user", System.getenv("LT_USERNAME")); -ltOptions.put("accessKey", System.getenv("LT_ACCESS_KEY")); -ltOptions.put("network", true); -ltOptions.put("video", true); -ltOptions.put("console", true); -ltOptions.put("playwrightClientVersion", "1.53.0"); - -Map capabilities = new LinkedHashMap<>(); -capabilities.put("LT:Options", ltOptions); - -String capsJson = new Gson().toJson(capabilities); -String cdpUrl = "wss://cdp.lambdatest.com/playwright?capabilities=" -+ URLEncoder.encode(capsJson, StandardCharsets.UTF_8); - -try (Playwright playwright = Playwright.create()) { -Browser browser = playwright.chromium().connect(cdpUrl); -BrowserContext context = browser.contexts().size() > 0 -? browser.contexts().get(0) : browser.newContext(); -Page page = context.pages().size() > 0 -? context.pages().get(0) : context.newPage(); - -page.navigate("https://duckduckgo.com", -new Page.NavigateOptions().setTimeout(30000)); -page.locator("[name=\"q\"]").fill("LambdaTest"); -page.locator("[name=\"q\"]").press("Enter"); -page.waitForTimeout(3000); - -String title = page.title(); -System.out.println("Page title: " + title); - -try { -if (title.contains("LambdaTest")) { -page.evaluate("_ => {}", -"lambdatest_action: {\"action\": \"setTestStatus\", \"arguments\": {\"status\": \"passed\", \"remark\": \"Title verified\"}}"); -} -} catch (Exception e) { -page.evaluate("_ => {}", -"lambdatest_action: {\"action\": \"setTestStatus\", \"arguments\": {\"status\": \"failed\", \"remark\": \"" + e.getMessage() + "\"}}"); -} - -page.close(); -browser.close(); -} -} -} -``` - -Run the test: - -```bash -mvn compile exec:java -Dexec.mainClass="com.lambdatest.PlaywrightAndroidTest" -``` - -```csharp title="PlaywrightAndroidTest.cs" -using Microsoft.Playwright; -using System.Text.Json; - -var capabilities = new Dictionary -{ -["LT:Options"] = new Dictionary -{ -["platformName"] = "android", -["deviceName"] = ".*", -["platformVersion"] = ".*", -["isRealMobile"] = true, -["build"] = "Playwright Android Build", -["name"] = "Playwright Android Test", -["user"] = Environment.GetEnvironmentVariable("LT_USERNAME")!, -["accessKey"] = Environment.GetEnvironmentVariable("LT_ACCESS_KEY")!, -["network"] = true, -["video"] = true, -["console"] = true, -["playwrightClientVersion"] = "1.53.0", -} -}; - -var capsJson = JsonSerializer.Serialize(capabilities); -var cdpUrl = $"wss://cdp.lambdatest.com/playwright?capabilities={Uri.EscapeDataString(capsJson)}"; - -using var playwright = await Playwright.CreateAsync(); -var browser = await playwright.Chromium.ConnectAsync(cdpUrl); -var context = browser.Contexts.Count > 0 -? browser.Contexts[0] : await browser.NewContextAsync(); -var page = context.Pages.Count > 0 -? context.Pages[0] : await context.NewPageAsync(); - -await page.GotoAsync("https://duckduckgo.com", new PageGotoOptions { Timeout = 30000 }); -await page.Locator("[name=\"q\"]").FillAsync("LambdaTest"); -await page.Locator("[name=\"q\"]").PressAsync("Enter"); -await page.WaitForTimeoutAsync(3000); - -var title = await page.TitleAsync(); -Console.WriteLine($"Page title: {title}"); - -try -{ -if (title.Contains("LambdaTest")) -{ -await page.EvaluateAsync("_ => {}", -"lambdatest_action: {\"action\": \"setTestStatus\", \"arguments\": {\"status\": \"passed\", \"remark\": \"Title verified\"}}"); -} -} -catch (Exception e) -{ -await page.EvaluateAsync("_ => {}", -$"lambdatest_action: {{\"action\": \"setTestStatus\", \"arguments\": {{\"status\": \"failed\", \"remark\": \"{e.Message}\"}}}}"); -} - -await page.CloseAsync(); -await browser.CloseAsync(); -``` - -Run the test: - -```bash -dotnet run -``` - -For Java, C#, and Python on Android, the CDP connection returns an existing browser context and page. Always check for existing contexts/pages before creating new ones, as shown in the examples above. - -## View your Playwright test results - -The TestMu AI Automation Dashboard is where you can see the results of your Playwright tests after running them on the TestMu AI platform. - -The below screenshot of TestMu AI Automation Dashboard shows the Playwright build on the left and the build sessions associated with the selected build on the right. - -## Capabilities Reference - -Configure these capabilities to control your Playwright Android tests. - -Use the [Playwright Android Capability Generator](https://www.lambdatest.com/playwright-android-capability-generator/) to generate capabilities for your tests. - -| Key | Expected Values | Description | Capability | -|-----------------|--------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------| -| platform | android | Specify the platform name | `const capability = { "LT:Options": {"platform": "android",}}` | -| platformVersion | 12 | Specify the platform version | `const capability = { "LT:Options": {"platformVersion": "12",}}` | -| deviceName | Pixel 5 | Specify the device name. | `const capability = { "LT:Options": {"deviceName": "Pixel 5",}}` | -| build | Playwright Android Build | Represent the build number for your test | ```const capability = {"LT:Options": {"build": "",}}``` | -| name | Playwright Android Test | Represents the name of a test | `const capability = {"LT:Options": {"name": "",}}` | -| projectName | Playwright Sample Project | Represents the name of a project | `const capability = {"LT:Options": {"projectName": "",}}` | -| tags | ["tag1", "tag2", "tag3"] | Group your Playwright tests | ``const capability = {"LT:Options": { "tags": ["tag1", "tag2", "tag3"], }}`` | -| buildTags | ["build1", "build2", "build3"] | Group your Playwright builds | `const capability = {"LT:Options": { "buildTags": ["build1", "build2", "build3"] }}` | -| network | true/false | Enable network logs | `const capability = { "LT:Options": {"network": true,}}` | -| console | true/false | Enable browser console logs | `const capabilities = { "LT:Options": {"console": true,}}` | -| video | true/false | Enable video recording of the entire screen | `const capability = { "LT:Options": {"video": true,}}` | -| tunnel | true/false | Enable tunnel for local testing | `const capability = { "LT:Options": {"tunnel": true,}}` | -| tunnelName | true/false | Specify tunnel name | `const capability = { "LT:Options": {"tunnelName": "",}}` | -| geoLocation | AR (Argentina) | Specify country code | `const capability = { "LT:Options": {"geoLocation": "AR",}}` | - -### Using REGEX for device name -When you run a test on a specific device, the exact device you selected may not be available. A regular expression (REGEX) widens the device search so you get any matching device. For example, to run on any Pixel device rather than one specific model, use a REGEX for `deviceName`. See [REGEX for App/Browser Automation](/support/docs/regular-expression-appium/) for details. - -## WebView Testing - -To test embedded WebViews on a real Android device, add the mandatory `isPwMobileWebviewTest` capability to your options ([sample script](https://github.com/LambdaTest/playwright-sample/blob/main/playwrightwebview.js)): - -```javascript title="playwrightwebview.js" -const capabilities = { -"LT:Options": { -"platformName": "android", -"isRealMobile": true, //if true, test will run on real devices -// highlight-next-line -"isPwMobileWebviewTest": true, //mandatory capability to enable WebView testing -}, -``` - -Run the test as usual (for example, `node playwrightwebview.js`) and check the [Automation dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) for results. diff --git a/static/docs/playwright-auto-healing.md b/static/docs/playwright-auto-healing.md deleted file mode 100644 index 571f84d9b..000000000 --- a/static/docs/playwright-auto-healing.md +++ /dev/null @@ -1,206 +0,0 @@ -# Auto Healing for Playwright Web Automation - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The TestMu AI Auto Healing feature for Playwright testing automatically recovers from certain types of failures during test execution. When enabled, it reduces test flakiness and improves test reliability by detecting broken locators and handling unexpected errors in your test suites. - -## Enabling Auto Healing - -To enable the Auto Healing feature, you need to pass the `autoHeal: true` as a capability in your Playwright configuration. For example: - -```js -const capabilities = { -browserName: "Chrome", -browserVersion: "latest", -"LT:Options": { -platform: "Windows 10", -build: "Playwright Auto Heal Build", -name: "Playwright Auto Heal Test", -user: process.env.LT_USERNAME, -accessKey: process.env.LT_ACCESS_KEY, -network: true, -video: true, -console: true, -autoHeal: true, -}, -}; -``` - -> **Note:** There are no Prerequisites required for enabling the auto-healing feature. You can enable the auto-healing feature directly via desired capabilities. - -### Language Preferences: - -For **JavaScript/NodeJS**, you can use the following code: - -```javascript -// playwright.config.js -module.exports = { -use: { -connectOptions: { -wsEndpoint: `wss://cdp.lambdatest.com/playwright?capabilities=${encodeURIComponent(JSON.stringify({ -'browserName': 'Chrome', -'browserVersion': 'latest', -'LT:Options': { -'platform': 'Windows 10', -'build': 'Playwright Auto Heal Build', -'name': 'Playwright Auto Heal Test', -'autoHeal': true, -'user': process.env.LT_USERNAME, -'accessKey': process.env.LT_ACCESS_KEY, -} -}))}` -} -} -}; -``` - -For **Python**, you can use the following code: - -```python -import asyncio -import json -import urllib.parse -from playwright.async_api import async_playwright - -capabilities = { -'browserName': 'Chrome', -'browserVersion': 'latest', -'LT:Options': { -'platform': 'Windows 10', -'build': 'Playwright Auto Heal Build', -'name': 'Playwright Auto Heal Test', -'autoHeal': True, -'user': 'your_username', -'accessKey': 'your_access_key', -} -} - -async def run(): -async with async_playwright() as playwright: -browser = await playwright.chromium.connect( -ws_endpoint=f"wss://cdp.lambdatest.com/playwright?capabilities={urllib.parse.quote(json.dumps(capabilities))}" -) -``` - -For **C#**, you can use the following code: - -```csharp -using Microsoft.Playwright; - -var capabilities = new Dictionary() -{ -["browserName"] = "Chrome", -["browserVersion"] = "latest", -["LT:Options"] = new Dictionary() -{ -["platform"] = "Windows 10", -["build"] = "Playwright Auto Heal Build", -["name"] = "Playwright Auto Heal Test", -["autoHeal"] = true, -["user"] = Environment.GetEnvironmentVariable("LT_USERNAME"), -["accessKey"] = Environment.GetEnvironmentVariable("LT_ACCESS_KEY"), -} -}; - -var browser = await Playwright.Chromium.ConnectAsync($"wss://cdp.lambdatest.com/playwright?capabilities={Uri.EscapeDataString(JsonSerializer.Serialize(capabilities))}"); -``` - -For **Java**, you can use the following code: - -```java -import com.microsoft.playwright.*; -import java.util.HashMap; -import java.util.Map; - -Map capabilities = new HashMap<>(); -capabilities.put("browserName", "Chrome"); -capabilities.put("browserVersion", "latest"); - -Map ltOptions = new HashMap<>(); -ltOptions.put("platform", "Windows 10"); -ltOptions.put("build", "Playwright Auto Heal Build"); -ltOptions.put("name", "Playwright Auto Heal Test"); -ltOptions.put("autoHeal", true); -ltOptions.put("user", System.getenv("LT_USERNAME")); -ltOptions.put("accessKey", System.getenv("LT_ACCESS_KEY")); - -capabilities.put("LT:Options", ltOptions); - -Browser browser = playwright.chromium().connect("wss://cdp.lambdatest.com/playwright?capabilities=" + URLEncoder.encode(new Gson().toJson(capabilities), "UTF-8")); -``` - -## How Auto Healing Works - -Auto-healing adjusts broken locators by merging attributes and context. During runtime, it monitors the web page to identify DOM (Document Object Model) changes. - -When an element is successfully located using Playwright's locator methods (`page.locator('#username').click()`, etc.), its DOM path is recorded. If that same element is later referenced on the same page and is missing, the system evaluates the current page and generates new locators for altered elements based on previous benchmarks. - -## Auto Detection of New Locator - -In some scenarios, web elements might change their locators due to updates in the web application. However, the Auto Healing feature can automatically detect the new locator and continue the test execution. - -Here is an example test case demonstrating this: - -```js -const { test } = require('../lambdatest-setup'); -const { expect } = require('@playwright/test'); - -test.describe('Browse LambdaTest in different search engines', () => { -test('Test Auto Heal', async ({ page }) => { -await page.goto('https://www.lambdatest.com/selenium-playground/auto-healing'); -await page.waitForTimeout(1000); // sleep 1s - -// Locate element before DOM change -let buttonBeforeDOMChange = await page.locator('#username'); // equivalent to FindElement ByID -await buttonBeforeDOMChange.click(); - -// Trigger DOM change -await page.click('p.selenium_btn'); -await page.waitForTimeout(1000); // sleep 1s - -// Try to locate the same element after DOM change -// Auto healing will automatically detect the new locator -let buttonAfterDOMChange = await page.locator('#username'); // again check for element -await buttonAfterDOMChange.click(); - -await page.waitForTimeout(1000); // sleep 1s -}); -}); -``` - -In the above test case, we are interacting with an element before and after a DOM change occurs on TestMu AI's auto-healing playground. The Auto Healing feature will automatically detect if the element's locator has changed and adapt accordingly, ensuring the test continues to execute successfully. - -To run the test, execute the below command: - -```bash -npx playwright test auto-heal.spec.js -``` - -## Benefits Of Auto Healing - -- **Increased Test Stability:** Playwright tests remain consistent even when the web application's UI undergoes minor changes, reducing flakiness. -- **Reduced Test Maintenance:** The system automatically adapts to evolving interfaces, reducing the manual effort required to update test scripts. -- **Reliable CI/CD Pipeline:** Stable tests feeding into CI/CD pipelines reduce unexpected failures and ensure smoother deployments. - -## Limitations of Auto Healing - -While the Auto Healing feature is designed to handle a wide range of issues, there are certain limitations to be aware of: - -- **Non-recoverable errors**: Auto Healing cannot recover from certain types of errors, such as Playwright browser initialization errors, network connectivity issues, or system-level failures. - -- **Test accuracy**: While Auto Healing can reduce test flakiness, it may also mask real issues in your web application or test scripts. It's important to review the logs and understand why a test needed healing. - -- **Performance impact**: While typically minimal, enabling Auto Healing can have a slight impact on test execution time due to the additional checks and recovery mechanisms. - -- **Limited scope**: Auto Healing works best with simple locator changes (ID, class, attribute modifications) and may not be effective for complete page redesigns or fundamental workflow changes. - -Auto Healing enhances test suite robustness but does not replace good test design. Review healed tests regularly for issues that may be masked by the feature. - -## Related Docs - -- [Selenium Auto Healing](/support/docs/auto-healing/):Auto-heal for Selenium web automation tests -- [Smart Heal for Appium](/support/docs/smart-heal-appium/):AI-powered self-healing for mobile app automation on real devices -- [Auto Healing in HyperExecute](/support/docs/hyperexecute-auto-healing/):Auto-heal for tests executed via HyperExecute -- [Auto-Heal in KaneAI](/support/docs/kaneai-auto-heal/):AI-native auto-heal with natural language understanding for KaneAI-authored tests - -> That was all you need to know for the Auto-Healing feature with Playwright. If you still have any questions for us, please feel free to let us know. Our experts are always available on **chat** to help you out with any roadblock regarding our product. Happy testing! diff --git a/static/docs/playwright-bundled-browser-support.md b/static/docs/playwright-bundled-browser-support.md deleted file mode 100644 index 8ce6fbf4c..000000000 --- a/static/docs/playwright-bundled-browser-support.md +++ /dev/null @@ -1,39 +0,0 @@ -# Playwright Bundled Browser Support - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -When you set the capability [`useSpecificBundleVersion: true`](/support/docs/capabilities-for-playwright/#:~:text=setHTTPCredentials%22%2C%20%22setStorageState%22%2C%20%22setGeolocation%22%5D%7D%7D-,useSpecificBundleVersion,-Boolean), then TestMu AI will select the browser (Chromium, Firefox, Webkit) version as per your local machine's PLaywright version. - -```yaml -const capabilities = { "LT:Options": {"useSpecificBundleVersion": true,}} -``` - -| Playwright Versions | Chromium | Firefox | Webkit | -|---------------------|----------|---------|--------| -|1.50| 130-133 except - 132, 126, 122 | 130-134, except - 131,133,126,122,120 | 18.0, 18.2 | -|1.49| 130-133 except - 132, 126, 122 | 130-134, except - 131,133,126,122,120 | 18.0, 18.2 | -|1.48| 130-133 except - 132, 126, 122 | 130-134, except - 131,133,126,122,120 | 18.0, 18.2 | -|1.47| 129 except - 132, 126, 122 | 130, except - 131,133,126,122,120 | 18.0 | -|1.46| 119-133, except - 132, 126, 122 | 118-134, except - 131,133, 126,122,120 | 17.4, 18.2 | -|1.45| 119-127, except - 126,122 | 119-127, except - 126,122,120 | 17.4 | -|1.44| 119-127, except - 126,122 | 119-127, except - 126,122,120 | 17.4 | -|1.43| 119-127, except - 126,122 | 119-127, except - 126,122,120 | 17.4 | -|1.42| 119-127, except - 126,122 | 119-127, except - 126,122,120 | 17.4 | -|1.41| 119-127, except - 126,122 | 119-127, except - 126,122,120 | 17.4 | -|1.40| 119-127, except - 126,122 | 119-127, except - 126,122,120 | 17.4 | -|1.39| 119-127, except - 126,122 | 119-127, except - 126,122,120 | 17.4 | -|1.38| 114-117 | 113-117 | 17 | -|1.37| 114-117 | 113-117 | 17 | -|1.36| 114-117 | 113-117 | 17 | -|1.35| 114-117 | 113-117 | 16.4 | -|1.34| 114-117 | 113-117 | 16.4 | -|1.33| 104-113 | 103-112 | 16.4 | -|1.32| 104-113 | 103-112 | 16.4 | -|1.31| 104-113 | 103-112 | 16.4 | -|1.30| 104-113 | 103-112 | 16.4 | -|1.29| 104-113 | 103-112 | 16.4 | -|1.28| 104-113 | 103-112 | 16.4 | -|1.27| 104-113 | 103-112 | 16.4 | -|1.26| 104-113 | 103-112 | 16 | -|1.25| 104-113 | 103-112 | 16 | -|1.24| 103-104 | 100-102 | 16 | diff --git a/static/docs/playwright-codeceptjs-on-hyperexecute.md b/static/docs/playwright-codeceptjs-on-hyperexecute.md deleted file mode 100644 index 653086b10..000000000 --- a/static/docs/playwright-codeceptjs-on-hyperexecute.md +++ /dev/null @@ -1,206 +0,0 @@ -# Running Codecept Framework Tests on HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -CodeceptJS is a versatile end-to-end testing framework for JavaScript that allows testing web applications using various libraries like Puppeteer and WebDriver. It stands out for its simplicity and ability to write tests in a more human-readable and expressive manner, making it accessible for both developers and non-developers. - -HyperExecute is an AI Native Test Orchestration Cloud Platform that empowers you to run **end-to-end** tests **quickly** and **efficiently**. It provides Just-in-Time (JIT) testing infrastructure with fast execution **speeds**, **smart orchestration**, and **detailed logs**. - -This guide details how to execute your **Codecept** framework tests on **HyperExecute** via two different methods: - -- [**Using Local System**](/support/docs/playwright-codeceptjs-on-hyperexecute/#1-testing-using-local-system) - You can use your own local machine to execute tests. -- [**Using Gitpod Platform**](/support/docs/playwright-codeceptjs-on-hyperexecute/#2-testing-using-gitpod) - Execute tests using GitPod. (Requires a [Gitpod](https://gitpod.io/login/) account) - -## 1. Testing Using Local System - -Follow the step-by-step guide to execute your test on HyperExecute. - -### Prerequisites - -To run the Tests on HyperExecute from your Local System, you are required: - -- Your TestMu AI [Username and Access key](/support/docs/hyperexecute-how-to-get-my-username-and-access-key/) -- [HyperExecute YAML](/support/docs/hyperexecute-yaml-version0.2/) file which contains all the necessary instructions. -- [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) in order to initiate a test execution Job . -- Setup the [Environmental Variable](/support/docs/hyperexecute-environment-variable-setup/) - -### Step 1: Configure Your Test Suite - -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** - -Download or Clone the code sample for the JUnit from the TestMu AI GitHub repository to run the tests on the HyperExecute. - - View on GitHub - -If you are using your own project, make sure you update the **Hub endpoint** in your tests file. - -By setting up the Hub endpoint, you establish the communication channel between your tests and the browser nodes, enabling effective test distribution and execution. - -> You can also use **dev** and **beta** browser versions. To generate capabilities for your test requirements, you can use our inbuilt 🔗 Capabilities Generator. - -### Step 2: Setup the CLI in your Test Suite - -After cloning / downloading the sample repo, you need to setup the CLI and the environment variables. - -#### Download the HyperExecute CLI - -The CLI is used for triggering the tests on HyperExecute. It is recommend to download the CLI binary on the host system and keep it in the root directory of the suite to perform the tests on HyperExecute. - -You can download the CLI for your desired platform from the below mentioned links: - -| Platform | HyperExecute CLI | -| ---------| ---------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| MacOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -#### Setup Environment Variable - -Now, you need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in the [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). - -Run the below mentioned commands in your terminal to setup the CLI and the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -### Step 3: Configure YAML in your Test Suite - -Configure your YAML file as per your use cases using **key value** pairs. - -In this sample YAML file, we have mentioned: - -- **version** of the YAML file -- **Timeouts** for executing your project -- **Mode of execution** is [Autosplit](/support/docs/hyperexecute-auto-split-strategy/). You can also opt for [Matrix](/support/docs/hyperexecute-matrix-multiplexing-strategy/) or [Hybrid](/support/docs/hyperexecute-hybrid-strategy/) mode. -- **Pre and Post** commands -- **Reports and Artefacts** that will be generated after the completion of tests -- and other necessary YAML Parameters - -```yaml ---- -version: "0.1" - -globalTimeout: 90 -testSuiteTimeout: 90 -testSuiteStep: 90 - -runson: linux - -autosplit: true - -retryOnFailure: false -maxRetries: 1 - -concurrency: 2 - -pre: -- npm install -- npx playwright install - -cacheKey: '{{ checksum "package-lock.json" }}' -cacheDirectories: -- node_modules - -testDiscovery: -type: raw -mode: remote -command: grep -lr 'Scenario' *test.js - -testRunnerCommand: npx codeceptjs run $test --steps - -jobLabel: [playwright-CodeceptJS, linux, autosplit] -``` - -### Step 4: Execute your Test Suite - -> **NOTE :** In case of MacOS, if you get a permission denied warning while executing CLI, simply run **`chmod u+x ./hyperexecute`** to allow permission. In case you get a security popup, allow it from your **System Preferences** → **Security & Privacy** → **General tab**. - -Run the below command in your terminal at the root folder of the project: - -```bash -./hyperexecute --config RELATIVE_PATH_OF_YOUR_YAML_FILE -``` - -OR use this command if you have not exported your username and access key in the step 2. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE `} - -### Step 5: Monitor the Test Execution - -Visit the [HyperExecute Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) and check your Job status. - -### Step 6: Download Artifacts and Reports - -HyperExecute also facilitates the provision to download the [Artifacts](/support/docs/hyperexecute-artifacts/) and [Reports](/support/docs/hyperexecute-reports/) on your local machine. Click on the corresponding button to download your generated artifacts and reports. - -## 2. Testing Using Gitpod - -You can also use the Gitpod platform to execute our sample repository. It will fetch all the sample codebases and trigger the CLI to execute the tests. - -Follow the below steps to run Test using Gitpod: - -**Step 1:** Click '**Open in Gitpod**' button. You will be redirected to Login/Signup page. This button is configured to redirect you to the Gitpod platform where you will be able to execute our sample repository. - -**Step 2:** Login with TestMu AI credentials. Once logged in, a pop-up confirmation will appear, asking you to **'Proceed'** to the Gitpod editor in a new tab. The current tab will display the HyperExecute Dashboard. - -**Step 3:** Choose your preferred editor (we recommend VS Code Editor) - -**Step 4:** As you are running a sample project, Fetching of the Test Scripts, [HyperExecute YAML](/support/docs/deep-dive-into-hyperexecute-yaml/), [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) and Triggering your tests using the `Execution Command` will be automated. - -**Step 5:** Once you see the `Job Link` in the logs, you can visit the [HyperExecute dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) to see the tests getting executed. - -You can also implement [Secret Keys](/support/docs/hyperexecute-how-to-save-and-manage-secrets/) in your YAML file. - -## Navigation in Automation Dashboard - -Every test run on the HyperExecute has a unique *jobId* associated with it. Each *jobId* can in turn constitute single (or multiple) *groupId*(s). You can visit [HyperExecute Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) for checking the status of the test execution. - -You can seamlessly navigate between JobId's and taskId's. You need to click on the *testID* to navigate from the HyperExecute logs to the Automation Dashboard. - -The snapshot below shows the videos, logs and other meta data for that specific *test_ID* - -> For any query or doubt, please feel free to contact us via **24×7 chat support** or you can also drop a mail to **support@testmuai.com**. -Happy testing! - -## Using the Playwright Agent Skill with TestMu AI - -The [playwright-skill](https://github.com/LambdaTest/agent-skills/tree/main/playwright-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The playwright-skill package includes: - -``` -playwright-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing Playwright Agent Skill - -Install a Playwright Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/playwright-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/playwright-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only playwright-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). diff --git a/static/docs/playwright-dotnet-on-hyperexecute.md b/static/docs/playwright-dotnet-on-hyperexecute.md deleted file mode 100644 index c47bdc4dd..000000000 --- a/static/docs/playwright-dotnet-on-hyperexecute.md +++ /dev/null @@ -1,217 +0,0 @@ -# Running DotNet Framework Tests on HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Playwright for .NET is a framework that enables browser automation and end-to-end testing using the .NET programming language, such as C#. Developed by Microsoft, Playwright offers a comprehensive .NET API for automating interactions with web pages in Chromium, Firefox, and WebKit browsers - -HyperExecute is an AI Native Test Orchestration Cloud Platform that empowers you to run **end-to-end** tests **quickly** and **efficiently**. It provides Just-in-Time (JIT) testing infrastructure with fast execution **speeds**, **smart orchestration**, and **detailed logs**. - -This guide details how to execute your **DotNet** framework tests on **HyperExecute** via two different methods: - -- [**Using Local System**](/support/docs/playwright-dotnet-on-hyperexecute/#1-testing-using-local-system) - You can use your own local machine to execute tests. -- [**Using Gitpod Platform**](/support/docs/playwright-dotnet-on-hyperexecute/#2-testing-using-gitpod) - Execute tests using GitPod. (Requires a [Gitpod](https://gitpod.io/login/) account) - -## 1. Testing Using Local System - -Follow the step-by-step guide to execute your test on HyperExecute. - -### Prerequisites - -To run the Tests on HyperExecute from your Local System, you are required: - -- Your TestMu AI [Username and Access key](/support/docs/hyperexecute-how-to-get-my-username-and-access-key/) -- [HyperExecute YAML](/support/docs/hyperexecute-yaml-version0.2/) file which contains all the necessary instructions. -- [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) in order to initiate a test execution Job . -- Setup the [Environmental Variable](/support/docs/hyperexecute-environment-variable-setup/) - -### Step 1: Configure Your Test Suite - -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** - -Download or Clone the code sample for the JUnit from the TestMu AI GitHub repository to run the tests on the HyperExecute. - - View on GitHub - -If you are using your own project, make sure you update the **Hub endpoint** in your tests file. - -By setting up the Hub endpoint, you establish the communication channel between your tests and the browser nodes, enabling effective test distribution and execution. - -> You can also use **dev** and **beta** browser versions. To generate capabilities for your test requirements, you can use our inbuilt 🔗 Capabilities Generator. - -### Step 2: Setup the CLI in your Test Suite - -After cloning / downloading the sample repo, you need to setup the CLI and the environment variables. - -#### Download the HyperExecute CLI - -The CLI is used for triggering the tests on HyperExecute. It is recommend to download the CLI binary on the host system and keep it in the root directory of the suite to perform the tests on HyperExecute. - -You can download the CLI for your desired platform from the below mentioned links: - -| Platform | HyperExecute CLI | -| ---------| ---------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| MacOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -#### Setup Environment Variable - -Now, you need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in the [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). - -Run the below mentioned commands in your terminal to setup the CLI and the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -### Step 3: Configure YAML in your Test Suite - -Configure your YAML file as per your use cases using **key value** pairs. - -In this sample YAML file, we have mentioned: - -- **version** of the YAML file -- **Timeouts** for executing your project -- **Mode of execution** is [Autosplit](/support/docs/hyperexecute-auto-split-strategy/). You can also opt for [Matrix](/support/docs/hyperexecute-matrix-multiplexing-strategy/) or [Hybrid](/support/docs/hyperexecute-hybrid-strategy/) mode. -- **Pre and Post** commands -- **Reports and Artefacts** that will be generated after the completion of tests -- and other necessary YAML Parameters - -```yaml ---- -version: 0.1 -globalTimeout: 90 -testSuiteTimeout: 90 -testSuiteStep: 90 - -runson: win - -concurrency: 3 - -autosplit: true - -retryOnFailure: true - -maxRetries: 5 - -env: -NUGET_PACKAGES: 'C:\nuget_global_cache' -NUGET_HTTP_CACHE_PATH: 'C:\nuget_http_cache' -NUGET_PLUGINS_CACHE_PATH: 'C:\nuget_plugins_cache' - -pre: -# https://docs.microsoft.com/en-us/dotnet/core/tools/dotnet-list-package -- npm install playwright@1.25.0 --save-exact -- dotnet list PlaywrightDotnetTests.csproj package > packages.txt -- nuget locals all -clear -- dotnet build -c Release - -runtime: -language: dotnet -version: "6.0.303" - -testDiscovery: -type: raw -mode: remote -command: grep -lr 'GotoAsync' -ir --include=*.cs - -post: -- cat yaml/linux/dotnet_playwright_hyperexecute_autosplit_sample.yaml - -testRunnerCommand: dotnet run $test - -jobLabel: [playwright-Dotnet, linux, autosplit] -``` - -### Step 4: Execute your Test Suite - -> **NOTE :** In case of MacOS, if you get a permission denied warning while executing CLI, simply run **`chmod u+x ./hyperexecute`** to allow permission. In case you get a security popup, allow it from your **System Preferences** → **Security & Privacy** → **General tab**. - -Run the below command in your terminal at the root folder of the project: - -```bash -./hyperexecute --config RELATIVE_PATH_OF_YOUR_YAML_FILE -``` - -OR use this command if you have not exported your username and access key in the step 2. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE `} - -### Step 5: Monitor the Test Execution - -Visit the [HyperExecute Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) and check your Job status. - -### Step 6: Download Artifacts and Reports - -HyperExecute also facilitates the provision to download the [Artifacts](/support/docs/hyperexecute-artifacts/) and [Reports](/support/docs/hyperexecute-reports/) on your local machine. Click on the corresponding button to download your generated artifacts and reports. - -## 2. Testing Using Gitpod - -You can also use the Gitpod platform to execute our sample repository. It will fetch all the sample codebases and trigger the CLI to execute the tests. - -Follow the below steps to run Test using Gitpod: - -**Step 1:** Click '**Open in Gitpod**' button. You will be redirected to Login/Signup page. This button is configured to redirect you to the Gitpod platform where you will be able to execute our sample repository. - -**Step 2:** Login with TestMu AI credentials. Once logged in, a pop-up confirmation will appear, asking you to **'Proceed'** to the Gitpod editor in a new tab. The current tab will display the HyperExecute Dashboard. - -**Step 3:** Choose your preferred editor (we recommend VS Code Editor) - -**Step 4:** As you are running a sample project, Fetching of the Test Scripts, [HyperExecute YAML](/support/docs/deep-dive-into-hyperexecute-yaml/), [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) and Triggering your tests using the `Execution Command` will be automated. - -**Step 5:** Once you see the `Job Link` in the logs, you can visit the [HyperExecute dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) to see the tests getting executed. - -You can also implement [Secret Keys](/support/docs/hyperexecute-how-to-save-and-manage-secrets/) in your YAML file. - -## Navigation in Automation Dashboard - -Every test run on the HyperExecute has a unique *jobId* associated with it. Each *jobId* can in turn constitute single (or multiple) *groupId*(s). You can visit [HyperExecute Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) for checking the status of the test execution. - -You can seamlessly navigate between JobId's and taskId's. You need to click on the *testID* to navigate from the HyperExecute logs to the Automation Dashboard. - -The snapshot below shows the videos, logs and other meta data for that specific *test_ID* - -> For any query or doubt, please feel free to contact us via **24×7 chat support** or you can also drop a mail to **support@testmuai.com**. -Happy testing! - -## Using the Playwright Agent Skill with TestMu AI - -The [playwright-skill](https://github.com/LambdaTest/agent-skills/tree/main/playwright-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The playwright-skill package includes: - -``` -playwright-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing Playwright Agent Skill - -Install a Playwright Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/playwright-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/playwright-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only playwright-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). diff --git a/static/docs/playwright-html-report.md b/static/docs/playwright-html-report.md deleted file mode 100644 index a1fd64f89..000000000 --- a/static/docs/playwright-html-report.md +++ /dev/null @@ -1,40 +0,0 @@ -# Consolidated Playwright HTML Report - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Playwright offers built-in HTML reports to help visualize the results of your test runs. These reports are detailed, user-friendly, and interactive, allowing you to analyze each test's status, logs, and errors in an organized manner. - -## Steps to Generate Consolidated HTML Report -Follow these steps to generate a consolidated Playwright HTML report: - -### Step 1: Update Your Playwright Configuration - -In your Playwright configuration file `playwright.config.js`, add the following code to enable the reporter: - -```javascript title="playwright.config.js" -import { devices } from '@playwright/test'; - -const config = { -...// -reporter: [['html', { open: 'never' }]], -...// -}; -``` - -- In the case of HyperExecute, we always have to set the value of `open: never`. - -- By default, the report is written into the **playwright-report folder** in the current working directory. The same location can be used in the report parameters in the YAML file. - -### Step 2: Configure the HyperExecute YAML File -In your HyperExecute YAML configuration, define the [`report`](/support/docs/deep-dive-into-hyperexecute-yaml/#report) parameters like this: - -```yaml title="hyperexecute.yaml" -report: true -partialReports: -frameworkName: playwright -location: playwright-report -type: html -``` - -### Step 3: Execute Your Tests -Run your Playwright tests on HyperExecute using the CLI. After your job completes, you can visit the HyperExecute dashboard to download and view the consolidated Plywright HTML report. diff --git a/static/docs/playwright-integration-with-hyperexecute.md b/static/docs/playwright-integration-with-hyperexecute.md deleted file mode 100644 index ad8aadd93..000000000 --- a/static/docs/playwright-integration-with-hyperexecute.md +++ /dev/null @@ -1,104 +0,0 @@ -# Playwright Integration with HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -In the ever-evolving realm of web development, the need for a comprehensive and efficient solution for cross-browser testing became paramount. Developers and QA engineers yearned for a framework that transcended the limitations of existing tools. They craved a solution that could address challenges like: - -- **Brittle and Flaky Tests:** Tests that randomly fail or pass, hindering reliable test results. - -- **Laborious Debugging:** Struggling to identify the root cause of issues across different browsers. - -- **Sequential Test Execution:** Slow test run times, especially for large test suites, delaying valuable feedback. - -Hence they turned to Playwright, a revolutionary framework designed to address these very roadblocks. With its robust capabilities. Playwright promised a new era of streamlined and dependable web application testing. - -As a web application developer or QA engineer, I want to leverage Playwright's powerful automation capabilities for cross-browser testing on a massive scale. So that I can ensure my application functions flawlessly across a wide range of real devices and browsers. - -However, for larger projects and complex testing scenarios, certain challenges emerged, including **Flaky Tests Detections**, **Test Parallelization**, **CI Pipeline Infrastructure**, **Fail Fasts mechanism**, **Detailed Reports** etc. Here comes a need for an effective companion to augment the capabilities of Playwright. - -## Advantages of Integrating Playwright with HyperExecute - -**HyperExecute**, a powerful platform with features designed to seamlessly complement Playwright and address its limitations via: - - Intelligently Orchestrate Your Tests - Use Smart Auto Split Strategy, Matrix Strategy or Hybrid Strategy modes to split and execute tests as per your requirements. - - Detailed Test Reports and Logs - Get a detailed test report for every job and stream the terminal logs in real time to understand everything that happened with your test. - - Fail Fast Mechanism - Run your jobs faster or fail them quickly to receive instant feedback and save your test time. - - Jobs Prioritization - Label the jobs that you want to run first on HyperExecute by giving them a higher priority. - - Comprehensive Artifacts of Your Job - HyperExecute also manages all of your test artifacts and provides you with the option to implement smart workflows on your test code without modifying it. - - AI Native Root Cause Analysis - Root Cause Analysis and Error Classification features in HyperExecute to view different kinds of errors and directly land on corrective measures or fixes. - - Background Services - Triggers long-running Application Servers tasks like running WebApps or Databases and generates the logs report in the Dashboard. - - Auto Healing - Auto Healing Feature allows you to automatically rеcovеr from cеrtain typеs of failurеs during thе еxеcution of your tеst scripts. - - Projects - Setup your custom projects using HyperExecute GUI itself and initiate the tests execution from the platform only. - - Jobs Archiving - Jobs Archiving allows you to archive the non important jobs so that you and your team can focus only on required ones. - - Smart Caching - Repeatedly fetching data during tests can slow down execution. HyperExecute's intelligent caching stores frequently accessed data, significantly speeding up test runs and optimizing resource utilization. - - Mute Your Flaky Tests - Mute scenarios that have been continuously failing for a pre-defined number of times, ignore expected failures, achieve better runtimes and faster feedback on the executed Jobs. - - Auto Retries - Occasional test failures happen. HyperExecute's configurable retry mechanism allows you to automatically re-run flaky or failed tests based on defined criteria, ensuring accurate results and preventing false negatives. - - Secure Upload, Zero Hassle - HyperExecute streamlines your CI/CD workflows by eliminating the need for manual test script uploads. Configure the `sourcePayload` parameter in your HyperExecute YAML file, and it will securely access your test scripts directly from your Git repository using secure access tokens. - - Incremental Updates for Seamless Testing - If your codebase changes are less than 75%, HyperExecute intelligently identifies and uploads only the updated or added parts, efficiently mapping the remainder from previous uploads. This approach minimizes wait times and streamlines your testing cycles, leading to faster results and a smoother overall testing experience. - -## Using the Playwright Agent Skill with TestMu AI - -The [playwright-skill](https://github.com/LambdaTest/agent-skills/tree/main/playwright-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The playwright-skill package includes: - -``` -playwright-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing Playwright Agent Skill - -Install a Playwright Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/playwright-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/playwright-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only playwright-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). diff --git a/static/docs/playwright-ios-device.md b/static/docs/playwright-ios-device.md deleted file mode 100644 index 305d8f8a3..000000000 --- a/static/docs/playwright-ios-device.md +++ /dev/null @@ -1,510 +0,0 @@ -# Getting Started With Playwright Testing on iOS Real Devices - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Playwright test automation on real iOS devices is now supported on TestMu AI across **Node.js, Java, C#, and Python**. Test on latest iPhone and iPad Safari combinations to catch device-specific issues that mobile emulation may miss. Integrate with your existing CI pipeline, and access logs and debugging artifacts for each test run. - -This guide will cover the basics of getting started with Playwright testing on iOS devices on the TestMu AI platform. - -**Currently in BETA** - -Playwright testing on real iOS devices is currently in **Beta**. To enable this feature for your organization, please contact your account team to have the feature flag turned on. - -**Supported Versions** -- Playwright versions **v1.53.0** and above (until **v1.6.0**) are supported for iOS real device testing. -- All languages use the **stock Playwright packages**, with no custom forks or client-side changes required. -- Playwright v1.53.0 is currently supported for Playwright C# (for Android & iOS). - -## Prerequisites - -Set your TestMu AI username and access key in the environment variables. You can get your TestMu AI username and access key from your TestMu AI Profile > Account Settings > Password & Security. - -**Windows** - -```bash -set LT_USERNAME="YOUR_LAMBDATEST_USERNAME" -set LT_ACCESS_KEY="YOUR_LAMBDATEST_ACCESS_KEY" -``` - -**macOS/Linux** - -```bash -export LT_USERNAME="YOUR_LAMBDATEST_USERNAME" -export LT_ACCESS_KEY="YOUR_LAMBDATEST_ACCESS_KEY" -``` - -### Language-Specific Setup - -| Language | Supported Playwright Versions | -|----------|-------------------------------| -| **JavaScript (Node.js)** | Up to **v1.59.0** | -| **Java, Python, C#** | Up to **v1.53.2** | - -Install the Playwright package: - -```bash -npm install playwright -``` - -Install the Playwright package: - -```bash -pip install playwright -``` - -Add the Playwright dependency to your `pom.xml`: - -```xml - -com.microsoft.playwright -playwright -1.54.0 - -``` - -Add the Playwright NuGet package: - -```bash -dotnet add package Microsoft.Playwright -``` - -## Run Your First Test - -```javascript title="playwright-ios-test.js" -const { webkit } = require("playwright"); - -(async () => { -const capabilities = { -"LT:Options": { -"platformName": "ios", -"deviceName": "iPhone 16", -"platformVersion": "18", -"isRealMobile": true, -"build": "Playwright iOS Build", -"name": "Playwright iOS Test", -"user": process.env.LT_USERNAME, -"accessKey": process.env.LT_ACCESS_KEY, -"network": true, -"video": true, -"console": true, -}, -}; - -const browser = await webkit.connect( -`wss://cdp.lambdatest.com/playwright?capabilities=${encodeURIComponent( -JSON.stringify(capabilities) -)}` -); - -const context = await browser.newContext(); -const page = await context.newPage(); - -await page.goto("https://duckduckgo.com", { timeout: 30000 }); -await page.locator('[name="q"]').fill("LambdaTest"); -await page.locator('[name="q"]').press("Enter"); -await page.waitForTimeout(3000); - -const title = await page.title(); -console.log("Page title:", title); - -try { -if (title.includes("LambdaTest")) { -await page.evaluate( -(_) => {}, -`lambdatest_action: ${JSON.stringify({ -action: "setTestStatus", -arguments: { status: "passed", remark: "Title verified" }, -})}` -); -} -} catch (e) { -await page.evaluate( -(_) => {}, -`lambdatest_action: ${JSON.stringify({ -action: "setTestStatus", -arguments: { status: "failed", remark: e.message }, -})}` -); -} - -await page.close(); -await context.close(); -await browser.close(); -})(); -``` - -Run the test: - -```bash -node playwright-ios-test.js -``` - -```python title="playwright_ios_test.py" -import os, json, urllib.parse -from playwright.sync_api import sync_playwright - -def main(): -capabilities = { -"LT:Options": { -"platformName": "ios", -"deviceName": "iPhone 16", -"platformVersion": "18", -"isRealMobile": True, -"build": "Playwright iOS Build", -"name": "Playwright iOS Test", -"user": os.environ["LT_USERNAME"], -"accessKey": os.environ["LT_ACCESS_KEY"], -"network": True, -"video": True, -"console": True, -} -} - -ws_endpoint = ( -f"wss://cdp.lambdatest.com/playwright?capabilities=" -f"{urllib.parse.quote(json.dumps(capabilities))}" -) - -with sync_playwright() as p: -browser = p.webkit.connect(ws_endpoint) -context = browser.new_context() -page = context.new_page() - -page.goto("https://duckduckgo.com", timeout=30000) -page.locator('[name="q"]').fill("LambdaTest") -page.locator('[name="q"]').press("Enter") -page.wait_for_timeout(3000) - -title = page.title() -print(f"Page title: {title}") - -try: -if "LambdaTest" in title: -page.evaluate( -"_ => {}", -'lambdatest_action: {"action": "setTestStatus", "arguments": {"status": "passed", "remark": "Title verified"}}', -) -except Exception as e: -page.evaluate( -"_ => {}", -f'lambdatest_action: {json.dumps({"action": "setTestStatus", "arguments": {"status": "failed", "remark": str(e)}})}', -) - -page.close() -context.close() -browser.close() - -if __name__ == "__main__": -main() -``` - -Run the test: - -```bash -python playwright_ios_test.py -``` - -```java title="PlaywrightIosTest.java" -package com.lambdatest; - -import com.microsoft.playwright.*; -import com.google.gson.Gson; -import java.net.URLEncoder; -import java.nio.charset.StandardCharsets; -import java.util.Map; - -public class PlaywrightIosTest { -public static void main(String[] args) { -Map ltOptions = Map.of( -"platformName", "ios", -"deviceName", "iPhone 16", -"platformVersion", "18", -"isRealMobile", true, -"build", "Playwright iOS Build", -"name", "Playwright iOS Test", -"user", System.getenv("LT_USERNAME"), -"accessKey", System.getenv("LT_ACCESS_KEY"), -"network", true, -"video", true, -"console", true -); - -Map capabilities = Map.of("LT:Options", ltOptions); -String capsJson = new Gson().toJson(capabilities); -String wsEndpoint = "wss://cdp.lambdatest.com/playwright?capabilities=" -+ URLEncoder.encode(capsJson, StandardCharsets.UTF_8); - -try (Playwright playwright = Playwright.create()) { -Browser browser = playwright.webkit().connect(wsEndpoint); -BrowserContext context = browser.newContext(); -Page page = context.newPage(); - -page.navigate("https://duckduckgo.com", -new Page.NavigateOptions().setTimeout(30000)); -page.locator("[name=\"q\"]").fill("LambdaTest"); -page.locator("[name=\"q\"]").press("Enter"); -page.waitForTimeout(3000); - -String title = page.title(); -System.out.println("Page title: " + title); - -try { -if (title.contains("LambdaTest")) { -page.evaluate("_ => {}", -"lambdatest_action: {\"action\": \"setTestStatus\", \"arguments\": {\"status\": \"passed\", \"remark\": \"Title verified\"}}"); -} -} catch (Exception e) { -page.evaluate("_ => {}", -"lambdatest_action: {\"action\": \"setTestStatus\", \"arguments\": {\"status\": \"failed\", \"remark\": \"" + e.getMessage() + "\"}}"); -} - -page.close(); -context.close(); -browser.close(); -} -} -} -``` - -Run the test: - -```bash -mvn compile exec:java -Dexec.mainClass="com.lambdatest.PlaywrightIosTest" -``` - -```csharp title="PlaywrightIosTest.cs" -using Microsoft.Playwright; -using System.Text.Json; -using System.Web; - -var capabilities = new Dictionary -{ -["LT:Options"] = new Dictionary -{ -["platformName"] = "ios", -["deviceName"] = "iPhone 16", -["platformVersion"] = "18", -["isRealMobile"] = true, -["build"] = "Playwright iOS Build", -["name"] = "Playwright iOS Test", -["user"] = Environment.GetEnvironmentVariable("LT_USERNAME")!, -["accessKey"] = Environment.GetEnvironmentVariable("LT_ACCESS_KEY")!, -["network"] = true, -["video"] = true, -["console"] = true, -["playwrightClientVersion"] = "1.53.0", -} -}; - -var capsJson = JsonSerializer.Serialize(capabilities); -var wsEndpoint = $"wss://cdp.lambdatest.com/playwright?capabilities={HttpUtility.UrlEncode(capsJson)}"; - -using var playwright = await Playwright.CreateAsync(); -var browser = await playwright.Webkit.ConnectAsync(wsEndpoint); -var context = await browser.NewContextAsync(); -var page = await context.NewPageAsync(); - -await page.GotoAsync("https://duckduckgo.com", new PageGotoOptions { Timeout = 30000 }); -await page.Locator("[name=\"q\"]").FillAsync("LambdaTest"); -await page.Locator("[name=\"q\"]").PressAsync("Enter"); -await page.WaitForTimeoutAsync(3000); - -var title = await page.TitleAsync(); -Console.WriteLine($"Page title: {title}"); - -try -{ -if (title.Contains("LambdaTest")) -{ -await page.EvaluateAsync("_ => {}", -"lambdatest_action: {\"action\": \"setTestStatus\", \"arguments\": {\"status\": \"passed\", \"remark\": \"Title verified\"}}"); -} -} -catch (Exception e) -{ -await page.EvaluateAsync("_ => {}", -$"lambdatest_action: {{\"action\": \"setTestStatus\", \"arguments\": {{\"status\": \"failed\", \"remark\": \"{e.Message}\"}}}}"); -} - -await page.CloseAsync(); -await context.CloseAsync(); -await browser.CloseAsync(); -``` - -Run the test: - -```bash -dotnet run -``` - -## Apple Pay Automation - -Automate the Apple Pay checkout flow on a real iOS device using Playwright over the TestMu AI CDP endpoint (`wss://cdp.lambdatest.com/playwright`). When enabled, the platform provisions Wallet, a sandbox card, and the device passcode on the real iPhone — so you never interact with Face ID / Touch ID or set up Wallet manually. - -- Apple Pay runs on **WebKit/Safari** and is supported across **all languages** available for Playwright iOS testing. The hook calls use the same `lambdatest_action` server-side channel shown under [Run Your First Test](#run-your-first-test), so the same syntax applies in every language. -- To enable Apple Pay for your organization, please contact us via **24×7 chat support** or drop a mail to **support@testmuai.com**. - -### Capabilities - -| Capability | Type | Default | Required / Optional | Description | -|------------|------|---------|---------------------|-------------| -| **applePay** | Boolean | false | Mandatory | Enables Apple Pay on the session — provisions Wallet, a sandbox card, and the device passcode on supported real iOS devices. | -| **applePayCardType** | Array | None | Optional | Preferred payment network(s) in priority order. Supported values: `["master", "visa", "amex", "discover"]`. The first network is preferred; the rest act as fallbacks. If omitted, a default sandbox card is provisioned. | - -Add the Apple Pay keys to the **same `LT:Options` object** you already use to start your Playwright session (see [Run Your First Test](#run-your-first-test)): - -```javascript -const capabilities = { -"LT:Options": { -// ...your existing iOS capabilities (platformName, deviceName, platformVersion, user, accessKey, etc.) -// highlight-start -"applePay": true, -"applePayCardType": ["master", "visa"], // priority order — master preferred, visa as fallback -// highlight-end -}, -}; -``` - -#### Passcode Capabilities - -Adding a card to Wallet requires a device passcode: - -- **Public cloud** — no extra capability is needed. The confirm hook handles the passcode automatically. -- **Private cloud** — use the `passcode` capability to set a custom passcode value directly on the device. Add it inside `LT:Options` alongside `applePay`: - -```javascript -// Private cloud only — set a custom passcode -"LT:Options": { /* ...other caps */, "applePay": true, "passcode": "654321" } -``` - -On **iOS 26**, the `lambda-applepay` confirm hook enters the device passcode automatically — the custom `passcode` on private cloud, or the default passcode on public cloud. No separate passcode step is required. - -### Validation - -Before either Apple Pay hook executes, the gateway validates: - -1. **iOS version ≥ 14** — Apple Pay hooks are rejected on older platform versions. -2. **Apple Pay capability present** — `applePay: true` must be set in `LT:Options`. - -If either check fails, the hook is not executed and an error is returned to the session. - -### Hooks - -Apple Pay hooks are invoked through the TestMu AI server-side action channel — the native Apple Pay sheet is not reachable by Playwright directly. A small reusable wrapper keeps the calls readable: - -```javascript -async function ltAction(page, action, args = {}) { -return page.evaluate( -(_) => {}, -`lambdatest_action: ${JSON.stringify({ action, arguments: args })}` -); -} -``` - -#### Hook 1 — `lambda-applepay-details` (pre-fill the sheet) - -Sets shipping, billing, and contact details on the Apple Pay sheet. Call it **before** launching the sheet. Optional — use it when your merchant requires shipping/contact info. - -```javascript -await ltAction(page, "lambda-applepay-details", { -shippingDetails: { -firstName: "John", lastName: "Doe", -street: "1 Infinite Loop", city: "Cupertino", -state: "California", postalCode: "95014", country: "United States", -}, -billingDetails: { -firstName: "John", lastName: "Doe", -street: "1 Infinite Loop", city: "Cupertino", -state: "California", postalCode: "95014", country: "United States", -email: "john.doe@example.com", phone: "+14085551234", -}, -contact: { -firstName: "John", lastName: "Doe", -email: "john.doe@example.com", phone: "+14085551234", -}, -}); -``` - -#### Hook 2 — `lambda-applepay` (confirm / authorize payment) - -Confirms the native Apple Pay sheet to authorize the transaction. - -```javascript -await ltAction(page, "lambda-applepay", { confirm: true }); -``` - -On **iOS 26**, the confirm hook automatically enters the device passcode — one call confirms the sheet and authorizes the payment end to end. On earlier iOS versions, the passcode is entered as a separate step after confirm. - -### End-to-End Example - -```javascript title="apple-pay.spec.js" -const { webkit } = require("playwright"); - -// Reusable wrapper for any TestMu AI server-side action. -async function ltAction(page, action, args = {}) { -return page.evaluate( -(_) => {}, -`lambdatest_action: ${JSON.stringify({ action, arguments: args })}` -); -} - -(async () => { -const capabilities = { -"LT:Options": { -platformName: "ios", -deviceName: "iPhone 16", -platformVersion: "26", -isRealMobile: true, -user: process.env.LT_USERNAME, -accessKey: process.env.LT_ACCESS_KEY, -build: "Apple Pay 26.0", -name: "Apple Pay via Playwright", -applePay: true, -applePayCardType: ["master", "visa"], -}, -}; - -const browser = await webkit.connect( -`wss://cdp.lambdatest.com/playwright?capabilities=${encodeURIComponent( -JSON.stringify(capabilities) -)}` -); - -const context = await browser.newContext(); -const page = await context.newPage(); - -// Navigate to your checkout page and trigger the Apple Pay sheet here... - -// Optional: pre-fill shipping / billing / contact on the sheet. -await ltAction(page, "lambda-applepay-details", { -billingDetails: { -firstName: "John", lastName: "Doe", -street: "1 Infinite Loop", city: "Cupertino", -state: "California", postalCode: "95014", country: "United States", -email: "john.doe@example.com", phone: "+14085551234", -}, -}); - -// Confirm the sheet. On iOS 26 the passcode is entered automatically. -await ltAction(page, "lambda-applepay", { confirm: true }); - -// Assert your post-payment state (swap for a real locator on your app). -// await page.getByText(/order confirmed/i).waitFor(); - -await page.close(); -await context.close(); -await browser.close(); -})(); -``` - -The `ltAction` helper is generic — reuse it for `setTestStatus`, `smartui.takeScreenshot`, or any other TestMu AI action. - -## View your Playwright test results - -The TestMu AI Automation Dashboard is where you can see the results of your Playwright iOS tests after running them on the TestMu AI platform. - -The below screenshot of TestMu AI Automation Dashboard shows the Playwright build on the left and the build sessions associated with the selected build on the right. - -- Safari is the supported browser for iOS real device testing. All four languages (**Node.js, Java, C#, and Python**) are supported using stock Playwright packages. - -- Playwright testing on real iOS devices is currently supported on latest iOS versions (iOS 17, iOS 18, and iOS 26) across both iPhones and iPads. diff --git a/static/docs/playwright-jest-on-hyperexecute.md b/static/docs/playwright-jest-on-hyperexecute.md deleted file mode 100644 index d62b7024e..000000000 --- a/static/docs/playwright-jest-on-hyperexecute.md +++ /dev/null @@ -1,206 +0,0 @@ -# Running Jest Framework Tests on HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Jest is a popular JavaScript testing framework primarily used for testing React applications, although it can be employed for any JavaScript project. Developed by Facebook, Jest is known for its simplicity and speed. It provides a testing environment, assertion utilities, and features like snapshot testing, parallel test execution, and mocking to ensure efficient and reliable testing of JavaScript codebases. - -HyperExecute is an AI Native Test Orchestration Cloud Platform that empowers you to run **end-to-end** tests **quickly** and **efficiently**. It provides Just-in-Time (JIT) testing infrastructure with fast execution **speeds**, **smart orchestration**, and **detailed logs**. - -This guide details how to execute your **Jest** framework tests on **HyperExecute** via two different methods: - -- [**Using Local System**](/support/docs/playwright-jest-on-hyperexecute/#1-testing-using-local-system) - You can use your own local machine to execute tests. -- [**Using Gitpod Platform**](/support/docs/playwright-jest-on-hyperexecute/#2-testing-using-gitpod) - Execute tests using GitPod. (Requires a [Gitpod](https://gitpod.io/login/) account) - -## 1. Testing Using Local System - -Follow the step-by-step guide to execute your test on HyperExecute. - -### Prerequisites - -To run the Tests on HyperExecute from your Local System, you are required: - -- Your TestMu AI [Username and Access key](/support/docs/hyperexecute-how-to-get-my-username-and-access-key/) -- [HyperExecute YAML](/support/docs/hyperexecute-yaml-version0.2/) file which contains all the necessary instructions. -- [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) in order to initiate a test execution Job . -- Setup the [Environmental Variable](/support/docs/hyperexecute-environment-variable-setup/) - -### Step 1: Configure Your Test Suite - -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** - -Download or Clone the code sample for the JUnit from the TestMu AI GitHub repository to run the tests on the HyperExecute. - - View on GitHub - -If you are using your own project, make sure you update the **Hub endpoint** in your tests file. - -By setting up the Hub endpoint, you establish the communication channel between your tests and the browser nodes, enabling effective test distribution and execution. - -> You can also use **dev** and **beta** browser versions. To generate capabilities for your test requirements, you can use our inbuilt 🔗 Capabilities Generator. - -### Step 2: Setup the CLI in your Test Suite - -After cloning / downloading the sample repo, you need to setup the CLI and the environment variables. - -#### Download the HyperExecute CLI - -The CLI is used for triggering the tests on HyperExecute. It is recommend to download the CLI binary on the host system and keep it in the root directory of the suite to perform the tests on HyperExecute. - -You can download the CLI for your desired platform from the below mentioned links: - -| Platform | HyperExecute CLI | -| ---------| ---------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| MacOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -#### Setup Environment Variable - -Now, you need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in the [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). - -Run the below mentioned commands in your terminal to setup the CLI and the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -### Step 3: Configure YAML in your Test Suite - -Configure your YAML file as per your use cases using **key value** pairs. - -In this sample YAML file, we have mentioned: - -- **version** of the YAML file -- **Timeouts** for executing your project -- **Mode of execution** is [Autosplit](/support/docs/hyperexecute-auto-split-strategy/). You can also opt for [Matrix](/support/docs/hyperexecute-matrix-multiplexing-strategy/) or [Hybrid](/support/docs/hyperexecute-hybrid-strategy/) mode. -- **Pre and Post** commands -- **Reports and Artefacts** that will be generated after the completion of tests -- and other necessary YAML Parameters - -```yaml ---- -version: "0.1" - -globalTimeout: 90 -testSuiteTimeout: 90 -testSuiteStep: 90 - -runson: linux - -autosplit: true - -retryOnFailure: false -maxRetries: 1 - -concurrency: 2 - -pre: -- npm install -- npx playwright install - -cacheKey: '{{ checksum "package-lock.json" }}' -cacheDirectories: -- node_modules - -testDiscovery: -type: raw -mode: remote -command: grep -lr 'describe' *.test.js - -testRunnerCommand: npm test -- $test - -jobLabel: [playwright-JestJS, linux, autosplit] -``` - -### Step 4: Execute your Test Suite - -> **NOTE :** In case of MacOS, if you get a permission denied warning while executing CLI, simply run **`chmod u+x ./hyperexecute`** to allow permission. In case you get a security popup, allow it from your **System Preferences** → **Security & Privacy** → **General tab**. - -Run the below command in your terminal at the root folder of the project: - -```bash -./hyperexecute --config RELATIVE_PATH_OF_YOUR_YAML_FILE -``` - -OR use this command if you have not exported your username and access key in the step 2. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE `} - -### Step 5: Monitor the Test Execution - -Visit the [HyperExecute Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) and check your Job status. - -### Step 6: Download Artifacts and Reports - -HyperExecute also facilitates the provision to download the [Artifacts](/support/docs/hyperexecute-artifacts/) and [Reports](/support/docs/hyperexecute-reports/) on your local machine. Click on the corresponding button to download your generated artifacts and reports. - -## 2. Testing Using Gitpod - -You can also use the Gitpod platform to execute our sample repository. It will fetch all the sample codebases and trigger the CLI to execute the tests. - -Follow the below steps to run Test using Gitpod: - -**Step 1:** Click '**Open in Gitpod**' button. You will be redirected to Login/Signup page. This button is configured to redirect you to the Gitpod platform where you will be able to execute our sample repository. - -**Step 2:** Login with TestMu AI credentials. Once logged in, a pop-up confirmation will appear, asking you to **'Proceed'** to the Gitpod editor in a new tab. The current tab will display the HyperExecute Dashboard. - -**Step 3:** Choose your preferred editor (we recommend VS Code Editor) - -**Step 4:** As you are running a sample project, Fetching of the Test Scripts, [HyperExecute YAML](/support/docs/deep-dive-into-hyperexecute-yaml/), [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) and Triggering your tests using the `Execution Command` will be automated. - -**Step 5:** Once you see the `Job Link` in the logs, you can visit the [HyperExecute dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) to see the tests getting executed. - -You can also implement [Secret Keys](/support/docs/hyperexecute-how-to-save-and-manage-secrets/) in your YAML file. - -## Navigation in Automation Dashboard - -Every test run on the HyperExecute has a unique *jobId* associated with it. Each *jobId* can in turn constitute single (or multiple) *groupId*(s). You can visit [HyperExecute Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) for checking the status of the test execution. - -You can seamlessly navigate between JobId's and taskId's. You need to click on the *testID* to navigate from the HyperExecute logs to the Automation Dashboard. - -The snapshot below shows the videos, logs and other meta data for that specific *test_ID* - -> For any query or doubt, please feel free to contact us via **24×7 chat support** or you can also drop a mail to **support@testmuai.com**. -Happy testing! - -## Using the Jest Agent Skill with TestMu AI - -The [jest-skill](https://github.com/LambdaTest/agent-skills/tree/main/jest-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The jest-skill package includes: - -``` -jest-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing Jest Agent Skill - -Install a Jest Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/jest-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/jest-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only jest-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). diff --git a/static/docs/playwright-junit-on-hyperexecute.md b/static/docs/playwright-junit-on-hyperexecute.md deleted file mode 100644 index e21443fb0..000000000 --- a/static/docs/playwright-junit-on-hyperexecute.md +++ /dev/null @@ -1,224 +0,0 @@ -# Running JUnit Framework Tests on HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -JUnit is a widely-used testing framework for Java applications, designed to simplify and enhance the testing process for developers. It provides a flexible and powerful platform for running test suites, enabling effective unit testing, integration testing, and end-to-end testing of Java applications - -HyperExecute is an AI Native Test Orchestration Cloud Platform that empowers you to run **end-to-end** tests **quickly** and **efficiently**. It provides Just-in-Time (JIT) testing infrastructure with fast execution **speeds**, **smart orchestration**, and **detailed logs**. - -This guide details how to execute your **JUnit** framework tests on **HyperExecute** via two different methods: - -- [**Using Local System**](/support/docs/playwright-junit-on-hyperexecute/#1-testing-using-local-system) - You can use your own local machine to execute tests. -- [**Using Gitpod Platform**](/support/docs/playwright-junit-on-hyperexecute/#2-testing-using-gitpod) - Execute tests using GitPod. (Requires a [Gitpod](https://gitpod.io/login/) account) - -## 1. Testing Using Local System - -Follow the step-by-step guide to execute your test on HyperExecute. - -### Prerequisites - -To run the Tests on HyperExecute from your Local System, you are required: - -- Your TestMu AI [Username and Access key](/support/docs/hyperexecute-how-to-get-my-username-and-access-key/) -- [HyperExecute YAML](/support/docs/hyperexecute-yaml-version0.2/) file which contains all the necessary instructions. -- [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) in order to initiate a test execution Job . -- Setup the [Environmental Variable](/support/docs/hyperexecute-environment-variable-setup/) - -### Step 1: Configure Your Test Suite - -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** - -Download or Clone the code sample for the JUnit from the TestMu AI GitHub repository to run the tests on the HyperExecute. - - View on GitHub - -If you are using your own project, make sure you update the **Hub endpoint** in your tests file. - -By setting up the Hub endpoint, you establish the communication channel between your tests and the browser nodes, enabling effective test distribution and execution. - -> You can also use **dev** and **beta** browser versions. To generate capabilities for your test requirements, you can use our inbuilt 🔗 Capabilities Generator. - -### Step 2: Setup the CLI in your Test Suite - -After cloning / downloading the sample repo, you need to setup the CLI and the environment variables. - -#### Download the HyperExecute CLI - -The CLI is used for triggering the tests on HyperExecute. It is recommend to download the CLI binary on the host system and keep it in the root directory of the suite to perform the tests on HyperExecute. - -You can download the CLI for your desired platform from the below mentioned links: - -| Platform | HyperExecute CLI | -| ---------| ---------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| MacOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -#### Setup Environment Variable - -Now, you need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in the [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). - -Run the below mentioned commands in your terminal to setup the CLI and the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -### Step 3: Configure YAML in your Test Suite - -Configure your YAML file as per your use cases using **key value** pairs. - -In this sample YAML file, we have mentioned: - -- **version** of the YAML file -- **Timeouts** for executing your project -- **Mode of execution** is [Autosplit](/support/docs/hyperexecute-auto-split-strategy/). You can also opt for [Matrix](/support/docs/hyperexecute-matrix-multiplexing-strategy/) or [Hybrid](/support/docs/hyperexecute-hybrid-strategy/) mode. -- **Pre and Post** commands -- **Reports and Artefacts** that will be generated after the completion of tests -- and other necessary YAML Parameters - -```yaml ---- -version: 0.1 -globalTimeout: 150 -testSuiteTimeout: 150 -testSuiteStep: 150 - -runson: linux - -autosplit: true -retryOnFailure: false - -maxRetries: 1 -concurrency: 1 - -env: -# PAT: ${{ .secrets.testKey }} -CACHE_DIR: m2_cache_dir -TARGET_OS: linux - -cacheKey: '{{ checksum "pom.xml" }}' -cacheDirectories: -- ${CACHE_DIR} - -shell: bash - -pre: -# install playwright version to be used. -- npm install playwright@1.23.0 --save-exact -# Download and install packages in the CACHE_DIR. -# Skip execution of the tests in the pre step -- mvn -Dmaven.repo.local=${CACHE_DIR} -Dmaven.test.skip=true clean install - -post: -- cat yaml/linux/junit_hyperexecute_autosplit_sample.yaml - -mergeArtifacts: true - -uploadArtefacts: -- name: Surefire-Report -path: -- target/surefire-reports/** - -testDiscovery: -type: raw -mode: remote -command: grep 'public class' src/test/java/SampleTest.java | awk '{print$3}' - -testRunnerCommand: mvn -Dplatname=linux -Dmaven.repo.local=m2_cache_dir -Dtest=$test test surefire-report:report - -jobLabel: [playwright-Junit, linux, autosplit] -``` - -### Step 4: Execute your Test Suite - -> **NOTE :** In case of MacOS, if you get a permission denied warning while executing CLI, simply run **`chmod u+x ./hyperexecute`** to allow permission. In case you get a security popup, allow it from your **System Preferences** → **Security & Privacy** → **General tab**. - -Run the below command in your terminal at the root folder of the project: - -```bash -./hyperexecute --config RELATIVE_PATH_OF_YOUR_YAML_FILE -``` - -OR use this command if you have not exported your username and access key in the step 2. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE `} - -### Step 5: Monitor the Test Execution - -Visit the [HyperExecute Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) and check your Job status. - -### Step 6: Download Artifacts and Reports - -HyperExecute also facilitates the provision to download the [Artifacts](/support/docs/hyperexecute-artifacts/) and [Reports](/support/docs/hyperexecute-reports/) on your local machine. Click on the corresponding button to download your generated artifacts and reports. - -## 2. Testing Using Gitpod - -You can also use the Gitpod platform to execute our sample repository. It will fetch all the sample codebases and trigger the CLI to execute the tests. - -Follow the below steps to run Test using Gitpod: - -**Step 1:** Click '**Open in Gitpod**' button. You will be redirected to Login/Signup page. This button is configured to redirect you to the Gitpod platform where you will be able to execute our sample repository. - -**Step 2:** Login with TestMu AI credentials. Once logged in, a pop-up confirmation will appear, asking you to **'Proceed'** to the Gitpod editor in a new tab. The current tab will display the HyperExecute Dashboard. - -**Step 3:** Choose your preferred editor (we recommend VS Code Editor) - -**Step 4:** As you are running a sample project, Fetching of the Test Scripts, [HyperExecute YAML](/support/docs/deep-dive-into-hyperexecute-yaml/), [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) and Triggering your tests using the `Execution Command` will be automated. - -**Step 5:** Once you see the `Job Link` in the logs, you can visit the [HyperExecute dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) to see the tests getting executed. - -You can also implement [Secret Keys](/support/docs/hyperexecute-how-to-save-and-manage-secrets/) in your YAML file. - -## Navigation in Automation Dashboard - -Every test run on the HyperExecute has a unique *jobId* associated with it. Each *jobId* can in turn constitute single (or multiple) *groupId*(s). You can visit [HyperExecute Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) for checking the status of the test execution. - -You can seamlessly navigate between JobId's and taskId's. You need to click on the *testID* to navigate from the HyperExecute logs to the Automation Dashboard. - -The snapshot below shows the videos, logs and other meta data for that specific *test_ID* - -> For any query or doubt, please feel free to contact us via **24×7 chat support** or you can also drop a mail to **support@testmuai.com**. -Happy testing! - -## Using the JUnit 5 Agent Skill with TestMu AI - -The [junit-5-skill](https://github.com/LambdaTest/agent-skills/tree/main/junit-5-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The junit-5-skill package includes: - -``` -junit-5-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing JUnit 5 Agent Skill - -Install a JUnit 5 Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/junit-5-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/junit-5-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only junit-5-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). diff --git a/static/docs/playwright-lighthouse-library.md b/static/docs/playwright-lighthouse-library.md deleted file mode 100644 index b38dd2115..000000000 --- a/static/docs/playwright-lighthouse-library.md +++ /dev/null @@ -1,86 +0,0 @@ -# Generate Lighthouse Reports With Playwright Lighthouse Library - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -A Lighthouse Report is a performance, accessibility, SEO, and best practices evaluation report generated using [Google Lighthouse](https://developers.google.com/web/tools/lighthouse), an open-source, automated tool developed by Google. It is commonly used to assess and optimize the quality of web pages. - -TestMu AI lets you perform Playwright automation while allowing you to generate Lighthouse reports with Playwright Lighthouse Library in a single test. - -> Lighthouse Report is supported on **Chrome**, **MicrosoftEdge** and **Chromium** browsers. - -**Sample repository** -The code sample for generating the Lighthouse Performance Metrics in the Playwright test can be found on **TestMu AI's GitHub Repository**. You can either download or clone the repository to quickly run your tests. View on GitHub - -## Prerequisites -- Ensure that you have the Playwright Lighthouse Library installed in your web project. -```bash -npm install playwright-lighthouse -``` - -- Export the *LIGHTHOUSE_LAMBDATEST* environment variable to your project environment. -```bash -export LIGHTHOUSE_LAMBDATEST='true' -``` - -## Write your Test Script - -Generating lighthouse report within the test might increase the test duration. It is advisable to generate the lighthouse reports only in the required tests. - -You can easily add the Lighthouse Performance Metrics of the website you want to test by using the `lighthouseReport` action via the test. Following is the code snippet for generating Lighthouse report from the Playwright if you are using JavaScript: -```js title="playwright-lighthouse-report.js" -await page.evaluate(_ => {}, `lambdatest_action: ${JSON.stringify({ -action: 'lighthouseReport', -arguments: { url: 'https://www.example.com' } -})}`) -``` - -### For Authenticated Pages -> This feature is only supported on **Windows** and **macOS** platforms. - -The feature allows you to generate Lighthouse reports for authenticated web pages using Playwright. This is particularly useful for testing performance, accessibility, SEO, and other quality metrics for pages that require authentication. By leveraging authentication tokens, you can analyze restricted pages in your Playwright tests on both **Windows** and **macOS** environments. - -```javascript -await page.evaluate(() => {}, `lambdatest_action: ${JSON.stringify({ -action: 'lighthouseReport', -arguments: { url: 'https://www.example.com', -args: `--extra-headers -${JSON.stringify({ authtoken: "YOUR_AUTH_TOKEN" })}` -} })}` ); -``` - -```javascript -await page.evaluate(() => {}, `lambdatest_action: ${JSON.stringify({ -action: 'lighthouseReport', -arguments: { url: 'https://www.example.com', -args: '--extra-headers -"{\\"authtoken\\": \\"YOUR_AUTH_TOKEN\\"}"' -} })}`); -``` - -### Sample Test Script - -This following script performs web automation testing using the Playwright and Lighthouse libraries on the TestMu AI platform. The script navigates to the DuckDuckGo search engine and searches for the term "Playwright". After that, the script runs a Lighthouse audit on the specified URL (https://duckduckgo.com) with defined performance thresholds and report formats. - -```javascript reference title="playwright-lighthouse-report.js" -https://github.com/LambdaTest/playwright-sample/blob/main/playwright-lighthouse-report.js -``` - -## Trigger your Test on TestMu AI -### Set up your authentication - -Make sure you have your TestMu AI credentials with you to run test automation scripts on TestMu AI. To obtain your access credentials, [purchase a plan](https://billing.lambdatest.com/billing/plans) or access the [Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://appautomation.lambdatest.com/). Then, set TestMu AI `Username` and `Access Key` in environment variables with following commands. - - {`export LT_USERNAME=${ YOUR_LAMBDATEST_USERNAME()} -export LT_ACCESS_KEY=${ YOUR_LAMBDATEST_ACCESS_KEY()}`} - - {`set LT_USERNAME=${ YOUR_LAMBDATEST_USERNAME()} -set LT_ACCESS_KEY=${ YOUR_LAMBDATEST_ACCESS_KEY()}`} - -### Run your test -Pass the below command in the terminal to run the test: -```bash -node RELATIVE_PATH_OF_YOUR_TEST_FILE -``` - -### View Your Test Results -To view your test runs that uses Playwright Lighthouse Library, go to the TestMu AI [Web Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/). diff --git a/static/docs/playwright-python-on-hyperexecute.md b/static/docs/playwright-python-on-hyperexecute.md deleted file mode 100644 index 792339ad6..000000000 --- a/static/docs/playwright-python-on-hyperexecute.md +++ /dev/null @@ -1,213 +0,0 @@ -# Running Python Framework Tests on HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Playwright is a powerful browser automation library that allows developers to automate browser interactions. When using Python with Playwright, developers can leverage a robust testing framework to create automated tests for web applications. This combination enables end-to-end testing, ensuring the functionality, performance, and reliability of web applications. - -HyperExecute is an AI Native Test Orchestration Cloud Platform that empowers you to run **end-to-end** tests **quickly** and **efficiently**. It provides Just-in-Time (JIT) testing infrastructure with fast execution **speeds**, **smart orchestration**, and **detailed logs**. - -This guide details how to execute your **Python** framework tests on **HyperExecute** via two different methods: - -- [**Using Local System**](/support/docs/playwright-python-on-hyperexecute/#1-testing-using-local-system) - You can use your own local machine to execute tests. -- [**Using Gitpod Platform**](/support/docs/playwright-python-on-hyperexecute/#2-testing-using-gitpod) - Execute tests using GitPod. (Requires a [Gitpod](https://gitpod.io/login/) account) - -## 1. Testing Using Local System - -Follow the step-by-step guide to execute your test on HyperExecute. - -### Prerequisites - -To run the Tests on HyperExecute from your Local System, you are required: - -- Your TestMu AI [Username and Access key](/support/docs/hyperexecute-how-to-get-my-username-and-access-key/) -- [HyperExecute YAML](/support/docs/hyperexecute-yaml-version0.2/) file which contains all the necessary instructions. -- [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) in order to initiate a test execution Job . -- Setup the [Environmental Variable](/support/docs/hyperexecute-environment-variable-setup/) - -### Step 1: Configure Your Test Suite - -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** - -Download or Clone the code sample for the JUnit from the TestMu AI GitHub repository to run the tests on the HyperExecute. - - View on GitHub - -If you are using your own project, make sure you update the **Hub endpoint** in your tests file. - -By setting up the Hub endpoint, you establish the communication channel between your tests and the browser nodes, enabling effective test distribution and execution. - -> You can also use **dev** and **beta** browser versions. To generate capabilities for your test requirements, you can use our inbuilt 🔗 Capabilities Generator. - -### Step 2: Setup the CLI in your Test Suite - -After cloning / downloading the sample repo, you need to setup the CLI and the environment variables. - -#### Download the HyperExecute CLI - -The CLI is used for triggering the tests on HyperExecute. It is recommend to download the CLI binary on the host system and keep it in the root directory of the suite to perform the tests on HyperExecute. - -You can download the CLI for your desired platform from the below mentioned links: - -| Platform | HyperExecute CLI | -| ---------| ---------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| MacOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -#### Setup Environment Variable - -Now, you need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in the [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). - -Run the below mentioned commands in your terminal to setup the CLI and the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -### Step 3: Configure YAML in your Test Suite - -Configure your YAML file as per your use cases using **key value** pairs. - -In this sample YAML file, we have mentioned: - -- **version** of the YAML file -- **Timeouts** for executing your project -- **Mode of execution** is [Autosplit](/support/docs/hyperexecute-auto-split-strategy/). You can also opt for [Matrix](/support/docs/hyperexecute-matrix-multiplexing-strategy/) or [Hybrid](/support/docs/hyperexecute-hybrid-strategy/) mode. -- **Pre and Post** commands -- **Reports and Artefacts** that will be generated after the completion of tests -- and other necessary YAML Parameters - -```yaml ---- -version: "0.1" - -globalTimeout: 90 -testSuiteTimeout: 90 -testSuiteStep: 90 - -runson: linux - -autosplit: true - -retryOnFailure: true -maxRetries: 1 - -concurrency: 1 - -env: -TARGET_OS: "LINUX" - -cacheKey: '{{ checksum "requirement.txt" }}' -cacheDirectories: -- CacheDir - -pre: -- pip3 install -r requirement.txt --cache-dir CacheDir -- playwright install -- npm install playwright --save-exact - -post: -- cat yaml/linux/.hyperexecute_autosplits.yaml - -testDiscovery: -type: raw -mode: remote -command: grep -lr 'def' *.py - -testRunnerCommand: python $test - -jobLabel: [playwright-Python, linux, autosplit] -``` - -### Step 4: Execute your Test Suite - -> **NOTE :** In case of MacOS, if you get a permission denied warning while executing CLI, simply run **`chmod u+x ./hyperexecute`** to allow permission. In case you get a security popup, allow it from your **System Preferences** → **Security & Privacy** → **General tab**. - -Run the below command in your terminal at the root folder of the project: - -```bash -./hyperexecute --config RELATIVE_PATH_OF_YOUR_YAML_FILE -``` - -OR use this command if you have not exported your username and access key in the step 2. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE `} - -### Step 5: Monitor the Test Execution - -Visit the [HyperExecute Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) and check your Job status. - -### Step 6: Download Artifacts and Reports - -HyperExecute also facilitates the provision to download the [Artifacts](/support/docs/hyperexecute-artifacts/) and [Reports](/support/docs/hyperexecute-reports/) on your local machine. Click on the corresponding button to download your generated artifacts and reports. - -## 2. Testing Using Gitpod - -You can also use the Gitpod platform to execute our sample repository. It will fetch all the sample codebases and trigger the CLI to execute the tests. - -Follow the below steps to run Test using Gitpod: - -**Step 1:** Click '**Open in Gitpod**' button. You will be redirected to Login/Signup page. This button is configured to redirect you to the Gitpod platform where you will be able to execute our sample repository. - -**Step 2:** Login with TestMu AI credentials. Once logged in, a pop-up confirmation will appear, asking you to **'Proceed'** to the Gitpod editor in a new tab. The current tab will display the HyperExecute Dashboard. - -**Step 3:** Choose your preferred editor (we recommend VS Code Editor) - -**Step 4:** As you are running a sample project, Fetching of the Test Scripts, [HyperExecute YAML](/support/docs/deep-dive-into-hyperexecute-yaml/), [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) and Triggering your tests using the `Execution Command` will be automated. - -**Step 5:** Once you see the `Job Link` in the logs, you can visit the [HyperExecute dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) to see the tests getting executed. - -You can also implement [Secret Keys](/support/docs/hyperexecute-how-to-save-and-manage-secrets/) in your YAML file. - -## Navigation in Automation Dashboard - -Every test run on the HyperExecute has a unique *jobId* associated with it. Each *jobId* can in turn constitute single (or multiple) *groupId*(s). You can visit [HyperExecute Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) for checking the status of the test execution. - -You can seamlessly navigate between JobId's and taskId's. You need to click on the *testID* to navigate from the HyperExecute logs to the Automation Dashboard. - -The snapshot below shows the videos, logs and other meta data for that specific *test_ID* - -> For any query or doubt, please feel free to contact us via **24×7 chat support** or you can also drop a mail to **support@testmuai.com**. -Happy testing! - -## Using the Playwright Agent Skill with TestMu AI - -The [playwright-skill](https://github.com/LambdaTest/agent-skills/tree/main/playwright-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The playwright-skill package includes: - -``` -playwright-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing Playwright Agent Skill - -Install a Playwright Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/playwright-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/playwright-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only playwright-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). diff --git a/static/docs/playwright-real-device-on-hyperexecute.md b/static/docs/playwright-real-device-on-hyperexecute.md deleted file mode 100644 index 8d3ef6dd7..000000000 --- a/static/docs/playwright-real-device-on-hyperexecute.md +++ /dev/null @@ -1,132 +0,0 @@ -# Running Playwright Real Device Tests on HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This guide explains how to run your **Playwright** tests on **Real devices (Android and iOS)** using HyperExecute: - -## Prerequisites -Before running your tests, ensure you have: - -- Your TestMu AI [Username and Access key](/support/docs/hyperexecute-how-to-get-my-username-and-access-key/) -- [HyperExecute YAML](/support/docs/hyperexecute-yaml-version0.2/) file with the test configuration. -- [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) in order to initiate a test execution Job . - -## Step 1: Set Up Your Test Project -You can use your own Playwright project or try the sample repo:. For demo purposes, we are using the sample repository. - -**Sample repo** -Download or Clone the code sample for the Playwright Real Device from the TestMu AI GitHub repository to run the tests on the HyperExecute. - - View on GitHub - -### Configure Desired Capabilities -Update your test configuration with the required capabilities. - -```javascript title="playwright-android-real-device.js" reference -https://github.com/LambdaTest/LambdaTest-HyperExecute-Playwright-Appium-Sample/blob/main/playwright-android-real-device.js -``` - -```javascript title="playwright-ios-real-device.js" reference -https://github.com/LambdaTest/LambdaTest-HyperExecute-Playwright-Appium-Sample/blob/main/playwright-ios-real-device.js -``` - -> You can also use **dev** and **beta** browser versions. To generate capabilities for your test requirements, you can use our inbuilt 🔗 Capabilities Generator. - -### Configure the CDP URL -You will have to update the **CDP (Chrome DevTools Protocol) URL** which is a protocol used for communication between the browser and the developer tools. - -```javascript title="playwright-ios-real-device.js" -const browser = await chromium.connect({ -wsEndpoint: `wss://cdp.lambdatest.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` -}) -``` - -## Step 2: Download the CLI in your Project -The CLI is used for triggering the tests on HyperExecute. It is recommend to download the CLI binary on the host system and keep it in the root directory of the project to execute the tests on HyperExecute. - -You can download the CLI for your desired platform from the below mentioned links: - -| Host Machine | HyperExecute CLI | -| ---------| ---------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| MacOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -## Step 3: Configure YAML in your Test Suite -Use the sample YAML below to configure your execution: - -```yaml title="hyperexecute_android.yaml" reference -https://github.com/LambdaTest/LambdaTest-HyperExecute-Playwright-Appium-Sample/blob/main/hyperexecute_android.yaml -``` - -```yaml title="hyperexecute_ios.yaml" reference -https://github.com/LambdaTest/LambdaTest-HyperExecute-Playwright-Appium-Sample/blob/main/hyperexecute_ios.yaml -``` - -To Run test on Mobile Containers in a Particular Region - -```yaml title="hyperexecute.yaml" -dynamicAllocation: true - -framework: -name: appium -args: -playwrightRD : true -region: ap # supported regions -> ap, eu, us -reservation: false -``` -- If **`reservation : false`**, it means that it will allocate the device from any region. If you want to allocate the device of any specific region, keep the **`reservation : true`**. - -## Step 4: Execute your Test Suite - -> **NOTE :** In case of MacOS, if you get a permission denied warning while executing CLI, simply run **`chmod u+x ./hyperexecute`** to allow permission. In case you get a security popup, allow it from your **System Preferences** → **Security & Privacy** → **General tab**. - -Run the below command in your terminal at the root folder of the project: - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE `} - -## Step 5: Monitor the Test Execution - -Visit the [HyperExecute Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) and check your Job status. - -## Step 6: Download Artifacts and Reports - -HyperExecute also facilitates the provision to download the [Artifacts](/support/docs/hyperexecute-artifacts/) and [Reports](/support/docs/hyperexecute-reports/) on your local machine. Click on the corresponding button to download your generated artifacts and reports. - -## Using the Playwright Agent Skill with TestMu AI - -The [playwright-skill](https://github.com/LambdaTest/agent-skills/tree/main/playwright-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The playwright-skill package includes: - -``` -playwright-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing Playwright Agent Skill - -Install a Playwright Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/playwright-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/playwright-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only playwright-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). diff --git a/static/docs/playwright-sdk.md b/static/docs/playwright-sdk.md deleted file mode 100644 index 8c9a62feb..000000000 --- a/static/docs/playwright-sdk.md +++ /dev/null @@ -1,208 +0,0 @@ -# Playwright SDK - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This document guides you through setting up and using the TestMu AI Playwright-Node.js SDK to run your Playwright tests on the TestMu AI platform. - -## Overview -The TestMu AI Playwright Node SDK is a command-line interface (CLI) tool designed to streamline the process of running your existing Playwright tests on the TestMu AI cloud grid. It eliminates the need for code modifications and provides a simple, configuration-driven approach to test execution. - -Key benefits include: - -* **Effortless Execution**: Run your entire test suite on the TestMu AI platform with a single, simple command. The SDK handles all the underlying complexity of connecting to the grid. -* **Zero Code Changes**: Integrate your existing Playwright tests without modifying your test scripts. All configuration is managed externally in the lambdatest.yml file. -* **Centralized Capabilities**: Define and manage all your test environments, including browser, platform, and resolution combinations, in a single, easy-to-read YAML file. -* **Local & Private Testing**: Seamlessly test internal, pre-production, or locally hosted websites using the TestMu AI Tunnel, which can be managed automatically by the SDK. -* **CI/CD Integration**: The command-line nature of the SDK makes it trivial to integrate into any CI/CD pipeline (like Jenkins, GitHub Actions, CircleCI, etc.) for continuous testing. -* **Rich Test Artifacts**: All test runs are automatically reported to your TestMu AI Dashboard, complete with video recordings, step-by-step screenshots, browser logs, and network logs for easy debugging. - -## Prerequisites - -Before proceeding, ensure the following requirements are met: - -* A TestMu AI account. A free account can be created from the TestMu AI website. -* Your TestMu AI username and access key, which are available in your [TestMu AI profile](https://www.testmuai.com/login/). -* A local installation of Node.js and npm. -* An existing Playwright project. - -## Installation - -To get started, install the SDK as a development dependency within your project's root directory. - -```bash -npm install @lambdatest/playwright-node-sdk -``` - -## Configuration - -The SDK is configured via a `lambdatest.yml` file located in your project root. This YAML file orchestrates your test execution, defining credentials, capabilities, and other run settings. - -### 1. Initializing the Configuration File - -You can generate a template `lambdatest.yml` file by running the following command. You may also provide an optional filename. - -```bash -npx playwright-node-sdk --init [(optional: filename)] -``` - -Below is the structure and explanation of the configuration options available in the `lambdatest.yml` file. - -```yaml -# Access your LambdaTest credentials from https://accounts.lambdatest.com/security/username-accesskey -# It's recommended to use environment variables for your credentials. -user: {YOUR_LAMBDATEST_USERNAME} -accessKey: {YOUR_LAMBDATEST_ACCESS_KEY} - -# Build name for your test run -build: "Playwright Node.js SDK Build" - -# Name of the test suite -name: "Playwright Test Suite" - -# Global base configuration -framework: "playwright" # Framework name -source: "node-js-playwright-sample-sdk:v1" # GitHub branch and tag - -# Optional: Build and project tags for organization and filtering -buildTags: ["playwright", "nodejs", "typescript"] -tags: ["smoke", "regression"] -projectName: "Playwright Node SDK Testing" - -# Define the number of parallel sessions for each configuration -# Check your parallel session limits at https://billing.lambdatest.com/billing/subscriptions -parallelsPerPlatform: 2 - -# Optional: Advanced settings applied to all tests -network: false # To capture network HAR logs (may impact test execution speed) -console: false # To capture browser console logs (may impact test execution speed) -video: true # To capture video for debugging (may impact test execution speed) -tunnel: false # To test pages behind a VPN/private network or from a local system. Refer to https://www.lambdatest.com/support/docs/testing-locally-hosted-pages/ -dedicatedProxy: false -idleTimeout: 300 -timezone: "UTC" -visual: true # To capture a screenshot for each command (may impact test execution speed) -headless: false # To run tests in headless mode. Note: Video will not be available for headless tests. -commandLog: true # To capture commands that are executed on the LambdaTest platform -resolution: "1280x960" # To set the global resolution/viewport size to execute tests - -# Tunnel-specific tests -tunnel_settings: -# tunnel: true -# autostart: true -# tunnelName: "app-behind-firewall-localhost-test-tunnel" -# localdirectory: "./log" -# verbose: true -# loglevel: "debug" - -# Global Playwright configuration (inherited by all platforms unless overridden) -playwrightConfigOptions: -testDir: "tests" -# use: -# viewport: { width: 1280, height: 720 } - -# Refer to the capability generator for detailed configurations: https://www.lambdatest.com/capabilities-generator/ -platforms: -- platform: "macOS Sequoia" # Or "Windows 11", "android", "ios" -browserName: "chrome" # Or pw-firefox, pw-webkit -# You can specify an exact browser version or use "latest", "latest-1", etc. -browserVersion: "latest" -# For mobile testing -# deviceName: "Pixel 7" # Or "iPhone 14" -# platformVersion: "13" # Or "16" -# isRealMobile: true -# ltOptions: -# Geolocation-specific tests -# geoLocation: "GB" -# Network communication tests (enable network logging) -# network: true -# console: true -# fullHar: true # To capture request and response body (might slow the test execution) -# playwrightConfigOptions: -# testDir: "additional-tests" -# testMatch: ["**/mobile-responsive.spec.ts"] -# # For mobile testing -# use: -# viewport: { width: 390, height: 844 } -``` - -### 2. Credentials Configuration - -In the `lambdatest.yml` file, replace the placeholder values for `user` and `accessKey` with your TestMu AI credentials. - -### 3. Platform Configuration - -The `platforms` array defines the target environments (OS/browser combinations) for your test execution. You can specify multiple platform objects. For a complete list of valid platform configurations, consult the [TestMu AI Capabilities Generator](https://www.lambdatest.com/capabilities-generator/). - -### 4. Run Settings - -The `run_settings` section is not explicitly present in the provided YAML, but you can control test execution behavior by passing standard Playwright CLI arguments directly to the `playwright-node-sdk` command. - -### 5. Playwright Project Options - -The `playwrightConfigOptions` section provides a powerful mechanism to override your project's native Playwright configuration for the TestMu AI run. This allows for dynamic adjustments without altering your source `playwright.config.js`. - -* **testDir**: Specifies the location of your test files. - -```yaml -playwrightConfigOptions: -testDir: './tests' -``` - -* **testMatch** and **testIgnore**: Defines glob patterns to include or exclude specific test files from the run. - -```yaml -playwrightConfigOptions: -testMatch: /.*\.e2e\.js/ -testIgnore: /.*\.spec\.js/ -``` - -* **setup** and **teardown**: Specifies scripts to be executed before and after your test suite. - -```yaml -playwrightConfigOptions: -setup: -- name: 'setup1' -testMatch: '**/*.setup.ts' -teardown: 'teardown1' -teardown: -- name: 'teardown1' -testMatch: '/teardown\.ts/' -``` - -* **use**: Sets global options that are applied to all tests within the run. - -```yaml -playwrightConfigOptions: -use: -baseURL: 'https://www.example.com' -``` - -### 6. Tunnel Configuration - -The `tunnel_settings` section is used for testing applications that are not publicly accessible (e.g., on localhost or behind a firewall). - -* **tunnel: true**: Enables the TestMu AI Tunnel feature. -* **autostart: true**: Instructs the SDK to automatically manage the tunnel's lifecycle (start and stop). - -## Running Your Tests - -To execute your tests on the TestMu AI platform, you will use the `playwright-node-sdk` CLI wrapper followed by the standard `playwright test` command. - -```bash -npx playwright-node-sdk playwright test --config Optional path-of-config.yml -``` - -### Execution Workflow - -When this command is run, the SDK performs the following steps: - -1. Reads and parses the `lambdatest.yml` configuration file. -2. If `tunnel: true` and `autostart: true` are set, it establishes a secure connection via the TestMu AI Tunnel. -3. Generates a temporary, in-memory Playwright configuration that includes the necessary hooks to connect to the TestMu AI grid. -4. Executes your Playwright tests using this temporary configuration. -5. Upon test completion, it terminates the TestMu AI Tunnel (if started) and cleans up any temporary artifacts. - -You can pass any standard Playwright CLI options directly to this command. For instance, to run a specific test file: - -```bash -npx playwright-node-sdk playwright test tests/my-test.spec.js diff --git a/static/docs/playwright-test-execution-setup.md b/static/docs/playwright-test-execution-setup.md deleted file mode 100644 index d38cbfe5a..000000000 --- a/static/docs/playwright-test-execution-setup.md +++ /dev/null @@ -1,293 +0,0 @@ -# Playwright - Test Execution Setup - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Whenever you run a Playwright test, you must specify the operating system and the browser you wish to use. The TestMu AI Desired Capabilities Generator allows you to automatically create the capabilities class needed to run your Playwright automation scripts on TestMu AI. - -In this document, learn how to configure the desired capability for selecting browsers and OS, organzing tests, changing desktop resolution, and more for your Playwright tests. - -## Choosing Browser And OS - -To perform Playwright testing on TestMu AI, you need to define the `browserName`, `browserVersion`, and `platform` capabilities in your automation scripts. - -| Key | Expected Values | Description | Capability| -| -------- | -----| ------------ | --------------| -| browserName | Chrome, Microsoft Edge **Playwright Bundled Browsers**: pw-chromium, pw-webkit, and pw-firefox | Specify the browser to test on | `const capability = {"browserName": "pw-webkit"}`| -| browserVersion | Chrome 83 & above, Edge 83 & above | Specify the browser version to test on | `const capability = {"browserVersion": "latest"}`| -| platform | **Windows**: 11, 10, 8, 8.1, 7 **macOS**: Monterey, Big Sur, Catiline, Mojave | Specify the platform name | `const capability = { "LT:Options": {"platform": "Windows 10",}}` | - -### Playwright Supported Browsers - -Following is a list of compatible Playwright bundled browsers that you can use to run tests on the TestMu AI platform. - -| Playwright Versions | Compatible Browsers | -| --------------------|---------------------| -| v1.44 | **Chromium** 125.0 **Firefox** 125.0.1 **WebKit** 17.4 | -| v1.43 | **Chromium** 124.0 **Firefox** 124.0 **WebKit** 17.4 | -| v1.42 | **Chromium** 123.0 **Firefox** 123.0 **WebKit** 17.4 | -| v1.41 | **Chromium** 121.0 **Firefox** 121.0 **WebKit** 17.4 | -| v1.40 | **Chromium** 120.0 **Firefox** 119.0 **WebKit** 17.4 | -| v1.39 | **Chromium** 119.0 **Firefox** 118.0 **WebKit** 17.4 | -| v1.38 | **Chromium** 117.0 **Firefox** 117.0 **WebKit** 17.0 | -| v1.37 | **Chromium** 116.0 **Firefox** 115.0 **WebKit** 17.0 | -| v1.36 | **Chromium** 115.0 **Firefox** 115.0 **WebKit** 17.0 | -| v1.35 | **Chromium** 115.0 **Firefox** 113.0 **WebKit** 16.4 | -| v1.34 | **Chromium** 115.0 **Firefox** 113.0 **WebKit** 16.4 | -| v1.33 | **Chromium** 113.0 **Firefox** 112.0 **WebKit** 16.4 | -| v1.32 | **Chromium** 112.0 **Firefox** 111.0 **WebKit** 16.4 | -| v1.31 | **Chromium** 111.0 **Firefox** 109.0 **WebKit** 16.4 | -| v1.30 | **Chromium** 110.0 **Firefox** 108.0 **WebKit** 16.4 | -| v1.29 | **Chromium** 109.0 **Firefox** 107.0 **WebKit** 16.0 | -| v1.28 | **Chromium** 108.0 **Firefox** 106.0 **WebKit** 16.4 | -| v1.27 | **Chromium** 107.0 **Firefox** 105.0 **WebKit** 16.0 | -| v1.26 | **Chromium** 106.0 **Firefox** 104.0 **WebKit** 16.0 | -| v1.25 | **Chromium** 105.0 **Firefox** 103.0 **WebKit** 16.0 | -| v1.24 | **Chromium** 103.0 **Firefox** 102.0 **WebKit** 16.0 | -| v1.23 | **Chromium** 103.0 **Firefox** 100.0 **WebKit** 15.4 | -| v1.22 | **Chromium** 102.0 **Firefox** 99.0 **WebKit** 15.4 | -| v1.21 | **Chromium** 101.0 **Firefox** 98.0 **WebKit** 15.4 | -| v1.20 | **Chromium** 101.0 **Firefox** 97.0 **WebKit** 15.4 | -| v1.19 | **Chromium** 100.0 **Firefox** 96.0 **WebKit** 15.4 | -| v1.18 | **Chromium** 99.0 **Firefox** 95.0 **WebKit** 15.4 | -| v1.17 | **Chromium** 98.0 **Firefox** 94.0 **WebKit** 15.4 | -| v1.16 | **Chromium** 97.0 **Firefox** 93.0 **WebKit** 15.4 | -| v1.15 | **Chromium** 96.0 **Firefox** 92.0 **WebKit** 15.0 | - -### Bundled Browser Versions -When you set the `useSpecificBundleVersion: true` capability, TestMu AI selects the browser (Chromium, Firefox, WebKit) version that matches your local machine's Playwright version: - -```yaml -const capabilities = { "LT:Options": {"useSpecificBundleVersion": true,}} -``` - -| Playwright Versions | Chromium | Firefox | Webkit | -|---------------------|----------|---------|--------| -|1.50| 130-133 except - 132, 126, 122 | 130-134, except - 131,133,126,122,120 | 18.0, 18.2 | -|1.49| 130-133 except - 132, 126, 122 | 130-134, except - 131,133,126,122,120 | 18.0, 18.2 | -|1.48| 130-133 except - 132, 126, 122 | 130-134, except - 131,133,126,122,120 | 18.0, 18.2 | -|1.47| 129 except - 132, 126, 122 | 130, except - 131,133,126,122,120 | 18.0 | -|1.46| 119-133, except - 132, 126, 122 | 118-134, except - 131,133, 126,122,120 | 17.4, 18.2 | -|1.45| 119-127, except - 126,122 | 119-127, except - 126,122,120 | 17.4 | -|1.44| 119-127, except - 126,122 | 119-127, except - 126,122,120 | 17.4 | -|1.43| 119-127, except - 126,122 | 119-127, except - 126,122,120 | 17.4 | -|1.42| 119-127, except - 126,122 | 119-127, except - 126,122,120 | 17.4 | -|1.41| 119-127, except - 126,122 | 119-127, except - 126,122,120 | 17.4 | -|1.40| 119-127, except - 126,122 | 119-127, except - 126,122,120 | 17.4 | -|1.39| 119-127, except - 126,122 | 119-127, except - 126,122,120 | 17.4 | -|1.38| 114-117 | 113-117 | 17 | -|1.37| 114-117 | 113-117 | 17 | -|1.36| 114-117 | 113-117 | 17 | -|1.35| 114-117 | 113-117 | 16.4 | -|1.34| 114-117 | 113-117 | 16.4 | -|1.33| 104-113 | 103-112 | 16.4 | -|1.32| 104-113 | 103-112 | 16.4 | -|1.31| 104-113 | 103-112 | 16.4 | -|1.30| 104-113 | 103-112 | 16.4 | -|1.29| 104-113 | 103-112 | 16.4 | -|1.28| 104-113 | 103-112 | 16.4 | -|1.27| 104-113 | 103-112 | 16.4 | -|1.26| 104-113 | 103-112 | 16 | -|1.25| 104-113 | 103-112 | 16 | -|1.24| 103-104 | 100-102 | 16 | - -## Organizing Tests - -You can name your test cases and categorize your Playwright builds by build number, build name, test and build tags for easier analysis. Use the following capabilities to organize your Playwright tests. - -| Key | Values | Description | Capability | -| -------- | -----| ------------ | -------- | -| build | Playwright Sample Build | Represent the build number for your test |```const capability = {"LT:Options": {"build": "",}}``` | -| name | Playwright Sample Test | Represents the name of a test | `const capability = {"LT:Options": {"name": "",}}` | -| projectName | Playwright Sample Project | Represents the name of a project | `const capability = {"LT:Options": {"projectName": "",}}` | -| tags | ["tag1", "tag2", "tag3"] | Group your Playwright tests |``const capability = {"LT:Options": { "tags": ["tag1", "tag2", "tag3"], }}`` | -| buildTags | ["build1", "build2", "build3"] | Group your Playwright builds |`const capability = {"LT:Options": { "buildTags": ["build1", "build2", "build3"] }}` | - -## Changing Desktop Resolutions - -Playwright tests run with a `1920x1080` resolution by default for desktop browsers. With our `resolution` capability, you can set a different screen resolution for your tests. - -| Capability | Description | Expected Values | Example | -| -------- | -----| ------------ | -----------------------------| -| resolution | Specifying your desktop resolution before initiating the test | String, **Default value**: 1920x1080 **Windows 11 & 10**: `1024x768, 1280x800, 1280x1024, 1366x768, 1440x900, 1680x1050, 1600x1200, 1920x1200, 1920x1080 and 2048x1536` **macOS**: `1024x768, 1280x960, 1280x1024, 1600x1200 and 1920x1080` | `const capabilities = { 'LT:Options': {'resolution': '2048x1536',}}` | - -## Changing Browser Window Size - -The default viewport size for Playwright tests is 1280x720. If you wish to maximize the browser window during your Playwright test, you can do it as shown in the code below. - -```js -// Create context with given viewport -const context = await browser.newContext({ -viewport: { width: 1280, height: 1024 } -}); -``` -The above Playwright test runs on a default viewport of 1280x1024. However, you can also select any other viewport when creating a `new context`. - -If you resize viewport for individual page then run the below command. - -```js -await page.setViewportSize({ width: 1600, height: 1200 }); -``` - -## Setting Browser Options - -Playwright lets you set browser arguments for Chromium based browsers. When selecting Chrome and Edge, you can set browser options using `'goog:chromeOptions'` and `'ms:edgeOptions'` capability as shown below. - -**For Chrome** - -```js -const { chromium } = require('playwright'); - -const capabilities = { -'browserName': 'Chrome', // Browsers allowed: `Chrome`, `MicrosoftEdge`, `pw-chromium`, `pw-firefox` and `pw-webkit` -'browserVersion': 'latest', -'LT:Options': { -'platform': 'Windows 10', -'build': 'Playwright Sample Build', -'name': 'Playwright Sample Test', -'user': process.env.LT_USERNAME, -'accessKey': process.env.LT_ACCESS_KEY, -'goog:chromeOptions':[ -'--user-agent=', -], - -} -} - -const browser = await chromium.connect({ -wsEndpoint: `wss://cdp.lambdatest.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` -}) -``` - - **For Edge** - -```js -const { chromium } = require('playwright'); - -const capabilities = { -'browserName': 'Chrome', // Browsers allowed: `Chrome`, `MicrosoftEdge`, `pw-chromium`, `pw-firefox` and `pw-webkit` -'browserVersion': 'latest', -'LT:Options': { -'platform': 'Windows 10', -'build': 'Playwright Sample Build', -'name': 'Playwright Sample Test', -'user': process.env.LT_USERNAME, -'accessKey': process.env.LT_ACCESS_KEY, -'ms:edgeOptions':[ -'--user-agent=', -], - -} -} - -const browser = await chromium.connect({ -wsEndpoint: `wss://cdp.lambdatest.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` -}) -``` - ->**Note**: You can define any Chromium based argument from the [list of Chromium command line arguments](https://peter.sh/experiments/chromium-command-line-switches/). - -## Getting Session Details - -When you run a Playwright test on TestMu AI, it generates different logs for each test session. To get relevant information for the specific session, use the snippet provided below in your Playwright test scripts. - -```js -let response = JSON.parse(await page.evaluate(_ => {}, `lambdatest_action: ${JSON.stringify({ action: 'getTestDetails' })}`)) -console.log(response); -``` - -Upon executing the script, you will get the details for the particular test session as a JSON response as shown below. - -```js -{ -data: { -test_id: 'RWA9R-AC4V8-GJRET-YHBO2', -build_id: 9415381, -name: 'Playwright Sample Test', -user_id: 1001150, -username: 'kaustubhd', -test_type: 'playwright', -platform: 'win10', -browser: 'Chrome', -browser_version: '107.0', -status_ind: 'running', -session_id: 'RWA9R-AC4V8-GJRET-YHBO2', -build_name: 'Playwright Sample Build', -create_timestamp: '2022-12-07 10:53:16', -start_timestamp: '2022-12-07 10:53:16', -remark: 'running', -console_logs_url: 'https://api.lambdatest.com/automation/api/v1/sessions/RWA9R-AC4V8-GJRET-YHBO2/log/console', -network_logs_url: 'https://api.lambdatest.com/automation/api/v1/sessions/RWA9R-AC4V8-GJRET-YHBO2/log/network', -command_logs_url: 'https://api.lambdatest.com/automation/api/v1/sessions/RWA9R-AC4V8-GJRET-YHBO2/log/command', -video_url: 'https://automation.lambdatest.com/public/video?testID=RWA9R-AC4V8-GJRET-YHBO2&auth=50471585dd55293d204fe87f63400f5f', -screenshot_url: 'https://video-bundler.lambdatest.com/RWA9R-AC4V8-GJRET-YHBO2/screenshots.zip?orgId=588659&createTimestamp=1670410396&testCreateTimestamp=2022-12-07%2010:53:16' -}, -message: 'Retrieve session was successful', -status: 'success' -} -``` - -## Configuring Geolocation - -You can run your Playwright tests across 53+ geographies using the `'geoLocation'` capability as shown below. - -| Key | Values | Description | Capability | -| -------- | -----| ------------ | -------- | -| geoLocation | US | Represents the geolocation - United States |`const capabilities = { "LT:Options": { 'geoLocation': 'US', }}` | - -## Debugging Tests - -By specifying the capabilities for the debugging tools, you can debug and fix your failed Playwright test sessions using network logs, console logs, and video logs. - -| Key | Values | Description | Desired Capability | -| -------- | -----| ------------ | --------------| -| network | true/false | Enable network logs | `const capabilities = { 'LT:Options': {'network': true,}}` | -| console | true/false | Enable browser console logs | `const capabilities = { 'LT:Options': {'console': true,}}` | -| video | true/false | Enable Video recording of the entire screen | `const capabilities = { 'LT:Options': {'video': true,}}` | - -## Mark Tests As Passed Or Failed - -While running Playwright tests on the TestMu AI platform, you may come across a scenario in which a test that failed in your local instance turns up to be successful on TestMu AI. For verifying expected behavior, it is critical to identify automated tests as **Passed** or **Failed** based on your testing requirements. - -By default, the Status of each test that runs successfully is marked as **Completed**, and if there are any issues, the Status is marked as **Failed**. - -Shown below is syntax how to mark Playwright tests as **Passed** or **Failed**. - -1. To mark test status as **passed**. - -``` -await page.evaluate(_ => {}, `lambdatest_action: ${JSON.stringify({ action: 'setTestStatus', arguments: { status:'passed', remark: 'Title matched' } })}`) -``` - -2. To mark test status as **failed**. - -``` -await page.evaluate(_ => {}, `lambdatest_action: ${JSON.stringify({ action: 'setTestStatus', arguments: { status:'failed', remark: 'Title not matched' } })}`) -``` - -## Capabilities Reference - -The TestMu AI Capability Generator can auto-create the capabilities class for your Playwright scripts. The full set of capabilities you can configure: - -| Key | Expected Values | Description |Capability| -|----|----------|----------|------------| -| browserName| Chrome, Microsoft Edge **Playwright Bundled Browsers**: pw-chromium, pw-webkit, and pw-firefox| Specify the browser to test on | `const capability = {"browserName": "pw-webkit"}` | -| browserVersion | Chrome 83 & above, Edge 83 & above | Specify the browser version to test on | `const capability = {"browserVersion": "latest"}` | -| platform | **Windows**: 11, 10, 8, 8.1, 7 **macOS**: Monterey, Big Sur, Catiline, Mojave| Specify the platform name | `const capability = { "LT:Options": {"platform": "Windows 10",}}`| -| build | Playwright Sample Build | Represent the build number for your test | ```const capability = {"LT:Options": {"build": "",}}``` | -| name| Playwright Sample Test| Represents the name of a test| `const capability = {"LT:Options": {"name": "",}}`| -| projectName | Playwright Sample Project| Represents the name of a project | `const capability = {"LT:Options": {"projectName": "",}}`| -| tags| ["tag1", "tag2", "tag3"]| Group your Playwright tests | ``const capability = {"LT:Options": { "tags": ["tag1", "tag2", "tag3"], }}``| -| buildTags| ["build1", "build2","build3"]| Group your Playwright builds| `const capability = {"LT:Options": { "buildTags": ["build1", "build2", "build3"] }}` | -| resolution| String, **Default value**: 1920x1080 **Windows 11 & 10**: `1024x768, 1280x800, 1280x1024, 1366x768, 1440x900, 1680x1050, 1600x1200, 1920x1200, 1920x1080 and 2048x1536` **macOS**: `1024x768, 1280x960, 1280x1024, 1600x1200 and 1920x1080` | Specifying your desktop resolution before initiating the test | `const capabilities = { "LT:Options": {"resolution": "2048x1536",}}`| -| projectName| My Test| Represent the name of your project| `const capability = { "LT:Options": {"projectName": "",}}`| -| network| true/false| Enable network logs| `const capability = { "LT:Options": {"network": true,}}`| -| console| true/false| Enable browser console logs| `const capabilities = { "LT:Options": {"console": true,}}`| -| video| true/false| Enable video recording of the entire screen| `const capability = { "LT:Options": {"video": true,}}`| -| visual| true/false| Captures screenshot for every command| `const capability = { "LT:Options": {"visual": true,}}`| -| tunnel| true/false| Enable tunnel for local testing| `const capability = { "LT:Options": {"tunnel": true,}}`| -| tunnelName | true /false | Specify tunnel name | `const capability = { "LT:Options": {"tunnelName": "",}}`| -| geoLocation | AR (Argentina) | Specify country code | `const capability = { "LT:Options": {"geoLocation": "AR",}}`| -| idleTimeout | number| Specifies the timeout of the commands in seconds. Default value: 300 Max value: 1800 If a value greater than 1800 is added, idleTimeout will be set to 1800.| `const capability = { "LT:Options": {"idleTimeout": "",}}`| -| lambdaMaskCommands | array | `sendType` or `sendFill` Masks the type method of the ElementHandle and Locator class. `sendPress` Masks the press method of the ElementHandle and Locator class. `setHTTPCredentials` Masks the HTTP credentials parameter in the newContext and newPage method of the browser class. `setStorageState` Masks the storage state parameter in the newContext and newPage method of the browser class `setGeolocation` Masks the Geolocation parameter in the newContext and newPage method of the browser class.|`const capability = { "LT:Options": {'lambdaMaskCommands': ["sendType", "sendFill", "sendPress", "setHTTPCredentials", "setStorageState", "setGeolocation"]}}` | -| useSpecificBundleVersion | Boolean | If set **`true`:** TestMu AI will select the playwright server version as per the browser version you have passed in the capabilities. Check the [supported browser version list](/support/docs/playwright-test-execution-setup/#playwright-supported-browsers). If the value set in the `browser version` capability is supported by multiple Playwright versions, TestMu AI checks your Playwright client version and sets the Playwright version accordingly. If set **`false` :** TestMu AI will run your playwright tests with the same version as setup in your local system and the browser version will also be used the compatible one as per the [supported browser version list](/support/docs/playwright-test-execution-setup/#playwright-supported-browsers) and not your defined one. | `const capability = { "LT:Options": {"useSpecificBundleVersion": true,}}` | -| lambdaSetBrowserPosition | | The `lambdaSetBrowserPosition` function is designed to arrange two browser windows on the screen, ensuring each occupies exactly half of the available screen space. This layout facilitates side-by-side browser comparisons or multitasking workflows. OS Supported - Windows Browsers supported : Chrome, Microsoft Edge , pw-chroium, pw-webkit & pw-firefox | `const capability = {action: 'lambdaSetBrowserPosition',}` | diff --git a/static/docs/playwright-test-runner.md b/static/docs/playwright-test-runner.md deleted file mode 100644 index 3d7a98589..000000000 --- a/static/docs/playwright-test-runner.md +++ /dev/null @@ -1,163 +0,0 @@ -# Playwright Testing With Playwright Test Runner - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Playwright Test Runner is used for end-to-end automated testing of websites and web apps across all major browsers. You can run parallel tests, get context isolation out of the box, capture videos, screenshots, and other test artifacts on test failure, and use fixtures with Playwright test runner. - -TestMu AI enables you to run Playwright tests with the Playwright test runner across 40+ real browser and operating system combinations. This guide will outline the fundamentals of getting started with Playwright testing on the TestMu AI platform using the Playwright test runner. - -## Prerequisites - ->Note: All the code samples in this documentation can be found in the TestMu AI's Repository on GitHub. You can either download or clone the repository to quickly run your tests. - View on GitHub - -1. Clone the TestMu AI-Playwright repository on your system. - -2. Install the npm dependencies. - -``` -npm install -``` - -3. In order to run your Playwright tests with Playwright test runner, you will need to set your TestMu AI username and access key in the environment variables. Click the **Access Key** button at the top-right of the Automation Dashboard to access it. - -**Windows** - -```js -set LT_USERNAME="YOUR_LAMBDATEST_USERNAME" -set LT_ACCESS_KEY="YOUR_LAMBDATEST_ACCESS_KEY" -``` - -**macOS/Linux** - -```js -export LT_USERNAME="YOUR_LAMBDATEST_USERNAME" -export LT_ACCESS_KEY="YOUR_LAMBDATEST_ACCESS_KEY" -``` - -## Running Playwright Tests With Playwright Test Runner - -In your `playwright.config.js` file, add the browserName, browserVersion, and platform in the below projects configuration. - -```js -const { devices } = require('@playwright/test') -// Playwright config to run tests on LambdaTest platform and local -const config = { -testDir: 'tests', -testMatch: '**/*.spec.js', -timeout: 60000, -projects: [ -// -- LambdaTest Config -- -// name in the format: browserName:browserVersion:platform@lambdatest -// Browsers allowed: `Chrome`, `MicrosoftEdge`, `pw-chromium`, `pw-firefox` and `pw-webkit` -// Use additional configuration options provided by Playwright if required: https://playwright.dev/docs/api/class-testconfig -{ -name: 'chrome:latest:MacOS Catalina@lambdatest', -use: { -viewport: { width: 1920, height: 1080 } -} -}, -{ -name: 'MicrosoftEdge:90:Windows 10@lambdatest', -use: { -...devices['iPhone 12 Pro Max'] -} -}, -] -} - -module.exports = config -``` -Pass the below command to run the test. - -``` -npm run test -``` - -Visit the TestMu AI Automation dashboard to view the results of your executed test with Playwright test runner. - -## Testing With Playwright Test When Migrating To TestMu AI - -If you are migrating test suites to TestMu AI, then follow the below steps. - -1. Add the `lambdatest-setup.js` to your project route. - -2. Include the `playwright.config.js` in your project in the below format. - -```js -const { devices } = require('@playwright/test') - -// Playwright config to run tests on LambdaTest platform and local -const config = { -testDir: 'tests', -testMatch: '**/*.spec.js', -timeout: 60000, -use: { -viewport: null -}, -projects: [ -// -- LambdaTest Config -- -// name in the format: browserName:browserVersion:platform@lambdatest -// Browsers allowed: `Chrome`, `MicrosoftEdge`, `pw-chromium`, `pw-firefox` and `pw-webkit` -// Use additional configuration options provided by Playwright if required: https://playwright.dev/docs/api/class-testconfig -{ -name: 'chrome:latest:MacOS Catalina@lambdatest', -use: { -viewport: { width: 1920, height: 1080 } -} -}, -{ -name: 'chrome:latest:Windows 10@lambdatest', -use: { -viewport: { width: 1280, height: 720 } -} -}, -{ -name: 'MicrosoftEdge:90:Windows 10@lambdatest', -use: { -...devices['iPhone 12 Pro Max'] -} -}, -{ -name: 'pw-firefox:latest:Windows 10@lambdatest', -use: { -viewport: { width: 1280, height: 720 } -} -}, -{ -name: 'pw-webkit:latest:Windows 10@lambdatest', -use: { -viewport: { width: 1920, height: 1080 } -} -} - -] -} - -``` - -3. Add your test script path in `playwright.config.js`. - -4. Import the test object from `lambdatest-setup.js` and run your tests. - -```js -const { test } = require('../lambdatest-setup') -const { expect } = require('@playwright/test') - -test.describe('Browse LambdaTest in different search engines', () => { -test('Search LambdaTest on Bing', async ({ page }) => { -await page.goto('https://www.bing.com') -const element = await page.$('[aria-label="Enter your search term"]') -await element.click() -await element.type('LambdaTest') -await element.press('Enter') -const title = await page.title() - -console.log('Page title:: ', title) -// Use the expect API for assertions provided by playwright -expect(title).toEqual(expect.stringContaining('LambdaTest')) -}) -}) -``` - -To generate Playwright tests with AI coding assistants, see [Run Tests With Agent Skills](/support/docs/playwright-agent-skills/). diff --git a/static/docs/playwright-testing.md b/static/docs/playwright-testing.md deleted file mode 100644 index 23c6e0524..000000000 --- a/static/docs/playwright-testing.md +++ /dev/null @@ -1,229 +0,0 @@ -# Getting Started With Playwright Testing - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Playwright is a Node.js library that uses a single API to automate Chromium, Firefox, and WebKit. It is designed to enable powerful, reliable, and efficient [automated browser testing](https://www.lambdatest.com/automated-browser-testing). Playwright can also automate Microsoft Edge since it is built on the open-source Chromium web framework. - -TestMu AI allows you to run Playwright tests across 40+ real browsers and operating system combinations. This guide will cover the basics of getting started with Playwright testing on the TestMu AI platform. - -*You can run tests using Playwright versions **v1.15.0** to the latest.* - -## Prerequisites - ->Note: All the code samples in this documentation can be found in the TestMu AI's Repository on GitHub. You can either download or clone the repository to quickly run your tests. - View on GitHub - -1. Clone the TestMu AI-Playwright repository on your system. - -2. Install the npm dependencies. - -``` -npm install -``` - -3. Add browserWSEndpoint (browser end point URL) in your test script. - -```js -wsEndpoint: `wss://cdp.lambdatest.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` -``` - -4. In order to run your Playwright tests, you will need to set your TestMu AI username and access key in the environment variables. Click the **Access Key** button at the top-right of the Automation Dashboard to access it. - -**Windows** - -```js -set LT_USERNAME="YOUR_LAMBDATEST_USERNAME" -set LT_ACCESS_KEY="YOUR_LAMBDATEST_ACCESS_KEY" -``` - -**macOS/Linux** - -```js -export LT_USERNAME="YOUR_LAMBDATEST_USERNAME" -export LT_ACCESS_KEY="YOUR_LAMBDATEST_ACCESS_KEY" -``` - -## Run Your First Test - -1. Add the below code snippet in your test scripts. - -```js -(async () => { -const capabilities = { -'browserName': 'Chrome', // Browsers allowed: `Chrome`, `MicrosoftEdge`, `pw-chromium`, `pw-firefox` and `pw-webkit` -'browserVersion': 'latest', -'LT:Options': { -'platform': 'Windows 10', -'build': 'Playwright Sample Build', -'name': 'Playwright Sample Test', -'user': process.env.LT_USERNAME, -'accessKey': process.env.LT_ACCESS_KEY, -'network': true, -'video': true, -'console': true -} -} - -const browser = await chromium.connect({ -wsEndpoint: `wss://cdp.lambdatest.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` -}) -``` - -Once you are done with the above-mentioned steps, you can initiate your first Playwright test on TestMu AI. - -The below test script searches the term 'TestMu AI' on Bing. - -```js -const { chromium } = require('playwright') -const { expect } = require('@playwright/test'); - -(async () => { -const capabilities = { -'browserName': 'Chrome', // Browsers allowed: `Chrome`, `MicrosoftEdge`, `pw-chromium`, `pw-firefox` and `pw-webkit` -'browserVersion': 'latest', -'LT:Options': { -'platform': 'Windows 10', -'build': 'Playwright Sample Build', -'name': 'Playwright Sample Test', -'user': process.env.LT_USERNAME, -'accessKey': process.env.LT_ACCESS_KEY, -'network': true, -'video': true, -'console': true -} -} - -const browser = await chromium.connect({ -wsEndpoint: `wss://cdp.lambdatest.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` -}) - -const page = await browser.newPage() - -await page.goto('https://www.bing.com') - -const element = await page.$('[aria-label="Enter your search term"]') -await element.click() -await element.type('LambdaTest') -await element.press('Enter') -const title = await page.title() - -try { -expect(title).toEqual('LambdaTest - Search') -// Mark the test as completed or failed -await page.evaluate(_ => {}, `lambdatest_action: ${JSON.stringify({ action: 'setTestStatus', arguments: { status: 'passed', remark: 'Title matched' } })}`) -} catch { -await page.evaluate(_ => {}, `lambdatest_action: ${JSON.stringify({ action: 'setTestStatus', arguments: { status: 'failed', remark: 'Title not matched' } })}`) -} - -await browser.close() -})() - -``` - -2. Pass the below command to run the test. - -``` -node playwright-single.js -``` - -## Run Tests in Parallel - -Once your first test runs, you can execute the same test across multiple browser and OS configurations at once. Define a `capabilities` array (one entry per configuration) and run each through a shared function. - -The example below runs the Bing search test on three configurations: Chrome on Windows 10, Microsoft Edge on Windows 8, and Chrome on macOS Big Sur. - -```js title="playwright-parallel.js" -const { chromium } = require('playwright') -const { expect } = require('@playwright/test') - -const parallelTests = async (capability) => { -console.log('Initialising test:: ', capability['LT:Options']['name']) - -const browser = await chromium.connect({ -wsEndpoint: `wss://cdp.lambdatest.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(capability))}` -}) - -const page = await browser.newPage() - -await page.goto('https://www.bing.com') - -const element = await page.$('[aria-label="Enter your search term"]') -await element.click() -await element.type('LambdaTest') -await element.press('Enter') -const title = await page.title() - -try { -expect(title).toEqual('LambdaTest - Search') -// Mark the test as completed or failed -await page.evaluate(_ => {}, `lambdatest_action: ${JSON.stringify({ action: 'setTestStatus', arguments: { status: 'passed', remark: 'Title matched' } })}`) -} catch { -await page.evaluate(_ => {}, `lambdatest_action: ${JSON.stringify({ action: 'setTestStatus', arguments: { status: 'failed', remark: 'Title not matched' } })}`) -} - -await browser.close() -} - -// Capabilities array with the respective configuration for the parallel tests -const capabilities = [ -{ -'browserName': 'Chrome', // Browsers allowed: `Chrome`, `MicrosoftEdge`, `pw-chromium`, `pw-firefox` and `pw-webkit` -'browserVersion': 'latest', -'LT:Options': { -'platform': 'Windows 10', -'build': 'Playwright Sample Build', -'name': 'Playwright Sample Test on Windows 10 - Chrome', -'user': process.env.LT_USERNAME, -'accessKey': process.env.LT_ACCESS_KEY, -'network': true, -'video': true, -'console': true -} -}, -{ -'browserName': 'MicrosoftEdge', -'browserVersion': 'latest', -'LT:Options': { -'platform': 'Windows 8', -'build': 'Playwright Sample Build', -'name': 'Playwright Sample Test on Windows 8 - MicrosoftEdge', -'user': process.env.LT_USERNAME, -'accessKey': process.env.LT_ACCESS_KEY, -'network': true, -'video': true, -'console': true -} -}, -{ -'browserName': 'Chrome', -'browserVersion': 'latest', -'LT:Options': { -'platform': 'MacOS Big sur', -'build': 'Playwright Sample Build', -'name': 'Playwright Sample Test on MacOS Big sur - Chrome', -'user': process.env.LT_USERNAME, -'accessKey': process.env.LT_ACCESS_KEY, -'network': true, -'video': true, -'console': true -} -}] - -capabilities.forEach(async (capability) => { -await parallelTests(capability) -}) -``` - -Run the parallel test: - -```bash -node playwright-parallel.js -``` - -## View your Playwright test results - -The TestMu AI Automation Dashboard is where you can see the results of your Playwright tests after running them on the TestMu AI platform. - -The below screenshot of TestMu AI Automation Dashboard shows the Playwright build on the left and the build sessions associated with the selected build on the right. - -On clicking the session name of the respective test, you can view the details of Playwright test session that you just executed. For example, the below screenshot shows a test execution details of Playwright test like Test Name, Test ID, selected configurations, test logs, basic info, input config, and test session video. diff --git a/static/docs/playwright-tests-in-ci-cd.md b/static/docs/playwright-tests-in-ci-cd.md deleted file mode 100644 index a37304348..000000000 --- a/static/docs/playwright-tests-in-ci-cd.md +++ /dev/null @@ -1,25 +0,0 @@ -# Playwright Testing With CI/CD - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI integrates with a variety of CI/CD solutions to help you speed up your go-to-market launch by automating your delivery pipeline. Similarly, you can run Playwright tests in CI/CD across 40+ browsers in the cloud, reducing lead time by multiple folds and shipping quality builds faster. - -Running Playwright tests through CI/CD pipeline involves three stages: - -1. **Build**: It consists of developing an web app (or code) and hosting it on a staging environment for testing. -2. **Test**: It entails running Playwright tests on the TestMu AI platform. -3. **Deploy**: Based on your CI/CD configuration, the deploy stage may or may not present. - -## Running Playwright Tests Via CI/CD - -TestMu AI offers [integration with CI/CD tools](/support/docs/integrations-with-ci-cd-tools/) like Jenkins, Bamboo, AWS CodePipeline, and more. In order to run Playwright tests in CI/CD, you need to perform few steps in the **Test** stage of your CI/CD pipeline. - -1. Ensure the website (or code) is hosted on staging environment. - -2. Import test data in existing databases in order to initiate the tests. - -3. If the website under test in hosted on local server, then you will need to configure the TestMu AI tunnel by setting `tunnel` capability to `true` for running local tests via CI/CD on the TestMu AI. However, if the staging website is publicly accessible then this step is not required. - -4. Ensure your Playwright test scripts uses `chromium.connect` method to connect to the CDP endpoint at TestMu AI. Additional parameters for assigning a specific browser and OS combination to your TestMu AI test are contained in the `capabilities` variable. - -To generate Playwright tests with AI coding assistants, see [Run Tests With Agent Skills](/support/docs/playwright-agent-skills/). diff --git a/static/docs/playwright-vanillajs-on-hyperexecute.md b/static/docs/playwright-vanillajs-on-hyperexecute.md deleted file mode 100644 index 6d43a6357..000000000 --- a/static/docs/playwright-vanillajs-on-hyperexecute.md +++ /dev/null @@ -1,206 +0,0 @@ -# Running Vanilla Framework Tests on HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Playwright is a Node.js library that uses a single API to automate Chromium, Firefox, and WebKit. It is designed to enable powerful, reliable, and efficient automated browser testing. Playwright can also automate Microsoft Edge since it is built on the open-source Chromium web framework. - -HyperExecute is an AI Native Test Orchestration Cloud Platform that empowers you to run **end-to-end** tests **quickly** and **efficiently**. It provides Just-in-Time (JIT) testing infrastructure with fast execution **speeds**, **smart orchestration**, and **detailed logs**. - -This guide details how to execute your **Vanilla** framework tests on **HyperExecute** via two different methods: - -- [**Using Local System**](/support/docs/playwright-vanillajs-on-hyperexecute/#1-testing-using-local-system) - You can use your own local machine to execute tests. -- [**Using Gitpod Platform**](/support/docs/playwright-vanillajs-on-hyperexecute/#2-testing-using-gitpod) - Execute tests using GitPod. (Requires a [Gitpod](https://gitpod.io/login/) account) - -## 1. Testing Using Local System - -Follow the step-by-step guide to execute your test on HyperExecute. - -### Prerequisites - -To run the Tests on HyperExecute from your Local System, you are required: - -- Your TestMu AI [Username and Access key](/support/docs/hyperexecute-how-to-get-my-username-and-access-key/) -- [HyperExecute YAML](/support/docs/hyperexecute-yaml-version0.2/) file which contains all the necessary instructions. -- [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) in order to initiate a test execution Job . -- Setup the [Environmental Variable](/support/docs/hyperexecute-environment-variable-setup/) - -### Step 1: Configure Your Test Suite - -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** - -Download or Clone the code sample for the JUnit from the TestMu AI GitHub repository to run the tests on the HyperExecute. - - View on GitHub - -If you are using your own project, make sure you update the **Hub endpoint** in your tests file. - -By setting up the Hub endpoint, you establish the communication channel between your tests and the browser nodes, enabling effective test distribution and execution. - -> You can also use **dev** and **beta** browser versions. To generate capabilities for your test requirements, you can use our inbuilt 🔗 Capabilities Generator. - -### Step 2: Setup the CLI in your Test Suite - -After cloning / downloading the sample repo, you need to setup the CLI and the environment variables. - -#### Download the HyperExecute CLI - -The CLI is used for triggering the tests on HyperExecute. It is recommend to download the CLI binary on the host system and keep it in the root directory of the suite to perform the tests on HyperExecute. - -You can download the CLI for your desired platform from the below mentioned links: - -| Platform | HyperExecute CLI | -| ---------| ---------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| MacOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -#### Setup Environment Variable - -Now, you need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in the [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). - -Run the below mentioned commands in your terminal to setup the CLI and the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -### Step 3: Configure YAML in your Test Suite - -Configure your YAML file as per your use cases using **key value** pairs. - -In this sample YAML file, we have mentioned: - -- **version** of the YAML file -- **Timeouts** for executing your project -- **Mode of execution** is [Autosplit](/support/docs/hyperexecute-auto-split-strategy/). You can also opt for [Matrix](/support/docs/hyperexecute-matrix-multiplexing-strategy/) or [Hybrid](/support/docs/hyperexecute-hybrid-strategy/) mode. -- **Pre and Post** commands -- **Reports and Artefacts** that will be generated after the completion of tests -- and other necessary YAML Parameters - -```yaml ---- -version: "0.1" - -globalTimeout: 90 -testSuiteTimeout: 90 -testSuiteStep: 90 - -runson: linux - -autosplit: true - -retryOnFailure: false -maxRetries: 1 - -concurrency: 2 - -pre: -- npm install -- npx playwright install - -cacheKey: '{{ checksum "package-lock.json" }}' -cacheDirectories: -- node_modules - -testDiscovery: -type: raw -mode: remote -command: grep -nri 'describe' tests | sed 's/:test.*//' - -testRunnerCommand: npx playwright test $test - -jobLabel: [playwright-VanillaJS, linux, autosplit] -``` - -### Step 4: Execute your Test Suite - -> **NOTE :** In case of MacOS, if you get a permission denied warning while executing CLI, simply run **`chmod u+x ./hyperexecute`** to allow permission. In case you get a security popup, allow it from your **System Preferences** → **Security & Privacy** → **General tab**. - -Run the below command in your terminal at the root folder of the project: - -```bash -./hyperexecute --config RELATIVE_PATH_OF_YOUR_YAML_FILE -``` - -OR use this command if you have not exported your username and access key in the step 2. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE `} - -### Step 5: Monitor the Test Execution - -Visit the [HyperExecute Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) and check your Job status. - -### Step 6: Download Artifacts and Reports - -HyperExecute also facilitates the provision to download the [Artifacts](/support/docs/hyperexecute-artifacts/) and [Reports](/support/docs/hyperexecute-reports/) on your local machine. Click on the corresponding button to download your generated artifacts and reports. - -## 2. Testing Using Gitpod - -You can also use the Gitpod platform to execute our sample repository. It will fetch all the sample codebases and trigger the CLI to execute the tests. - -Follow the below steps to run Test using Gitpod: - -**Step 1:** Click '**Open in Gitpod**' button. You will be redirected to Login/Signup page. This button is configured to redirect you to the Gitpod platform where you will be able to execute our sample repository. - -**Step 2:** Login with TestMu AI credentials. Once logged in, a pop-up confirmation will appear, asking you to **'Proceed'** to the Gitpod editor in a new tab. The current tab will display the HyperExecute Dashboard. - -**Step 3:** Choose your preferred editor (we recommend VS Code Editor) - -**Step 4:** As you are running a sample project, Fetching of the Test Scripts, [HyperExecute YAML](/support/docs/deep-dive-into-hyperexecute-yaml/), [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) and Triggering your tests using the `Execution Command` will be automated. - -**Step 5:** Once you see the `Job Link` in the logs, you can visit the [HyperExecute dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) to see the tests getting executed. - -You can also implement [Secret Keys](/support/docs/hyperexecute-how-to-save-and-manage-secrets/) in your YAML file. - -## Navigation in Automation Dashboard - -Every test run on the HyperExecute has a unique *jobId* associated with it. Each *jobId* can in turn constitute single (or multiple) *groupId*(s). You can visit [HyperExecute Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) for checking the status of the test execution. - -You can seamlessly navigate between JobId's and taskId's. You need to click on the *testID* to navigate from the HyperExecute logs to the Automation Dashboard. - -The snapshot below shows the videos, logs and other meta data for that specific *test_ID* - -> For any query or doubt, please feel free to contact us via **24×7 chat support** or you can also drop a mail to **support@testmuai.com**. -Happy testing! - -## Using the Playwright Agent Skill with TestMu AI - -The [playwright-skill](https://github.com/LambdaTest/agent-skills/tree/main/playwright-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The playwright-skill package includes: - -``` -playwright-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing Playwright Agent Skill - -Install a Playwright Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/playwright-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/playwright-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only playwright-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). diff --git a/static/docs/playwright-visual-regression.md b/static/docs/playwright-visual-regression.md deleted file mode 100644 index db9ebd3bb..000000000 --- a/static/docs/playwright-visual-regression.md +++ /dev/null @@ -1,405 +0,0 @@ -# Getting started with SmartUI using Playwright - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This documentation will act as your step-by-step guide in to perform Playwright test with SmartUI. -- Basic understanding of Playwright is required. -- Go to [SmartUI](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) and login along with your credentials. -- Copy `LT_USERNAME` and `LT_ACCESS_KEY` credentials from `Access Key` button on the top right of the dashboard. - -```bash -export LT_USERNAME="YOUR_USERNAME" -export LT_ACCESS_KEY="YOUR ACCESS KEY" -``` - -```bash -set LT_USERNAME="YOUR_USERNAME" -set LT_ACCESS_KEY="YOUR ACCESS KEY" -``` - -```powershell -$env:LT_USERNAME="YOUR_USERNAME" -$env:LT_ACCESS_KEY="YOUR ACCESS KEY" -``` - -The following steps will guide you in running your first Visual Regression test on TestMu AI platform - - -### Step 1: Create a SmartUI Project - -The first step is to create a project with the application in which we will combine all your **builds** run on the project. -To create a SmartUI Project, follow these steps: - -1. Go to [Projects page](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) -2. Click on the `new project` button -3. Select the platform as Web for executing your `PlayWright` tests. -4. Add name of the project, approvers for the changes found, tags for any filter or easy navigation. -5. Click on the **Submit**. - -### Step 2: Configure your test with Playwright Desired Capabilities - -Once you have created a SmartUI Project, you can generate screenshots by running automation scripts. Follow the below steps to successfully generate screenshots - - -1. Please clone the following sample Github repo - -```bash -git clone https://github.com/LambdaTest/playwright-sample -``` - -2. Install the node modules using the command - -```bash -npm i -``` - -3. Set up the TestMu AI credentials by using the commands below in the terminal.The account details are available on your [TestMu AI Profile](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile) page. - -For macOS: -```bash -export LT_USERNAME=LT_USERNAME -export LT_ACCESS_KEY=LT_ACCESS_KEY -``` -For Linux: -```bash -export LT_USERNAME=LT_USERNAME -export LT_ACCESS_KEY=LT_ACCESS_KEY -``` -For Windows: -```bash -set LT_USERNAME=LT_USERNAME -set LT_ACCESS_KEY=LT_ACCESS_KEY -``` - -1. Edit the required capabilities in your test file `playwright-smartui.js`. - -```javascript title="Add the following code snippet to run SmartUI with Playwright in playwright-smartui.js file" -const { chromium } = require('playwright') -const { expect } = require('@playwright/test'); - -(async () => { -const capabilities = { -'browserName': 'Chrome', // Browsers allowed: `Chrome`, `MicrosoftEdge`, `pw-chromium`, `pw-firefox` and `pw-webkit` -'browserVersion': 'latest', -'LT:Options': { -'platform': 'Windows 10', -'build': 'Playwright Sample Build', -'name': 'Playwright Sample Test', -'user': process.env.LT_USERNAME, -'accessKey': process.env.LT_ACCESS_KEY, -'network': true, -'video': true, -'console': true, -"smartUIProjectName": "" //Add the required SmartUI Project name -} -} - -const browser = await chromium.connect({ -wsEndpoint: `wss://cdp.lambdatest.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` -}) - -const page = await browser.newPage() - -await page.goto('https://www.bing.com') -// Add the following command in order to take screenshot in SmartUI -await page.evaluate((_) => {}, -`lambdatest_action: ${JSON.stringify({ action: "smartui.takeScreenshot", arguments: { fullPage: true, screenshotName: "" } })}`); // Add a relevant screenshot name here - -const element = await page.$('[aria-label="Enter your search term"]') -await element.click() -await element.type('LambdaTest') -await element.press('Enter') -const title = await page.title() - -try { -expect(title).toEqual('LambdaTest - Search') -// Mark the test as completed or failed -await page.evaluate(_ => {}, `lambdatest_action: ${JSON.stringify({ action: 'setTestStatus', arguments: { status: 'passed', remark: 'Title matched' } })}`) -} catch { -await page.evaluate(_ => {}, `lambdatest_action: ${JSON.stringify({ action: 'setTestStatus', arguments: { status: 'failed', remark: 'Title not matched' } })}`) -} - -await browser.close() -})() -``` - -- Execute the test using the following command - -```bash -node playwright-smartui.js -``` - -- You can check the executed builds over at [TestMu AI SmartUI](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/). - -## Fetching Screenshot Status and Results using WebHook - -To retrieve the status and results of a captured screenshot, you can utilize the provided webhook. Follow the steps below to fetch this information: - -```javascript -// Add this code snippet within your script to fetch the screenshot status during runtime - -response = await page.evaluate(_ => {}, `lambdatest_action: ${JSON.stringify({action: 'smartui.fetchScreenshotStatus', arguments: { screenshotName: "" }})}`) - -console.log(response) -``` -A sample response for the same can be seen below - -```json -screenshotStatus response: { -screenshotsData: [ -{ -screenshotName: '', -screenshotURL: '', -screenshotStatus: 'Approved', // current status of the screenshot -approvedBy: 'system', // Approver details -misMatchPercentage: 31.8, // percentage mismatch of the screenshot -threshold: 5000, // Threshold set for the screenshot -browserName: 'chrome', // Browser used for capturing the screenshot -resolution: '1920x1080' // Resolution of the screenshot -}, - -buildId: '', -projectName: '' -} -``` -The following are the description of the parameters: - -| Parameter | Description | -| -------------------- | ------------------------------------------------------------- | -| `screenshotName` | Name of the screenshot for which the result is to be fetched. | -| `screenshotURL` | URL of the screenshot captured. | -| `screenshotStatus` | Status of the screenshot captured. | -| `approvedBy` | Approver of the screenshot | -| `misMatchPercentage` | Percentage of mismatch in the screenshot captured. | -| `threshold` | Threshold set for the screenshot captured. | -| `browserName` | Browser used for capturing the screenshot. | -| `resolution` | Resolution of the screenshot captured. | -| `buildId` | ID of the build in which the screenshot was captured. | -| `projectName` | Name of the project in which the screenshot was captured. | - -By including the above code, you will be able to obtain the status of the captured screenshot. If you specify a particular `Screenshot Name`, the response will contain the result for that specific screenshot. - -Please note that the screenshot name should be provided within the screenshotName argument. - -## GitHub App Integration With Playwright - -Experience effortless collaboration and streamlined testing workflows with SmartUI's GitHub app integration for Playwright users. Now, you can seamlessly link your GitHub repositories to your SmartUI accounts. This integration empowers teams to effortlessly share test scripts, review and track changes, and provide feedback within the familiar GitHub environment. - -To add this integration, you can add the following capability: - -```bash -const capabilities: { -... -"smartUIProjectName": "" -"github": { -"url": "https://api.github.com/repos/OWNER/REPO/statuses/commitId" -} -... -} -``` -**For step by step guide for `Github Integration` click [here](/support/docs/smartui-github-app-integration/)** - -For additional information about Playwright framework please explore the documentation [here](/support/docs/playwright-testing/) - -## Advanced Options for Screenshot Comparison - - **Build Configuration** - If you have multiple screenshots running the same test suite and want to run the comparison for the same test suite, want to add a build as a baseline from your test suite or need to access more SmartUI Build Config Options, click [here](/support/docs/smart-ui-build-options/). - - **Handling Dynamic Data** - In case if you have any dynamic elements that are not in the same position across test runs, you can ignore or select a specific area to be removed from the comparison. For accessing such HTML DOM Config and Options, see [Handling Dynamic Data with DOM Configuration](/support/docs/html-dom-smartui-options). - -## Best Practices - -### Capability Configuration - -- Always set `visual: true` in your capabilities to enable SmartUI -- Use consistent project and build names across test runs -- Set meaningful test names for better organization - -**Example:** -```javascript -const capabilities = { -browserName: 'Chrome', -browserVersion: 'latest', -platformName: 'Windows 10', -'LT:Options': { -username: process.env.LT_USERNAME, -accessKey: process.env.LT_ACCESS_KEY, -visual: true, -name: 'Homepage Visual Test', -build: 'Release 1.0', -'smartUI.project': 'MyProject', -'smartUI.build': 'Build-1.0' -} -}; -``` - -### Screenshot Timing - -- Wait for page elements to load before capturing screenshots -- Use Playwright's wait methods for dynamic content -- Consider page load time when setting up tests - -**Example:** -```javascript -await page.goto('https://example.com'); -await page.waitForSelector('.main-content', { state: 'visible' }); -await page.waitForLoadState('networkidle'); -``` - -### Screenshot Naming - -- Use descriptive, consistent names -- Include context (page, component, state) in names -- Avoid special characters - -### Baseline Management - -- Establish baselines from stable builds -- Review and approve baselines before using -- Update baselines when intentional changes are made - -### Viewport Selection - -- Test on viewports that match your user base -- Include mobile, tablet, and desktop viewports -- Consider both portrait and landscape orientations - -### Viewport Selection - -- Test on viewports that match your user base -- Include mobile, tablet, and desktop viewports -- Consider both portrait and landscape orientations - -## Troubleshooting - -### Issue: Screenshots Not Captured - -**Symptoms**: Tests run but no screenshots appear in SmartUI dashboard - -**Possible Causes**: -- `visual: true` not set in capabilities -- Incorrect project name -- Network connectivity issues -- Credentials not set correctly - -**Solutions**: -1. Verify `visual: true` is set in capabilities: -```javascript -'LT:Options': { -visual: true, // Must be set -// ... other options -} -``` - -2. Check project name matches exactly (case-sensitive): -```javascript -'smartUI.project': 'ExactProjectName' -``` - -3. Verify credentials are set: -```bash -echo $LT_USERNAME -echo $LT_ACCESS_KEY -``` - -4. Check network connectivity to TestMu AI - -### Issue: "Project Not Found" Error - -**Symptoms**: Error indicating SmartUI project cannot be found - -**Possible Causes**: -- Project name typo or mismatch -- Project deleted -- Wrong account credentials - -**Solutions**: -1. Verify project exists in SmartUI dashboard -2. Copy project name directly from dashboard -3. Check credentials match the account with the project -4. Ensure project name is in capabilities, not just in dashboard - -### Issue: Screenshots Show Blank Pages - -**Symptoms**: Screenshots captured but show blank or incomplete content - -**Possible Causes**: -- Page not fully loaded -- JavaScript not executed -- Timing issues -- Viewport issues - -**Solutions**: -1. Add explicit waits before screenshot: -```javascript -await page.waitForSelector('#content', { state: 'visible' }); -await page.waitForLoadState('networkidle'); -``` - -2. Wait for specific elements to be visible: -```javascript -await page.waitForSelector('.main-content', { state: 'visible' }); -``` - -3. Increase wait time for slow-loading pages - -4. Check viewport size matches expected dimensions - -### Issue: Build Name Conflicts - -**Symptoms**: Screenshots appear in wrong build or build name issues - -**Possible Causes**: -- Build name not set consistently -- Special characters in build name -- Build name conflicts - -**Solutions**: -1. Set build name in capabilities: -```javascript -'smartUI.build': 'ConsistentBuildName' -``` - -2. Avoid special characters in build names - -3. Use consistent naming convention across team - -### Issue: Mismatch Percentage Unexpected - -**Symptoms**: Mismatch percentage higher or lower than expected - -**Possible Causes**: -- Threshold settings -- Dynamic content not ignored -- Rendering differences -- Baseline issues - -**Solutions**: -1. Review threshold settings in project settings - -2. Use `ignoreDOM` for dynamic content: -```javascript -'smartUI.options': { -'ignoreDOM': { -'id': ['timestamp', 'user-id'] -} -} -``` - -3. Check baseline is correct and up-to-date - -4. Review comparison settings in project - -### Getting Help - -If you encounter issues not covered here: - -- Review [SmartUI Build Options](/support/docs/smart-ui-build-options) documentation -- Check [Advanced Test Settings](/support/docs/test-settings-options/) for comparison options -- Visit [TestMu AI Support](https://www.testmuai.com/support) for additional resources -- Contact support at support@testmuai.com or use [24/7 Chat Support](https://www.testmuai.com/support) - -## Additional Resources - -- [SmartUI Build Options](/support/docs/smart-ui-build-options) -- [Advanced Test Settings](/support/docs/test-settings-options/) -- [Handling Dynamic Data](/support/docs/smartui-handle-dynamic-data) -- [Project Settings](/support/docs/smartui-project-settings) diff --git a/static/docs/playwright-webview-test.md b/static/docs/playwright-webview-test.md deleted file mode 100644 index 687203c97..000000000 --- a/static/docs/playwright-webview-test.md +++ /dev/null @@ -1,70 +0,0 @@ -# Playwright Webview Testing - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -WebView testing is an essential process for ensuring the functionality and reliability of applications that rely on embedded web browsers. This guide provides detailed steps to perform WebView testing using Playwright. It is designed for developers, QA engineers, and anyone involved in software quality assurance. - -## Steps to perform Webview testing - -### Step 1: Setup your test suite - -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**sample repo** -Download or Clone the code sample from the TestMu AI GitHub repository to run your tests. - - View on GitHub - -- If you are using your own project, configure the `browserWSEndpoint` (browser end point URL) in your test script to run the test on TestMu AI - -```javascript -wsEndpoint: `wss://cdp.lambdatest.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` -``` - -- Here is the sample test script for your reference: - -```javascript reference title="playwrightwebview.js" -https://github.com/LambdaTest/playwright-sample/blob/main/playwrightwebview.js -``` - -### Step 2: Update the dependencies -Before proceeding forward, run the below command to update the outdated dependencies - -```bash -npm install -``` - -### Step 3: Update your test Capabilities -Include the following field in the capabilities object to enable WebView testing: - -```javascript title="playwrightwebview.js" -const capabilities = { -"LT:Options": { -"platformName": "android", -"isRealMobile": true, //if true, test will run on real devices -// highlight-next-line -"isPwMobileWebviewTest": true, //mandatory capability to enable WebView testing -}, -``` - -Use our [Capability Generator](https://www.lambdatest.com/capabilities-generator/) to select from a wide range of options for customizing your tests. - -### Step 4: Setup your TestMu AI credentials - -In your terminal (as per your respective Operating System), run these command to setup your TestMu AI credentials. -> You can see your credentials below if you have logged into our platform. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -### Step 5: Execute your test -Replace the `TEST_FILE_NAME` in the below command with your desired test file to execute that particular test on TestMu AI: - -```bash -node playwrightwebview.js -``` - -Visit the [TestMu AI Web Automation](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) page to check the status of your test execution. diff --git a/static/docs/playwright-with-cucumberjs.md b/static/docs/playwright-with-cucumberjs.md deleted file mode 100644 index 2b029d330..000000000 --- a/static/docs/playwright-with-cucumberjs.md +++ /dev/null @@ -1,161 +0,0 @@ -# Playwright Testing With Cucumber.js - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Cucumber.js is a JavaScript-based open-source framework for web automation testing. It runs on Node.js and latest web browsers. Cucumber.js allows you to write and execute tests in Gherkin - a non-technical and human-readable language. - -TestMu AI enables you to run Playwright tests with Cucumber.js test runner on a browser farm of 40+ real browser and operating system combinations. This guide will outline the fundamentals of getting started with Playwright testing on the TestMu AI platform using the Cucumber.js test runner. - -## Prerequisites - ->Note: All the code samples in this documentation can be found in the TestMu AI's Repository on GitHub. You can either download or clone the repository to quickly run your tests. - View on GitHub - -1. Clone the TestMu AI-Playwright repository on your system. - -2. Install the npm dependencies. - -``` -npm install -``` - -3. In order to run your Playwright tests with Playwright test runner, you will need to set your TestMu AI username and access key in the environment variables. Click the **Access Key** button at the top-right of the Automation Dashboard to access it. - -**Windows** - -```js -set LT_USERNAME="YOUR_LAMBDATEST_USERNAME" -set LT_ACCESS_KEY="YOUR_LAMBDATEST_ACCESS_KEY" -``` - -**macOS/Linux** - -```js -export LT_USERNAME="YOUR_LAMBDATEST_USERNAME" -export LT_ACCESS_KEY="YOUR_LAMBDATEST_ACCESS_KEY" -``` - -## Running Playwright Tests With Cucumber.js - -In your `setup.js` file, specify your TestMu AI Username and TestMu AI Access Key, add the browserName, browserVersion, and platform. - -```js -const { setWorldConstructor, World, Before, After} = require("@cucumber/cucumber"); -const { chromium } = require('playwright') - -class CustomWorld extends World{ -async setTestStatus(status, remark) { -await page.evaluate(_ => {}, `lambdatest_action: ${JSON.stringify({ action: 'setTestStatus', arguments: { status, remark } })}`) -} -} - -Before(async (scenario) => { -const capabilities = { -'browserName': 'Chrome', // Browsers allowed: `Chrome`, `MicrosoftEdge`, `pw-chromium`, `pw-firefox` and `pw-webkit` -'browserVersion': 'latest', -'LT:Options': { -'platform': 'Windows 10', -'build': 'Playwright Sample Build with Cucumber Runner', -'name': scenario.pickle.name, -'user': process.env.LT_USERNAME, -'accessKey': process.env.LT_ACCESS_KEY, -'network': true, -'video': true, -'console': true, -'tunnel': false, // Add tunnel configuration if testing locally hosted webpage -'tunnelName': '' // Optional -} -} - -// Create page and browser globals to be used in the scenarios -global.browser = await chromium.connect({ -wsEndpoint: `wss://cdp.lambdatest.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` -}) - -const context = await global.browser.newContext(); - -global.page = await context.newPage(); -}) - -After(async () => { -await global.browser.close() -}) - -setWorldConstructor(CustomWorld); -``` -Pass the below command to run the test. - -``` -npm run test -``` - -Visit the TestMu AI Automation dashboard to view the results of your executed test with Playwright test runner. - -## Testing With Cucumber.js When Migrating To TestMu AI - -1. In your `setup.js` file, add before and after code block for setting up and closing the remote browsers. - -```js -Before(async (scenario) => { -const capabilities = { -'browserName': 'Chrome', // Browsers allowed: `Chrome`, `MicrosoftEdge`, `pw-chromium`, `pw-firefox` and `pw-webkit` -'browserVersion': 'latest', -'LT:Options': { -'platform': 'Windows 10', -'build': 'Playwright Sample Build with Cucumber Runner', -'name': scenario.pickle.name, -'user': process.env.LT_USERNAME, -'accessKey': process.env.LT_ACCESS_KEY, -'network': true, -'video': true, -'console': true, -'tunnel': false, // Add tunnel configuration if testing locally hosted webpage -'tunnelName': '' // Optional -} -} - -// Create page and browser globals to be used in the scenarios -global.browser = await chromium.connect({ -wsEndpoint: `wss://cdp.lambdatest.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` -}) - -const context = await global.browser.newContext(); - -global.page = await context.newPage(); -}) - -After(async () => { -await global.browser.close() -}) - -setWorldConstructor(CustomWorld); -``` - -2. In the class CustomWorld, create a function `setTestStatus` that will mark your test passed or failed on the TestMu AI platform. - -```js -class CustomWorld extends World{ -async setTestStatus(status, remark) { -await page.evaluate(_ => {}, `lambdatest_action: ${JSON.stringify({ action: 'setTestStatus', arguments: { status, remark } })}`) -} -} -``` - -3. After any assertions in your script mark the test status as passed as shown below: - -```js -try { -assert.equal(title, -"How to use HyperExecute for scalable and reliable web automation testing | TestMu AI", -"Page title does not match"); - -await this.setTestStatus("passed", "Title matched"); -} catch (e) { -await this.setTestStatus("failed", e); -throw(e); -} -``` - -4. After that, you can run your test. - -To generate Playwright tests with AI coding assistants, see [Run Tests With Agent Skills](/support/docs/playwright-agent-skills/). diff --git a/static/docs/plugins-and-extensions.md b/static/docs/plugins-and-extensions.md deleted file mode 100644 index 15db36575..000000000 --- a/static/docs/plugins-and-extensions.md +++ /dev/null @@ -1,30 +0,0 @@ -# Plugins and Extensions - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI offers you plugins with various third-party application to help you perform cross browser testing on 3000+ real browsers with pace and convenience. Here are all the plugins and extensions available to integrate TestMu AI and perform testing in a jiffy. - -Chrome Extension - -Jenkins Plugin - -WordPress Plugin - -npm Plugin For TestCafe - -fastlane Plugin - -Azure Pipeline Extension - -TeamCity Plugin - -TestMu AI VS Code Extension - -LambdaMax XCode Plugin - -TestMu AI Docker Tunnel Extension - -Gradle Plugin - -**Got something to recommend?** -We would love to hear it. Drop the suggestion in the **24/7 chat support** for any plugin that you may want us to integrate with. diff --git a/static/docs/practitest-integration.md b/static/docs/practitest-integration.md deleted file mode 100644 index cae9d01e4..000000000 --- a/static/docs/practitest-integration.md +++ /dev/null @@ -1,45 +0,0 @@ -# PractiTest Integration with TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -[PractiTest](https://www.practitest.com/) is a QA and test management solution to provide users with better control of their testing and development process. Enabling organizations to guarantee visibility and communication with relevant stakeholders. It also helps project development teams to streamline and manage their testing processes while providing management with a clear and simple view of their project status at all times. - -TestMu AI's integration with PractiTest will further help to log bugs to your projects on PractiTest. You can easily find and report any cross browser compatibility issues to your team instantly. - -## Integrate PractiTest from your TestMu AI Account - -**Step 1:** Login to your TestMu AI account. You should have Admin or User level access to see and install integrations. - -**Step 2:** Click on Settings -> Integrations -> Project Management. - -**Step 3:** Click on **Connect** button of PractiTest block. - -**Step 4:** Now, provide your PractiTest API Token and Teamwork site address to establish integration with TestMu AI and click on install button. - -**How to Get your API Token** - -Visit your PractiTest Dashboard -> Settings -> API Tokens. - -## Lodge your First Bug - -**Step 1:** Start with any type of testing, for the demo purpose we are going with the **Real Time Testing**. - -**Step 2:** Enter your **Project URL** and configure for **browser** and **operating system** of your choice & hit Start. - -**Step 3:** After the VM is launched and operable. You can perform testing on your web-app for finding bugs. If a bug gets revealed, then you need to click on the Bug icon from the left panel for capturing a screenshot of the same. - -**Step 4:** After clicking on "**Mark as Bug**" button a PractiTest specific form would open up. Fill the fields as per your requirement. - -- Select which project should the ticket go under. -- Set which Task List should the bug go under. -- Set the priority of the issue. -- Mention the Title as well to easily organize your bugs. -- Write a description to help relate the cause of the issue or the task. - -## Uninstall PractiTest Integration - -**Step 1:** Login to your TestMu AI account. You should have Admin or User level access to see and install integrations. - -**Step 2:** Click on Settings -> Integrations -> Project Management. - -**Step 3:** Click on the **Remove** button. diff --git a/static/docs/pre-loaded-chrome-extension.md b/static/docs/pre-loaded-chrome-extension.md deleted file mode 100644 index b296267ab..000000000 --- a/static/docs/pre-loaded-chrome-extension.md +++ /dev/null @@ -1,36 +0,0 @@ -# Pre-Loaded Extension for Chrome - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The Pre-Loaded Chrome Extensions feature in TestMu AI's real-time testing allows users to install and use Chrome extensions during your testing sessions. This capability provides the flexibility to test web applications with various extensions, such as ad blockers, password managers, and developer tools, replicating real user environments more accurately. - -> The Pre-Loaded Extension is only accessible on **Chrome** browsers. - -## Prerequisites -- A TestMu AI account. -- Access to the real-time testing feature in the TestMu AI platform. -- Chrome extensions to be tested - -## Accessing the Chrome Extensions Feature - -**Step 1:** Select on the **Real Time** > Desktop Web Browser Testing from the sidebar. - -**Step 2:** Click on the Chrome Extension drop down button in the dashboard. - -**Step 3:** Click on the **Add New Extension** button and then paste the url of the chrome extension that you want to apply and click on the + icon. Your chrome extension will be added. - -**Step 4:** Now again click on the Chrome Extension drop down button and select all the extensions that you want to apply for your testing purpose and click on the **Apply** button. - -## Removing the Chrome Extension - -**Step 1:** Navigate to Real Time Desktop Web Browser Testing console. - -**Step 2:** Unselect all those extensions that you want to remove, and click on **Apply** button. - -## Use Cases for Chrome Extensions in Real-Time Testing -- **Testing with Ad Blockers :** Ensure that your web application functions correctly when ad blockers are enabled. -- **Testing Third-Party Integrations:** Validate integrations with password managers, video downloaders, or custom automation scripts. -- **Debugging with Developer Tools :** Use developer-focused extensions like Lighthouse, React DevTools, or Web Developer to inspect your web application. -- **Accessibility Testing :** Install extensions such as Axe or WAVE to perform accessibility audits during manual testing. - -> If you have any questions or need further information about Pre-Loaded Chrome Extension, please feel free to reach us at our **24*7 Chat Support** or email us at [support@testmuai.com](mailto:support@testmuai.com). diff --git a/static/docs/private-dependencies-cypress.md b/static/docs/private-dependencies-cypress.md deleted file mode 100644 index 30ca33a23..000000000 --- a/static/docs/private-dependencies-cypress.md +++ /dev/null @@ -1,50 +0,0 @@ -# Execute Cypress Tests Including Private Dependencies - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This article guides you on how to execute your **Cypress** tests including the **private dependencies** on **TestMu AI platform**. -Earlier if your project includes any private dependency, the tests will error out with the **LambdaError** on our platform but now we have built the support for handling the private dependencies. - -> In this demo we are demonstrating using [TestMu AI's Sample Cypress Cloud Repo](https://github.com/LambdaTest/Cypress-Cloud). You can find all the resources used for this article in the linked repo. - -## Steps To Execute Using Private Dependencies - -Assuming that you have run a Cypress test on TestMu AI platform ( if not, this article [here](/support/docs/getting-started-with-cypress-testing/) will guide you in running your first Cypress test on TestMu AI), you need to follow these steps below: - -### Update TestMu AI Cypress CLI - -- Before getting started, you would have to update the **lambdatest-cypress cli** using the command below: - -```bash -npm install -g lambdatest-cypress-cli -``` - -- The CLI version should be the latest i.e. is 2.3.0. You can check the CLI version by running the below command. - -```bash -lambdatest-cypress --version -``` - -### Add Private Dependencies - -You can either add private dependency in the `package.json` file or in the `lambdatest-config.json` file. Refer to the below steps for both approaches. - -- **If passing private dependency in package.json file ** - - Add the private dependency and create `.npmrc` file in the root directory of your project as highlighted below and define the same in `.npmrc` file as well. - -- **If passing private dependency in lambdatest-config.json file -** - - If you define the dependencies in the `lambdatest-config.json` file, it will override the `package.json` file and our cloud grid will read the dependencies from the `lambdatest-config.json` file. You can define the dependencies in the npm_dependencies object available in the run_settings of `lambdatest-config.json` file. Also, here as well, you have to create `.npmrc` file and there you have to define the private dependency. - - > If you are using an **access token** in the private dependency and you are accessing its value from the **environment variable**, in that case, you have to add `dep_token` capability and define all those in environment variables in it. You can pass multiple environment variables as comma-separated. - - Refer the below screenshot for the help. - - > All the keys present in **dep_tokens** will be replaced in the `.npmrc` file and if the key is not present in the environment variable, it will throw an error. - -After doing this setup, run your tests on the **TestMu AI platform** and now the test will execute perfectly with the **private dependencies**. - -> To download private dependencies over tunnel follow this [documentation](/support/docs/npm-via-tunnel/). - -> That's all! In case you have any questions or need any additional information, you could reach out at our **[24X7 Chat Support]** or mail us directly at support@testmuai.com. diff --git a/static/docs/private-device-allocation-to-teams.md b/static/docs/private-device-allocation-to-teams.md deleted file mode 100644 index d833d1939..000000000 --- a/static/docs/private-device-allocation-to-teams.md +++ /dev/null @@ -1,53 +0,0 @@ -# Private Device Allocation to Teams - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The Teams feature for Private Cloud Devices enables organization admins to divide and allocate private cloud devices among teams within their organization which ensures effective resource management. Team members then gain exclusive access to the allocated devices, ensuring efficient and secure testing workflows. - -> To enable it for your organization, please contact us via **24×7 chat support** or you can also drop a mail to **support@testmuai.com**. - -## Steps to Allocate Devices to Teams - -> Only the organization's Admins have the authority to allocate or deallocate devices. - -### Step 1: Create or Select a Team -- Go to the Teams section under the Organization Settings -- Create your [Organization](/support/docs/team-management/#create-an-organization) -- Then, create your [Team](/support/docs/team-management/#create-your-first-team) and assign members by selecting users from your organization. - -### Step 2: Allocate Devices to the Team -In the **Private Cloud Devices** tab section, distribute your devices amongst your teams for restricted access via: - - - Click on the **Bulk Distribute** button and select all the devices that you want to allocate to the teams. - - Click on the **Distribute** button. In the popup, choose the teams from the dropdown list. - - Click on the **Allocate** button. -The devices will now appear in the selected team's device pool, accessible only to its members. - - - Click on the **Allocate** button from the right side of your desired device. - - In the popup, choose the teams from the dropdown list. - - Click on the **Allocate** button. -The selected device will now appear in the team's device pool, accessible only to its members. - -> You can also see tags of all those teams to which those devices are allocated. - -## Deallocate any Device from Teams -- Click on the **Deallocate** button from the right side of your desired device. - -The selected device will no longer appear in all those allocated team's device pool. - -## Edit Allocation of any Device -To edit allocation of any device, or simply if you want to add/remove any team allocation for any particular device then: - -- Click on the **Edit Allocation** button from the right side of your desired device. -- Select or Deselect your required teams. -- Click on the **Allocate** button. - -A device in a busy state will not be allocated or deallocated. - -## Filter your Devices -In the topbar, you have plenty of options to filter your devices based on: - -- You can select from the drop down whether you want to see **All Devices**, **Allocated Devices**, or all the **Available Devices** in the table. -- You can also search for a particular device based on the **device name**, or its **UDID**. -- **OS :** Select which OS devices you want to allocate or deallocate. -- **Teams :** Select from which team you want to filter your device and perform allocation or deallocation of devices. diff --git a/static/docs/protractor-on-hyperexecute-grid.md b/static/docs/protractor-on-hyperexecute-grid.md deleted file mode 100644 index 89e3fb74c..000000000 --- a/static/docs/protractor-on-hyperexecute-grid.md +++ /dev/null @@ -1,228 +0,0 @@ -# Running Protractor Framework Tests on HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Protractor is a widely-used testing framework for Java applications, designed to simplify and enhance the testing process for developers. It provides a flexible and powerful platform for running test suites, enabling effective unit testing, integration testing, and end-to-end testing of Java applications - -HyperExecute is an AI Native Test Orchestration Cloud Platform that empowers you to run **end-to-end** tests **quickly** and **efficiently**. It provides Just-in-Time (JIT) testing infrastructure with fast execution **speeds**, **smart orchestration**, and **detailed logs**. - -This guide details how to execute your **Protractor** framework tests on **HyperExecute** via two different methods: - -- [**Using Local System**](/support/docs/protractor-on-hyperexecute-grid/#1-testing-using-local-system) - You can use your own local machine to execute tests. -- [**Using Gitpod Platform**](/support/docs/protractor-on-hyperexecute-grid/#2-testing-using-gitpod) - Execute tests using GitPod. (Requires a [Gitpod](https://gitpod.io/login/) account) - -## 1. Testing Using Local System - -Follow the step-by-step guide to execute your test on HyperExecute. - -### Prerequisites - -To run the Tests on HyperExecute from your Local System, you are required: - -- Your TestMu AI [Username and Access key](/support/docs/hyperexecute-how-to-get-my-username-and-access-key/) -- [HyperExecute YAML](/support/docs/hyperexecute-yaml-version0.2/) file which contains all the necessary instructions. -- [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) in order to initiate a test execution Job . -- Setup the [Environmental Variable](/support/docs/hyperexecute-environment-variable-setup/) - -### Step 1: Configure Your Test Suite - -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** - -Download or Clone the code sample for the JUnit from the TestMu AI GitHub repository to run the tests on the HyperExecute. - - View on GitHub - -If you are using your own project, make sure you update the **Hub endpoint** in your tests file. - -By setting up the Hub endpoint, you establish the communication channel between your tests and the browser nodes, enabling effective test distribution and execution. - -Configure the desired capabilities based on your test requirements. For example: - -```bash -capabilities = { -build: 'protractor-LambdaTest-Single', -browserName: 'chrome', -version:'latest', -platform: 'Windows 10', -}, -``` - -> You can also use **dev** and **beta** browser versions. To generate capabilities for your test requirements, you can use our inbuilt 🔗 Capabilities Generator. - -### Step 2: Setup the CLI in your Test Suite - -After cloning / downloading the sample repo, you need to setup the CLI and the environment variables. - -#### Download the HyperExecute CLI - -The CLI is used for triggering the tests on HyperExecute. It is recommend to download the CLI binary on the host system and keep it in the root directory of the suite to perform the tests on HyperExecute. - -You can download the CLI for your desired platform from the below mentioned links: - -| Platform | HyperExecute CLI | -| ---------| ---------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| MacOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -#### Setup Environment Variable - -Now, you need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in the [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). - -Run the below mentioned commands in your terminal to setup the CLI and the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -### Step 3: Configure YAML in your Test Suite - -Configure your YAML file as per your use cases using **key value** pairs. - -In this sample YAML file, we have mentioned: - -- **version** of the YAML file -- **Timeouts** for executing your project -- **Mode of execution** is [Autosplit](/support/docs/hyperexecute-auto-split-strategy/). You can also opt for [Matrix](/support/docs/hyperexecute-matrix-multiplexing-strategy/) or [Hybrid](/support/docs/hyperexecute-hybrid-strategy/) mode. -- **Pre and Post** commands -- **Reports and Artefacts** that will be generated after the completion of tests -- and other necessary YAML Parameters - -```yaml ---- -version: 0.1 -globalTimeout: 90 -testSuiteTimeout: 90 -testSuiteStep: 90 - -runson: linux - -autosplit: true -retryOnFailure: true - -maxRetries: 1 -concurrency: 2 - -cacheKey: '{{ checksum "package-lock.json" }}' -cacheDirectories: -- node_modules - -pre: -- npm install -g protractor -- npm install - -mergeArtifacts: true - -uploadArtefacts: -- name: Reports -path: -- ProtractorTestReport.html -- xmlresults.xml - -report: true -partialReports: -type: json -location: / -frameworkName: extent - -testDiscovery: -type: raw -mode: remote -command: grep -nri 'describe' specs -ir --include=*.js | sed 's/:.*//' -testRunnerCommand: protractor conf/single.conf.js --specs=$test --browser=chrome - -jobLabel: [selenium-protractor, linux, autosplit] -``` - -### Step 4: Execute your Test Suite - -> **NOTE :** In case of MacOS, if you get a permission denied warning while executing CLI, simply run **`chmod u+x ./hyperexecute`** to allow permission. In case you get a security popup, allow it from your **System Preferences** → **Security & Privacy** → **General tab**. - -Run the below command in your terminal at the root folder of the project: - -```bash -./hyperexecute --config RELATIVE_PATH_OF_YOUR_YAML_FILE -``` - -OR use this command if you have not exported your username and access key in the step 2. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE `} - -### Step 5: Monitor the Test Execution - -Visit the [HyperExecute Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) and check your Job status. - -### Step 6: Download Artifacts and Reports - -HyperExecute also facilitates the provision to download the [Artifacts](/support/docs/hyperexecute-artifacts/) and [Reports](/support/docs/hyperexecute-reports/) on your local machine. Click on the corresponding button to download your generated artifacts and reports. - -## 2. Testing Using Gitpod - -You can also use the Gitpod platform to execute our sample repository. It will fetch all the sample codebases and trigger the CLI to execute the tests. - -Follow the below steps to run Test using Gitpod: - -**Step 1:** Click '**Open in Gitpod**' button. You will be redirected to Login/Signup page. This button is configured to redirect you to the Gitpod platform where you will be able to execute our sample repository. - -**Step 2:** Login with TestMu AI credentials. Once logged in, a pop-up confirmation will appear, asking you to **'Proceed'** to the Gitpod editor in a new tab. The current tab will display the HyperExecute Dashboard. - -**Step 3:** Choose your preferred editor (we recommend VS Code Editor) - -**Step 4:** As you are running a sample project, Fetching of the Test Scripts, [HyperExecute YAML](/support/docs/deep-dive-into-hyperexecute-yaml/), [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) and Triggering your tests using the `Execution Command` will be automated. - -**Step 5:** Once you see the `Job Link` in the logs, you can visit the [HyperExecute dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) to see the tests getting executed. - -You can also implement [Secret Keys](/support/docs/hyperexecute-how-to-save-and-manage-secrets/) in your YAML file. - -## Navigation in Automation Dashboard - -Every test run on the HyperExecute has a unique *jobId* associated with it. Each *jobId* can in turn constitute single (or multiple) *groupId*(s). You can visit [HyperExecute Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) for checking the status of the test execution. - -You can seamlessly navigate between JobId's and taskId's. You need to click on the *testID* to navigate from the HyperExecute logs to the Automation Dashboard. - -The snapshot below shows the videos, logs and other meta data for that specific *test_ID* - -> For any query or doubt, please feel free to contact us via **24×7 chat support** or you can also drop a mail to **support@testmuai.com**. -Happy testing! - -## Using the Protractor Agent Skill with TestMu AI - -The [protractor-skill](https://github.com/LambdaTest/agent-skills/tree/main/protractor-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The protractor-skill package includes: - -``` -protractor-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing Protractor Agent Skill - -Install a Protractor Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/protractor-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/protractor-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only protractor-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). diff --git a/static/docs/protractor-with-selenium-running-protractor-automation-scripts-on-testmu-selenium-grid.md b/static/docs/protractor-with-selenium-running-protractor-automation-scripts-on-testmu-selenium-grid.md deleted file mode 100644 index 72622f7f3..000000000 --- a/static/docs/protractor-with-selenium-running-protractor-automation-scripts-on-testmu-selenium-grid.md +++ /dev/null @@ -1,104 +0,0 @@ -# Selenium With Protractor - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Run Protractor tests on the TestMu AI cloud grid. This guide covers setup, running a sample test, configuring capabilities, and testing locally hosted pages. - -**Deprecated Framework** -Protractor reached end-of-life in August 2023. Consider migrating to [Playwright](/support/docs/playwright-testing/) or [WebDriverIO](/support/docs/webdriverio-with-selenium-running-webdriverio-automation-scripts-on-testmu-selenium-grid/). This guide is maintained for existing users. - -**Sample repo** -All the code used in this guide is available in the sample repository. - - View on GitHub - -## Prerequisites - -Complete these steps before running Protractor Selenium tests on TestMu AI. - -1. Create a [TestMu AI account](https://www.testmuai.com/register/?redirectTo=https://accounts.lambdatest.com/dashboard) and get your username and access key from the dashboard. -2. Install **NodeJS** v6 or newer from [nodejs.org](https://nodejs.org/en/). -3. Install **npm** from the [official npm website](https://www.npmjs.com/). -4. Download [Selenium JavaScript bindings](https://www.selenium.dev/downloads/) from the official website. - -## Step 1: Clone the Sample Project - -Clone the TestMu AI Protractor Selenium sample repository to your local machine. - -```bash -git clone https://github.com/LambdaTest/protractor-selenium-sample -cd protractor-selenium-sample -``` - -Install the required dependencies: -```bash -npm install -``` - -## Step 2: Set Your Credentials - -Set your TestMu AI username and access key as environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -## Step 3: Configure Your Test Capabilities - -Update the capabilities object to define the browser and platform settings. - -```js -capabilities = { -build: 'protractor-LambdaTest-Single', -browserName: 'chrome', -browserVersion: 'latest', -platformName: 'Windows 10', -}, -``` - -Generate capabilities for your test requirements with the [Capabilities Generator](https://www.testmuai.com/capabilities-generator/). - -## Step 4: Run the Test - -Execute the test using the following command. - -**Single test:** -```bash -npm run single -``` - -**Parallel tests:** -```bash -npm run parallel -``` - -## Step 5: View Your Results - -After running the test, view your results on the [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build). - -The dashboard provides: -- Video recordings of each test session -- Screenshots captured at each step -- Console logs from the browser -- Network logs for debugging -- Detailed command logs - -## Run Protractor Tests Using Agent Skills - -Use AI coding assistants to generate and run Protractor tests with the TestMu AI Agent Skill. - -The [protractor-skill](https://github.com/LambdaTest/agent-skills/tree/main/protractor-skill) is part of [TestMu AI Agent Skills](https://github.com/LambdaTest/agent-skills/) - structured packages that teach AI coding assistants how to write production-grade test automation. - -Install the skill: - -```bash -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/protractor-skill .claude/skills/ - -# For Cursor / Copilot -cp -r agent-skills/protractor-skill .cursor/skills/ -``` - -Install all available framework skills at once by cloning the repository directly into your tool's skills directory (e.g., `.claude/skills/`, `.cursor/skills/`). diff --git a/static/docs/public-cloud-vs-private-cloud.md b/static/docs/public-cloud-vs-private-cloud.md deleted file mode 100644 index 091e44a5b..000000000 --- a/static/docs/public-cloud-vs-private-cloud.md +++ /dev/null @@ -1,25 +0,0 @@ -# Public Cloud VS Private Cloud - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -In mobile app development, thorough testing on real devices is essential for performance and seamless user experience. Platforms like Public and Private Device Clouds offer access to a variety of real devices for testing, each with differences in accessibility, security, and features. Understanding these nuances is crucial for organizations to choose the right option. - -Our detailed document compares Public Device Cloud and Private Device Cloud, focusing on their benefits for real device testing. It considers factors such as device access, session duration, settings control, and testing capabilities allowing organizations to make informed decisions about their testing strategies based on their needs and goals. - -## Why Choose Private Cloud - -Here is the detailed comparison of Public Cloud and Private Cloud: - -| Features | Public Device Cloud | Private Device Cloud | -| -------- | ------------------- | -------------------- | -| Device Access | Accessible by only a single user during session, but shared for different test sessions.| Only accessible by the specific organization and no access to anyone else. | -| Session Duration | Maximum session duration is 6 hours.| Maximum session duration is 12 hours.| -| Session Retention | Devices undergo a factory reset after each session.| Users have the option to retain the session or reset as per their need. | -| Settings Access | Devices settings not accessible.| Complete access to all device applications including settings.| -| SIM-Based Testing | Not available.| Available for comprehensive testing.| -| Apple Pay Testing | Not Supported.| Supported for thorough testing scenarios.| -| Bluetooth-Based Use Cases | Bluetooth functionality not available for testing.| Bluetooth functionality available for testing on dedicated devices.| -| Assistive Touch | Unavailable.| Available only on dedicated devices.| -| Device Status | Unavailable.| Real time device status with active user name.| - -To conclude, both Public Device Cloud and Private Device Cloud are useful for real device testing, but the best choice depends on an organization's needs. Private Device Cloud offers **exclusive access, longer session duration, control over device retention, advanced testing features,** and **clear device usage status** making it ideal for organizations prioritizing security, customization, and thorough testing. By using Private Device Cloud, organizations can improve their testing, work more efficiently, and provide better mobile apps to users. diff --git a/static/docs/puppeteer-agent-skills.md b/static/docs/puppeteer-agent-skills.md deleted file mode 100644 index 283d9a9b5..000000000 --- a/static/docs/puppeteer-agent-skills.md +++ /dev/null @@ -1,118 +0,0 @@ -# Run Your Puppeteer Tests Using Agent Skills - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Instead of manually setting up your Puppeteer project, you can use **Agent Skills** to let your AI coding assistant generate production-ready Puppeteer automation for you. - -The [puppeteer-skill](https://github.com/LambdaTest/agent-skills/tree/main/puppeteer-skill) is part of [TestMu AI Agent Skills](https://github.com/LambdaTest/agent-skills/) - structured packages that teach AI coding assistants how to write production-grade test automation code. - -## What Are Agent Skills? - -Agent Skills are self-contained packages of instructions, code patterns, debugging guides, and CI/CD configurations for specific testing frameworks. Once installed, your AI assistant works like a **Senior QA automation architect** with knowledge of: - -- Proper project structure for each framework -- Correct dependency versions and configurations -- Both local and TestMu AI cloud execution patterns -- Common pitfalls and debugging approaches -- CI/CD integration with GitHub Actions - -## Prerequisites - -- A TestMu AI account. If you don't have one, sign up for free. -- Your TestMu AI **Username** and **Access Key** from the [Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/). -- An AI coding assistant: [Claude Code](https://claude.ai/code), [GitHub Copilot](https://github.com/features/copilot), [Cursor](https://cursor.sh/), [Gemini CLI](https://github.com/google-gemini/gemini-cli), or any compatible tool. - -## Installing the Puppeteer Agent Skill - -Clone the repository and copy the Puppeteer skill into your AI tool's skills directory: - -```bash -git clone https://github.com/LambdaTest/agent-skills.git - -# For Claude Code -cp -r agent-skills/puppeteer-skill .claude/skills/ - -# For Cursor -cp -r agent-skills/puppeteer-skill .cursor/skills/ - -# For GitHub Copilot -cp -r agent-skills/puppeteer-skill .github/skills/ - -# For Gemini CLI -cp -r agent-skills/puppeteer-skill .gemini/skills/ -``` - -If you prefer installing **all available framework skills**, clone the repository directly into your tool's skills directory (e.g., `.claude/skills/`, `.cursor/skills/`). - -## Set Up Your Authentication - -Configure your TestMu AI credentials as environment variables: - -```bash -export LT_USERNAME="YOUR_LAMBDATEST_USERNAME" -export LT_ACCESS_KEY="YOUR_LAMBDATEST_ACCESS_KEY" -``` - -## Running Your First Test Using Agent Skills - -Once installed, simply ask your AI assistant to write tests in natural language: - -``` -"Write Puppeteer tests to scrape product data and run them on TestMu AI cloud" -``` - -``` -"Set up Puppeteer with Jest for E2E testing and generate PDF reports" -``` - -``` -"Run headless Chrome tests on TestMu AI with network interception" -``` - -The Agent Skill automatically handles: - -1. **Project setup** - Creates the right folder structure, dependencies, and config files -2. **Language detection** - Maps your request to JavaScript or TypeScript -3. **Execution target** - Configures local or TestMu AI cloud execution via CDP WebSocket -4. **Best practices** - Uses `waitForSelector`, proper navigation handling, and network interception patterns - -## What's Included in the Puppeteer Skill - -``` -puppeteer-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -| Section | Details | -|---------|---------| -| **Project Setup** | Dependencies, versions, configuration files, project structure | -| **Core Patterns** | Complete, runnable code examples with proper wait strategies | -| **Cloud Integration** | TestMu AI CDP WebSocket connection and capabilities configuration | -| **CI/CD Integration** | GitHub Actions workflows with reporting and parallel execution | -| **Debugging Guide** | 12+ common issues mapped to solutions | -| **Best Practices** | Actionable recommendations for production code | - -## Supported Frameworks with Dedicated Agent Skills - -The following framework docs under Puppeteer reference their own dedicated Agent Skills: - -| Framework | Agent Skill | Documentation | -|-----------|------------|---------------| -| Jest | [jest-skill](https://github.com/LambdaTest/agent-skills/tree/main/jest-skill) | [Puppeteer with Jest](/docs/puppeteer-testing-with-jest/) | -| Mocha | [mocha-skill](https://github.com/LambdaTest/agent-skills/tree/main/mocha-skill) | [Puppeteer with Mocha](/docs/puppeteer-testing-with-mocha/) | -| pytest (Pyppeteer) | [pytest-skill](https://github.com/LambdaTest/agent-skills/tree/main/pytest-skill) | [Puppeteer with pytest](/docs/puppeteer-testing-with-pytest-pyppeteer/) | - -## Supported AI Tools - -| Tool | Type | Installation Path | -|------|------|-------------------| -| Claude Code | CLI | `.claude/skills/` | -| GitHub Copilot | Extension | `.github/skills/` | -| Cursor | IDE | `.cursor/skills/` | -| Gemini CLI | CLI | `.gemini/skills/` | -| Codex CLI | CLI | `.codex/skills/` | -| OpenCode | CLI | `.opencode/skills/` | -| Claude.ai | Web | Settings > Features > Skills | diff --git a/static/docs/puppeteer-test-execution-setup.md b/static/docs/puppeteer-test-execution-setup.md deleted file mode 100644 index 2e07c1aae..000000000 --- a/static/docs/puppeteer-test-execution-setup.md +++ /dev/null @@ -1,202 +0,0 @@ -# Puppeteer - Test Execution Setup - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Whenever you run a Puppeteer test, you must specify the operating system and the browser you wish to use. The TestMu AI Desired Capabilities Generator allows you to automatically create the capabilities class needed to run your Puppeteer automation scripts on TestMu AI. - -In this guide, learn how to configure the desired capability for selecting browsers and OS, organzing tests, changing desktop resolution, and more for your Puppeteer tests. - -## Choosing Browser And OS - -To perform Puppeteer testing on TestMu AI, you need to define the `browserName`, `browserVersion`, and `platform` capabilities in your automation scripts. - -| Key | Expected Values | Description | Example | -| -------- | -----| ------- | ----------------- | -| browserName | Chrome, Edge | Specify the browser to test on | `const capability = {"browserName": "Chrome"}` -| browserVersion | Chrome 83 & above, Edge 83 & above | Specify the browser version to test on | `const capability = {"browserVersion": "113.0"}` -| platform | **Windows**: 11, 10, 8, 8.1, 7 **macOS**: Monterey, Big Sur, Catiline, Mojave | Specify the platform name | `const capability = { "LT:Options": {"platform": "Windows 10",}}` - -## Organizing Tests - -You can name your test cases and categorize your builds by build, and name for easier analysis. You will need to use the name, and build capabilities to organize Puppeteer automated tests. - -| Key | Values | Description | Capability| -| -------- | -----| ------------ | ---------| -| build | Puppeteer Sample Build | Represent the build number for your test | `const capability = { "LT:Options": {"build": "",}}` -| name | Puppeteer Sample Test | Represents the name of a test | `const capability = { "LT:Options": {"name": "",}}` -| projectName | Example: My Test | Represent the name of your project | `const capability = { "LT:Options": {"projectName": "",}}` | -| tags | ["tag1", "tag2", "tag3"] | Group your Puppeteer tests |``const capability = {"LT:Options": { "tags": ["tag1", "tag2", "tag3"], }}`` | -| buildTags | ["build1", "build2", "build3"] | Group your Puppeteer builds |`const capability = {"LT:Options": { "buildTags": ["build1", "build2", "build3"] }}` | - -Shown below is the script that configure the `build` and `name` capabilities. - -```js -# add test code after initializing your browser -'use strict'; -const { strict } = require('once'); -const puppeteer = require('puppeteer'); -const expect = require('chai').expect; - -(async () => { -const capabilities = { -'browserName': 'Chrome', -'browserVersion': 'latest', -'LT:Options': { -'platform': 'Windows 10', -'build': 'puppeteer-build-1', -'name': 'My first Puppeteer test', -'resolution':'1366x768', -'user': process.env.LT_USERNAME || "LT_USERNAME", -'accessKey': process.env.LT_ACCESS_KEY || "LT_ACCESS_KEY", -'network': true -} -}; - -try { -const browser = await puppeteer.connect({ -browserWSEndpoint: -`wss://cdp.lambdatest.com/puppeteer?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}`, -}); - -const page = await browser.newPage(); -await page.setViewport({ -width: 1024, -height: 768, -deviceScaleFactor: 1, -}); -console.log("Navigating to LambdaTest"); -await page.goto('https://www.lambdatest.com/'); -console.log("Navigating to Pricing"); -await page.goto('https://www.lambdatest.com/pricing'); -console.log("Navigating to Automation"); -await page.goto('https://www.lambdatest.com/automation-testing'); -console.log("Closing browser"); -await browser.close(); - -} catch (e) { -console.log("Error - ", e); -} -})(); -``` - -## Getting Session Details - -Each Puppeteer test generates a different log on TestMu AI. To get the information relevant to your test session, use the snippet provided below in your Puppeteer test scripts. - -```js -let response = await page.evaluate(_ => {}, `lambdatest_action: ${JSON.stringify({ action: 'getTestDetails' })}`) -console.log("Test details: ", JSON.parse(response).data); -``` - -Upon executing the script, you will get the details for the particular test session as a JSON response as shown below. - -```js -{ -data: { -test_id: 'RWA9R-AC4V8-GJRET-YHBO2', -build_id: 9415381, -name: 'Puppeteer Sample Test', -user_id: 1001150, -username: 'newuser', -test_type: 'puppeteer', -platform: 'win10', -browser: 'Chrome', -browser_version: '107.0', -status_ind: 'running', -session_id: 'RWA9R-AC4V8-GJRET-YHBO2', -build_name: 'Puppeteer Sample Build', -create_timestamp: '2022-12-07 10:53:16', -start_timestamp: '2022-12-07 10:53:16', -remark: 'running', -console_logs_url: 'https://api.lambdatest.com/automation/api/v1/sessions/RWA9R-AC4V8-GJRET-YHBO2/log/console', -network_logs_url: 'https://api.lambdatest.com/automation/api/v1/sessions/RWA9R-AC4V8-GJRET-YHBO2/log/network', -command_logs_url: 'https://api.lambdatest.com/automation/api/v1/sessions/RWA9R-AC4V8-GJRET-YHBO2/log/command', -video_url: 'https://automation.lambdatest.com/public/video?testID=RWA9R-AC4V8-GJRET-YHBO2&auth=50471585dd55293d204fe87f63400f5f', -screenshot_url: 'https://video-bundler.lambdatest.com/RWA9R-AC4V8-GJRET-YHBO2/screenshots.zip?orgId=588659&createTimestamp=1670410396&testCreateTimestamp=2022-12-07%2010:53:16' -}, -message: 'Retrieve session was successful', -status: 'success' -} -``` - -## Changing Desktop Resolutions - -Puppeteer tests run with a `1920x1080` resolution by default for desktop browsers. With our `resolution` capability, you can set a different screen resolution for your tests. - -| Capability | Description | Expected Values | Example | -| -------- | -----| ------------ | -----------------------------| -| resolution | Specifying your desktop resolution before initiating the test | String, **Default value**: 1920x1080 **Windows 11 & 10**: `1024x768, 1280x800, 1280x1024, 1366x768, 1440x900, 1680x1050, 1600x1200, 1920x1200, 1920x1080 and 2048x1536` **macOS**: `1024x768, 1280x960, 1280x1024, 1600x1200 and 1920x1080` | const caps = {'resolution': '1024x768',}; | - -## Changing Browser Window Size - -If you wish to modify the browser window size during your Puppeteer test, you can do it as shown in the code below. - -```js - -await page.setViewport({ -width: 1024, -height: 768, -deviceScaleFactor: 1, -}); -``` - -## Naming your Project - -You can give your project a name of your choice by using the `projectName` key. - -| Key | Values | Description | Desired Capability | -| -------- | -----| ------------ | --------------| -| projectName | Example: My Test | Represent the name of your project | `const capabilities = { 'LT:Options': {'projectName': 'My Test',}}` | - -## Debugging Tests - -By specifying the capabilities for the debugging tools, you can debug and fix your failed Puppeteer test sessions using network logs, console logs, and video logs. - -| Key | Values | Description | Desired Capability | -| -------- | -----| ------------ | --------------| -| network | true/false | Enable network logs | `const capabilities = { 'LT:Options': {'network': true,}}` | -| console | true/false | Enable browser console logs | `const capabilities = { 'LT:Options': {'console': true,}}` | -| video | true/false | Enable Video recording of the entire screen | `const capabilities = { 'LT:Options': {'video': true,}}` | - -## Mark Tests As Passed Or Failed - -While running Puppeteer tests on the TestMu AI platform, you may come across a scenario in which a test that failed in your local instance turns up to be successful on TestMu AI. For verifying expected behavior, it is critical to identify automated tests as **Passed** or **Failed** based on your testing requirements. - -By default, the Status of each test that runs successfully is marked as **Completed**, and if there are any issues, the Status is marked as **Failed**. - -Shown below is syntax how to mark Puppeteer tests as **Passed** or **Failed**. - -1. To mark test status as **passed**. - -``` -await page.evaluate(_ => {}, `lambdatest_action: ${JSON.stringify({ action: 'setTestStatus', arguments: { status:'passed', remark: 'Title matched' } })}`) -``` - -2. To mark test status as **failed**. - -``` -await page.evaluate(_ => {}, `lambdatest_action: ${JSON.stringify({ action: 'setTestStatus', arguments: { status:'failed', remark: 'Title not matched' } })}`) -``` - -## Capabilities Reference - -The TestMu AI Capability Generator can auto-create the capabilities class for your Puppeteer scripts. The full set of capabilities you can configure: - -| Key | Expected Values | Description | Capability | -| -------- | -----| ------- | ----------------- | -| browserName | Chrome, Edge | Specify the browser to test on | `const capability = {"browserName": "Chrome"}` -| browserVersion | Chrome 83 & above, Edge 83 & above | Specify the browser version to test on | `const capability = {"browserVersion": "113.0"}` -| platform | **Windows**: 11, 10, 8, 8.1, 7 **macOS**: Monterey, Big Sur, Catiline, Mojave | Specify the platform name | `const capability = { "LT:Options": {"platform": "Windows 10",}}` -| build | Puppeteer Sample Build | Represent the build number for your test | `const capability = { "LT:Options": {"build": "",}}` -| name | Puppeteer Sample Test | Represents the name of a test | `const capability = { "LT:Options": {"name": "",}}` -| resolution | Specifying your desktop resolution before initiating the test | String, **Default value**: 1920x1080 **Windows 11 & 10**: `1024x768, 1280x800, 1280x1024, 1366x768, 1440x900, 1680x1050, 1600x1200, 1920x1200, 1920x1080 and 2048x1536` **macOS**: `1024x768, 1280x960, 1280x1024, 1600x1200 and 1920x1080` | `const capability = {"LT:Options": {"resolution": '1024x768'}}` | -| projectName | Example: My Test | Represent the name of your project | `const capability = { "LT:Options": {"projectName": "",}}` | -| tags | ["tag1", "tag2", "tag3"] | Group your Puppeteer tests |``const capability = {"LT:Options": { "tags": ["tag1", "tag2", "tag3"], }}`` | -| buildTags | ["build1", "build2", "build3"] | Group your Puppeteer builds |`const capability = {"LT:Options": { "buildTags": ["build1", "build2", "build3"] }}` | -| network | true/false | Enable network logs | `const capability = { "LT:Options": {"network": true,}}` | -| console | true/false | Enable browser console logs | `const capabilities = { "LT:Options": {"console": true,}}` | -| video | true/false | Enable video recording of the entire screen | `const capability = { "LT:Options": {"video": true,}}` | -| tunnel | true/false | Enable tunnel for local testing | `const capability = { "LT:Options": {"tunnel": true,}}` | -| tunnelName | true/false | Specify tunnel name | `const capability = { "LT:Options": {"tunnelName": "",}}` | -| geoLocation | AR (Argentina) | Specify country code | `const capability = { "LT:Options": {"geoLocation": "AR",}}` | -| idleTimeout | number| Specifies the timeout of the commands in seconds. Default value: 300 Max value: 1800 If a value greater than 1800 is added, idleTimeout will be set to 1800.| `const capability = { "LT:Options": {"idleTimeout": "",}}`| diff --git a/static/docs/puppeteer-testing-with-codecept.md b/static/docs/puppeteer-testing-with-codecept.md deleted file mode 100644 index eb5156d85..000000000 --- a/static/docs/puppeteer-testing-with-codecept.md +++ /dev/null @@ -1,130 +0,0 @@ -# Puppeteer Testing With CodeceptJS - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Learn how to run your Puppeteer tests with CodeceptJS across 40+ real browsers and operating systems on the TestMu AI platform. - -## Prerequisites - -1. You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** -Download or clone the code sample for the Puppeteer Codecept from the TestMu AI GitHub repository to run the tests. - - View on GitHub - -```js -git clone https://github.com/LambdaTest/puppeteer-sample.git -cd puppeteer-sample -cd puppeteer-codecept - -``` - -2. Install the npm dependencies. - -``` -npm install -``` - -3. If you are running Codecept for the first time, run the below command: - -``` -npm install codeceptjs puppeteer --save -``` - -4. To run Puppeteer tests with CodeceptJS, set your TestMu AI username and access key in the environment variables. Click the **Access Key** button at the top-right of the Automation Dashboard to access it. - -**Windows** - -```js -set LT_USERNAME="YOUR_LAMBDATEST_USERNAME" -set LT_ACCESS_KEY="YOUR_LAMBDATEST_ACCESS_KEY" -``` - -**macOS/Linux** - -```js -export LT_USERNAME="YOUR_LAMBDATEST_USERNAME" -export LT_ACCESS_KEY="YOUR_LAMBDATEST_ACCESS_KEY" -``` - -## Running Tests Using CodeceptJS - ->**Test Scenario**: The below test script searches TestMu AI on DuckDuckGo and verifies the website title. - -1. Navigate to the `codecept.conf.js` file in the `puppeteer-codecept` directory. - -```js -const { setHeadlessWhen } = require('@codeceptjs/configure'); - -// turn on headless mode when running with HEADLESS=true environment variable -// export HEADLESS=true && npx codeceptjs run -setHeadlessWhen(process.env.HEADLESS); - -const caps_chrome = { -browserName : 'Chrome', -browserVersion : 'latest', -'LT:Options' : { -platform : 'Windows 10', -build : 'Sample Puppeteer-Codecept', -name : 'Puppeteer-Codecept test on Chrome', -resolution : '1366x768', -user : process.env.LT_USERNAME, -accessKey : process.env.LT_USER_KEY, -network : true -} -}; - -const caps_edge = { -browserName : 'MicrosoftEdge', -browserVersion : 'latest', -'LT:Options' : { -platform : 'Windows 10', -build : 'Sample Puppeteer-Codecept', -name : 'Puppeteer-Codecept test on Edge', -resolution : '1366x768', -user : process.env.LT_USERNAME, -accessKey : process.env.LT_USER_KEY, -network : true -} -}; - -exports.config = { -tests: './specs/*.spec.js', -output: './output', -helpers: { -Puppeteer: { -chrome: { -browserWSEndpoint : `wss://cdp.lambdatest.com/puppeteer?capabilities=${encodeURIComponent(JSON.stringify(caps_chrome))}`, -"ignoreHTTPSErrors": true -} -} -}, -include: { -I: './steps_file.js' -}, -bootstrap: null, -mocha: {}, -name: 'codeceptjs-example', -plugins: { -pauseOnFail: {}, -retryFailedStep: { -enabled: true -}, -tryTo: { -enabled: true -}, -screenshotOnFail: { -enabled: true -} -} -} -``` - -2. Now pass the below command to run your test. - -``` -npx codeceptjs run -``` - -3. Visit the [TestMu AI Web Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) to see your test results. diff --git a/static/docs/puppeteer-testing-with-jest.md b/static/docs/puppeteer-testing-with-jest.md deleted file mode 100644 index 5bcc98ceb..000000000 --- a/static/docs/puppeteer-testing-with-jest.md +++ /dev/null @@ -1,129 +0,0 @@ -# Puppeteer Testing With Jest - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Jest is a Facebook-maintained framework for developing frontend and backend unit tests in JavaScript and executing end-to-end testing using Playwright. It provides you with the flexibility to target a single test, delivering structured output, or an entire suite for common assertions. - -TestMu AI allows you to run Puppeteer tests with Jest on a browser farm of 40+ real browsers and operating system combinations. - -This guide will cover the basics of getting started with Puppeteer testing with Jest on the TestMu AI platform. - -## Prerequisites - ->Note: All the code samples in this documentation can be found in the TestMu AI's Repository on GitHub. You can either download or clone the repository to quickly run your tests. - View on GitHub - -1. Clone the TestMu AI-Puppeteer repository on your system. - -2. Install the npm dependencies. - -``` -npm install -``` - -3. In order to run your Puppeteer tests with Jest, you will need to set your TestMu AI username and access key in the environment variables. Click the **Access Key** button at the top-right of the Automation Dashboard to access it. - -**Windows** - -```js -set LT_USERNAME="YOUR_LAMBDATEST_USERNAME" -set LT_ACCESS_KEY="YOUR_LAMBDATEST_ACCESS_KEY" -``` - -**macOS/Linux** - -```js -export LT_USERNAME="YOUR_LAMBDATEST_USERNAME" -export LT_ACCESS_KEY="YOUR_LAMBDATEST_ACCESS_KEY" -``` - -## Running Tests Using Jest - ->**Test Scenario**: The below test script searches TestMu AI on DuckDuckGo and verifies the website title. - -1. To run the Puppeteer tests using Jest on TestMu AI, you need make some tweaks to the `jest-puppeteer.config.js` file. - -```js -const caps_chrome = { -browserName : 'Chrome', -browserVersion : 'latest', -'LT:Options' : { -platform : 'Windows 10', -build : 'Sample Puppeteer-Jest', -name : 'Puppeteer-jest test on Chrome', -resolution : '1366x768', -user : process.env.LT_USERNAME, -accessKey : process.env.LT_USER_KEY, -network : true -} -}; - -const caps_edge = { -browserName : 'MicrosoftEdge', -browserVersion : 'latest', -'LT:Options' : { -platform : 'Windows 10', -build : 'Sample Puppeteer-Jest', -name : 'Puppeteer-jest test on Edge', -resolution : '1366x768', -user : process.env.LT_USERNAME, -accessKey : process.env.LT_USER_KEY, -network : true -} -}; - -module.exports = { -connect : { -browserWSEndpoint : `wss://cdp.lambdatest.com/puppeteer?capabilities=${encodeURIComponent( -JSON.stringify(caps_chrome) -)}` -} -}; - -``` - -2. Pass the below command to run the test. - -``` -npm run test -``` - -3. Visit the TestMu AI Automation Dashboard to see the results of your Puppeteer Jest tests. - -## Using the Jest Agent Skill with TestMu AI - -The [jest-skill](https://github.com/LambdaTest/agent-skills/tree/main/jest-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The jest-skill package includes: - -``` -jest-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing Jest Agent Skill - -Install a Jest Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/jest-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/jest-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only jest-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). diff --git a/static/docs/puppeteer-testing-with-mocha.md b/static/docs/puppeteer-testing-with-mocha.md deleted file mode 100644 index 5d7da2d42..000000000 --- a/static/docs/puppeteer-testing-with-mocha.md +++ /dev/null @@ -1,170 +0,0 @@ -# Puppeteer Testing With Mocha - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Mocha is a JavaScript automation testing framework to run the tests on Node.js and the browser. It enables you to run test cases serially, providing greater flexibility and accurate test reporting while mapping unidentified bugs to the correct test cases. - -TestMu AI allows you to run Puppeteer tests with Mocha on a browser farm of 40+ real browsers and operating system combinations. - -This guide will cover the basics of getting started with Puppeteer testing with Mocha on the TestMu AI platform. - -## Prerequisites - ->Note: All the code samples in this documentation can be found in the TestMu AI's Repository on GitHub. You can either download or clone the repository to quickly run your tests. - View on GitHub - -1. Clone the TestMu AI-Puppeteer repository on your system. - -2. Install the npm dependencies. - -``` -npm install -``` - -3. In order to run your Puppeteer tests with Mocha, you will need to set your TestMu AI username and access key in the environment variables. Click the **Access Key** button at the top-right of the Automation Dashboard to access it. - -**Windows** - -```js -set LT_USERNAME="YOUR_LAMBDATEST_USERNAME" -set LT_ACCESS_KEY="YOUR_LAMBDATEST_ACCESS_KEY" -``` - -**macOS/Linux** - -```js -export LT_USERNAME="YOUR_LAMBDATEST_USERNAME" -export LT_ACCESS_KEY="YOUR_LAMBDATEST_ACCESS_KEY" -``` - -## Running Tests Using Mocha - ->**Test Scenario**: The below test script searches TestMu AI on DuckDuckGo and verifies the website title. - -1. To run the Puppeteer Mocha tests on TestMu AI, you need make some tweaks to the `google.spec.js` file. - -```js -const puppeteer = require('puppeteer'); -const expect = require('chai').expect; -const caps_chrome = { -browserName : 'Chrome', -browserVersion : 'latest', -'LT:Options' : { -platform : 'Windows 10', -build : 'Sample Puppeteer-Mocha', -name : 'Puppeteer-mocha test on Chrome', -resolution : '1366x768', -user : process.env.LT_USERNAME, -accessKey : process.env.LT_ACCESS_KEY, -network : true -} -}; -const caps_edge = { -browserName : 'MicrosoftEdge', -browserVersion : 'latest', -'LT:Options' : { -platform : 'Windows 10', -build : 'Sample Puppeteer-Mocha', -name : 'Puppeteer-mocha test on Edge', -resolution : '1366x768', -user : process.env.LT_USERNAME, -accessKey : process.env.LT_ACCESS_KEY, -network : true -} -}; - -let browser = null; -let page = null; -describe('Search Text', () => { -beforeEach(async () => { -browser = await puppeteer.connect({ -browserWSEndpoint : `wss://cdp.lambdatest.com/puppeteer?capabilities=${encodeURIComponent( -JSON.stringify(caps_chrome) -)}` -}); -page = await browser.newPage(); -}); - -it('should be titled "Google"', async () => { -let text = 'Google'; -await page.goto('https://www.duckduckgo.com'); -var element = await page.$('[name="q"]'); -await element.click(); -await element.type(text); -await Promise.all([ -page.keyboard.press('Enter'), -page.waitForNavigation() -]); -var title = await page.title(); -try { -expect(title).equal(text + ' at DuckDuckGo', 'Expected page title is incorrect!'); -await page.evaluate( -(_) => {}, -`lambdatest_action: ${JSON.stringify({ -action : 'setTestStatus', -arguments : { status: 'passed', remark: 'assertion passed' } -})}` -); -} catch (e) { -await page.evaluate( -(_) => {}, -`lambdatest_action: ${JSON.stringify({ -action : 'setTestStatus', -arguments : { status: 'failed', remark: e.name } -})}` -); -} -}); - -afterEach(async () => { -await page.close(); -await browser.close(); -}); -}); -``` - -2. Now pass the below command to run your test. - -``` -npm run test -``` - -3. Visit the TestMu AI Automation Dashboard to see the results of your Puppeteer Mocha tests. - -## Using the Mocha Agent Skill with TestMu AI - -The [mocha-skill](https://github.com/LambdaTest/agent-skills/tree/main/mocha-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The mocha-skill package includes: - -``` -mocha-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing Mocha Agent Skill - -Install a Mocha Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/mocha-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/mocha-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only mocha-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). diff --git a/static/docs/puppeteer-testing-with-pytest-pyppeteer.md b/static/docs/puppeteer-testing-with-pytest-pyppeteer.md deleted file mode 100644 index f6bb3d3aa..000000000 --- a/static/docs/puppeteer-testing-with-pytest-pyppeteer.md +++ /dev/null @@ -1,128 +0,0 @@ -# Puppeteer Testing With CodeceptJS - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Learn how to run your Puppeteer tests with CodeceptJS across 40+ real browsers and operating systems on the TestMu AI platform. - -## Prerequisites - ->Note: All the code samples in this documentation can be found in the TestMu AI's Repository on GitHub. You can either download or clone the repository to quickly run your tests. - View on GitHub - -1. Clone the puppeteer-sample repository on your system and navigate to the `pytest-pyppeteer` directory. -``` -cd pytest-pyppeteer -``` - -2. Create a virtual environment using the following commands: - -``` -virtualenv venv -``` - -``` -source venv/bin/activate -``` - -3. Install the necessary configurations. - -``` -poetry install -``` - -4. Install the necessary dependencies - -``` -pip install - r requirements.txt -``` - -5. In order to run your pyppeteer tests, you will need to set your TestMu AI username and access key in the environment variables. Click the **Access Key** button at the top-right of the Automation Dashboard to access it. - -##### Windows - -```sh -set LT_USERNAME="YOUR_LAMBDATEST_USERNAME" -set LT_ACCESS_KEY="YOUR_LAMBDATEST_ACCESS_KEY" -``` - -##### macOS/Linux - -```sh -export LT_USERNAME="YOUR_LAMBDATEST_USERNAME" -export LT_ACCESS_KEY="YOUR_LAMBDATEST_ACCESS_KEY" -``` - -## Running Your First Pyppeteer Test - -The first test script navigates to DuckduckGo and searches for TestMu AI. The second test script navigates to Brave search and searches for TestMu AI. Both the tests are executed using Chrome (latest) on Windows 11. - -Once you are done with the steps 1 thru' 5, you can initiate your first Pyppeteer test on TestMu AI. - -Run the following command on the terminal to run the Pyppeteer tests in parallel. - -``` -pytest --verbose --capture=no -s -n 2 tests/test_pytest_pyppeteer_1.py \ -tests/test_pytest_pyppeteer_2.py -``` - -## View your Pyppeteer test results - -The [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) is where you can see the results of your Pyppeteer tests after running them on the TestMu AI platform. - -The below screenshot of TestMu AI Automation Dashboard shows the pyppeteer build on the left and the build sessions associated with the selected build on the right. - -On clicking the session name of the respective test, you can view the details of pyppeteer test session that you just executed. For example, the below screenshot shows a test execution details of pyppeteer test like Test Name, Test ID, selected configurations, test logs, basic info, input config, and test session video. - -## Run Pyppeteer Tests In Gitpod - -Select the button below to try this demo in [Gitpod](https://www.gitpod.io/) - -* After the Gitpod session launches, navigate to the terminal and run the following commands to save your [TestMu AI Credentials](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile) to Gitpod as environment variables: - -``` -eval $(gp env -e LT_USERNAME=******) -eval $(gp env -e LT_ACCESS_KEY=******) -``` - -* Click the following link if you're unsure how to [access your TestMu AI credentials.](/support/docs/using-environment-variables-for-authentication-credentials/). Also, if you start a new terminal in Gitpod, you have to run the following command to reset environment variables: -``` -eval $(gp env -e) -``` - -## Using the Pytest Agent Skill with TestMu AI - -The [pytest-skill](https://github.com/LambdaTest/agent-skills/tree/main/pytest-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The pytest-skill package includes: - -``` -pytest-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing Pytest Agent Skill - -Install a Pytest Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/pytest-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/pytest-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only pytest-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). diff --git a/static/docs/puppeteer-testing.md b/static/docs/puppeteer-testing.md deleted file mode 100644 index 0a6bdb2b1..000000000 --- a/static/docs/puppeteer-testing.md +++ /dev/null @@ -1,123 +0,0 @@ -# Getting Started With Puppeteer Testing - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Puppeteer is a Node package that automates the Chrome browser for web testing. Over the DevTools Protocol, it provides a high-level API for controlling headless Chrome or Chromium. It may also be set to run full (non-headless) Chrome or Chromium. - -TestMu AI allows you to run Puppeteer tests on a browser farm of 40+ real browsers and operating system combinations. This guide will cover the basics of getting started with Puppeteer testing on the TestMu AI platform. - -## Prerequisites - ->Note: All the code samples in this documentation can be found in the TestMu AI's Repository on GitHub. You can either download or clone the repository to quickly run your tests. - View on GitHub - -1. Clone the TestMu AI-Puppeteer repository on your system. - -2. Install the npm dependencies. - -``` -npm install -``` - -3. Add browserWSEndpoint (browser end point URL) in your test script. - -```js -`wss://cdp.lambdatest.com/puppeteer?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}` -``` - -4. In order to run your Puppeteer tests, you will need to set your TestMu AI username and access key in the environment variables. Click the **Access Key** button at the top-right of the Automation Dashboard to access it. - -**Windows** - -```js -set LT_USERNAME="YOUR_LAMBDATEST_USERNAME" -set LT_ACCESS_KEY="YOUR_LAMBDATEST_ACCESS_KEY" -``` - -**macOS/Linux** - -```js -export LT_USERNAME="YOUR_LAMBDATEST_USERNAME" -export LT_ACCESS_KEY="YOUR_LAMBDATEST_ACCESS_KEY" -``` - -## Run Your First Puppeteer Test - -Shown below are the steps on running Puppeteer tests on the TestMu AI platform. - -1. Clone the [TestMu AI-Puppeteer GitHub repository](https://github.com/LambdaTest/puppeteer-sample) and switch to the cloned directory. - -```js -git clone https://github.com/LambdaTest/puppeteer-sample.git -cd puppeteer-sample -``` - -2. Ensure you have npm dependencies installed. - -3. Configure your TestMu AI authentication credentials. - -Once you are done with the above-mentioned steps, you can initiate your first Puppeteer test on TestMu AI. - ->**Test Scenario**: The below test script runs on Chrome browser running Windows 10. It visits the TestMu AI platform, clicks on the Pricing page, and navigates to the Automation Testing page. - -```js -'use strict'; -const { strict } = require('once'); -const puppeteer = require('puppeteer'); -const expect = require('chai').expect; - -(async () => { -const capabilities = { -'browserName': 'Chrome', -'browserVersion': 'latest', -'LT:Options': { -'platform': 'Windows 10', -'build': 'puppeteer-build-1', -'name': 'My first Puppeteer test', -'resolution':'1366x768', -'user': process.env.LT_USERNAME || "Your Username", -'accessKey': process.env.LT_ACCESS_KEY || "Your Access Key",, -'network': true -} -}; - -try { -const browser = await puppeteer.connect({ -browserWSEndpoint: -`wss://cdp.lambdatest.com/puppeteer?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}`, -}); - -const page = await browser.newPage(); -await page.setViewport({ -width: 1024, -height: 768, -deviceScaleFactor: 1, -}); -console.log("Navigating to LambdaTest"); -await page.goto('https://www.lambdatest.com/'); -console.log("Navigating to Pricing"); -await page.goto('https://www.lambdatest.com/pricing'); -console.log("Navigating to Automation"); -await page.goto('https://www.lambdatest.com/automation-testing'); -console.log("Closing browser"); -await browser.close(); - -} catch (e) { -console.log("Error - ", e); -} -})(); -``` - -4. Pass the below command to run the test. - -``` -node navigation.js -``` - -## View your Puppeteer test results - -The TestMu AI Automation Dashboard is where you can see the results of your Puppeteer tests after running them on the TestMu AI platform. - -The below screenshot of TestMu AI Automation Dashboard shows the Puppeteer build on the left and the build sessions associated with the selected build on the right. - -On clicking the session name of the respective test, you can view the details of Puppeteer test session that you just executed. For example, the below screenshot shows a test execution details of Puppeteer test like Test Name, Test ID, selected configurations, test logs, basic info, input config, and test session video. diff --git a/static/docs/puppeteer-tests-in-ci-cd.md b/static/docs/puppeteer-tests-in-ci-cd.md deleted file mode 100644 index 97171fcb3..000000000 --- a/static/docs/puppeteer-tests-in-ci-cd.md +++ /dev/null @@ -1,61 +0,0 @@ -# Puppeteer Testing With CI/CD - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI integrates with a variety of CI/CD solutions to help you pace up your go-to-market launch by automating your delivery pipeline. Similarly, you can run Puppeteer tests in CI/CD across 40+ browsers in the cloud, reducing lead time by 10X and shipping quality builds faster. - -Running Puppeteer tests through CI/CD pipeline involves three stages: - -1. **Build**: It consists of developing an web app (or code) and hosting it on a staging environment for testing. -2. **Test**: It entails running Puppeteer tests on the TestMu AI platform. -3. **Deploy**: Based on your CI/CD configuration, the deploy stage may or may not present. - -## Running Puppeteer Tests Via CI/CD - -TestMu AI offers [integration with CI/CD tools](/support/docs/integrations-with-ci-cd-tools/) like Jenkins, Bamboo, AWS CodePipeline, and more. In order to run Puppeteer tests in CI/CD, you need to perform few steps in the **Test** stage of your CI/CD pipeline. - -1. Ensure the website (or code) is hosted on staging environment. - -2. Import test data in existing databases in order to initiate the tests. - -3. If the website under test in hosted on local server, then you will need to configure the TestMu AI tunnel by setting `tunnel` capability to `true` for running local tests via CI/CD on the TestMu AI. However, if the staging website is publicly accessible then this step is not required. - -4. Ensure your Puppeteer test scripts uses `puppeteer.connect` method to connect to the CDP endpoint at TestMu AI. Additional parameters for assigning a specific browser and OS combination to your TestMu AI test are contained in the `capabilities` variable. - -## Using the Puppeteer Agent Skill with TestMu AI - -The [puppeteer-skill](https://github.com/LambdaTest/agent-skills/tree/main/puppeteer-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The puppeteer-skill package includes: - -``` -puppeteer-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing Puppeteer Agent Skill - -Install a Puppeteer Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/puppeteer-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/puppeteer-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only puppeteer-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). diff --git a/static/docs/puppeteer-visual-regression.md b/static/docs/puppeteer-visual-regression.md deleted file mode 100644 index ad9b6823c..000000000 --- a/static/docs/puppeteer-visual-regression.md +++ /dev/null @@ -1,131 +0,0 @@ -# Getting started with SmartUI using Puppeteer - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This documentation will act as your step-by-step guide in to perform Puppteer test with SmartUI. - -### Prerequisites for running SmartUI with Puppeteer - -- Basic understanding of Puppeteer is required. -- Go to [SmartUI](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) and login along with your credentials. -- Copy `LT_USERNAME` and `LT_ACCESS_KEY` credentials from `Access Key` button on the top right of the dashboard. - -```bash -export LT_USERNAME="YOUR_USERNAME" -``` - -```bash -set LT_USERNAME="YOUR_USERNAME" -``` - -```powershell -$env:LT_USERNAME="YOUR_USERNAME" -``` - -The following steps will guide you in running your first Visual Regression test on TestMu AI platform - - -### **Step 1:** Create a SmartUI Project - -The first step is to create a project with the application in which we will combine all your **builds** run on the project. -To create a SmartUI Project, follow these steps: - -1. Go to [Projects page](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) -2. Click on the `new project` button -3. Select the platform as Web for executing your `Puppeteer` tests. -4. Add name of the project, approvers for the changes found, tags for any filter or easy navigation. -5. Click on the **Submit**. - -### **Step 2:** Configure your test with Puppeteer Desired Capabilities - -Once you have created a SmartUI Project, you can generate screenshots by running automation scripts. Follow the below steps to successfully generate screenshots - - -1. Please clone the following sample GitHub repo (`https://github.com/LambdaTest/puppeteer-sample`). - -```bash -git clone https://github.com/LambdaTest/puppeteer-sample.git -``` - -2. Install the node modules using the command: - -```bash -npm i -``` - -3. Set up the TestMu AI credentials by following the instructions mentioned in the `README.md` file. -4. Edit the required capabilities in your test file `navigation.js`. - -```javascript title="Add the following code snippet to run SmartUI with Puppeteer in ./navigation.js" -"use strict"; -const { strict } = require("once"); -const puppeteer = require("puppeteer"); -const expect = require("chai").expect; - -(async () => { -const capabilities = { -browserName: "Chrome", -browserVersion: "latest", -"LT:Options": { -platform: "Windows 10", -build: "puppeteer-build-1", -name: "My first Puppeteer test", -resolution: "1366x768", -user: process.env.LT_USERNAME || "Your Username", -accessKey: process.env.LT_ACCESS_KEY || "Your Access Key", -network: true, -smartUIProjectName: "Testing Puppeteer Connection", // Add your SmartUI Project Name here -smartUIBuildName: "My First Build", // Replace with your build name of choice here -// smartUIBaseline: false, // (Optional) To set your current build as baseline to compare -}, -}; - -try { -const browser = await puppeteer.connect({ -browserWSEndpoint: `wss://cdp.lambdatest.com/puppeteer?capabilities=${encodeURIComponent( -JSON.stringify(capabilities) -)}`, -}); - -const page = await browser.newPage(); -await page.setViewport({ -width: 1024, -height: 768, -deviceScaleFactor: 1, -}); -console.log("Navigating to LambdaTest"); -await page.goto("https://www.lambdatest.com/"); -await page.evaluate((_) => {}, -`lambdatest_action: ${JSON.stringify({ action: "smartui.takeScreenshot", arguments: { fullPage: true, screenshotName: "Navigating to LambdaTest" } })}`); -console.log("Navigating to Pricing"); -await page.goto("https://www.lambdatest.com/pricing"); -await page.evaluate((_) => {}, -`lambdatest_action: ${JSON.stringify({ action: "smartui.takeScreenshot", arguments: { fullPage: true, screenshotName: "Navigating to Pricing" } })}`); -console.log("Navigating to Automation"); - -await page.goto("https://www.lambdatest.com/automation-testing"); -await page.evaluate((_) => {}, -`lambdatest_action: ${JSON.stringify({ action: "smartui.takeScreenshot", arguments: { fullPage: true, screenshotName: "Navigating to Automation" } })}`); -console.log("Closing browser"); -await browser.close(); -} catch (e) { -console.log("Error - ", e); -} -})(); -``` - -### **Step 3:** Executing the SmartUI Test Suite on Cloud - -Execute the test using the following command to run the test suite using `puppeteer` - -```bash -node navigation.js -``` - -- You can check the executed builds over at [TestMu AI SmartUI](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/). - -For additional information about Puppteer framework please explore the documentation [here](/docs/puppeteer-testing/) - -## Advanced Options for Screenshot Comparison - - **Build Configuration** - If you have multiple screenshots running the same test suite and want to run the comparison for the same test suite, want to add a build as a baseline from your test suite or need to access more SmartUI Build Config Options, click [here](/support/docs/smart-ui-build-options/). - - **Handling Dynamic Data** - In case if you have any dynamic elements that are not in the same position across test runs, you can ignore or select a specific area to be removed from the comparison. For accessing such HTML DOM Config and Options, see [Handling Dynamic Data with DOM Configuration](/support/docs/html-dom-smartui-options). diff --git a/static/docs/pytest-on-hyperexecute-grid.md b/static/docs/pytest-on-hyperexecute-grid.md deleted file mode 100644 index e2e49635c..000000000 --- a/static/docs/pytest-on-hyperexecute-grid.md +++ /dev/null @@ -1,236 +0,0 @@ -# Running PyTest Framework Tests on HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -PyTest is a testing framework for Python that is widely used with Selenium for automated testing. Known for its simplicity and powerful features, Pytest allows efficient test organization, parameterization, and fixtures. - -HyperExecute is an AI Native Test Orchestration Cloud Platform that empowers you to run **end-to-end** tests **quickly** and **efficiently**. It provides Just-in-Time (JIT) testing infrastructure with fast execution **speeds**, **smart orchestration**, and **detailed logs**. - -This guide details how to execute your **PyTest** framework tests on **HyperExecute** via two different methods: - -- [**Using Local System**](/support/docs/pytest-on-hyperexecute-grid/#1-testing-using-local-system) - You can use your own local machine to execute tests. -- [**Using Gitpod Platform**](/support/docs/pytest-on-hyperexecute-grid/#2-testing-using-gitpod) - Execute tests using GitPod. (Requires a [Gitpod](https://gitpod.io/login/) account) - -## 1. Testing Using Local System - -Follow the step-by-step guide to execute your test on HyperExecute. - -### Prerequisites - -To run the Tests on HyperExecute from your Local System, you are required: - -- Your TestMu AI [Username and Access key](/support/docs/hyperexecute-how-to-get-my-username-and-access-key/) -- [HyperExecute YAML](/support/docs/hyperexecute-yaml-version0.2/) file which contains all the necessary instructions. -- [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) in order to initiate a test execution Job . -- Setup the [Environmental Variable](/support/docs/hyperexecute-environment-variable-setup/) - -### Step 1: Configure Your Test Suite - -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** - -Download or Clone the code sample for the JUnit from the TestMu AI GitHub repository to run the tests on the HyperExecute. - - View on GitHub - -If you are using your own project, make sure you update the **Hub endpoint** in your tests file. - -By setting up the Hub endpoint, you establish the communication channel between your tests and the browser nodes, enabling effective test distribution and execution. - -Configure the desired capabilities based on your test requirements. For example: - -```bash -capabilities = { -"build": "Sample PY Build", -"platformName": "Windows 11", -"browserName": "Chrome", -"browserVersion": "latest", -} -``` - -> You can also use **dev** and **beta** browser versions. To generate capabilities for your test requirements, you can use our inbuilt 🔗 Capabilities Generator. - -### Step 2: Setup the CLI in your Test Suite - -After cloning / downloading the sample repo, you need to setup the CLI and the environment variables. - -#### Download the HyperExecute CLI - -The CLI is used for triggering the tests on HyperExecute. It is recommend to download the CLI binary on the host system and keep it in the root directory of the suite to perform the tests on HyperExecute. - -You can download the CLI for your desired platform from the below mentioned links: - -| Platform | HyperExecute CLI | -| ---------| ---------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| MacOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -#### Setup Environment Variable - -Now, you need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in the [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). - -Run the below mentioned commands in your terminal to setup the CLI and the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -### Step 3: Configure YAML in your Test Suite - -Configure your YAML file as per your use cases using **key value** pairs. - -In this sample YAML file, we have mentioned: - -- **version** of the YAML file -- **Timeouts** for executing your project -- **Mode of execution** is [Autosplit](/support/docs/hyperexecute-auto-split-strategy/). You can also opt for [Matrix](/support/docs/hyperexecute-matrix-multiplexing-strategy/) or [Hybrid](/support/docs/hyperexecute-hybrid-strategy/) mode. -- **Pre and Post** commands -- **Reports and Artefacts** that will be generated after the completion of tests -- and other necessary YAML Parameters - -```yaml ---- -version: 0.1 -globalTimeout: 90 -testSuiteTimeout: 90 -testSuiteStep: 90 - -runson: linux - -autosplit: true -retryOnFailure: true - -maxRetries: 1 -concurrency: 2 - -env: -# PAT: ${{ .secrets.testKey }} -TARGET_OS: LINUX - -cacheKey: '{{ checksum "requirements.txt" }}' -cacheDirectories: -- CacheDir -pre: -- pip install py -- pip install -r requirements.txt --cache-dir CacheDir -post: -- cat yaml/linux/pytest_hyperexecute_autosplit_sample.yaml - -runtime: -language: python -version: "3" - -mergeArtifacts: true -uploadArtefacts: -- name: TestReports -path: -- reports/** - -report: true -partialReports: -type: json -location: reports/ -frameworkName: extent - -testDiscovery: -type: raw -mode: remote -command: grep -nri 'class' tests -ir --include=\*.py | sed 's/:.*//' - -testRunnerCommand: pytest -s --verbose --html=reports/report.html $test - -jobLabel: [selenium-pytest, linux, autosplit] -``` - -### Step 4: Execute your Test Suite - -> **NOTE :** In case of MacOS, if you get a permission denied warning while executing CLI, simply run **`chmod u+x ./hyperexecute`** to allow permission. In case you get a security popup, allow it from your **System Preferences** → **Security & Privacy** → **General tab**. - -Run the below command in your terminal at the root folder of the project: - -```bash -./hyperexecute --config RELATIVE_PATH_OF_YOUR_YAML_FILE -``` - -OR use this command if you have not exported your username and access key in the step 2. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE `} - -### Step 5: Monitor the Test Execution - -Visit the [HyperExecute Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) and check your Job status. - -### Step 6: Download Artifacts and Reports - -HyperExecute also facilitates the provision to download the [Artifacts](/support/docs/hyperexecute-artifacts/) and [Reports](/support/docs/hyperexecute-reports/) on your local machine. Click on the corresponding button to download your generated artifacts and reports. - -## 2. Testing Using Gitpod - -You can also use the Gitpod platform to execute our sample repository. It will fetch all the sample codebases and trigger the CLI to execute the tests. - -Follow the below steps to run Test using Gitpod: - -**Step 1:** Click '**Open in Gitpod**' button. You will be redirected to Login/Signup page. This button is configured to redirect you to the Gitpod platform where you will be able to execute our sample repository. - -**Step 2:** Login with TestMu AI credentials. Once logged in, a pop-up confirmation will appear, asking you to **'Proceed'** to the Gitpod editor in a new tab. The current tab will display the HyperExecute Dashboard. - -**Step 3:** Choose your preferred editor (we recommend VS Code Editor) - -**Step 4:** As you are running a sample project, Fetching of the Test Scripts, [HyperExecute YAML](/support/docs/deep-dive-into-hyperexecute-yaml/), [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) and Triggering your tests using the `Execution Command` will be automated. - -**Step 5:** Once you see the `Job Link` in the logs, you can visit the [HyperExecute dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) to see the tests getting executed. - -You can also implement [Secret Keys](/support/docs/hyperexecute-how-to-save-and-manage-secrets/) in your YAML file. - -## Navigation in Automation Dashboard - -Every test run on the HyperExecute has a unique *jobId* associated with it. Each *jobId* can in turn constitute single (or multiple) *groupId*(s). You can visit [HyperExecute Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) for checking the status of the test execution. - -You can seamlessly navigate between JobId's and taskId's. You need to click on the *testID* to navigate from the HyperExecute logs to the Automation Dashboard. - -The snapshot below shows the videos, logs and other meta data for that specific *test_ID* - -> For any query or doubt, please feel free to contact us via **24×7 chat support** or you can also drop a mail to **support@testmuai.com**. -Happy testing! - -## Using the Pytest Agent Skill with TestMu AI - -The [pytest-skill](https://github.com/LambdaTest/agent-skills/tree/main/pytest-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The pytest-skill package includes: - -``` -pytest-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing Pytest Agent Skill - -Install a Pytest Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/pytest-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/pytest-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only pytest-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). diff --git a/static/docs/pytest-on-hyperexecute.md b/static/docs/pytest-on-hyperexecute.md deleted file mode 100644 index b9f2ba34d..000000000 --- a/static/docs/pytest-on-hyperexecute.md +++ /dev/null @@ -1,213 +0,0 @@ -# Running PyTest Framework Tests on HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Pytest is a popular Python testing framework that can be seamlessly integrated with Playwright for efficient end-to-end testing of web applications. Pytest simplifies the testing process with its straightforward syntax and rich set of features. - -HyperExecute is an AI Native Test Orchestration Cloud Platform that empowers you to run **end-to-end** tests **quickly** and **efficiently**. It provides Just-in-Time (JIT) testing infrastructure with fast execution **speeds**, **smart orchestration**, and **detailed logs**. - -This guide details how to execute your **PyTest** framework tests on **HyperExecute** via two different methods: - -- [**Using Local System**](/support/docs/pytest-on-hyperexecute/#1-testing-using-local-system) - You can use your own local machine to execute tests. -- [**Using Gitpod Platform**](/support/docs/pytest-on-hyperexecute/#2-testing-using-gitpod) - Execute tests using GitPod. (Requires a [Gitpod](https://gitpod.io/login/) account) - -## 1. Testing Using Local System - -Follow the step-by-step guide to execute your test on HyperExecute. - -### Prerequisites - -To run the Tests on HyperExecute from your Local System, you are required: - -- Your TestMu AI [Username and Access key](/support/docs/hyperexecute-how-to-get-my-username-and-access-key/) -- [HyperExecute YAML](/support/docs/hyperexecute-yaml-version0.2/) file which contains all the necessary instructions. -- [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) in order to initiate a test execution Job . -- Setup the [Environmental Variable](/support/docs/hyperexecute-environment-variable-setup/) - -### Step 1: Configure Your Test Suite - -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** - -Download or Clone the code sample for the JUnit from the TestMu AI GitHub repository to run the tests on the HyperExecute. - - View on GitHub - -If you are using your own project, make sure you update the **Hub endpoint** in your tests file. - -By setting up the Hub endpoint, you establish the communication channel between your tests and the browser nodes, enabling effective test distribution and execution. - -> You can also use **dev** and **beta** browser versions. To generate capabilities for your test requirements, you can use our inbuilt 🔗 Capabilities Generator. - -### Step 2: Setup the CLI in your Test Suite - -After cloning / downloading the sample repo, you need to setup the CLI and the environment variables. - -#### Download the HyperExecute CLI - -The CLI is used for triggering the tests on HyperExecute. It is recommend to download the CLI binary on the host system and keep it in the root directory of the suite to perform the tests on HyperExecute. - -You can download the CLI for your desired platform from the below mentioned links: - -| Platform | HyperExecute CLI | -| ---------| ---------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| MacOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -#### Setup Environment Variable - -Now, you need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in the [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). - -Run the below mentioned commands in your terminal to setup the CLI and the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -### Step 3: Configure YAML in your Test Suite - -Configure your YAML file as per your use cases using **key value** pairs. - -In this sample YAML file, we have mentioned: - -- **version** of the YAML file -- **Timeouts** for executing your project -- **Mode of execution** is [Autosplit](/support/docs/hyperexecute-auto-split-strategy/). You can also opt for [Matrix](/support/docs/hyperexecute-matrix-multiplexing-strategy/) or [Hybrid](/support/docs/hyperexecute-hybrid-strategy/) mode. -- **Pre and Post** commands -- **Reports and Artefacts** that will be generated after the completion of tests -- and other necessary YAML Parameters - -```yaml ---- -version: "0.1" - -globalTimeout: 90 -testSuiteTimeout: 90 -testSuiteStep: 90 - -runson: linux - -autosplit: true - -retryOnFailure: true -maxRetries: 1 - -concurrency: 1 - -env: -TARGET_OS: "LINUX" - -cacheKey: '{{ checksum "requirement.txt" }}' -cacheDirectories: -- CacheDir - -pre: -- npm install playwright --save-exact -- pip install -r requirement.txt --cache-dir CacheDir -- playwright install - -post: -- cat yaml/linux/.hyperexecute_autosplits.yaml - -testDiscovery: -type: raw -mode: remote -command: grep -lr 'def' *.py - -testRunnerCommand: pytest $test - -jobLabel: [playwright-pytest, linux, autosplit] -``` - -### Step 4: Execute your Test Suite - -> **NOTE :** In case of MacOS, if you get a permission denied warning while executing CLI, simply run **`chmod u+x ./hyperexecute`** to allow permission. In case you get a security popup, allow it from your **System Preferences** → **Security & Privacy** → **General tab**. - -Run the below command in your terminal at the root folder of the project: - -```bash -./hyperexecute --config RELATIVE_PATH_OF_YOUR_YAML_FILE -``` - -OR use this command if you have not exported your username and access key in the step 2. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE `} - -### Step 5: Monitor the Test Execution - -Visit the [HyperExecute Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) and check your Job status. - -### Step 6: Download Artifacts and Reports - -HyperExecute also facilitates the provision to download the [Artifacts](/support/docs/hyperexecute-artifacts/) and [Reports](/support/docs/hyperexecute-reports/) on your local machine. Click on the corresponding button to download your generated artifacts and reports. - -## 2. Testing Using Gitpod - -You can also use the Gitpod platform to execute our sample repository. It will fetch all the sample codebases and trigger the CLI to execute the tests. - -Follow the below steps to run Test using Gitpod: - -**Step 1:** Click '**Open in Gitpod**' button. You will be redirected to Login/Signup page. This button is configured to redirect you to the Gitpod platform where you will be able to execute our sample repository. - -**Step 2:** Login with TestMu AI credentials. Once logged in, a pop-up confirmation will appear, asking you to **'Proceed'** to the Gitpod editor in a new tab. The current tab will display the HyperExecute Dashboard. - -**Step 3:** Choose your preferred editor (we recommend VS Code Editor) - -**Step 4:** As you are running a sample project, Fetching of the Test Scripts, [HyperExecute YAML](/support/docs/deep-dive-into-hyperexecute-yaml/), [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) and Triggering your tests using the `Execution Command` will be automated. - -**Step 5:** Once you see the `Job Link` in the logs, you can visit the [HyperExecute dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) to see the tests getting executed. - -You can also implement [Secret Keys](/support/docs/hyperexecute-how-to-save-and-manage-secrets/) in your YAML file. - -## Navigation in Automation Dashboard - -Every test run on the HyperExecute has a unique *jobId* associated with it. Each *jobId* can in turn constitute single (or multiple) *groupId*(s). You can visit [HyperExecute Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) for checking the status of the test execution. - -You can seamlessly navigate between JobId's and taskId's. You need to click on the *testID* to navigate from the HyperExecute logs to the Automation Dashboard. - -The snapshot below shows the videos, logs and other meta data for that specific *test_ID* - -> For any query or doubt, please feel free to contact us via **24×7 chat support** or you can also drop a mail to **support@testmuai.com**. -Happy testing! - -## Using the Pytest Agent Skill with TestMu AI - -The [pytest-skill](https://github.com/LambdaTest/agent-skills/tree/main/pytest-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The pytest-skill package includes: - -``` -pytest-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing Pytest Agent Skill - -Install a Pytest Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/pytest-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/pytest-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only pytest-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). diff --git a/static/docs/pytest-with-selenium-running-pytest-automation-script-on-testmu-selenium-grid.md b/static/docs/pytest-with-selenium-running-pytest-automation-script-on-testmu-selenium-grid.md deleted file mode 100644 index eb996f148..000000000 --- a/static/docs/pytest-with-selenium-running-pytest-automation-script-on-testmu-selenium-grid.md +++ /dev/null @@ -1,115 +0,0 @@ -# Selenium With PyTest - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Run PyTest tests on the TestMu AI cloud grid. This guide covers setup, running a sample test, configuring capabilities, and testing locally hosted pages. - -**Sample repo** -All the code used in this guide is available in the sample repository. - - View on GitHub - -## Prerequisites - -Complete these steps before running your first PyTest Selenium test. - -1. Install the latest Python build from the [official website](https://www.python.org/downloads/). -2. Verify that **pip** is installed in your system. Install **pip** from [pip documentation](https://pip.pypa.io/en/stable/installation/). -3. Download the latest **Selenium Client** and its **WebDriver bindings** from the [official website](https://www.selenium.dev/downloads/). -4. Install **virtualenv** to isolate the build from other setups and ensure tests run with the specified module versions. -```bash -pip install virtualenv -``` - -## Step 1: Clone the Sample Project - -Clone the repository and set up the environment. - -```bash -git clone https://github.com/LambdaTest/pytest-selenium-sample -cd pytest-selenium-sample -``` - -Create a virtual environment and activate it: -```bash -virtualenv venv -source venv/bin/activate -``` - -Install the required packages: -```bash -pip install -r requirements.txt -``` - -## Step 2: Set Your Credentials - -Configure your credentials to connect to the TestMu AI Selenium Grid. - -Set TestMu AI `Username` and `Access Key` in environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -## Step 3: Configure Your Test Capabilities - -Define browser, version, and OS settings for your test run. - -In the `conftest.py` file, update your test capabilities. This code passes browser, browser version, and operating system information, along with TestMu AI Selenium grid capabilities via the capabilities object. - -```python -capabilities = { -"build": "Sample PY Build", -"platformName": "Windows 11", -"browserName": "Chrome", -"browserVersion": "latest", -} -``` - -**Capabilities Generator** -Use the TestMu AI [Capabilities Generator](https://www.testmuai.com/capabilities-generator/) to auto-generate the capabilities class for your test requirements. - -## Step 4: Run the Test - -Execute the PyTest Selenium test from the command line. - -Navigate to the `tests` directory and run: -```bash -cd tests //navigate to tests directory -python lt_sample_todo.py -``` - -To run parallel tests: -```bash -pytest -s -n=2 tests/lt_sample_todo.py -``` - -## Step 5: View Your Results - -Check the test output on the console and the TestMu AI dashboard. - -Visit the [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) to view your test results. The dashboard provides: - -- Text logs for each test step -- Screenshots captured during execution -- Video recordings of the full test session - -## Run PyTest Tests Using Agent Skills - -Use AI coding assistants to generate and run PyTest tests with the TestMu AI Agent Skill. - -The [pytest-skill](https://github.com/LambdaTest/agent-skills/tree/main/pytest-skill) is part of [TestMu AI Agent Skills](https://github.com/LambdaTest/agent-skills/) - structured packages that teach AI coding assistants how to write production-grade test automation. - -Install the skill: - -```bash -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/pytest-skill .claude/skills/ - -# For Cursor / Copilot -cp -r agent-skills/pytest-skill .cursor/skills/ -``` - -Install all available framework skills at once by cloning the repository directly into your tool's skills directory (e.g., `.claude/skills/`, `.cursor/skills/`). diff --git a/static/docs/python-with-playwright.md b/static/docs/python-with-playwright.md deleted file mode 100644 index f2ed5aad4..000000000 --- a/static/docs/python-with-playwright.md +++ /dev/null @@ -1,109 +0,0 @@ -# Python with Playwright: Running Your First Test - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Learn how to use Playwright with Python to automate web application testing across 50+ real browsers and operating systems on TestMu AI cloud platform. - -## Prerequisites - -1. You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** -Download or clone the code sample for the Playwright Python from the TestMu AI GitHub repository to run the tests. - - View on GitHub - -```js -git clone https://github.com/LambdaTest/playwright-sample.git -cd playwright-sample -cd playwright-python -``` - -2. Install the Python dependencies. - -``` -pip install -r requirements.txt -``` - -3. A TestMu AI Username and Access key. You can get it from your TestMu AI Profile section. Don't have an account, sign up for free. - -4. To run Playwright tests, set your TestMu AI Username and Access key in the Environment Variables. - -## Run your Playwright tests with Python - -Navigate to the `playwright_sample.py` file in the `playwright-python` directory. - -```py -import json -import os -import urllib -import subprocess - -from playwright.sync_api import sync_playwright - -capabilities = { -'browserName': 'Chrome', # Browsers allowed: `Chrome`, `MicrosoftEdge`, `pw-chromium`, `pw-firefox` and `pw-webkit` -'browserVersion': 'latest', -'LT:Options': { -'platform': 'Windows 10', -'build': 'Playwright Python Build', -'name': 'Playwright Python Test', -'user': os.getenv('LT_USERNAME'), -'accessKey': os.getenv('LT_ACCESS_KEY'), -'network': True, -'video': True, -'console': True, -'tunnel': False, # Add tunnel configuration if testing locally hosted webpage -'tunnelName': '', # Optional -'geoLocation': '', # country code can be fetched from https://www.lambdatest.com/capabilities-generator/ -} -} - -def run(playwright): -playwrightVersion = str(subprocess.getoutput('playwright --version')).strip().split(" ")[1] -capabilities['LT:Options']['playwrightClientVersion'] = playwrightVersion - -lt_cdp_url = 'wss://cdp.lambdatest.com/playwright?capabilities=' + urllib.parse.quote( -json.dumps(capabilities)) -browser = playwright.chromium.connect(lt_cdp_url) -page = browser.new_page() -try: -page.goto("https://www.bing.com/") -page.fill("[aria-label='Enter your search term'] > input", 'LambdaTest') -page.keyboard.press("Enter") -page.wait_for_timeout(1000) - -title = page.title() - -print("Title:: ", title) - -if "LambdaTest" in title: -set_test_status(page, "passed", "Title matched") -else: -set_test_status(page, "failed", "Title did not match") -except Exception as err: -print("Error:: ", err) -set_test_status(page, "failed", str(err)) - -browser.close() - -def set_test_status(page, status, remark): -page.evaluate("_ => {}", -"lambdatest_action: {\"action\": \"setTestStatus\", \"arguments\": {\"status\":\"" + status + "\", \"remark\": \"" + remark + "\"}}"); - -with sync_playwright() as playwright: -run(playwright) - -``` - -Pass the below command in the terminal to run the test. - -```js -npm run test -``` - -## View your test results - -Go to the [TestMu AI Web Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) to see your Playwright Python test results. - -To generate Playwright tests with AI coding assistants, see [Run Tests With Agent Skills](/support/docs/playwright-agent-skills/). diff --git a/static/docs/python-with-selenium-running-python-automation-scripts-on-testmu-selenium-grid.md b/static/docs/python-with-selenium-running-python-automation-scripts-on-testmu-selenium-grid.md deleted file mode 100644 index 4029b28d9..000000000 --- a/static/docs/python-with-selenium-running-python-automation-scripts-on-testmu-selenium-grid.md +++ /dev/null @@ -1,170 +0,0 @@ -# Selenium With Python - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Run your Python Selenium tests on the TestMu AI cloud grid across 3000+ browser and OS combinations. The setup is the same for every framework: you connect to the grid and pass your capabilities. This guide covers that shared flow once, then gives you a per-framework quickstart in the tabs below. - -## Prerequisites - -1. [Create a TestMu AI account](https://www.testmuai.com/register/) if you don't have one. -2. Get your **Username** and **Access Key** from the [TestMu AI Dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard). -3. Install [Python](https://www.python.org/downloads/) and pip. -4. Install the Selenium client and WebDriver bindings, and (recommended) `virtualenv` to isolate dependencies. - -## Set your credentials - -Every framework authenticates the same way: your Username and Access Key are read from environment variables. Set them once. Pick your operating system: - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME=${ YOUR_LAMBDATEST_USERNAME()} -set LT_ACCESS_KEY=${ YOUR_LAMBDATEST_ACCESS_KEY()}`} - -## How the sample test works - -Every framework below connects to the grid at `hub.lambdatest.com/wd/hub` and passes your browser and OS choices through a capabilities dictionary: - -```python -capabilities = { -"build": "your build name", -"name": "your test name", -"platformName": "Windows 10", -"browserName": "Chrome", -"browserVersion": "latest", -} -``` - -What changes between frameworks is only how those capabilities are supplied: inline, a `conftest.py`, a `.robot` variables block, or a JSON config. That is what each tab covers. - -Use the [Capabilities Generator](https://www.testmuai.com/capabilities-generator/) to build a capabilities block for any browser, version, and OS combination. - -## Run a test in your framework - -Each tab lists the framework-specific pieces. Clone the matching repo (it contains the full, ready-to-run project), then run. - -The standard-library `unittest` framework connects a remote WebDriver to the grid, with capabilities inline in the test. - -1. Clone the [sample GitHub project](https://github.com/LambdaTest/Python-UnitTest-Selenium): - -```bash -git clone https://github.com/LambdaTest/Python-UnitTest-Selenium -cd Python-UnitTest-Selenium -``` - -2. Set your browser and OS in the capabilities dictionary: - -```python -capabilities = { -"build": "UnitTest-Selenium-Sample", -"name": "UnitTest-Selenium-Test", -"platformName": "Windows 11", -"browserName": "Chrome", -"browserVersion": "latest", -} -``` - -3. Run the test: - -```bash -python lambdatest_test.py -``` - -pytest keeps capabilities in `conftest.py` and runs in parallel via `pytest-xdist`. - -1. Clone the [sample GitHub project](https://github.com/LambdaTest/pytest-selenium-sample): - -```bash -git clone https://github.com/LambdaTest/pytest-selenium-sample -cd pytest-selenium-sample -``` - -2. Set your browser and OS in `conftest.py`: - -```python title="conftest.py" -capabilities = { -"build": "Sample PY Build", -"platformName": "Windows 11", -"browserName": "Chrome", -"browserVersion": "latest", -} -``` - -3. Run a single test, or in parallel: - -```bash -python tests/lt_sample_todo.py -pytest -s -n=2 tests/lt_sample_todo.py -``` - -Robot Framework uses Selenium2Library, with capabilities and the grid URL declared as variables in a `.robot` file, run through a Makefile. - -1. Clone the [sample GitHub project](https://github.com/LambdaTest/Robot-Selenium-Sample): - -```bash -git clone https://github.com/LambdaTest/Robot-Selenium-Sample -cd Robot-Selenium-Sample -``` - -2. Set your browser and OS in the variables block of `common.robot`: - -```robotframework title="common.robot" -*** Settings *** -Library Selenium2Library - -*** Variables *** -@{_tmp} -... browserName: %{browserName}, -... platformName: %{platform}, -... browserVersion: %{version}, -... name: RobotFramework Lambda Test -${BROWSER} %{ROBOT_BROWSER} -${CAPABILITIES} ${EMPTY.join(${_tmp})} -${REMOTE_URL} https://${KEY}@hub.lambdatest.com/wd/hub -``` - -3. Run a single test, or all in parallel: - -```bash -make test_Windows_10_chrome_latest -make run_all_in_parallel -``` - -Behave is BDD for Python: feature files plus step definitions, with capabilities in a JSON config. - -1. Clone the [sample GitHub project](https://github.com/LambdaTest/Python-Behave-Selenium): - -```bash -git clone https://github.com/LambdaTest/Python-Behave-Selenium -cd Python-Behave-Selenium -``` - -2. Set your browser and OS in `config/config.json`: - -```json title="config/config.json" -[ -{ -"platformName": "Windows 10", -"browserName": "chrome", -"browserVersion": "latest", -"build": "Behave Selenium Sample", -"name": "Behave Sample Test" -} -] -``` - -3. Run the test: - -```bash -behave features/test.feature -``` - -## Legacy frameworks - -This framework is deprecated and kept only for existing suites. For new projects, use one of the frameworks above. - -- **Lettuce** (unmaintained since 2016, Python 2 only): [sample GitHub project](https://github.com/LambdaTest/sample-lettuce). Set your browser and OS in `config.json`, then run `python tests/lt_sample_todo.py`. Migrate to Behave or pytest. - -## View your results - -Your test results, including video, network logs, and command-by-command execution, appear on the [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build). diff --git a/static/docs/pyunit-on-hyperexecute-grid.md b/static/docs/pyunit-on-hyperexecute-grid.md deleted file mode 100644 index 7ffd0d2e7..000000000 --- a/static/docs/pyunit-on-hyperexecute-grid.md +++ /dev/null @@ -1,195 +0,0 @@ -# Running PyUnit Framework Tests on HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -PyUnit, also known as unittest, is a testing framework in Python that is commonly used with Selenium for automated testing. It provides a structure for organizing and executing test cases, along with features like test fixtures, assertions, and test discovery. - -HyperExecute is an AI Native Test Orchestration Cloud Platform that empowers you to run **end-to-end** tests **quickly** and **efficiently**. It provides Just-in-Time (JIT) testing infrastructure with fast execution **speeds**, **smart orchestration**, and **detailed logs**. - -This guide details how to execute your **PyUnit** framework tests on **HyperExecute** via two different methods: - -- [**Using Local System**](/support/docs/pyunit-on-hyperexecute-grid/#1-testing-using-local-system) - You can use your own local machine to execute tests. -- [**Using Gitpod Platform**](/support/docs/pyunit-on-hyperexecute-grid/#2-testing-using-gitpod) - Execute tests using GitPod. (Requires a [Gitpod](https://gitpod.io/login/) account) - -## 1. Testing Using Local System - -Follow the step-by-step guide to execute your test on HyperExecute. - -### Prerequisites - -To run the Tests on HyperExecute from your Local System, you are required: - -- Your TestMu AI [Username and Access key](/support/docs/hyperexecute-how-to-get-my-username-and-access-key/) -- [HyperExecute YAML](/support/docs/hyperexecute-yaml-version0.2/) file which contains all the necessary instructions. -- [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) in order to initiate a test execution Job . -- Setup the [Environmental Variable](/support/docs/hyperexecute-environment-variable-setup/) - -### Step 1: Configure Your Test Suite - -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** - -Download or Clone the code sample for the JUnit from the TestMu AI GitHub repository to run the tests on the HyperExecute. - - View on GitHub - -If you are using your own project, make sure you update the **Hub endpoint** in your tests file. - -By setting up the Hub endpoint, you establish the communication channel between your tests and the browser nodes, enabling effective test distribution and execution. - -Configure the desired capabilities based on your test requirements. For example: - -```bash -capabilities = { -"build": "UnitTest-Selenium-Sample", -"name": "UnitTest-Selenium-Test", -"platformName": "Windows 11", -"browserName": "Chrome", -"browserVersion": "latest", -} -``` - -> You can also use **dev** and **beta** browser versions. To generate capabilities for your test requirements, you can use our inbuilt 🔗 Capabilities Generator. - -### Step 2: Setup the CLI in your Test Suite - -After cloning / downloading the sample repo, you need to setup the CLI and the environment variables. - -#### Download the HyperExecute CLI - -The CLI is used for triggering the tests on HyperExecute. It is recommend to download the CLI binary on the host system and keep it in the root directory of the suite to perform the tests on HyperExecute. - -You can download the CLI for your desired platform from the below mentioned links: - -| Platform | HyperExecute CLI | -| ---------| ---------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| MacOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -#### Setup Environment Variable - -Now, you need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in the [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). - -Run the below mentioned commands in your terminal to setup the CLI and the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -### Step 3: Configure YAML in your Test Suite - -Configure your YAML file as per your use cases using **key value** pairs. - -In this sample YAML file, we have mentioned: - -- **version** of the YAML file -- **Timeouts** for executing your project -- **Mode of execution** is [Autosplit](/support/docs/hyperexecute-auto-split-strategy/). You can also opt for [Matrix](/support/docs/hyperexecute-matrix-multiplexing-strategy/) or [Hybrid](/support/docs/hyperexecute-hybrid-strategy/) mode. -- **Pre and Post** commands -- **Reports and Artefacts** that will be generated after the completion of tests -- and other necessary YAML Parameters - -```yaml ---- -version: 0.1 -globalTimeout: 90 -testSuiteTimeout: 90 -testSuiteStep: 90 - -runson: linux - -autosplit: true -retryOnFailure: true - -maxRetries: 1 -concurrency: 2 - -env: -# PAT: ${{ .secrets.testKey }} -TARGET_OS: LINUX - -cacheKey: '{{ checksum "requirements.txt" }}' -cacheDirectories: -- CacheDir -pre: -- pip3 install -r requirements.txt --cache-dir CacheDir -post: -- cat yaml/linux/pyunit_hyperexecute_autosplit_sample.yaml - -mergeArtifacts: true - -uploadArtefacts: -- name: TestReport -path: -- example_*/** - -report: true -partialReports: -type: json -location: / -frameworkName: extent - -testDiscovery: -type: raw -mode: remote -command: grep -nri 'HyperTestPyUnit' tests -ir --include=\*.py | sed 's/:.*//' - -testRunnerCommand: python3 $test - -jobLabel: [selenium-pyunit, linux, autosplit] -``` - -### Step 4: Execute your Test Suite - -> **NOTE :** In case of MacOS, if you get a permission denied warning while executing CLI, simply run **`chmod u+x ./hyperexecute`** to allow permission. In case you get a security popup, allow it from your **System Preferences** → **Security & Privacy** → **General tab**. - -Run the below command in your terminal at the root folder of the project: - -```bash -./hyperexecute --config RELATIVE_PATH_OF_YOUR_YAML_FILE -``` - -OR use this command if you have not exported your username and access key in the step 2. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE `} - -### Step 5: Monitor the Test Execution - -Visit the [HyperExecute Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) and check your Job status. - -### Step 6: Download Artifacts and Reports - -HyperExecute also facilitates the provision to download the [Artifacts](/support/docs/hyperexecute-artifacts/) and [Reports](/support/docs/hyperexecute-reports/) on your local machine. Click on the corresponding button to download your generated artifacts and reports. - -## 2. Testing Using Gitpod - -You can also use the Gitpod platform to execute our sample repository. It will fetch all the sample codebases and trigger the CLI to execute the tests. - -Follow the below steps to run Test using Gitpod: - -**Step 1:** Click '**Open in Gitpod**' button. You will be redirected to Login/Signup page. This button is configured to redirect you to the Gitpod platform where you will be able to execute our sample repository. - -**Step 2:** Login with TestMu AI credentials. Once logged in, a pop-up confirmation will appear, asking you to **'Proceed'** to the Gitpod editor in a new tab. The current tab will display the HyperExecute Dashboard. - -**Step 3:** Choose your preferred editor (we recommend VS Code Editor) - -**Step 4:** As you are running a sample project, Fetching of the Test Scripts, [HyperExecute YAML](/support/docs/deep-dive-into-hyperexecute-yaml/), [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) and Triggering your tests using the `Execution Command` will be automated. - -**Step 5:** Once you see the `Job Link` in the logs, you can visit the [HyperExecute dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) to see the tests getting executed. - -You can also implement [Secret Keys](/support/docs/hyperexecute-how-to-save-and-manage-secrets/) in your YAML file. - -## Navigation in Automation Dashboard - -Every test run on the HyperExecute has a unique *jobId* associated with it. Each *jobId* can in turn constitute single (or multiple) *groupId*(s). You can visit [HyperExecute Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) for checking the status of the test execution. - -You can seamlessly navigate between JobId's and taskId's. You need to click on the *testID* to navigate from the HyperExecute logs to the Automation Dashboard. - -The snapshot below shows the videos, logs and other meta data for that specific *test_ID* - -> For any query or doubt, please feel free to contact us via **24×7 chat support** or you can also drop a mail to **support@testmuai.com**. -Happy testing! diff --git a/static/docs/qmetry-integration.md b/static/docs/qmetry-integration.md deleted file mode 100644 index cfebbf2d1..000000000 --- a/static/docs/qmetry-integration.md +++ /dev/null @@ -1,60 +0,0 @@ -# Running Your QMetry Tests On TestMu AI Selenium Grid - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -QMetry is a codeless platform tailored for DevOps and Agile testing teams. It can be used to prepare and execute test scenarios on multiple platforms very easily. Not only that, but QMetry also allows you to record test cases in almost all major languages and frameworks, including the QMetry framework. -TestMu AI now integrates with the QMetry platform to help you execute cross-browser testing of your website, web app, or your mobile website, over 3000+ real desktop devices, mobile viewports, and operating system combinations on its cloud-based selenium grid. This document is a comprehensive guide to help you understand how to integrate QMetry with your TestMu AI account. -## How To Setup Your TestMu AI Account With QMetry? - -Once you have your project set up in the QMetry app and test suite ready, you need to specify and activate your TestMu AI account in the QMetry app to execute it over the TestMu AI's cloud-based Selenium Grid. Below is the step by step guide to help you do just that: -1. Once you have your test suite ready, click on the **Execute** button on the app, and select **Remote**. - -2. In the window that opens, navigate to **TestMu AI** tab. Here, you will need to enter your account details and the [desired capabilities](/docs/selenium-automation-capabilities/) to execute the test over your TestMu AI account. - -3. Now, you need to provide access of your TestMu AI account to the QMetry app. To do so, add your _username_ and _access key_ over their corresponding fields, and click on **FETCH PLATFORM LIST** button. This will verify your credentials and connect your TestMu AI account with the QMetry app successfully, based on your current plan. - ->To get your Username and Access Key, go to your TestMu AI automation dashboard and click on the "key" icon left to the help button, on the top right corner. - ->If you are on _[Lite](https://www.testmuai.com/pricing)_ or _[Live](https://www.testmuai.com/pricing)_ plan, and your provided automation minutes have been exhausted; the above step will throw an error. In such a case, you can wait for the next availability of automation minutes, or you can [upgrade](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/billing/plans) your account. To learn more about how to manage and upgrade your account, refer [this guide](/docs/manage-subscriptions/). -6. Once done, you will be able to select your desired capabilities in their corresponding fields, like the platform, browser, browser version, resolution, etc. To know more about desired capabilities, refer to [this guide](/docs/selenium-automation-capabilities/). - -7. Click on **Save**, once you have selected the desired capabilities. The current account info and the capabilities will be saved with a name that you provide. - -That's it. Your QMetry app has been integrated successfully with your TestMu AI account to execute your tests on its cloud-based Selenium Grid. -## Running Tests From Your QMetry App On TestMu AI Platform - -For demo purpose, we have recorded a test scenario, which includes: - -```javascript -- Opening [LambdaTest ToDo App](https://lambdatest.github.io/sample-todo-app/) -- Check the first item in the list -- Check the second item in the list -- Add a new item in the list, with text "Yey, Let's add it to list". -``` - -For the desired capabilities, we have chosen the following details: - -```javascript -platform: macOS High Sierra -browserName: Chrome -version: 87 -resolution: 1280x800 -network: true -visual: true -``` - -Now just select the profile, and hit **Start**. - -The test execution will then start, and you can see the same on the automation dashboard of your TestMu AI account. - -Once the test is complete, you can view the test result on the automation dashboard by clicking on the test case. You can see and verify if the test has been passed or failed in the Automation Logs. - -For each test executed overthe TestMu AI platform, you can view logs, [mark bugs](/docs/mark-as-bug-in-automation-testing/), and do much more. - -## Running QMetry Tests on HyperExecute - -If you have completed the QMetry setup above, you can also run your QMetry test scripts on [HyperExecute](/support/docs/getting-started-with-hyperexecute/) for faster, parallelized test execution. HyperExecute supports the QMetry framework and its native reporting, so your existing QMetry test suite can be executed at scale without any changes to your test scripts. - -To get started, create a [HyperExecute Project](/support/docs/hyperexecute-projects/) connected to your repository containing the QMetry test scripts, configure your [HyperExecute YAML](/support/docs/hyperexecute-yaml-parameters/), and trigger the job via the [CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) or a [Workflow](/support/docs/hyperexecute-workflows/). - -> Congrats! You have successfully integrated the QMetry app with your TestMu AI account and ran your test cases over the TestMu AI's cloud based Seleniunm Grid. If you come across any doubts or questions, reach out to us via our **24/7 chat support** or mail us at [support@testmuai.com](mailto:support@testmuai.com). Happy testing! 🙂 diff --git a/static/docs/qtest-integration.md b/static/docs/qtest-integration.md deleted file mode 100644 index 8a162b774..000000000 --- a/static/docs/qtest-integration.md +++ /dev/null @@ -1,32 +0,0 @@ -# qTest Integration With TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -qTest is a comprehensive test management tool designed for Project Management, Bug Tracking, and Test Management. It embraces a centralized test management approach, facilitating seamless communication and accelerating task development across QA teams and stakeholders. With qTest Launch, you can run your automated tests on a variety of devices and real devices provided by TestMu AI. - -## Prerequisites - -1. qTest Host - - Setup host for qTest: [Download](https://support-hub.tricentis.com/open?sys_kb_id=194a54eedb4f5c181ea7bb13f3961950&id=kb_article_view&number=KB0015571) and copy the file agentctl-[version]-mac-x64-full.tgz to a desired installation directory, e.g. /usr/local/agentctl-[version]-mac-x64-full.tgz - - Follow this guide for more in-depth information: [qTest Host Setup](https://documentation.tricentis.com/qtest/10200/en/content/qtest_launch/qtest_automation_host_2_install_upgrade_guides/qtest_automation_host_installation_and_upgrade_guide_master_list.htm) - -2. Acquire your qTest API Token - - * Login to qTest Manager as an Administrator and access the Resources page. - * Expand the APIs and SDK section. - -3. Setup host and agent - - Start the host by running the following command: -``` -/path/to/agentctl-[version]$ ./agentctl start -``` - Navigate to [http://localhost:6789](http://localhost:6789) and add your details. - -4. Run Tests - - Now once you have set up host, you can setup agents as per your requirements and run your tests using qTest Launch on TestMu AI. - - For example to run Junit Automated tests, you can refer the following github repository: [automate-junit-qtest](https://github.com/LambdaTest/automate-junit-qtest/tree/main) and its readme file for more information. diff --git a/static/docs/queuing-appium-tests.md b/static/docs/queuing-appium-tests.md deleted file mode 100644 index 5c57c5ded..000000000 --- a/static/docs/queuing-appium-tests.md +++ /dev/null @@ -1,31 +0,0 @@ -# Queuing Appium Tests - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -To avoid over-exploitation of TestMu AI, we have imposed a capacity constraint on the number of tests queued on our platform. The maximum number of queued items for your TestMu AI account is determined by the number of concurrent sessions you are registered for. Using the formula below, you can calculate the maximum number of Appium test cases you are entitled queue. - -```js -Maximum number of Appium test cases that can be queued = n + 150 - -Here, n = number of concurrent sessions. -``` - -For example, if your TestMu AI account enables 10 concurrent sessions, your queue can hold a maximum of (10 + 150) queued test cases, or 160 queued test cases. TestMu AI will handle the scheduling and parallel execution of Appium test cases in your queue. - -If you send more tests for execution than your concurrency limit allows, they will be queued until the queue hits their maximum holding capacity. So, if you have a concurrency of 100 but send 200 tests, the "additional" tests will be queued and performed as soon as the first round of tests is completed. After the number of running tests reaches 99 in this case, one test from the queue will start running, and when the next test ends, another test from the queue will start running. This process is repeated until all of the queued tests are completed. - -> -Here are some key points to keep in mind while running Appium automated tests on TestMu AI platform. -1. As mentioned above, the number of tests that can be queued for a single instance is limited. This limit is determined by the amount of concurrent TestMu AI sessions, and exceeding your maximum queuing capacity will result in the test cases being discarded. -2. There is also a maximum time restriction for how long a test can be in the queue. So, if you have automated test cases in a queue for more than 15 minutes, they will be timed out and will not be executed. - -## Rate Limiter and Penalty - -An account can exceed the authorized queuing limit seven times per month. For example, if an account has 100 concurrent session plans, user is allowed to queue n+150=250 requests. If one attempts to exceed the limit, the requests will be declined. At the account level, rejection events are counted monthly. So, before terminating an account, the user can request (n+150)*x (default 7 times in a month), where x is a constant per account. - -* After (n+150)*2 request rejections due to exceeding the permitted queuing threshold in a month, the admin will get an email alert for service exploitation. -* After (n+150)*4 request rejections due to exceeding the allowed queuing threshold in a month, the admin will get an email alert for service exploitation. -* After (n+150)*6 request rejections due to exceeding the authorized queuing threshold in a month, the admin will get an email alert for service exploitation with last warning message. -* After (n+150)*7 request rejections due to exceeding the allowable queuing threshold in a month, Admin will receive an email notifying account suspension with a help link for account re-activation. - -> That's all about queuing your Appium tests. If you still have any questions for us, please feel free to let us know. Our experts are always **available on chat** to help you out with any roadblock regarding our product. diff --git a/static/docs/quick-guide-to-run-node-js-tests-on-testmu-selenium-grid.md b/static/docs/quick-guide-to-run-node-js-tests-on-testmu-selenium-grid.md deleted file mode 100644 index 45b396665..000000000 --- a/static/docs/quick-guide-to-run-node-js-tests-on-testmu-selenium-grid.md +++ /dev/null @@ -1,111 +0,0 @@ -# Quick Guide To Run Node.js Tests On Cloud Selenium Grid - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI’s [Selenium grid](https://www.testmuai.com/blog/why-selenium-grid-is-ideal-for-automated-browser-testing/) enables you to perform end-to-end automation tests on a secure, reliable, and scalable Selenium cloud infrastructure. You can run your Node.js automation test scripts on 3000+ browser and operating system environments giving you higher test coverage and significantly less built times. This document will help you quickly get started with running your Node.js test scripts on the TestMu AI Selenium automation grid. - -## Prerequisites - -- The first step is to download and install Node.js and node package manager or npm. We recommend using the latest version of node.js. - -- You can download it from [official NodeJS website](https://nodejs.org/en/). - -- If you have npm already installed, you may want to upgrade it to the latest version. Here the code you can run in your terminal to upgrade npm. - - npm install npm@latest -g - -## Installing Selenium Dependencies For Node.js - -The next step is to install Selenium dependencies for Node.js using npm. Here’s the command that you can run: - - npm i selenium-webdriver - -## Sample Node.js & Selenium Test To Get You Started - -Once you have installed the necessary dependencies, you are all set to run your Node.js automation scripts on the TestMu AI Selenium grid. To get started, you can try out the code below. It’s a simple Node.js script that opens up Google.com, searches TestMu AI in Google search, and then outputs the search result page's title. You can also fork it from our [Node.js GitHub repository](https://github.com/LambdaTest/nodejs-selenium-sample). Make sure to create the JS file in the same folder where you have downloaded Selenium. - -## Sample Node.js & Selenium Automation Script - -Do make sure to add the **Username** and **Key** in the code below. You can find your username and access key using the key icon on the [Automation dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/) or in your [Profile](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile) section. - -```javascript -/* -LambdaTest selenium automation sample example -Configuration ----------- -username: Username can be found at automation dashboard -accessKey: AccessKey can be generated from automation dashboard or profile section - -Result -------- -Execute NodeJS Automation Tests on LambdaTest Distributed Selenium Grid -*/ -const webdriver = require('selenium-webdriver'); - -/* -Setup remote driver -Params ----------- -platform : Supported platform - (Windows 10, Windows 8.1, Windows 8, Windows 7, macOS High Sierra, macOS Sierra, OS X El Capitan, OS X Yosemite, OS X Mavericks) -browserName : Supported platform - (chrome, firefox, Internet Explorer, MicrosoftEdge, Safari) -version : Supported list of version can be found at https://www.testmuai.com/capabilities-generator/ -*/ - -// username: Username can be found at automation dashboard -const USERNAME = '{YOUR_LAMBDATEST_USERNAME}'; - -// AccessKey: AccessKey can be generated from automation dashboard or profile section -const KEY = '{YOUR_LAMBDATEST_ACCESS_KEY}'; - -// gridUrl: gridUrl can be found at automation dashboard -const GRID_HOST = 'hub.lambdatest.com/wd/hub'; - -function searchTextOnGoogle() { -// Setup Input capabilities -const capabilities = { -platform: 'windows 10', -browserName: 'chrome', -version: '67.0', -resolution: '1280x800', -network: true, -visual: true, -console: true, -video: true, -name: 'Test 1', // name of the test -build: 'NodeJS build' // name of the build -} - -// URL: https://{username}:{accessKey}@hub.lambdatest.com/wd/hub -const gridUrl = 'https://' + USERNAME + ':' + KEY + '@' + GRID_HOST; - -// setup and build selenium driver object -const driver = new webdriver.Builder() -.usingServer(gridUrl) -.withCapabilities(capabilities) -.build(); - -// navigate to a url, search for a text and get title of page -driver.get('https://www.google.com/ncr').then(function() { -driver.findElement(webdriver.By.name('q')).sendKeys('LambdaTest\n').then(function() { -driver.getTitle().then(function(title) { -setTimeout(function() { -console.log(title); -driver.quit(); -}, 5000); -}); -}); -}); -} -searchTextOnGoogle(); -``` -You can execute the test from your terminal through this command. - - node index.js - -Do note the use of the remote webdriver here. We have passed browser environment configurations through the capabilities class. You can set the environment details as per your requirements. - -To set up your own choice of environment for the test in the code, you can leverage our Capability Generator tool. - -TestMu AI Capability Generator Tool - -> **Tear Down**: The tear down method helps the TestMu AI platform understand when the test’s execution is finished. It’s important to tear down each test, or else it will give timeout errors. diff --git a/static/docs/quick-guide-to-run-php-tests-on-testmu-selenium-grid.md b/static/docs/quick-guide-to-run-php-tests-on-testmu-selenium-grid.md deleted file mode 100644 index 8ea2321cc..000000000 --- a/static/docs/quick-guide-to-run-php-tests-on-testmu-selenium-grid.md +++ /dev/null @@ -1,144 +0,0 @@ -# Quick Guide To Run PHP Tests on TestMu AI Selenium Grid - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI's cloud-based Selenium grid can be leveraged to run your automation test scripts on 3000+ different browser and operating system environments. It’s a scalable, reliable, and secure online [Selenium grid](https://www.testmuai.com/selenium-automation) infrastructure that not only helps you in increasing test coverage, but also cut down execution of your test automation builds by significant margin. This post will help you get started with running your PHP based Selenium automation scripts on TestMu AI Selenium grid. - -## Prerequisites - -Running PHP scripts require your local system to have both PHP and Composer. Here’s how you can setup your PHP environment for running Selenium scripts. - -- First step is to install the latest PHP build. Latest MacOS systems come with PHP pre-installed, however for windows users and linux users, you can download PHP from [here](https://www.php.net/manual/en/install.php). - -- Next step is to install Composer. Open up terminal and navigate to the folder that have PHP installed. Run the following command. - -```javascript -php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');" -php -r "if (hash_file('sha384', 'composer-setup.php') === '93b54496392c062774670ac18b134c3b3a95e5a5e5c8f1a9f115f203b75bf9a129d5daa8ba6a13e2cc8a1da0806388a8') { echo 'Installer verified'; } else { echo 'Installer corrupt'; unlink('composer-setup.php'); } echo PHP_EOL;" -php composer-setup.php -php -r "unlink('composer-setup.php');" -``` -- If you are using XAMPP or MAMP, you may have to open up PHP shell by pressing the button shown in picture below: - -Once you have composer installed, you can leverage Composer to download and install Selenium dependencies. For that, first create a composer.json file in your directory and add the following JSON code: - -```javascript -{ -"require": { -"phpunit/phpunit-selenium": "*", -"facebook/webdriver": "dev-master" -} -} -``` -## Installing Selenium Dependencies For PHP - -Now all you would have to do is to run composer to install Selenium. You can do that by running following command in your terminal: - - composer install - -If you are using XAMPP or MAMP you may have to be a little more specific. - - php composer.phar install - -## Sample PHP & Selenium Test To Get You started - -Once you have installed and setup PHP and Selenium, you are now ready to run PHP automation scripts on TestMu AI Selenium Grid. Checkout the simple sample code below. This is a simple PHP and Selenium automation script that opens up google.com, searches TestMu AI and outputs the title of the resultant google search page. You can also fork it from our [PHP GitHub repository](https://github.com/LambdaTest/php-selenium-sample). - -### PHP Sample Test - -```javascript -setCapability('browserName',$LT_BROWSER); -$desired_capabilities->setCapability('version', $LT_BROWSER_VERSION); -$desired_capabilities->setCapability('platform', $LT_PLATFORM); -$desired_capabilities->setCapability('name', "Php"); -$desired_capabilities->setCapability('build', "Php Build"); -$desired_capabilities->setCapability('network', true); -$desired_capabilities->setCapability('visual', true); -$desired_capabilities->setCapability('video ', true); -$desired_capabilities->setCapability('console', true); - -/* -Setup remote driver -Params ----------- -Execute test: navigate google.com search LambdaTest -Result -------- -print title -*/ -self::$driver = RemoteWebDriver::create($url, $desired_capabilities); - -self::$driver->get("https://www.google.com/ncr"); - -$element = self::$driver->findElement(WebDriverBy::name("q")); -if($element) { -$element->sendKeys("LambdaTest"); -$element->submit(); -} - -print self::$driver->getTitle(); -self::$driver->quit(); -} -} - -$lambdaTest = new LambdaTest(); -$lambdaTest->searchTextOnGoogle(); - -?> -``` - -You can execute the test from your terminal through this command. - - vendor/bin/phpunit lambdatest.php - -Do note the use of remote webdriver here. We have passed browser environment configurations through capabilities class. You can set your environment details as you wish. - -To setup your own environment for the test you can leverage our Capability Generator tool. - -TestMu AI Capability Generator Tool - -> **Tear Down**: The tear down method helps TestMu AI platform understand when the execution of a test is finished. It’s important to tear down each test or else it will give timeout errors diff --git a/static/docs/quick-guide-to-run-ruby-tests-on-testmu-selenium-grid.md b/static/docs/quick-guide-to-run-ruby-tests-on-testmu-selenium-grid.md deleted file mode 100644 index 80777e11a..000000000 --- a/static/docs/quick-guide-to-run-ruby-tests-on-testmu-selenium-grid.md +++ /dev/null @@ -1,123 +0,0 @@ -# Quick Guide To Run Ruby Tests On TestMu AI Selenium Grid - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI Selenium Automation Grid is a secure, scalable, and reliable [Selenium Grid](https://www.testmuai.com/blog/why-selenium-grid-is-ideal-for-automated-browser-testing/) on cloud that can help you run your Ruby automation scripts on 3000+ different browsers and operating system environments. The grid is specifically built for high performance and gives you not only a greater browser coverage but also an ability to run automation tests in parallel. This helps you reduce your automation test execution time significantly. This post will help you quickly get started with running Ruby test scripts on TestMu AI Selenium Grid. - -## Prerequisites - -First, you would have to install Ruby and gem on your local system. We recommend installing the latest version of Ruby. Installing these platforms is a little different in each operating system - -- For windows, you can download the latest installed the [official website](https://rubyinstaller.org/downloads/). - -- If you are using Linux or Ubuntu OS, you can run a simple apt command like below. You can also refer to this doc to know more about which command suits your OS better. - - $ sudo apt-get install ruby-full - -- If you are using MacOS you can run a [homebrew](https://brew.sh/) command like this- - - $ brew install ruby - -## Installing Selenium Dependencies For Ruby - -Once you have Ruby and Gem setup, you would now have to install Selenium dependencies. You can install Selenium via simple gem command- - - gem install selenium-webdriver - -## Sample Ruby & Selenium Test To Get You started - -Once you have installed necessary dependencies, you are all set to run your Ruby automation scripts on TestMu AI Selenium grid. To get started you can try out the code below. It’s a simple Ruby scripts that opens up [Google.com](https://www.google.com/), searches TestMu AI in Google search bar, and then displays the title of the search result page. You can also fork it from our [Ruby GitHub repository](https://github.com/LambdaTest/ruby-selenium-sample). - -```ruby -//Sample Ruby and Selenium Automation Script - -require 'selenium-webdriver' -require 'test/unit' - -class LtTest < Test::Unit::TestCase -""" -LambdaTest selenium automation sample example -Configuration ----------- -username: Username can be found at automation dashboard -accessToken: AccessToken can be generated from automation dashboard or profile section - -Result -------- -Execute Ruby Automation Tests on LambdaTest Distributed Selenium Grid -""" - -def setup -""" -Setup remote driver -Params ----------- -platform : Supported platform - (Windows 10, Windows 8.1, Windows 8, Windows 7, macOS High Sierra, macOS Sierra, OS X El Capitan, OS X Yosemite, OS X Mavericks) -browserName : Supported platform - (chrome, firefox, Internet Explorer, MicrosoftEdge) -version : Supported list of version can be found at https://www.testmuai.com/capabilities-generator/ - -Result -------- -""" -username= "{YOUR_LAMBDATEST_USERNAME}" -accessToken= "{YOUR_LAMBDATEST_ACCESS_KEY}" -gridUrl = "hub.lambdatest.com/wd/hub" - -caps = { -:browserName => "chrome", -:version => "67.0", -:platform => "win10", -:name => "LambdaTest ruby google search name", -:build => "LambdaTest ruby google search build", -:network => true, -:visual => true, -:video => true, -:console => true -} - -puts (caps) -# URL: https://{username}:{accessToken}@hub.lambdatest.com/wd/hub -@driver = Selenium::WebDriver.for(:remote, -:url => "https://"+username+":"+accessToken+"@"+gridUrl, -:desired_capabilities => caps) -end - -def test_Login -""" -Setup remote driver -Params ----------- -Execute test: navigate google.com search LambdaTest -Result -------- -print title -""" -puts("Searching LambdaTest on google.com ") -sleep(15) -@driver.get("https://www.google.com/ncr") -elem = @driver.find_element(:name, 'q') -elem.send_keys("LambdaTest.com") -elem.submit() - -puts("Printing title of current page :"+@driver.title) -end -def teardown -""" -Quit selenium driver -""" -@driver.quit -end -end -``` - -Here’s how you can execute the test from your terminal. - - ruby google-search-lambdatest.rb - -The first thing to note in this code is the use of remote webdriver. To access TestMu AI grid, you would have to invoke remote webdriver instead of local webdrivers. In addition, the browser enthronement details are passed on through desired_capabilities. - -In this example we are running a test on **Windows 10**, **Chrome 67 environment**. You can test your own choice of an environment by changing the parameters in the object. You can leverage our capabilities generator tool to create code for your environments. - -LambdaTest Capability Generator Tool - -> **Tear Down**: The tear down method helps TestMu AI platform understand when the execution of a test is finished. It’s important to tear down each test or else it will give timeout errors. diff --git a/static/docs/rally-integration.md b/static/docs/rally-integration.md deleted file mode 100644 index 826db0d10..000000000 --- a/static/docs/rally-integration.md +++ /dev/null @@ -1,72 +0,0 @@ -# Rally Integration with TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Rally is a powerful cloud-based project management solution that streamlines project tracking, bug management, and team collaboration through its robust customization and automation capabilities. - -The TestMu AI integration enables you to capture and log bugs directly from your test sessions to Rally, complete with screenshots and test environment details, making your testing workflow more efficient. - -> Rally Integration with TestMu AI is available for both freemium and premium plans. - -## Integrate Rally With TestMu AI - -**Step 1:** Login to your TestMu AI account. You should have administrator or user level access. - -**Step 2:** Navigate to `Settings` -> `Integrations` -> `Bug Tracker`. - -**Step 3:** Click on **Install** on the Rally block. - -**Step 4:** Fill in the integration details: - - **Rally API Key** - Required for authentication - - **Host URL** - Required only for on-premises/self-hosted instances - -**How to Generate API Key in Rally** - -- Log in to your Rally workspace -- Go to User Settings → API Keys -- Click "Create New API Key" -- Set permissions to "Full Access" -- Copy the generated API key - -> **Note**: By default, we use `https://rally1.rallydev.com` as host. - -**Step 5:** Click **Install**. TestMu AI will now be integrated with Rally. - -## Log your First Bug - -**Step 1:** Ensure you have access to the appropriate Rally workspace and project. - -**Step 2:** While testing on TestMu AI, if you encounter a bug, click on the **Mark as Bug** button available in the test session toolbar. - -**Step 3:** In the bug logging form, provide the necessary details. The available fields are defined by the defect attributes in your Rally workspace: - -- **Project:** Select the specific project in Rally (required) -- **Owner:** Assign the defect to a team member -- **Name:** Enter a concise summary of the bug (required) -- **Description:** Provide detailed information about the bug - -- any other required fields defined in your rally attribute for defects - -**Step 4:** Click on the **Create Issue** button. The bug will be logged into your Rally, and you can view it directly from your Rally dashboard or the TestMu AI Issue Tracker. - -## Resync Rally Integration - -To ensure that your Rally integration stays up-to-date with the latest workspaces and users: - -**Step 1:** Navigate to `Settings` -> `Integrations` in your TestMu AI account. - -**Step 2:** Locate the Rally integration and click on the **Resync** button. - -This will: -- Fetch the latest projects and data from your Rally -- Update any changes in project configurations - -If your API key has expired, you will be prompted to update your Rally integration credentials. - -## Uninstall Rally Integration - -**Step 1:** Navigate to `Settings` -> `Integrations` in your TestMu AI account. - -**Step 2:** Locate the Rally integration and click on the **Remove** button to uninstall the integration. - -> That was all you need to know for TestMu AI + Rally Integration. Increase your productivity with our integrations. If you still have any questions for us, please feel free to let us know. Our experts are always available on **chat** to help you out with any roadblock. diff --git a/static/docs/ranorex-integration-with-testmu.md b/static/docs/ranorex-integration-with-testmu.md deleted file mode 100644 index ae123ba44..000000000 --- a/static/docs/ranorex-integration-with-testmu.md +++ /dev/null @@ -1,68 +0,0 @@ -# Ranorex Integration With TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Ranorex is a GUI test automation framework which helps to execute E2E(End-to-End) testing of web, desktop, and mobile applications. Ranorex Core API is powered with Selenium WebDriver helping you to maximize your cross browser testing in an efficient manner. - -Selenium, as we know, has 4 variants, RC, IDE, WebDriver, & Grid. [Selenium IDE](https://www.testmuai.com/blog/selenium-ide-what-is-it-why-is-it-must-for-every-qa/) is built with the purpose of facilitating automated record-and-replay testing over Mozilla Firefox browsers. However, ever since Mozilla Firefox released its version 55, the IDE’s compatibility got disrupted and isn’t replaced till now. Ranorex helps you to overcome this scenario by optimizing their Core API in such manner that users of Selenium WebDriver could get to unleash the full potential of all Selenium variants irrespective of browser updates. - -TestMu AI offers integration with Ranorex to help you perform automated cross browser testing of your web app on over 3000+ real browsers & browser versions through an on-cloud Selenium Grid. You could perform tests on your locally hosted files and can execute parallel testing to trim down your test cycles drastically. - -In this documentation, we will demonstrate how to set up your Selenium Server using Ranorex, and how to connect it to TestMu AI cloud-based [Selenium Grid](https://www.testmuai.com/blog/why-selenium-grid-is-ideal-for-automated-browser-testing/). You will be going through the below sections: - -## Getting Started With Ranorex - -**Step 1:**  Open Ranorex & click on **"New solution via wizard"**  from the left menu. - -**Step 2:**  Select the method for **Web testing**  over different browsers. Then hit on the button that says **"Click here"**. - -**Step 3:**  Provide the **Solution name**  & **Location**  where you wish to store the new solution & hit continue. - -**Step 4:**  Specify the test URL and browsers on which you wish to perform cross browser testing. - -**Step 5:**  Select your recording behaviour as **"Do not use whitelisting"**. - -**Step 6:**  And you are done creating your new test automation solution. Now, you can finish the wizard. - -## Setting Up WebDriver Endpoint For TestMu AI Selenium Grid - -**Step 1:**  After successfully creating a solution, you will notice that a default test suite would be configured in Ranorex. - -**Step 2:**  Add the **end point** to run the test suite on Selenium Grid offered by TestMu AI. You can do so by going to the tab for **"Endpoint"**   Select **WebDriver**   Provide an **Endpoint name**  & **Address(Hub URL)**. Click on **Add endpoint**. - ->**Note:**  You can check your connection by clicking on **Test connection**. - -**Step 3:**  You have successfully setup a WebDriver endpoint for TestMu AI Selenium Grid Hub URL. - ->**Note:** You need to select this WebDriver endpoint to perform automated cross browser testing on 3000+ real browsers & browser versions using the TestMu AI Selenium Grid. - -## Performing Cross Browser Testing On TestMu AI Selenium Grid Through Ranorex - -**Step 1:**  After successful addition of TestMu AI Selenium Grid Hub-URL, comes the time to perform cross browser testing. For browser compatibility testing, you would need to open you test suite and select the particular browser you wish to run your test on. - ->**Note:**  In order to run your test on a browser which is not present in the drop-down, you would need to add a new variable for that browser. For example, if you wish to perform cross browser testing on Safari browser then you can implement it with the help of below variables: ->Variable name- Mac ->Variable value – Safari - -**Step 2:**  Now, under the tab **"Endpoint"**, you need to add a **new configuration**  to set up the DesiredCapabilities for TestMu AI Selenium Grid. - -**Step 3:**  Fetch your DesiredCapabilities from TestMu AI [Selenium Capability Generator](https://www.testmuai.com/capabilities-generator/). - -**Step 4:**  Copy the DesiredCapabilities & add them in Endpoint configurations by placing them in **"Capabilities JSON"**. Don’t forget to provide a name as well. - -**Step 5:**  Now, from the drop-down for **Active configuration**, select the latest configuration you added. - -**Step 6:**  Now, go to recording tab & **add new actions**  after clicking the **Record**  button. - ->Keep in mind, to choose the Endpoint as **Admin localhost**  while recording the test cases. - -**Step 7:**  You will find your actions being recorded in the Ranorex interface. - ->**Important:**  Once you are finished recording you actions, TestMu AI recommends you to choose the repository item in the same instance where you recording is halted. This would help you to avoid any time out issue. -> - -**Step 8:**  After you are done configuring the open browser, recording, & close browser file, come back to your test-suite & ensure that you have **WebDriver** set as an Endpoint to run the test on TestMu AI. - -**Step 9:**  Hit the **run button**. You will notice two output screens. One on the right side, is displaying the running test case. And on the left side, you can see that the test ran on the specific configuration that you chose under **Active configuration**  drop-down, along with all the the steps you recorded. - -**Step 10:**  Kudos! You have successfully performed automated cross browser testing on TestMu AI Selenium Grid from your Ranorex instance. You can confirm the text execution on your TestMu AI account. You could also find all the test details such as metadata, network logs, command logs, Selenium logs, command-by-command screenshot, and more on the TestMu AI dashboard. You can also extract these details using our TestMu AI [Selenium Automation API](/api-doc/). Happy testing! diff --git a/static/docs/rbac-roles-and-permissions.md b/static/docs/rbac-roles-and-permissions.md deleted file mode 100644 index 6bf4af03f..000000000 --- a/static/docs/rbac-roles-and-permissions.md +++ /dev/null @@ -1,129 +0,0 @@ -# Roles and Permissions (RBAC) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI **Roles and Permissions (RBAC)** allows organization admins to define fine-grained access control across products, projects, and features. With Custom Roles and Permissions, you can control which team members have access to specific products, what actions they can perform, and ensure that each user only sees what is relevant to their role. - -This is especially useful for large enterprises with multiple teams where different groups need access to different parts of the platform, for example, one team working exclusively with KaneAI and Test Manager, while another only needs access to SmartUI and Automation. - -Custom Roles and Permissions is an enterprise-level feature. Please contact your account manager or reach out to [support@testmuai.com](mailto:support@testmuai.com) to get it enabled for your organization. - -## Prerequisites - -- You must be an **Admin** of your TestMu AI organization. -- Ensure your organization has the RBAC feature enabled by the TestMu AI team. -- Familiarity with [User Management](/support/docs/user-management/) and [Team Management](/support/docs/team-management/) is recommended. - -## Understanding Default Roles - -Navigate to **Organization Settings** > **Custom Roles & Permissions** from the left sidebar. You will see the **Custom Roles & Permissions** page displaying all existing roles in your organization. - -TestMu AI provides three default roles out of the box: - -| Role | Description | Entity Permissions | -|------|-------------|-------------------| -| **Admin** | Full access to all products and entities, plus organization-level administration such as settings, billing, integrations, and user and team management. | List, Read, Create, Update, Delete, Execute on all entities, plus organization administration | -| **User** | Full permissions on all products and entities, but without organization-level administration. | List, Read, Create, Update, Delete, Execute on all entities | -| **Guest** | Read-only access across products. | List, Read only on entities | - -Roles are auto-applied at login. - -When a user is assigned more than one role, their effective permissions are the combination of all assigned roles, and the most permissive access applies. Permission changes take effect on the user's next request or page load, no re-login is required. - -## Creating and Applying Custom Roles - -Custom roles give you the flexibility to tailor access permissions to match your organization's specific needs. Unlike the default roles (Admin, User, Guest), which cannot be edited, custom roles allow you to define exactly which products, entities, and actions each team member can access. To set up custom permissions, you need to create a custom role. - -**Step 1:** On the **Custom Roles & Permissions** page, click the **Create New Role** button in the top-right corner. - -**Step 2:** In the **Create New Role** dialog, enter a **Role Name** that clearly describes the role's purpose (e.g., "SmartUI Tester", "KaneAI Developer", "QA Lead"). - -**Step 3:** You can configure [product-level access](#product-level-access) or entity-level access based on your requirements. Entity-level access is covered in detail below. - -**Step 4:** Click **Create** to complete the process. - -**Step 5:** Assign the custom role to users or teams based on your use case: - -- **To assign to individual users:** Navigate to **Organization Settings** > **Members**, click on the user's row or the options menu, and assign the custom role. - -- **To assign to a team:** Navigate to **Organization Settings** > **Teams**, select a team, and assign the desired custom role. All members of the team will automatically inherit the same permissions. - -### Entity Level Access - -Entity-level access allows you to configure granular permissions for specific items and product areas within the platform. Each product exposes its own set of entities, and you can grant specific permissions on each one. - -Entity-level access is available for the following products: - -| Product | Entities | -|---------|----------| -| **Test Manager** | Projects, Test Runs, Test Cases and Test Case Instances | -| **Automation** | Projects, Builds, Test Case Instances | -| **HyperExecute** | Projects, Workflows, Organization Settings | -| **SmartUI** | Projects, Builds | -| **Analytics & Insights** | Projects (applied to dashboards and reports, read-only) | - -Each product exposes a different set of entities. Here is what each one represents. - -**Test Manager** - -- **Projects**: A workspace that groups all test cases, test runs, and related assets for one application or team. -- **Test Runs**: A grouping of test cases with chosen browser, device, and OS configurations, created to execute and track them together. -- **Test Cases**: Individual test scenarios stored in a centralized repository and reused across multiple test runs. -- **Test Case Instances**: The per-configuration executions of a test case inside a test run, each tracked with its own status. - -**Automation** - -- **Projects**: A container that groups related automation builds and sessions and serves as the unit you scope access to. -- **Builds**: A named grouping of automation test sessions, set through the build capability, used to collect and filter related executions. -- **Test Case Instances**: Individual automation test sessions executed on a specific browser, OS, or device configuration within a build. - -**HyperExecute** - -- **Projects**: A centralized space for organizing and managing test executions by grouping similar tests, controlling access, and tracking progress. -- **Workflows**: Schedule-driven orchestration that runs HyperExecute jobs on a defined schedule, without requiring CI/CD integration. -- **Organization Settings**: Organization-level HyperExecute preferences, such as auto-mute tests and test viewing preferences. - -**SmartUI** - -- **Projects**: A container that groups all visual regression builds and screenshots for a single application. -- **Builds**: A named collection of screenshots grouped for visual comparison and baseline management within a project. - -**Analytics & Insights** - -- **Projects**: The read-only reporting scope that controls which dashboards and reports a user can view. - -Entity-level access for **App Automation** is planned for an upcoming release. The exact set of entities, and the actions available on each, can vary from one product to another. - -#### Select Specific Entities - -When creating or editing a role, you can assign permissions to specific items rather than to all items of a type, for example, granting access only to selected Test Manager projects or to specific HyperExecute projects. - -#### Granular Control - -For supported products, you can configure fine-grained permissions on each entity using the following permission levels: - -- **List**: View items in a list. -- **Read**: View item details. -- **Create**: Create new items. -- **Update**: Modify existing items. -- **Delete**: Remove items. -- **Execute**: Run or trigger items (for example, abort a build or trigger a HyperExecute job). - -Not every permission level applies to every entity. For example, Analytics & Insights is a read-only reporting surface, so only **List** and **Read** apply there. - -## Product Level Access - -Product-level access lets you control which TestMu AI products a user can see and interact with. This is the most impactful RBAC setting as it determines what shows up on a user's dashboard, sidebar, and which URLs they can visit. - -When creating or editing a role, add **List of Products** as an entity and select only the products this role should have access to, such as KaneAI, HyperExecute, SmartUI, and more. - -### What happens when a product is restricted? - -- The product is **hidden from the sidebar** and **dashboard** for the user. -- If a user tries to directly access a restricted product URL, they are redirected to a **403 page**. - -### When to use Product Level Access - -- **Security & compliance**:Restrict sensitive products like Insights or Settings to admins and leads only, ensuring regular testers don't have access to org-level analytics or configurations. -- **Onboarding new members**:New team members can be given a limited role with access to only Real Time and Automation while they ramp up, then gradually expanded to include KaneAI and HyperExecute. -- **Client or vendor access**:External contractors working on a specific product area (e.g., mobile testing) can be restricted to only Real Device and Automation without exposing the rest of the platform. diff --git a/static/docs/real-device-adb-shell.md b/static/docs/real-device-adb-shell.md deleted file mode 100644 index 93558ca84..000000000 --- a/static/docs/real-device-adb-shell.md +++ /dev/null @@ -1,72 +0,0 @@ -# Execute ADB shell commands - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The integration of Android Debug Bridge (ADB) shell support within the TestMu AI Manual Testing platform for app testing on Android devices empowers users to run supported ADB shell commands directly on devices during manual testing sessions, receiving instant output. This feature addresses the challenge of remote device communication, enabling users to interact with devices as if they were physically present. - -## Benefits - -- Unlock new testing scenarios that require direct device interaction, simplifying the development process. -- Speed up testing by easily accessing native device capabilities and APIs, reducing reliance on framework-specific commands. -- Gain better control over device configurations to execute specific commands tailored to your test cases. - -## Accessing ADB Shell Terminal - -**Step 1:** Login to your TestMu AI account and navigate to **App Testing** under **Real Device** from the left sidebar. - -**Step 2:** Choose the OS version (Android), select your application and device from the device listing, and start your test session. - -**Step 3:** Access **ADB Shell** -- Once in the test session, find the **ADB Shell** icon on the top bar. -- Click the ADB Shell icon to open the terminal and start running supported ADB shell commands. -- With a single click, you can clear the terminal content or download the terminal output as a `.txt` file. - -## Supported ADB Commands -To ensure security and compatibility, we have a list of ADB commands that can be executed within our Real Device Cloud. Please refer to the following list of supported commands: - -### Supported ADB Commands - Public Device - -| Command | Description | -| -------------------------------------- | --------------------------------------------------------------------------------- | -| `am start` | Launches an activity on the device. Useful for opening apps or specific screens. | -| `am force-stop` | Force-stops an application by its package name. | -| `am compat enable` | Enables app compatibility options. | -| `cat /sdcard/Download/` | Displays the content of a specific file inside the Download folder. | -| `cat /sdcard/Pictures/` | Displays the content of a specific file inside the Pictures folder. | -| `cat /sdcard/Movies/` | Displays the content of a specific file inside the Movies folder. | -| `cat /proc/version` | Displays kernel version and build info. | -| `dumpsys` | Dumps system service information (battery, activity, memory, etc.). | -| `echo` | Prints text to the terminal. Useful for testing output or scripting. | -| `getprop` | Gets system properties. Often used for device diagnostics. | -| `grep` | Searches for text patterns. Useful for filtering logs or command output. | -| `input` | Simulates user input like taps, swipes, or key events. | -| `log.tag.FA` | Firebase Analytics SDK logs (e.g., event recording, session start). | -| `log.tag.FA-SVC` | Firebase background service logs (e.g., data uploads, scheduled jobs). | -| `ls /sdcard/Download` | Lists all files and folders located in the device's Download folder. | -| `ls /sdcard/Pictures` | Lists all files and folders located in the device's Pictures folder. | -| `ls /sdcard/Movies` | Lists all files and folders located in the device's Movies folder. | -| `pm clear` | Clears all data associated with a package (app). | -| `pwd` | Displays the current working directory. | -| `setprop debug.firebase.analytics.app` | Enables Firebase Analytics debug mode for a specific app. | -| `wm fixed-to-user-rotation` | Controls screen rotation policy. | - -### Supported ADB Commands - Private Device - -All the public commands along with the following: - -| Command | Description | -| -------------------------------------- | --------------------------------------------------------------------------------- | -| `am instrument` | Executes instrumented tests. Often used in automated testing pipelines. | -| `am broadcast` | Sends custom broadcast messages for triggering in-app actions. | -| `am` | All other app manager commands. | -| `cmd connectivity airplane-mode` | Toggles airplane mode on/off (requires permissions). | -| `logcat` | Streams real-time logs from device, including app logs. | -| `ls` | Lists files and directories. Commonly used to inspect folders like `/sdcard/Download`. | -| `mkdir` | Creates a new directory. | -| `pm` | All other package manager commands. | -| `rm` | Removes files or directories. | -| `setprop` | Sets custom system properties (e.g., for debugging frameworks). | - -`adb shell` is already pre-applied in our **terminal environment**. Rewriting it explicitly may prevent the command from executing properly. - -> If the command you require is not listed, please reach out at our 24x7 Chat Support or you could also mail us at support@testmuai.com. diff --git a/static/docs/real-device-app-testing.md b/static/docs/real-device-app-testing.md deleted file mode 100644 index c64f8159f..000000000 --- a/static/docs/real-device-app-testing.md +++ /dev/null @@ -1,25 +0,0 @@ -# Getting Started with Real Device App Testing - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Real device app testing is the process of testing a mobile application to ensure that their functionality and usability is not comprised when installed across multiple Android and iOS devices. With TestMu AI, you can test mobile applications manually to ensure they perform seamlessly across all Android emulators and iOS simulators. - -**Step 1:** Login to your TestMu AI account. Go to "**App Testing**" under “**Real Device**” from the left navigation menu. - -**Step 2**: Choose the OS version of your choice. ( Android or iOS ) - -**Step 3**: You can upload an application directly from your files or by entering a URL. To select an app, simply scroll through the uploaded files or use the search bar to find a specific application. The uploading time may vary depending on the size of the application. - -In the case of **Android**, you can install from Play Store or App Center. - -In the case of **iOS**, you can install from the App Store, install from Test Flight or from App Center. - -**Step 4**: You can select your preferred region from the **Region** dropdown. Choose from the available regions i.e., North America, Europe, Asia, or select All. If you would like to choose devices from specific regions, please reach out to [support@testmuai.com](mailto:support@testmuai.com) to get the option enabled. - -**Step 5**: Select the **Tunnel** option in the top right corner (optional) to test your locally hosted and privately hosted websites. - -For detailed instructions on how to activate the tunnel, please refer to the detailed [support documentation](/support/docs/testing-locally-hosted-pages/). - -**Step 6**: Select the preferred **Brand**,**Device Name** and **OS Version** to easily navigate to your required device. You can also use the search bar to select your devices. - -**Step 7**: Click on **Start** to execute your test. diff --git a/static/docs/real-devices-integration-with-hyperexecute.md b/static/docs/real-devices-integration-with-hyperexecute.md deleted file mode 100644 index fe4dc6167..000000000 --- a/static/docs/real-devices-integration-with-hyperexecute.md +++ /dev/null @@ -1,47 +0,0 @@ -# Real Devices Integration with HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Till now you have executed your test scripts on the [Real Device Automation](/support/docs/appium-python-pytest/). This Document will not only explain to you about how to use Real Devices on HyperExecute but also the superpower you just unlocked with HyperExecute - Test Orchestration and Intelligence. - -HyperExecute is a smart test orchestration platform to run end-to-end tests at the fastest speed possible. HyperExecute lets you achieve an accelerated time to market by providing a test infrastructure that offers optimal speed, test orchestration, and detailed execution logs. - -**Let's first take a look at the different ways of executing tests on HyperExecute:** - -## 1. Triggering a job on HyperExecute -In order to trigger a job on HyperExecute, it requires the [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) and a set of instructions inside the [HyperExecute Yaml](/support/docs/deep-dive-into-hyperexecute-yaml/) file. When you trigger the CLI command, it connects with the HyperExecute APIs which then distributes the tests across different machines/nodes for parallel execution as shown in the diagram below: - -## 2. Real Devices Test Execution on HyperExecute -In case of Real devices test execution the test scripts from the machines run the actual tests on connected real devices as shown in the diagram below: - -## 3. Triggering through local machine -When you are triggering your test cases from the local machine the flow of data is as shown in the diagram below: - -## 4. Triggering through Jenkins -In order to execute a job on HyperExecute via Jenkins, you would need to perform 2 simple steps : - -You can read more about it [here](/support/docs/hyperexecute-integration-with-ci-cd-tools/). When you want the job to be initiated from the Jenkins Pipeline, it would look as shown in the diagram below: - -## 5. How to Trigger Real Device tests on HyperExecute ? -You can seamlessly trigger your favorite mobile app framework test scripts with HyperExecute. - - Appium Testing for Apps - Test your native Android and iOS mobile applications on Appium automation cloud. - - Appium Testing for WebApps - Test your Android and iOS web applications on Appium automation cloud. - - Espresso Testing - Test your Android mobile applications on Espresso automation cloud. - - Maestro Testing - Test your Android mobile applications on Maestro automation cloud. - - XCUI Testing - Test your iOS applications on XCUI automation cloud. - -## Support - -We at TestMu AI take pride in our Support system. We ensure that we do the heavy lifting while our customers can focus on the test scripts and strategy. As a first step in the POC, our support engineers conduct a working session over Zoom or Gmeet to understand your use case and provide you with the right configuration to execute your test cases on HyperExecute. Our support engineers are always available over slack, mail and calls to ensure that we provide you with the right kind of support and set you up for a win.  - -Please feel free to give us a shout if you have any questions or require any assistance. diff --git a/static/docs/real-time-desktop-browser-testing-tools.md b/static/docs/real-time-desktop-browser-testing-tools.md deleted file mode 100644 index 2ea8bde22..000000000 --- a/static/docs/real-time-desktop-browser-testing-tools.md +++ /dev/null @@ -1,53 +0,0 @@ -# How to use Testing Tools for Desktop Browsers? - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI offers a comprehensive suite of real-time desktop web browser testing tools designed to enhance the efficiency and accuracy of your testing process. This guide provides detailed instructions on how to utilize each tool to its fullest potential. - -## Screenshot -Capture high-quality screenshots of the current screen on your test device instantly. Screenshots are essential for documenting bugs, UI issues, and other anomalies during testing. Click on the Screenshot icon in the sidebar to take a snapshot of the current screen and will automatically start downloading in your local machine and it will be saved in the [gallery](/support/docs/real-time-desktop-browser-testing-tools/#gallery) as well. - -## Record Session -Record your entire test session for detailed analysis and playback. Click on the Record Session button to begin recording. A timer will display the recording duration, helping you pinpoint specific moments during playback. To end the recording, click the Stop button. The recorded video will be saved automatically in the [gallery](/support/docs/real-time-desktop-browser-testing-tools/#gallery) for review. - -## Gallery -The Gallery stores all screenshots and video recordings from your test sessions in one place, allowing you to easily review, download, or manage your media files. To access, select Gallery from the left toolbar and you'll find the saved screenshots and videos. To download the screenshots or videos to your local machine, click on the Download icon. You can also mark them as bug by clicking on the Bug icon. - -## Mark as Bug -This feature allows you to mark any bug in your UI during the session. You can mark the bug in your UI via different shapes, texts and of different colors. You can also configure different project management tools with your session to create a ticket for the same. You can also share that bug, save the image to the gallery or download that screenshot. - -## Resolution -Optimize your tests for different screen sizes and resolutions to ensure your web application looks and functions correctly on all devices. Adjust the resolution settings to simulate how your website will appear on different displays. - -## Scaling -The Scaling feature allows you to adjust the display scale of your web application during testing to simulate different zoom levels or device pixel ratios. This helps ensure your web application looks and functions correctly at various zoom levels, as users might have different display settings on their devices. - -## IP Geolocation -Simulate website and mobile experiences from over 45 countries using local IP addresses. -- **Why Use IP Geolocation? :** Test localized features such as language preferences, currency, regional product offerings, and content visibility based on a specific location. -- **How to Enable? :** Use the IP Geolocation feature to verify that your app's traffic originates from a specific country, providing a comprehensive global testing approach. - -## Files and Media -The [Upload and Download Files](/support/docs/real-time-upload-and-download-files/) feature allows to manage files directly within remote testing environments. This functionality streamlines the testing process by allowing seamless file transfers between local and remote environments across desktop and mobile platforms. - -## Chrome Extension -Enhance your testing capabilities with TestMu AI by leveraging various Chrome extensions that provide additional functionalities and streamline your testing process. These extensions can help automate repetitive tasks, assist in accessibility checks, and integrate AI tools into your workflow. - -## Screen Reader -Ensure your web applications are accessible to all users, including those with disabilities, using TestMu AI's accessibility testing tools. This tool helps identify and address potential accessibility issues to enhance your website's usability and compliance with accessibility standards like WCAG (Web Content Accessibility Guidelines). - -## Settings -Customize your test environment with various settings: - -- **Idle Timeout:** Set the idle timeout duration for your test session (default is 5 minutes, maximum is 60 minutes). -- **Time Zone:** Select the time zone that aligns with your testing needs to ensure accurate testing across different regions. -- **Keyboard Input:** Change the keyboard language during the test session for localized testing scenarios. - -## Switch -Efficiently switch between different devices, browsers, and their OS versions without ending the current test session. - -## On Screen Keyboard -This allows testers to simulate keyboard input directly on the virtual devices during real-time testing sessions. This feature is particularly useful when testing on devices with different keyboard layouts or when physical keyboard access is limited or unavailable. - -## End Session -Conclude your testing session by clicking on the End Session button. diff --git a/static/docs/real-time-locally-hosted-pages.md b/static/docs/real-time-locally-hosted-pages.md deleted file mode 100644 index dd58699f3..000000000 --- a/static/docs/real-time-locally-hosted-pages.md +++ /dev/null @@ -1,13 +0,0 @@ -# Locally Hosted Pages - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The [TestMu AI Tunnel](/support/docs/testing-locally-hosted-pages/) feature enables you to perform secure and real-time testing of locally hosted websites, private servers, and web applications that are behind a corporate firewall. By creating a secure connection between your local environment and the TestMu AI cloud, the Tunnel allows you to access and test internal applications on various browsers and devices. - -## Use Cases -- **Testing Locally Hosted Websites :** Useful for validating websites or web applications still in the development phase that are hosted on a local machine or development server. -- **Testing Staged Applications :** Ideal for testing applications hosted in staging environments that are not yet deployed to production. -- **Testing Web Apps Behind a Corporate Firewall :** Allows you to securely access and test applications that are protected by a corporate firewall. -- **End-to-End Testing in CI Environments :** Facilitates end-to-end testing scenarios that require access to local databases, APIs, or other internal services. - -> 📕 Learn more about the [TestMu AI Tunnel](/support/docs/testing-locally-hosted-pages/) from the documentation. diff --git a/static/docs/real-time-mobile-app-testing-tools.md b/static/docs/real-time-mobile-app-testing-tools.md deleted file mode 100644 index 099bf614e..000000000 --- a/static/docs/real-time-mobile-app-testing-tools.md +++ /dev/null @@ -1,56 +0,0 @@ -# How to use Testing Tools for Mobile Applications? - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI offers a comprehensive suite of real-time mobile app testing tools designed to enhance the efficiency of your testing process. This guide provides detailed instructions on how to utilize each and every tool. - -## Device Controls -The Device Controls feature allows you to simulate various actions on your test device during a testing session, enabling comprehensive testing of your web applications' behavior in real-world scenarios. - -- **Volume Control:** Adjust the device volume directly within your test session to test audio-related features, such as media playback or notifications, ensuring they function correctly at different volume levels. -- **Lock Device:** Seamlessly lock or unlock the device during your testing session without interrupting your workflow. This is useful for verifying app behavior upon device unlocking or handling lock-screen notifications. -- **Shake Device:** Simulate a device shake to test functionalities that rely on shake gestures, such as undo actions or triggering specific in-app events. -- **Rotate Device:** Rotate the device's orientation between portrait and landscape modes to test the responsiveness and adaptability of your web application. This feature helps you ensure your application layout adjusts properly and remains user-friendly across different orientations. - -## App Controls -The App Controls feature provides programmatic control over the applications installed on the emulator or real device: - -- **Install New App:** Programmatically install new applications on the test device without manual intervention. -- **Kill Existing App:** Terminate running apps to test scenarios like crash recovery, app restart, or handling unexpected closures. -- **Uninstall App:** Uninstall the app to validate installation flows, or clean up the environment between tests. - -## Screenshot -Capture high-quality screenshots of the current screen on your test device instantly. Screenshots are essential for documenting bugs, UI issues, and other anomalies during testing. Click on the Screenshot icon in the sidebar to take a snapshot of the current screen and will automatically start downloading in your local machine and it will be saved in the [gallery](/support/docs/real-time-mobile-app-testing-tools/#gallery) as well. - -## Record Session -Record your entire test session for detailed analysis and playback. Click on the Record Session button to begin recording. A timer will display the recording duration, helping you pinpoint specific moments during playback. To end the recording, click the Stop button. The recorded video will be saved automatically in the [gallery](/support/docs/real-time-mobile-app-testing-tools/#gallery) for review. - -## Gallery -All screenshots and video recordings from test sessions are stored in the Gallery. To access, select Gallery from the left toolbar and you'll find the saved screenshots and videos. To download the screenshots or videos to your local machine, click on the Download icon. You can also mark them as bug by clicking on the Bug icon. - -## Mark as Bug -This feature allows you to mark any bug in your UI during the session. You can mark the bug in your UI via different shapes, texts and of different colors. You can also configure different project management tools with your session to create a ticket for the same. You can also share that bug, save the image to the gallery or download that screenshot. - -## IP Geolocation -The IP Geolocation feature enables you to simulate website and mobile experiences from over 45 countries by using local IP addresses. This allows you to test localized features such as pricing, languages, and product offerings. With IP Geolocation enabled, you can verify that your app's traffic is coming from a specific country, allowing for comprehensive global testing. - -## GPS Location -GPS Geolocation testing allows you to simulate specific device locations using exact GPS coordinates to evaluate location-based app scenarios. You can enter your preferred location by entering the name of the location. However, if your preferred location is not showing, you can specify the exact location by entering the latitude and longitude of the location. - -## Files and Media -The [Upload and Download Files](/support/docs/real-time-upload-and-download-files/) feature allows to manage files directly within remote testing environments. This functionality streamlines the testing process by allowing seamless file transfers between local and remote environments across desktop and mobile platforms. - -## Network Throttling -TestMu AI's network throttling feature empowers you to simulate real-world network conditions (2G, 3G, LTE) during testing. This allows you to assess your application performance under varying internet speeds, ensuring a seamless user experience across different network environments. You can leverage predefined network profiles or craft custom settings for comprehensive testing. - -## Settings -The settings section offer the following features: -- **Idle Timeout :** This feature allows you to set the idle timeout for your test session. The default Idle Timeout is 5 minutes and the maximum limit is 60 minutes. -- **Time Zone :** With this feature, you can select the time zone of your preferred region that align with your testing needs. -- **Keyboard Input** With this feature, you can change the language in your test session for a more localized testing experience. - -## Switch -Easily switch between different devices, browsers, and OS versions without ending your current test session. - -## End Session -By clicking on the end session button, you can close your current test session. diff --git a/static/docs/real-time-mobile-browser-testing-tools.md b/static/docs/real-time-mobile-browser-testing-tools.md deleted file mode 100644 index aab9e63e4..000000000 --- a/static/docs/real-time-mobile-browser-testing-tools.md +++ /dev/null @@ -1,50 +0,0 @@ -# How to use Testing Tools for Mobile Browsers? - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI offers a comprehensive suite of real-time mobile web browser testing tools designed to improve your testing experience. This guide provides detailed instructions on how to use each of the tools. - -## Device Controls -The Device Controls feature allows you to simulate various actions on your test device during a testing session, enabling comprehensive testing of your web applications' behavior in real-world scenarios. - -- **Volume Control:** Adjust the device volume directly within your test session to test audio-related features, such as media playback or notifications, ensuring they function correctly at different volume levels. -- **Lock Device:** Seamlessly lock or unlock the device during your testing session without interrupting your workflow. This is useful for verifying app behavior upon device unlocking or handling lock-screen notifications. -- **Shake Device:** Simulate a device shake to test functionalities that rely on shake gestures, such as undo actions or triggering specific in-app events. -- **Rotate Device:** Rotate the device's orientation between portrait and landscape modes to test the responsiveness and adaptability of your web application. This feature helps you ensure your application layout adjusts properly and remains user-friendly across different orientations. - -## Screenshot -Capture high-quality screenshots of the current screen on your test device instantly. Screenshots are essential for documenting bugs, UI issues, and other anomalies during testing. Click on the Screenshot icon in the sidebar to take a snapshot of the current screen and will automatically start downloading in your local machine and it will be saved in the [gallery](/support/docs/real-time-mobile-browser-testing-tools/#gallery) as well. - -## Record Session -Record your entire test session for detailed analysis and playback. Click on the Record Session button to begin recording. A timer will display the recording duration, helping you pinpoint specific moments during playback. To end the recording, click the Stop button. The recorded video will be saved automatically in the [gallery](/support/docs/real-time-mobile-browser-testing-tools/#gallery) for review. - -## Gallery -All screenshots and video recordings from test sessions are stored in the Gallery. To access, select Gallery from the left toolbar and you'll find the saved screenshots and videos. To download the screenshots or videos to your local machine, click on the Download icon. You can also mark them as bug by clicking on the Bug icon. - -## Mark as Bug -Highlight and report any bugs found during your testing session using the Mark as Bug feature. - -- **How to Mark a Bug:** Use various shapes (rectangles, circles), text annotations, and colors to highlight bugs directly on the screen. -- **Integration with Project Management Tools:** Create bug tickets directly from the test session by integrating with tools like Jira, Trello, and Asana. -- **Save and Share:** Save the marked image to the gallery, download it for local use, or share it directly with your team. - -## IP Geolocation -The IP Geolocation feature enables you to simulate website and mobile experiences from over 45 countries by using local IP addresses. This allows you to test localized features such as pricing, languages, and product offerings. With IP Geolocation enabled, you can verify that your app's traffic is coming from a specific country, allowing for comprehensive global testing. - -## Files and Media -The [Upload and Download Files](/support/docs/real-time-upload-and-download-files/) feature allows to manage files directly within remote testing environments. This functionality streamlines the testing process by allowing seamless file transfers between local and remote environments across desktop and mobile platforms. - -## Network Throttling -TestMu AI's network throttling feature empowers you to simulate real-world network conditions (2G, 3G, LTE) during testing. This allows you to assess your application performance under varying internet speeds, ensuring a seamless user experience across different network environments. You can leverage predefined network profiles or craft custom settings for comprehensive testing. - -## Settings -The settings section offer the following features: -- **Idle Timeout :** This feature allows you to set the idle timeout for your test session. The default Idle Timeout is 5 minutes and the maximum limit is 60 minutes. -- **Time Zone :** With this feature, you can select the time zone of your preferred region that align with your testing needs. -- **Keyboard Input** With this feature, you can change the language in your test session for a more localized testing experience. - -## Switch -This feature allows you to switch between devices, browser and their OS versions without exiting your current test session. - -## End Session -By clicking on the end session button, you can close your current test session. diff --git a/static/docs/real-time-recents-and-favourites.md b/static/docs/real-time-recents-and-favourites.md deleted file mode 100644 index b793039b7..000000000 --- a/static/docs/real-time-recents-and-favourites.md +++ /dev/null @@ -1,14 +0,0 @@ -# Recents and Favourites - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The **Recents** and **Favourites** feature, located under the Quick Links section, helps users easily access frequently used resources. This functionality provides an easy way to revisit recently accessed items or mark important resources for quick retrieval. - -## Accessing the "Recents and Favourites" -The Recents tab displays a list of items that you have accessed recently. This helps you quickly return to resources without needing to search for them. To add an item to your favourites, click the **star** icon next to it in the Recents tab. The "Favourites" tab shows all items you have marked as favourites, allowing quick access to frequently used resources. - -- Navigate to the **Quick Links** section in the sidebar of the Real Time Web Browser or App Testing. -- Select **Recents** to view the list of the recently created Browser or App tests. -- Switch the tab to **Favourites** to view items that have been marked as favorites for quick access. - -> You can also re-launch those tests from the recents or favorites section. diff --git a/static/docs/real-time-testing-troubleshooting.md b/static/docs/real-time-testing-troubleshooting.md deleted file mode 100644 index 6d9879756..000000000 --- a/static/docs/real-time-testing-troubleshooting.md +++ /dev/null @@ -1,19 +0,0 @@ -# Real Time Testing Troubleshooting - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Real-Time Testing allows you to perform live-interactive cross browser testing of your website on more than 3000+ real browsers & browser versions running on various operating systems in the cloud. TestMu AI provides you real-time testing experience by virtual machines hosted on our cloud servers. - -Click here to know more about [Real Time Testing at TestMu AI](/support/docs/getting-started-with-desktop-browser-real-time-testing/) - -We have observed that in specific scenarios, users face issues while connecting to our Selenium Grid. This document will help you troubleshoot such scenarios. - -## Why Real-Time Testing Is Failing? - -It might be possible the WebSockets are either blocked on your network or on your local browsers, which are used to access the TestMu AI platform. You can validate if the website is blocked using the below link: - -[**https://www.websocket.org/echo.html**](https://www.websocket.org/echo.html) - -In case your website is blocked, we request you to ask your network admin to open up the web sockets for the TestMu AI platform. - -> For better performance of the TestMu AI platform, we recommend using the latest Google Chrome browser version to access the platform. diff --git a/static/docs/real-time-upload-and-download-files.md b/static/docs/real-time-upload-and-download-files.md deleted file mode 100644 index abc11552c..000000000 --- a/static/docs/real-time-upload-and-download-files.md +++ /dev/null @@ -1,79 +0,0 @@ -# Upload and Download Files - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The Upload and Download Files feature allows you to manage files directly within remote testing environments. This functionality enhances the testing process by allowing seamless file transfers between local and remote environments across desktop and mobile platforms. - -## Key Benefits - -- Direct file upload to remote testing environments -- Cross-platform file management support -- Integrated download capability for test outputs -- Seamless integration with existing manual testing workflows - -## Steps to Upload or Download the Files - -**Step 1 :** Log in to the [Web Browser Testing](https://www.testmuai.com/login/?redirectTo=https://app.lambdatest.com/console/realtime/browser/desktop) dashboard. - -**Step 2 :** Select your required OS and relevant browser, and device from the listing to launch your test session. - -**Step 3 :** Click on Files and Media from the toolbar. - -**Step 4 :** Click **Uploads / Downloads** button Upload or Downlaod the file. - -**Step 1 :** Log in to the [Virtual Mobile](https://www.testmuai.com/login/?redirectTo=https://app.lambdatest.com/console/realtime/browser/mobile) dashboard from the **Web Browser Testing** section. - -**Step 2 :** Select your required OS and relevant browser, and device from the listing to launch your test session. - -**Step 3 :** Click on Files and Media from the toolbar. - -**Step 4 :** Click **Uploads / Downloads** button Upload or Downlaod the file. - -**Step 1 :** Log in to the [Virtual Mobile](https://www.testmuai.com/login/?redirectTo=https://app.lambdatest.com/console/realtime/app) dashboard from the **App Testing** section. - -**Step 2 :** Select your OS and the required device, upload your app and then launch your test session. - -**Step 3 :** Click on Files and Media from the toolbar. - -**Step 4 :** Click **Uploads / Downloads** button Upload or Downlaod the file. - -## Supported Platform Compatibility - -- **Supported Versions :** Android 7 and above. - -#### File Upload Path -- **Android 10 and above :** Uploaded files will be accessible through the file picker in the "Downloads" folder. -- **Below Android 10 :** Files will be uploaded to the "Downloads" folder due to the absence of a "Documents" folder. - -- **Supported Versions :** iOS 13 and above. -- **Upcoming Support :** iOS 18 support is planned for a future release. -- **Upload Location :** Files can be accessed from the "Downloads" section in the Files app. - -#### Browser-Specific Download Locations: -- **Firefox :** Files are saved in the "Firefox" folder. -- **Chromium Browsers (e.g., Chrome, Edge) :** Files are stored in the "Chromium" folder. -- **Safari :** Files are downloaded to the "Downloads" folder. - -- **Supported Versions :** Windows versions newer than XP and Windows 7 with Internet Explorer 8, 9, or 10. -- **Download Location :** Files are stored in the C:/ or E:/Downloads folder. -- **Upload Location :** Uploaded files appear in the "Downloads" folder. - -- **Supported Versions :** macOS Sierra (version 10.12) and above. -- **Download Location :** Downloaded files are accessible under "Downloads" in Finder. -- **Upload Location :** Files can be found in the "Downloads" folder after uploading. - -## File Types and Sizes - -| File Type | Supported Formats | Maximum File Size | -|-----------|-------------------|-------------------| -| Image | JPG, JPEG, PNG, GIF, BMP | 15 MB | -| Video | MP4, MOV, 3GP | 15 MB | -| Non-media | XLS, XLSX, DOC, DOCX, PDF, CSV, TXT, PPT, Contacts | 15 MB | - -| File Type | Supported Formats | -|-----------|-------------------| -|File Size | The maximum total file size supported for download is 50 MB. The combined size of all the files you want to download must not exceed 50 MB. | - -- **Number of File Uploads per Session :** You can upload 5 files max per testing session. - -- **File Storage Duration :** All files uploaded to TestMu AI servers, whether media or non-media, will be retained for a period of 30 days. After this period, the files will be automatically purged from the servers. diff --git a/static/docs/record-and-replay-tests.md b/static/docs/record-and-replay-tests.md deleted file mode 100644 index 97e852dd4..000000000 --- a/static/docs/record-and-replay-tests.md +++ /dev/null @@ -1,34 +0,0 @@ -# Record and Replay Tests using LT Debug 2.0 - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -With LT Debug 2.0, you now have the ability to record user interactions through the Chrome Recorder Panel and replay or retest them as automated tests on a cloud grid on over 3000+ real browsers and operating system combinations. This eliminates the need to download code or manually run tests on your local machine, thus streamlining and optimizing your testing process. - -## Prerequisites - -1. Install the [LT Debug Chrome Extension](https://chrome.google.com/webstore/detail/lt-debug/kofahhnmgobkidipanhejacffiigppcd). -2. Login to your TestMu AI Account via LT Debug Dashboard. Don't have an account, sign up for free. - -## Recording User Interactions with Chrome Recorder - -1. To initiate a recording in Chrome DevTools, first, open the developer tools panel by inspecting the webpage. - -2. Once the panel is open, navigate to the **Recorder** tab. - -3. From there, select **Create a new recording**. - -4. Now click **Start recording** to begin capturing user interactions on the page. - -5. Once you have completed recording the user flow in Chrome DevTools, you can end the recording process by clicking on **End recording**. - -## Recording and Replaying Tests on TestMu AI - -After ending the recording process in Chrome DevTools, you will notice a TestMu AI button appear on the screen. - -1. To replay or retest the recorded user flow on TestMu AI, simply click on the TestMu AI button. - -2. Configure your test capabilities like Browser/OS Version, Build Name, and Test Name, etc, on which you wish to run the recorded interactions, and click **Start Test**. - -3. Once the test successful runs, it will generate the test IDs and a link to TestMu AI Automation Dashboard. - -4. To view your record automated test results, visit the [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build). diff --git a/static/docs/redmine-integration.md b/static/docs/redmine-integration.md deleted file mode 100644 index a8e9a93e4..000000000 --- a/static/docs/redmine-integration.md +++ /dev/null @@ -1,79 +0,0 @@ -# Redmine Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - ->Redmine is an open-source, cross-platform, and cross-database project management tool. It is coded in Ruby on Rails, and it helps in collaboration, bug tracking, managing multiple projects, and allows users to have numerous roles. It has an issue tracking system that enables developers to identify each issue status and prioritize it accordingly. Besides, it has additional features such as email notification, feeds, wikis, forums, and time tracker. - -Using TestMu AI integration with Redmine, you'll be able to log bugs in your Redmine project from TestMu AI in just one click. You can grab a screenshot of a problem, highlight an exception, annotate additional detail, select an assignee, include a fitting title and a summary to help convey further actions. You can do all of this directly from TestMu AI. - -## How to Integrate Redmine with your TestMu AI account? - -**Step 1:** Login to your TestMu AI account. You would need to have Admin or User level access to see and install integrations. - -**Step 2:** Select **'Integration'** from the left navigation menu bar. This will guide you to a screen where you will find a list of third party applications, available to integrate with your TestMu AI account. - -**Step 3:** Click on **'Add'** under the block that says **'Redmine'**. - -**Step 4:** Once you click on add, you will be guided to the screen below. You will be prompted to enter your **'Domain'** and **'API token'**. - ->* ** What is Domain?** -The domain name is any website's name having unique IP address that the user enters in the URL tab. It helps you recall the name of the website. You can select a domain name based on your organization's work area. -> ->* ** What is an API Token?** -When two different applications communicate, a server-to – server connection is established. The API token is used to authenticate user identity from one server to another by extracting user-owned resources accessible on the server from where the communication is initiated. The API token is required to preserve the protection and integrity of any application. They're just as valuable to the end user as they are to the application. API tokens are important if the end user logs to the application via a two step authentication process. ->* **Where can I get my API Token?** -Make sure you have your Redmine credentials. If not registered, you can register yourself by navigating to the **'Register'** button on the top-left, then login to your Redmine account and click on **'My Account'** on the top-left. Navigate to **'Show'** under **'API access key'** header on the left. Copy this API access token as you will need to provide it on the TestMu AI application to integrate with Redmine. This API token is auto-generated though you can regenerate it as per your need. -> - -**Step 5:** Then enter your **'Domain'** and paste the **'API Token'** into the field provided at TestMu AI & press the **'Install'** button. - -**Step 6:** Go to Integrations again and you will be able to notice a green tick indicating that Redmine is successfully installed. You are all set to experience one-click bug logging to share your issues directly from your TestMu AI account to your teammates on Redmine project. - -## How to Mark your First Bug through Redmine Integration? - -**Step 1:** In order to log bugs from your TestMu AI, firstly you would have to create a Project in the Redmine environment by navigating to **'Projects>New Project>Create'**. - -**Step 2:** Go for any of the test from the TestMu AI's left navigation menu. For demo, we will be taking **'Real Time Testing'** option. After that, enter a URL of the web-app you need to test in the dialog box. Then, select any configuration for browser and operating system of your choice & click on **'Start'**. - -**Step 3:** After the VM is launched and operable. You can perform testing on your web-app for finding bugs. If a bug gets revealed, then you need to click on the **'Bug icon'** from the left panel for capturing a screenshot of the same. - -**Step 4:** After a screenshot is captured, you can annotate any issue or a task with an **in-built image editor**. Once you are done highlighting the bug, click on the button that says **'Mark as Bug'**. - -**Step 5:** After clicking on **'Mark as Bug'** button a form would open up. Fill the fields as per your requirement. -> -* You can select the name of your **'Project'**. -* You can choose the **'Assignee'** to whom you would assign your project. -* You can assign the **'Priority'** of your task. -* You can give suitable **'Subject'** for your test. -* You also get to post a **'Description'** to help relate the cause of the issue. -* You can select the issue **‘Status’** of the project. - -**Note**: To mark bugs, you will need to change one of the your statuses to Default. To do this, navigate to **Administration > Issue Statuses**. - -**Step 1**: Click on the status that you wish to change to Default. - -**Step 2**: Mark the **‘Default value’** checkbox as checked and click on **‘Save’**. - -**Step 3**: After clicking the save button, you’ll notice a green tick that notifies your selected status is now set as default. - -**Step 6:** Click on **'Create Issue'** & observe it being successfully marked through a single click effort. Once you press the button, you will get **prompt messages** on top of your Virtual Machine indicating the progress of bug logging. You will get a prompt message in few seconds mentioning **Bug successfully marked** indicating that the card has been generated in your dashboard. - -**Step 7:** Visit your Redmine dashboard by navigating to **'Home'** button on the top-left. Select your project and you will be able to notice the logged issue right away in the **'Issue tracking'** section. - -**Step 8:** To view the log bug details, navigate to **'Issues'**. - -**Step 9:** Then click on the subject's name by navigating to **'Subject'**, that you have assigned while creating your ticket. - -You will find all the data that you provided through TestMu AI already presented in it. TestMu AI automatically includes test environment details and related screenshots in the card as attachments. - -## How to Remove Redmine Integration? - ->You can work with one integration at a time. So if you would want to integrate to a similar third party application, then you would have to **remove** your current integration. Here is how you can do that. - -**Step 1:** Login to your TestMu AI account. - -**Step 2:** Select **'Settings'** from the left navigation menu bar & click on **'Integrations'**. This will guide you to a screen where you will find 3rd party applications, available to integrate with your TestMu AI account. - -**Step 3:** Click on **'REMOVE'**. You can find the remove button right under the **'Redmine'** block which would be highlighted with a green tick. - ->That was all you need to know for TestMu AI + Redmine Integration. Increase your productivity with our integrations. If you still have any questions for us, please feel free to let us know. Our experts are always **available on chat** to help you out with any roadblock regarding our product. Happy testing! diff --git a/static/docs/regular-expression-appium.md b/static/docs/regular-expression-appium.md deleted file mode 100644 index a3ed6fd73..000000000 --- a/static/docs/regular-expression-appium.md +++ /dev/null @@ -1,40 +0,0 @@ -# Regular Expression - Appium - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Regular Expressions (RegEx) are powerful tools for searching and manipulating strings. When running tests on TestMu AI with a specific device, there may be times when the exact device you selected isn't available. In such cases, RegEx will help you widen your search criteria to find any available device that matches your requirements. - -TestMu AI provides RegEx support for both **deviceName** and **platformVersion**, making it easier to find the right devices for your tests. - -**Note:** Regular Expression support is currently available **only for App Automation**. -It is **not supported for Web Automation on Virtual Devices** at this time. - -## Why Use RegEx? - -Imagine you want to run a test on an iPhone. You don't need a specific model, just any iPhone. Using **RegEx**, you can specify this requirement, and TestMu AI will allocate any available iPhone. This flexibility can significantly broaden the search and increase the chances of finding a suitable device quickly. - -## Regex Characters for `deviceName` - -| REGEX CHARACTERS | DESCRIPTION | EXAMPLE | -|------------------|-------------|---------| -| `.*` | The characters `.*` are used to include all the devices that match the string passed.**In the example given**:You'll be allocated any device from the Inventory that's an iPhone or a Pixel device respectively.See detailed examples for using this RegEx here:a) **For iPhone**: [regex101 example](https://regex101.com/r/4BOgRs/1)b) **For Pixel**: [regex101 example](https://regex101.com/r/7dovT2/1) | `"(iPhone.*)"``"(Pixel.*)"` | -| `,` | This character `,` is used to include more combinations of different devices in a single expression.You can also use `OR` operator in these cases. **In the example given**:1. We have added 2 regex characters `.*` and `,` to show that using both in combination, we'll fetch any device from the Inventory that contains the string Pixel, Nexus, Galaxy.2. You'll be allocated any available Pixel or Galaxy device.| `"(Pixel.*),(Galaxy.*)"` | -| `[]` | The `[]` is used to match special/reserved characters such as `+`,`.`,`(`,`)` etc. in the device name search. Not using the `[]` may treat these characters as regex characters.**In the example given**:1. You'll be allocated any iPad Pro 12.9 (2022).2. In case you send `"iPad Pro 12.9 (2022).*"` you will not get the desired allocated device.| `"iPad Pro 12[.]9 [(]2022[)].*"` | -| `[]` | The `[]` is used to include a range of devices in the combination mentioned using a **single character only**.**In the example given**:Fetching any device available from Pixel 3, Pixel 3a, Pixel 4, Pixel 5, Pixel 6, Pixel 6 Pro, etc.See detailed examples for using this RegEx here:[regex101 example](https://regex101.com/r/H4IvpF/1) | `"(Pixel [3456].*)"` `"(Pixel [3-6].*)"` | -| `$` | The `$` character indicates the end of a string. **In the example given**:1. Allocating only Pixel 6 device & will neglect the Pixel 6 Pro devices.2. Allocates only Pixel 3 or Pixel 6 rejecting others like Pixel 3a, Pixel 6 Pro etc.See detailed examples for using this RegEx here:a) **For Pixel 6**: [regex101 example](https://regex101.com/r/8qYuq6/1)b) **For Pixel 6 & 3**: [regex101 example](https://regex101.com/r/9p7sqe/1) | `"(Pixel 6$)"``"(Pixel [36]$)"` | -| `^` | The `^` character serves multiple purposes, it negates a character set when used inside brackets (e.g., [^8] matches any except 8), signifies that the match must occur at the beginning of the line when placed at the start of a pattern and, in combination with a negative lookahead (?!), it excludes specific patterns from matching. **In the example given**: 1.The first part of this regex `.*` matches Oneplus devices except those containing 8 in name. 2. Excludes all the devices containing `Tab` | `"(OnePlus [^8].*)"` `"(^(?!.*Tab).*)"`| -| \| | The \| is used to match any of the specified devices in the list. **In the example given**: You will be allocated any iPad Pro 12.9 (2020), iPad Pro 12.9 (2021), or iPad Pro 12.9 (2022) | "iPad Pro 12.9 (202(0\|1\|2))" | - -## Regex Characters for `platformVersion` - -| REGEX CHARACTERS | DESCRIPTION | EXAMPLE | -|------------------|-------------|---------| -| `.*` | The characters `.*` are used to match any version that starts with the given string.**In the example given**:You'll be allocated any platform version that starts with 13 or 14. | `"(13.*)"``"(14.*)"` | -| `,` | This character `,` is used to include more combinations of different versions.**In the example given**:1. We have added 2 regex characters `.*` and `,` to show that using both in combination, we'll fetch any platform version that contains the string 13, 14, or 15.2. You'll be allocated any available platform version 13, 14, or 15.| `"(13.*),(14.*),(15.*)"` | -| `[]` | The `[]` is used to include a range of versions in the combination mentioned using a **single character only**.**In the example given**:Fetching any platform version available from 12, 13, 15 | `"1[235]"` | -| `^` | The `^` is used to negate a character from the search. **In the example given**:This matches os versions starting with 1 and not having immediate character = 5. | `"1[^5].*"` | -| \| | The \| is used to match any of the specified platform versions in the list. **In the example given**: 1. We have added 2 regex characters \| and `.*` to show that using both in combination, we'll fetch any platform version that contains the string 13 or 14. 2. You'll be allocated any available platform version 13 or 14.| "(1(3\|4).*)" | - -> To run tests on **minor OS versions** while using `platformVersion` capability, please reach out at our **24x7 Chat Support** or you could also mail us at [support@testmuai.com](mailto:support@testmuai.com). - -You can pass both `deviceName` and `platformVersion` regex patterns together or use any one of them as per your requirements. This flexibility allows you to match specific devices, versions, or combinations based on your needs. diff --git a/static/docs/regular-expression-espresso.md b/static/docs/regular-expression-espresso.md deleted file mode 100644 index ecf3fd0a2..000000000 --- a/static/docs/regular-expression-espresso.md +++ /dev/null @@ -1,29 +0,0 @@ -# Regular Expression - Espresso - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Regular Expression or REGEX is widely used to make searching/find characters in a string. - -When you run a test on TestMu AI using a specific device, there may be scenarios, in which the particular device that you selected isn’t available. In these scenarios, REGEX will help you widen the search request for devices to run the test on. - -To quote an example, consider you want to run the test on an iPhone. It doesn’t matter which one, but it’s got to be an iPhone. In these cases, we can use REGEX to specify exactly that. You’ll now be allotted any iPhone that’s available which widens the search of devices considerably. - -Platform version is not mandatory only if `deviceName` is passed using regex. If there is no regex in `deviceName`, passing `platformVersion` is mandatory. - -Usual way to pass the `deviceName` and `platformVersion` looks like this: - -```java -"device" : ["Pixel 6 Pro-12"] -``` - -Passing `deviceName` using REGEX: -```java -"device" : ["Pixel.*-12"] -``` - -**Right way to pass REGEX:** - -``Pixel.*-12`` -``Pixel.*`` - -REGEX is currently possible on the `deviceName` as of now and *not* on the ``platformVerison``. diff --git a/static/docs/regular-expression-xcui.md b/static/docs/regular-expression-xcui.md deleted file mode 100644 index a29c68024..000000000 --- a/static/docs/regular-expression-xcui.md +++ /dev/null @@ -1,29 +0,0 @@ -# Regular Expression - XCUI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Regular Expression or REGEX is widely used to make searching/find characters in a string. - -When you run a test on TestMu AI using a specific device, there may be scenarios, in which the particular device that you selected isn’t available. In these scenarios, REGEX will help you widen the search request for devices to run the test on. - -To quote an example, consider you want to run the test on an iPhone. It doesn’t matter which one, but it’s got to be an iPhone. In these cases, we can use REGEX to specify exactly that. You’ll now be allotted any iPhone that’s available which widens the search of devices considerably. - -Platform version is not mandatory only if `deviceName` is passed using regex. If there is no regex in `deviceName`, passing `platformVersion` is mandatory. - -Usual way to pass the `deviceName` and `platformVersion` looks like this: - -```java -"device" : ["Pixel 6 Pro-12"] -``` - -Passing `deviceName` using REGEX: -```java -"device" : ["Pixel.*-12"] -``` - -**Right way to pass REGEX:** - -``Pixel.*-12`` -``Pixel.*`` - -REGEX is currently possible on the `deviceName` as of now and *not* on the ``platformVerison``. diff --git a/static/docs/report-portal-cypress.md b/static/docs/report-portal-cypress.md deleted file mode 100644 index d3783be71..000000000 --- a/static/docs/report-portal-cypress.md +++ /dev/null @@ -1,23 +0,0 @@ -# ReportPortal.io Integration With TestMu AI For Cypress - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This article will guide you on how to integrate the **TestMu AI** platform with **ReportPortal.io** platform for running your **Cypress** automation tests. Before you can get started, make sure you have an account on [ReportPortal.io](http://reportportal.io/). - -> By default, the TestMu AI **Cypress-Multi-Reporter** mechanism generates **mochaawesome**. To override it with some other reporting options (report portal in this case), you have to create a different file to define the reporting configuration. - -## Steps To Integrate - -- Navigate to [ReportPortal.io](http://reportportal.io/) and login into your account. Then open your **Report Portal IO Profile**. - -- Copy the Report Portal Credentials - -- Open your Cypress project and create a new file for defining report portal configuration/credentials - -- Define the file name in the "`reporter_config_file`" capability of `lambdatest-config.json` file as mentioned in the below screenshot - -- Define the **reportportal.io** dependency (`@reportportal/agent-js-cypress`) in your **lambdatest-config.json** or **package.json** file - -- Integration is now done, open the Dashboard to see the results. - -> That's all! you have successfully integrated **ReportPortal.io** and **TestMu AI** for running your **Cypress** tests. In case you have any questions or need any additional information, you could reach out at our **24X7 Chat Support** or mail us directly at support@testmuai.com. diff --git a/static/docs/reqnroll-automation-with-selenium-appium-on-testmu.md b/static/docs/reqnroll-automation-with-selenium-appium-on-testmu.md deleted file mode 100644 index 07d1f0a95..000000000 --- a/static/docs/reqnroll-automation-with-selenium-appium-on-testmu.md +++ /dev/null @@ -1,81 +0,0 @@ -# Selenium With Reqnroll - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Run Reqnroll tests on the TestMu AI cloud grid. This guide covers setup, running a sample test, configuring capabilities, and testing locally hosted pages. - -**Sample repo** -All the code used in this guide is available in the sample repository. - - View on GitHub - -## Prerequisites - -Complete the following steps before running Reqnroll Selenium tests. - -1. Download and install **Selenium WebDriver** from the [official website](https://www.selenium.dev/downloads/). -2. Install the latest version of **Reqnroll**. -3. Install the **.Net** framework for developing applications using C#. -4. Download [Selenium WebDriver Language Binding](https://www.selenium.dev/downloads/) for C# and extract them to the appropriate folder. -5. Install a [.NET Core SDK](https://dotnet.microsoft.com/en-us/download) of 2.1 or greater version. -6. Get your TestMu AI Username and Access Key from the [TestMu AI Dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard). - -## Step 1: Clone the Sample Project - -Clone the repository and set up the virtual environment. - -```bash -virtualenv venv -``` - -Navigate to the newly created virtual environment: - -```bash -source venv/bin/activate -``` - -## Step 2: Set Your Credentials - -Configure your credentials to connect to the TestMu AI grid. - -Fetch the TestMu AI Credentials from the [TestMu AI Profile Section](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard). You can export the environment variables by running the following commands in your terminal. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -## Step 3: Configure Your Test Capabilities - -Update the TestMu AI Credentials in the Makefile or export them as environment variables as shown above. - -**Capabilities Generator** -Use the TestMu AI [Capabilities Generator](https://www.testmuai.com/capabilities-generator/) to auto-generate the capabilities class for your test requirements. - -## Step 4: Run the Test - -Build and execute the Reqnroll tests on the TestMu AI platform. - -Clean temporary files: -```bash -make clean -``` - -Build the automation project: -```bash -make build -``` - -Run the Reqnroll tests: -```bash -make reqnroll-automation-test -``` - -## Step 5: View Your Results - -Check the test output on the console and the TestMu AI dashboard. - -Navigate to the [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/) to check the status of the test execution. - -All the scenarios that are a part of respective *.feature(s)* files successfully executed on TestMu AI. diff --git a/static/docs/reqnroll-on-hyperexecute-grid.md b/static/docs/reqnroll-on-hyperexecute-grid.md deleted file mode 100644 index d00f6591e..000000000 --- a/static/docs/reqnroll-on-hyperexecute-grid.md +++ /dev/null @@ -1,172 +0,0 @@ -# Running Reqnroll Framework Tests on HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Reqnroll is an open-source behavior-driven development (BDD) framework for .NET, the successor to SpecFlow. It allows you to define test scenarios in plain language using Gherkin syntax, making collaboration between technical and non-technical team members seamless. - -HyperExecute is an AI Native Test Orchestration Cloud Platform that empowers you to run **end-to-end** tests **quickly** and **efficiently**. It provides Just-in-Time (JIT) testing infrastructure with fast execution **speeds**, **smart orchestration**, and **detailed logs**. - -This guide details how to execute your **Reqnroll** framework tests on **HyperExecute** via two different methods: - -- [**Using Local System**](/support/docs/reqnroll-on-hyperexecute-grid/#1-testing-using-local-system) - You can use your own local machine to execute tests. -- [**Using Gitpod Platform**](/support/docs/reqnroll-on-hyperexecute-grid/#2-testing-using-gitpod) - Execute tests using GitPod. (Requires a [Gitpod](https://gitpod.io/login/) account) - -## Testing Using Local System - -Follow the step-by-step guide to execute your test on HyperExecute. - -### Prerequisites - -To run the Tests on HyperExecute from your Local System, you are required: - -- Your TestMu AI [Username and Access key](/support/docs/hyperexecute-how-to-get-my-username-and-access-key/) -- [HyperExecute YAML](/support/docs/hyperexecute-yaml-version0.2/) file which contains all the necessary instructions. -- [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) in order to initiate a test execution Job . -- Setup the [Environmental Variable](/support/docs/hyperexecute-environment-variable-setup/) - -### Step 1: Configure Your Test Suite - -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** - -Download or Clone the code sample for the Reqnroll from the TestMu AI GitHub repository to run the tests on the HyperExecute. - - View on GitHub - -If you are using your own project, make sure you update the **Hub endpoint** in your tests file. - -By setting up the Hub endpoint, you establish the communication channel between your tests and the browser nodes, enabling effective test distribution and execution. - -Configure the desired capabilities based on your test requirements. For example: - -```bash -DesiredCapabilities capabilities = new DesiredCapabilities(); -capabilities.SetCapability(CapabilityType.BrowserName, "Chrome"); -capabilities.SetCapability(CapabilityType.Version, "96"); -capabilities.SetCapability(CapabilityType.Platform, "Windows 10"); -``` - -> You can also use **dev** and **beta** browser versions. To generate capabilities for your test requirements, you can use our inbuilt 🔗 Capabilities Generator. - -### Step 2: Setup the CLI in your Test Suite - -After cloning / downloading the sample repo, you need to setup the CLI and the environment variables. - -#### Download the HyperExecute CLI - -The CLI is used for triggering the tests on HyperExecute. It is recommend to download the CLI binary on the host system and keep it in the root directory of the suite to perform the tests on HyperExecute. - -You can download the CLI for your desired platform from the below mentioned links: - -| Platform | HyperExecute CLI | -| ---------| ---------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| MacOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -#### Setup Environment Variable - -Now, you need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in the [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). - -Run the below mentioned commands in your terminal to setup the CLI and the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -### Step 3: Configure YAML in your Test Suite - -Configure your YAML file as per your use cases using **key value** pairs. - -In this sample YAML file, we have mentioned: - -- **version** of the YAML file -- **Timeouts** for executing your project -- **Mode of execution** is [Autosplit](/support/docs/hyperexecute-auto-split-strategy/). You can also opt for [Matrix](/support/docs/hyperexecute-matrix-multiplexing-strategy/) or [Hybrid](/support/docs/hyperexecute-hybrid-strategy/) mode. -- **Pre and Post** commands -- **Reports and Artefacts** that will be generated after the completion of tests -- and other necessary YAML Parameters - -```yaml ---- -version: 0.1 -globalTimeout: 90 -testSuiteTimeout: 90 -testSuiteStep: 90 - -runson: linux - -autosplit: true -retryOnFailure: true - -maxRetries: 1 -concurrency: 4 - -env: -NUGET_PACKAGES: '/home/ltuser/.nuget/packages/' -NUGET_HTTP_CACHE_PATH: '/home/ltuser/.local/share/NuGet/v3-cache' -NUGET_PLUGINS_CACHE_PATH: '/home/ltuser/.local/share/NuGet/plugins-cache' - -pre: -- dotnet restore -- dotnet list reqnroll.cloud.csproj package > packages.txt -- dotnet build -c Release - -cacheKey: '{{ checksum "packages.txt" }}' - -post: -- cat yaml/linux/reqnroll_hyperexecute_autosplit_sample.yaml - -mergeArtifacts: true - -uploadArtefacts: -- name: Execution_Report -path: -- Report/** -- Reports/** -- name: Execution_Screenshots -path: -- Screenshots/**/** -- Reports/**/Screenshots/** - -testDiscovery: -type: raw -mode: remote -command: grep -rni 'Features' -e '@' --include=\*.feature | sed 's/.*@//' - -testRunnerCommand: dotnet test --logger "console;verbosity=detailed" --filter "(Category=$test)" - -jobLabel: [selenium-reqnroll, linux, autosplit] -``` - -### Step 4: Execute your Test Suite - -> **NOTE :** In case of MacOS, if you get a permission denied warning while executing CLI, simply run **`chmod u+x ./hyperexecute`** to allow permission. In case you get a security popup, allow it from your **System Preferences** → **Security & Privacy** → **General tab**. - -Run the below command in your terminal at the root folder of the project: - -```bash -./hyperexecute --config RELATIVE_PATH_OF_YOUR_YAML_FILE -``` - -OR use this command if you have not exported your username and access key in the step 2. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE `} - -### Step 5: Monitor the Test Execution - -Visit the [HyperExecute Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) and check your Job status. - -Every test run on the HyperExecute has a unique *jobId* associated with it. Each *jobId* can in turn constitute single (or multiple) *groupId*(s). You can visit [HyperExecute Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) for checking the status of the test execution. - -You can seamlessly navigate between JobId's and taskId's. You need to click on the *testID* to navigate from the HyperExecute logs to the Automation Dashboard. - -### Step 6: Download Artifacts and Reports - -HyperExecute also facilitates the provision to download the [Artifacts](/support/docs/hyperexecute-artifacts/) and [Reports](/support/docs/hyperexecute-reports/) on your local machine. Click on the corresponding button to download your generated artifacts and reports. - -> For any query or doubt, please feel free to contact us via **24×7 chat support** or you can also drop a mail to **support@testmuai.com**. -Happy testing! diff --git a/static/docs/rerun-automation-tests.md b/static/docs/rerun-automation-tests.md deleted file mode 100644 index 83dbfeeab..000000000 --- a/static/docs/rerun-automation-tests.md +++ /dev/null @@ -1,76 +0,0 @@ -# Re-Run Automation Tests - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -When a test fails, you often need to run it again, either to confirm a fix or to check whether the failure was a flake. The **Re-Run** button on the Test Detail Page lets you do this from the dashboard, without re-triggering your suite from CI or your local machine. - -Re-Run offers two options: - -- **Replay** - runs the test again on the grid. -- **Reproduce** - replays the test against the data captured from the original run. Requires HAR logging. - -**Limited Availability** -Re-Run is not generally available yet and may not be enabled for your account. Its behavior may change before general availability. To get it enabled for your organization, reach out to our **24/7 chat support** or email us at [support@testmuai.com](mailto:support@testmuai.com). - -## Re-Run a Test - -1. Open the [Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) and select a build. -2. Select a test from the Test List Panel to open its Test Detail Page. -3. Click the dropdown arrow on the **Re-Run** button, in the top-right of the test summary. -4. Choose **Replay** or **Reproduce**. - -The **Re-runs** field in the test summary shows how many times the test has been re-run. - -## Replay - -**Replay** runs the test again on the grid, against your application as it exists now. Unlike Reproduce, it does not use the captured data from the original run, so the result reflects the current state of your application and environment. - -Use Replay to: - -- **Confirm a fix** - re-run a previously failing test after deploying a change. -- **Identify flaky tests** - run the same test repeatedly to see whether failures are intermittent. For a broader view, see [Flaky Test Detection](/support/docs/test-intelligence-flakiness-test-detection/). -- **Re-test after an environment change** - verify a test once a dependency or test environment issue is resolved. - -## Reproduce - -**Reproduce** replays the test against the network and DOM data captured during the original run, rather than against your live application. This replicates the conditions the test originally encountered, which is what makes it useful for intermittent failures that will not recur on demand. - -Reproduce reads the HAR (HTTP Archive) network logs recorded during the original test. If those logs were not captured, the option is unavailable. - -A test is eligible for Reproduce only if both of the following are true: - -- It ran with the `network.full.har` capability set to `true`. -- It finished with a passed, failed, or completed status. Tests that did not run to completion cannot be reproduced. - -### Enable Reproduce - -Reproduce requires HAR logging on the original run. Pass the following capability in your test: - -```javascript -"network.full.har" : true -``` - -If the original test ran without this capability, **Reproduce** appears greyed out in the Re-Run menu with the tooltip: - -> No Network Logs found, use network.full.har:true to replicate the test states. - -In this case, add the capability and run the test again. The new run is then eligible for Reproduce. - -`network.full.har` applies to web tests. Enabling it also populates the **Network** tab with a waterfall chart. See [HAR Log Viewer With Waterfall](/support/docs/har-log-viewer-with-waterfall/) for how to read it. - -## Choosing Between Replay and Reproduce - -| | Replay | Reproduce | -| --- | --- | --- | -| Runs against | Your live application | Network and DOM data captured from the original run | -| Prerequisite | None | `network.full.har : true`, and a passed, failed, or completed original run | -| Reflects | The current state of your app | The state at the time of the original run | -| Best for | Confirming fixes, detecting flakes | Investigating intermittent failures that will not recur | - -If you are not sure which to use, start with **Replay**. Reach for **Reproduce** when a failure will not recur on a live run and you need the original conditions back. - -## Next Steps - -- [Explore the Automation Dashboard](/support/docs/inside-testmu-platform/) -- [HAR Log Viewer With Waterfall](/support/docs/har-log-viewer-with-waterfall/) -- [Debugging Options](/support/docs/debugging-options/) diff --git a/static/docs/responsive-testing.md b/static/docs/responsive-testing.md deleted file mode 100644 index a64c4c808..000000000 --- a/static/docs/responsive-testing.md +++ /dev/null @@ -1,25 +0,0 @@ -# Responsive Testing - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -> This feature is **deprecated** and as a result, no further support will be provided for this feature by the TestMu AI Organization. - -As the word responsive suggests, this type of testing is used to check the appearance of a website with RWD(Responsive Web Design). This design helps a website render the content in a flexible yet appropriate manner based on the devices & OS on which it gets summoned. At TestMu AI, you can [check RWD](https://www.testmuai.com/responsive-test-online) through 50+ different devices in a single go! - -### How To Do Responsive Testing? - -**Step 1:** Click ‘**Responsive**’ sub menu under [Visual UI Testing](https://www.testmuai.com/smart-visual-ui-testing) menu in menu box. - -**Step 2:** Input the URL for which you wish to test responsiveness. Select your monitor size (diagonal length in inches). Click ‘Generate’ to [generate screenshots](https://www.testmuai.com/full-page-screen-capture). - -**Step 3:** On the right side of the page, you will find various mobile and desktop combinations. On clicking them, you can test the responsiveness for those particular devices. - -### How to record issues? - -**Step 1:** On the ‘**Responsive**’ page, under the menu ‘**Visual UI Testing**’, you will find a camera icon; click that icon. - -**Step 2:** The image editor will pop up where you can comment about the screenshot and check the mark as an issue check box. - -**Step 3:** Finally, click save and done on the pop up to record the issue. Enter the required details and share the bug with your favorite project management tool. - -Let us know if you have any questions, drop us an email at support@testmuai.com, or simply give us a shout. We are here to help you speed your test suites 24/7. Happy testing! 🙂 diff --git a/static/docs/rktracer-integration.md b/static/docs/rktracer-integration.md deleted file mode 100644 index 308d28324..000000000 --- a/static/docs/rktracer-integration.md +++ /dev/null @@ -1,53 +0,0 @@ -# RKTracer Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Developed by RKValidate, RKTracer is a Code coverage tool capable of generating coverage data for different programming languages like C, C++, C#, Java, Kotlin, JavaScript, Python, etc; regardless of what compiler, cross compiler, embedded targets, or servers you use to test your apps. - -With TestMu AI and RKTracer, you can generate Code coverage for your web and mobile apps across 3000+ real browsers, devices, and operating system combinations. - -## RKTracer With Web Automation On TestMu AI - -1. Download the TestMu AI [Java-TestNG GitHub repository](https://github.com/LambdaTest/Java-TestNG-Selenium/). - -2. To run the single test using TestNG, pass the build command by prefixing it with `"rktracer"` - -``` -rktracer mvn test -D suite=single.xml -``` - -3. Pass the below command to get the test results. - -``` -rkresults -``` - -## RKTracer With App Automation On TestMu AI - -1. Build your app with the RKTracer tool embedded. -2. Login to your TestMu AI account and perform the automated testing of your app. -3. Navigate to the [App Automation dashboard](https://www.testmuai.com/login/?redirectTo=https://appautomation.lambdatest.com/build) on TestMu AI. -4. Download the Device log file from TestMu AI App Automation dashboard. - -5. Place the downloaded Device log file in your app directory. -6. Convert the file from `JSON` to `.txt` with below command `json2rk devicelog.json > rk-coverage.txt`. - -7. Pass the below command to get the test results. - -``` -rkresults -``` - -## RKTracer With Real Time App Testing On TestMu AI - -1. Build your app with the RKTracer tool embedded. -2. Login to your TestMu AI account and perform the real-time testing of your app. -3. Download the App logs of the real-time app test and place them in the build code directory. - -4. Open the terminal from that folder and run the command `node har2rk.js `. It will convert the harfile into `rkcoverage.txt`. - -5. Run the below command to get the results. - -``` -"rkresults" -``` diff --git a/static/docs/robot-on-hyperexecute-grid.md b/static/docs/robot-on-hyperexecute-grid.md deleted file mode 100644 index ba9b4b3ff..000000000 --- a/static/docs/robot-on-hyperexecute-grid.md +++ /dev/null @@ -1,233 +0,0 @@ -# Running Robot Framework Tests on HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Robot Framework is a powerful open-source automation framework for Selenium that supports keyword-driven and behavior-driven development. With an easy-to-read syntax and a rich set of libraries, it simplifies the creation and execution of Selenium test scripts. - -HyperExecute is an AI Native Test Orchestration Cloud Platform that empowers you to run **end-to-end** tests **quickly** and **efficiently**. It provides Just-in-Time (JIT) testing infrastructure with fast execution **speeds**, **smart orchestration**, and **detailed logs**. - -This guide details how to execute your **Robot** framework tests on **HyperExecute** via two different methods: - -- [**Using Local System**](/support/docs/robot-on-hyperexecute-grid/#1-testing-using-local-system) - You can use your own local machine to execute tests. -- [**Using Gitpod Platform**](/support/docs/robot-on-hyperexecute-grid/#2-testing-using-gitpod) - Execute tests using GitPod. (Requires a [Gitpod](https://gitpod.io/login/) account) - -**Mainframe 3270 suites** -Testing a mainframe terminal with Robot Framework? See [Mainframe Testing with HyperExecute](/support/docs/mainframe-testing-with-hyperexecute/) for a suite that installs the `s3270`/`x3270` emulators, runs mock hosts as background services, and records the `x3270` GUI. - -## 1. Testing Using Local System - -Follow the step-by-step guide to execute your test on HyperExecute. - -### Prerequisites - -To run the Tests on HyperExecute from your Local System, you are required: - -- Your TestMu AI [Username and Access key](/support/docs/hyperexecute-how-to-get-my-username-and-access-key/) -- [HyperExecute YAML](/support/docs/hyperexecute-yaml-version0.2/) file which contains all the necessary instructions. -- [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) in order to initiate a test execution Job . -- Setup the [Environmental Variable](/support/docs/hyperexecute-environment-variable-setup/) - -### Step 1: Configure Your Test Suite - -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** - -Download or Clone the code sample for the JUnit from the TestMu AI GitHub repository to run the tests on the HyperExecute. - - View on GitHub - -If you are using your own project, make sure you update the **Hub endpoint** in your tests file. - -By setting up the Hub endpoint, you establish the communication channel between your tests and the browser nodes, enabling effective test distribution and execution. - -> You can also use **dev** and **beta** browser versions. To generate capabilities for your test requirements, you can use our inbuilt 🔗 Capabilities Generator. - -### Step 2: Setup the CLI in your Test Suite - -After cloning / downloading the sample repo, you need to setup the CLI and the environment variables. - -#### Download the HyperExecute CLI - -The CLI is used for triggering the tests on HyperExecute. It is recommend to download the CLI binary on the host system and keep it in the root directory of the suite to perform the tests on HyperExecute. - -You can download the CLI for your desired platform from the below mentioned links: - -| Platform | HyperExecute CLI | -| ---------| ---------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| MacOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -#### Setup Environment Variable - -Now, you need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in the [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). - -Run the below mentioned commands in your terminal to setup the CLI and the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -### Step 3: Configure YAML in your Test Suite - -Configure your YAML file as per your use cases using **key value** pairs. - -In this sample YAML file, we have mentioned: - -- **version** of the YAML file -- **Timeouts** for executing your project -- **Mode of execution** is [Autosplit](/support/docs/hyperexecute-auto-split-strategy/). You can also opt for [Matrix](/support/docs/hyperexecute-matrix-multiplexing-strategy/) or [Hybrid](/support/docs/hyperexecute-hybrid-strategy/) mode. -- **Pre and Post** commands -- **Reports and Artefacts** that will be generated after the completion of tests -- and other necessary YAML Parameters - -```yaml ---- -version: 0.1 -globalTimeout: 90 -testSuiteTimeout: 90 -testSuiteStep: 90 - -runson: linux - -autosplit: true -retryOnFailure: true - -maxRetries: 1 -concurrency: 2 - -env: -# PAT: ${{ .secrets.testKey }} -TARGET_OS: LINUX - -# Dependency caching for Windows -cacheKey: '{{ checksum "requirements.txt" }}' -cacheDirectories: -- pip_cache -- poetry_cache -pre: -# Robot Framework and Robot Selenium Library need to be installed globally -# Rest of the packages can be installed in venv -- pip3 install -r requirements.txt --cache-dir pip_cache -- poetry config virtualenvs.path poetry_cache -- poetry install -post: -- cat yaml/linux/robot_hyperexecute_autosplit_sample.yaml - -mergeArtifacts: true - -uploadArtefacts: -- name: HTML_Reports -path: -- /*.html -- name: XML_Reports -path: -- /*.xml - -report: true -partialReports: -type: json -location: / -frameworkName: extent - -testDiscovery: -type: raw -mode: remote -command: grep 'test_linux' Makefile | sed 's/\(.*\):/\1 /' - -testRunnerCommand: make $test - -jobLabel: [selenium-robot, linux, autosplit] -``` - -### Step 4: Execute your Test Suite - -> **NOTE :** In case of MacOS, if you get a permission denied warning while executing CLI, simply run **`chmod u+x ./hyperexecute`** to allow permission. In case you get a security popup, allow it from your **System Preferences** → **Security & Privacy** → **General tab**. - -Run the below command in your terminal at the root folder of the project: - -```bash -./hyperexecute --config RELATIVE_PATH_OF_YOUR_YAML_FILE -``` - -OR use this command if you have not exported your username and access key in the step 2. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE `} - -### Step 5: Monitor the Test Execution - -Visit the [HyperExecute Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) and check your Job status. - -### Step 6: Download Artifacts and Reports - -HyperExecute also facilitates the provision to download the [Artifacts](/support/docs/hyperexecute-artifacts/) and [Reports](/support/docs/hyperexecute-reports/) on your local machine. Click on the corresponding button to download your generated artifacts and reports. - -## 2. Testing Using Gitpod - -You can also use the Gitpod platform to execute our sample repository. It will fetch all the sample codebases and trigger the CLI to execute the tests. - -Follow the below steps to run Test using Gitpod: - -**Step 1:** Click '**Open in Gitpod**' button. You will be redirected to Login/Signup page. This button is configured to redirect you to the Gitpod platform where you will be able to execute our sample repository. - -**Step 2:** Login with TestMu AI credentials. Once logged in, a pop-up confirmation will appear, asking you to **'Proceed'** to the Gitpod editor in a new tab. The current tab will display the HyperExecute Dashboard. - -**Step 3:** Choose your preferred editor (we recommend VS Code Editor) - -**Step 4:** As you are running a sample project, Fetching of the Test Scripts, [HyperExecute YAML](/support/docs/deep-dive-into-hyperexecute-yaml/), [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) and Triggering your tests using the `Execution Command` will be automated. - -**Step 5:** Once you see the `Job Link` in the logs, you can visit the [HyperExecute dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) to see the tests getting executed. - -You can also implement [Secret Keys](/support/docs/hyperexecute-how-to-save-and-manage-secrets/) in your YAML file. - -## Navigation in Automation Dashboard - -Every test run on the HyperExecute has a unique *jobId* associated with it. Each *jobId* can in turn constitute single (or multiple) *groupId*(s). You can visit [HyperExecute Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) for checking the status of the test execution. - -You can seamlessly navigate between JobId's and taskId's. You need to click on the *testID* to navigate from the HyperExecute logs to the Automation Dashboard. - -The snapshot below shows the videos, logs and other meta data for that specific *test_ID* - -> For any query or doubt, please feel free to contact us via **24×7 chat support** or you can also drop a mail to **support@testmuai.com**. -Happy testing! - -## Using the Robot Framework Agent Skill with TestMu AI - -The [robot-framework-skill](https://github.com/LambdaTest/agent-skills/tree/main/robot-framework-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The robot-framework-skill package includes: - -``` -robot-framework-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing Robot Framework Agent Skill - -Install a Robot Framework Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/robot-framework-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/robot-framework-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only robot-framework-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). diff --git a/static/docs/robot-report.md b/static/docs/robot-report.md deleted file mode 100644 index 4d5357f72..000000000 --- a/static/docs/robot-report.md +++ /dev/null @@ -1,27 +0,0 @@ -# Robot Report - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Robot Framework is a generic, open-source automation framework designed for acceptance testing, acceptance test-driven development (ATDD), and robotic process automation (RPA). It is widely used in software testing due to its simplicity, extensibility, and ease of use. The framework is keyword-driven, meaning that tests are written using predefined keywords, making it accessible even to non-programmers. - -> Ensure your project is configured with the Robot framework and all test scripts are in place. - -## Steps to Generate Robot Reports on HyperExecute - -**Step 1:** Configure all the necessary dependencies and configurations to your project. - -**Step 2:** Configure the report parameters in the HyperExecute YAML file - -Add the following configurations in your YAML file: - -```yaml -testRunnerCommand: YOUR_RUNNER_COMMAND --outputdir Reports - -report: true -partialReports: -type: json -location: Reports -frameworkName: robot -``` - -**Step 3:** Now execute your job by triggering the HyperExecute CLI. You can visit the HyperExecute dashboard to download the report after job completion. diff --git a/static/docs/robot-with-selenium-running-robot-automation-scripts-on-testmu-selenium-grid.md b/static/docs/robot-with-selenium-running-robot-automation-scripts-on-testmu-selenium-grid.md deleted file mode 100644 index f9034d97a..000000000 --- a/static/docs/robot-with-selenium-running-robot-automation-scripts-on-testmu-selenium-grid.md +++ /dev/null @@ -1,131 +0,0 @@ -# Selenium With Robot Framework - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Run Robot Framework tests on the TestMu AI cloud grid. This guide covers setup, running a sample test, configuring capabilities, and testing locally hosted pages. - -**Sample repo** -All the code used in this guide is available in the sample repository. - - View on GitHub - -## Prerequisites - -Complete these steps before running your first Robot Framework Selenium test. - -1. Install the latest Python build from the [official website](https://www.python.org/downloads/). -2. Verify that **pip** is installed in your system. Install **pip** from [official pip documentation](https://pip.pypa.io/en/stable/installation/). -3. Download the latest **Selenium Client** and its **WebDriver bindings** from the [official website](https://www.selenium.dev/downloads/). -4. Install **virtualenv** to isolate the build from other setups and ensure tests run with the specified module versions. -```bash -pip install virtualenv -``` - -## Step 1: Clone the Sample Project - -Clone the repository and set up the environment. - -```bash -git clone https://github.com/LambdaTest/Robot-Selenium-Sample -cd Robot-Selenium-Sample -``` - -Create a virtual environment and activate it: -```bash -virtualenv venv -source venv/bin/activate -``` - -Install the required packages: -```bash -pip install -r requirements.txt -``` - -## Step 2: Set Your Credentials - -Configure your credentials to connect to the TestMu AI Selenium Grid. - -Set TestMu AI `Username` and `Access Key` in environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -## Step 3: Configure Your Test Capabilities - -Define browser, version, and OS settings for your test run. - -Here is `common.robot` file to setup mandatory details to run at TestMu AI. -```python -*** Settings *** -Library Selenium2Library - -*** Variables *** - -@{_tmp} -... browserName: %{browserName}, -... platformName: %{platform}, -... browserVersion: %{version}, -... name: RobotFramework Lambda Test - -${BROWSER} %{ROBOT_BROWSER} -${CAPABILITIES} ${EMPTY.join(${_tmp})} -${KEY} : -${REMOTE_URL} https://${KEY}@hub.lambdatest.com/wd/hub - -*** Keywords *** - -Open test browser -Open browser https://lambdatest.github.io/sample-todo-app/ browser=${BROWSER} -... remote_url=${REMOTE_URL} -... desired_capabilities=${CAPABILITIES} - -Close test browser -Close all browsers -``` - -**Capabilities Generator** -Use the TestMu AI [Capabilities Generator](https://www.testmuai.com/capabilities-generator/) to auto-generate the capabilities class for your test requirements. - -## Step 4: Run the Test - -Execute the Robot Framework test from the command line. - -```bash -make test_Windows_10_chrome_latest -``` - -To run parallel tests: -```bash -make run_all_in_parallel -``` - -## Step 5: View Your Results - -Check the test output on the console and the TestMu AI dashboard. - -Visit the [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) to view your test results. The dashboard provides: - -- Text logs for each test step -- Screenshots captured during execution -- Video recordings of the full test session - -## Run Robot Framework Tests Using Agent Skills - -Use AI coding assistants to generate and run Robot Framework tests with the TestMu AI Agent Skill. - -The [robot-framework-skill](https://github.com/LambdaTest/agent-skills/tree/main/robot-framework-skill) is part of [TestMu AI Agent Skills](https://github.com/LambdaTest/agent-skills/) - structured packages that teach AI coding assistants how to write production-grade test automation. - -Install the skill: - -```bash -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/robot-framework-skill .claude/skills/ - -# For Cursor / Copilot -cp -r agent-skills/robot-framework-skill .cursor/skills/ -``` - -Install all available framework skills at once by cloning the repository directly into your tool's skills directory (e.g., `.claude/skills/`, `.cursor/skills/`). diff --git a/static/docs/rocketchat-integration-with-testmu.md b/static/docs/rocketchat-integration-with-testmu.md deleted file mode 100644 index 25dbe1b92..000000000 --- a/static/docs/rocketchat-integration-with-testmu.md +++ /dev/null @@ -1,57 +0,0 @@ -# Rocket.Chat Integration With TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Rocket.Chat is a free, open-source communication platform that allows collaboration, file sharing, and real-time chat sharing. It's cross-platform and open source, allowing users to customize and extend its functionality to meet their team's or business needs. - -The TestMu AI Rocket.Chat Integration enables users to share or push bugs to their specified channel, capture screenshots, annotations, and issue information, with the fields marked as bugs displayed on Rocket.Chat for that testing instance. - -> Rocket.Chat Integration with TestMu AI is available for freemium as well as premium plan. - -## Integrate Rocket.Chat from your TestMu AI Account - -**Step 1:** Login to your TestMu AI account. You should have Admin or User level access to see and install integrations. - -**Step 2:** Click on Settings -> Integrations -> Communication. - -**Step 3:** Click on **Connect** button of `Rocket Chat` block. - -**Step 4:** Now, provide your Rocket.Chat Webhook URL to establish integration with TestMu AI and click on **install** button. - -**generate your webhook url** - -- Visit your Rocket.Chat account -> **Workspace** tab. -- Click on the **Integrations** -> click on **New** button to generate your new webhook. -- Toggle the enable button to true. -- Enter the details and click on the **Save** button -> copy your Webhook URL. - -## Lodge your First Bug - -> Note: If you are using Rocket.Chat for the first time, then make sure to create a project for yourself. It is a pre-requisite in order to push screenshots from your TestMu AI account. - -**Step 1:** Start with any type of testing, for the demo purpose we are going with the **Real Time Testing**. - -**Step 2:** Enter your Project URL and configure for browser and operating system of your choice & hit **Start**. - -**Step 3:** After the VM is launched and operable. You can perform testing on your web-app for finding bugs. If a bug gets revealed, then you need to click on the Bug icon from the left panel for capturing a screenshot of the same. - -**Step 4:** After clicking on "Mark as Bug" button a Teamwork specific form would open up. Fill the fields as per your requirement. -- Select which project should the ticket go under. -- Set which Task List should the bug go under. -- Set the priority of the issue. -- Mention the Title as well to easily organize your bugs. -- Write a description to help relate the cause of the issue or the task. - -**Step 5:** Click on **Mark as Bug** button. Now go to your dashboard and check a ticket will be created for the same. - -**Step 6:** Go to your Rocket.Chat dashboard and you can check on your assigned channel, a notification is sent for the same. - -## Uninstall Rocket.Chat Integration - -**Step 1:** Login to your TestMu AI account. You should have Admin or User level access to see and install integrations. - -**Step 2:** Click on Settings -> Integrations -> Communication. - -**Step 3:** Click on the **Remove** button. - ->That was all you need to know for TestMu AI + Rocket.Chat Integration. Increase your productivity multifold with our integrations. If you still have any questions for us, please feel free to let us know. Our experts are always **available on chat** to help you out with any roadblock regarding our product. Happy testing! diff --git a/static/docs/role-based-access-control.md b/static/docs/role-based-access-control.md deleted file mode 100644 index 0594578d8..000000000 --- a/static/docs/role-based-access-control.md +++ /dev/null @@ -1,64 +0,0 @@ -# TestMu AI Role-Based Access Control (RBAC) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -For every enterprise that uses cloud services, access management for those cloud resources is vital. TestMu AI RBAC is an authorization system that allows seamless access management of TestMu AI resources. With TestMu AI role-based access control, you can control who has access to TestMu AI resources, how they can leverage those resources, and what domains they can have access to. - -The TestMu AI RBAC will help you in creating the different teams and allot concurrency to those teams. In this way, you can perform the test automation by dividing the concurrency among the teams. - -## How To Add A New Team Member? - -**Step 1**: Login to your TestMu AI account and navigate to Profile icon on the top-right. Click on **'Manage Teams'**. - -**Step 2**: From the Team Members tab, click on **'Invite'** - -**Step 3**: Enter the user's e-mail address, select the user's role and then select the Group. Click on **'Send Invites'**. - -> -**Note**: You can also add multiple users by selecting **'ADD ANOTHER'** or **'ADD MULTIPLE'**. - -Invites will be sent to the selected users which they need to accept from their end. After accepting the invites, they will be on your team member list. - -## How To Create A New Group? - -**Step 1**: Login to your TestMu AI account and navigate to Profile icon on the top-right. Click on **'Manage Teams'**. - -**Step 2**: Select the **'Groups'** tab and click on **'Create Group'**. - -**Step 3**: Fill in the group name of your choice, add the team members to be added to the group and then allot the maximum concurrency. After providing the details, click on **'Create'**. - -Your new group will be created successfully. - -## How To View A Group? - -**Step 1**: Login to your TestMu AI account and navigate to Profile icon on the top-right. Click on **'Manage Teams'**. - -**Step 2**: Select the **'Groups'** tab. Navigate to the group that you want to view. Click on **'See Users'** below the _User_ category. - -Your group details will be displayed. - -## How To Edit A Group? - -**Step 1**: Login to your TestMu AI account and navigate to Profile icon on the top-right. Click on **'Manage Teams'**. - -**Step 2**: Select the **'Groups'** tab. Navigate to the group that you want to edit. Click on edit icon below the _Max Concurrency_ category. - -**Step 3**: You can edit the required fields such as Group name, add or delete users, and change concurrency. After editing, click on **'Update'**. - -Your group details will be updated. - -> -**Note**: You can also edit your Group details by selecting **'See Users'** under the _User_ category. - -Then click on **'Edit'**. - -## How To Delete A Group? - -**Step 1**: Login to your TestMu AI account and navigate to Profile icon on the top-right. Click on **'Manage Teams'**. - -**Step 2**: Select the **'Groups'** tab. Navigate to the group that you want to delete. Click on delete icon below the _Max Concurrency_ category. - - Your selected group will be deleted. - -> -In case you have any queries regarding TestMu AI RBAC or need any additional information, please reach out at our **24*7 Chat Portal** or drop us a mail to [**support@testmuai.com**](mailto:support@testmuai.com). diff --git a/static/docs/rspec-with-selenium-running-rspec-automation-scripts-on-testmu-selenium-grid.md b/static/docs/rspec-with-selenium-running-rspec-automation-scripts-on-testmu-selenium-grid.md deleted file mode 100644 index cd036afea..000000000 --- a/static/docs/rspec-with-selenium-running-rspec-automation-scripts-on-testmu-selenium-grid.md +++ /dev/null @@ -1,103 +0,0 @@ -# Selenium With RSpec - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Run RSpec tests on the TestMu AI cloud grid. This guide covers setup, running a sample test, configuring capabilities, and testing locally hosted pages. - -**Sample repo** -All the code used in this guide is available in the sample repository. - - View on GitHub - -## Prerequisites - -Complete the following steps before you start Ruby automation testing with Selenium. - -1. Install Ruby and gem on your local system. Follow these instructions to install on different operating systems. - * For **Windows**, download from the [Ruby Installer for Windows](https://rubyinstaller.org/downloads/). - * For **Linux** or **Ubuntu**, run a simple apt command like below: -```bash -sudo apt-get install ruby-full -``` - * For **macOS**, run a [Homebrew](https://brew.sh/) command like this: -```bash -brew install ruby -``` -2. Install the [parallel_tests](https://github.com/grosser/parallel_tests) gem to run tests in parallel. -3. Get the TestMu AI binary file for running tests on your locally hosted web pages. - -## Step 1: Clone the Sample Project - -Clone the repository and install dependencies. - -```bash -git clone https://github.com/LambdaTest/RSpec-Selenium-Sample.git -cd RSpec-Selenium-Sample -``` - -Install project dependencies: -```bash -bundle install -``` - -## Step 2: Set Your Credentials - -Configure your credentials to connect to the TestMu AI Selenium Grid. - -Set TestMu AI Username and Access Key in environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -## Step 3: Configure Your Test Capabilities - -Define browser, version, and OS settings for your test run. - -In the test script, update your test capabilities. The capabilities are loaded from the config YAML files and passed to the Selenium RemoteWebDriver. - -**Capabilities Generator** -Use the TestMu AI [Capabilities Generator](https://www.testmuai.com/capabilities-generator/) to auto-generate the capabilities class for your test requirements. - -## Step 4: Run the Test - -Execute the RSpec test from the command line. - -```bash -bundle exec rake single -``` - -To run parallel tests: -```bash -bundle exec rake parallel -``` - -## Step 5: View Your Results - -Check the test output on the console and the TestMu AI dashboard. - -Visit the [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) to view your test results. The dashboard provides: - -- Text logs for each test step -- Screenshots captured during execution -- Video recordings of the full test session - -## Run RSpec Tests Using Agent Skills - -Use AI coding assistants to generate and run RSpec tests with the TestMu AI Agent Skill. - -The [rspec-skill](https://github.com/LambdaTest/agent-skills/tree/main/rspec-skill) is part of [TestMu AI Agent Skills](https://github.com/LambdaTest/agent-skills/) - structured packages that teach AI coding assistants how to write production-grade test automation. - -Install the skill: - -```bash -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/rspec-skill .claude/skills/ - -# For Cursor / Copilot -cp -r agent-skills/rspec-skill .cursor/skills/ -``` - -Install all available framework skills at once by cloning the repository directly into your tool's skills directory (e.g., `.claude/skills/`, `.cursor/skills/`). diff --git a/static/docs/ruby-on-hyperexecute-grid.md b/static/docs/ruby-on-hyperexecute-grid.md deleted file mode 100644 index 4fd38a1ff..000000000 --- a/static/docs/ruby-on-hyperexecute-grid.md +++ /dev/null @@ -1,177 +0,0 @@ -# Running Ruby Framework Tests on HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -RSpec is a behavior-driven development (BDD) framework for Ruby commonly used in conjunction with Selenium for automated testing. It facilitates expressive and readable specifications, making it easier to define and manage Selenium test cases. RSpec supports a variety of assertions and provides powerful reporting tools, enhancing the efficiency and clarity of Selenium test scripts written in Ruby. - -HyperExecute is an AI Native Test Orchestration Cloud Platform that empowers you to run **end-to-end** tests **quickly** and **efficiently**. It provides Just-in-Time (JIT) testing infrastructure with fast execution **speeds**, **smart orchestration**, and **detailed logs**. - -This guide details how to execute your **RSpec** framework tests on **HyperExecute** via two different methods: - -- [**Using Local System**](/support/docs/ruby-on-hyperexecute-grid/#1-testing-using-local-system) - You can use your own local machine to execute tests. -- [**Using Gitpod Platform**](/support/docs/ruby-on-hyperexecute-grid/#2-testing-using-gitpod) - Execute tests using GitPod. (Requires a [Gitpod](https://gitpod.io/login/) account) - -## 1. Testing Using Local System - -Follow the step-by-step guide to execute your test on HyperExecute. - -### Prerequisites - -To run the Tests on HyperExecute from your Local System, you are required: - -- Your TestMu AI [Username and Access key](/support/docs/hyperexecute-how-to-get-my-username-and-access-key/) -- [HyperExecute YAML](/support/docs/hyperexecute-yaml-version0.2/) file which contains all the necessary instructions. -- [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) in order to initiate a test execution Job . -- Setup the [Environmental Variable](/support/docs/hyperexecute-environment-variable-setup/) - -### Step 1: Configure Your Test Suite - -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** - -Download or Clone the code sample for the JUnit from the TestMu AI GitHub repository to run the tests on the HyperExecute. - - View on GitHub - -If you are using your own project, make sure you update the **Hub endpoint** in your tests file. - -By setting up the Hub endpoint, you establish the communication channel between your tests and the browser nodes, enabling effective test distribution and execution. - -> You can also use **dev** and **beta** browser versions. To generate capabilities for your test requirements, you can use our inbuilt 🔗 Capabilities Generator. - -### Step 2: Setup the CLI in your Test Suite - -After cloning / downloading the sample repo, you need to setup the CLI and the environment variables. - -#### Download the HyperExecute CLI - -The CLI is used for triggering the tests on HyperExecute. It is recommend to download the CLI binary on the host system and keep it in the root directory of the suite to perform the tests on HyperExecute. - -You can download the CLI for your desired platform from the below mentioned links: - -| Platform | HyperExecute CLI | -| ---------| ---------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| MacOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -#### Setup Environment Variable - -Now, you need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in the [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). - -Run the below mentioned commands in your terminal to setup the CLI and the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -### Step 3: Configure YAML in your Test Suite - -Configure your YAML file as per your use cases using **key value** pairs. - -In this sample YAML file, we have mentioned: - -- **version** of the YAML file -- **Timeouts** for executing your project -- **Mode of execution** is [Autosplit](/support/docs/hyperexecute-auto-split-strategy/). You can also opt for [Matrix](/support/docs/hyperexecute-matrix-multiplexing-strategy/) or [Hybrid](/support/docs/hyperexecute-hybrid-strategy/) mode. -- **Pre and Post** commands -- **Reports and Artefacts** that will be generated after the completion of tests -- and other necessary YAML Parameters - -```yaml ---- -version: "0.1" -globalTimeout: 90 -testSuiteTimeout: 90 -testSuiteStep: 90 - -runson: linux - -autosplit: true -retryOnFailure: true -maxRetries: 1 -concurrency: 2 - -pre: -- bundle install --deployment - -cacheKey: '{{ checksum "Gemfile.lock" }}' -cacheDirectories: -- /vendor - -env: -CONFIG_NAME: "linux" - -testDiscovery: -type: raw -mode: remote -command: grep -nri -E 'it ".*." do' spec/*.rb | awk '{print$1}' | sed 's/\(.*\):/\1 /' - -testRunnerCommand: bundle exec rspec "$test" --format html --out rspec_results.html - -mergeArtifacts: true -uploadArtefacts: -- name: report -path: -- rspec_results.html - -report: true -partialReports: -location: rspec_results.html -type: html - -jobLabel: [selenium-Rspec, linux, autosplit] -``` - -### Step 4: Execute your Test Suite - -> **NOTE :** In case of MacOS, if you get a permission denied warning while executing CLI, simply run **`chmod u+x ./hyperexecute`** to allow permission. In case you get a security popup, allow it from your **System Preferences** → **Security & Privacy** → **General tab**. - -Run the below command in your terminal at the root folder of the project: - -```bash -./hyperexecute --config RELATIVE_PATH_OF_YOUR_YAML_FILE -``` - -OR use this command if you have not exported your username and access key in the step 2. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE `} - -### Step 5: Monitor the Test Execution - -Visit the [HyperExecute Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) and check your Job status. - -### Step 6: Download Artifacts and Reports - -HyperExecute also facilitates the provision to download the [Artifacts](/support/docs/hyperexecute-artifacts/) and [Reports](/support/docs/hyperexecute-reports/) on your local machine. Click on the corresponding button to download your generated artifacts and reports. - -## 2. Testing Using Gitpod - -You can also use the Gitpod platform to execute our sample repository. It will fetch all the sample codebases and trigger the CLI to execute the tests. - -Follow the below steps to run Test using Gitpod: - -**Step 1:** Click '**Open in Gitpod**' button. You will be redirected to Login/Signup page. This button is configured to redirect you to the Gitpod platform where you will be able to execute our sample repository. - -**Step 2:** Login with TestMu AI credentials. Once logged in, a pop-up confirmation will appear, asking you to **'Proceed'** to the Gitpod editor in a new tab. The current tab will display the HyperExecute Dashboard. - -**Step 3:** Choose your preferred editor (we recommend VS Code Editor) - -**Step 4:** As you are running a sample project, Fetching of the Test Scripts, [HyperExecute YAML](/support/docs/deep-dive-into-hyperexecute-yaml/), [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) and Triggering your tests using the `Execution Command` will be automated. - -**Step 5:** Once you see the `Job Link` in the logs, you can visit the [HyperExecute dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) to see the tests getting executed. - -You can also implement [Secret Keys](/support/docs/hyperexecute-how-to-save-and-manage-secrets/) in your YAML file. - -## Navigation in Automation Dashboard - -Every test run on the HyperExecute has a unique *jobId* associated with it. Each *jobId* can in turn constitute single (or multiple) *groupId*(s). You can visit [HyperExecute Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) for checking the status of the test execution. - -You can seamlessly navigate between JobId's and taskId's. You need to click on the *testID* to navigate from the HyperExecute logs to the Automation Dashboard. - -The snapshot below shows the videos, logs and other meta data for that specific *test_ID* - -> For any query or doubt, please feel free to contact us via **24×7 chat support** or you can also drop a mail to **support@testmuai.com**. diff --git a/static/docs/ruby-with-capybara-running-automation-scripts-on-testmu-selenium-grid.md b/static/docs/ruby-with-capybara-running-automation-scripts-on-testmu-selenium-grid.md deleted file mode 100644 index 53382b1c8..000000000 --- a/static/docs/ruby-with-capybara-running-automation-scripts-on-testmu-selenium-grid.md +++ /dev/null @@ -1,118 +0,0 @@ -# Selenium With Capybara - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Run Capybara tests on the TestMu AI cloud grid. This guide covers setup, running a sample test, configuring capabilities, and testing locally hosted pages. - -**Sample repo** -All the code used in this guide is available in the sample repository. - - View on GitHub - -## Prerequisites - -Complete the following steps before you start Ruby automation testing with Selenium. - -1. Install Ruby and gem on your local system. Follow these instructions to install on different operating systems. - * For **Windows**, download from the [Ruby Installer for Windows](https://rubyinstaller.org/downloads/). - * For **Linux** or **Ubuntu**, run a simple apt command like below: -```bash -sudo apt-get install ruby-full -``` - * For **macOS**, run a [Homebrew](https://brew.sh/) command like this: -```bash -brew install ruby -``` -2. Install the [parallel_tests](https://github.com/grosser/parallel_tests) gem to run tests in parallel. -3. Get the TestMu AI binary file for running tests on your locally hosted web pages. - -## Step 1: Clone the Sample Project - -Clone the repository and install dependencies. - -```bash -git clone https://github.com/LambdaTest/Capybara-Cucumber-Ruby.git -cd Capybara-Cucumber-Ruby -``` - -Install project dependencies: -```bash -bundle install -``` - -## Step 2: Set Your Credentials - -Configure your credentials to connect to the TestMu AI Selenium Grid. - -Set TestMu AI Username and Access Key in environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -## Step 3: Configure Your Test Capabilities - -Define browser, version, and OS settings for your test run. - -In the test script, update your test capabilities. This code passes browser, browser version, and operating system information, along with TestMu AI Selenium grid capabilities via the capabilities object. - -```ruby -@caps = { -"browserName"=>lt_browser, -"browserVersion"=>lt_browser_version, -"platformName"=>lt_os, -"resolution"=>lt_res, -"build"=>"capybara-lambdatest", -"name"=>"single-Test-Jenkins", -"video"=>true, -"network"=>true, -"console"=>true, -"visual"=>true -} -``` - -**Capabilities Generator** -Use the TestMu AI [Capabilities Generator](https://www.testmuai.com/capabilities-generator/) to auto-generate the capabilities class for your test requirements. - -## Step 4: Run the Test - -Execute the Capybara test from the command line. - -```bash -bundle exec rake single -``` - -To run parallel tests: -```bash -bundle exec rake parallel -``` - -## Step 5: View Your Results - -Check the test output on the console and the TestMu AI dashboard. - -Visit the [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) to view your test results. The dashboard provides: - -- Text logs for each test step -- Screenshots captured during execution -- Video recordings of the full test session - -## Run Capybara Tests Using Agent Skills - -Use AI coding assistants to generate and run Capybara tests with the TestMu AI Agent Skill. - -The [capybara-skill](https://github.com/LambdaTest/agent-skills/tree/main/capybara-skill) is part of [TestMu AI Agent Skills](https://github.com/LambdaTest/agent-skills/) - structured packages that teach AI coding assistants how to write production-grade test automation. - -Install the skill: - -```bash -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/capybara-skill .claude/skills/ - -# For Cursor / Copilot -cp -r agent-skills/capybara-skill .cursor/skills/ -``` - -Install all available framework skills at once by cloning the repository directly into your tool's skills directory (e.g., `.claude/skills/`, `.cursor/skills/`). diff --git a/static/docs/ruby-with-selenium-running-ruby-automation-scripts-on-testmu-selenium-grid.md b/static/docs/ruby-with-selenium-running-ruby-automation-scripts-on-testmu-selenium-grid.md deleted file mode 100644 index ffeb8ca27..000000000 --- a/static/docs/ruby-with-selenium-running-ruby-automation-scripts-on-testmu-selenium-grid.md +++ /dev/null @@ -1,121 +0,0 @@ -# Selenium With Ruby - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Run your Ruby Selenium tests on the TestMu AI cloud grid across 3000+ browser and OS combinations. The setup is the same for every framework: you connect to the grid and pass your capabilities. This guide covers that shared flow once, then gives you a per-framework quickstart in the tabs below. - -## Prerequisites - -1. [Create a TestMu AI account](https://www.testmuai.com/register/) if you don't have one. -2. Get your **Username** and **Access Key** from the [TestMu AI Dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard). -3. Install [Ruby](https://www.ruby-lang.org/en/downloads/) and the `gem` package manager. -4. Install the `selenium-webdriver` gem (and `parallel_tests` for parallel runs), pulled in by the sample projects via Bundler. - -## Set your credentials - -Every framework authenticates the same way: your Username and Access Key are read from environment variables. Set them once. Pick your operating system: - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME=${ YOUR_LAMBDATEST_USERNAME()} -set LT_ACCESS_KEY=${ YOUR_LAMBDATEST_ACCESS_KEY()}`} - -## How the sample test works - -Every framework below connects to the grid and passes your browser and OS choices through a capabilities hash: - -```ruby -caps = { -:browserName => "chrome", -:version => "latest", -:platform => "Windows 10", -:name => "Ruby Selenium Test", -:build => "Ruby Selenium Build", -:video => true -} -``` - -What changes between frameworks is only how the test is structured and launched. That is what each tab covers. - -Use the [Capabilities Generator](https://www.testmuai.com/capabilities-generator/) to build a capabilities block for any browser, version, and OS combination. - -## Run a test in your framework - -Each tab lists the framework-specific pieces. Clone the matching repo (it contains the full, ready-to-run project), then run. - -Capybara runs with Cucumber for BDD-style tests, driven by Bundler and Rake tasks. - -1. Clone the [sample GitHub project](https://github.com/LambdaTest/Capybara-Cucumber-Ruby): - -```bash -git clone https://github.com/LambdaTest/Capybara-Cucumber-Ruby -cd Capybara-Cucumber-Ruby -``` - -2. Set your browser and OS in the capabilities hash: - -```ruby -@caps = { -"browserName" => "chrome", -"browserVersion" => "latest", -"platformName" => "Windows 10", -"build" => "capybara-lambdatest", -"name" => "single-Test" -} -``` - -3. Run a single test, or in parallel: - -```bash -bundle exec rake single -bundle exec rake parallel -``` - -RSpec loads capabilities from config YAML files and passes them to the remote WebDriver, run via Rake tasks. - -1. Clone the [sample GitHub project](https://github.com/LambdaTest/RSpec-Selenium-Sample): - -```bash -git clone https://github.com/LambdaTest/RSpec-Selenium-Sample -cd RSpec-Selenium-Sample -``` - -2. Set your browser and OS in the config YAML. -3. Run a single test, or in parallel: - -```bash -bundle exec rake single -bundle exec rake parallel -``` - -Test::Unit runs a Ruby test file directly, with capabilities inline, and no Bundler task runner. - -1. Clone the [sample GitHub project](https://github.com/LambdaTest-sample-test-frameworks/Ruby-UnitTest-Selenium): - -```bash -git clone https://github.com/LambdaTest-sample-test-frameworks/Ruby-UnitTest-Selenium -cd Ruby-UnitTest-Selenium -``` - -2. Set your browser and OS in the capabilities hash: - -```ruby -caps = { -:browserName => "chrome", -:browser_version => "latest", -:platform_name => "Windows 10", -:name => "UnitTest Sample Test", -:build => "UnitTest Selenium Build" -} -``` - -3. Run the test: - -```bash -ruby LambdaTest.rb -``` - -## View your results - -Your test results, including video, network logs, and command-by-command execution, appear on the [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build). diff --git a/static/docs/run-espresso-tests-in-bitrise.md b/static/docs/run-espresso-tests-in-bitrise.md deleted file mode 100644 index 2dd2c189b..000000000 --- a/static/docs/run-espresso-tests-in-bitrise.md +++ /dev/null @@ -1,64 +0,0 @@ -# Running Espresso Tests In Bitrise CI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Bitrise offers a mobile CI/CD platform as a service (PaaS) to accelerate the automation of your software development projects. It provides different mobile-first features like exhaustive mobile stack coverage, store code anywhere in cloud or on-premise, and zero hardware setup that make building, testing, and deploying faster and easier. - -Using TestMu AI with Bitrise, you can effortlessly perform [Espresso testing](https://www.testmuai.com/espresso-automation-testing) of your mobile apps on an [online device farm](https://www.testmuai.com/online-device-farm) of real Android devices. - -## Prerequisites - -1. Ensure you have access to TestMu AI [real device cloud](https://www.testmuai.com/real-device-cloud). If not, please [contact sales](https://www.testmuai.com/contact-us). - -2. A Bitrise account. - -3. A GitHub / BitBucket / GitLab or other repository to connect with Bitrise. - -**Note** -If you are getting started with Bitrise CI, please refer to our documentation on [running Appium tests in Bitrise CI](/support/docs/bitrise-integration/). - -## Configuring The TestMu AI App Automate - Espresso Step In Bitrise - -1. Use the search bar to find **TestMu AI App Automate - Espresso**. - -2. Click **TestMu AI App Automate- Espresso** card. - -3. In **Input Variables**, enter **App apk path** and **Test suite apk path**. - -> You can add a public link if your files are accessible through public link. In case, you don't an apk link handy, you can use TestMu AI’s sample application Proverbial for running your first test. -* [Proverbial APK](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk) -* [Test Suite APK](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android_expressotest.apk) - -* In **TestMu AI username**, click Select secret variable. - -* Enter the key (it can be anything), your TestMu AI username and click **Add new**. - -* In **TestMu AI access key**, click Select secret variable. - -* Enter the key (it can be anything), your TestMu AI access key and click **Add new**. - -4. Provide the list of device and operating system version on which you wish to run your tests. - -Other options: - -| Key | Value | Capability Description -| -------- | -----| ------------ | -| Device List | TYPE: STRING DEFAULT: Untitled `"Galaxy S10-9", "OnePlus 6T-10"`| Add the devices here on which the test will be executed. Devices are comma separated. | -|Build Name | TYPE: STRING DEFAULT: Untitled `Android Small Run` | You can group your tests like a job containing multiple tests. | -| Queue Timeout | TYPE: STRING DEFAULT: 600 `300` | Enter the time in seconds after which you want your build to timeout from queue. | -| Idle Timeout | TYPE: STRING DEFAULT: 120 `120` | Enter the time in seconds for maximum running time on a test in the build. | -| Tunnel | TYPE: BOOLEAN `True`OR `False` | To test local applications with TestMu AI. | -| Tunnel name | TYPE: STRING `RabbitHole` | Name of the tunnel. | -| Device logs | TYPE: BOOLEAN DEFAULT: FALSE`True`OR `False`| Enable Device Logs that will show information on the actions performed by the device. | -| Network logs | TYPE: BOOLEAN DEFAULT: FALSE`True`OR `False` | Enable Network Logs that will show the information on the data transmitted & received by the device. | -| Video | TYPE: BOOLEAN DEFAULT: TRUE`True`OR `False` | Video recording of the screen. | - -[Espresso Testing On TestMu AI](/support/docs/getting-started-with-espresso-testing/) - -[Automation Capabilities Generator For Selenium and Appium](https://www.testmuai.com/capabilities-generator/) - -5. Click **⌘ + S** to save the step. - -6. Now go back to the build and click **Rebuild** to build the step. - -7. Once the build process is complete, visit TestMu AI App Automation Dashboard to view your test results. diff --git a/static/docs/run-geb-tests-on-selenium-grid.md b/static/docs/run-geb-tests-on-selenium-grid.md deleted file mode 100644 index 54e9d4828..000000000 --- a/static/docs/run-geb-tests-on-selenium-grid.md +++ /dev/null @@ -1,111 +0,0 @@ -# Selenium With Geb - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Run Geb tests on the TestMu AI cloud grid. This guide covers setup, running a sample test, configuring capabilities, and testing locally hosted pages. - -**Sample repo** -All the code used in this guide is available in the sample repository. - - View on GitHub - -## Prerequisites - -Make sure you have the following set up before you start. - -1. [Create a TestMu AI account](https://www.testmuai.com/register/) if you don't have one. -2. Get your **Username** and **Access Key** from the [TestMu AI Dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard). -3. Install the **Java development environment** (Java 11 recommended). -4. Install **Maven**. Download it from [the official website](https://maven.apache.org/) or install it on **Linux/MacOS** using [**Homebrew**](https://brew.sh/). - -## Step 1: Clone the Sample Project - -Pull the sample repo to your local machine and navigate into the project directory. - -```bash -git clone https://github.com/LambdaTest/geb-parallel-sample-code -cd geb-parallel-sample-code -``` - -Compile Maven dependencies: - -```bash -mvn compile -``` - -You may also want to run the command below to check for outdated dependencies. - -```bash -mvn versions:display-dependency-updates -``` - -## Step 2: Set Your Credentials - -Add your TestMu AI credentials as environment variables so the test can authenticate with the grid. - -Visit the [TestMu AI Dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard), navigate to the left sidebar, and select **Credentials**. Copy your **Username** and **Access Key**, then set them as environment variables: - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -## Step 3: Configure Your Test Capabilities - -Define the browser, version, and OS for your test run. - -Update the `capabilities.json` file with your desired browser, version, and platform: - -```json title="capabilities.json" -{ -"build": "GebFirstTest", -"platformName": "Windows 10", -"browserName": "firefox", -"browserVersion": "latest" -} -``` - -Use the [Capabilities Generator](https://www.testmuai.com/capabilities-generator/) to auto-generate capabilities for any browser, version, and OS combination. - -## Step 4: Run the Test - -Trigger the test from your terminal. - -```bash -./run.sh -``` - -```bash -mvn -Dlambdageb.capabilities=capabilities clean test -``` - -## Step 5: View Your Results - -Check the Automation Dashboard to see exactly what happened during your test. - -Visit the [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) to see your test results. Each session includes: - -- **Video recording** of the full test execution -- **Screenshots** captured at each step -- **Console logs** from the browser -- **Network logs** for every request and response -- **Selenium command logs** showing each driver action - -## Run Geb Tests Using Agent Skills - -Use AI coding assistants to generate and run Geb tests with the TestMu AI Agent Skill. - -The [geb-skill](https://github.com/LambdaTest/agent-skills/tree/main/geb-skill) is part of [TestMu AI Agent Skills](https://github.com/LambdaTest/agent-skills/) - structured packages that teach AI coding assistants how to write production-grade test automation. - -Install the skill: - -```bash -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/geb-skill .claude/skills/ - -# For Cursor / Copilot -cp -r agent-skills/geb-skill .cursor/skills/ -``` - -Install all available framework skills at once by cloning the repository directly into your tool's skills directory (e.g., `.claude/skills/`, `.cursor/skills/`). diff --git a/static/docs/run-oxygen-automation-scripts-on-testmu.md b/static/docs/run-oxygen-automation-scripts-on-testmu.md deleted file mode 100644 index 2945b4883..000000000 --- a/static/docs/run-oxygen-automation-scripts-on-testmu.md +++ /dev/null @@ -1,26 +0,0 @@ -# Run Oxygen Automation Scripts On TestMu AI Selenium Grid - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Oxygen is an open-source test automation framework offering a low code approach for Selenium and Appium tests. The scripts are based on JavaScript and look almost as plain as English, making them intuitive. - -Oxygen supports TestMu AI Selenium Grid, meaning you can configure your TestMu AI account on your Oxygen instance to perform cross browser testing on 3000+ real browser running on real operating systems, on-cloud. That way, you can eliminate the need for configuring in-house Selenium Grid infrastructure and run your Oxygen IDE test over a wider test coverage. - -## Prerequisites - -* Make sure to [download](https://docs.oxygenhq.org/download-installation-start/download-and-installation/download-oxygen-ide) the Oxygen framework for your operating system. -* Retrieve your TestMu AI authentication credentials i.e., your TestMu AI Username, Access Key and Hub URL. You can do so by visiting the [TestMu AI automation dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/) and clicking on the key icon near the help button. - -## Configure TestMu AI Authentication Variables In Oxygen - -**Step 1:** Open the Oxygen framework and click on the cloud icon in the upper-right corner. - -**Step 2:** Turn on the toggle for TestMu AI to run your Oxygen automation scripts on TestMu AI Selenium Grid of 3000+ real browsers. - -**Step 3:** Once you turn the toggle on, you need to fill the mandatory fields to run your Oxygen tests on TestMu AI Selenium Grid. These mandatory fields include your Hub URL, your TestMu AI Username, and your TestMu AI Access Token. - -You can also Capture Browser Console, Capture Network Logs, Take Screenshots, and do Video Recording for your Oxygen test execution over TestMu AI Selenium Grid by enabling the checkboxes. After selecting the preferred checkboxes and filling the mandatory fields, click on **Save & Close**. - -**Step 4:** You will now find a drop-down beside the **Run** button, which says Local, indicating that the tests are being executed on your local machine. To run the tests on the TestMu AI cloud, you would need to select the value for TestMu AI. - -Kudos! That is all you need to do for integrating your Oxygen framework with TestMu AI Selenium Grid. Now, as you run the test, it would be executed over the Selenium Grid offered by TestMu AI, which would enable you to ensure a sturdy cross browser compatible web-application. diff --git a/static/docs/run-parallel-tests-with-selenium.md b/static/docs/run-parallel-tests-with-selenium.md deleted file mode 100644 index ea42da46b..000000000 --- a/static/docs/run-parallel-tests-with-selenium.md +++ /dev/null @@ -1,177 +0,0 @@ -# Run Parallel Tests With Selenium - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Running tests sequentially multiplies wait time across every test. Parallel execution on the TestMu AI cloud grid lets you run hundreds of tests simultaneously across different browser and OS combinations, cutting build times from hours to minutes. - -## How Parallel Testing Works - -Each parallel test opens its own session on the grid. Your framework splits the test suite into threads or processes, and each one connects to a separate browser instance on TestMu AI. - -``` -Your Machine -├── Thread 1 → Chrome / Windows 11 -├── Thread 2 → Firefox / Windows 10 -├── Thread 3 → Safari / macOS Sonoma -└── Thread 4 → Edge / Windows 11 -``` - -The number of sessions you can run simultaneously depends on your TestMu AI plan's concurrency limit. - -## Run Parallel Tests by Framework - -Each framework handles parallelism differently. Below are the commands and configurations for the most common frameworks. - -TestNG uses `parallel="tests"` and `thread-count` in the `testng.xml` file to run tests in parallel. - -```xml title="parallel.testng.xml" - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -``` - -Run with Maven: - -```bash -mvn test -D suite=parallel.xml -``` - -JUnit uses a custom `Parallelized` runner or JUnit 5's `junit.jupiter.execution.parallel.enabled` property. - -**JUnit 5** - add to `junit-platform.properties`: - -```properties -junit.jupiter.execution.parallel.enabled=true -junit.jupiter.execution.parallel.mode.default=concurrent -junit.jupiter.execution.parallel.config.strategy=fixed -junit.jupiter.execution.parallel.config.fixed.parallelism=3 -``` - -**JUnit 4** - run with Maven parallel profile: - -```bash -mvn test -P parallel -``` - -Use the `pytest-xdist` plugin to run tests in parallel. - -Install it: - -```bash -pip install pytest-xdist -``` - -Run with a specified number of workers: - -```bash -pytest -n 3 -``` - -Each worker opens its own browser session on the TestMu AI grid. - -Use the `--parallel` flag (Mocha 8+) to run test files in parallel: - -```bash -mocha --parallel --jobs 3 -``` - -Or use a test runner like `mocha-parallel-tests`: - -```bash -npm install mocha-parallel-tests -mocha-parallel-tests --max-parallel 3 -``` - -NUnit runs tests in parallel using the `[Parallelizable]` attribute: - -```csharp -[TestFixture] -[Parallelizable(ParallelScope.All)] -public class ParallelTests -{ -// Each test method runs in its own thread -} -``` - -Run from the command line: - -```bash -dotnet test -``` - -NUnit handles thread management automatically. - -Use the `parallel_tests` gem: - -```bash -gem install parallel_tests -``` - -Run with a specified number of processes: - -```bash -parallel_rspec -n 3 spec/ -``` - -Use the `paratest` package for parallel PHPUnit execution: - -```bash -composer require brianium/paratest -``` - -Run with a specified number of processes: - -```bash -vendor/bin/paratest -p 3 -``` - -xUnit runs tests in parallel by default across test collections. Run with: - -```bash -dotnet test --filter "profile=parallel" -``` - -## Best Practices - -Follow these guidelines to get the most out of parallel execution. - -- **Match thread count to your concurrency limit.** Running more threads than your plan allows queues the excess sessions. Check your concurrency limit in the [TestMu AI Dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard). -- **Keep tests independent.** Parallel tests must not share state, test data, or depend on execution order. Each test should set up and tear down its own data. -- **Always call `driver.quit()`.** If a test exits without quitting, the session stays open until the idle timeout expires, wasting concurrency slots. -- **Use unique build and test names.** Set the `build` and `name` capabilities to identify which parallel thread ran which test on the dashboard. -- **Start small.** Begin with 2-3 parallel threads and increase gradually. Monitor the [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) for failures caused by test interference. - -## Verify Your Parallel Runs - -Check the dashboard to confirm tests ran simultaneously. - -Visit the [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) and open your build. If tests ran in parallel, you will see multiple sessions with overlapping start times. The build duration should be roughly equal to the duration of the longest individual test, not the sum of all tests. diff --git a/static/docs/run-python-tests-on-selenium-grid.md b/static/docs/run-python-tests-on-selenium-grid.md deleted file mode 100644 index aca10421f..000000000 --- a/static/docs/run-python-tests-on-selenium-grid.md +++ /dev/null @@ -1,134 +0,0 @@ -# Quick Guide To Run Python Tests On TestMu AI Selenium Grid - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI Selenium Automation Grid is a cloud based scalable [Selenium testing](https://www.testmuai.com/selenium-automation) platform which enables you to run your automation scripts on 3000+ different browsers and operating systems. You can leverage TestMu AI Selenium grid to not only decrease the build time of Python automation scripts, but also increase your test coverage and check for more range of browsers for cross browser compatibility. This post will help you in getting started with configuring and running your Python based automation test scripts on TestMu AI Selenium cloud platform. - -## Prerequisites - -- Make sure that you have the latest Python build installed. You can download the latest Python build from [this page](https://www.python.org/downloads/). - -- You would need Python package manager, pip, as well. Usually latest python installers come prepacked with pip. However, if you don’t have pip, you can install it using the below command: - - curl https://bootstrap.pypa.io/get-pip.py | python - -## Installing Selenium Dependencies For Python - -Make sure that you have installed Selenium dependencies before executing your tests. You can use pip to install Selenium using following command: - - pip install selenium - export PYTHONWARNINGS="ignore:Unverified HTTPS request" //Disable SSL warning - -## Sample Python & Selenium Test To Get You started - -Now we are all set to execute the first Python test on TestMu AI Selenium Grid. Checkout the sample code below. This is a simple python code that uses TestMu AI Selenium grid to open up Google. You can also download it from our [GitHub repo](https://github.com/LambdaTest/python-selenium-sample). - -### Sample Python & Selenium Automation Script - -``` -""" -LambdaTest Selenium automation sample example -Configuration ----------- -username: Username can be found at automation dashboard -accessToken: AccessToken can be generated from automation dashboard or profile section - -Result -------- -Execute Python Automation Tests on LambdaTest Distributed Selenium Grid -""" -import unittest -import time -from selenium import webdriver -from selenium.webdriver.common.keys import Keys - -class LTAutomate(unittest.TestCase): - -""" -Setup remote driver -Params ----------- -platform : Supported platform - (Windows 10, Windows 8.1, Windows 8, Windows 7, macOS High Sierra, macOS Sierra, OS X El Capitan, OS X Yosemite, OS X Mavericks) -browserName : Supported platform - (chrome, firefox, Internet Explorer, MicrosoftEdge) -version : Supported list of version can be found at https://www.testmuai.com/capabilities-generator/ - -Result -------- -""" -def setUp(self): -# username: Username can be found at automation dashboard -username="{YOUR_LAMBDATEST_USERNAME}" -# accessToken: AccessToken can be generated from automation dashboard or profile section -accessToken="{YOUR_LAMBDATEST_ACCESS_KEY}" -# gridUrl: gridUrl can be found at automation dashboard -gridUrl = "hub.lambdatest.com/wd/hub" - -desired_cap = { -'platform' : "win10", -'browserName' : "chrome", -'version' : "67.0", -# Resolution of machine -"resolution": "1024x768", -"name": "LambdaTest python google search test ", -"build": "LambdaTest python google search build", -"network": True, -"video": True, -"visual": True, -"console": True, -} - -# URL: https://{username}:{accessToken}@hub.lambdatest.com/wd/hub -url = "https://"+username+":"+accessToken+"@"+gridUrl - -print("Initiating remote driver on platform: "+desired_cap["platform"]+" browser: "+desired_cap["browserName"]+" version: "+desired_cap["version"]) -self.driver = webdriver.Remote( -desired_capabilities=desired_cap, -command_executor= url -) - -""" -Setup remote driver -Params ----------- -Execute test: navigate google.com search LambdaTest -Result -------- -print title -""" -def test_search_in_google(self): -driver = self.driver -print("Driver initiated successfully. Navigate url") -driver.get("https://www.google.com/ncr") - -print("Searching lambdatest on google.com ") -time.sleep(8) -elem = driver.find_element_by_name("q") -elem.send_keys("lambdatest.com") -elem.submit() - -print("Printing title of current page :"+driver.title) -driver.execute_script("lambda-status=passed") -print("Requesting to mark test : pass") - -""" -Quit selenium driver -""" -def tearDown(self): -self.driver.quit() - -if __name__ == "__main__": -unittest.main() - -``` - -You can execute the test from your terminal through this command. - - python google-search-lambdatest.py - -The first thing to note here, is the use of remote WebDriver. Also checkout how the test passed on the enthronement configuration via desired_cap class. - -To use your choice of environment for the test you can leverage our Capability Generator tool. - -TestMu AI Capability Generator Tool - -> **Tear Down**: The tear down method helps TestMu AI platform understand when the execution of a test is finished. It’s important to tear down each test or else it will give timeout errors. diff --git a/static/docs/run-selenium-ide-tests-on-testmu-selenium-cloud-grid.md b/static/docs/run-selenium-ide-tests-on-testmu-selenium-cloud-grid.md deleted file mode 100644 index ea4ae035a..000000000 --- a/static/docs/run-selenium-ide-tests-on-testmu-selenium-cloud-grid.md +++ /dev/null @@ -1,98 +0,0 @@ -# Run Selenium IDE Tests with TestMu AI Selenium Grid - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -[Selenium IDE](https://www.testmuai.com/blog/selenium-ide-what-is-it-why-is-it-must-for-every-qa/) is one of the most popular record and playback tool for testing which is widely used for automated browser testing. The Selenium IDE is a very easy-to-use solution for getting started with automation testing. It also provides instant feedback and has cross-browser and operating system compatibility, with the help of Selenium IDE (SIDE) Runner. - -In this document, we will see how to run Selenium IDE tests on TestMu AI cloud [Selenium Grid](https://www.testmuai.com/blog/why-selenium-grid-is-ideal-for-automated-browser-testing/). - -## Prerequisites - -* [Node.js](https://nodejs.org/en/download/) programming language (version 8 or 10). -* [NPM or Node.js Package Manager](https://www.npmjs.com/get-npm). -* Selenium IDE Plugin, for either [Google Chrome](https://chrome.google.com/webstore/detail/selenium-ide/mooikfkahbdckldjjndioackbalphokd?hl=en) and [Mozilla Firefox](https://addons.mozilla.org/en-US/firefox/addon/selenium-ide/). -* Selenium IDE (SIDE) Runner. You can download SIDE runner using below code: - -```javascript -npm install -g selenium-side-runner -``` - -* You also need your TestMu AI authentication credentials, which includes your TestMu AI username, and access key. These will be set up as the Secrets for configuring the pipeline. To get your Username and Access Key, go to your [TestMu AI automation dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/) and click on the "key" icon left to the help button, on the top right corner. - -For the demonstration of executing Selenium IDE tests on the TestMu AI grid, we’ll create a test suite that comprises of four test cases- two to perform a search for ‘TestMu AI’ on Google & DuckDuckGo, and the other two to perform a search for ‘TestMu AI Blog’ on Google & DuckDuckGo. - -## Enabling Parallel Testing for Selenium IDE - -The parallel testing on the [Cloud Selenium Grid](https://www.testmuai.com/selenium-automation) works only when the tests are executed using the SIDE Runner, and not the Selenium IDE. So we will enable the Parallel testing for Selenium IDE. - -* Right click on the test suite and go to Settings. -* In the Settings window, tick the **Run in Parallel** checkbox. -* Save the SIDE project. - -We have successfully activated Parallel Testing in Selenium IDE. Now we can execute tests Selenium IDE tests in parallel on TestMu AI Selenium Cloud Grid. - -## Executing Selenium IDE Tests on TestMu AI Selenium Cloud Grid - -1. **Set the [Desired Capabilities in Selenium](/docs/selenium-automation-capabilities/).** For this article, we have set the Chrome browser (version 72.0) for Windows 10, to run our Selenium IDE tests. - - The [TestMu AI Selenium Desired Capabilities Generator](https://www.testmuai.com/capabilities-generator/) is used to generate the capabilities. Below are the desired capabilities used in this article for the required test combination, provided in Python Language. -```javascript -capabilities = { -"platform" : "Windows 10", -"browserName" : "Chrome", -"version" : "72.0" -} -``` -```javascript -In the string format, it will be represented as: **"browserName='chrome' version='72.0' platform='Windows 10'"** -``` - -2. **Get the Grid URL for executing tests on TestMu AI.** Get your username and access key from your TestMu AI's Profile section (as shown in the prerequisites), and replace it in the below URL. - -```javascript -https://user-name:access-key@hub.lambdatest.com/wd/hub -``` - -3. **Specify the desired capabilities along with the Grid URL.** Replace the desired capabilities and the Grid URL from the above steps in the below code: - -```javascript -selenium-side-runner --server --capabilities -``` - -4. **Mention the number of parallel processes to be executed at a time.** You can use any of the below syntax to specify the count of parallel processes to be executed at the above mentioned Grid URL: -```javascript -selenium-side-runner --w --server https://user-name:access-key@hub.lambdatest.com/wd/hub -``` -OR - -```javascript -selenium-side-runner --max-workers --server https://user-name:access-key@hub.lambdatest.com/wd/hub -``` - - For this article, we have set the number of parallel processes as '4' - -5. Now we have all the components of the SIDE Runner command to execute the tests. So we will **run the Selenium IDE tests on TestMu AI Selenium Cloud Grid.** Navigate to the directory where the .SIDE file is located (here it is located at "_TestMu AI-IDE.side_") and execute the below command in the terminal, after replacing the GRID URL: - -```javascript -selenium-side-runner -w 4 "LambdaTest-IDE.side" --server https://user-name:acces-key@hub.lambdatest.com/wd/hub -c "browserName='chrome' version='72.0' platform='Windows 10'" -``` - -We have successfully executed all the steps to run Selenium IDE tests on TestMu AI Selenium Cloud Grid. Now you can see our tests being executed along with their status in the [Automation Timeline of your TestMu AI profile](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/timeline). - -Below are the execution snapshots of the SIDE Runner and the automation timeline instance, that shows the successful execution of the desired tests: - -## How To Run Multiple Test Cases Within Same Test Suite? - -You might have lots of individual tests in a single test suite to test all features of a site. When you run these tests normally, you might experience that the test suites get broken into individual tests, thereby requiring a new session and the variables being passed again each time. - -Selenium IDE provides the feature to overcome this as well. In order to run multiple test cases within the same test suite with a single instance of a browser, the Persist session option in the Selenium IDE can be used. Below are the steps for how to do so: - -1. Click over the three dot option of the test suite. Choose the setting option. -2. Then choose persist session option and submit. -3. For the second query, you pass the variable value through the SIDE Runner using command line. You can use the SIDE Runner's "**param**" flag to pass the values. For example: - -```javascript -selenium-side-runner --params "a='example-value' a.b='another example-value' a.b.c=[1,2,3]" -``` - -Congrats! We have successfully executed our Selenium IDE tests on TestMu AI Selenium Cloud Grid, which allows you to perform live interactive and automated [cross browser testing](https://www.testmuai.com) on 3000+ real browsers and operating systems online. diff --git a/static/docs/run-settings.md b/static/docs/run-settings.md deleted file mode 100644 index ef76bb836..000000000 --- a/static/docs/run-settings.md +++ /dev/null @@ -1,432 +0,0 @@ -# Configuring Cypress Test Execution - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -You can specify Cypress CLI flags to run on TestMu AI in two ways: - -1. Adding the CLI flag details in `lambdatest-config.json` file -2. Using the CLI - -If a glob pattern path is passed in the specs flag using CLI, you must enclose it within double quotes as shown: -`lambdatest-cypress-cli run --env=stage --verbose --specs "./cypress/integration/examples/*"` - -## Cypress Config File -#### Using `lambdatest-config.json` -You can use the `cypress_config_file` key in `run_settings` option to specify the Cypress configuration file. - -> **NOTE :** Cypress 10 and above versions automatically identify the `cypress.config.js` file in the project. Therefore, you don't need to specify the `cypress_config.js` capability. - -```javascript title="lambdatest-config.json" -{ -"run_settings": { -"cypress_config_file": "cypress.json", -"video": true -} -} -``` - -```javascript title="lambdatest-config.json" -// In case of Cypress 10 and above, you can specify your custom config files by using the --cy flag. -lambdatest-cypress-cli run --env=stage,video=true --verbose --specs "./cypress/integration/examples/*" -``` - -#### Using CLI -You can specify the Cypress configuration file using the below CLI flag with `run` command. - -| Flag | Purpose | Type | -| ---------------------------- | ----------------------- | ------ | -| --ccf, --cypress-config-file | Path of the config file | String | - -## Specifying Spec Files -#### Using `lambdatest-config.json` -You can use the *specs* key in *run_settings* option to specify the Spec files. - -```javascript title="lambdatest-config.json" -{ -"run_settings": { -"specs": "**/*.cy.js", -} -} -``` - -```javascript title="lambdatest-config.json" -{ -"run_settings": { -"specs": "./cypress/integration/examples/*.spec.js", -} -} - -``` - -#### Using CLI -You can specify the Spec files using the below CLI flag with `run` command. - -| Flag | Purpose | Type | -| ----------- | --------------------------------------------- | ------ | -| -s, --specs | Path of the spec file or directory or pattern | String | - -## Exclude Specs Files -You can use the *exclude_specs* key in *run_settings* option to specify the spec files that you want to exclude from the test execution. - -```javascript title="lambdatest-config.json" -{ -"run_settings": { -"exclude_specs": "./examples/assertion.spec.js, ./examples/connectors.spec.js", -} -} -``` - -```javascript title="lambdatest-config.json" -{ -"run_settings": { -"exclude_specs": "./examples/assertion.spec.js, ./examples/connectors.spec.js", -} -} -``` - -## Geolocation -#### Using `lambdatest-config.json` -You can use the *geo_location* key in *run_settings* option to specify the Spec files. - -```javascript title="lambdatest-config.json" -{ -"run_settings": { -"geo_location": "", -} -} -``` - -```javascript title="lambdatest-config.json" -{ -"run_settings": { -"geo_location": "", -} -} -``` - -#### Using CLI -You can specify the geolocation files using the below CLI flag with `run` command. - -| Flag | Purpose | Type | -| --------------------- | ------------------------- | ------ | -| --geo, --geo_location | Pass the Geo country code | String | - -## Resolution -#### Using `lambdatest-config.json` -Use the *resolution* key in *run_settings* option to specify the resolution. - -```javascript title="lambdatest-config.json" -{ -"run_settings": { -"resolution": "1024x768", -} -} -``` - -```javascript title="lambdatest-config.json" -{ -"run_settings": { -"resolution": "1024x768", -} -} -``` - -#### Using CLI -You can specify the system's resolution using the below CLI flag with `run` command. - -| Flag | Purpose | Type | -| ------ | ------------------- | ------ | -| --res | Pass the resolution | String | - -*It will change the resolution of the system but the viewport in which Cypress launches the website is controlled by Cypress.* - -**Supported resolutions**: `1024x768`, `1280x960`, `1280x1024`, `1600x1200`, `1920x1080`, `2048x1536`, `2560x1440` - -The `resolution` capability above sets the VM's screen resolution. To control the browser **viewport** that Cypress renders the app in (which `resolution` does not change), use Cypress's own viewport and full-screen options. - -#### Test on viewports -When you define a viewport, Cypress uses the browser's scaling to size the app: a large viewport scales the app down, a small one scales it up. Set the viewport globally with `viewportWidth` and `viewportHeight` in the Cypress config, or per test: - -```js -cy.viewport(550, 750) // Set viewport to 550px x 750px -cy.viewport('iphone-6') // Set viewport to 375px x 667px -``` - -For more details, see the [official Cypress documentation](https://docs.cypress.io/api/commands/viewport#Syntax). - -#### Test in full-screen mode - -Full-screen mode together with viewports gives the best results for screen-resolution testing. - -Use the `before:browser:launch` event to change the browser options. - -For **Cypress v9 and below**, use the following script in the `plugin/index.js` file: - -```js -module.exports = (on, config) => { -on('before:browser:launch', (browser = {}, launchOptions) => { -if (browser.family === 'chromium' && browser.name !== 'electron') { -launchOptions.args.push('--start-fullscreen') - -return launchOptions -} - -if (browser.name === 'electron') { -launchOptions.preferences.fullscreen = true - -return launchOptions -} -}) -} -``` - -For **Cypress v10 and above**, add the below code in the `cypress.config.js` file: - -```js -const { defineConfig } = require('cypress') - -module.exports = defineConfig({ - -e2e: { -setupNodeEvents(on, config) { -on('before:browser:launch', (browser = {}, launchOptions) => { -if (browser.family === 'chromium' && browser.name !== 'electron') { -launchOptions.args.push('--start-fullscreen') - -return launchOptions -} - -if (browser.name === 'electron') { -launchOptions.preferences.fullscreen = true - -return launchOptions -} -}) -} -} -}) -``` - -## Excluding Files From Test Uploads -#### Using `lambdatest-config.json` -You can use the *ignore_files* key in *run_settings* option to ignore or exclude any particular files while uploading your tests. - -```javascript title="lambdatest-config.json" -{ -"run_settings": { -"ignore_files": "", -} -} -``` - -```javascript title="lambdatest-config.json" -{ -"run_settings": { -"ignore_files": "", -} -} -``` - -#### Using CLI -Specify the files that you want to exclude using the below CLI flag with `run` command. - -| Flag | Purpose | Type | -| ------------------- | ---------------------------------- | ------ | -| --if, --ignore_files| Files to ignore in the zip project | String | - -## Max Duration for Test Session Timeout -When running tests, if you encounter a situation where a particular test is running for an extended period and causing other parallel tests to be blocked, in this case, you can use the *max duration* flag to mitigate the issue. This flag allows you to specify a maximum time limit for the test to run before it is automatically stopped, thus freeing up resources for other tests to execute. - -The valid input for `max_duration` must be an integer, and it can only be within the range of 2 to 240 minutes. -#### Using `lambdatest-config.json` -You can use the `max_duration` key in *run_settings* option to specify the session timeout. - -```javascript title="lambdatest-config.json" -{ -"run_settings": { -"max_duration":2 -} -} -``` - -```javascript title="lambdatest-config.json" -{ -"run_settings": { -"max_duration":2 -} -} -``` - -#### Using CLI - You can specify the timeout using the below CLI flag with `run` command. - -| Flag | Purpose | Type | -| -------------- | ------------------------------ | ------- | -| --max_duration | idle timeout of a test session | Integer | - -## Organizing Tests -You can name your test cases and categorize your Cypress builds by build number, build name, test and build tags for easier analysis. Use the following keys to organize your tests. - -| Key | Description | Type | -| ------------ | -------------------------------------------- | ------ | -| build-name | Represent the name of the build for your test | String | -| project-name | Represents the name of your project | String | -| tags | Group your tests | String | - -**Example**: - -```javascript title="lambdatest-config.json" -{ -"run_settings": { -"build-name": "Cypress 10 Demo", -"project-name": "DemoTest", -"tags": "Sprint1", -} -} -``` - -## Headless Browser -You can run Headless tests with Cypress by the `headless` key to `true`. - -| Key | Description | Type | -| -------- | ------------------------- | ------- | -| headless | Run test in Headless mode | Boolean | - -**Example**: - -```javascript title="lambdatest-config.json" -{ -"run_settings": { -"headless": "true", -} -} -``` - -## Capture Network Logs -You can generate Network logs with Cypress by setting the `network` key to `true`. - -| Key | Description | Type | -| -------- | ------------ | -------- | -| headless | Record network packets while the test is running | Boolean | - -**Example**: - -```javascript title="lambdatest-config.json" -{ -"run_settings": { -"network": "true", -} -} -``` - -## NPM Package Dependencies -In order to run your tests on TestMu AI, we refer to your `package.json` and use those dependencies and devDependencies. Since, `package.json` may contain several dependencies which may not be required to run your Cypress tests. We recommend to use `npm_dependencies` parameter to list down the required dependencies to run your test, because it will reduce your build time on TestMu AI. - -Below are the ways through which TestMu AI detects the dependencies which has to be installed before running the test on TestMu AI. - -#### Using `npm_dependencies` -Inside `run_settings` of `lambdatest-config.json`, you can provide the list of NPM dependencies: - -```javascript title="lambdatest-config.json" -"run_settings": { -"npm_dependencies": { -"cypress": "9.0.0", -}, -}, -``` - -#### Using `package.json` -TestMu AI will automatically detects the dependencies that has to be installed from `package.json`. - -```javascript title="package.json" -"devDependencies": { -"cypress": "9.2.1", -"eslint": "7.0.0", -"eslint-plugin-json-format": "2.0.1", -"eslint-plugin-mocha": "5.3.0", -}, -``` - -**note** -It's recommended to use `npm_dependencies` instead of `package.json` because `package.json` may contain the dependencies which are not actually used while running the Cypress tests and also, installing these unwanted dependencies will increase the build time. - -#### Install npm packages via a tunnel -Use the `npm_via_tunnel` flag to route npm install traffic through a tunnel. This is useful when installing packages from a private registry, or in a restricted environment where the public npm registry isn't directly reachable. Add it to `run_settings`: - -```bash -"npm_via_tunnel": true -``` - -You must already have a tunnel configured and running. To download private dependencies, see [Include Private Dependencies](/support/docs/private-dependencies-cypress/). - -## Specific Node Version -TestMu AI allows you to specify the Node.js version for running your tests using the ``useNodeVersion`` capability in the `run_settings` section of the `lambdatest-config.json` file. This capability provides greater flexibility and ensures compatibility with your project's dependencies and environment. - -> Supported Versions are 18, 20 and 22. The exact version will be used are `18.17.1`, `20.18.0`, and `22.11.0` respectively - -```javascript title="lambdatest-config.json" -"run_settings": { -"useNodeVersion":"20" -} -``` - -## Environment Variables -You can set environment variables for your Cypress tests in three ways: via the Cypress config file, via `cypress.env.json`, or via the TestMu AI Cypress CLI. If you set variables via both the CLI and `cypress.env.json`, the `cypress.env.json` file is ignored and only the CLI values are used. - -#### Via the config file -**Cypress 9.** A sample `cypress.json`: - -```bash -{ -...... -"env":{ -"CYPRESS_BASE_URL":"https://example.cypress.io/", -"ACTIONS_URL": "commands/actions", -"WINDOW_URL": "commands/window" -}, -...... -} -``` - -**Cypress 10.** A sample `cypress.config.js`: - -```bash -module.exports = defineConfig({ -env: { -'CYPRESS_BASE_URL':'https://example.cypress.io/', -'ACTIONS_URL' : 'commands/actions', -'WINDOW_URL': 'commands/window' -}, -``` - -Use them in your test spec: - -```bash -describe('Sample test', () => { -it('test case - actions', () => { -cy.visit(Cypress.env('CYPRESS_BASE_URL') + Cypress.env('ACTIONS_URL')) -cy.wait(3000) -}) -it('test case - window', () => { -cy.visit(Cypress.env('CYPRESS_BASE_URL') + Cypress.env('WINDOW_URL')) -cy.wait(3000) -}) -}) -``` - -#### Via `cypress.env.json` - -```bash -{ -"CYPRESS_BASE_URL":"https://example.cypress.io/", -"ACTIONS_URL" : "commands/actions", -"WINDOW_URL": "commands/window" -} -``` - -#### Via the Cypress CLI -Add variables with the `--envs` parameter: - -```bash -lambdatest-cypress run --envs "CYPRESS_BASE_URL=https://example.cypress.io/,ACTIONS_URL=commands/actions,WINDOW_URL=commands/window" -``` diff --git a/static/docs/run-your-cypress-tests-in-parallel.md b/static/docs/run-your-cypress-tests-in-parallel.md deleted file mode 100644 index 46d4674c6..000000000 --- a/static/docs/run-your-cypress-tests-in-parallel.md +++ /dev/null @@ -1,91 +0,0 @@ -# Run Your Cypress Tests In Parallel - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -If you want to run your tests parallelly on the TestMu AI platform, you can do that using either of the following two ways: - -### 1. Run your Cypress tests Parallelly using the CLI - -To perform Cypress testing parallelly using the CLI, you need to use the "`--parallels`" option while running your tests. - -**Syntax:** - -```bash ---parallels -``` - -For example, if you want to run your Cypress tests on 5 parallel sessions, you can use the below command: - -```bash -lambdatest-cypress run --parallels 5 -``` - -### 2. Run your Cypress tests Parallelly using the lambdatest-configuration.json file - -To run your Cypress testing parallelly, you can also use the **lambdatest-configuration.json** file, using the '**parallels**' key. - -**Syntax:** - -```json -{ -"run_settings": { -... -"parallels": , -... -} -} -``` - -For example, if you want to run your Cypress tests on 5 parallel sessions, use the below settings: - -```json -{ -"run_settings": { -... -"parallels": 5, -... -} -} -``` - -**NOTE** - -The number of parallel test running on the TestMu AI platform at a time, is based on the concurrency plan of your TestMu AI account. In case the number of parallel tests is more than the concurrency plan, the remaining tests will get queued and run after the existing tests are complete. For example, suppose you have a concurrency plan of 5 parallel sessions and want to run 50 tests parallelly. In this case, only 5 parallel tests will be executed at a time, and the rest 45 will be queued. As the test finishes running, the queued tests will be moved to the running state based on availability. - -## Using the Cypress Agent Skill with TestMu AI - -The [cypress-skill](https://github.com/LambdaTest/agent-skills/tree/main/cypress-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The cypress-skill package includes: - -``` -cypress-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing Cypress Agent Skill - -Install a Cypress Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/cypress-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/cypress-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only cypress-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). diff --git a/static/docs/running-cucumber-scripts-with-testng-and-selenium.md b/static/docs/running-cucumber-scripts-with-testng-and-selenium.md deleted file mode 100644 index f029ebbef..000000000 --- a/static/docs/running-cucumber-scripts-with-testng-and-selenium.md +++ /dev/null @@ -1,263 +0,0 @@ -# Selenium With Cucumber - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Run Cucumber tests on the TestMu AI cloud grid. This guide covers setup, running a sample test, configuring capabilities, and testing locally hosted pages. - -**Sample repo** -All the code used in this guide is available in the sample repository. - - View on GitHub - -## Prerequisites - -Make sure you have the following set up before you start. - -1. [Create a TestMu AI account](https://www.testmuai.com/register/) if you don't have one. -2. Get your **Username** and **Access Key** from the [TestMu AI Dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard). -3. Install the **Java development environment** (Java 11 recommended). -4. Install **Maven**. Download it from [the official website](https://maven.apache.org/) or install it on **Linux/MacOS** using [**Homebrew**](https://brew.sh/). - -## Step 1: Clone the Sample Project - -Pull the sample repo to your local machine and navigate into the project directory. - -```bash -git clone https://github.com/LambdaTest/cucumber-testng-sample -cd cucumber-testng-sample -``` - -You may also want to run the command below to check for outdated dependencies. - -```bash -mvn versions:display-dependency-updates -``` - -## Step 2: Set Your Credentials - -Add your TestMu AI credentials as environment variables so the test can authenticate with the grid. - -Visit the [TestMu AI Dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard), navigate to the left sidebar, and select **Credentials**. Copy your **Username** and **Access Key**, then set them as environment variables: - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -## Step 3: Configure Your Test Capabilities - -Define the browser, version, and OS for your test run. - -```java -ChromeOptions browserOptions = new ChromeOptions(); -browserOptions.setPlatformName(platform); -browserOptions.setBrowserVersion("latest"); - -HashMap ltOptions = new HashMap(); -ltOptions.put("build", "Your Build Name"); -ltOptions.put("w3c", true); -browserOptions.setCapability("LT:Options", ltOptions); -``` - -Use the [Capabilities Generator](https://www.testmuai.com/capabilities-generator/) to auto-generate capabilities for any browser, version, and OS combination. - -Below is the sample feature file for Cucumber: - -```bash -Feature: Add new item to ToDO list - -Scenario: Lambdatest ToDO Scenario - -Given user is on home Page -When select First Item -Then select second item -Then add new item -Then verify added item -``` - -Here is the `TestRunner` file to automate the feature file through Selenium using TestNG: - -```java title="TestRunner" -package MyRunner; - -import java.net.URL; -import java.util.HashMap; - -import org.openqa.selenium.chrome.ChromeOptions; -import org.openqa.selenium.remote.RemoteWebDriver; -import org.testng.annotations.AfterClass; -import org.testng.annotations.AfterMethod; -import org.testng.annotations.BeforeClass; -import org.testng.annotations.BeforeMethod; -import org.testng.annotations.DataProvider; -import org.testng.annotations.Parameters; -import org.testng.annotations.Test; - -import cucumber.api.CucumberOptions; -import cucumber.api.testng.CucumberFeatureWrapper; -import cucumber.api.testng.TestNGCucumberRunner; - -@CucumberOptions( -features = "src/main/java/Features", -glue = {"stepDefinitions"}, -tags = {"~@Ignore"}, -format = { -"pretty", -"html:target/cucumber-reports/cucumber-pretty", -"json:target/cucumber-reports/CucumberTestReport.json", -"rerun:target/cucumber-reports/rerun.txt" -},plugin = "json:target/cucumber-reports/CucumberTestReport.json") - -public class TestRunner { - -private TestNGCucumberRunner testNGCucumberRunner; - -public static RemoteWebDriver connection; - -@BeforeClass(alwaysRun = true) -public void setUpCucumber() { -testNGCucumberRunner = new TestNGCucumberRunner(this.getClass()); -} - -@BeforeMethod(alwaysRun = true) -@Parameters({ "browser", "version", "platform" }) -public void setUpClass(String browser, String version, String platform) throws Exception { - -String username = System.getenv("LT_USERNAME") == null ? "YOUR LT_USERNAME" : System.getenv("LT_USERNAME"); -String accesskey = System.getenv("LT_ACCESS_KEY") == null ? "YOUR LT_ACCESS_KEY" : System.getenv("LT_ACCESS_KEY"); - -ChromeOptions browserOptions = new ChromeOptions(); -browserOptions.setPlatformName(platform); -browserOptions.setBrowserVersion("latest"); - -HashMap ltOptions = new HashMap(); -ltOptions.put("build", "Your Build Name"); -ltOptions.put("w3c", true); -browserOptions.setCapability("LT:Options", ltOptions); -String gridURL = "https://" + username + ":" + accesskey + "@hub.lambdatest.com/wd/hub"; -System.out.println(gridURL); -connection = new RemoteWebDriver(new URL(gridURL), browserOptions); -System.out.println(browserOptions); -System.out.println(connection); -} - -@Test(groups = "cucumber", description = "Runs Cucumber Feature", dataProvider = "features") -public void feature(CucumberFeatureWrapper cucumberFeature) { -testNGCucumberRunner.runCucumber(cucumberFeature.getCucumberFeature()); -} - -@DataProvider -public Object[][] features() { -return testNGCucumberRunner.provideFeatures(); -} - -@AfterClass(alwaysRun = true) -public void tearDownClass() throws Exception { -testNGCucumberRunner.finish(); -} -} -``` - -Below are the step definitions: - -```java title="ToDoStepDefinition.java" -package stepDefinitions; - -import org.openqa.selenium.By; -import org.openqa.selenium.remote.RemoteWebDriver; -import org.testng.Assert; - -import cucumber.api.Scenario; -import cucumber.api.java.After; -import cucumber.api.java.Before; -import cucumber.api.java.en.Given; -import cucumber.api.java.en.Then; -import cucumber.api.java.en.When; -import MyRunner.*; - -public class ToDoStepDefinition extends TestRunner { - -public RemoteWebDriver driver = this.connection; - -@Before -public void updateName(Scenario scenario) { -driver.executeScript("lambda-name="+scenario.getName()); -} - -@Given("^user is on home Page$") -public void user_already_on_home_page() { -System.out.println(driver.getCapabilities()); -driver.get("https://lambdatest.github.io/sample-todo-app/"); - -} - -@When("^select First Item$") -public void select_first_item() { -driver.findElement(By.name("li1")).click(); -} - -@Then("^select second item$") -public void select_second_item() { -driver.findElement(By.name("li2")).click(); -} - -@Then("^add new item$") -public void add_new_item() { -driver.findElement(By.id("sampletodotext")).clear(); -driver.findElement(By.id("sampletodotext")).sendKeys("Yey, Let's add it to list"); -driver.findElement(By.id("addbutton")).click(); -} - -@Then("^verify added item$") -public void verify_added_item() { -String item = driver.findElement(By.xpath("/html/body/div/div/div/ul/li[6]/span")).getText(); -Assert.assertTrue(item.contains("Yey, Let's add it to list")); -} - -@After -public void close_the_browser(Scenario scenario) { -driver.executeScript("lambda-status=" + (scenario.isFailed() ? "failed" : "passed")); -driver.quit(); -} - -} -``` - -## Step 4: Run the Test - -Trigger the test from your terminal. - -```bash -mvn test -``` - -## Step 5: View Your Results - -Check the Automation Dashboard to see exactly what happened during your test. - -Visit the [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) to see your test results. Each session includes: - -- **Video recording** of the full test execution -- **Screenshots** captured at each step -- **Console logs** from the browser -- **Network logs** for every request and response -- **Selenium command logs** showing each driver action - -## Run Cucumber Tests Using Agent Skills - -Use AI coding assistants to generate and run Cucumber tests with the TestMu AI Agent Skill. - -The [cucumber-skill](https://github.com/LambdaTest/agent-skills/tree/main/cucumber-skill) is part of [TestMu AI Agent Skills](https://github.com/LambdaTest/agent-skills/) - structured packages that teach AI coding assistants how to write production-grade test automation. - -Install the skill: - -```bash -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/cucumber-skill .claude/skills/ - -# For Cursor / Copilot -cp -r agent-skills/cucumber-skill .cursor/skills/ -``` - -Install all available framework skills at once by cloning the repository directly into your tool's skills directory (e.g., `.claude/skills/`, `.cursor/skills/`). diff --git a/static/docs/running-gauge-tests-on-testmu-selenium-grid.md b/static/docs/running-gauge-tests-on-testmu-selenium-grid.md deleted file mode 100644 index 006e0f9ed..000000000 --- a/static/docs/running-gauge-tests-on-testmu-selenium-grid.md +++ /dev/null @@ -1,180 +0,0 @@ -# Selenium With Gauge - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Run Gauge tests on the TestMu AI cloud grid. This guide covers setup, running a sample test, configuring capabilities, and testing locally hosted pages. - -**Sample repo** -All the code used in this guide is available in the sample repository. - - View on GitHub - -## Prerequisites - -Make sure you have the following set up before you start. - -1. [Create a TestMu AI account](https://www.testmuai.com/register/) if you don't have one. -2. Get your **Username** and **Access Key** from the [TestMu AI Dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard). -3. Install the **Java development environment** (Java 11 recommended). -4. Install **Maven**. Download it from [the official website](https://maven.apache.org/) or install it on **Linux/MacOS** using [**Homebrew**](https://brew.sh/). -5. Install the **Gauge** framework from [the official Gauge website](https://docs.gauge.org/getting_started/installing-gauge.html?os=windows&language=null&ide=null). - -## Step 1: Clone the Sample Project - -Pull the sample repo to your local machine and navigate into the project directory. - -```bash -git clone https://github.com/LambdaTest/gauge-selenium-sample -cd gauge-selenium-sample -``` - -You may also want to run the command below to check for outdated dependencies. - -```bash -mvn versions:display-dependency-updates -``` - -Install the mandatory Selenium dependencies for Maven: - -```bash -mvn compile -``` - -## Step 2: Set Your Credentials - -Add your TestMu AI credentials as environment variables so the test can authenticate with the grid. - -Visit the [TestMu AI Dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard), navigate to the left sidebar, and select **Credentials**. Copy your **Username** and **Access Key**, then set them as environment variables: - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -## Step 3: Configure Your Test Capabilities - -Define the browser, version, and OS for your test run. - -```java -ChromeOptions browserOptions = new ChromeOptions(); -browserOptions.setPlatformName("Windows 10"); -browserOptions.setBrowserVersion("latest"); - -HashMap ltOptions = new HashMap(); -ltOptions.put("build", "LambdaTestSampleApp"); -ltOptions.put("name", "LambdaTestJavaSample"); -ltOptions.put("network", true); // To enable network logs -ltOptions.put("visual", true); // To enable step by step screenshot -ltOptions.put("video", true); // To enable video recording -ltOptions.put("console", true); // To capture console logs -ltOptions.put("w3c", true); -browserOptions.setCapability("LT:Options", ltOptions); -``` - -Use the [Capabilities Generator](https://www.testmuai.com/capabilities-generator/) to auto-generate capabilities for any browser, version, and OS combination. - -Below is the sample Gauge test used in this project: - -```java title="StepImplementation_ToDo.java" -package driver.driver; - -import com.thoughtworks.gauge.Step; -import driver.driver.DriverFactory; -import org.openqa.selenium.By; -import org.openqa.selenium.WebDriver; -import org.openqa.selenium.WebElement; - -import java.util.concurrent.TimeUnit; - -import static org.junit.Assert.assertEquals; - -public class StepImplementation_ToDo { - -private final WebDriver driver; - -public StepImplementation_ToDo() { -this.driver = DriverFactory.getDriver(); -} - -@Step("Open the todo app") -public void gotoApp() throws InterruptedException { - -System.out.println(DriverFactory.getDriver()); -driver.get("https://lambdatest.github.io/sample-todo-app/"); -String title = driver.getTitle(); -assertEquals(title,"Sample page - lambdatest.com"); -} - -@Step("Select the desired items") -public void selectItems() throws InterruptedException { - -WebElement itemOne = driver.findElement(By.name("li1")); -itemOne.click(); - -driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS); - -WebElement itemThird = driver.findElement(By.name("li3")); -itemThird.click(); -} - -@Step("Add new item ") -public void addNewItem(String itemName) throws InterruptedException { - -WebElement addItem = driver.findElement(By.id("sampletodotext")); -addItem.sendKeys(itemName); -Thread.sleep(2000); - -WebElement addButton = driver.findElement(By.id("addbutton")); -addButton.click(); -} -} -``` - -## Step 4: Run the Test - -Trigger the test from your terminal. - -```bash -mvn test -``` - -or - -```bash -mvn clean install -``` - -Gauge framework supports parallel test execution on the TestMu AI grid. To run parallel tests, update more than one browser specification in the env folder of the project. Once the test executes, all parallel browser specifications hit the TestMu AI grid and run simultaneously. - -To remove a browser from parallel execution, delete its folder or comment out the code in it. - -## Step 5: View Your Results - -Check the Automation Dashboard to see exactly what happened during your test. - -Visit the [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) to see your test results. Each session includes: - -- **Video recording** of the full test execution -- **Screenshots** captured at each step -- **Console logs** from the browser -- **Network logs** for every request and response -- **Selenium command logs** showing each driver action - -## Run Gauge Tests Using Agent Skills - -Use AI coding assistants to generate and run Gauge tests with the TestMu AI Agent Skill. - -The [gauge-skill](https://github.com/LambdaTest/agent-skills/tree/main/gauge-skill) is part of [TestMu AI Agent Skills](https://github.com/LambdaTest/agent-skills/) - structured packages that teach AI coding assistants how to write production-grade test automation. - -Install the skill: - -```bash -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/gauge-skill .claude/skills/ - -# For Cursor / Copilot -cp -r agent-skills/gauge-skill .cursor/skills/ -``` - -Install all available framework skills at once by cloning the repository directly into your tool's skills directory (e.g., `.claude/skills/`, `.cursor/skills/`). diff --git a/static/docs/running-groovy-script-in-katalon-with-testmu-selenium-grid.md b/static/docs/running-groovy-script-in-katalon-with-testmu-selenium-grid.md deleted file mode 100644 index 81ec12f79..000000000 --- a/static/docs/running-groovy-script-in-katalon-with-testmu-selenium-grid.md +++ /dev/null @@ -1,108 +0,0 @@ -# Running Groovy Script In Katalon With TestMu AI Selenium Grid - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -As you perform testing with Katalon Studio, you can write or edit test automation scripts in the **Script mode** using Groovy programming language. This tutorial will help you run your Groovy script in Katalon with TestMu AI Selenium Grid. - -**Step 1:** Open your Katalon Studio instance & go to **Tests Explorer** from the left side-bar. - -**Step 2:** Right-click on the **Test Listener** folder & click on the new listener. - -**Step 3:** Name the new listener & paste the below code in the new listener. - -```javascript -import static com.kms.katalon.core.checkpoint.CheckpointFactory.findCheckpoint -import static com.kms.katalon.core.testcase.TestCaseFactory.findTestCase -import static com.kms.katalon.core.testdata.TestDataFactory.findTestData -import static com.kms.katalon.core.testobject.ObjectRepository.findTestObject - -import com.kms.katalon.core.checkpoint.Checkpoint as Checkpoint -import com.kms.katalon.core.model.FailureHandling as FailureHandling -import com.kms.katalon.core.testcase.TestCase as TestCase -import com.kms.katalon.core.testdata.TestData as TestData -import com.kms.katalon.core.testobject.TestObject as TestObject - -import com.kms.katalon.core.webservice.keyword.WSBuiltInKeywords as WS -import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI -import com.kms.katalon.core.mobile.keyword.MobileBuiltInKeywords as Mobile - -import com.kms.katalon.core.annotation.BeforeTestCase -import com.kms.katalon.core.annotation.BeforeTestSuite -import com.kms.katalon.core.annotation.TearDown -import com.kms.katalon.core.annotation.AfterTestCase -import com.kms.katalon.core.annotation.AfterTestSuite -import com.kms.katalon.core.context.TestCaseContext -import com.kms.katalon.core.context.TestSuiteContext - -import com.kms.katalon.core.webui.driver.ExistingRemoteWebDriver - -import com.kms.katalon.core.configuration.RunConfiguration - -class LambdaListener { - -TestSuiteContext suiteContext; - -/** -* Executes before every test case starts. -* @param testCaseContext related information of the executed test case. -*/ -@BeforeTestCase -def sampleBeforeTestCase(TestCaseContext testCaseContext) { - -//To set the test name at LambdaTest. -RunConfiguration.setDriverPreferencesProperty("Remote", "name", testCaseContext.getTestCaseId()); -if (suiteContext != null){ -RunConfiguration.setDriverPreferencesProperty("Remote","build", suiteContext.getTestSuiteId()); -} -println testCaseContext.getTestCaseId(); -println RunConfiguration.getDriverPreferencesProperties(); -} -/** -* Executes after every test case ends. -* @param testCaseContext related information of the executed test case. -*/ -@com.kms.katalon.core.annotation.TearDown -@AfterTestCase -def sampleAfterTestCase(TestCaseContext testCaseContext) { -//To set the status of test at LambdaTest. -String result="failed"; -if(testCaseContext.getTestCaseStatus().equalsIgnoreCase("PASSED")){ -result="passed" -} -try{ -WebUI.executeJavaScript("lambda-status="+result,null) -}catch (Exception e) -{ -println e.toString() -} -finally { -WebUI.closeBrowser() -} -println testCaseContext.getTestCaseId() -println testCaseContext.getTestCaseStatus() -} - -/** -* Executes before every test suite starts. -* @param testSuiteContext: related information of the executed test suite. -*/ -@BeforeTestSuite -def sampleBeforeTestSuite(TestSuiteContext testSuiteContext) { -suiteContext=testSuiteContext -//To Set the build Name at LambdaTest. -RunConfiguration.setDriverPreferencesProperty("Remote","build", suiteContext.getTestSuiteId()); -println testSuiteContext.getTestSuiteId() -} - -/** -* Executes after every test suite ends. -* @param testSuiteContext: related information of the executed test suite. -*/ -@AfterTestSuite -def sampleAfterTestSuite(TestSuiteContext testSuiteContext) { -println testSuiteContext.getTestSuiteId() -} -} -``` - -That is all. Now, you can effortlessly run Groovy Scripts in Katalon for triggering your tests over TestMu AI cloud-based cross browser testing platform. Happy testing! ?? diff --git a/static/docs/running-unit-testing-in-python-on-testmu-selenium-grid.md b/static/docs/running-unit-testing-in-python-on-testmu-selenium-grid.md deleted file mode 100644 index ecc3bc443..000000000 --- a/static/docs/running-unit-testing-in-python-on-testmu-selenium-grid.md +++ /dev/null @@ -1,109 +0,0 @@ -# Selenium With Python Unittest - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Run Python Unittest tests on the TestMu AI cloud grid. This guide covers setup, running a sample test, configuring capabilities, and testing locally hosted pages. - -**Sample repo** -All the code used in this guide is available in the sample repository. - - View on GitHub - -## Prerequisites - -Complete these steps before running your first Unittest Selenium test. - -1. Install the latest Python build from the [official website](https://www.python.org/downloads/). -2. Verify that **pip** is installed in your system. Install **pip** from [pip documentation](https://pip.pypa.io/en/stable/installation/). -3. Download the latest **Selenium Client** and its **WebDriver bindings** from the [official website](https://www.selenium.dev/downloads/). -4. Install **virtualenv** to isolate the build from other setups and ensure tests run with the specified module versions. -```bash -pip install virtualenv -``` - -## Step 1: Clone the Sample Project - -Clone the repository and set up the environment. - -```bash -git clone https://github.com/LambdaTest/Python-UnitTest-Selenium -cd Python-UnitTest-Selenium -``` - -Create a virtual environment and activate it: -```bash -virtualenv venv -source venv/bin/activate -``` - -Install the required packages: -```bash -pip install -r requirements.txt -``` - -## Step 2: Set Your Credentials - -Configure your credentials to connect to the TestMu AI Selenium Grid. - -Set TestMu AI `Username` and `Access Key` in environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -## Step 3: Configure Your Test Capabilities - -Define browser, version, and OS settings for your test run. - -In the test script, update your test capabilities. This code passes browser, browser version, and operating system information, along with TestMu AI Selenium grid capabilities via the capabilities object. - -```python -capabilities = { -"build": "UnitTest-Selenium-Sample", -"name": "UnitTest-Selenium-Test", -"platformName": "Windows 11", -"browserName": "Chrome", -"browserVersion": "latest", -} -``` - -**Capabilities Generator** -Use the TestMu AI [Capabilities Generator](https://www.testmuai.com/capabilities-generator/) to auto-generate the capabilities class for your test requirements. - -## Step 4: Run the Test - -Execute the Unittest Selenium test from the command line. - -```bash -python lambdatest_test.py -``` - -## Step 5: View Your Results - -Check the test output on the console and the TestMu AI dashboard. - -Visit the [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) to view your test results. The dashboard provides: - -- Text logs for each test step -- Screenshots captured during execution -- Video recordings of the full test session - -## Run Python Unittest Tests Using Agent Skills - -Use AI coding assistants to generate and run Python Unittest tests with the TestMu AI Agent Skill. - -The [unittest-skill](https://github.com/LambdaTest/agent-skills/tree/main/unittest-skill) is part of [TestMu AI Agent Skills](https://github.com/LambdaTest/agent-skills/) - structured packages that teach AI coding assistants how to write production-grade test automation. - -Install the skill: - -```bash -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/unittest-skill .claude/skills/ - -# For Cursor / Copilot -cp -r agent-skills/unittest-skill .cursor/skills/ -``` - -Install all available framework skills at once by cloning the repository directly into your tool's skills directory (e.g., `.claude/skills/`, `.cursor/skills/`). diff --git a/static/docs/running-your-first-cypress-test.md b/static/docs/running-your-first-cypress-test.md deleted file mode 100644 index 8073d85bb..000000000 --- a/static/docs/running-your-first-cypress-test.md +++ /dev/null @@ -1,133 +0,0 @@ -# Running Your First Cypress Test On TestMu AI Platform - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -To demonstrate Cypress testing on the TestMu AI platform, we will use the [Cypress Cloud repo](https://github.com/LambdaTest/Cypress-Cloud) in the demo below. - -**Step-1:** Clone the [TestMu AI-Cypress Cloud](https://github.com/LambdaTest/Cypress-Cloud) GitHub repo and switch to the cloned directory. - -```bash -git clone https://github.com/LambdaTest/Cypress-Cloud.git -cd Cypress-Cloud -``` - -**Step-2:** Setup the **TestMu AI-Cypress CLI** and configure the configuration file, as shown in the Prerequisites before. A file named `lambdatest-config.json` is generated in your project using the below command: - -```bash -lambdatest-cypress init -``` - -Here, we have used the below configuration as default and generated it in the `lambdatest-config.json` file. You need to set up the authentication by using TestMu AI credentials. You can check [here](/support/docs/authentication/) for more details about authentication. - -```json -{ -"lambdatest_auth": { -"username": "", -"access_key": "" -}, -"browsers": [ -{ -"browser": "Chrome", -"platform": "Windows 10", -"versions": ["86.0"] -}, -{ -"browser": "Firefox", -"platform": "Windows 10", -"versions": ["82.0"] -} -], -"run_settings": { -"cypress_config_file": "cypress.json", -"build_name": "build-name", -"parallels": 1, -"specs": "./*.spec.js", -"ignore_files": "", -"npm_dependencies": { -"cypress": "6.1.0" -}, -"feature_file_suppport": false -}, -"tunnel_settings": { -"tunnel": false, -"tunnelName": null -} -} -``` - -Also in `run-settings` section you need to specify the path of your `spec.js` file on which you want to run the test on. Here we will pass the path of a **sample to do** spec.js file for our demo. - -```json -"specs": "./cypress/integration/1-getting-started/todo.spec.js" -``` - -**Note** -In this demo, all occurrences of ``http://localhost:8080`` have been replaced with [https://example.cypress.io](https://example.cypress.io) to prevent running the Cypress tests locally. Alternatively, if you want to run your tests locally, refer to the [**Run locally**](/support/docs/running-your-first-cypress-test/#running-your-cypress-tests-locally-on-lambdatest-platform) section below. - -**Step-3:** Execute your tests using the following command in the terminal: - -```bash -lambdatest-cypress run -``` - -## View Your Cypress Testing Results On TestMu AI Platform - -As soon as the tests starts executing, you can view them running. Just visit your TestMu AI Automation Dashboard. - -**info** -For each test, you can view the live video feed, screenshots for each test run, console logs, terminal logs and do much more using the **TestMu AI platform**. - -If the test gets executed successfully, you will see a green tick on the Timeline view and a **"Completed"** message on the Automation logs view of your Automation dashboard. If not, then you will see a red cross and a **"Failed"** message respectively. - -## Running Your Cypress Tests Locally On TestMu AI Platform - -To run your tests locally on the TestMu AI platform, you need to setup [TestMu AI tunnel](/docs/testing-locally-hosted-pages/), and execute commands using the CLI, or [Download UnderPass](/docs/underpass-tunnel-application/), our GUI based desktop app. Once you have the TestMu AI tunnel or Underpass set up and started, you can use the TestMu AI platform to run your Cypress tests locally. - -Now you need to activate the tunnel capability in the `lambdatest_config.json` file under the section "connection_settings" as shown below: - -```json title="lambdatest_config.json" -"connection_settings": { -"tunnel": true, -"tunnel_name": "lt-cypress-tunnel" -}, -``` - -You can provide the name of the **TestMu AI tunnel** as per your requirements. - -## Using the Cypress Agent Skill with TestMu AI - -The [cypress-skill](https://github.com/LambdaTest/agent-skills/tree/main/cypress-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The cypress-skill package includes: - -``` -cypress-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing Cypress Agent Skill - -Install a Cypress Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/cypress-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/cypress-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only cypress-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). diff --git a/static/docs/saucelabs-to-hyperexecute-migrate.md b/static/docs/saucelabs-to-hyperexecute-migrate.md deleted file mode 100644 index 1dc7a2490..000000000 --- a/static/docs/saucelabs-to-hyperexecute-migrate.md +++ /dev/null @@ -1,205 +0,0 @@ -# Migrate from SauceLabs to HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Overview - -Sauce Labs and HyperExecute are both cloud-based testing platforms that provide a wide range of browsers and devices for running automated tests. However, HyperExecute provides you with a variety of [features](/support/docs/key-features-of-hyperexecute) to help you manage and analyze your test results more easily and efficiently. - -By making the switch to HyperExecute, you're not just migrating; you're upgrading your entire testing experience. This document ensures that you have all the information required for a successful migration. Whether you're driven by cost-efficiency, scalability, performance, or advanced features, HyperExecute is the perfect choice for your automated testing needs. - -## Why Migrate to HyperExecute? - -There are several benefits to migrating your tests from Sauce Labs to HyperExecute, including: - -- **High Speed, High-Quality Testing :** HyperExecute is renowned for its fast and reliable performance. Realize shorter test cycles and quicker feedback, enabling you to deliver high-quality applications at lightning speed. - -- **Test Orchestration :** HyperExecute is a cloud-based Test Orchestration platform that enables teams to run tests efficiently across multiple devices and browsers, featuring parallel execution, smart test distribution, automatic test reordering, retry logic, and real-time reporting and analytics. - -- **Cost-Efficiency :** HyperExecute doesn't just offer superior [features](/support/docs/key-features-of-hyperexecute); it's also cost-effective. By reducing testing time and enhancing productivity, you can optimize your testing budget, allocating resources to other critical areas of your project. - -- **Seamless Integration :** HyperExecute provides out-of-the-box support for popular tools such as [Tosca](/support/docs/tosca-integration-with-hyperexecute/), [Katalon](/support/docs/katalon-integration-with-hyperexecute/), and [Accelq](/support/docs/hyperexecute-accelq-integration/). This means you can effortlessly integrate your testing workflow with your preferred tools, streamlining your testing processes. - -- **CI/CD Tools :** We offer robust support for [CI/CD](/support/docs/hyperexecute-integration-with-ci-cd-tools/) tools, allowing you to automate testing and release processes seamlessly, resulting in faster software delivery. - -## Steps to Migrate - -To migrate your Test from SauceLabs to HyperExecute, you need to follow the below mentioned steps: - -> For demonstration purposes, we'll be using Selenium with the TestNG framework in our examples. - -### Step 1: Create your account in HyperExecute - -Before starting the migration process, make sure you have created your account in [HyperExecute](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/) and that you have also configured your HyperExecute [UserName and Access Key](/support/docs/hyperexecute-how-to-get-my-username-and-access-key/). - -### Step 2: Configure CLI and Environment Variables - -Since we will be performing the test from our local system, thus in order to initiate a test execution Job, you are required to download HyperExecute [CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) and set the environment variable. - -The *HyperExecute CLI* is used for triggering tests on HyperExecute. It is recommended to download the CLI binary on the host system to perform the tests on HyperExecute. The CLI download site for various platforms is displayed below: - -| Platform | HyperExecute CLI download location | -| ---------| --------------------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| macOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -Run the below mentioned commands in the terminal to setup the CLI and the environment variables. - -For macOS / Linux: - -```bash -export LT_USERNAME=YOUR_LT_USERNAME -export LT_ACCESS_KEY=YOUR_LT_ACCESS_KEY -``` - -For Windows: - -```bash -set LT_USERNAME=YOUR_LT_USERNAME -set LT_ACCESS_KEY=YOUR_LT_ACCESS_KEY -``` - -### Step 3: Configure and Update your Test Scripts - -- Add the [HyperExecute YAML](/support/docs/deep-dive-into-hyperexecute-yaml/) file in the root folder of your test code repository. - -**Sample YAML File** - -```yaml ---- -version: 0.2 -globalTimeout: 150 -testSuiteTimeout: 150 -testSuiteStep: 150 - -runson: win - -autosplit: true -retryOnFailure: true - -maxRetries: 1 -concurrency: 4 -shell: bash -env: -# PAT: ${{ .secrets.testKey }} -CACHE_DIR: m2_cache_dir - -cacheKey: '{{ checksum "pom.xml" }}' -cacheDirectories: -- .m2 - -pre: -# Skip execution of the tests in the pre step -- mvn -Dmaven.repo.local=./.m2 dependency:resolve - -post: -- ls target/surefire-reports/ - -mergeArtifacts: true - -uploadArtefacts: -- name: ExecutionSnapshots -path: -- target/surefire-reports/html/** - -report: true -partialReports: -location: target/surefire-reports/html -type: html -frameworkName: extent - -framework: -name: maven/testng -defaultReports: false -flags: -- "-Dplatname=mac" - -jobLabel: [selenium-testng, mac, autosplit, migration] - -``` - -- Replace the SauceLabs Hooks with [TestMu AI Hooks](/support/docs/lambda-hooks/) (if required). -- Update the SauceLabs Dependencies and Capabilities. - -**Sauce Labs** - -```java -MutableCapabilities sauceOptions = new MutableCapabilities(); -sauceOptions.setCapability("username", System.getenv("SAUCE_USERNAME")); -sauceOptions.setCapability("accessKey", System.getenv("SAUCE_ACCESS_KEY")); -sauceOptions.setCapability("name", method.getName()); -sauceOptions.setCapability("browserVersion", "latest"); -``` - -**HyperExecute** - -```java -DesiredCapabilities capabilities = new DesiredCapabilities(); -capabilities.setCapability("build", "[HyperExecute - 2] Demonstration of the TestNG Framework"); -capabilities.setCapability("name", "[HyperExecute - 2] Demonstration of the TestNG Framework"); - -capabilities.setCapability("platform", System.getenv("HYPEREXECUTE_PLATFORM")); -capabilities.setCapability("browserName", browser); -capabilities.setCapability("version", version); - -capabilities.setCapability("tunnel",false); -capabilities.setCapability("network",true); -capabilities.setCapability("console",true); -capabilities.setCapability("visual",true); -``` - -#### Update the **Hub URL** and **Authentication** in the Test Scripts - -For a Java-based implementation, here are the changes in the script as far as the authentication stage is concerned: - -**Sauce Labs** -``` js -// test.java -public static final String user_name = "SauceLabs_UserName"; -public static final String access_key = "SauceLabs_AccessKey"; -``` - -**HyperExecute** -``` js -// test.java -public static final String user_name = "LambdaTest_UserName"; -public static final String access_key = "LambdaTest_AccessKey"; -``` - -For a Java-based implementation, here are the changes in the script as far as the Hub URL is concerned: - -**Sauce Labs** -``` js -// test.java -public static final String Hub_URL = "https://" + user_name + ":" + access_key + "@ondemand.us-west-1.saucelabs.com:443/wd/hub"; -``` -**TestMu AI** -``` js -// test.java -public static final String Hub_URL = "https://" + user_name + ":" + access_key + "@hub.lambdatest.com/wd/hub"; -``` - -### Step 4: Execute your Job in HyperExecute - -Run the below command in your downloaded CLI to trigger your test scripts in the HyperExecute Platform. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE `} - -> In case of MAC, if you get a permission denied warning while executing CLI, simply run `chmod u+x ./hyperexecute` to allow permission. In case you get a security popup, allow it from your **System Preferences** → **Security & Privacy** → **General tab**. - -Here is the output in the [HyperExecute Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute/jobs). - -## Troubleshooting - -If you are having trouble migrating your tests from Sauce Labs to HyperExecute, here are some possible solutions: - -- #### You may not have updated your Test Scripts. - - To fix this, you will need to replace the Sauce Labs specific code with the equivalent HyperExecute code. You can use the [HyperExecute Capabilities Generator](https://www.testmuai.com/capabilities-generator/) to help you generate the correct desired capabilities for your tests. - -- #### Your Job is executed, but you are getting an error in the HyperExecute Dashboard. - - To fix this, you can check the [documentation](/support/docs/hyperexecute-how-to-debug-job/) explaining how you can debug a failed job on your own. - -> If you are still having trouble migrating your tests, you can contact **HyperExecute 24×7 Chat Support** or you can also drop a mail to **support@testmuai.com**. for assistance. diff --git a/static/docs/saucelabs-to-testmu-migration-guide.md b/static/docs/saucelabs-to-testmu-migration-guide.md deleted file mode 100644 index 6046bf982..000000000 --- a/static/docs/saucelabs-to-testmu-migration-guide.md +++ /dev/null @@ -1,254 +0,0 @@ -# Migrating From Sauce Labs To TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI and Sauce Labs both offer the cloud-based Selenium Grid. Hence, you can easily migrate your existing Selenium test automation scripts (or suites) from Sauce Labs to TestMu AI. - -[Note: This guide covers the migration of tests running on the cloud grid that uses Selenium 4 and Selenium 3] - -## Authentication -Firstly, you need to change the authentication in your configuration settings of your test suite. For running tests on TestMu AI Selenium Grid, you need to have a valid user_name and access_key to perform tests on our Grid. In case you do not have an account on TestMu AI, visit the TestMu AI signup page and create a new account. - -When migrating your Selenium 4 tests from BrowserStack to TestMu AI, the following updates are required in your existing code: - -1. Get TestMu AI Credentials: You can find these credentials under Account Settings > [Password & Security](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/security/username-accesskey) and copy your Username and Access Key, then add them to the .env file to keep them safe from public exposure. - -2. Create .env file: Securely store your TestMu AI credentials, create a .env file in the root of your project and add the following values: - -```js -public static final String user_name = "SauceLabs_UserName"; -public static final String access_key = "SauceLabs_AccessKey"; -``` - -``` -LT_USERNAME="" -LT_ACCESS_KEY="" -``` - -Once the .env file is set up, ensure your test framework correctly reads these variables at runtime. This helps keep your authentication secure and avoids hard-coding credentials within your scripts. With the credentials in place, you’re now ready to update your Hub URL for TestMu AI execution. - -## Changes in Hub URL -You need to now change the hub URL in the configuration settings of your test suite. Hub URL is of type String and it defines the Hub location to which the Selenium tests would be submitted for execution. - -```js -@ondemand.us-west-1.saucelabs.com:443/wd/hub -``` - -```js -@hub.lambdatest.com/wd/hub -``` - -## Configuring Selenium 4 Tests on TestMu AI -Migrating your existing tests running on Sauce Labs Cloud to TestMu AI Selenium cloud can be done with just a few lines of change. In this guide, we will see how you can use TestMu AI desired capabilities in your tests, authenticate your test session, and run tests on the browsers in our cloud. - -### TestMu AI Automation Capabilities - -Capabilities generator let you configure the desired capabilities (or capabilities) which are configuration options that let you set the following: -* Desired browser -* Desired browser version -* Desired platform (or operating system) - -Optionally, you can also choose the Selenium version and other advanced options present in the Selenium Capabilities Generator. For this migration guide, we have only restricted to the three capabilities listed above. - -To generate capabilities use [TestMu AI Capabilities Generator](https://www.testmuai.com/capabilities-generator/) to define key automation testing parameters, such as browser, version, operating system, and additional test settings. - -For the migration, we have considered Java-based Selenium automation tests. Shown below are the screenshots of capabilities generator of Sauce Labs and TestMu AI: - -```js -SafariOptions browserOptions = new SafariOptions(); -browserOptions.setPlatformName("macOS 15"); -browserOptions.setBrowserVersion("latest"); -Map sauceOptions = new HashMap<>(); -sauceOptions.put("username", "YOUR_USERNAME"); -sauceOptions.put("accessKey", "YOUR_ACCESS_KEY"); -sauceOptions.put("build", ""); -sauceOptions.put("name", ""); -sauceOptions.put("armRequired", true); -browserOptions.setCapability("sauce:options", sauceOptions); -``` - -```js -SafariOptions browserOptions = new SafariOptions(); -browserOptions.setPlatformName("MacOS Tahoe"); -browserOptions.setBrowserVersion("26"); -HashMap ltOptions = new HashMap(); -ltOptions.put("username", ""); -ltOptions.put("accessKey", ""); -ltOptions.put("w3c", true); -browserOptions.setCapability("LT:Options", ltOptions); -``` - -## Configuring Selenium 3 Tests on TestMu AI -Migrating your existing tests running on Sauce Labs Cloud to TestMu AI Selenium cloud can be done with just a few lines of change. In this guide, we will see how you can use TestMu AI desired capabilities in your tests, authenticate your test session, and run tests on the browsers in our cloud. - -### TestMu AI Automation Capabilities - -Capabilities generator let you configure the desired capabilities (or capabilities) which are configuration options that let you set the following: -* Desired browser -* Desired browser version -* Desired platform (or operating system) - -Optionally, you can also choose the Selenium version and other advanced options present in the Selenium Capabilities Generator. For this migration guide, we have only restricted to the three capabilities listed above. - -To generate capabilities use [TestMu AI Capabilities Generator](https://www.testmuai.com/capabilities-generator/) to define key automation testing parameters, such as browser, version, operating system, and additional test settings. - -For the migration, we have considered Java-based Selenium automation tests. Shown below are the screenshots of capabilities generator of Sauce Labs and TestMu AI: - -```js -SafariOptions browserOptions = new SafariOptions(); -browserOptions.setCapability("platformName", "macOS 15"); -browserOptions.setCapability("browserVersion", "latest"); -Map sauceOptions = new HashMap<>(); -sauceOptions.put("build", ""); -sauceOptions.put("name", ""); -sauceOptions.put("username", "YOUR_USERNAME"); -sauceOptions.put("accessKey", "YOUR_ACCESS_KEY"); -sauceOptions.put("armRequired", true); -browserOptions.setCapability("sauce:options", sauceOptions); -``` - -```js -DesiredCapabilities capabilities = new DesiredCapabilities(); -capabilities.setCapability("browserName", "Safari"); -capabilities.setCapability("browserVersion", "26"); -HashMap ltOptions = new HashMap(); -ltOptions.put("username", ""); -ltOptions.put("accessKey", ""); -ltOptions.put("platformName", "MacOS Tahoe"); -ltOptions.put("visual", true); -ltOptions.put("video", true); -capabilities.setCapability("LT:Options", ltOptions); -``` - -## Hands On Guide - TestMu AI Migration - -Let’s walk through a practical example demonstrating how to migrate a Selenium 4 test to TestMu AI. In this scenario, we launch the latest Chrome browser on a Windows 10 machine. - -**Test Scenario**: - -This test script performs a basic text validation on the website [TestMu AI eCommerce Playground](https://ecommerce-playground.lambdatest.io/) and shows the expected execution results when running the test in the TestMu AI cloud. - -```java -// TextValidationTest.java -import org.openqa.selenium.WebDriver; -import org.openqa.selenium.remote.RemoteWebDriver; -import org.openqa.selenium.JavascriptExecutor; -import org.openqa.selenium.safari.SafariOptions; - -import java.net.URL; -import java.util.HashMap; - -public class TextValidationTest { - -public static void main(String[] args) throws Exception { - -String username = System.getenv("LT_USERNAME") == null ? -"Your LT Username" : System.getenv("LT_USERNAME"); - -String authkey = System.getenv("LT_ACCESS_KEY") == null ? -"Your LT AccessKey\n" : System.getenv("LT_ACCESS_KEY"); - -String GRID_URL = "https://" + username + ":" + authkey + "@hub.lambdatest.com/wd/hub"; - -SafariOptions browserOptions = new SafariOptions(); -browserOptions.setPlatformName("MacOS Tahoe"); -browserOptions.setBrowserVersion("26"); -HashMap ltOptions = new HashMap(); -ltOptions.put("username", ""); -ltOptions.put("accessKey", ""); -ltOptions.put("project", "Text Validation Test"); -ltOptions.put("build", "Text Validation Test Build"); -ltOptions.put("w3c", true); -browserOptions.setCapability("LT:Options", ltOptions); - -WebDriver driver = new RemoteWebDriver(new URL(GRID_URL), browserOptions); - -try { - -driver.get("https://ecommerce-playground.lambdatest.io/"); - -String expectedText = "This is a dummy website for Web Automation Testing"; -boolean isTextPresent = driver.getPageSource().contains(expectedText); - -if (isTextPresent) { -((JavascriptExecutor) driver).executeScript("lambda-status=passed"); -System.out.println("✔ Text validation PASSED"); -} else { -((JavascriptExecutor) driver).executeScript("lambda-status=failed"); -System.out.println("✘ Text validation FAILED"); -} - -} catch (Exception e) { -((JavascriptExecutor) driver).executeScript("lambda-status=pass"); -e.printStackTrace(); -} finally { -driver.quit(); // 🔹 Correctly placed – runs even if test fails -} -} -} -``` - -```java -// TextValidationTest.java – Selenium 3 Configuration -import org.openqa.selenium.WebDriver; -import org.openqa.selenium.remote.DesiredCapabilities; -import org.openqa.selenium.remote.RemoteWebDriver; -import org.openqa.selenium.JavascriptExecutor; - -import java.net.URL; -import java.util.HashMap; - -public class TextValidationTest { - -public static void main(String[] args) throws Exception { - -String username = System.getenv("LT_USERNAME") == null ? -"Your LT Username" : System.getenv("LT_USERNAME"); - -String authkey = System.getenv("LT_ACCESS_KEY") == null ? -"Your LT AccessKey" : System.getenv("LT_ACCESS_KEY"); - -String GRID_URL = "https://" + username + ":" + authkey + "@hub.lambdatest.com/wd/hub"; - -DesiredCapabilities capabilities = new DesiredCapabilities(); -capabilities.setCapability("browserName", "Safari"); -capabilities.setCapability("browserVersion", "26"); -HashMap ltOptions = new HashMap(); -ltOptions.put("username", ""); -ltOptions.put("accessKey", ""); -ltOptions.put("platformName", "MacOS Tahoe"); -ltOptions.put("visual", true); -ltOptions.put("video", true); -capabilities.setCapability("LT:Options", ltOptions); - -WebDriver driver = new RemoteWebDriver(new URL(GRID_URL), capabilities); - -try { - -driver.get("https://ecommerce-playground.lambdatest.io/"); - -String expectedText = "This is a dummy website for Web Automation Testing"; -boolean isTextPresent = driver.getPageSource().contains(expectedText); - -if (isTextPresent) { -((JavascriptExecutor) driver).executeScript("lambda-status=passed"); -System.out.println("✔ Text validation PASSED"); -} else { -((JavascriptExecutor) driver).executeScript("lambda-status=failed"); -System.out.println("✘ Text validation FAILED"); -} - -} catch (Exception e) { -((JavascriptExecutor) driver).executeScript("lambda-status=pass"); -e.printStackTrace(); -} finally { -driver.quit(); // 🔹 Correctly placed – runs even if test fails -} -} -} - -``` - -**Result** - -Visit TestMu AI Web Automation dashboard to view your test execution result. diff --git a/static/docs/scim.md b/static/docs/scim.md deleted file mode 100644 index 4148c3688..000000000 --- a/static/docs/scim.md +++ /dev/null @@ -1,727 +0,0 @@ -# SCIM Provisioning: Users & Groups - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -SCIM (System for Cross-domain Identity Management) lets your Identity Provider (IDP) automatically manage users and groups within your TestMu AI organization, no manual account setup required. - -| Capability | What it does | -|---|---| -| **User Provisioning** | Auto-create, update, and deactivate user accounts | -| **Group Provisioning** | Sync IDP groups and map them to TestMu AI teams, concurrency groups, or sub-orgs | -| **Role Assignment** | Set organization roles (Admin / User / Guest) from your IDP | - -> **SSO is required.** If you haven't set up SSO yet, see [Getting Started with SSO](/support/docs/single-sign-on/) first. - -## Setup {#setup} - -**What you need:** Enterprise plan, SSO configured, Admin access, and an IDP that supports SCIM 2.0 (Okta, Azure AD, JumpCloud, etc.). - -### Step 1: Copy SCIM Credentials - -Go to **Settings** > **Organization Settings** > **Security** tab. Copy the **SCIM Base URL** and **Bearer Token**. - -### Step 2: Configure Your IDP - -Paste the SCIM Base URL and Bearer Token into your IDP's provisioning settings. - -Full walkthrough: [Okta SCIM Guide](/support/docs/scim/okta/) - -1. **Applications** > your TestMu AI app > **Provisioning** tab > **Configure API Integration** -2. Check **Enable API Integration**, paste credentials, click **Test API Credentials** > **Save** -3. Under **To App**, enable: Create Users, Update User Attributes, Deactivate Users -4. **Assignments** tab > assign users or groups -5. *(For groups)* **Push Groups** tab > **Push Groups** > Find by name or rule - -Member changes in pushed Okta groups are automatically synced to TestMu AI. - -Full walkthrough: [Azure AD SCIM Guide](/support/docs/scim/azure/) - -1. **Enterprise Applications** > your TestMu AI app > **Provisioning** > set to **Automatic** -2. Under **Admin Credentials**, paste SCIM Base URL (Tenant URL) and Bearer Token (Secret Token) > **Test Connection** > **Save** -3. Under **Mappings**, enable user and group provisioning -4. Under **Users and groups**, assign what you want to provision -5. Start a provisioning cycle (or wait for the 40-minute auto sync) - -Full walkthrough: [JumpCloud SCIM Guide](/support/docs/jumpcloud-scim/) - -1. **SSO Applications** > your TestMu AI app > **Identity Management** tab -2. Enable **SCIM Provisioning**, paste credentials -3. Configure attribute mappings (userName, name, active) -4. **User Groups** tab > select groups > **Activate** > **Save** - -Full walkthrough: [PingOne SCIM Guide](/support/docs/pingone-scim/) - -1. In PingOne, go to **Integrations** > **Provisioning** > create a new **SCIM Outbound** connection -2. Enter the **SCIM Base URL** and **Bearer Token** from TestMu AI -3. Configure preferences: set **User Identifier** to `workEmail` and enable **Create**, **Update**, **Disable** users -4. Create a provisioning **Rule** to select which users and groups to sync - -Any SCIM 2.0-compliant IDP works. Use these settings: - -| Setting | Value | -|---|---| -| **SCIM Base URL** | From Organization Settings > Security | -| **Authentication** | Bearer Token (HTTP Header) | -| **SCIM Version** | 2.0 | -| **Users Resource** | `/Users` | -| **Groups Resource** | `/Groups` | - -**Custom Attribute Schema URNs**: add these to your IDP's SCIM custom attribute configuration to send role and group assignments: - -| Purpose | Schema URN | -|---|---| -| **User extension** | `urn:ietf:params:scim:schemas:extension:LambdaTest:2.0:User` | -| **Group extension** | `urn:ietf:params:scim:schemas:extension:LambdaTest:2.0:Group` | - -- **User extension attributes:** `OrganizationRole` (Admin / User / Guest), `LambdatestGroup` (concurrency group name) -- **Group extension attributes:** `LambdatestRoles` (array of Admin / User / Guest, applied to all group members) - -## User Provisioning {#user-provisioning} - -### How It Works - -| Scenario | What happens | -|---|---| -| New user (email doesn't exist) | Account created and added to your org | -| Existing user (same org) | Attributes (role, active) updated | -| Existing user (different org) | **Not** provisioned, invite via team invite first | - -### Schema & Attributes - -```json -{ -"schemas": [ -"urn:ietf:params:scim:schemas:core:2.0:User", -"urn:ietf:params:scim:schemas:extension:LambdaTest:2.0:User" -], -"userName": "jane@company.com", -"active": true, -"name": { "givenName": "Jane", "familyName": "Doe", "formatted": "Jane Doe" }, -"urn:ietf:params:scim:schemas:extension:LambdaTest:2.0:User": { -"OrganizationRole": "User", -"LambdatestGroup": "Engineering" -} -} -``` - -**Standard Attributes** - -| Attribute | Required | Notes | -|---|---|---| -| `userName` | Yes | Must be a valid email. **Cannot be changed after creation.** | -| `active` | Yes | `true` = enabled, `false` = deactivated | -| `name` | Yes | `givenName`, `familyName`, `formatted` | - -**Custom Attributes** (LambdaTest Extension) - -These attributes are part of the `urn:ietf:params:scim:schemas:extension:LambdaTest:2.0:User` extension schema. To send them from your IDP, add this schema URN to your IDP's custom attribute configuration. - -| Attribute | Required | Values | Notes | -|---|---|---|---| -| `OrganizationRole` | No | `Admin`, `User`, `Guest` | Sets the user's [organization role](#roles). Defaults to `User` if not provided. | -| `LambdatestGroup` | No | _(concurrency group name)_ | Assigns the user to a concurrency group by name. The group must already exist. Contact support to enable concurrency groups. | - -For PATCH operations, use the fully qualified SCIM path: -- **OrganizationRole:** `urn:ietf:params:scim:schemas:extension:LambdaTest:2.0:User:OrganizationRole` -- **LambdatestGroup:** `urn:ietf:params:scim:schemas:extension:LambdaTest:2.0:User:LambdatestGroup` - -> **What can be updated:** `OrganizationRole`, `LambdatestGroup`, and `active` can be updated via SCIM. `userName` is immutable after creation. `name` can only be changed from TestMu AI Account Settings. - -### User API Operations - -**Request:** POST `https://auth.lambdatest.com/api/scim/Users` - -```json -{ -"schemas": [ -"urn:ietf:params:scim:schemas:core:2.0:User", -"urn:ietf:params:scim:schemas:extension:LambdaTest:2.0:User" -], -"userName": "jane@company.com", -"active": true, -"name": { "givenName": "Jane", "familyName": "Doe", "formatted": "Jane Doe" }, -"urn:ietf:params:scim:schemas:extension:LambdaTest:2.0:User": { -"OrganizationRole": "User", -"LambdatestGroup": "Engineering" -} -} -``` - -**Response:** `201 Created` - -```json -{ -"schemas": [ -"urn:ietf:params:scim:schemas:core:2.0:User", -"urn:ietf:params:scim:schemas:extension:LambdaTest:2.0:User" -], -"id": "23123", -"userName": "jane@company.com", -"active": true, -"name": { "givenName": "Jane", "familyName": "Doe", "formatted": "Jane Doe" }, -"urn:ietf:params:scim:schemas:extension:LambdaTest:2.0:User": { -"OrganizationRole": "User", -"LambdatestGroup": "Engineering" -} -} -``` - -**Request:** GET `https://auth.lambdatest.com/api/scim/Users/{id}` - -**Response:** `200 OK` - -```json -{ -"schemas": [ -"urn:ietf:params:scim:schemas:core:2.0:User", -"urn:ietf:params:scim:schemas:extension:LambdaTest:2.0:User" -], -"id": "23123", -"userName": "jane@company.com", -"active": true, -"name": { "givenName": "Jane", "familyName": "Doe", "formatted": "Jane Doe" }, -"urn:ietf:params:scim:schemas:extension:LambdaTest:2.0:User": { -"OrganizationRole": "User" -} -} -``` - -**Errors:** `404 Not Found` if user doesn't exist. `400 Bad Request` if user belongs to a different org. - -**Request:** GET `https://auth.lambdatest.com/api/scim/Users` - -Filter by email: `?filter=userName eq "jane@company.com"` - -**Response:** `200 OK` - -```json -{ -"schemas": ["urn:ietf:params:scim:api:messages:2.0:ListResponse"], -"totalResults": 1, -"startIndex": 1, -"itemsPerPage": 20, -"Resources": [ -{ -"schemas": [ -"urn:ietf:params:scim:schemas:core:2.0:User", -"urn:ietf:params:scim:schemas:extension:LambdaTest:2.0:User" -], -"id": "23123", -"userName": "jane@company.com", -"active": true, -"name": { "givenName": "Jane", "familyName": "Doe", "formatted": "Jane Doe" }, -"urn:ietf:params:scim:schemas:extension:LambdaTest:2.0:User": { -"OrganizationRole": "User" -} -} -] -} -``` - -**Zero results:** Returns `"totalResults": 0` with empty `"Resources": []`. - -**Errors:** `400 Bad Request` if user belongs to a different org. - -**Request:** PUT `https://auth.lambdatest.com/api/scim/Users/{id}` - -```json -{ -"schemas": ["urn:ietf:params:scim:schemas:core:2.0:User"], -"urn:ietf:params:scim:schemas:extension:LambdaTest:2.0:User": { -"OrganizationRole": "Admin" -}, -"active": true -} -``` - -**Response:** `200 OK`, returns the full updated user object. - -**Errors:** `404 Not Found` if user doesn't exist. `400 Bad Request` if user belongs to a different org. - -**Request:** PATCH `https://auth.lambdatest.com/api/scim/Users/{id}` - -```json -{ -"schemas": ["urn:ietf:params:scim:api:messages:2.0:PatchOp"], -"Operations": [ -{ "op": "Replace", "path": "active", "value": false }, -{ "op": "Replace", "path": "urn:ietf:params:scim:schemas:extension:LambdaTest:2.0:User:OrganizationRole", "value": "Guest" }, -{ "op": "Replace", "path": "urn:ietf:params:scim:schemas:extension:LambdaTest:2.0:User:LambdatestGroup", "value": "QA-Team" } -] -} -``` - -**Patchable paths:** - -| Path | Value | -|---|---| -| `active` | `true` / `false` | -| `urn:ietf:params:scim:schemas:extension:``LambdaTest:2.0:User:OrganizationRole` | `Admin`, `User`, `Guest` | -| `urn:ietf:params:scim:schemas:extension:``LambdaTest:2.0:User:LambdatestGroup` | _(concurrency group name)_ | - -**Response:** `200 OK`, returns the full updated user object. - -**Errors:** `404 Not Found` if user doesn't exist. `400 Bad Request` if user belongs to a different org. - -**Request:** PATCH `https://auth.lambdatest.com/api/scim/Users/{id}` - -```json -{ -"schemas": ["urn:ietf:params:scim:api:messages:2.0:PatchOp"], -"Operations": [ -{ "op": "Replace", "path": "active", "value": false } -] -} -``` - -**Response:** `200 OK`, returns user object with `"active": false`. - -**Errors:** `404 Not Found` if user doesn't exist. `400 Bad Request` if user belongs to a different org. - -**Request:** DELETE `https://auth.lambdatest.com/api/scim/Users/{id}` - -**Response:** `204 No Content` - -DELETE only **deactivates** the account. It does not permanently delete it. For permanent deletion, the user must request it from TestMu AI Account Settings. - -**Errors:** `404 Not Found` if user doesn't exist. `400 Bad Request` if user belongs to a different org. - -## Group Provisioning {#group-provisioning} - -**Quick Start** -Push groups from your IDP → approve the mapping in the **SCIM Group Provisioning** dashboard → members are synced automatically. - -### How It Works - -Groups and members are stored **as soon as your IDP pushes them**, even before any mapping is configured. Mapping only controls **where** members are assigned. - - Step 1: Your IDP (automatic) - Groups & members pushed via SCIM - - ↓ - - Step 2: LambdaTest (automatic) - Group stored & roles applied to members - Members get roles immediately, even without mapping - - ↓ - - Step 3: Admin (manual) or Mapping Rules (automatic) - Group mapped to a LambdaTest entity - - ↙ - ↓ - ↘ - - Team - Additive - Multiple teams allowed - - Concurrency Group - Exclusive - User can only belong to one - - Sub-Organization - Exclusive - User can only belong to one - -### Enabling & Disabling - -Group provisioning is not enabled by default. Reach out to our **24/7 chat support** or email [support@testmuai.com](mailto:support@testmuai.com) to get it activated for your organization. - -Once activated, you can control it from **Settings** > **Organization Settings** > **Security** > **SCIM Group Provisioning**. - -> **When toggled OFF:** New IDP group operations (create/update/delete) are rejected with `403`. Existing groups, mappings, and assignments are preserved. Nothing is deleted. Toggle back ON to resume syncing. - -### Group Schema & Attributes - -```json -{ -"schemas": [ -"urn:ietf:params:scim:schemas:core:2.0:Group", -"urn:ietf:params:scim:schemas:extension:LambdaTest:2.0:Group" -], -"displayName": "eng-backend", -"members": [{ "value": "12345" }, { "value": "67890" }], -"urn:ietf:params:scim:schemas:extension:LambdaTest:2.0:Group": { -"LambdatestRoles": ["User"] -} -} -``` - -**Standard Attributes** - -| Attribute | Required | Notes | -|---|---|---| -| `displayName` | Yes | Must be **unique** within your org | -| `members` | No | Array of `{ "value": "" }` | - -**Custom Attributes** (LambdaTest Extension) - -These attributes are part of the `urn:ietf:params:scim:schemas:extension:LambdaTest:2.0:Group` extension schema. To send them from your IDP, add this schema URN to your IDP's custom attribute configuration. - -| Attribute | Required | Values | Notes | -|---|---|---|---| -| `LambdatestRoles` | No | `Admin`, `User`, `Guest` | Assigns [organization roles](#roles) to **all members** of the group. Highest role wins if a user is in multiple groups. | - -For PATCH operations, use the fully qualified SCIM path: -- **LambdatestRoles:** `urn:ietf:params:scim:schemas:extension:LambdaTest:2.0:Group:LambdatestRoles` - -### Mapping Groups to LambdaTest Entities - -Once a group is pushed, it needs to be **mapped** to tell TestMu AI what to do with its members. Select your target entity type below to see the details relevant to you: - -**Feature Activation** -**Teams**, **Concurrency Groups**, and **Sub-Organizations** are not enabled by default. Reach out to our **24/7 chat support** or email [support@testmuai.com](mailto:support@testmuai.com) to get them activated for your organization before mapping groups to these entities. - -**Teams are additive**: a user can belong to multiple teams at once, so there are no conflicts. - -| | | -|---|---| -| **Auto-create** | Yes, if the team doesn't exist, it's created automatically | -| **On member removal** | Removed from team (unless another SCIM group also maps them there) | -| **On group rename** | Team is **automatically renamed** to match the IDP group name | -| **Conflicts** | None, teams are never exclusive | - -This is the simplest and most common mapping. If you're just starting out, **Team is the recommended choice**. - -**Concurrency groups are exclusive**: a user can only belong to one at a time. - -| | | -|---|---| -| **Auto-create** | Yes, if the group doesn't exist, it's created automatically | -| **On member removal** | User moved back to the org's **default concurrency group** | -| **On group rename** | Concurrency group is **automatically renamed** to match the IDP group name | -| **Conflicts** | Yes, if the same user is in two SCIM groups mapped to **different** concurrency groups. See [Conflicts](#conflicts). | - -**Sub-organizations are exclusive**: a user can only belong to one at a time. Sub-orgs also **conflict with concurrency groups and teams** from other SCIM groups (cross-type conflict). - -| | | -|---|---| -| **Auto-create** | No, sub-orgs must be created manually first (they have billing and setup requirements) | -| **On member removal** | User moved back to the **root organization** | -| **On group rename** | Sub-org is **not** renamed (sub-orgs have independent naming) | -| **Conflicts** | Yes, two types: (1) same user in two groups mapped to **different** sub-orgs, or (2) same user in a sub-org group **and** a concurrency group/team group from a different SCIM group | - -**Why cross-type conflicts?** Moving a user to a sub-org takes them out of the parent org's resource pool entirely. Team and concurrency group assignments in the parent org become invalid. - -A single SCIM group can only be mapped to **one** TestMu AI entity (one team, one concurrency group, or one sub-org). To assign the same users to multiple entities, use separate IDP groups. - -> **Mapping statuses:** `Pending` → `Approved` / `Auto-Approved` (members synced) or `Rejected` (no sync). If no mapping rule matches, the group stays **Pending** until an admin maps it manually. - -**To map manually:** Go to **SCIM Group Provisioning** dashboard > click a Pending group > select target type and entity > **Approve**. - -### Mapping Rules (Automatic Mapping) - -Instead of mapping each group manually, create rules that auto-match groups by name. - -Matches group names **starting with** a pattern (case-insensitive). - -`eng-` matches `eng-backend`, `eng-frontend`, `ENG-DevOps` - -Matches group names against a **regular expression**. - -`^qa-.*-team$` matches `qa-mobile-team`, `qa-web-team` - -Matches **every group**. Use as a low-priority catch-all fallback. - -**Each rule has an auto-approve toggle:** -- **ON** → finds (or creates) the target entity by name → mapping approved → members synced immediately. -- **OFF** → creates a Pending mapping → admin approves manually. - -**Rules are evaluated by priority** (highest first). First match wins. If **no rule matches**, the group stays Pending until an admin maps it manually. - -| Priority | Rule | Target | Auto-Approve | Example match | -|---|---|---|---|---| -| 3 | Prefix: `eng-` | Team | ON | `eng-backend` → auto-approved Team | -| 2 | Prefix: `qa-` | Team | ON | `qa-mobile` → auto-approved Team | -| 1 | Match All | Concurrency Group | OFF | `design-ops` → Pending | - -**To create a rule:** - -1. Go to **SCIM Group Provisioning** dashboard -2. Click **Add Mapping Rule** -3. Select a **rule type** (Prefix, Regex, or Match All) -4. Enter the **pattern** to match group names against -5. Choose the **target type** (Team, Concurrency Group, or Sub-Organization) -6. Toggle **auto-approve** ON if you want matched groups to be approved automatically -7. Click **Save** - -### Role Assignment {#roles} - -Roles can be set per-user (User extension `OrganizationRole`) or per-group (Group extension `LambdatestRoles`). Roles work **independently of mappings**, even unmapped groups apply their roles to members immediately. - -When a user is in **multiple groups with different roles**, the highest-priority role wins: **Admin > User > Guest** - -| Group | Role | -|---|---| -| `eng-team` | User | -| `org-admins` | Admin | -| **Effective role** | **Admin** (highest wins) | - -**Roles can be upgraded and downgraded.** The effective role is always the **highest** across all current group memberships. Removing a user from one group only downgrades their role if no other group provides it. - -| Scenario | What happens | -|---|---| -| User added to a group with `Admin` role | Role upgraded to Admin (if currently lower) | -| User removed from the `Admin` group | Role recomputed, drops to next highest (e.g., User) if no other group gives Admin | -| All groups removed, or no roles set | Role defaults to **User** | -| Group's `LambdatestRoles` changed from `Admin` to `Guest` | All members' roles recomputed, may downgrade | - -### One Group per Entity - -Each TestMu AI entity (team, concurrency group, or sub-org) can only be mapped from **one SCIM group at a time**. If you try to create a second mapping to the same entity, the request is rejected. This ensures a clear ownership model: one IDP group controls one TestMu AI entity. - -### Conflicts {#conflicts} - -**Teams don't have conflicts** -If you're only mapping to **Teams**, you can skip this section entirely. Teams are additive: no conflicts possible. - -Conflicts happen when a user belongs to multiple SCIM groups that compete for the **same exclusive slot**. When a conflict occurs, the user **keeps their current assignment** until an admin resolves it. Nothing changes automatically. - -**When do conflicts happen?** - -When the same user is in two SCIM groups mapped to **different** concurrency groups. Example: Group A → "QA Pool" and Group B → "Dev Pool". The user can only be in one. - -When the same user is in two SCIM groups mapped to **different** sub-orgs. Example: Group X → "US Team" and Group Y → "EU Team". The user can only be in one. - -When a user is mapped to a **sub-organization** from one SCIM group and a **concurrency group or team** from another SCIM group. Example: Group A maps to Sub-Org "US Team" and Group B maps to Concurrency Group "Dev Pool". - -**Why does this conflict?** Moving a user to a sub-org takes them out of the parent org's resource pool entirely. Team and concurrency group assignments in the parent org become invalid. - -**To resolve:** - -1. Go to **SCIM Group Provisioning** dashboard > **Conflicts** tab -2. Each conflict card shows the **Current** group (where the user is now) and the **Incoming** group (the one trying to claim the user) -3. Click **Keep Current** or **Use Incoming** -4. TestMu AI remembers this decision. The same combination won't create a new conflict - -**To avoid conflicts** -Prefer **teams** when users need to be in multiple groups. Teams never create conflicts. Only use concurrency groups and sub-orgs when you need exclusive assignment. - -### Deleted Target {#target-deleted} - -If an admin deletes a team, concurrency group, or sub-org that has an active SCIM mapping, the mapping is flagged as `target_deleted` and the status changes to **Pending**. - -**Manual re-mapping required** -When a target is deleted, the mapping **will not auto-create a replacement**, even if a matching mapping rule with auto-approve exists. This is intentional: auto-creating the same entity that was just deleted would cause a loop. An admin must manually update the mapping to point to a new (or recreated) target. - -**To fix a `target_deleted` mapping:** - -1. Go to **SCIM Group Provisioning** dashboard -2. Find the group with `target_deleted` status -3. Click the group and select a **new target entity** -4. Click **Approve**: members will be synced to the new target - -### Group API Operations - -**Request:** POST `https://auth.lambdatest.com/api/scim/Groups` - -```json -{ -"schemas": [ -"urn:ietf:params:scim:schemas:core:2.0:Group", -"urn:ietf:params:scim:schemas:extension:LambdaTest:2.0:Group" -], -"displayName": "eng-backend", -"members": [ -{ "value": "23123" }, -{ "value": "23456" } -], -"urn:ietf:params:scim:schemas:extension:LambdaTest:2.0:Group": { -"LambdatestRoles": ["User"] -} -} -``` - -**Response:** `201 Created` - -```json -{ -"schemas": [ -"urn:ietf:params:scim:schemas:core:2.0:Group", -"urn:ietf:params:scim:schemas:extension:LambdaTest:2.0:Group" -], -"id": "50001", -"displayName": "eng-backend", -"members": [ -{ "value": "23123", "display": "jane@company.com" }, -{ "value": "23456", "display": "bob@company.com" } -], -"urn:ietf:params:scim:schemas:extension:LambdaTest:2.0:Group": { -"LambdatestRoles": ["User"] -}, -"meta": { -"resourceType": "Group", -"created": "2025-01-15T10:30:00Z", -"lastModified": "2025-01-15T10:30:00Z" -} -} -``` - -**Request:** GET `https://auth.lambdatest.com/api/scim/Groups` - -Filter by name: `?filter=displayName eq "eng-backend"` | Paginate: `?startIndex=1&count=20` - -**Response:** `200 OK` - -```json -{ -"schemas": ["urn:ietf:params:scim:api:messages:2.0:ListResponse"], -"totalResults": 1, -"startIndex": 1, -"itemsPerPage": 20, -"Resources": [ -{ -"schemas": [ -"urn:ietf:params:scim:schemas:core:2.0:Group", -"urn:ietf:params:scim:schemas:extension:LambdaTest:2.0:Group" -], -"id": "50001", -"displayName": "eng-backend", -"members": [ -{ "value": "23123", "display": "jane@company.com" }, -{ "value": "23456", "display": "bob@company.com" } -], -"urn:ietf:params:scim:schemas:extension:LambdaTest:2.0:Group": { -"LambdatestRoles": ["User"] -}, -"meta": { -"resourceType": "Group", -"created": "2025-01-15T10:30:00Z", -"lastModified": "2025-01-15T10:30:00Z" -} -} -] -} -``` - -**Zero results:** Returns `"totalResults": 0` with empty `"Resources": []`. - -**PUT** (full replace) to `https://auth.lambdatest.com/api/scim/Groups/{id}`. Replaces entire membership list. Members removed from the list are unassigned from all mapped entities. - -**PATCH** (partial) to `https://auth.lambdatest.com/api/scim/Groups/{id}`: - -```json -{ -"schemas": ["urn:ietf:params:scim:api:messages:2.0:PatchOp"], -"Operations": [ -{ -"op": "Add", -"path": "members", -"value": [{ "value": "99999" }] -} -] -} -``` - -```json -{ -"schemas": ["urn:ietf:params:scim:api:messages:2.0:PatchOp"], -"Operations": [ -{ -"op": "Remove", -"path": "members", -"value": [{ "value": "12345" }] -} -] -} -``` - -```json -{ -"schemas": ["urn:ietf:params:scim:api:messages:2.0:PatchOp"], -"Operations": [ -{ -"op": "Replace", -"path": "displayName", -"value": "eng-platform" -} -] -} -``` - -```json -{ -"schemas": ["urn:ietf:params:scim:api:messages:2.0:PatchOp"], -"Operations": [ -{ -"op": "Replace", -"path": "urn:ietf:params:scim:schemas:extension:LambdaTest:2.0:Group:LambdatestRoles", -"value": ["Admin"] -} -] -} -``` - -**Response:** `200 OK`, returns the updated group object with new membership and `meta.lastModified` timestamp. - -**Errors:** `404 Not Found` if group doesn't exist. `409 Conflict` if renamed to an existing `displayName`. - -**Request:** DELETE `https://auth.lambdatest.com/api/scim/Groups/{id}` - -**Response:** `204 No Content` - -**On delete:** group is soft-deleted, roles recomputed, members safely unassigned from all mapped entities (checking other group memberships first), mappings rejected, related conflicts auto-resolved. - -**Errors:** `404 Not Found` if group doesn't exist. - -## What Happens When... {#sync-behavior} - -Quick reference for common scenarios. Everything below is handled automatically, no action needed unless noted. - -| You do this in your IDP | What happens in LambdaTest | Action needed? | -|---|---|---| -| **Rename a group** | Group name updated. Mapped team/concurrency group **renamed to match**. Mapping rules re-evaluated. | Only if mapping reverted to Pending | -| **Delete a group** | Soft-deleted. Members safely unassigned. Roles recomputed. Conflicts auto-resolved. | No | -| **Add a member to a group** | Added to all mapped entities. Role recomputed. | Only if a [conflict](#conflicts) is created | -| **Remove a member from a group** | Unassigned from mapped entities (only if no other group maps them there). Role recomputed. | No | -| **Re-push a previously deleted group** | Group restored. Members must be re-pushed. Mapping rules re-evaluated. | Depends on rules | -| **Change roles on a group** | All members' roles recomputed immediately. | No | - -| You do this in LambdaTest | What happens | Important | -|---|---|---| -| **Rename a team / group / sub-org** | Works fine, but the next IDP group rename will overwrite it. | To control names, rename **in your IDP** | -| **Delete a mapped entity** | Mapping flagged as `target_deleted`, reverts to Pending. Auto-create is blocked. | [Manually re-map](#target-deleted) to a new target | -| **Manually remove a member from a team** | Removal is immediate but **temporary**. Next IDP sync re-adds them. | Remove **in your IDP** instead | -| **Manually assign user to concurrency group / sub-org** | SCIM overrides non-SCIM assignments on next sync. | Use SCIM groups for exclusive assignments | -| **Manually change a SCIM-managed user's role** | May be overwritten on next IDP sync. | Manage roles **in your IDP** | - -## Troubleshooting {#troubleshooting} - -### API Errors - -| Error | Cause | Fix | -|---|---|---| -| `401 Unauthorized` | Invalid or expired Bearer Token | Regenerate in Organization Settings > Security | -| `403 Forbidden` | Group Provisioning is disabled | Enable the toggle in Security settings | -| `404 Not Found` | ID doesn't exist or belongs to a different org | Verify via a List call | -| `409 Conflict` | Group with same `displayName` already exists | Rename or delete the existing group | -| `400 Bad Request` | User exists in a different organization | Invite them via team invite first | - -### Common Issues - -| Issue | Solution | -|---|---| -| Members not appearing in teams or sub-orgs | Group mapping is still **Pending**. Approve it in the dashboard or create an auto-approve mapping rule. | -| Members are in the SCIM group but not in the sub-org | Check for [conflicts](#conflicts). The user may belong to another group with a competing exclusive mapping. | -| User has an unexpected role | Check **all** SCIM group memberships. Roles follow highest-wins (Admin > User > Guest). The user may inherit Admin from another group. | -| User keeps getting re-added after manual removal | SCIM is the source of truth. Remove the user **in your IDP** instead. | -| Group mapping reverted to Pending | The group was renamed (rules re-evaluated) or the target entity was deleted (`target_deleted`). If renamed, rules may auto-approve. If deleted, [manual re-mapping](#target-deleted) is required. | -| Auto-approve didn't create my sub-organization | Sub-orgs are never auto-created (billing/setup required). Create the sub-org first, then approve manually. | -| Can't map two SCIM groups to the same team | Each entity can only be mapped from one SCIM group. Use a single group, or merge in your IDP. | - -### FAQ - -| Question | Answer | -|---|---| -| Can a group be mapped to multiple targets? | No. A single SCIM group can only map to **one** entity (team, concurrency group, or sub-org). To assign the same users to multiple entities, use separate IDP groups. | -| Can two SCIM groups map to the same entity? | No. Each entity can only be owned by one SCIM group. This prevents conflicting membership lists. | -| Can I disable group provisioning without losing data? | Yes. The toggle only blocks new IDP operations. Existing groups, mappings, and assignments are preserved. Toggle back ON to resume. | -| Can I restore a deleted group? | Yes. Push a group with the same `displayName` from your IDP. The soft-deleted record is restored. Members need to be re-pushed. | -| Can roles be downgraded? | Yes. Roles are recomputed across all groups. If the highest role is removed, the effective role drops to the next highest. Defaults to **User** if none set. | -| What happens to a conflict when one group is deleted? | The conflict is **auto-resolved** in favor of the remaining group. No admin action needed. | -| What's the difference between Approved and Auto-Approved? | Both sync members identically. **Auto-Approved** = mapping rule matched automatically. **Approved** = admin approved manually. | -| Do mapping rules overwrite existing mappings? | No. Rules only apply when a group is first created or renamed. Existing mappings (manual or auto) are not overwritten. | - -> That's all you need to know about SCIM Provisioning with TestMu AI. In case you have any questions please feel free to reach out to us via the **24/7 chat support** or email us over [support@testmuai.com](mailto:support@testmuai.com). diff --git a/static/docs/screen-reader-on-accessibility.md b/static/docs/screen-reader-on-accessibility.md deleted file mode 100644 index d963fbe30..000000000 --- a/static/docs/screen-reader-on-accessibility.md +++ /dev/null @@ -1,38 +0,0 @@ -# Screen Reader - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -A Screen Reader is an assistive technology tool that converts digital content on screens into audio output or Braille for users with visual impairments. More than a simple text-to-speech tool, it acts as a comprehensive digital interpreter, transforming complex visual interfaces into navigable auditory or tactile experiences. It helps users with visual disabilities to interact with digital content with unprecedented independence and depth. - -## Why Do We Need Screen Readers? -Screen readers are essential for making digital environments accessible to visually impaired users. It helps to: - -- **Enhance Inclusivity :** Users with visual disabilities can access and interact with websites, applications, and digital content. -- **Support Compliance :** Ensure adherence to accessibility standards like WCAG (Web Content Accessibility Guidelines) and legal regulations such as ADA (Americans with Disabilities Act) or Section 508. -- **Improve Usability :** Highlight areas where the user experience can be improved for all users, not just those with visual impairments. -- **Boost Engagement :** Make websites and applications more usable for a diverse audience, improving user satisfaction and reach. - -## Screen Reader Support in TestMu AI Accessibility Testing -TestMu AI supports the following screen readers during Accessibility Testing: - -- **NVDA (NonVisual Desktop Access)** – Windows. See [NVDA on Windows](/support/docs/accessibility-nvda-windows/). -- **JAWS (Job Access With Speech)** – Windows. See [JAWS on Windows](/support/docs/accessibility-jaws-windows/). -- **VoiceOver** – macOS. See [VoiceOver on macOS](/support/docs/accessibility-voiceover-macos/). -- **TalkBack** – Android. See [TalkBack on Android](/support/docs/accessibility-talkback-android/). - -These tools allow developers and testers to evaluate how well their applications or websites perform for visually impaired users, ensuring accessibility compliance and usability. - -## Onboarding: pick a screen reader path - -| If you test on… | Start here | -|------------------|------------| -| Windows desktop web | [NVDA on Windows](/support/docs/accessibility-nvda-windows/) (free, common) or [JAWS on Windows](/support/docs/accessibility-jaws-windows/) (enterprise) | -| macOS desktop web | [VoiceOver on macOS](/support/docs/accessibility-voiceover-macos/) | -| Android app or web | [TalkBack on Android](/support/docs/accessibility-talkback-android/) (often paired with [App Scanner](/support/docs/accessibility-app-scanner/)) | - -**Suggested order of operations:** run an automated or DevTools scan first, export the top issues, then walk the same URLs with a screen reader to confirm real-world impact before filing engineering bugs. - -## Related docs - -- [Accessibility Testing (Overview)](/support/docs/accessibility-testing/) -- [Choosing the Right Accessibility Tool](/support/docs/accessibility-choosing-the-right-tool/) diff --git a/static/docs/screen-reader-on-real-devices-app.md b/static/docs/screen-reader-on-real-devices-app.md deleted file mode 100644 index fa636464c..000000000 --- a/static/docs/screen-reader-on-real-devices-app.md +++ /dev/null @@ -1,50 +0,0 @@ -# Screen Reader (TalkBack) on Android Devices for App Testing - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Manual accessibility testing is crucial to ensure that your mobile apps and websites are fully compliant with the Web Content Accessibility Guidelines (WCAG). This guarantees that your websites are usable by people with disabilities, regardless of their device or browsing capabilities. - -TestMu AI's Live Screen Reader feature empowers you to perform manual accessibility testing on real Android devices, allowing you to navigate through app elements with spoken descriptions of the UI, similar to the [Google TalkBack](https://support.google.com/accessibility/android/topic/3529932?hl=en&ref_topic=9078845) functionality. This feature enables comprehensive accessibility testing to ensure compliance with WCAG standards. - -## Accessing Screen Reader in Real Device App Testing - -**Step 1:** Navigate to **Real Devices** > **App Testing**. - -**Step 2:** Select the app you want to test and choose a device. Once selected, click on the **Start button** to launch the session. - -> Make sure the device supports accessibility features, look for the **Accessibility icon** next to the device name. - -**Step 3:** After the session starts, open the **Screen Reader** option from the sidebar and use the **toggle to enable or disable the TalkBack** functionality. - -## Supported Devices - - Screen feature is available on below android devices only to ensure a smooth rollout. We'll expand support to more devices based on feedback and performance insights only. - -| Device |Android Version | -|--------|--------------| -|Google Pixel 8 Pro|Android 14| -|Samsung Galaxy S23|Android 13| -|Google Pixel 7 Pro|Android 13| -|Samsung Galaxy S22 5G|Android 12| -|Google Pixel 6 Pro|Android 12| - -## How to Navigate using Keyboard Shortcuts - -Users accessing your mobile website on an Android device with TalkBack enabled can interact through touch gestures or by using keyboard shortcuts via a Bluetooth keyboard. Both options enable them to carry out "TalkBack actions," such as navigating between elements or having content read aloud. - -These two navigation methods function similarly, performing the same actions within the UI, ensuring that your site remains fully accessible, no matter the approach. Once your website meets WCAG standards using one method, it will be compatible with the other as well. - -For the **Screen Reader** feature on android devices , use the following keyboard shortcuts to trigger the different actions : - -| TalkBack Action | MacOS Keyboard Shortcut | Windows Keyboard Shortcut | -|-----------------|-------------------------|---------------------------| -| Move to previous/next item | Control + Option + (Left/Right arrow) | Ctrl + Alt + (Left/Right arrow) | -| Activate/open element | Control + Option + Space | Ctrl + Alt + Space| -| Go back | Esc|Esc | -| Pause or resume speech |Control + Option + Z |Ctrl + Alt + Z | -| Read from next item |Control + Option + P | Ctrl + Alt + P| -| Move up the slider | Control + Option + U|Ctrl + Alt + U| -| Move down the slider | Control + Option + D| Ctrl + Alt + D | -| Go to home screen | Control + Option + H|Ctrl + Alt + H | -| Go to recents | Control + Option + R| Ctrl + Alt + R| -| Go to quick settings | Control + Option + Q|Ctrl + Alt + Q | diff --git a/static/docs/screen-reader-on-real-devices-browser.md b/static/docs/screen-reader-on-real-devices-browser.md deleted file mode 100644 index 50ed398ce..000000000 --- a/static/docs/screen-reader-on-real-devices-browser.md +++ /dev/null @@ -1,50 +0,0 @@ -# Screen Reader (TalkBack) on Android Devices for Web Testing - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Manual accessibility testing is crucial to ensure that your mobile apps and websites are fully compliant with the Web Content Accessibility Guidelines (WCAG). This guarantees that your websites are usable by people with disabilities, regardless of their device or browsing capabilities. - -TestMu AI's Live Screen Reader feature empowers you to perform manual accessibility testing on real Android devices, allowing you to navigate through app elements with spoken descriptions of the UI, similar to the [Google TalkBack](https://support.google.com/accessibility/android/topic/3529932?hl=en&ref_topic=9078845) functionality. This feature enables comprehensive accessibility testing to ensure compliance with WCAG standards. - -> This feature is currently in **Beta** - -## Accessing Screen Reader in Real Device Browser Testing - -**Step 1:** Click on the **Real Devices** > **Browser Testing** - -**Step 2:** Select your required device and add your url to test and start session. - -**Step 3:** Click on the **screen reader** option from toolbar and use toggle to enable and disable talkBack feature on your device. - -## Supported Devices - - Screen feature is available on below android devices only to ensure a smooth rollout. We'll expand support to more devices based on feedback and performance insights only. - -| Device |Android Version | -|--------|--------------| -|Google Pixel 8 Pro|Android 14| -|Samsung Galaxy S23|Android 13| -|Google Pixel 7 Pro|Android 13| -|Samsung Galaxy S22 5G|Android 12| -|Google Pixel 6 Pro|Android 12| - -## How to Navigate using Keyboard Shortcuts - -Users accessing your mobile website on an Android device with TalkBack enabled can interact through touch gestures or by using keyboard shortcuts via a Bluetooth keyboard. Both options enable them to carry out "TalkBack actions," such as navigating between elements or having content read aloud. - -These two navigation methods function similarly, performing the same actions within the UI, ensuring that your site remains fully accessible, no matter the approach. Once your website meets WCAG standards using one method, it will be compatible with the other as well. - -For the **Screen Reader** feature on android devices , use the following keyboard shortcuts to trigger the different actions : - -| TalkBack Action | MacOS Keyboard Shortcut | Windows Keyboard Shortcut | -|-----------------|-------------------------|---------------------------| -| Move to previous/next item | Control + Option + (Left/Right arrow) | Ctrl + (Left/Right arrow) | -| Activate/open element | Control + Option + Space | Ctrl + Alt + Space| -| Go back | Esc|Esc | -| Pause or resume speech |Control + Option + Z |Ctrl + Alt + Z | -| Read from next item |Control + Option + P | Ctrl + Alt + P| -| Move up the slider | Control + Option + U|Ctrl + Alt + U| -| Move down the slider | Control + Option + D| Ctrl + Alt + D | -| Go to home screen | Control + Option + H|Ctrl + Alt + H | -| Go to recents | Control + Option + R| Ctrl + Alt + R| -| Go to quick settings | Control + Option + Q|Ctrl + Alt + Q | diff --git a/static/docs/screen-reader-voiceover-real-devices-app.md b/static/docs/screen-reader-voiceover-real-devices-app.md deleted file mode 100644 index b0c0f7961..000000000 --- a/static/docs/screen-reader-voiceover-real-devices-app.md +++ /dev/null @@ -1,75 +0,0 @@ -# Screen Reader (VoiceOver) on iOS Devices for App Testing - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Manual accessibility testing ensures that your app or website is fully accessible and compliant with Web Content Accessibility Guidelines (WCAG). With iOS VoiceOver feature on TestMu AI, you can test the accessibility of your website on supported real iOS devices. - -Ensuring accessibility with VoiceOver is essential for creating inclusive digital experiences that accommodate users with visual impairments. VoiceOver provides spoken feedback, allowing users to navigate, interact, and understand UI elements without relying on visual cues. - -This feature is currently in **Beta**. Please reach out to our [Support](https://www.testmuai.com/support/) team for access. - -## Accessing Screen Reader in Real Device App Testing - -VoiceOver-enabled devices are available in a separate device pool. To access them, you need to enable the VoiceOver toggle from Advanced Settings before selecting a device. - -**Step 1:** Click on **Real Devices** > **App Testing/Browser Testing**. - -**Step 2:** Click on **Advanced Settings** and navigate to the **Device Control** section. - -**Step 3:** Enable the **VoiceOver** toggle. This filters the device list to show only VoiceOver-enabled devices from the dedicated pool. - -**Step 4:** Select an iOS device from the filtered list and click **Start** to launch your session. Once the session starts, the VoiceOver feature will be active on the device. - -> Make sure your system's audio is turned on to hear the spoken descriptions provided by VoiceOver. - -## Supported Devices - -The VoiceOver feature is currently supported on selected iOS devices, allowing accurate accessibility testing. Below is the list of supported devices, with more being added over time to enhance accessibility coverage: - -| Device | iOS Version | -|--------|--------------| -| iPhone 15 | iOS 17 | -| iPhone 16 | iOS 18 | -| iPhone 17 | iOS 26 | - -## How to Navigate using Keyboard Shortcuts - -The navigation behavior on iOS, such as moving to the next or previous element, auto-reading from the start of the page, and interacting with UI elements, can be performed using keyboard shortcuts when **VoiceOver** is enabled. - -On a real iOS device, VoiceOver navigation can be executed via keyboard shortcuts and touch gestures as well . However, when performing accessibility testing, using keyboard shortcuts is the most efficient way to validate VoiceOver interactions. - -Ensuring that your website or application is accessible using **keyboard shortcuts** guarantees that it will also be accessible through touch gestures. If a site is fully navigable and interactable using VoiceOver keyboard commands, it will be equally accessible through swipe gestures and VoiceOver touch commands. - -To navigate and interact using VoiceOver on iOS in a Manual test session, use the following keyboard shortcuts: - -## VoiceOver Keyboard Shortcuts - -Below is a list of VoiceOver keyboard shortcuts, using `Control+Option` on macOS and `Ctrl+Alt` on Windows. These commands allow seamless navigation and interaction with UI elements. - -| Action | macOS (Control+Option) | Windows (Ctrl+Alt) | -|------------------------------------------------------|------------------------------------------------------|------------------------------------------------------| -| Turn on VoiceOver Help | Control+Option+K | Ctrl+Alt+K | -| Turn off VoiceOver Help | Option+Esc | Windows+Esc | -| Select the next or previous item | Control+Option+Right Arrow / Control+Option+Left Arrow | Ctrl+Alt+Right Arrow / Ctrl+Alt+Left Arrow | -| Activate the selected item | Control+Option+Space bar | Ctrl+Alt+Space bar | -| Touch and hold the selected item | Control+Option+Shift+M | Ctrl+Alt+Shift+M | -| Read from the current position | Control+Option+A | Ctrl+Alt+A | -| Read from the top | Control+Option+B | Ctrl+Alt+B | -| Pause reading | Control+Option+Hyphen | Ctrl+Alt+Hyphen | -| Copy the last spoken text to the clipboard | Control+Option+Shift+C | Ctrl+Alt+Shift+C | -| Search for text | Control+Option+F | Ctrl+Alt+F | -| Mute or unmute VoiceOver | Control+Option+S | Ctrl+Alt+S | -| Go to the Home Screen | Control+Option+H | Ctrl+Alt+H | -| Move to the status bar | Control+Option+M, then Option+Up Arrow | Ctrl+Alt+M, then Alt+Up Arrow | -| Open Control centre | (Control+Option+M), then Option+Down Arrow | (Ctrl+Alt+M), then Alt+Down Arrow | -| Open the App Switcher | Control+Option+H+H | Ctrl+Alt+H+H | -| Open the Item Chooser | Control+Option+I | Ctrl+Alt+I | -| Change the label of the selected item | Control+Option+Slash | Ctrl+Alt+Slash | -| Swipe up or down | Control+Option+Up Arrow / Control+Option+Down Arrow | Ctrl+Alt+Up Arrow / Ctrl+Alt+Down Arrow | -| Turn the rotor left or right | Control+Option+Command+Left Arrow / Right Arrow | Ctrl+Alt+Command+Left Arrow / Right Arrow | -| Adjust the setting specified by the rotor | Control+Option+Command+Up Arrow / Down Arrow | Ctrl+Alt+Command+Up Arrow / Down Arrow | -| Return to the previous screen | Option+Esc | Windows+Esc+EscEsc | - -By incorporating **VoiceOver** keyboard navigation into your accessibility testing workflow, you can identify usability barriers early, ensuring compliance with WCAG standards and delivering an **inclusive digital experience for all users**. - - If the keyboard shortcuts do not work initially, click on the device screen in your test session to ensure that it is in focus. diff --git a/static/docs/screen-reader-voiceover-real-devices-browser.md b/static/docs/screen-reader-voiceover-real-devices-browser.md deleted file mode 100644 index 9a2728bfe..000000000 --- a/static/docs/screen-reader-voiceover-real-devices-browser.md +++ /dev/null @@ -1,75 +0,0 @@ -# Screen Reader (VoiceOver) on iOS Devices for Web Testing - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Manual accessibility testing ensures that your website is fully accessible and compliant with Web Content Accessibility Guidelines (WCAG). With iOS VoiceOver feature on TestMu AI, you can test the accessibility of your website on supported real iOS devices. - -Ensuring accessibility with VoiceOver is essential for creating inclusive digital experiences that accommodate users with visual impairments. VoiceOver provides spoken feedback, allowing users to navigate, interact, and understand UI elements without relying on visual cues. - -This feature is currently in **Beta**. Please reach out to our [Support](https://www.testmuai.com/support/) team for access. - -## Accessing Screen Reader in Real Device Browser Testing - -VoiceOver-enabled devices are available in a separate device pool. To access them, you need to enable the VoiceOver toggle from Advanced Settings before selecting a device. - -**Step 1:** Click on **Real Devices** > **Browser Testing**. - -**Step 2:** Click on **Advanced Settings** and navigate to the **Device Control** section. - -**Step 3:** Enable the **VoiceOver** toggle. This filters the device list to show only VoiceOver-enabled devices from the dedicated pool. - -**Step 4:** Select an iOS device from the filtered list, enter your URL, and click **Start** to launch your session. Once the session starts, the VoiceOver feature will be active on the device. - -> Make sure your system's audio is turned on to hear the spoken descriptions provided by VoiceOver. - -## Supported Devices - -The VoiceOver feature is currently supported on selected iOS devices, allowing accurate accessibility testing. Below is the list of supported devices, with more being added over time to enhance accessibility coverage: - -| Device | iOS Version | -|--------|--------------| -| iPhone 15 | iOS 17 | -| iPhone 16 | iOS 18 | -| iPhone 17 | iOS 26 | - -## How to Navigate using Keyboard Shortcuts - -The navigation behavior on iOS, such as moving to the next or previous element, auto-reading from the start of the page, and interacting with UI elements, can be performed using keyboard shortcuts when **VoiceOver** is enabled. - -On a real iOS device, VoiceOver navigation can be executed via keyboard shortcuts and touch gestures as well. However, when performing accessibility testing, using keyboard shortcuts is the most efficient way to validate VoiceOver interactions. - -Ensuring that your website or application is accessible using **keyboard shortcuts** guarantees that it will also be accessible through touch gestures. If a site is fully navigable and interactable using VoiceOver keyboard commands, it will be equally accessible through swipe gestures and VoiceOver touch commands. - -To navigate and interact using VoiceOver on iOS in a Manual test session, use the following keyboard shortcuts: - -## VoiceOver Keyboard Shortcuts - -Below is a list of VoiceOver keyboard shortcuts, using `Control+Option` on macOS and `Ctrl+Alt` on Windows. These commands allow seamless navigation and interaction with UI elements. - -| Action | macOS (Control+Option) | Windows (Ctrl+Alt) | -|------------------------------------------------------|------------------------------------------------------|------------------------------------------------------| -| Turn on VoiceOver Help | Control+Option+K | Ctrl+Alt+K | -| Turn off VoiceOver Help | Option+Esc | Windows+Esc | -| Select the next or previous item | Control+Option+Right Arrow / Control+Option+Left Arrow | Ctrl+Alt+Right Arrow / Ctrl+Alt+Left Arrow | -| Activate the selected item | Control+Option+Space bar | Ctrl+Alt+Space bar | -| Touch and hold the selected item | Control+Option+Shift+M | Ctrl+Alt+Shift+M | -| Read from the current position | Control+Option+A | Ctrl+Alt+A | -| Read from the top | Control+Option+B | Ctrl+Alt+B | -| Pause reading | Control+Option+Hyphen | Ctrl+Alt+Hyphen | -| Copy the last spoken text to the clipboard | Control+Option+Shift+C | Ctrl+Alt+Shift+C | -| Search for text | Control+Option+F | Ctrl+Alt+F | -| Mute or unmute VoiceOver | Control+Option+S | Ctrl+Alt+S | -| Go to the Home Screen | Control+Option+H | Ctrl+Alt+H | -| Move to the status bar | Control+Option+M, then Option+Up Arrow | Ctrl+Alt+M, then Alt+Up Arrow | -| Open Control centre | (Control+Option+M), then Option+Down Arrow | (Ctrl+Alt+M), then Alt+Down Arrow | -| Open the App Switcher | Control+Option+H+H | Ctrl+Alt+H+H | -| Open the Item Chooser | Control+Option+I | Ctrl+Alt+I | -| Change the label of the selected item | Control+Option+Slash | Ctrl+Alt+Slash | -| Swipe up or down | Control+Option+Up Arrow / Control+Option+Down Arrow | Ctrl+Alt+Up Arrow / Ctrl+Alt+Down Arrow | -| Turn the rotor left or right | Control+Option+Command+Left Arrow / Right Arrow | Ctrl+Alt+Command+Left Arrow / Right Arrow | -| Adjust the setting specified by the rotor | Control+Option+Command+Up Arrow / Down Arrow | Ctrl+Alt+Command+Up Arrow / Down Arrow | -| Return to the previous screen | Option+Esc | Windows+Esc+EscEsc | - -By incorporating **VoiceOver** keyboard navigation into your accessibility testing workflow, you can identify usability barriers early, ensuring compliance with WCAG standards and delivering an **inclusive digital experience for all users**. - - If the keyboard shortcuts do not work initially, click on the device screen in your test session to ensure that it is in focus. diff --git a/static/docs/screen-resolution-testing-with-cypress.md b/static/docs/screen-resolution-testing-with-cypress.md deleted file mode 100644 index 8e95e4057..000000000 --- a/static/docs/screen-resolution-testing-with-cypress.md +++ /dev/null @@ -1,82 +0,0 @@ -# Cypress Testing On Different Screen Resolutions - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -As part of a responsive design process, it is important to ensure that websites or web applications under test run properly on different screen sizes. - -TestMu AI lets you test websites with Cypress across different screen resolutions on over 50+ browser versions on cloud. - -Cypress provides website testing on different resolutions using - -* Viewports -* Full Screen mode - -## Test on Viewports - -When you define a viewport, Cypress automatically uses the browser's scaling capabilities to customize the size of the web application. Therefore, on defining a large viewport, Cypress will scale down the web application. If you specify a small viewport, Cypress will upscale it. - -To set the viewport's width and height globally, you need to define `viewportWidth` and `viewportHeight` in the Cypress configuration file. - -**Syntax** - -```js -cy.viewport(550, 750) // Set viewport to 550px x 750px -cy.viewport('iphone-6') // Set viewport to 375px x 667px -``` -For more details, please check the [official Cypress documentation](https://docs.cypress.io/api/commands/viewport#Syntax). -## Test on Full Screen mode - -The use of full-screen mode in conjunction with viewports can help you achieve good results when performing screen resolution testing for web applications. - -To test your web applications in full screen mode, use the following event to change the browser options. - -```js -before:browser:launch -``` - -For **Cypress v9 and below**, use the following script in the `plugin/index.js` file. - -```js -module.exports = (on, config) => { -on('before:browser:launch', (browser = {}, launchOptions) => { -if (browser.family === 'chromium' && browser.name !== 'electron') { -launchOptions.args.push('--start-fullscreen') - -return launchOptions -} - -if (browser.name === 'electron') { -launchOptions.preferences.fullscreen = true - -return launchOptions -} -}) -} -``` - -For **Cypress v10 and above**, you can add the below code in `cypress.config.js` file. - -```js -const { defineConfig } = require('cypress') - -module.exports = defineConfig({ - -e2e: { -setupNodeEvents(on, config) { -on('before:browser:launch', (browser = {}, launchOptions) => { -if (browser.family === 'chromium' && browser.name !== 'electron') { -launchOptions.args.push('--start-fullscreen') - -return launchOptions -} - -if (browser.name === 'electron') { -launchOptions.preferences.fullscreen = true - -return launchOptions -} -}) -} -} -}) -``` diff --git a/static/docs/scroll-sync.md b/static/docs/scroll-sync.md deleted file mode 100644 index 7a9911125..000000000 --- a/static/docs/scroll-sync.md +++ /dev/null @@ -1,20 +0,0 @@ -# What Is Scroll Sync In LT Browser? - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -With [LT Browser](https://www.testmuai.com/lt-browser/), you can sync your scrolls from your device to web view and vice versa. It helps you to scroll through different devices so that scrolling on one device will scroll the same amount on another device. You can compare your website side-by-side over other devices via mirrored interactions. - -To get started, you need to download the executable file from here. - -Download LT Browser For Windows - -Download LT Browser For macOS - -Download LT Browser For Linux - -To enable the Scroll sync, navigate to the **'Settings'** and click on the **'Scroll Sync'**. - -With now Scroll sync enabled you can compare your web-pages side by side. - -> -That is all about the Scroll Sync in LT Browser. Don't forget to share you experience or questions with us. You can drop us an email through [support@testmuai.com](mailto:support@testmuai.com) or you could wave us a hello over our **24*7 Customer Chat Support**. Happy testing! diff --git a/static/docs/select-pre-installed-apps.md b/static/docs/select-pre-installed-apps.md deleted file mode 100644 index 2dfda6ae6..000000000 --- a/static/docs/select-pre-installed-apps.md +++ /dev/null @@ -1,21 +0,0 @@ -# Select Pre-Installed App in Session - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI empowers developers with a comprehensive testing environment, enabling thorough examination of mobile applications and websites across real devices and browsers. One standout feature is the ability to start testing sessions with pre-installed applications, enhancing efficiency, and reducing test start time, in **private cloud** setups where data retention and reusability are paramount. - -Let's delve into the streamlined steps to initiate a testing session with pre-installed applications: - -**Step 1.** Login to your TestMu AI account. Go to **App Testing** under **Real Device** from the left sidebar. - -**Step 2.** Enable the toggle to enter into your **Private Cloud**. - -**Step 3.** Select your preferred operating system version from the private device pool. - -**Step 4.** Within the **App Testing** section, locate and select the option to start a session with pre-installed applications. - -**Step 5.** Upon initiating the session, gain instant access to the pre-installed applications on the designated device. This seamless integration accelerates testing scenarios, significantly reducing start time. - -By adhering to the above steps, you can seamlessly commence testing sessions with pre-installed applications on TestMu AI. This feature not only expedites the testing process but also enhances efficiency, making it particularly advantageous for private cloud environments. With data retained within the private cloud devices, TestMu AI ensures enhanced flexibility and productivity in testing workflows. - -> **Note:** To access the pre-installed apps, you need to have a private cloud plan. diff --git a/static/docs/selenide-tests-with-testmu-online-selenium-grid-for-automated-cross-browser-testing.md b/static/docs/selenide-tests-with-testmu-online-selenium-grid-for-automated-cross-browser-testing.md deleted file mode 100644 index ee337cd5e..000000000 --- a/static/docs/selenide-tests-with-testmu-online-selenium-grid-for-automated-cross-browser-testing.md +++ /dev/null @@ -1,303 +0,0 @@ -# Selenium With Selenide - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Run Selenide tests on the TestMu AI cloud grid. This guide covers setup, running a sample test, configuring capabilities, and testing locally hosted pages. - -**Sample repo** -All the code used in this guide is available in the sample repository. - - View on GitHub - -## Prerequisites - -Make sure you have the following set up before you start. - -1. [Create a TestMu AI account](https://www.testmuai.com/register/) if you don't have one. -2. Get your **Username** and **Access Key** from the [TestMu AI Dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard). -3. Install the **Java development environment** (Java 11 recommended). -4. Install **Maven**. Download it from [the official website](https://maven.apache.org/) or install it on **Linux/MacOS** using [**Homebrew**](https://brew.sh/). - -## Step 1: Clone the Sample Project - -Pull the sample repo to your local machine and navigate into the project directory. - -```bash -git clone https://github.com/LambdaTest/selenide-testng-sample -cd selenide-testng-sample -``` - -You may also want to run the command below to check for outdated dependencies. - -```bash -mvn versions:display-dependency-updates -``` - -## Step 2: Set Your Credentials - -Add your TestMu AI credentials as environment variables so the test can authenticate with the grid. - -Visit the [TestMu AI Dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard), navigate to the left sidebar, and select **Credentials**. Copy your **Username** and **Access Key**, then set them as environment variables: - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -**Note** - -You can also mention your TestMu AI credentials in each of the config files located at `selenide-testng-sample/src/test/resources/conf/`. - -## Step 3: Configure Your Test Capabilities - -Define the browser, version, and OS for your test run. - -```java -ChromeOptions browserOptions = new ChromeOptions(); -browserOptions.setPlatformName("Windows 10"); -browserOptions.setBrowserVersion("latest"); - -HashMap ltOptions = new HashMap(); -ltOptions.put("build", "LambdaTestSampleApp"); -ltOptions.put("name", "LambdaTestJavaSample"); -ltOptions.put("network", true); // To enable network logs -ltOptions.put("visual", true); // To enable step by step screenshot -ltOptions.put("video", true); // To enable video recording -ltOptions.put("console", true); // To capture console logs -ltOptions.put("w3c", true); -browserOptions.setCapability("LT:Options", ltOptions); -``` - -Use the [Capabilities Generator](https://www.testmuai.com/capabilities-generator/) to auto-generate capabilities for any browser, version, and OS combination. - -Below is the sample Selenide test used in this project: - -```java title="Selenide To Do" -package com.lambdatest; - -import java.io.FileReader; -import java.net.URL; -import java.util.HashMap; -import java.util.Iterator; -import java.util.Map; -import java.util.concurrent.TimeUnit; - -import org.json.simple.JSONObject; -import org.json.simple.parser.JSONParser; -import org.openqa.selenium.chrome.ChromeOptions; -import org.openqa.selenium.remote.RemoteWebDriver; -import org.testng.annotations.AfterMethod; -import org.testng.annotations.BeforeMethod; -import org.testng.annotations.Parameters; -import static com.codeborne.selenide.Selenide.$; -import static com.codeborne.selenide.Selenide.open; -import static com.codeborne.selenide.Selenide.sleep; -import static com.codeborne.selenide.Selenide.title; - -import org.openqa.selenium.By; -import org.testng.Assert; -import org.testng.annotations.Test; - -import com.codeborne.selenide.WebDriverRunner; - -public class LambdaTestSetup { -public RemoteWebDriver driver; -public String status="failed"; - -public static String username; -public static String accessKey; -public static String sessionId; - -@BeforeMethod(alwaysRun = true) -@Parameters(value = { "config", "environment" }) -public void setUp(String config_file, String environment) throws Exception { -JSONParser parser = new JSONParser(); -JSONObject config = (JSONObject) parser.parse(new FileReader("src/test/resources/conf/" + config_file)); -JSONObject envs = (JSONObject) config.get("environments"); - -ChromeOptions browserOptions = new ChromeOptions(); -HashMap ltOptions = new HashMap(); - -Map envCapabilities = (Map) envs.get(environment); -Iterator it = envCapabilities.entrySet().iterator(); -while (it.hasNext()) { -Map.Entry pair = (Map.Entry) it.next(); -String key = pair.getKey().toString(); -if (key.equals("platformName")) { -browserOptions.setPlatformName(pair.getValue().toString()); -} else if (key.equals("browserVersion")) { -browserOptions.setBrowserVersion(pair.getValue().toString()); -} else { -ltOptions.put(key, pair.getValue().toString()); -} -} - -Map commonCapabilities = (Map) config.get("capabilities"); -it = commonCapabilities.entrySet().iterator(); -while (it.hasNext()) { -Map.Entry pair = (Map.Entry) it.next(); -if (!ltOptions.containsKey(pair.getKey().toString())) { -ltOptions.put(pair.getKey().toString(), -(pair.getValue().toString().equalsIgnoreCase("true") -|| (pair.getValue().toString().equalsIgnoreCase("false")) -? Boolean.parseBoolean(pair.getValue().toString()) -: pair.getValue().toString())); -} -} -ltOptions.put("name", this.getClass().getName()); -ltOptions.put("w3c", true); -browserOptions.setCapability("LT:Options", ltOptions); - -username = System.getenv("LT_USERNAME"); -if (username == null) { -username = (String) config.get("user"); -} - -accessKey = System.getenv("LT_ACCESS_KEY"); -if (accessKey == null) { -accessKey = (String) config.get("key"); -} - -driver = new RemoteWebDriver( -new URL("https://" + username + ":" + accessKey + "@" + config.get("server") + "/wd/hub"), browserOptions); -driver.manage().timeouts().implicitlyWait(5, TimeUnit.SECONDS); -sessionId = driver.getSessionId().toString(); - -WebDriverRunner.setWebDriver(driver); -} - -@Test -public void test() throws Exception { - -open("http://www.google.co.uk"); - -$(By.name("q")).setValue("LambdaTest").pressEnter(); - -sleep(2000); - -Assert.assertEquals(title(), "LambdaTest - Google Search"); - -status = "passed"; -} - -@AfterMethod(alwaysRun = true) -public void tearDown() throws Exception { -driver.executeScript("lambda-status="+status); -driver.quit(); -} - -} -``` - -## Step 4: Run the Test - -Trigger the test from your terminal. - -First, install the mandatory Selenium dependencies for Maven: - -```bash -mvn compile -``` - -Then execute the tests: - -```bash -mvn test -P single -``` - -```bash -mvn test -P parallel -``` - -For parallel execution, the sample project includes a `parallel.testng.xml` that runs the test across multiple browsers simultaneously: - -```xml title="parallel.testng.xml" - - - - - - - - - - - - - - - - - - - - - - - - - - - -``` - -```json title="parallel.config.json" -{ -"server": "hub.lambdatest.com", -"user": "YOUR_USERNAME", -"key": "YOUR_ACCESS_KEY", - -"capabilities": { -"build": "Java Selenide Parallel" -}, - -"environments": { -"chrome": { -"platformName": "Windows 10", -"browserName": "chrome", -"browserVersion": "latest" -}, -"firefox": { -"platformName": "Windows 10", -"browserName": "firefox", -"browserVersion": "latest" -}, -"safari": { -"platformName": "macOS Mojave", -"browserName": "safari", -"browserVersion": "latest" -} -} -} -``` - -## Step 5: View Your Results - -Check the Automation Dashboard to see exactly what happened during your test. - -Visit the [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) to see your test results. Each session includes: - -- **Video recording** of the full test execution -- **Screenshots** captured at each step -- **Console logs** from the browser -- **Network logs** for every request and response -- **Selenium command logs** showing each driver action - -## Run Selenide Tests Using Agent Skills - -Use AI coding assistants to generate and run Selenide tests with the TestMu AI Agent Skill. - -The [selenide-skill](https://github.com/LambdaTest/agent-skills/tree/main/selenide-skill) is part of [TestMu AI Agent Skills](https://github.com/LambdaTest/agent-skills/) - structured packages that teach AI coding assistants how to write production-grade test automation. - -Install the skill: - -```bash -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/selenide-skill .claude/skills/ - -# For Cursor / Copilot -cp -r agent-skills/selenide-skill .cursor/skills/ -``` - -Install all available framework skills at once by cloning the repository directly into your tool's skills directory (e.g., `.claude/skills/`, `.cursor/skills/`). diff --git a/static/docs/selenified-integration-with-testmu.md b/static/docs/selenified-integration-with-testmu.md deleted file mode 100644 index c0d4ce48a..000000000 --- a/static/docs/selenified-integration-with-testmu.md +++ /dev/null @@ -1,59 +0,0 @@ -# Selenified Integration With TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Previously known as SecureCI™ Testing Framework, Selenified is an open-source test automation framework as a wrapper to Selenium with extra features such as detailed **traceable reporting**, **custom reporting**, ability to invoke **dynamic waits**, etc. so you could test your web-application across the web and even API level. The framework is built on top of TestNG, making it intuitive for beginners and offers support with multiple build tools such as Maven, Gradle, and Ant, making it easier to integrate with your DevOps build environments. - -With TestMu AI integration to Selenified, you can perform cross browser testing in parallel on more than 2000 real browsers available at an online Selenium Grid offered by TestMu AI. So you don't have to worry about configuring an in-house testing infrastructure as TestMu AI will take care of that for you. - -## Prerequisites - -* Basic understanding of running tests from Selenified. If you haven't run a Selenified test before, you can refer to their [official GitHub repository][1]. -* TestMu AI Hub URL -* **TestMu AI Authentication Credentials** -You will need your TestMu AI authentication credentials i.e., your TestMu AI username, access key, and HubURL. You need to set them up as your environment variables. You can retrieve them from your **[TestMu AI automation dashboard][2]** by clicking on the key icon near the help button. - - * For Linux/Mac: - -``` -$ export LT_USERNAME= {YOUR_LAMBDATEST_USERNAME} -$ export LT_ACCESS_KEY= {YOUR_LAMBDATEST_ACCESS_KEY} -``` - - * For Windows: -``` -$ set LT_USERNAME= {YOUR_LAMBDATEST_USERNAME} -$ set LT_ACCESS_KEY= {YOUR_LAMBDATEST_ACCESS_KEY} -``` -## Running Selenified Tests On TestMu AI Cloud Grid - -You can run your Selenified test cases on a cloud-based Selenium Grid, such as TestMu AI, by declaring any of the two aspects into your test automation script. - -1. Feeding a remote Hub URL. -2. Configuring authentication credentials as the environment variables. - -### Hub URL - -By default, your Selenified tests would be executed over a standalone server. To run your tests using a remote server for TestMu AI Selenium Grid, you will need to provide your TestMu AI HubURL. - -``` --Dhub=https://LT_USERNAME:LT_ACCESS_KEY@hub.lambdatest.com -``` - ->**Note:** If required, you can also declare extra parameters such as a particular port or IP. - -### Authentication Credentials - -You can also run your Selenified test automation scripts on TestMu AI Selenium Grid by simply declaring your TestMu AI authentication credentials as the environment variables in your Selenified test automation script. - ->**Note:** TestMu AI clubs your test cases under a group and assigns them under a build. If you wish to override the build name, then you can do so by invoking the `buildName` parameter in your Selenified script. - -``` --Dhub=https://LT_USERNAME:LT_ACCESS_KEY@hub.lambdatest.com -``` - -That is all you need to tweak to run your Selenified tests on TestMu AI Selenium Grid. In case you have any questions, feel free to share them with us in the comments or wave them to our **24/7 in-app customer chat support**. You could also email us through [support@testmuai.com][3]. Happy testing! 🙂 - -[1]: https://github.com/Coveros/selenified -[2]: https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com -[3]: mailto:support%40lambdatest.com diff --git a/static/docs/selenium-add-test-meta-data.md b/static/docs/selenium-add-test-meta-data.md deleted file mode 100644 index 7fef34fc9..000000000 --- a/static/docs/selenium-add-test-meta-data.md +++ /dev/null @@ -1,196 +0,0 @@ -# Add Test Metadata Using the customData Capability - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The `customData` capability allows you to associate additional metadata with test runs, enabling better traceability, debugging, and reporting. This metadata can include information like issue tracker links, test case IDs, and other critical test context. By embedding this metadata in test configurations, your team can integrate with existing tools and workflows, such as GitHub, Jira, or any test management system. - -## How to Add Custom Metadata for Running Automation Tests on TestMu AI - -Add the `customData` capability to your test script with the metadata fields you need. - -To add custom metadata in your automation tests, add the capability `customData` in your test script with all the metadata information that we support: - -```java title="Test.java" -ltOptions.put("customData", new HashMap() {{ -put("jiraTicket", "JIRA-12345"); -put("githubPR", "https://github.com/organization/repo/pull/678"); -put("testDescription", "This test validates login functionality under high load."); -}}); -``` - -```javascript title="Test.js" -"LT:Options": { -"customData": { -"jiraTicket": "JIRA-12345", -"githubPR": "https://github.com/organization/repo/pull/678", -"testDescription": "This test validates login functionality under high load." -} -} -``` - -```typescript title="Test.ts" -"LT:Options": { -customData: { -jiraTicket: "JIRA-12345", -githubPR: "https://github.com/organization/repo/pull/678", -testDescription: "This test validates login functionality under high load." -} -} -``` - -```php title="Test.php" -"LT:Options" => [ -"customData" => [ -"jiraTicket" => "JIRA-12345", -"githubPR" => "https://github.com/organization/repo/pull/678", -"testDescription" => "This test validates login functionality under high load." -] -] -``` - -```python title="Test.py" -"LT:Options": { -"customData": { -"jiraTicket": "JIRA-12345", -"githubPR": "https://github.com/organization/repo/pull/678", -"testDescription": "This test validates login functionality under high load." -} -} -``` - -```csharp title="Test.cs" -ltOptions.Add("customData", new Dictionary{ -{ "jiraTicket", "JIRA-12345" }, -{ "githubPR", "https://github.com/organization/repo/pull/678" }, -{ "testDescription", "This test validates login functionality under high load." } -}); -``` - -```ruby title="Test.rb" -"LT:Options" => { -customData: { -jiraTicket: "JIRA-12345", -githubPR: "https://github.com/organization/repo/pull/678", -testDescription: "This test validates login functionality under high load." -} -} -``` - -## Use Cases for `customData` - -See how teams use the customData capability in different testing workflows. - -### 1. Enhanced Reporting With GitHub and Jira Links -**Scenario:** A QA team wants to include direct links to GitHub pull requests or Jira issues related to a test. This helps developers and testers quickly access related code changes or tasks when a test fails. - -```javascript title="Test.js" -'customData': { -"jiraTicket": "JIRA-12345", -"githubPR": "https://github.com/organization/repo/pull/678", -"testDescription": "This test validates login functionality under high load." -} -``` - -- **`jiraTicket`:** Links the test to the corresponding Jira issue for easy navigation. -- **`githubPR`:** Links to the pull request that introduced the changes being tested. -- **`testDescription`:** Provides a brief description of the test's purpose. - -### 2. Linking Test Management Systems -**Scenario:** The team uses a test management tool (e.g., TestRail, Zephyr) to manage test cases. Adding the test case ID ensures results link back to the test plan. - -```javascript title="Test.js" -'customData': { -"testCaseID": "TC-56789", -"testSuite": "Regression Suite", -"priority": "High", -"owner": "qa_team@company.com" -} -``` - -- **`testCaseID`:** Maps the execution to a specific test case in the test management system. -- **`testSuite`:** Specifies the test suite or category the test belongs to. -- **`priority`:** Indicates the importance or severity of the test. -- **`owner`:** Identifies the owner or responsible party for the test. - -### 3. Debugging With Environment Metadata -**Scenario:** When debugging test failures, include information about the environment or build being tested. - -```javascript title="Test.js" -'customData': { -"buildNumber": "1234", -"environment": "Staging", -"apiVersion": "v1.2.3", -"releaseTag": "v1.2.3-rc1" -} -``` - -- **`buildNumber`:** Identifies the specific build of the application being tested. -- **`environment`:** Indicates the environment (e.g., Development, Staging, Production) the test ran in. -- **`apiVersion`:** Provides the API version being tested. -- **`releaseTag`:** Links the test to a specific release or tag in the version control system. - -### 4. Capturing User Story or Feature Metadata -**Scenario:** A product manager wants test results linked to specific user stories or features for tracking progress on new functionality. - -```javascript title="Test.js" -'customData': { -"featureID": "FEAT-9876", -"userStory": "As a user, I want to reset my password securely.", -"sprint": "Sprint 45" -} -``` - -- **`featureID`:** Links the test to a specific feature ID in the product backlog. -- **`userStory`:** Describes the user story being validated. -- **`sprint`:** Indicates the sprint or iteration in which the feature is being developed. - -### 5. Tracking Third-Party Dependencies -**Scenario:** A test depends on third-party APIs or integrations, and tracking the versions or configurations of these dependencies is critical. - -```javascript title="Test.js" -'customData': { -"thirdPartyAPI": "Stripe", -"apiVersion": "2023-01-15", -"status": "Active" -} -``` - -- **`thirdPartyAPI`:** Identifies the external service used. -- **`apiVersion`:** Specifies the version of the API. -- **`status`:** Indicates the status or availability of the dependency. - -### 6. Integrating Test Runs With CI/CD Pipelines -**Scenario:** A DevOps team wants to include pipeline-specific metadata in the test report to track CI/CD execution details. - -```javascript title="Test.js" -'customData': { -"pipelineID": "Pipeline-001", -"jobID": "Job-456", -"triggeredBy": "GitHub Actions", -"commitHash": "a1b2c3d4e5f67890" -} -``` - -- **`pipelineID`:** Tracks the pipeline in which the test ran. -- **`jobID`:** Identifies the specific CI/CD job. -- **`triggeredBy`:** Indicates the trigger source (e.g., manual, GitHub Actions, Jenkins). -- **`commitHash`:** Links the test to a specific commit in the version control system. - -## Limitations - -Review these constraints before using the customData capability. - -- **Payload Size:** The `customData` capability is limited to 1 KB of JSON data. Larger payloads are not accepted. - - Use concise key names and avoid unnecessary fields. - - Prioritize critical metadata to stay within the limit. - -- **Readability:** Adding too many fields may reduce the readability of the metadata. Be selective in the information you include. - -## Best Practices - -Follow these guidelines to get the most out of your custom metadata. - -- **Keep Metadata Concise:** Use meaningful but short key names and values. -- **Align With Workflows:** Structure customData to integrate with tools like GitHub, Jira, and test management systems. -- **Validate Data Size:** Include a validation step in your scripts to ensure the payload is under 1 KB. -- **Automate Metadata Generation:** Use scripts or CI/CD tools to dynamically populate customData fields, reducing manual effort. diff --git a/static/docs/selenium-agent-skills.md b/static/docs/selenium-agent-skills.md deleted file mode 100644 index d8b4bd592..000000000 --- a/static/docs/selenium-agent-skills.md +++ /dev/null @@ -1,183 +0,0 @@ -# Run Selenium Tests Using Agent Skills - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Instead of manually setting up your Selenium project, use **Agent Skills** to let your AI coding assistant generate production-ready test automation for you. The [selenium-skill](https://github.com/LambdaTest/agent-skills/tree/main/selenium-skill) is part of [TestMu AI Agent Skills](https://github.com/LambdaTest/agent-skills/) - structured packages that teach AI assistants how to write production-grade test code. - -## What Are Agent Skills? - -Self-contained packages of instructions, code patterns, and configurations for specific testing frameworks. - -Once installed, your AI assistant works like a senior QA automation architect with knowledge of: - -- Proper project structure for each framework -- Correct dependency versions and configurations -- Both local and TestMu AI cloud execution patterns -- Common pitfalls and debugging approaches -- CI/CD integration with GitHub Actions - -## Prerequisites - -Make sure you have the following set up before you start. - -1. [Create a TestMu AI account](https://www.testmuai.com/register/) if you don't have one. -2. Get your **Username** and **Access Key** from the [TestMu AI Dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard). -3. Install an AI coding assistant: [Claude Code](https://claude.ai/code), [GitHub Copilot](https://github.com/features/copilot), [Cursor](https://cursor.sh/), [Gemini CLI](https://github.com/google-gemini/gemini-cli), or any compatible tool. - -## Step 1: Install the Selenium Agent Skill - -Clone the repository and copy the Selenium skill into your AI tool's skills directory. - -```bash -git clone https://github.com/LambdaTest/agent-skills.git - -# For Claude Code -cp -r agent-skills/selenium-skill .claude/skills/ - -# For Cursor -cp -r agent-skills/selenium-skill .cursor/skills/ - -# For GitHub Copilot -cp -r agent-skills/selenium-skill .github/skills/ - -# For Gemini CLI -cp -r agent-skills/selenium-skill .gemini/skills/ -``` - -Install **all available framework skills** at once by cloning the repository directly into your tool's skills directory (e.g., `.claude/skills/`, `.cursor/skills/`). - -## Step 2: Set Your Credentials - -Add your TestMu AI credentials as environment variables so the generated tests can authenticate with the grid. - -Visit the [TestMu AI Dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard), navigate to the left sidebar, and select **Credentials**. Copy your **Username** and **Access Key**, then run: - -```bash -export LT_USERNAME="YOUR_LAMBDATEST_USERNAME" -export LT_ACCESS_KEY="YOUR_LAMBDATEST_ACCESS_KEY" -``` - -## Step 3: Run Your First Test - -Ask your AI assistant to write tests in natural language. - -``` -"Write Selenium tests for the login page and run them on TestMu AI cloud using Chrome and Firefox" -``` - -``` -"Set up a Selenium Java project with TestNG, Page Object Model, and run tests on TestMu AI" -``` - -``` -"Execute Selenium tests across 5 browsers in parallel on TestMu AI, tag the build as 'release-v1.0'" -``` - -The Agent Skill automatically handles: - -1. **Project setup** - Creates the right folder structure, dependencies, and config files -2. **Language detection** - Maps your request to the correct language (Java, Python, JavaScript, C#, Ruby, or PHP) -3. **Execution target** - Configures local or TestMu AI cloud execution based on your request -4. **Best practices** - Uses explicit waits, proper locator strategies, and Page Object Model patterns - -## What's Included in the Selenium Skill - -A breakdown of everything the skill package contains. - -``` -selenium-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -| Section | Details | -|---------|---------| -| **Project Setup** | Dependencies, versions, configuration files, project structure | -| **Core Patterns** | Complete, runnable code examples with best practices | -| **Cloud Integration** | TestMu AI RemoteWebDriver and capabilities configuration | -| **CI/CD Integration** | GitHub Actions workflows with reporting and parallel execution | -| **Debugging Guide** | 11+ common exceptions mapped to solutions | -| **Best Practices** | 14+ actionable recommendations for production code | - -## Supported Languages and Frameworks - -The Selenium Agent Skill supports 6 languages. Each framework below also has its own dedicated Agent Skill with framework-specific patterns. - -### Java - -| Framework | Agent Skill | Documentation | -|-----------|------------|---------------| -| TestNG | [testng-skill](https://github.com/LambdaTest/agent-skills/tree/main/testng-skill) | [TestNG with Selenium](/docs/testng-with-selenium-running-java-automation-scripts-on-testmu-selenium-grid/) | -| JUnit 5 | [junit-5-skill](https://github.com/LambdaTest/agent-skills/tree/main/junit-5-skill) | [JUnit with Selenium](/docs/junit-with-selenium-running-junit-automation-scripts-on-testmu-selenium-grid/) | -| Cucumber | [cucumber-skill](https://github.com/LambdaTest/agent-skills/tree/main/cucumber-skill) | [Cucumber with Selenium](/docs/running-cucumber-scripts-with-testng-and-selenium/) | -| Selenide | [selenide-skill](https://github.com/LambdaTest/agent-skills/tree/main/selenide-skill) | [Selenide Framework](/docs/selenide-tests-with-testmu-online-selenium-grid-for-automated-cross-browser-testing/) | -| Gauge | [gauge-skill](https://github.com/LambdaTest/agent-skills/tree/main/gauge-skill) | [Gauge Framework](/docs/running-gauge-tests-on-testmu-selenium-grid/) | -| Geb | [geb-skill](https://github.com/LambdaTest/agent-skills/tree/main/geb-skill) | [Geb Framework](/docs/run-geb-tests-on-selenium-grid/) | -| Serenity BDD | [serenity-bdd-skill](https://github.com/LambdaTest/agent-skills/tree/main/serenity-bdd-skill) | [Serenity BDD](/docs/serenity-test-on-selenium-grid/) | - -### JavaScript - -| Framework | Agent Skill | Documentation | -|-----------|------------|---------------| -| Mocha | [mocha-skill](https://github.com/LambdaTest/agent-skills/tree/main/mocha-skill) | [Mocha with Selenium](/docs/automation-testing-with-mocha-and-selenium/) | -| Jest | [jest-skill](https://github.com/LambdaTest/agent-skills/tree/main/jest-skill) | [Jest with Selenium](/docs/automation-testing-with-selenium-and-jest/) | -| WebdriverIO | [webdriverio-skill](https://github.com/LambdaTest/agent-skills/tree/main/webdriverio-skill) | [WebdriverIO](/docs/webdriverio-with-selenium-running-webdriverio-automation-scripts-on-testmu-selenium-grid/) | -| Nightwatch.js | [nightwatchjs-skill](https://github.com/LambdaTest/agent-skills/tree/main/nightwatchjs-skill) | [Nightwatch.js](/docs/nightwatch-with-selenium-running-nightwatch-automation-scripts-on-testmu-selenium-grid/) | -| Protractor | [protractor-skill](https://github.com/LambdaTest/agent-skills/tree/main/protractor-skill) | [Protractor](/docs/protractor-with-selenium-running-protractor-automation-scripts-on-testmu-selenium-grid/) | -| TestCafe | [testcafe-skill](https://github.com/LambdaTest/agent-skills/tree/main/testcafe-skill) | [TestCafe](/docs/npm-plugin-for-testcafe-integration-with-testmu/) | -| Jasmine | [jasmine-skill](https://github.com/LambdaTest/agent-skills/tree/main/jasmine-skill) | [Jasmine with Karma](/docs/jasmine-with-karma-running-jasmine-tests-on-testmu-selenium-grid/) | -| Karma | [karma-skill](https://github.com/LambdaTest/agent-skills/tree/main/karma-skill) | [Karma Integration](/docs/karma-integration-with-testmu/) | -| CucumberJS | [cucumber-skill](https://github.com/LambdaTest/agent-skills/tree/main/cucumber-skill) | [CucumberJS](/docs/cucumberjs-with-selenium-running-cucumberjs-automation-scripts-on-testmu-selenium-grid/) | -| NemoJS | [nemojs-skill](https://github.com/LambdaTest/agent-skills/tree/main/nemojs-skill) | [NemoJS](/docs/selenium-tests-with-nemojs/) | - -### C# - -| Framework | Agent Skill | Documentation | -|-----------|------------|---------------| -| NUnit | [nunit-skill](https://github.com/LambdaTest/agent-skills/tree/main/nunit-skill) | [NUnit with Selenium](/docs/nunit-with-selenium-running-nunit-automation-scripts-on-testmu-selenium-grid/) | -| MSTest | [mstest-skill](https://github.com/LambdaTest/agent-skills/tree/main/mstest-skill) | [MSTest with Selenium](/docs/mstest-with-selenium-running-mstest-automation-scripts-on-testmu-selenium-grid/) | -| xUnit | [xunit-skill](https://github.com/LambdaTest/agent-skills/tree/main/xunit-skill) | [xUnit with Selenium](/docs/selenium-with-xunit/) | -| SpecFlow | [specflow-skill](https://github.com/LambdaTest/agent-skills/tree/main/specflow-skill) | [SpecFlow with Selenium](/docs/specflow-with-selenium-running-specflow-automation-scripts-on-testmu-selenium-grid/) | - -### Python - -| Framework | Agent Skill | Documentation | -|-----------|------------|---------------| -| pytest | [pytest-skill](https://github.com/LambdaTest/agent-skills/tree/main/pytest-skill) | [pytest with Selenium](/docs/pytest-with-selenium-running-pytest-automation-script-on-testmu-selenium-grid/) | -| unittest | [unittest-skill](https://github.com/LambdaTest/agent-skills/tree/main/unittest-skill) | [unittest](/docs/running-unit-testing-in-python-on-testmu-selenium-grid/) | -| Robot Framework | [robot-framework-skill](https://github.com/LambdaTest/agent-skills/tree/main/robot-framework-skill) | [Robot Framework](/docs/robot-with-selenium-running-robot-automation-scripts-on-testmu-selenium-grid/) | -| Behave | [behave-skill](https://github.com/LambdaTest/agent-skills/tree/main/behave-skill) | [Behave with Selenium](/docs/behave-with-selenium-running-behave-automation-scripts-on-testmu-selenium-grid/) | -| Lettuce | [lettuce-skill](https://github.com/LambdaTest/agent-skills/tree/main/lettuce-skill) | [Lettuce with Selenium](/docs/automation-testing-selenium-with-lettuce/) | - -### PHP - -| Framework | Agent Skill | Documentation | -|-----------|------------|---------------| -| PHPUnit | [phpunit-skill](https://github.com/LambdaTest/agent-skills/tree/main/phpunit-skill) | [PHPUnit](/docs/phpunit-with-selenium-running-phpunit-automation-scripts-on-testmu-selenium-grid/) | -| Behat | [behat-skill](https://github.com/LambdaTest/agent-skills/tree/main/behat-skill) | [Behat](/docs/behat-with-selenium-running-behat-automation-scripts-on-testmu-selenium-grid/) | -| Laravel Dusk | [laravel-dusk-skill](https://github.com/LambdaTest/agent-skills/tree/main/laravel-dusk-skill) | [Laravel Dusk](/docs/laravel-dusk-with-selenium-running-laravel-dusk-automation-scripts-on-testmu-selenium-grid/) | -| Codeception | [codeception-skill](https://github.com/LambdaTest/agent-skills/tree/main/codeception-skill) | [Codeception](/docs/codeception-integration-with-testmu/) | - -### Ruby - -| Framework | Agent Skill | Documentation | -|-----------|------------|---------------| -| RSpec | [rspec-skill](https://github.com/LambdaTest/agent-skills/tree/main/rspec-skill) | [RSpec](/docs/rspec-with-selenium-running-rspec-automation-scripts-on-testmu-selenium-grid/) | -| Capybara | [capybara-skill](https://github.com/LambdaTest/agent-skills/tree/main/capybara-skill) | [Capybara](/docs/ruby-with-capybara-running-automation-scripts-on-testmu-selenium-grid/) | -| Test::Unit | [testunit-skill](https://github.com/LambdaTest/agent-skills/tree/main/testunit-skill) | [Test::Unit](/docs/testunit-with-selenium-running-testunit-automation-scripts-on-testmu-selenium-grid/) | - -## Supported AI Tools - -Pick the tool you use and install the skill to the corresponding path. - -| Tool | Type | Installation Path | -|------|------|-------------------| -| Claude Code | CLI | `.claude/skills/` | -| GitHub Copilot | Extension | `.github/skills/` | -| Cursor | IDE | `.cursor/skills/` | -| Gemini CLI | CLI | `.gemini/skills/` | -| Codex CLI | CLI | `.codex/skills/` | -| OpenCode | CLI | `.opencode/skills/` | -| Claude.ai | Web | Settings > Features > Skills | diff --git a/static/docs/selenium-automation-capabilities.md b/static/docs/selenium-automation-capabilities.md deleted file mode 100644 index a2e271a0f..000000000 --- a/static/docs/selenium-automation-capabilities.md +++ /dev/null @@ -1,144 +0,0 @@ -# Desired Capabilities in Selenium - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Desired Capabilities in Selenium test scripts, encoded as key-value pairs, allow you to define specific test environment requirements. These requirements include operating systems, browser combinations, versions, and other parameters supported by TestMu AI's Cloud Grid to run your [automation framework tests](/support/docs/getting-started-with-testmu-automation/#supported-languages-and-frameworks). - -You can also use TestMu AI's [Automation Capabilities Generator](https://www.testmuai.com/capabilities-generator/) to autogenerate your required capabilities class to run Selenium and Appium automation test scripts on the TestMu AI [Selenium Grid](https://www.testmuai.com/blog/why-selenium-grid-is-ideal-for-automated-browser-testing/). - -You can generate the capabilities in all six programming languages: Java, PHP, Ruby, JavaScript, C#, and Python. - ->The values provided for any capability are both **space-insensitive** and **case-insensitive**. This means that any value you provide as **Mac OS X 10.13** or **mac os x 10.13** would be correct. - -## Default Capabilities - -These are the built-in capabilities that Selenium supports for browser session configuration. - -| Key | Values | Capability Description | -|-----|--------|------------------------| -| `javascriptEnabled` | true, false | Enables or disables JavaScript execution support based on the provided input.| -| `databaseEnabled` | true, false | Determines whether a session can communicate with the database. | -| `locationContextEnabled` | true, false | Determines whether a session can use features related to different geographic locations.| -| `applicationCacheEnabled` | true, false | Determines whether a session can use the application cache.| -| `browserConnectionEnabled` | true, false | Determines the browser connectivity for the query involved in the session. Also checks whether the end user can disable the session if required. | -| `webStorageEnabled` | true, false | Determines whether a session can connect with [storage objects](https://www.w3.org/TR/2009/WD-webstorage-20091029/).| -| `acceptSslCerts` | true, false | Determines whether a session should accept all SSL certificates by default.| -| `unexpectedAlertBehaviour` | accept, dismiss, ignore | Determines the browser action for handling alerts before pointing the UnhandledAlertException. | -| `elementScrollBehavior` | integer | Determines the scrolling behavior based on the provided input. You can align elements to top(0) or bottom(1) with respect to the elements scroll-ability inside the viewport. By default the value is set to 1, aligning the scroll behavior to top. This is compatible with IE and Firefox (since 2.36). | -| `preventCrossSiteTracking` | true, false | Blocks or unblocks cross-site tracking. | - -## Selenium 3 Capabilities (and below) - -Use these capabilities to configure browser, platform, and test metadata for Selenium version 3 and below. - -| Key | Values | Capability Description | Default Value | Alias | Value Type | Example | Possible Errors | -|-----|--------|------------------------|---------------|--------|------------|---------|-----------------| -| `browserName` | ChromeFirefoxSafariMS Edge - Microsoft EdgeOpera IE - Internet Explorer | This is a mandatory and case-sensitive capability that represents the name of the selected browser. `capabilities.setCapability("browserName", "Chrome");` | Mandatory |browser |String |`capabilities.setCapability("browserName","chrome");` |Case Sensitive Invalid browser error | -| `version` | Example: 101.0 (Beta) | This capability sets the version of the browser mentioned above. The version you provide as a desired capability must be valid. `capabilities.setCapability("version","78.0");`| -| `latest` | - | The latest keyword runs your test cases on the most recent browser version available at TestMu AI. `capabilities.setCapability("version","latest");` | -| `latest-N` | latest-5 | This allows you to execute your test on an older browser version. Here, N represents an integer. If you want to test your website on a browser version released 5 versions before the latest one, use the below capability: `capabilities.setCapability("version","latest-5");`| -| `platformName` | **Supported Windows:** Windows 11Windows 10Windows 8.1Windows 8Windows 7 **Supported macOS:** MontereyBig SurCatalinaMojaveHigh SierraSierraEl CapitanMavericksYosemite | If you do not declare a platform, TestMu AI maps your test to a relevant operating system for the selected browser. To run Selenium automation testing on Windows 11, add the below line of code in your Desired Selenium Capabilities class: ``` capabilities.setCapability("platformName", "Windows 11"); ``` For macOS Monterey, add the below line: ``` capabilities.setCapability("platformName", "macOS Monterey"); ``` |select on the basis of browser. For web Linuxwin10 |platform,OS |String |```capabilities.setCapability("platformName", "Windows 10");``` |Case Sensitive Invalid Platform Error | -| `resolution` | 1024x7681280x8001280x10241366x7681440x9001680x10501600x12001920x12001920x10802048x1536These values are OS Dependent, Verify from Capability generator String: "1920x1200" | This capability specifies a particular screen resolution for your VM (Virtual Machine). By default, the resolution is 1024x768. ``` capabilities.setCapability("resolution","1600x1200"); ``` |1920 x1080 |viewport |String |```capabilities.setCapability("resolution","1600x1200");``` |Case Sensitive Invalid Resolution error | -| `name` | Example: ToDo Sample Test name: "Login testcase"| Represents the name of a test. ``` capabilities.setCapability("name", "ToDo Sample Test"); ``` | test-id |testname,sessionname,test |String |```capabilities.setCapability("name", "ToDo Sample Test");``` |Case Sensitive > 255 char | -| `driver_version` | Example: "driver_version" : "99.0" | Specifies the driver version of the browser you want to run the test on. Provide a valid driver version corresponding to the browser used. ``` capabilities.setCapability("driver_version","99.0");| -| `build` | build: "Regression Job #567" Example: 5. | Represents the build number for your test. ` capabilities.setCapability("build", "5.1");` Note: Build string should be less than 255 characters. |Untitled Build |buildName,job,jobName |String |`capabilities.setCapability("build", "5.1");` Note: Build string should be less than 255 characters. |Case Sensitive > 255 char -| `projectName` |Example: My Test projectName: "Xaviar Project" |Represents the name of a project. ` capabilities.setCapability("projectName", "Test");` | Untitled |projectName,project |String |`caps.setCapability("projectName","Test");` |Case Sensitive > 255 char | -| tags | Example: {"Tag 1", "Tag 2", "Tag 3"} tags: {"Staging", "Critical, "Sanity"} | Groups your automation tests. `String[] tags = {"Tags"};``caps.setCapability("tags", tags);` For more details, visit our [documentation](/support/docs/group-tests-using-custom-tags/).| | |String |```String[] tags = {"Tags"};``` ```caps.setCapability("tags", tags);```For more details, visit our documentation. |Case Sensitive > 50 char | -| `buildTags` | Example: {"Build Tags"} buildTags: {"Build Tags","Staging"} | Groups your test builds. `String[] buildTagList = {"Build Tag"};` `caps.setCapability("buildTags", buildTagList);` For more details, visit our [documentation](/support/docs/group-and-filter-your-test-builds-using-build-tags/). | | |String |```String[] buildTagList = {"Build Tag"};``` ```caps.setCapability("buildTags", buildTagList);``` For more details, visit our documentation. -|`lambda:loadExtension`| Example: `https://automation-prod-user-files.s3.amazonaws.com/extensions/orgId-XXXX/2.1.0_0.zip` | Tests `.zip` extension files uploaded on TestMu AI storage. ```String[] extension = {"https://automation-prod-user-files.s3.amazonaws.com/extensions/orgId-XXXX/2.1.0_0.zip"};``````capabilities.setCapability("lambda:loadExtension", extension);```To upload an extension on TestMu AI storage, refer to our [documentation](/support/api-doc/#/extensions/UploadExtensions). | -|`lambda:userFiles`| Example:804402.png | Tests the uploaded files on TestMu AI. ``` capabilities.setCapability("lambda:userFiles", file); ``` For more details about uploading files, visit our [documentation](/support/docs/upload-files-using-testmu/). | -| -|`commandLog`| Example: true/false | Hides or shows the command logs. ``` capabilities.setCapability("commandLog", false); ``` | FALSE |commandLogs |Boolean | -|`systemLog`| Example: true/false | Hides or shows the Selenium/Appium logs. ``` capabilities.setCapability("systemLog", false); ``` | FALSE |seleniumLogs |Boolean | -|`network.http2` | network.http2: true | Enables HTTP2 support in Martian. Use this along with the network capability. | FALSE | |Boolean | -|`DisableXFHeaders`| DisableXFHeaders:true | Removes the X-Forwarded Headers added by Martian. | FALSE | | Boolean | -|`network.debug` | network.debug: true | Enables debug logging in Martian. Martian logs every request and response. | FALSE | | Boolean | -|`ignoreFfOptionsArgs` | ignoreFfOptionsArgs:true | Ignores args in moz:firefoxOptions. | FALSE | | Boolean | -|`updateBuildStatusOnSuccess` | true/false | Updates build status on replication of test name. | FALSE | | Boolean | - -## Selenium 4 Capabilities (and above) - -In addition to the capabilities supported in Selenium version 3 (above), Selenium version 4 introduces the following new capabilities. Use these to configure authentication, browser version, platform, and session settings for Selenium 4. - -| Key | Values | Capability Description | Default Value | Alias | Value Type | Example | Possible Errors | -|-----|--------|------------------------|---------------|--------|------------|---------|-----------------| -| `user` |```user: "Username"``` Example: YOUR_LAMBDATEST_USERNAME | Find this under your TestMu AI profile. If you are already logged into TestMu AI, the Desired Selenium Capabilities Generator automatically fetches your TestMu AI username under the desired Selenium capabilities class. ``` capabilities.setCapability("user","YOUR_LAMBDATEST_USERNAME"); ``` |Mandatory |username |String |If you are already logged into TestMu AI then the Desired Selenium Capabilities Generator automatically fetches your TestMu AI username under the desired Selenium capabilities class. ```capabilities.setCapability("user","YOUR_LAMBDATEST_USERNAME");```|401 Unauthorized case sensitive | -| `accessKey`|```accessKey: "Access Key"``` Example: YOUR_LAMBDATEST_ACCESS_KEY | Find this under your TestMu AI profile. If you are already logged into TestMu AI, the Desired Selenium Capabilities Generator automatically fetches your TestMu AI Access Key under the desired Selenium capabilities class. ``` capabilities.setCapability("accessKey","YOUR_LAMBDATEST_ACCESS_KEY");``` |Mandatory |key |String |```capabilities.setCapability(""accessKey"",""YOUR_LAMBDATEST_ACCESS_KEY"");``` |401 Unauthorized case sensitive | -| `driver_version`| Example: "driver_version" : "99.0"| Specifies the driver version of the browser you want to run the test on. Provide a valid driver version corresponding to the browser used. `capabilities.setCapability("driver_version","99.0");` | Dynamic map by system |driverVersion,driver |String | -| `browserVersion` | Ex: 101.0 (Beta) | Declares the specific browser version on which you want your test automation scripts to run using the Selenium Automation Grid. Provide a valid browser version in your desired Selenium capabilities class. ``` capabilities.setCapability("browserVersion","79.0"); ```You can use "latest", "latest-1" and so on to select the latest browser version. | latest |version |String |```capabilities.setCapability("browserVersion","91.0");``````capabilities.setCapability("browserVersion","latest");``` ```capabilities.setCapability("browserVersion","latest-1");``` |Case Sensitive Invalid version error | -| `latest` | - | The latest keyword runs your test cases on the most recent browser version available at TestMu AI. ``` capabilities.setCapability("version","latest"); ``` | -| `latest-N` | latest-5 | This allows you to execute your test on older browser versions. Here, N represents an integer. To test your website on the 5 most recently released browser versions, use the below capability: `capabilities.setCapability("version","latest-5");`| -| `platformName` | **Supported Windows:** Windows 11Windows 10Windows 8.1Windows 8Windows 7 **Supported macOS:** MontereyBig SurCatalinaMojaveHigh SierraSierraEl CapitanMavericksYosemite | Specifies the platform on which you want to run your Selenium automation testing. If you do not declare a platform, TestMu AI maps your test to a relevant operating system for the selected browser. To run Selenium automation testing on Windows 11, add the below line of code in your Desired Selenium Capabilities class: ``` capabilities.setCapability("platformName", "Windows 11"); ``` For macOS Monterey, add the below line: ``` capabilities.setCapability("platformName", "macOS Monterey"); ``` | Select based on browser. LinuxWindows 10|platform,OS | String |```capabilities.setCapability("platformName", "Windows 10");``` |Case Sensitive Invalid Platform Error | -| selenium_version | Example: `"selenium_version", "4.0.0"` | Defines the Selenium version for the test session. | Dynamic map by system |seleniumVersion,seVersion | String | -| idleTimeout | - | Defines the idle timeout for the session. | 600 sec |idle | - -## Advanced Capabilities - -These capabilities extend beyond Selenium defaults and give you more control over video, screenshots, network, console logs, tunnels, and other session settings. - -| Key | Values | Capability Description | Default Value | Alias | Value Type | Example | Possible Errors | -|-----|--------|-------------------------|---------------|-------|------------|---------|-----------------| -| `video` | true, false | Records video of the complete screen while the test runs. You can record up to 10 minutes. By default, video is set to true. To turn off video recording, use the Desired Capabilities Generator or set the flag in your capabilities class. `capabilities.setCapability("video",false);` | TRUE | |Boolean |`capabilities.setCapability("video",false);` |Case Sensitive Unable to generate video Video is not playing in video player | -| `visual` | true, false | Records command-by-command screenshots at each test step. By default the flag is set to off. **Note:** 1. Test execution time increases when set to `true`.             2. This capability takes a total of 150 screenshots. After that, use the screenshot command. `capabilities.setCapability("visual",true)`;|FALSE |debug |Boolean |`capabilities.setCapability("visual",true);` |Case sensitive Unable to generate screenshot Screenshot is broken | -| `network` |true, false | Captures a recording for network packets while the test runs. By default, the network flag is set to false. **Note**: Test execution time increases when set to 'true'. `capabilities.setCapability("network",true)`; | FALSE |networkLogs |Boolean |`capabilities.setCapability("network",true);` |Case sensitive Network logs not supported| -| `console` | true, false | Records JavaScript logs for the test. By default, it is set to false. **Note:** Test execution time increases when set to 'true'. `capabilities.setCapability("console",true)`; | FALSE |Capture browser console errors |String |JavaScript logs are recorded for the test. By default, it is set to false. `capabilities.setCapability("console","disable");` |Case Sensitive No error found Not supported on this browser | -| `tunnel` | true, false | Tests your locally hosted web application with Lambda Tunnel, which establishes a TCP with TLS 1.2 secure connection between your system and the cloud servers. By default, the tunnel flag is set to false. `capabilities.setCapability("tunnel",true);` Find more about Lambda Tunnel from our documentation to test on [locally hosted web applications.](/docs/testing-locally-hosted-pages/) | FALSE |local |Boolean |`capabilities.setCapability("tunnel",true);` Find more about TestMu AI Tunnel from our documentation to test on locally hosted web applications. |Case Sensitive Tunnel not running | -| `tunnelName` | Your desired tunnel name | Provides the tunnel identifier in your test suite capability when you want to execute the test through a particular tunnel. `capabilities.setCapability("tunnelName","your desired tunnel name")`;| |localName |String |`capabilities.setCapability("tunnelName","your desired tunnel name");` |Case Sensitive Tunnel not running Tunnel Name not exists | -| `timezone` | Example: UTC+07:00 | Sets a custom timezone for executing a test. `capabilities.setCapability("timezone","UTC+07:00")`; | UTC+00:00| |String |`capabilities.setCapability("timezone","UTC+07:00");` |Case sensitive Timezone not supported by OS Timezone is different | -| `verboseWebDriverLogging` | true, false | Enables detailed Selenium logs to debug issues using insights including command execution, communication with the browser, and responses. `capabilities.setCapability("verboseWebDriverLogging", true)` | FALSE | | Boolean | `capabilities.setCapability("verboseWebDriverLogging", true)` | -| `network.mask` | true, false | Masks network traffic during automated testing. Use this capability when you need secure or anonymized network interactions during test execution - for example, when dealing with sensitive data, internal environments, or compliance requirements.`capabilities.setCapability("network.mask", true)` | FALSE | | Boolean | `capabilities.setCapability("network.mask", true)` | Case Sensitive | - -## Explore more capabilities - - Headless Browser Testing - Run automated tests in browsers without a graphical user interface. - - Custom Chrome Profile - Configure a custom Chrome profile with specific extensions or settings for your tests. - - Setting Browser Options - Define preferences like browser size, language, and cookies for your tests. - - Network Throttling - Simulate different internet connection speeds to test website behavior under various network conditions. - - Custom DNS Mapping - Map custom domain names to specific IP addresses for your testing needs. - - Lighthouse Performance Metrics - Analyze website performance metrics within your tests using Lighthouse. - - Generate Multiple Lighthouse Reports - Run multiple Lighthouse audits and generate reports for comparison. - - Debugging Options - Use debugging tools provided by TestMu AI to troubleshoot your tests. - - Telemetry Logs - Access detailed logs generated during your test execution for analysis. - - SmartWait Functionality - Use TestMu AI's SmartWait feature to wait for elements to become interactive before proceeding with tests. - - CustomHeaders - Set custom headers in your test requests to simulate specific scenarios. - - Autohealing - Use TestMu AI's autohealing functionality to recover automatically from unexpected session issues. - - Command Annotations - Use TestMu AI's command annotations to control test execution flow. - - HAR Logs - Analyze network traffic during your tests using HTTP Archive (HAR) logs. - - Mask your Sensitive Data - Protect sensitive information like passwords or tokens from being exposed in test logs. - ->Got any questions? -Please reach out at our **24x7 Chat Support** or you could also mail us at support@testmuai.com. - -[1]: https://www.testmuai.com/capabilities-generator/ - -[3]: https://www.testmuai.com/list-of-browsers/ -[4]: https://www.testmuai.com/support/docs/testing-locally-hosted-pages/ -[5]: http://www.w3.org/TR/2009/WD-webstorage-20091029/ diff --git a/static/docs/selenium-bidi-integration.md b/static/docs/selenium-bidi-integration.md deleted file mode 100644 index fb3b00fc6..000000000 --- a/static/docs/selenium-bidi-integration.md +++ /dev/null @@ -1,72 +0,0 @@ -# BiDi Testing with Selenium WebDriver on TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Selenium WebDriver BiDi is a W3C standard protocol used to establish communication between a test script and a remote WebDriver server. It introduces bi-directional communication, meaning both the script and the browser can send requests and responses, leading to a more dynamic and reactive testing experience. - -> BiDi is compatible with **Chrome**, **Firefox**, and **Edge** browsers. For more information, refer to the [documentation](https://wpt.fyi/results/webdriver/tests/bidi?label=stable&label=master&aligned). - -## Why Use WebDriver BiDi Protocol? - -BiDi provides event-driven, bi-directional communication for more adaptable cross-browser automation. - -There are several reasons to use BiDi testing with Selenium WebDriver on TestMu AI: - -- **Better Test Maintainability:** Traditional WebDriver relies on a request-response model, where the test script dictates every action. BiDi introduces a standardized protocol, making your tests less vulnerable to breaking due to browser version changes. This reduces maintenance overhead as your tests adapt more gracefully to browser updates. - -- **Expanded Automation Capabilities:** BiDi's key strength lies in its bi-directional communication and event-driven architecture. This allows for a more dynamic and responsive testing experience compared to the traditional approach. Your tests can react to events happening within the browser itself, leading to more adaptable automation. - -- **Cross-Browser Support:** BiDi is a W3C standard, meaning it strives for consistent implementation across different browsers. This eliminates the need to write and maintain separate test scripts for each browser. With the TestMu AI browser grid, you can use BiDi for wider test coverage with a single, unified test suite. - -- **Fine-Grained Control over Browser Interactions:** BiDi offers access to lower-level browser functionalities that were not available with traditional WebDriver. You can now interact with browser features like console logs and network traffic, providing finer control over the testing process. This is useful for debugging complex test scenarios or monitoring browser behavior in detail. - -- **Future-Proofing Your Test Suite:** BiDi represents the future of web browser automation. By adopting BiDi early on, you future-proof your test suite and ensure compatibility with upcoming browser advancements that use this protocol. - -## Steps to Run Tests - -Set up and run BiDi tests using WebdriverIO on the cloud grid. - -Follow these steps to run tests on TestMu AI using **WebdriverIO with BiDi Protocol**: - -### Prerequisites -1. Install Node.js >= 12. -2. Get your TestMu AI [Username and Access Key](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard). - -### Step 1: Set Up the Project - -You can use your own project to configure and test it. For demo purposes, we use the sample repository. - -**Sample repo** -Download or Clone the code sample for the Selenium WebdriverIO BiDi from the TestMu AI GitHub repository to run the tests. - - View on GitHub - -Install all the necessary dependencies of the project by running the following command: - -```bash -npm install -``` - -### Step 2: Set Up Your Credentials - -1. Create a `.env` file in the root folder of your project. -2. Add your TestMu AI [Username and Access Key](/support/docs/using-environment-variables-for-authentication-credentials/) in place of `` and ``. - -```yaml -LT_USERNAME = -LT_ACCESS_KEY = -``` - -### Step 3: Trigger the Tests - -1. Pass the `webSocketUrl` as true in the `wdio.lambdatest.conf.js` file to enable BiDi support. - -```yaml -webSocketUrl: true -``` - -2. Run the following command in your terminal to trigger the tests on TestMu AI platform using the specified configuration. - -```bash -npm run wdio -``` diff --git a/static/docs/selenium-default-capabilities.md b/static/docs/selenium-default-capabilities.md deleted file mode 100644 index 58d90c419..000000000 --- a/static/docs/selenium-default-capabilities.md +++ /dev/null @@ -1,20 +0,0 @@ -# Selenium Default Capabilities - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Default Capabilities Reference - -These are the built-in capabilities that Selenium supports for browser session configuration. - -| Key | Values | Capability Description | -|-----|--------|------------------------| -| `javascriptEnabled` | true, false | Enables or disables JavaScript execution support based on the provided input.| -| `databaseEnabled` | true, false | Determines whether a session can communicate with the database. | -| `locationContextEnabled` | true, false | Determines whether a session can use features related to different geographic locations.| -| `applicationCacheEnabled` | true, false | Determines whether a session can use the application cache.| -| `browserConnectionEnabled` | true, false | Determines the browser connectivity for the query involved in the session. Also checks whether the end user can disable the session if required. | -| `webStorageEnabled` | true, false | Determines whether a session can connect with [storage objects](https://www.w3.org/TR/2009/WD-webstorage-20091029/).| -| `acceptSslCerts` | true, false | Determines whether a session should accept all SSL certificates by default.| -| `unexpectedAlertBehaviour` | accept, dismiss, ignore | Determines the browser action for handling alerts before pointing the UnhandledAlertException. | -| `elementScrollBehavior` | integer | Determines the scrolling behavior based on the provided input. You can align elements to top(0) or bottom(1) with respect to the elements scroll-ability inside the viewport. By default the value is set to 1, aligning the scroll behavior to top. This is compatible with IE and Firefox (since 2.36). | -| `preventCrossSiteTracking` | true, false | Blocks or unblocks cross-site tracking. | diff --git a/static/docs/selenium-faq.md b/static/docs/selenium-faq.md deleted file mode 100644 index 69769f4a8..000000000 --- a/static/docs/selenium-faq.md +++ /dev/null @@ -1,5 +0,0 @@ -# Frequently Asked Questions - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This document covers frequently asked questions from our customers about Selenium testing on TestMu AI. diff --git a/static/docs/selenium-four-capabilities.md b/static/docs/selenium-four-capabilities.md deleted file mode 100644 index b9f3fc7cb..000000000 --- a/static/docs/selenium-four-capabilities.md +++ /dev/null @@ -1,21 +0,0 @@ -# Selenium Capabilities for Version 4 and Above - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -In addition to the capabilities supported in [Selenium version 3](/support/docs/selenium-three-capabilities/), Selenium version 4 introduces the following new capabilities. - -## Capabilities Reference - -Use these capabilities to configure authentication, browser version, platform, and session settings for Selenium 4. - -| Key | Values | Capability Description | Default Value | Alias | Value Type | Example | Possible Errors | -|-----|--------|------------------------|---------------|--------|------------|---------|-----------------| -| `user` |```user: "Username"``` Example: YOUR_LAMBDATEST_USERNAME | Find this under your TestMu AI profile. If you are already logged into TestMu AI, the Desired Selenium Capabilities Generator automatically fetches your TestMu AI username under the desired Selenium capabilities class. ``` capabilities.setCapability("user","YOUR_LAMBDATEST_USERNAME"); ``` |Mandatory |username |String |If you are already logged into TestMu AI then the Desired Selenium Capabilities Generator automatically fetches your TestMu AI username under the desired Selenium capabilities class. ```capabilities.setCapability("user","YOUR_LAMBDATEST_USERNAME");```|401 Unauthorized case sensitive | -| `accessKey`|```accessKey: "Access Key"``` Example: YOUR_LAMBDATEST_ACCESS_KEY | Find this under your TestMu AI profile. If you are already logged into TestMu AI, the Desired Selenium Capabilities Generator automatically fetches your TestMu AI Access Key under the desired Selenium capabilities class. ``` capabilities.setCapability("accessKey","YOUR_LAMBDATEST_ACCESS_KEY");``` |Mandatory |key |String |```capabilities.setCapability(""accessKey"",""YOUR_LAMBDATEST_ACCESS_KEY"");``` |401 Unauthorized case sensitive | -| `driver_version`| Example: "driver_version" : "99.0"| Specifies the driver version of the browser you want to run the test on. Provide a valid driver version corresponding to the browser used. `capabilities.setCapability("driver_version","99.0");` | Dynamic map by system |driverVersion,driver |String | -| `browserVersion` | Ex: 101.0 (Beta) | Declares the specific browser version on which you want your test automation scripts to run using the Selenium Automation Grid. Provide a valid browser version in your desired Selenium capabilities class. ``` capabilities.setCapability("browserVersion","79.0"); ```You can use "latest", "latest-1" and so on to select the latest browser version. | latest |version |String |```capabilities.setCapability("browserVersion","91.0");``````capabilities.setCapability("browserVersion","latest");``` ```capabilities.setCapability("browserVersion","latest-1");``` |Case Sensitive Invalid version error | -| `latest` | - | The latest keyword runs your test cases on the most recent browser version available at TestMu AI. ``` capabilities.setCapability("version","latest"); ``` | -| `latest-N` | latest-5 | This allows you to execute your test on older browser versions. Here, N represents an integer. To test your website on the 5 most recently released browser versions, use the below capability: `capabilities.setCapability("version","latest-5");`| -| `platformName` | **Supported Windows:** Windows 11Windows 10Windows 8.1Windows 8Windows 7 **Supported macOS:** MontereyBig SurCatalinaMojaveHigh SierraSierraEl CapitanMavericksYosemite | Specifies the platform on which you want to run your Selenium automation testing. If you do not declare a platform, TestMu AI maps your test to a relevant operating system for the selected browser. To run Selenium automation testing on Windows 11, add the below line of code in your Desired Selenium Capabilities class: ``` capabilities.setCapability("platformName", "Windows 11"); ``` For macOS Monterey, add the below line: ``` capabilities.setCapability("platformName", "macOS Monterey"); ``` | Select based on browser. LinuxWindows 10|platform,OS | String |```capabilities.setCapability("platformName", "Windows 10");``` |Case Sensitive Invalid Platform Error | -| selenium_version | Example: `"selenium_version", "4.0.0"` | Defines the Selenium version for the test session. | Dynamic map by system |seleniumVersion,seVersion | String | -| idleTimeout | - | Defines the idle timeout for the session. | 600 sec |idle | diff --git a/static/docs/selenium-geolocation-capabilities.md b/static/docs/selenium-geolocation-capabilities.md deleted file mode 100644 index 16dbb4fb8..000000000 --- a/static/docs/selenium-geolocation-capabilities.md +++ /dev/null @@ -1,555 +0,0 @@ -# Selenium Testing from Different Geolocations - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI allows you to run your tests from different geographies across the world. - -**NOTE** -You cannot perform Geolocation testing with locally hosted web applications using TestMu AI Tunnel in a single test case. If you wish to perform both, then you can leverage parallel testing to run two tests simultaneously, one for Geolocation and the other for TestMu AI Tunnel. - -| Key | Values | Capability Description | Alias | Value Type | Example | Possible Errors | -|-----|--------|-------------------------|-------|------------|---------|-----------------| -| `geoLocation` | Example: US | This capability lets you run Selenium automation tests from different countries. `capabilities.setCapability("geoLocation","US");` |geo | String |`capabilities.setCapability("geoLocation","US");`|Case Sensitive Invalid location error - -> **NOTE:** Verify whether your geolocation IP address is working via [MaxMind's GeoIP Web Services](https://www.maxmind.com/en/geoip-web-services-demo). - -You can run Selenium automation tests from the following countries. - -| Country | Country Codes | -|---------|---------------| -| Albania | "AL" | -| Andorra | "AD" | -| Argentina | "AR" | -| Armenia | "AM" | -| Australia | "AU" | -| Austria | "AT" | -| Azerbaijan | "AZ" | -| Bahrain | "BH" | -| Bangladesh | "BD" | -| Belarus | "BY" | -| Belgium | "BE" | -| Bosnia and Herzegovina| "BA" | -| Brazil | "BR" | -| Bulgaria | "BG" | -| Cambodia | "KH" | -| Canada | "CA" | -| Chile | "CL" | -| China | "CN" | -| Colombia | "CO" | -| Costa Rica | "CR" | -| Croatia | "HR" | -| Curacao | "CW" | -| Cyprus | "CY" | -| Czech Republic | "CZ" | -| Denmark | "DK" | -| Dominican Republic| "DO" | -| Ecuador | "EC" | -| Egypt | "EG" | -| El Salvador | "SV" | -| Estonia | "EE" | -| Finland | "FI" | -| France | "FR" | -| Georgia | "GE" | -| Germany | "DE" | -| Greece | "GR" | -| Guatemala | "GT" | -| Honduras | "HN" | -| Hong Kong | "HK" | -| Hungary | "HU" | -| Iceland | "IS" | -| India | "IN" | -| Indonesia | "ID" | -| Ireland | "IE" | -| Isle of Man | "IM" | -| Israel | "IL" | -| Italy | "IT" | -| Jamaica | "JM" | -| Japan | "JP" | -| Jordan | "JO" | -| Kazakhstan | "KZ" | -| Kenya | "KE" | -| Korea | "KR" | -| Kuwait | "KW" | -| Kyrgyzstan | "KG" | -| Latvia | "LV" | -| Lebanon | "LB" | -| Lithuania | "LT" | -| Luxembourg | "LU" | -| Malaysia | "MY" | -| Malta | "MT" | -| Mexico | "MX" | -| Moldova | "MD" | -| Mongolia | "MN" | -| Morocco | "MA" | -| Netherlands | "NL" | -| New Zealand | "NZ" | -| Nicaragua | "NI" | -| Nigeria | "NG" | -| Norway | "NO" | -| Oman | "OM" | -| Pakistan | "PK" | -| Panama | "PA" | -| Paraguay | "PY" | -| Peru | "PE" | -| Philippines | "PH" | -| Poland | "PL" | -| Portugal | "PT" | -| Puerto Rico | "PR" | -| Qatar | "QA" | -| Romania | "RO" | -| Russian Federation| "RU" | -| Saudi Arabia | "SA" | -| Serbia | "RS" | -| Singapore | "SG" | -| Slovakia | "SK" | -| Slovenia | "SI" | -| South Africa | "ZA" | -| Spain | "ES" | -| Sweden | "SE" | -| Switzerland | "CH" | -| Taiwan | "TW" | -| Tanzania | "TZ" | -| Thailand | "TH" | -| Tunisia | "TN" | -| Turkey | "TR" | -| Ukraine | "UA" | -| United Arab Emirates| "AE" | -| United Kingdom | "GB" | -| United States | "US" | -| Uruguay | "UY" | -| Uzbekistan | "UZ" | -| Venezuela | "VE" | -| Vietnam | "VN" | - -## List of Supported Timezones - -The following timezones are supported for automation testing on the TestMu AI platform. - -| City | Timezone | -| -------------- | -------------------------------------------------------------- | -| Abidjan | {"Africa/Abidjan", "Greenwich Standard Time"}, | -| Accra | {"Africa/Accra", "Greenwich Standard Time"}, | -| Addis_Ababa | {"Africa/Addis_Ababa", "E. Africa Standard Time"}, | -| Algiers | {"Africa/Algiers", "W. Central Africa Standard Time"}, | -| Asmara | {"Africa/Asmara", "nan"}, | -| Bamako | {"Africa/Bamako", "Greenwich Standard Time"}, | -| Bangui | {"Africa/Bangui", "W. Central Africa Standard Time"}, | -| Banjul | {"Africa/Banjul", "Greenwich Standard Time"}, | -| Bissau | {"Africa/Bissau", "Greenwich Standard Time"}, | -| Blantyre | {"Africa/Blantyre", "South Africa Standard Time"}, | -| Brazzaville | {"Africa/Brazzaville", "W. Central Africa Standard Time"}, | -| Bujumbura | {"Africa/Bujumbura", "South Africa Standard Time"}, | -| Cairo | {"Africa/Cairo", "Egypt Standard Time"}, | -| Casablanca | {"Africa/Casablanca", "Morocco Standard Time"}, | -| Ceuta | {"Africa/Ceuta", "Romance Standard Time"}, | -| Conakry | {"Africa/Conakry", "Greenwich Standard Time"}, | -| Dakar | {"Africa/Dakar", "Greenwich Standard Time"}, | -| Dar_es_Salaam | {"Africa/Dar_es_Salaam", "E. Africa Standard Time"}, | -| Djibouti | {"Africa/Djibouti", "E. Africa Standard Time"}, | -| Douala | {"Africa/Douala", "W. Central Africa Standard Time"}, | -| El_Aaiun | {"Africa/El_Aaiun", "Morocco Standard Time"}, | -| Freetown | {"Africa/Freetown", "Greenwich Standard Time"}, | -| Gaborone | {"Africa/Gaborone", "South Africa Standard Time"}, | -| Harare | {"Africa/Harare", "South Africa Standard Time"}, | -| Johannesburg | {"Africa/Johannesburg", "South Africa Standard Time"}, | -| Juba | {"Africa/Juba", "E. Africa Standard Time"}, | -| Kampala | {"Africa/Kampala", "E. Africa Standard Time"}, | -| Khartoum | {"Africa/Khartoum", "E. Africa Standard Time"}, | -| Kigali | {"Africa/Kigali", "South Africa Standard Time"}, | -| Kinshasa | {"Africa/Kinshasa", "W. Central Africa Standard Time"}, | -| Lagos | {"Africa/Lagos", "W. Central Africa Standard Time"}, | -| Libreville | {"Africa/Libreville", "W. Central Africa Standard Time"}, | -| Lome | {"Africa/Lome", "Greenwich Standard Time"}, | -| Luanda | {"Africa/Luanda", "W. Central Africa Standard Time"}, | -| Lubumbashi | {"Africa/Lubumbashi", "South Africa Standard Time"}, | -| Lusaka | {"Africa/Lusaka", "South Africa Standard Time"}, | -| Malabo | {"Africa/Malabo", "W. Central Africa Standard Time"}, | -| Maputo | {"Africa/Maputo", "South Africa Standard Time"}, | -| Maseru | {"Africa/Maseru", "South Africa Standard Time"}, | -| Mbabane | {"Africa/Mbabane", "South Africa Standard Time"}, | -| Mogadishu | {"Africa/Mogadishu", "E. Africa Standard Time"}, | -| Monrovia | {"Africa/Monrovia", "Greenwich Standard Time"}, | -| Nairobi | {"Africa/Nairobi", "E. Africa Standard Time"}, | -| Ndjamena | {"Africa/Ndjamena", "W. Central Africa Standard Time"}, | -| Niamey | {"Africa/Niamey", "W. Central Africa Standard Time"}, | -| Nouakchott | {"Africa/Nouakchott", "Greenwich Standard Time"}, | -| Ouagadougou | {"Africa/Ouagadougou", "Greenwich Standard Time"}, | -| Porto-Novo | {"Africa/Porto-Novo", "W. Central Africa Standard Time"}, | -| Sao_Tome | {"Africa/Sao_Tome", "Greenwich Standard Time"}, | -| Tripoli | {"Africa/Tripoli", "Libya Standard Time"}, | -| Tunis | {"Africa/Tunis", "W. Central Africa Standard Time"}, | -| Windhoek | {"Africa/Windhoek", "Namibia Standard Time"}, | -| Adak | {"America/Adak", "nan"}, | -| Anchorage | {"America/Anchorage", "Alaskan Standard Time"}, | -| Anguilla | {"America/Anguilla", "SA Western Standard Time"}, | -| Antigua | {"America/Antigua", "SA Western Standard Time"}, | -| Araguaina | {"America/Araguaina", "SA Eastern Standard Time"}, | -| Buenos_Aires | {"America/Argentina/Buenos_Aires", "nan"}, | -| Catamarca | {"America/Argentina/Catamarca", "nan"}, | -| Cordoba | {"America/Argentina/Cordoba", "nan"}, | -| Jujuy | {"America/Argentina/Jujuy", "nan"}, | -| La_Rioja | {"America/Argentina/La_Rioja", "Argentina Standard Time"}, | -| Mendoza | {"America/Argentina/Mendoza", "nan"}, | -| Rio_Gallegos | {"America/Argentina/Rio_Gallegos", "Argentina Standard Time"}, | -| Salta | {"America/Argentina/Salta", "Argentina Standard Time"}, | -| San_Juan | {"America/Argentina/San_Juan", "Argentina Standard Time"}, | -| San_Luis | {"America/Argentina/San_Luis", "Argentina Standard Time"}, | -| Tucuman | {"America/Argentina/Tucuman", "Argentina Standard Time"}, | -| Ushuaia | {"America/Argentina/Ushuaia", "Argentina Standard Time"}, | -| Aruba | {"America/Aruba", "SA Western Standard Time"}, | -| Asuncion | {"America/Asuncion", "Paraguay Standard Time"}, | -| Atikokan | {"America/Atikokan", "nan"}, | -| Bahia | {"America/Bahia", "Bahia Standard Time"}, | -| Bahia_Banderas | {"America/Bahia_Banderas", "Central Standard Time (Mexico)"}, | -| Barbados | {"America/Barbados", "SA Western Standard Time"}, | -| Belem | {"America/Belem", "SA Eastern Standard Time"}, | -| Belize | {"America/Belize", "Central America Standard Time"}, | -| Blanc-Sablon | {"America/Blanc-Sablon", "SA Western Standard Time"}, | -| Boa_Vista | {"America/Boa_Vista", "SA Western Standard Time"}, | -| Bogota | {"America/Bogota", "SA Pacific Standard Time"}, | -| Boise | {"America/Boise", "Mountain Standard Time"}, | -| Cambridge_Bay | {"America/Cambridge_Bay", "Mountain Standard Time"}, | -| Campo_Grande | {"America/Campo_Grande", "Central Brazilian Standard Time"}, | -| Cancun | {"America/Cancun", "Eastern Standard Time (Mexico)"}, | -| Caracas | {"America/Caracas", "Venezuela Standard Time"}, | -| Cayenne | {"America/Cayenne", "SA Eastern Standard Time"}, | -| Cayman | {"America/Cayman", "SA Pacific Standard Time"}, | -| Chicago | {"America/Chicago", "Central Standard Time"}, | -| Chihuahua | {"America/Chihuahua", "Mountain Standard Time (Mexico)"}, | -| Costa_Rica | {"America/Costa_Rica", "Central America Standard Time"}, | -| Creston | {"America/Creston", "US Mountain Standard Time"}, | -| Cuiaba | {"America/Cuiaba", "Central Brazilian Standard Time"}, | -| Curacao | {"America/Curacao", "SA Western Standard Time"}, | -| Danmarkshavn | {"America/Danmarkshavn", "UTC"}, | -| Dawson | {"America/Dawson", "Pacific Standard Time"}, | -| Dawson_Creek | {"America/Dawson_Creek", "US Mountain Standard Time"}, | -| Denver | {"America/Denver", "Mountain Standard Time"}, | -| Detroit | {"America/Detroit", "Eastern Standard Time"}, | -| Dominica | {"America/Dominica", "SA Western Standard Time"}, | -| Edmonton | {"America/Edmonton", "Mountain Standard Time"}, | -| Eirunepe | {"America/Eirunepe", "SA Pacific Standard Time"}, | -| El_Salvador | {"America/El_Salvador", "Central America Standard Time"}, | -| Fortaleza | {"America/Fortaleza", "SA Eastern Standard Time"}, | -| Glace_Bay | {"America/Glace_Bay", "Atlantic Standard Time"}, | -| Godthab | {"America/Godthab", "Greenland Standard Time"}, | -| Goose_Bay | {"America/Goose_Bay", "Atlantic Standard Time"}, | -| Grand_Turk | {"America/Grand_Turk", "SA Western Standard Time"}, | -| Grenada | {"America/Grenada", "SA Western Standard Time"}, | -| Guadeloupe | {"America/Guadeloupe", "SA Western Standard Time"}, | -| Guatemala | {"America/Guatemala", "Central America Standard Time"}, | -| Guayaquil | {"America/Guayaquil", "SA Pacific Standard Time"}, | -| Guyana | {"America/Guyana", "SA Western Standard Time"}, | -| Halifax | {"America/Halifax", "Atlantic Standard Time"}, | -| Havana | {"America/Havana", "Eastern Standard Time"}, | -| Hermosillo | {"America/Hermosillo", "US Mountain Standard Time"}, | -| Indianapolis | {"America/Indiana/Indianapolis", "nan"}, | -| Knox | {"America/Indiana/Knox", "Central Standard Time"}, | -| Marengo | {"America/Indiana/Marengo", "US Eastern Standard Time"}, | -| Petersburg | {"America/Indiana/Petersburg", "Eastern Standard Time"}, | -| Tell_City | {"America/Indiana/Tell_City", "Central Standard Time"}, | -| Vevay | {"America/Indiana/Vevay", "US Eastern Standard Time"}, | -| Vincennes | {"America/Indiana/Vincennes", "Eastern Standard Time"}, | -| Winamac | {"America/Indiana/Winamac", "Eastern Standard Time"}, | -| Inuvik | {"America/Inuvik", "Mountain Standard Time"}, | -| Iqaluit | {"America/Iqaluit", "Eastern Standard Time"}, | -| Jamaica | {"America/Jamaica", "SA Pacific Standard Time"}, | -| Juneau | {"America/Juneau", "Alaskan Standard Time"}, | -| Louisville | {"America/Kentucky/Louisville", "nan"}, | -| Monticello | {"America/Kentucky/Monticello", "Eastern Standard Time"}, | -| Kralendijk | {"America/Kralendijk", "SA Western Standard Time"}, | -| La_Paz | {"America/La_Paz", "SA Western Standard Time"}, | -| Lima | {"America/Lima", "SA Pacific Standard Time"}, | -| Los_Angeles | {"America/Los_Angeles", "Pacific Standard Time"}, | -| Lower_Princes | {"America/Lower_Princes", "SA Western Standard Time"}, | -| Maceio | {"America/Maceio", "SA Eastern Standard Time"}, | -| Managua | {"America/Managua", "Central America Standard Time"}, | -| Manaus | {"America/Manaus", "SA Western Standard Time"}, | -| Marigot | {"America/Marigot", "SA Western Standard Time"}, | -| Martinique | {"America/Martinique", "SA Western Standard Time"}, | -| Matamoros | {"America/Matamoros", "Central Standard Time"}, | -| Mazatlan | {"America/Mazatlan", "Mountain Standard Time (Mexico)"}, | -| Menominee | {"America/Menominee", "Central Standard Time"}, | -| Merida | {"America/Merida", "Central Standard Time (Mexico)"}, | -| Metlakatla | {"America/Metlakatla", "nan"}, | -| Mexico_City | {"America/Mexico_City", "Central Standard Time (Mexico)"}, | -| Miquelon | {"America/Miquelon", "nan"}, | -| Moncton | {"America/Moncton", "Atlantic Standard Time"}, | -| Monterrey | {"America/Monterrey", "Central Standard Time (Mexico)"}, | -| Montevideo | {"America/Montevideo", "Montevideo Standard Time"}, | -| Montreal | {"America/Montreal", "Eastern Standard Time"}, | -| Montserrat | {"America/Montserrat", "SA Western Standard Time"}, | -| Nassau | {"America/Nassau", "Eastern Standard Time"}, | -| New_York | {"America/New_York", "Eastern Standard Time"}, | -| Nipigon | {"America/Nipigon", "Eastern Standard Time"}, | -| Nome | {"America/Nome", "Alaskan Standard Time"}, | -| Noronha | {"America/Noronha", "UTC-02"}, | -| Beulah | {"America/North_Dakota/Beulah", "Central Standard Time"}, | -| Center | {"America/North_Dakota/Center", "Central Standard Time"}, | -| New_Salem | {"America/North_Dakota/New_Salem", "Central Standard Time"}, | -| Ojinaga | {"America/Ojinaga", "Mountain Standard Time"}, | -| Panama | {"America/Panama", "SA Pacific Standard Time"}, | -| Pangnirtung | {"America/Pangnirtung", "Eastern Standard Time"}, | -| Paramaribo | {"America/Paramaribo", "SA Eastern Standard Time"}, | -| Phoenix | {"America/Phoenix", "US Mountain Standard Time"}, | -| Port-au-Prince | {"America/Port-au-Prince", "Eastern Standard Time"}, | -| Port_of_Spain | {"America/Port_of_Spain", "SA Western Standard Time"}, | -| Porto_Velho | {"America/Porto_Velho", "SA Western Standard Time"}, | -| Puerto_Rico | {"America/Puerto_Rico", "SA Western Standard Time"}, | -| Rainy_River | {"America/Rainy_River", "Central Standard Time"}, | -| Rankin_Inlet | {"America/Rankin_Inlet", "Central Standard Time"}, | -| Recife | {"America/Recife", "SA Eastern Standard Time"}, | -| Regina | {"America/Regina", "Canada Central Standard Time"}, | -| Resolute | {"America/Resolute", "Central Standard Time"}, | -| Rio_Branco | {"America/Rio_Branco", "SA Pacific Standard Time"}, | -| Santa_Isabel | {"America/Santa_Isabel", "Pacific Standard Time (Mexico)"}, | -| Santarem | {"America/Santarem", "SA Eastern Standard Time"}, | -| Santiago | {"America/Santiago", "Pacific SA Standard Time"}, | -| Santo_Domingo | {"America/Santo_Domingo", "SA Western Standard Time"}, | -| Sao_Paulo | {"America/Sao_Paulo", "E. South America Standard Time"}, | -| Scoresbysund | {"America/Scoresbysund", "Azores Standard Time"}, | -| Shiprock | {"America/Shiprock", "nan"}, | -| Sitka | {"America/Sitka", "Alaskan Standard Time"}, | -| St_Barthelemy | {"America/St_Barthelemy", "SA Western Standard Time"}, | -| St_Johns | {"America/St_Johns", "Newfoundland Standard Time"}, | -| St_Kitts | {"America/St_Kitts", "SA Western Standard Time"}, | -| St_Lucia | {"America/St_Lucia", "SA Western Standard Time"}, | -| St_Thomas | {"America/St_Thomas", "SA Western Standard Time"}, | -| St_Vincent | {"America/St_Vincent", "SA Western Standard Time"}, | -| Swift_Current | {"America/Swift_Current", "Canada Central Standard Time"}, | -| Tegucigalpa | {"America/Tegucigalpa", "Central America Standard Time"}, | -| Thule | {"America/Thule", "Atlantic Standard Time"}, | -| Thunder_Bay | {"America/Thunder_Bay", "Eastern Standard Time"}, | -| Tijuana | {"America/Tijuana", "Pacific Standard Time"}, | -| Toronto | {"America/Toronto", "Eastern Standard Time"}, | -| Tortola | {"America/Tortola", "SA Western Standard Time"}, | -| Vancouver | {"America/Vancouver", "Pacific Standard Time"}, | -| Whitehorse | {"America/Whitehorse", "Pacific Standard Time"}, | -| Winnipeg | {"America/Winnipeg", "Central Standard Time"}, | -| Yakutat | {"America/Yakutat", "Alaskan Standard Time"}, | -| Yellowknife | {"America/Yellowknife", "Mountain Standard Time"}, | -| Casey | {"Antarctica/Casey", "W. Australia Standard Time"}, | -| Davis | {"Antarctica/Davis", "SE Asia Standard Time"}, | -| DumontDUrville | {"Antarctica/DumontDUrville", "West Pacific Standard Time"}, | -| Macquarie | {"Antarctica/Macquarie", "Central Pacific Standard Time"}, | -| Mawson | {"Antarctica/Mawson", "West Asia Standard Time"}, | -| McMurdo | {"Antarctica/McMurdo", "New Zealand Standard Time"}, | -| Palmer | {"Antarctica/Palmer", "Pacific SA Standard Time"}, | -| Rothera | {"Antarctica/Rothera", "SA Eastern Standard Time"}, | -| South_Pole | {"Antarctica/South_Pole", "nan"}, | -| Syowa | {"Antarctica/Syowa", "E. Africa Standard Time"}, | -| Troll | {"Antarctica/Troll", "nan"}, | -| Vostok | {"Antarctica/Vostok", "Central Asia Standard Time"}, | -| Longyearbyen | {"Arctic/Longyearbyen", "W. Europe Standard Time"}, | -| Aden | {"Asia/Aden", "Arab Standard Time"}, | -| Almaty | {"Asia/Almaty", "Central Asia Standard Time"}, | -| Amman | {"Asia/Amman", "Jordan Standard Time"}, | -| Anadyr | {"Asia/Anadyr", "Russia Time Zone 11"}, | -| Aqtau | {"Asia/Aqtau", "West Asia Standard Time"}, | -| Aqtobe | {"Asia/Aqtobe", "West Asia Standard Time"}, | -| Ashgabat | {"Asia/Ashgabat", "West Asia Standard Time"}, | -| Baghdad | {"Asia/Baghdad", "Arabic Standard Time"}, | -| Bahrain | {"Asia/Bahrain", "Arab Standard Time"}, | -| Baku | {"Asia/Baku", "Azerbaijan Standard Time"}, | -| Bangkok | {"Asia/Bangkok", "SE Asia Standard Time"}, | -| Beirut | {"Asia/Beirut", "Middle East Standard Time"}, | -| Bishkek | {"Asia/Bishkek", "Central Asia Standard Time"}, | -| Brunei | {"Asia/Brunei", "Singapore Standard Time"}, | -| Chita | {"Asia/Chita", "North Asia East Standard Time"}, | -| Choibalsan | {"Asia/Choibalsan", "Ulaanbaatar Standard Time"}, | -| Chongqing | {"Asia/Chongqing", "nan"}, | -| Colombo | {"Asia/Colombo", "Sri Lanka Standard Time"}, | -| Damascus | {"Asia/Damascus", "Syria Standard Time"}, | -| Dhaka | {"Asia/Dhaka", "Bangladesh Standard Time"}, | -| Dili | {"Asia/Dili", "Tokyo Standard Time"}, | -| Dubai | {"Asia/Dubai", "Arabian Standard Time"}, | -| Dushanbe | {"Asia/Dushanbe", "West Asia Standard Time"}, | -| Gaza | {"Asia/Gaza", "nan"}, | -| Harbin | {"Asia/Harbin", "nan"}, | -| Hebron | {"Asia/Hebron", "nan"}, | -| Ho_Chi_Minh | {"Asia/Ho_Chi_Minh", "nan"}, | -| Hong_Kong | {"Asia/Hong_Kong", "China Standard Time"}, | -| Hovd | {"Asia/Hovd", "SE Asia Standard Time"}, | -| Irkutsk | {"Asia/Irkutsk", "North Asia East Standard Time"}, | -| Jakarta | {"Asia/Jakarta", "SE Asia Standard Time"}, | -| Jayapura | {"Asia/Jayapura", "Tokyo Standard Time"}, | -| Jerusalem | {"Asia/Jerusalem", "Israel Standard Time"}, | -| Kabul | {"Asia/Kabul", "Afghanistan Standard Time"}, | -| Kamchatka | {"Asia/Kamchatka", "Russia Time Zone 11"}, | -| Karachi | {"Asia/Karachi", "Pakistan Standard Time"}, | -| Kashgar | {"Asia/Kashgar", "nan"}, | -| Kathmandu | {"Asia/Kathmandu", "nan"}, | -| Katmandu | {"Asia/Katmandu", "Nepal Standard Time"}, | -| Khandyga | {"Asia/Khandyga", "Yakutsk Standard Time"}, | -| Kolkata | {"Asia/Kolkata", "India Standard Time"}, | -| Krasnoyarsk | {"Asia/Krasnoyarsk", "North Asia Standard Time"}, | -| Kuala_Lumpur | {"Asia/Kuala_Lumpur", "Singapore Standard Time"}, | -| Kuching | {"Asia/Kuching", "Singapore Standard Time"}, | -| Kuwait | {"Asia/Kuwait", "Arab Standard Time"}, | -| Macau | {"Asia/Macau", "China Standard Time"}, | -| Magadan | {"Asia/Magadan", "Magadan Standard Time"}, | -| Makassar | {"Asia/Makassar", "Singapore Standard Time"}, | -| Manila | {"Asia/Manila", "Singapore Standard Time"}, | -| Muscat | {"Asia/Muscat", "Arabian Standard Time"}, | -| Nicosia | {"Asia/Nicosia", "GTB Standard Time"}, | -| Novokuznetsk | {"Asia/Novokuznetsk", "North Asia Standard Time"}, | -| Novosibirsk | {"Asia/Novosibirsk", "N. Central Asia Standard Time"}, | -| Omsk | {"Asia/Omsk", "N. Central Asia Standard Time"}, | -| Oral | {"Asia/Oral", "West Asia Standard Time"}, | -| Phnom_Penh | {"Asia/Phnom_Penh", "SE Asia Standard Time"}, | -| Pontianak | {"Asia/Pontianak", "SE Asia Standard Time"}, | -| Pyongyang | {"Asia/Pyongyang", "nan"}, | -| Qatar | {"Asia/Qatar", "Arab Standard Time"}, | -| Qyzylorda | {"Asia/Qyzylorda", "Central Asia Standard Time"}, | -| Rangoon | {"Asia/Rangoon", "Myanmar Standard Time"}, | -| Riyadh | {"Asia/Riyadh", "Arab Standard Time"}, | -| Sakhalin | {"Asia/Sakhalin", "Vladivostok Standard Time"}, | -| Samarkand | {"Asia/Samarkand", "West Asia Standard Time"}, | -| Seoul | {"Asia/Seoul", "Korea Standard Time"}, | -| Shanghai | {"Asia/Shanghai", "China Standard Time"}, | -| Singapore | {"Asia/Singapore", "Singapore Standard Time"}, | -| Srednekolymsk | {"Asia/Srednekolymsk", "Russia Time Zone 10"}, | -| Taipei | {"Asia/Taipei", "Taipei Standard Time"}, | -| Tashkent | {"Asia/Tashkent", "West Asia Standard Time"}, | -| Tbilisi | {"Asia/Tbilisi", "Georgian Standard Time"}, | -| Tehran | {"Asia/Tehran", "Iran Standard Time"}, | -| Thimphu | {"Asia/Thimphu", "Bangladesh Standard Time"}, | -| Tokyo | {"Asia/Tokyo", "Tokyo Standard Time"}, | -| Ulaanbaatar | {"Asia/Ulaanbaatar", "Ulaanbaatar Standard Time"}, | -| Urumqi | {"Asia/Urumqi", "Central Asia Standard Time"}, | -| Ust-Nera | {"Asia/Ust-Nera", "Vladivostok Standard Time"}, | -| Vientiane | {"Asia/Vientiane", "SE Asia Standard Time"}, | -| Vladivostok | {"Asia/Vladivostok", "Vladivostok Standard Time"}, | -| Yakutsk | {"Asia/Yakutsk", "Yakutsk Standard Time"}, | -| Yekaterinburg | {"Asia/Yekaterinburg", "Ekaterinburg Standard Time"}, | -| Yerevan | {"Asia/Yerevan", "Caucasus Standard Time"}, | -| Azores | {"Atlantic/Azores", "Azores Standard Time"}, | -| Bermuda | {"Atlantic/Bermuda", "Atlantic Standard Time"}, | -| Canary | {"Atlantic/Canary", "GMT Standard Time"}, | -| Cape_Verde | {"Atlantic/Cape_Verde", "Cape Verde Standard Time"}, | -| Faroe | {"Atlantic/Faroe", "nan"}, | -| Madeira | {"Atlantic/Madeira", "GMT Standard Time"}, | -| Reykjavik | {"Atlantic/Reykjavik", "Greenwich Standard Time"}, | -| South_Georgia | {"Atlantic/South_Georgia", "UTC-02"}, | -| St_Helena | {"Atlantic/St_Helena", "Greenwich Standard Time"}, | -| Stanley | {"Atlantic/Stanley", "SA Eastern Standard Time"}, | -| Adelaide | {"Australia/Adelaide", "Cen. Australia Standard Time"}, | -| Brisbane | {"Australia/Brisbane", "E. Australia Standard Time"}, | -| Broken_Hill | {"Australia/Broken_Hill", "Cen. Australia Standard Time"}, | -| Currie | {"Australia/Currie", "Tasmania Standard Time"}, | -| Darwin | {"Australia/Darwin", "AUS Central Standard Time"}, | -| Eucla | {"Australia/Eucla", "nan"}, | -| Hobart | {"Australia/Hobart", "Tasmania Standard Time"}, | -| Lindeman | {"Australia/Lindeman", "E. Australia Standard Time"}, | -| Lord_Howe | {"Australia/Lord_Howe", "nan"}, | -| Melbourne | {"Australia/Melbourne", "AUS Eastern Standard Time"}, | -| Perth | {"Australia/Perth", "W. Australia Standard Time"}, | -| Sydney | {"Australia/Sydney", "AUS Eastern Standard Time"}, | -| Amsterdam | {"Europe/Amsterdam", "W. Europe Standard Time"}, | -| Andorra | {"Europe/Andorra", "W. Europe Standard Time"}, | -| Athens | {"Europe/Athens", "GTB Standard Time"}, | -| Belgrade | {"Europe/Belgrade", "Central Europe Standard Time"}, | -| Berlin | {"Europe/Berlin", "W. Europe Standard Time"}, | -| Bratislava | {"Europe/Bratislava", "Central Europe Standard Time"}, | -| Brussels | {"Europe/Brussels", "Romance Standard Time"}, | -| Bucharest | {"Europe/Bucharest", "GTB Standard Time"}, | -| Budapest | {"Europe/Budapest", "Central Europe Standard Time"}, | -| Busingen | {"Europe/Busingen", "W. Europe Standard Time"}, | -| Chisinau | {"Europe/Chisinau", "GTB Standard Time"}, | -| Copenhagen | {"Europe/Copenhagen", "Romance Standard Time"}, | -| Dublin | {"Europe/Dublin", "GMT Standard Time"}, | -| Gibraltar | {"Europe/Gibraltar", "W. Europe Standard Time"}, | -| Guernsey | {"Europe/Guernsey", "GMT Standard Time"}, | -| Helsinki | {"Europe/Helsinki", "FLE Standard Time"}, | -| Isle_of_Man | {"Europe/Isle_of_Man", "GMT Standard Time"}, | -| Istanbul | {"Europe/Istanbul", "Turkey Standard Time"}, | -| Jersey | {"Europe/Jersey", "GMT Standard Time"}, | -| Kaliningrad | {"Europe/Kaliningrad", "Kaliningrad Standard Time"}, | -| Kiev | {"Europe/Kiev", "FLE Standard Time"}, | -| Lisbon | {"Europe/Lisbon", "GMT Standard Time"}, | -| Ljubljana | {"Europe/Ljubljana", "Central Europe Standard Time"}, | -| London | {"Europe/London", "GMT Standard Time"}, | -| Luxembourg | {"Europe/Luxembourg", "W. Europe Standard Time"}, | -| Madrid | {"Europe/Madrid", "Romance Standard Time"}, | -| Malta | {"Europe/Malta", "W. Europe Standard Time"}, | -| Mariehamn | {"Europe/Mariehamn", "FLE Standard Time"}, | -| Minsk | {"Europe/Minsk", "Belarus Standard Time"}, | -| Monaco | {"Europe/Monaco", "W. Europe Standard Time"}, | -| Moscow | {"Europe/Moscow", "Russian Standard Time"}, | -| Oslo | {"Europe/Oslo", "W. Europe Standard Time"}, | -| Paris | {"Europe/Paris", "Romance Standard Time"}, | -| Podgorica | {"Europe/Podgorica", "Central Europe Standard Time"}, | -| Prague | {"Europe/Prague", "Central Europe Standard Time"}, | -| Riga | {"Europe/Riga", "FLE Standard Time"}, | -| Rome | {"Europe/Rome", "W. Europe Standard Time"}, | -| Samara | {"Europe/Samara", "Russia Time Zone 3"}, | -| San_Marino | {"Europe/San_Marino", "W. Europe Standard Time"}, | -| Sarajevo | {"Europe/Sarajevo", "Central European Standard Time"}, | -| Simferopol | {"Europe/Simferopol", "Russian Standard Time"}, | -| Skopje | {"Europe/Skopje", "Central European Standard Time"}, | -| Sofia | {"Europe/Sofia", "FLE Standard Time"}, | -| Stockholm | {"Europe/Stockholm", "W. Europe Standard Time"}, | -| Tallinn | {"Europe/Tallinn", "FLE Standard Time"}, | -| Tirane | {"Europe/Tirane", "Central Europe Standard Time"}, | -| Uzhgorod | {"Europe/Uzhgorod", "FLE Standard Time"}, | -| Vaduz | {"Europe/Vaduz", "W. Europe Standard Time"}, | -| Vatican | {"Europe/Vatican", "W. Europe Standard Time"}, | -| Vienna | {"Europe/Vienna", "W. Europe Standard Time"}, | -| Vilnius | {"Europe/Vilnius", "FLE Standard Time"}, | -| Volgograd | {"Europe/Volgograd", "Russian Standard Time"}, | -| Warsaw | {"Europe/Warsaw", "Central European Standard Time"}, | -| Zagreb | {"Europe/Zagreb", "Central European Standard Time"}, | -| Zaporozhye | {"Europe/Zaporozhye", "FLE Standard Time"}, | -| Zurich | {"Europe/Zurich", "W. Europe Standard Time"}, | -| GMT | {"GMT", "GMT Standard Time"}, | -| Antananarivo | {"Indian/Antananarivo", "E. Africa Standard Time"}, | -| Chagos | {"Indian/Chagos", "Central Asia Standard Time"}, | -| Christmas | {"Indian/Christmas", "SE Asia Standard Time"}, | -| Cocos | {"Indian/Cocos", "Myanmar Standard Time"}, | -| Comoro | {"Indian/Comoro", "E. Africa Standard Time"}, | -| Kerguelen | {"Indian/Kerguelen", "West Asia Standard Time"}, | -| Mahe | {"Indian/Mahe", "Mauritius Standard Time"}, | -| Maldives | {"Indian/Maldives", "West Asia Standard Time"}, | -| Mauritius | {"Indian/Mauritius", "Mauritius Standard Time"}, | -| Mayotte | {"Indian/Mayotte", "E. Africa Standard Time"}, | -| Reunion | {"Indian/Reunion", "Mauritius Standard Time"}, | -| Apia | {"Pacific/Apia", "Samoa Standard Time"}, | -| Auckland | {"Pacific/Auckland", "New Zealand Standard Time"}, | -| Bougainville | {"Pacific/Bougainville", "Central Pacific Standard Time"}, | -| Chatham | {"Pacific/Chatham", "nan"}, | -| Chuuk | {"Pacific/Chuuk", "nan"}, | -| Easter | {"Pacific/Easter", "nan"}, | -| Efate | {"Pacific/Efate", "Central Pacific Standard Time"}, | -| Enderbury | {"Pacific/Enderbury", "Tonga Standard Time"}, | -| Fakaofo | {"Pacific/Fakaofo", "Tonga Standard Time"}, | -| Fiji | {"Pacific/Fiji", "Fiji Standard Time"}, | -| Funafuti | {"Pacific/Funafuti", "UTC+12"}, | -| Galapagos | {"Pacific/Galapagos", "Central America Standard Time"}, | -| Gambier | {"Pacific/Gambier", "nan"}, | -| Guadalcanal | {"Pacific/Guadalcanal", "Central Pacific Standard Time"}, | -| Guam | {"Pacific/Guam", "West Pacific Standard Time"}, | -| Honolulu | {"Pacific/Honolulu", "Hawaiian Standard Time"}, | -| Johnston | {"Pacific/Johnston", "Hawaiian Standard Time"}, | -| Kiritimati | {"Pacific/Kiritimati", "Line Islands Standard Time"}, | -| Kosrae | {"Pacific/Kosrae", "Central Pacific Standard Time"}, | -| Kwajalein | {"Pacific/Kwajalein", "UTC+12"}, | -| Majuro | {"Pacific/Majuro", "UTC+12"}, | -| Marquesas | {"Pacific/Marquesas", "nan"}, | -| Midway | {"Pacific/Midway", "UTC-11"}, | -| Nauru | {"Pacific/Nauru", "UTC+12"}, | -| Niue | {"Pacific/Niue", "UTC-11"}, | -| Norfolk | {"Pacific/Norfolk", "nan"}, | -| Noumea | {"Pacific/Noumea", "Central Pacific Standard Time"}, | -| Pago_Pago | {"Pacific/Pago_Pago", "UTC-11"}, | -| Palau | {"Pacific/Palau", "Tokyo Standard Time"}, | -| Pitcairn | {"Pacific/Pitcairn", "nan"}, | -| Pohnpei | {"Pacific/Pohnpei", "nan"}, | -| Ponape | {"Pacific/Ponape", "Central Pacific Standard Time"}, | -| Port_Moresby | {"Pacific/Port_Moresby", "West Pacific Standard Time"}, | -| Rarotonga | {"Pacific/Rarotonga", "Hawaiian Standard Time"}, | -| Saipan | {"Pacific/Saipan", "West Pacific Standard Time"}, | -| Tahiti | {"Pacific/Tahiti", "Hawaiian Standard Time"}, | -| Tarawa | {"Pacific/Tarawa", "UTC+12"}, | -| Tongatapu | {"Pacific/Tongatapu", "Tonga Standard Time"}, | -| Truk | {"Pacific/Truk", "West Pacific Standard Time"}, | -| Wake | {"Pacific/Wake", "UTC+12"}, | -| Wallis | {"Pacific/Wallis", "UTC+12"}, | diff --git a/static/docs/selenium-hyperexecute-accessibility-tests.md b/static/docs/selenium-hyperexecute-accessibility-tests.md deleted file mode 100644 index 377641270..000000000 --- a/static/docs/selenium-hyperexecute-accessibility-tests.md +++ /dev/null @@ -1,64 +0,0 @@ -# HyperExecute integration: Selenium accessibility automation - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This page is the **HyperExecute** integration guide for **Accessibility Automation** when your tests use **Selenium**. It assumes you already run Selenium jobs on HyperExecute (YAML + CLI) and need the same **accessibility** session capabilities and reporting behavior on the grid, not a generic Selenium-only tutorial. For framework setup on local or other grids, start with **[Accessibility Automation](/support/docs/accessibility-automation/)** and the **[Selenium](/support/docs/accessibility-automation-test/)** setup doc. - -**Scope:** Selenium test code + `hyperexecute.yaml` orchestration on HyperExecute. If the Accessibility tab never appears on sessions, confirm `accessibility: true` (and related caps) in the process HyperExecute actually runs, and that Accessibility is enabled for your organization. Contact support if needed. - -## Prerequisites - -- A valid HyperExecute YAML file -- HyperExecute CLI installed locally -- Your [TestMu AI Username and Access key](/support/docs/using-environment-variables-for-authentication-credentials/) -- A Selenium project that already runs on HyperExecute - -## High-level flow (HyperExecute) - -1. Keep your Selenium suite configured for **HyperExecute** (YAML, discovery, `testRunnerCommand`). -2. Enable **Accessibility** capabilities in the same session options you pass to `RemoteWebDriver` (or equivalent). HyperExecute forwards them like any other capability. -3. Trigger the job with the **HyperExecute CLI** using your YAML. -4. When the build finishes, open the session in the **Automation** dashboard and use the **Accessibility** tab for the report. - -## Onboarding checklist (first HE + Accessibility run) - -1. **Confirm Selenium on HyperExecute** already runs a green build **without** Accessibility (isolate grid/YAML issues first). -2. **Add the same capability keys** you would use locally: `accessibility: true`, optional `accessibility.wcagVersion`, `accessibility.bestPractice`, `accessibility.needsReview`, and either **`lambda-accessibility-scan`** hooks in code or **`accessibility.autoscan`** ([settings reference](/support/docs/accessibility-automation-settings/)). -3. **Propagate credentials** in the HyperExecute job the same way as your other HE runs (`LT_USERNAME` / `LT_ACCESS_KEY` in env or secrets). -4. **Run a short suite** (one class or smoke tag) so the HE dashboard shows a single session to inspect. -5. Open the [Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard) → session → **Accessibility** tab. If the tab is missing, verify the session capabilities in the build log. - -## YAML and test code (integration point) - -HyperExecute does **not** replace Selenium capabilities. It **orchestrates** the same test JAR or Node project. The integration work is making sure the command HyperExecute runs is the profile that sets accessibility caps. Ensure: - -- Your **test process** still loads the code path that sets `accessibility` on `RemoteWebDriver` / options. -- **Discovery** in `hyperexecute.yaml` points at the same `mvn`/`gradle`/`npm` command you use locally for Accessibility runs. - -Example pattern (illustrative only, adapt to your repo’s actual `runson`, `pre`, and `testRunnerCommand`): - -```yaml -version: 0.1 -runson: linux -testSuiteStep: 90 -pre: -- npm install # or mvn dependency:go-offline -testRunnerCommand: npm run test:a11y # must invoke tests that set accessibility caps -``` - -If your team keeps caps in **environment-specific property files**, inject those files or env vars in `pre` or `env` blocks so CI matches local. - -## When something fails - -| Symptom | What to check | -|--------|----------------| -| HE green but no Accessibility tab | Session never received `accessibility: true`; confirm the command HyperExecute ran is the Accessibility-enabled profile. | -| Timeouts | Auto-scan on heavy SPAs increases duration; switch to hook-based scans for critical paths only. | - -## Related docs - -- [Accessibility Automation (Overview)](/support/docs/accessibility-automation/) -- [Selenium: Accessibility Automation setup](/support/docs/accessibility-automation-test/) -- [Configure Accessibility Automation](/support/docs/accessibility-automation-settings/) -- [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) -- [HyperExecute YAML parameters](/support/docs/hyperexecute-yaml-parameters/) diff --git a/static/docs/selenium-java-sdk.md b/static/docs/selenium-java-sdk.md deleted file mode 100644 index 3e6224eea..000000000 --- a/static/docs/selenium-java-sdk.md +++ /dev/null @@ -1,320 +0,0 @@ -# TestMu AI Selenium Java SDK - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The TestMu AI Selenium Java SDK enables you to run your existing Selenium TestNG tests on TestMu AI's cloud grid with **zero code changes**. Configure a YAML file, add a Maven dependency, and include a single listener to get started. - -## Key Features - -Review what the SDK offers out of the box. - -- **Zero Code Changes** - Run your existing Selenium tests without modifying your test code -- **Simple Configuration** - YAML-based configuration for credentials and capabilities -- **Thread-Safe by Default** - Built-in thread safety for parallel test execution -- **Easy Integration** - Add a dependency and listener to your project -- **Cloud Grid Access** - Run tests on 3000+ real browsers and operating systems - -## Prerequisites - -Complete these steps before using the SDK. - -1. [Create a TestMu AI account](https://www.testmuai.com/register/) if you do not have one. -2. Get your TestMu AI [Username and Access Key](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard). -3. Install Java 8 or higher. -4. Install and configure Maven. -5. Prepare an existing Java test project using TestNG (support for additional frameworks is coming soon). - -## Quick Start - -Get running on the cloud grid in three steps. - -### Step 1: Add Maven Dependency - -Add the TestMu AI Selenium Java SDK dependency to your `pom.xml`. Configure the agent to start automatically during your build process: - -```xml - -.... - -io.github.lambdatest -lambdatest-selenium-java-sdk -1.0.1 - - - - -.... - - -maven-compiler-plugin -3.7.0 - -10 - - - - -maven-dependency-plugin - - -getClasspathFilenames - -properties - - - -copy-lambdatest-agent -generate-test-resources - -copy - - - - -io.github.lambdatest -lambdatest-selenium-java-sdk -1.0.1 -lambdatest-agent.jar -${project.build.directory}/agents - - - - - - - - -org.apache.maven.plugins -maven-surefire-plugin -2.19.1 - - - -test - - - - - -${suite} - - --javaagent:${project.build.directory}/agents/lambdatest-agent.jar - - - - - - -``` - -Check the latest version of [testmuai-selenium-java-sdk](https://mvnrepository.com/artifact/io.github.lambdatest/lambdatest-selenium-java-sdk) and update the version accordingly. - -### Step 2: Create Configuration File - -Create a `lambdatest.yml` file in your project root directory with your TestMu AI credentials and desired capabilities: - -```yaml -# TestMu AI Configuration File for Tests -# This file is loaded automatically by the TestMu AI SDK - -username: YOUR_LAMBDATEST_USERNAME -accesskey: YOUR_LAMBDATEST_ACCESS_KEY - -platforms: -- browserName: Chrome -browserVersion: latest -platformName: Windows 10 - -# Test metadata -build: SDK Build Classes Methods v1 -project: SDK Project -name: SDK Test - -# TestMu AI features -video: true -network: false -console: true -visual: false - -# Advanced capabilities (optional) -resolution: 1920x1080 -tunnel: true -``` - -**Using Environment Variables** -You can also use environment variables in your YAML file: - -```yaml -username: ${LT_USERNAME} -accesskey: ${LT_ACCESS_KEY} -``` - -Set these environment variables before running your tests: - -- `LT_USERNAME` - Your TestMu AI username -- `LT_ACCESS_KEY` - Your TestMu AI access key - -### Step 3: Add TestNG Listener - -Add the TestMu AI status listener to your TestNG configuration: - -**Using testng.xml** - -```xml - - - - - - - - - - - -``` - -That's it! Your tests are now configured to run on TestMu AI. No code changes required in your existing test methods. - -## Thread Safety - -The SDK handles parallel execution without additional synchronization. - -The TestMu AI Selenium Java SDK is **thread-safe by default**, making it suitable for parallel test execution. You can run multiple tests concurrently without worrying about thread synchronization issues. - -### Parallel Execution Example - -The SDK fully supports parallel test execution with TestNG: - -```xml - - - - - - - - - - - - - - -``` - -## Example Test - -See how your existing tests work without any modifications. - -```java -import org.openqa.selenium.By; -import org.openqa.selenium.WebDriver; -import org.openqa.selenium.chrome.ChromeOptions; -import org.openqa.selenium.remote.RemoteWebDriver; -import org.testng.annotations.AfterMethod; -import org.testng.annotations.BeforeMethod; -import org.testng.annotations.Test; -import java.net.URL; - -public class BasicTest { -WebDriver driver; - -@BeforeMethod -public void setup() throws Exception { -ChromeOptions options = new ChromeOptions(); -options.setCapability("platformName", "Windows 10"); -options.setCapability("browserVersion", "latest"); - -// SDK automatically injects TestMu AI capabilities from lambdatest.yml -driver = new RemoteWebDriver( -new URL("https://hub.lambdatest.com/wd/hub"), -options -); -} - -@Test -public void testExample() { -driver.get("https://www.lambdatest.com"); -String title = driver.getTitle(); -System.out.println("Page title: " + title); -assert title.contains("LambdaTest"); -} - -@AfterMethod -public void teardown() { -if (driver != null) { -driver.quit(); -} -} -} -``` - -## Troubleshooting - -Resolve common issues when running tests with the SDK. - -### Common Issues - -**Issue: Driver not connecting to TestMu AI** - -- Verify credentials in `lambdatest.yml` or environment variables -- Check your TestMu AI account has active minutes - -**Issue: Tests not marked as passed/failed** - -- Ensure TestNG listener is configured correctly -- Verify driver session ID is available -- Check network connectivity to TestMu AI - -**Issue: Configuration not loading** - -- Ensure `lambdatest.yml` is in the project root directory -- Verify YAML syntax is correct - -## Additional Resources - -Find related guides and tools for further configuration. - -- [TestMu AI Selenium Grid Documentation](/support/docs/getting-started-with-testmu-automation) -- [Capabilities Generator](https://www.testmuai.com/capabilities-generator/) -- [GitHub Repository](https://github.com/Lambdatest/lambdatest-java-selenium-sdk) -- [TestMu AI Support](https://www.testmuai.com/support) - -## Changelog - -Track SDK version history and feature additions. - -### Version 1.0.1 (Latest) - -**New Features:** - -- **Enhanced Thread Safety:** Improved driver instance management for thread-safe parallel test execution -- **Tunnel Support**: Added support for TestMu AI Tunnel configuration in `lambdatest.yml` - - Configure tunnel settings directly in YAML configuration - - Automatic tunnel lifecycle management -- **Enhanced Configuration**: Improved YAML configuration parsing and validation -- **Better Error Handling**: More descriptive error messages for configuration issues - -**Improvements:** - -- Optimized SDK initialization and configuration loading -- Improved compatibility with TestNG execution - -### Version 1.0.0 (Initial Release) - -**Core Features:** - -- Zero code changes approach for running Selenium tests on TestMu AI -- YAML-based configuration (`lambdatest.yml`) -- Two TestNG listener integration for automatic test status updates -- Integration with TestMu AI Selenium Grid - -**Supported Features:** - -- Video recording -- Network logging -- Console logging -- Visual debugging -- Test metadata (build, project, name) diff --git a/static/docs/selenium-mask-your-data.md b/static/docs/selenium-mask-your-data.md deleted file mode 100644 index 3421097f4..000000000 --- a/static/docs/selenium-mask-your-data.md +++ /dev/null @@ -1,40 +0,0 @@ -# Mask Your Sensitive Data in Selenium Tests - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Overview - -Use the **maskCommands** capability to hide sensitive data sent to or retrieved from remote browsers at TestMu AI. - -When used, this capability hides all the keystrokes and other required values, and replaces them with **'*'** in test session logs (both text and raw). - -## Syntax - -Add the maskCommands capability to your desired capabilities configuration. - -**Syntax (Java)**: - -`capabilities.setCapability("lambdaMaskCommands", {""});` - -## Supported Values - -Use the following comma-separated values to specify what data to mask. - -| Value | Description | -|-------|-------------| -| `setValues` | Hide/Redact all the text sent via sendKeys command. | -| `setCookies` | Hide/Redact all the cookies set by the addCookie command.| -| `getCookies` | Hide/Redact all the cookie values obtained using the getCookies and getCookieNamed command.| - -## Language Examples - -The following examples show how to configure maskCommands in each supported language. - -| Language | Example | -|-------|-------------| -| Java |`capabilities.setCapability("lambdaMaskCommands", {"setValues", "setCookies, getCookies"});` | -| Node.js | `capabilities['lambdaMaskCommands'] = ["setValues", "setCookies", "getCookies"]`| -| C# | `capabilities.SetCapability("lambdaMaskCommands", {"setValues", "setCookies", "getCookies"});`| -| PHP | `$capabilities["lambdaMaskCommands"] = array("setValues", "setCookies", "getCookies")`| -| Python | `capabilities["lambdaMaskCommands"] = ["setValues", "setCookies", "getCookies"]`| -| Ruby |`capabilities["lambdaMaskCommands"] = ["setValues", "setCookies", "getCookies"]`| diff --git a/static/docs/selenium-set-browser-options.md b/static/docs/selenium-set-browser-options.md deleted file mode 100644 index b790b7eb9..000000000 --- a/static/docs/selenium-set-browser-options.md +++ /dev/null @@ -1,86 +0,0 @@ -# Setting Browser Options - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Browser-Specific Options - -Configure Chrome, Firefox, and Edge browser arguments using their respective Options classes. - -Selenium lets you set browser arguments for Chromium-based browsers. When selecting Chrome, Firefox, and Edge, you can set browser options using `ChromeOptions`, `EdgeOptions`, and `FirefoxOptions` capability as shown below. - -```java -import org.openqa.selenium.WebDriver; -import org.openqa.selenium.chrome.ChromeDriver; -import org.openqa.selenium.chrome.ChromeOptions; - -public class ChromeTest { -public static void main(String[] args) { -ChromeOptions options = new ChromeOptions(); -// Set Chrome capabilities -options.addArguments("--headless"); - -WebDriver driver = new ChromeDriver(options); -driver.get("http://example.com"); -// Your test code here -driver.quit(); -} -} -``` - -```java -import org.openqa.selenium.WebDriver; -import org.openqa.selenium.firefox.FirefoxDriver; -import org.openqa.selenium.firefox.FirefoxOptions; - -public class FirefoxTest { -public static void main(String[] args) { -FirefoxOptions options = new FirefoxOptions(); -// Set Firefox capabilities -options.addArguments("--headless"); - -WebDriver driver = new FirefoxDriver(options); -driver.get("http://example.com"); -// Your test code here -driver.quit(); -} -} -``` - -```java -import org.openqa.selenium.WebDriver; -import org.openqa.selenium.edge.EdgeDriver; -import org.openqa.selenium.edge.EdgeOptions; - -public class EdgeTest { -public static void main(String[] args) { -EdgeOptions options = new EdgeOptions(); -// Set Edge capabilities -options.addArguments("--headless"); - -WebDriver driver = new EdgeDriver(options); -driver.get("http://example.com"); -// Your test code here -driver.quit(); -} -} -``` - -## Automate IE Mode in Microsoft Edge - -Run IE Mode automation on Microsoft Edge for Windows 11 using the InternetExplorerOptions class. - -TestMu AI's Automation allows you to automate IE Mode in Microsoft Edge on Windows 11. Pass the below desired capability in your test script and execute your tests. - -```java -InternetExplorerOptions browserOptions = new InternetExplorerOptions(); -browserOptions.setPlatformName("Windows 11"); -browserOptions.setBrowserVersion("11.0"); -HashMap ltOptions = new HashMap(); -ltOptions.put("username", "Your LambdaTest Username"); -ltOptions.put("accessKey", "Your LambdaTest Access Key"); -ltOptions.put("project", "Untitled"); -ltOptions.put("w3c", true); -browserOptions.setCapability("LT:Options", ltOptions); -``` - -For a detailed guide on IE Mode testing, see [Test in IE Mode on Edge](/support/docs/ie-mode-on-edge/). diff --git a/static/docs/selenium-supported-languages-and-frameworks.md b/static/docs/selenium-supported-languages-and-frameworks.md deleted file mode 100644 index 10bf3e291..000000000 --- a/static/docs/selenium-supported-languages-and-frameworks.md +++ /dev/null @@ -1,7 +0,0 @@ -# Selenium Supported Languages and Frameworks - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI supports all languages and frameworks compatible with Selenium. Pick your language below to find setup instructions, working code samples, and framework-specific guides. - -If your framework isn't listed, contact us and we'll help you get running. diff --git a/static/docs/selenium-telemetry-logs.md b/static/docs/selenium-telemetry-logs.md deleted file mode 100644 index d77ae3e45..000000000 --- a/static/docs/selenium-telemetry-logs.md +++ /dev/null @@ -1,64 +0,0 @@ -# Debugging in Selenium Using Telemetry Logs - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI Automation allows you to perform parallel test execution through a standalone mode that connects to the Selenium server, where each test can have multiple requests. - -To trace each request, you can use Selenium 4 and OpenTelemetry integration to achieve greater visibility into request tracing and event logs. Tracing enables the monitoring of a request's life cycle and stores essential information about the request's critical operations. You can use this information to identify issues and optimize the performance of your web applications. - -Managing several requests, tracing logs, and troubleshooting request failures in a distributed environment can be challenging. In such instances, you can integrate Telemetry logs with Jaeger - a distributed tracing system - to quickly visualize and query traces. - -## Supported Operating Systems - -Telemetry logs are available on these operating systems. - -* Windows 11 -* Windows 10 -* macOS Ventura -* macOS Catalina -* macOS Mojave - -## Enabling Telemetry Logs - -Pass the `seTelemetryLogs` and `selenium_version` capabilities to activate telemetry logging. - -**Note** -TestMu AI Automation supports Telemetry logs on Selenium versions above 4.2.0. - -By default, Telemetry logs are disabled for Selenium automation on TestMu AI. To enable them in your test sessions: - -1. Pass the `seTelemetryLogs` capability in your test scripts and set it to true. - -```bash -caps.setCapability("seTelemetryLogs", true); -``` - -2. Pass the `selenium_version` capability in your test scripts and set it to `4.2.0`. - -```bash -caps.setCapability("selenium_version", "4.2.0"); -``` - -## Visualizing Telemetry Logs Using Jaeger - -Integrate Jaeger to visualize request traces from your test sessions. - -Jaeger is an open-source, end-to-end distributed tracing system to monitor and troubleshoot distributed systems. Using Jaeger, you can visualize the request traces of your tests run on TestMu AI. - -Here are the steps to integrate Jaeger with your test suites: - -1. Download Jaeger from its [official website](https://www.jaegertracing.io/download/#binaries). - -2. Extract the compressed (*.tar.gz* or *.zip*) Jaeger file to the directory where your test script resides. You will now have several Jaeger files like `jaeger-query`, `jaeger-collector`, etc., in your test script directory. - -3. Download Telemetry logs of your test session from the **Telemetry Logs** tab of your TestMu AI Automation Dashboard. - -4. Start the Jaeger UI to visualize the Telemetry logs by running the below command. - -```bash -jaeger-all-in-one -``` - -5. Visit `http://localhost:16686/search` and click the **Upload** tab to upload the downloaded Telemetry logs from the TestMu AI Automation Dashboard. - -After uploading, you can view the request traces in the Jaeger UI for your TestMu AI test sessions. diff --git a/static/docs/selenium-testing-with-concourse-ci-and-testmu.md b/static/docs/selenium-testing-with-concourse-ci-and-testmu.md deleted file mode 100644 index 0a7bc8984..000000000 --- a/static/docs/selenium-testing-with-concourse-ci-and-testmu.md +++ /dev/null @@ -1,97 +0,0 @@ -# Selenium Testing With Concourse CI And TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Concourse CI is an open-source CI/CD tool which is triggered over Docker containers. It offers a scalable and straightforward solution to your CI/CD pipelines with an intuitive UI that allows you to analyze the dashboard and validate your build execution. Offering a container-centric approach, Concourse CI offers a sturdy pipeline as each task in every job is handled in its container, controlling its dependencies. - -Concourse CI can be integrated with TestMu AI to perform automation testing with a cloud Selenium Grid of 3000+ real browsers for mobile and desktop. This document will help you understand how you can integrate your Concourse CI instance with TestMu AI Selenium Grid for continuous testing. - -## Prerequisites - -1. A TestMu AI account. It is free to get started. Sign up now. - -2. Install NPM(Node Package Manager). If you have npm already installed, you may want to upgrade it to latest version. Here the code you can run in your terminal to upgrade npm. - -``` -npm install npm@latest -g -``` - -3. TestMu AI Authentication Credentials i.e. your TestMu AI username, access key and HubURL. You need to set them up as your environment variables. You can retrieve them from your [TestMu AI automation dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/) by clicking on the key icon near the help button. - - * **For Linux/Mac:** - - $ export LT_USERNAME= `{YOUR_LAMBDATEST_USERNAME}` - - $ export LT_ACCESS_KEY= `{YOUR_LAMBDATEST_ACCESS_KEY}` - - * **For Windows:** - - $ set LT_USERNAME= `{YOUR_LAMBDATEST_USERNAME}` - - $ set LT_ACCESS_KEY= `{YOUR_LAMBDATEST_ACCESS_KEY}` - -4. Install Concourse and start a Concourse server at port 8080 i.e. ``http://127.0.0.1:8080``. - -5. Install fly CLI tool for Concourse CI/CD. We recommend using the latest fly version, you can check the fly version using: - -``` -$ fly -v -``` - -## Configuring Concourse-CI Pipeline - -> Note: Clone this [GitHub repository for Selenium and Concourse CI](https://github.com/LambdaTest/concourse-nodejs-selenium-sample). - -Open your terminal or command prompt. Login to your Concourse server and save the target. - -``` -$ fly -t ci login -c http://127.0.0.1:8080 -u test -p test -``` - -Next, we need to update the environment variables in the `YAML` file. To access your YAML file, you need to go to the project-folder/concourse-ci. Here you will find the YAML file i.e. `pipeline-config.yml`. - -```js -resources: -- name: nodejs-selenium-sample -type: git -icon: github -source: -url: https://github.com/sushobhit-lt/nodejs-selenium-sample.git - -jobs: -- name: 'Run over Lambdatest Hub' -public: true -plan: -- get: nodejs-selenium-sample -trigger: true -- task: 'Install dependencies and execute' -config: -platform: linux -image_resource: -type: registry-image -source: { repository: node, tag: "12" } -inputs: -- name: nodejs-selenium-sample -run: -path: /bin/sh -args: -- -c -- | -cd nodejs-selenium-sample -npm install -export LT_USERNAME=Your_LambdaTest_Username -export LT_ACCESS_KEY=Your_LambdaTest_Access_Key -node index.js -``` - -After you specify the right configuration in the `pipeline-config.yml`, you now need to create your concourse pipeline with the help of fly CLI. - -``` -$ fly -t ci set-pipeline -p nodejs-lambda-sample -c pipeline-config.yml -``` - -Now, you will find the nodejs-selenium-sample pipeline in the Concourse Web UI. You can run the pipeline over TestMu AI Selenium Grid from the UI itself. - -After you trigger the test, you will notice the different stages of test execution. The Concourse CI will get the nodejs-selenium-sample and execute it on TestMu AI after installing the right dependencies. - -Deploy your code in a reliable manner at scale using Concourse CI integration with TestMu AI, and ensure it looks robust across every browser to provide a seamless user experience to all your visitors. Happy Testing! diff --git a/static/docs/selenium-tests-behind-proxy.md b/static/docs/selenium-tests-behind-proxy.md deleted file mode 100644 index d95aa4445..000000000 --- a/static/docs/selenium-tests-behind-proxy.md +++ /dev/null @@ -1,88 +0,0 @@ -# Run Selenium Tests Behind the Proxy - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -If your system is behind a proxy or protected by a firewall, you cannot run Selenium test scripts directly on the TestMu AI Selenium grid. You need to define your proxy settings so the connection goes through as usual. - -This document shows you how to run your Selenium tests behind a proxy server. - -## How to Define Proxy Settings in Selenium C# - -Add the proxy host and port to the HttpCommandExecutor in your test code. - -You can define proxy programmatically by adding the below snippet in your code. - -```csharp -HttpCommandExecutor commandExecutor = new HttpCommandExecutor(new Uri("https://username:accesskey@hub.lambdatest.com/"), TimeSpan.FromSeconds(60)); -WebProxy myproxy = new WebProxy("proxy_host:proxy_port", false); -IWebDriver driver; -``` - -Below is the NUnit code that shows the insertion of the above code snippet to define proxy settings. - -```csharp -using System; -using System.Drawing.Text; -using System.Threading; -using OpenQA.Selenium; -using OpenQA.Selenium.Chrome; -using OpenQA.Selenium.Remote; -using NUnit.Framework; -using System.Net; -namespace TestSelenium3 -{ -public class Program -{ -static void setProxy() -{ -/* HttpWebRequest myWebRequest = (HttpWebRequest)WebRequest.Create("http://3.86.55.62:8888"); -// Obtain the 'Proxy' of the Default browser. -IWebProxy proxy = myWebRequest.Proxy; -// Print the Proxy Url to the console. -if (proxy != null) -{ -Console.WriteLine("Proxy: {0}", proxy.GetProxy(myWebRequest.RequestUri)); -} -else -{ -Console.WriteLine("Proxy is null; no proxy will be used"); -} -*/ -} -public static void Main(String[] args) -{ -// Init -setProxy(); -HttpCommandExecutor commandExecutor = new HttpCommandExecutor(new Uri("https://username:accesskey@hub.lambdatest.com/"), TimeSpan.FromSeconds(60)); -commandExecutor.Proxy = new WebProxy("proxy_host:proxy_port", false); -IWebDriver driver; -DesiredCapabilities capabilities = new DesiredCapabilities(); -capabilities.SetCapability("build", "your build name"); -capabilities.SetCapability("name", "your test name"); -capabilities.SetCapability("platform", "Windows 10"); -capabilities.SetCapability("browserName", "Chrome"); -capabilities.SetCapability("version", "89.0"); -driver = new RemoteWebDriver(commandExecutor, capabilities); - -Console.WriteLine("----------------------------" + capabilities + "#####################################"); -try -{ -/* -driver = new RemoteWebDriver(new Uri("https://username:accesskey@hub.lambdatest.com/"), capability); -Console.WriteLine("----------------------------" + driver + "#####################################"); -*/ -driver.Navigate().GoToUrl("https://www.google.com"); -} -catch (Exception e) -{ -Console.WriteLine(e); -Thread.Sleep(500000); -} -} -} -} -``` - -Execute the above NUnit test scripts, and you can run your Selenium tests directly on the TestMu AI Selenium grid behind the proxy server. - -If you have any questions or require an additional information, you can contact us at our **24/7 chat support**. You can also drop us a mail at support@testmuai.com. diff --git a/static/docs/selenium-tests-with-nemojs.md b/static/docs/selenium-tests-with-nemojs.md deleted file mode 100644 index 3ead38c45..000000000 --- a/static/docs/selenium-tests-with-nemojs.md +++ /dev/null @@ -1,106 +0,0 @@ -# Selenium With NemoJS - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Run NemoJS tests on the TestMu AI cloud grid. This guide covers setup, running a sample test, configuring capabilities, and testing locally hosted pages. - -**Sample repo** -All the code used in this guide is available in the sample repository. - - View on GitHub - -## Prerequisites - -Complete these steps before running NemoJS Selenium tests on TestMu AI. - -1. Create a [TestMu AI account](https://www.testmuai.com/register/?redirectTo=https://accounts.lambdatest.com/dashboard) and get your username and access key from the dashboard. -2. Install **NodeJS** from [nodejs.org](https://nodejs.org/en/). -3. Install **npm**: `npm init` -4. Install **NemoJS**: `npm install --save-dev nemo` - -## Step 1: Clone the Sample Project - -Clone the TestMu AI NemoJS sample repository to your local machine. - -```bash -git clone https://github.com/LambdaTest/Nemo-Lambdatest-sample -cd Nemo-Lambdatest-sample -``` - -Install the required dependencies: -```bash -npm install -``` - -## Step 2: Set Your Credentials - -Set your TestMu AI username and access key as environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -Set the environment variables in the `nemo.config.js` file with your TestMu AI Username and Access Key. - -## Step 3: Configure Your Test Capabilities - -Update the capabilities in `nemo.config.js` to define the browser and platform settings. - -``` js -"withCapabilities": [{ -"build": "LT Nemo Sample Tests", -"name": "LT Nemo sample test", -"platformName" : "Windows 10", -"browserName" : "Chrome", -"browserVersion" : "latest", -"user": "env:LT_USERNAME", -"accessKey": "env:LT_ACCESS_KEY" -}] -``` - -Generate capabilities for your test requirements with the [Capabilities Generator](https://www.testmuai.com/capabilities-generator/). - -## Step 4: Run the Test - -Execute the test using the following command. - -**Single test:** -``` js -npx nemo -P chrome -``` - -**Parallel tests:** -``` js -npx nemo -P chrome,chrome_2,firefox -``` - -## Step 5: View Your Results - -After running the test, view your results on the [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build). - -The dashboard provides: -- Video recordings of each test session -- Screenshots captured at each step -- Console logs from the browser -- Network logs for debugging -- Detailed command logs - -## Run NemoJS Tests Using Agent Skills - -Use AI coding assistants to generate and run NemoJS tests with the TestMu AI Agent Skill. - -The [nemojs-skill](https://github.com/LambdaTest/agent-skills/tree/main/nemojs-skill) is part of [TestMu AI Agent Skills](https://github.com/LambdaTest/agent-skills/) - structured packages that teach AI coding assistants how to write production-grade test automation. - -Install the skill: - -```bash -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/nemojs-skill .claude/skills/ - -# For Cursor / Copilot -cp -r agent-skills/nemojs-skill .cursor/skills/ -``` - -Install all available framework skills at once by cloning the repository directly into your tool's skills directory (e.g., `.claude/skills/`, `.cursor/skills/`). diff --git a/static/docs/selenium-three-capabilities.md b/static/docs/selenium-three-capabilities.md deleted file mode 100644 index 5df66b354..000000000 --- a/static/docs/selenium-three-capabilities.md +++ /dev/null @@ -1,31 +0,0 @@ -# Selenium Capabilities for Version 3 and Below - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Capabilities Reference - -Use these capabilities to configure browser, platform, and test metadata for Selenium version 3 and below. - -| Key | Values | Capability Description | Default Value | Alias | Value Type | Example | Possible Errors | -|-----|--------|------------------------|---------------|--------|------------|---------|-----------------| -| `browserName` | ChromeFirefoxSafariMS Edge - Microsoft EdgeOpera IE - Internet Explorer | This is a mandatory and case-sensitive capability that represents the name of the selected browser. `capabilities.setCapability("browserName", "Chrome");` | Mandatory |browser |String |`capabilities.setCapability("browserName","chrome");` |Case Sensitive Invalid browser error | -| `version` | Example: 101.0 (Beta) | This capability sets the version of the browser mentioned above. The version you provide as a desired capability must be valid. `capabilities.setCapability("version","78.0");`| -| `latest` | - | The latest keyword runs your test cases on the most recent browser version available at TestMu AI. `capabilities.setCapability("version","latest");` | -| `latest-N` | latest-5 | This allows you to execute your test on an older browser version. Here, N represents an integer. If you want to test your website on a browser version released 5 versions before the latest one, use the below capability: `capabilities.setCapability("version","latest-5");`| -| `platformName` | **Supported Windows:** Windows 11Windows 10Windows 8.1Windows 8Windows 7 **Supported macOS:** MontereyBig SurCatalinaMojaveHigh SierraSierraEl CapitanMavericksYosemite | If you do not declare a platform, TestMu AI maps your test to a relevant operating system for the selected browser. To run Selenium automation testing on Windows 11, add the below line of code in your Desired Selenium Capabilities class: ``` capabilities.setCapability("platformName", "Windows 11"); ``` For macOS Monterey, add the below line: ``` capabilities.setCapability("platformName", "macOS Monterey"); ``` |select on the basis of browser. For web Linuxwin10 |platform,OS |String |```capabilities.setCapability("platformName", "Windows 10");``` |Case Sensitive Invalid Platform Error | -| `resolution` | 1024x7681280x8001280x10241366x7681440x9001680x10501600x12001920x12001920x10802048x1536These values are OS Dependent, Verify from Capability generator String: "1920x1200" | This capability specifies a particular screen resolution for your VM (Virtual Machine). By default, the resolution is 1024x768. ``` capabilities.setCapability("resolution","1600x1200"); ``` |1920 x1080 |viewport |String |```capabilities.setCapability("resolution","1600x1200");``` |Case Sensitive Invalid Resolution error | -| `name` | Example: ToDo Sample Test name: "Login testcase"| Represents the name of a test. ``` capabilities.setCapability("name", "ToDo Sample Test"); ``` | test-id |testname,sessionname,test |String |```capabilities.setCapability("name", "ToDo Sample Test");``` |Case Sensitive > 255 char | -| `driver_version` | Example: "driver_version" : "99.0" | Specifies the driver version of the browser you want to run the test on. Provide a valid driver version corresponding to the browser used. ``` capabilities.setCapability("driver_version","99.0");| -| `build` | build: "Regression Job #567" Example: 5. | Represents the build number for your test. ` capabilities.setCapability("build", "5.1");` Note: Build string should be less than 255 characters. |Untitled Build |buildName,job,jobName |String |`capabilities.setCapability("build", "5.1");` Note: Build string should be less than 255 characters. |Case Sensitive > 255 char -| `projectName` |Example: My Test projectName: "Xaviar Project" |Represents the name of a project. ` capabilities.setCapability("projectName", "Test");` | Untitled |projectName,project |String |`caps.setCapability("projectName","Test");` |Case Sensitive > 255 char | -| tags | Example: {"Tag 1", "Tag 2", "Tag 3"} tags: {"Staging", "Critical, "Sanity"} | Groups your automation tests. `String[] tags = {"Tags"};``caps.setCapability("tags", tags);` For more details, visit our [documentation](/support/docs/group-tests-using-custom-tags/).| | |String |```String[] tags = {"Tags"};``` ```caps.setCapability("tags", tags);```For more details, visit our documentation. |Case Sensitive > 50 char | -| `buildTags` | Example: {"Build Tags"} buildTags: {"Build Tags","Staging"} | Groups your test builds. `String[] buildTagList = {"Build Tag"};` `caps.setCapability("buildTags", buildTagList);` For more details, visit our [documentation](/support/docs/group-and-filter-your-test-builds-using-build-tags/). | | |String |```String[] buildTagList = {"Build Tag"};``` ```caps.setCapability("buildTags", buildTagList);``` For more details, visit our documentation. -|`lambda:loadExtension`| Example: `https://automation-prod-user-files.s3.amazonaws.com/extensions/orgId-XXXX/2.1.0_0.zip` | Tests `.zip` extension files uploaded on TestMu AI storage. ```String[] extension = {"https://automation-prod-user-files.s3.amazonaws.com/extensions/orgId-XXXX/2.1.0_0.zip"};``````capabilities.setCapability("lambda:loadExtension", extension);```To upload an extension on TestMu AI storage, refer to our [documentation](/support/api-doc/#/extensions/UploadExtensions). | -|`lambda:userFiles`| Example:804402.png | Tests the uploaded files on TestMu AI. ``` capabilities.setCapability("lambda:userFiles", file); ``` For more details about uploading files, visit our [documentation](/support/docs/upload-files-using-testmu/). | -| -|`commandLog`| Example: true/false | Hides or shows the command logs. ``` capabilities.setCapability("commandLog", false); ``` | FALSE |commandLogs |Boolean | -|`systemLog`| Example: true/false | Hides or shows the Selenium/Appium logs. ``` capabilities.setCapability("systemLog", false); ``` | FALSE |seleniumLogs |Boolean | -|`network.http2` | network.http2: true | Enables HTTP2 support in Martian. Use this along with the network capability. | FALSE | |Boolean | -|`DisableXFHeaders`| DisableXFHeaders:true | Removes the X-Forwarded Headers added by Martian. | FALSE | | Boolean | -|`network.debug` | network.debug: true | Enables debug logging in Martian. Martian logs every request and response. | FALSE | | Boolean | -|`ignoreFfOptionsArgs` | ignoreFfOptionsArgs:true | Ignores args in moz:firefoxOptions. | FALSE | | Boolean | -|`updateBuildStatusOnSuccess` | true/false | Updates build status on replication of test name. | FALSE | | Boolean | diff --git a/static/docs/selenium-visual-regression.md b/static/docs/selenium-visual-regression.md deleted file mode 100644 index f76c6045f..000000000 --- a/static/docs/selenium-visual-regression.md +++ /dev/null @@ -1,649 +0,0 @@ -# Getting Started With Visual Regression Testing Using Selenium On SmartUI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Using the TestMu AI platform, perform regression testing in just one click and find Visual UI Regression bugs easily with the help of Smart Testing. This documentation will act as your step-by-step guide in performing successful Visual Regression tests. - -- Basic understanding of Selenium and WebDriver is required. -- Go to [`LambdaTest SmartUI`](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) and login along with your credentials. -- Access your Hub for your selenium **remote connection** instance at `hub.lambdatest.com/wd/hub`. -- Copy `LT_USERNAME` and `LT_ACCESS_KEY` credentials from `Access Key` button on the top right of the dashboard. - -```bash -export LT_USERNAME="YOUR_USERNAME" -export LT_ACCESS_KEY="YOUR ACCESS KEY" -``` - -```bash -set LT_USERNAME="YOUR_USERNAME" -set LT_ACCESS_KEY="YOUR ACCESS KEY" -``` - -```powershell -$env:LT_USERNAME="YOUR_USERNAME" -$env:LT_ACCESS_KEY="YOUR ACCESS KEY" -``` - -The following steps will guide you in running your first Visual Regression test on TestMu AI platform - - -### Step 1: Create a SmartUI Project - -The first step is to create a project with the application in which we will combine all your **builds** run on the project. -To create a SmartUI Project, follow these steps: - -1. Go to [Projects page](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) -2. Click on the `new project` button -3. Select the platform as Web for executing your `Selenium` tests. -4. Add name of the project, approvers for the changes found, tags for any filter or easy navigation. -5. Click on the **Submit**. - -### Step 2: Configure your test with Selenium Desired Capabilities - -Once you have created a SmartUI Project, you can generate screenshots by running automation scripts. Follow the below steps to successfully generate screenshots - - -1. Please clone the following sample Github repo (`https://github.com/LambdaTest/smartui-node-sample`). - -```bash -git clone https://github.com/LambdaTest/smartui-node-sample -``` - -2. Set up the TestMu AI credentials by following the instructions mentioned in the `README.md` file. -3. Edit the required capabilities for your test suite. - -```javascript title="Please replace your capabilities in your Remote Webdriver DesiredCapabilities configuration" -let capabilities = { -platform: "Windows 10", // Configure your OS for Selenium test -browserName: "chrome", // Configure your Browser for Selenium test -version: "latest", // Configure your Browser Version for Selenium test -visual: true, // Configure your Capture screenshot for Selenium test -name: "test session", // name of the test for Selenium -build: "Automation Build", // name of the build for Selenium -//highlight-start -"smartUI.project": "", // Replace the name of project with the new project name -"smartUI.build": "", // Replace the name of Build with the new Build name -"smartUI.baseline": false, // Enable if you want to update to a new baseline build -//highlight-end -}; - -// Connecting to the Lambdatest Selenium Cloud Grid with SmartUI -let gridUrl = -"https://" + -"" + -":" + -"" + -`hub.lambdatest.com/wd/hub`; - -// Here is your Remote WebDrive Connection -let driver = await new webdriver.Builder() -.usingServer(gridUrl) -.withCapabilities(capabilities) -.build(); -``` - -```python title="Please replace your capabilities in your Remote Webdriver DesiredCapabilities configuration" -options = ChromeOptions() -options.browser_version = "120.0" -options.platform_name = "Windows 10" -lt_options = {}; -lt_options["username"] = "USERNAME"; -lt_options["accessKey"] = "ACCESS_KEY"; -lt_options["project"] = "Untitled"; -#highlight-start -lt_options["smartUI.project"] = ""; -#highlight-end -lt_options["w3c"] = True; -lt_options["plugin"] = "python-python"; -options.set_capability('LT:Options', lt_options); - -options.set_capability('LT:Options', lt_options); - -#Connecting to the Lambdatest Selenium Cloud Grid with SmartUI -self.driver = webdriver.Remote( -command_executor= "https://" +"" +":" +"" + `hub.lambdatest.com/wd/hub`.format(username, access_key),options=options) -``` - -```ruby title="Please replace your capabilities in your Remote Webdriver DesiredCapabilities configuration" -options = Selenium::WebDriver::Options.chrome -options.browser_version = "120.0" -options.platform_name = "Windows 10" -lt_options = {}; -lt_options[:username] = "USERNAME"; -lt_options[:accessKey] = "ACCESS_KEY"; -lt_options[:project] = "Untitled"; -lt_options[:w3c] = true; -lt_options[:plugin] = "ruby-ruby"; -options.set_capability('LT:Options', lt_options); -#highlight-start -lt_options[:'smartUI.project']="SmartUI_Project_Name" -#highlight-end - -options.set_capability('LT:Options', lt_options); - -#Connecting to the Lambdatest Selenium Cloud Grid with SmartUI -self.driver = webdriver.Remote( -command_executor= "https://" +"" +":" +"" + `hub.lambdatest.com/wd/hub`.format(username, access_key),options=options) -``` - -```csharp title="Please replace your capabilities in your Remote Webdriver DesiredCapabilities configuration" -String LT_USERNAME = GetEnvironmentVariable("LT_USERNAME"); -String LT_ACCESS_KEY = GetEnvironmentVariable("LT_ACCESS_KEY"); -IWebDriver driver; -ChromeOptions capabilities = new ChromeOptions(); -capabilities.BrowserVersion = "latest"; -Dictionary ltOptions = new Dictionary(); -ltOptions.Add("username", LT_USERNAME); -ltOptions.Add("accessKey", LT_ACCESS_KEY); -ltOptions.Add("platformName", "Windows 10"); -ltOptions.Add("project", "Demo LT"); -ltOptions.Add("build", "C# Build"); -ltOptions.Add("sessionName", "C# Single Test"); -//highlight-start -ltOptions.Add("smartUI.project", ""); -//highlight-end -ltOptions.Add("w3c", true); -ltOptions.Add("plugin", "c#-c#"); -capabilities.AddAdditionalOption("LT:Options", ltOptions); - -// Connecting to the Lambdatest Selenium Cloud Grid with SmartUI -driver = new RemoteWebDriver(new Uri("https://hub.lambdatest.com/wd/hub/"), capabilities); -``` - -```java title="Please replace your capabilities in your Remote Webdriver DesiredCapabilities configuration" -ChromeOptions browserOptions = new ChromeOptions(); -browserOptions.setPlatformName("Windows 10"); -browserOptions.setBrowserVersion("120.0"); -HashMap ltOptions = new HashMap(); -ltOptions.put("username", ""); -ltOptions.put("accessKey", ""); -ltOptions.put("project", "Untitled"); -//highlight-start -ltOptions.put("smartUI.project", ""); -//highlight-end -ltOptions.put("w3c", true); -ltOptions.put("plugin", "java-java"); -browserOptions.setCapability("LT:Options", ltOptions); - -// Connecting to the Lambdatest Selenium Cloud Grid with SmartUI -driver = new RemoteWebDriver(new URL("https://" + username + ":" + accesskey + gridURL), capabilities); -``` - -- Installing dependency and for executing the test. - -``` -npm i && node -``` - -``` -pip install -r requirements.txt && python3 -``` - -``` -sudo gem install selenium-webdriver && ruby -``` - -``` -cd Packages nuget.exe install ..NUnitSeleniumpackages.config nmake clean build make -``` - -``` -mvn clean install && mvn test -P -``` - -Here you can explore multiple features and execute more such builds. - -## Step 3: Configure your Screenshot Capturing Options - -### For taking viewport screenshot - -This part of the code needs to be attached below the required segment of selenium script of which we would like to take the screenshot to test on. - -```javascript -let config = { -screenshotName: '' -}; -await driver.executeScript("smartui.takeScreenshot", config); -``` - -```python -config = { -'screenshotName': '' -} -driver.execute_script("smartui.takeScreenshot", config) -``` - -```ruby -config = { -'screenshotName' => '' -} -driver.execute_script("smartui.takeScreenshot", config) -``` - -```csharp -var config = new Dictionary { -{"screenshotName", ""} -}; -driver.ExecuteScript("smartui.takeScreenshot", config); -``` - -```java -Map config = new HashMap<>(); -config.put("screenshotName", ""); -((JavascriptExecutor)driver).executeScript("smartui.takeScreenshot", config); -``` - -### For capturing full page screenshot - -You can capture the full page screenshot for all different websites which have a scrolling functionality on their application user interface for your **Selenium** functional testing. - -Add the following **Webhook** to your test cases where we need to capture the screenshot of your application. - -```javascript -await driver.executeScript("smartui.takeScreenshot", { screenshotName: "", fullPage: true }); -``` - -```python -driver.execute_script("smartui.takeScreenshot", {"screenshotName": "", "fullPage": True}) -``` - -```ruby -driver.execute_script("smartui.takeScreenshot", {"screenshotName" => "", "fullPage" => true}) -``` - -```csharp -driver.ExecuteScript("smartui.takeScreenshot", new Dictionary { { "screenshotName", "" }, { "fullPage", true } }); -``` - -```java -Map config = new HashMap<>(); -config.put("screenshotName", ""); -config.put("fullPage", true); -((JavascriptExecutor)driver).executeScript("smartui.takeScreenshot", config); -``` - -The following feature is applicable for `Selenium >= 4.0.0 with Chrome` versions only. Please ensure that the `Lambdatest Capabilities` are updated in your project for executing full page screenshot(s). - -#### SmartScroll (For browsers other than Chrome) -For browsers other than chrome (Mozilla, Safari etc.) and for websites `Full Page Screenshot` fails, you can use smartscroll to take a full page screenshot of your web page by adding this hook in your test script. - -```javascript -await driver.executeScript("smartui.takeScreenshot", { screenshotName: "", fullPage: true, smartScroll: true }); -``` - -```python -driver.execute_script("smartui.takeScreenshot", {"screenshotName": "", "fullPage": True, "smartScroll": True}) -``` - -```ruby -driver.execute_script("smartui.takeScreenshot", {"screenshotName" => "", "fullPage" => true, "smartScroll" => true}) -``` - -```csharp -driver.ExecuteScript("smartui.takeScreenshot", new Dictionary { { "screenshotName", "" }, { "fullPage", true }, { "smartScroll", true } }); -``` - -```java -Map config = new HashMap<>(); -config.put("screenshotName", ""); -config.put("fullPage", true); -config.put("smartScroll", true); -((JavascriptExecutor)driver).executeScript("smartui.takeScreenshot", config); -``` - -### For capturing the screenshot of a specific element - -You can capture screenshots of targeted elements by leveraging various locator mechanisms such as XPath, CSS ID, class, and selectors. This precision-driven approach ensures accurate and specific visual regression testing for your web application. - -Add the following **Webhook** to your test cases where you need to capture the screenshot of a particular element on your UI. - -```javascript -let config = { -screenshotName: '', -elementType: 'xpath'/'css_selector'/'id'/'class', //Choose one from the following options - xpath, css_selector, id or class -element: '' //Add your required element here -await driver.executeScript("smartui.takeScreenshot", config); -``` - -```python -config = { -'screenshotName': '', -'elementType': 'xpath'/'css_selector'/'id'/'class', #Choose one from the following options - xpath, css_selector, id or class -'element': '' #Add your required element here -} -driver.execute_script("smartui.takeScreenshot", config) -``` - -```ruby -config = { -'screenshotName' => '', -'elementType'=> 'xpath'/'css_selector'/'id'/'class', #Choose one from the following options - xpath, css_selector, id or class -'element'=> '' #Add your required element here -} -driver.execute_script("smartui.takeScreenshot", config) -``` - -```csharp -var config = new Dictionary { -{"screenshotName", ""}, -{"elementType","xpath"/"css_selector"/"id"/"class"}, //Choose one from the following options - xpath, css_selector, id or class -{"element", ""} //Add your required element here -}; -driver.ExecuteScript("smartui.takeScreenshot", config); -``` - -```java -Map config = new HashMap<>(); -config.put("screenshotName", ""); -config.put("elementType", "xpath"/"css_selector"/"id"/"class"); //Choose one from the following options - xpath, css_selector, id or class -config.put("element", ""); //Add your required element here -((JavascriptExecutor)driver).executeScript("smartui.takeScreenshot", config); -``` - -## Step 4: View SmartUI Results - -You have successfully integrated SmartUI with your Selenium tests. Visit your [SmartUI project](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/projects) to view builds and compare snapshots between different test runs. - -You can see the SmartUI dashboard to view the results. This will help you identify the Mismatches from the existing `Baseline` build and do the required visual testing. - -## Fetching Screenshot Status and Results using WebHook - -You can fetch the status of the screenshot captured and the results of the screenshot captured using the following webhook. - -```javascript -driver.executeScript("smartui.fetchScreenshotStatus") -``` - -```python -driver.execute_script("smartui.fetchScreenshotStatus") -``` - -```ruby -driver.execute_script("smartui.fetchScreenshotStatus=") -``` - -```csharp -driver.ExecuteScript("smartui.fetchScreenshotStatus"); -``` - -```java -((JavascriptExecutor)driver).executeScript("smartui.fetchScreenshotStatus"); -``` - -For fetching results specific to a screenshot, you can use the following webhook by adding the `Screenshot Name` to the WebHook. - -```javascript -driver.executeScript("smartui.fetchScreenshotStatus=") -``` - -```python -driver.execute_script("smartui.fetchScreenshotStatus=") -``` - -```ruby -driver.execute_script("smartui.fetchScreenshotStatus=") -``` - -```csharp -driver.ExecuteScript("smartui.fetchScreenshotStatus="); -``` - -```java -((JavascriptExecutor)driver).executeScript("smartui.fetchScreenshotStatus="); -``` - -The following webhook will return the status of the screenshot captured. The status can be one of the following: - -```json -{ -screenshotsData: [ -{ -screenshotName: '', -screenshotURL: '', -screenshotStatus: 'Changes found', // Status of the screenshot -misMatchPercentage: 10.04, // Percentage of mismatch in the screenshot -threshold: 100, // Threshold set for the screenshot -browserName: 'chrome', // Browser used for capturing the screenshot -resolution: '1920x1080' // Resolution of the screenshot -} - -../Other Screenshots -], -buildId: '', -buildName: '', -projectName: '' -} -``` - -The following are the description of the parameters: - -| Parameter | Description | -| -------------------- | ------------------------------------------------------------- | -| `screenshotName` | Name of the screenshot for which the result is to be fetched. | -| `screenshotURL` | URL of the screenshot captured. | -| `screenshotStatus` | Status of the screenshot captured. | -| `misMatchPercentage` | Percentage of mismatch in the screenshot captured. | -| `threshold` | Threshold set for the screenshot captured. | -| `browserName` | Browser used for capturing the screenshot. | -| `resolution` | Resolution of the screenshot captured. | -| `buildId` | ID of the build in which the screenshot was captured. | -| `buildName` | Name of the build in which the screenshot was captured. | -| `projectName` | Name of the project in which the screenshot was captured. | - -**Please Note** - -The webhook will return the result of the screenshot captured only if the screenshot is processed and the result is available. - -This may take a few seconds to process the screenshot. Please ensure that you have added a delay before fetching the result of the screenshot. - -## Advanced Options for Screenshot Comparison - - **Build Configuration** - If you have multiple screenshots running the same test suite and want to run the comparison for the same test suite, want to add a build as a baseline from your test suite or need to access more SmartUI Build Config Options, click [here](/support/docs/smart-ui-build-options/). - - **Advanced Test Settings** - We offer multiple options for comparing the **Baseline** and the **Test Output** screenshots captured during your automation testing suites. To know more about our advanced P2P comparison options, click [here](/support/docs/test-settings-options/). - - **Handling Dynamic Data** - In case if you have any dynamic elements that are not in the same position across test runs, you can ignore or select a specific area to be removed from the comparison. For accessing such HTML DOM Config and Options, see [Handling Dynamic Data with DOM Configuration](/support/docs/html-dom-smartui-options). - -## Best Practices - -### Capability Configuration - -- Always set `visual: true` in your capabilities to enable SmartUI -- Use consistent project and build names across test runs -- Set meaningful test names for better organization - -**Example:** -```javascript -let capabilities = { -visual: true, -name: "Homepage Visual Test", -build: "Release 1.0", -"smartUI.project": "MyProject", -"smartUI.build": "Build-1.0" -}; -``` - -### Screenshot Timing - -- Wait for page elements to load before capturing screenshots -- Use explicit waits for dynamic content -- Consider page load time when setting up tests - -### Screenshot Naming - -- Use descriptive, consistent names -- Include context (page, component, state) in names -- Avoid special characters - -### Baseline Management - -- Establish baselines from stable builds -- Review and approve baselines before using -- Update baselines when intentional changes are made - -### Viewport Selection - -- Test on viewports that match your user base -- Include mobile, tablet, and desktop viewports -- Consider both portrait and landscape orientations - -### Viewport Selection - -- Test on viewports that match your user base -- Include mobile, tablet, and desktop viewports -- Consider both portrait and landscape orientations - -## Troubleshooting - -### Issue: Screenshots Not Captured - -**Symptoms**: Tests run but no screenshots appear in SmartUI dashboard - -**Possible Causes**: -- `visual: true` not set in capabilities -- Incorrect project name -- Network connectivity issues -- Credentials not set correctly - -**Solutions**: -1. Verify `visual: true` is set in capabilities: -```javascript -let capabilities = { -visual: true, // Must be set -// ... other capabilities -}; -``` - -2. Check project name matches exactly (case-sensitive): -```javascript -"smartUI.project": "ExactProjectName" -``` - -3. Verify credentials are set: -```bash -echo $LT_USERNAME -echo $LT_ACCESS_KEY -``` - -4. Check network connectivity to TestMu AI - -### Issue: "Project Not Found" Error - -**Symptoms**: Error indicating SmartUI project cannot be found - -**Possible Causes**: -- Project name typo or mismatch -- Project deleted -- Wrong account credentials - -**Solutions**: -1. Verify project exists in SmartUI dashboard -2. Copy project name directly from dashboard -3. Check credentials match the account with the project -4. Ensure project name is in capabilities, not just in dashboard - -### Issue: Screenshots Show Blank Pages - -**Symptoms**: Screenshots captured but show blank or incomplete content - -**Possible Causes**: -- Page not fully loaded -- JavaScript not executed -- Timing issues -- Viewport issues - -**Solutions**: -1. Add explicit waits before screenshot: -```javascript -await driver.wait(until.elementLocated(By.id('content')), 10000); -``` - -2. Wait for specific elements to be visible: -```javascript -await driver.wait(until.elementIsVisible(By.css('.main-content')), 10000); -``` - -3. Increase wait time for slow-loading pages - -4. Check viewport size matches expected dimensions - -### Issue: Build Name Conflicts - -**Symptoms**: Screenshots appear in wrong build or build name issues - -**Possible Causes**: -- Build name not set consistently -- Special characters in build name -- Build name conflicts - -**Solutions**: -1. Set build name in capabilities: -```javascript -"smartUI.build": "ConsistentBuildName" -``` - -2. Avoid special characters in build names - -3. Use consistent naming convention across team - -### Issue: Webhook Not Receiving Results - -**Symptoms**: Webhook configured but not receiving screenshot results - -**Possible Causes**: -- Webhook URL incorrect -- Screenshot not processed yet -- Network/firewall blocking webhook -- Delay too short - -**Solutions**: -1. Verify webhook URL is correct and accessible - -2. Add sufficient delay before fetching results: -```javascript -await new Promise(resolve => setTimeout(resolve, 10000)); // Wait 10 seconds -``` - -3. Check webhook endpoint is publicly accessible - -4. Verify webhook is receiving requests (check server logs) - -### Issue: Mismatch Percentage Unexpected - -**Symptoms**: Mismatch percentage higher or lower than expected - -**Possible Causes**: -- Threshold settings -- Dynamic content not ignored -- Rendering differences -- Baseline issues - -**Solutions**: -1. Review threshold settings in project settings - -2. Use `ignoreDOM` for dynamic content: -```javascript -"smartUI.options": { -"ignoreDOM": { -"id": ["timestamp", "user-id"] -} -} -``` - -3. Check baseline is correct and up-to-date - -4. Review comparison settings in project - -### Getting Help - -If you encounter issues not covered here: - -- Review [SmartUI Build Options](/support/docs/smart-ui-build-options) documentation -- Check [Advanced Test Settings](/support/docs/test-settings-options/) for comparison options -- Visit [TestMu AI Support](https://www.testmuai.com/support) for additional resources -- Contact support at support@testmuai.com or use [24/7 Chat Support](https://www.testmuai.com/support) - -## Additional Resources - -- [SmartUI Build Options](/support/docs/smart-ui-build-options) -- [Advanced Test Settings](/support/docs/test-settings-options/) -- [Handling Dynamic Data](/support/docs/smartui-handle-dynamic-data) -- [Project Settings](/support/docs/smartui-project-settings) diff --git a/static/docs/selenium-with-xunit.md b/static/docs/selenium-with-xunit.md deleted file mode 100644 index cdf162500..000000000 --- a/static/docs/selenium-with-xunit.md +++ /dev/null @@ -1,130 +0,0 @@ -# Selenium With xUnit - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Run xUnit Selenium tests on the TestMu AI cloud grid. This guide covers setup, running a sample test, configuring capabilities, and running tests in parallel. - -**Sample repo** -All the code used in this guide is available in the sample repository. - - View on GitHub - -## Prerequisites - -Make sure you have the following set up before you start. - -1. [Create a TestMu AI account](https://www.testmuai.com/register/) if you don't have one. -2. Get your **Username** and **Access Key** from the [TestMu AI Dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard). -3. Install [.NET Core SDK](https://dotnet.microsoft.com/download) 8.0 or later. -4. Download [Selenium WebDriver](https://www.selenium.dev/downloads/) Language Binding for C#. - -## Step 1: Clone the Sample Project - -Pull the sample repo to your local machine and navigate into the project directory. - -```bash -git clone https://github.com/LambdaTest/CSharp-xUnit-Selenium -cd CSharp-xUnit-Selenium -``` - -## Step 2: Set Your Credentials - -Add your TestMu AI credentials as environment variables so the test can authenticate with the grid. - -Visit the [TestMu AI Dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard), navigate to the left sidebar, and select **Credentials**. Copy your **Username** and **Access Key**, then set them as environment variables: - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -## Step 3: Configure Your Test Capabilities - -Define the browser, version, and OS for your test run. - -Update the `config.json` file with your desired capabilities: - -```csharp -{ -"server": "hub.lambdatest.com", -"user": "LT_USERNAME", -"key": "LT_ACCESS_KEY", - -"capabilities": { -"lt:options": { -"buildName": "xunit build", -"sessionName": "lambdatest xunit sample test", -"visual": "true", -"plugin": "xunit:sample" -} -}, - -"environments": [ -{ -"browserName": "chrome" -}, -{ -"browserName": "firefox" -}, -{ -"browserName": "safari" -} -], - -"TunnelOptions": { -"tunnel": false -} -} -``` - -Use the [Capabilities Generator](https://www.testmuai.com/capabilities-generator/) to auto-generate capabilities for any browser, version, and OS combination. - -## Step 4: Run the Test - -Build and run xUnit tests from Visual Studio or the command line. - -**From Visual Studio:** Build the solution and run tests from the Test Explorer. - -**From the terminal (macOS/Linux):** - -```bash -dotnet clean -dotnet test --filter "profile=single" -``` - -To run tests in parallel: - -```bash -dotnet test --filter "profile=parallel" -``` - -## Step 5: View Your Results - -Check the Automation Dashboard to see exactly what happened during your test. - -Visit the [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) to see your test results. Each session includes: - -- **Video recording** of the full test execution -- **Screenshots** captured at each step -- **Console logs** from the browser -- **Network logs** for every request and response -- **Selenium command logs** showing each driver action - -## Run xUnit Tests Using Agent Skills - -Use AI coding assistants to generate and run xUnit tests with the TestMu AI Agent Skill. - -The [xunit-skill](https://github.com/LambdaTest/agent-skills/tree/main/xunit-skill) is part of [TestMu AI Agent Skills](https://github.com/LambdaTest/agent-skills/) - structured packages that teach AI coding assistants how to write production-grade test automation. - -Install the skill: - -```bash -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/xunit-skill .claude/skills/ - -# For Cursor / Copilot -cp -r agent-skills/xunit-skill .cursor/skills/ -``` - -Install all available framework skills at once by cloning the repository directly into your tool's skills directory (e.g., `.claude/skills/`, `.cursor/skills/`). diff --git a/static/docs/semaphore-integration-with-hyperexecute.md b/static/docs/semaphore-integration-with-hyperexecute.md deleted file mode 100644 index 68c00fe04..000000000 --- a/static/docs/semaphore-integration-with-hyperexecute.md +++ /dev/null @@ -1,70 +0,0 @@ -# Semaphore Integration with HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Semaphore CI/CD is a cloud-based platform that helps engineering teams of all sizes build, test, and deploy their software faster and more reliably. It provides a powerful and flexible way to automate your entire CI/CD pipeline, from code builds to unit tests to production deployments. It is easy to get started with and use, even if you're new to CI/CD. It offers a wide range of features and integrations, so you can customize your pipelines to meet the specific needs of your team and project. - -This document will show you how to integrate Semaphore CI/CD with HyperExecute to greatly shorten your test cycles. - -## How to Integrate Semaphore with HyperExecute - -To integrate Semaphore Pipeline with HyperExecute, follow the below steps: - -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** -Download or Clone the code sample from the TestMu AI GitHub repository to run the tests on the HyperExecute. - - View on GitHub - -**Step 1:** Go to the [Semaphore CI/CD](https://semaphoreci.com/) website and login to the site using GitHub or Bitbucket. - -**Step 2:** Click on **Choose Repository** button. - -**Step 3:** Select the desired repository and click on **Next** button. - -**Step 4:** Click on **Continue to workflow setup** button. - -**Step 5:** Select the **Customize** button for custom setting in the workflow. - -**Step 6:** In the #Job1 add the below mentioned script. Make sure to replace the `username` and `key` with your TestMu AI [UserName and Access Key](/support/docs/hyperexecute-how-to-get-my-username-and-access-key/). - -```bash -curl -O https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute -chmod +x hyperexecute -./hyperexecute --user --key --config -``` - -Click on **Run the workflow** button. - -- Logs of the Pipeline executed. - -- Output in the [HyperExecute](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute/jobs) Dashboard - -## Sample Workflow File - -```bash -version: 1 - -image: ubuntu:latest # Adjust for macOS if needed - -stages: -- download # Descriptive stage name -- run-tests # Descriptive stage name - -jobs: -download: # Descriptive job name -stage: download -tasks: -- name: Download Hyperexecute CLI # Descriptive task name -command: wget https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute -- name: Make Hyperexecute executable # Descriptive task name -command: chmod u+x hyperexecute - -run-tests: # Descriptive job name -stage: run-tests -depends_on: download # Specify dependency on download job -tasks: -- name: Run Hyperexecute Tests # Descriptive task name -command: ./hyperexecute --user --key --config -``` diff --git a/static/docs/semaphore-integration-with-testmu.md b/static/docs/semaphore-integration-with-testmu.md deleted file mode 100644 index 522c77108..000000000 --- a/static/docs/semaphore-integration-with-testmu.md +++ /dev/null @@ -1,172 +0,0 @@ -# Semaphore CI Integration With TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Semaphore is a hosted continuous integration and deployment service used to test and deploy software projects hosted on GitHub and BitBucket. While open-source projects can use Semaphore for free in its full capacity, free use for private projects is limited to 100 builds per month. -TestMu AI now integrates with Semaphore CI to boost your go-to-market delivery. Perform automated cross browser testing hand in hand with your CI/CD pipeline and ensure your web-app renders seamlessly on more than 2000 real browsers and browser versions. Using TestMu AI Selenium Grid, you can fully automate your continuous testing process and trim down your test cycles significantly. - -## Prerequisites - -We will be performing a demonstration by using the Protractor framework - -* Global Dependencies - - * A [Git or GitHub](https://github.com/) repository - * Download and install node.js and node package manager or npm. - To install node.js with homebrew, use the below command. -``` -$ brew install node -``` - * If you have npm already installed, you may want to upgrade it to the latest version. Here the code you can run in your terminal to upgrade npm. -``` -npm install npm@latest -g -``` - -* TestMu AI Authentication Credentials - - * Be aware of your TestMu AI authentication credentials, i.e., your TestMu AI username, access key, and HubURL. You need to set them up as your environment variables. You can retrieve them from your TestMu AI automation dashboard by clicking on the key icon near the help button. -``` -$ export LT_USERNAME= -$ export LT_ACCESS_KEY= -``` - -## Integrating TestMu AI With Semaphore CI - -A few changes in the .semaphore/semaphore.yml are all you need to integrate TestMu AI With Semaphore CI. According to your test configurations, these changes would include replacing the environment variables such as access key, username, grid config, and so on. - -Here is a sample config file for integrating your TestMu AI account with your Semaphore CI instance. You can also find the below code at our [GitHub repository of Semaphore](https://github.com/LambdaTest/protractor-selenium-semaphore-sample). - -```js -version: v1.0 -name: Run Protractor Test In LambdaTest -agent: -machine: -type: e1-standard-2 -os_image: ubuntu1804 -execution_time_limit: -hours: 3 - -blocks: -- name: Set Env Variables -task: -jobs: -- name: NODE_ENV -commands: -- echo 'export LT_USERNAME="{your_lambdatest_username}"' >> ~/.bashrc -- echo 'export LT_ACCESS_KEY="{your_lambda_access_key}"' >> ~/.bashrc - -- name: Run Tests -task: -jobs: -- name: Run Protractor -commands: -- checkout -- cd /home/semaphore/protractor-selenium-semaphore-sample -- npm install -- node node_modules/protractor/bin/protractor single.conf.js -``` - -TestMu AI offers a Selenium [Desired Capabilities Generator](https://www.testmuai.com/capabilities-generator/) to fetch coded values for your desired test configurations. -For Instance, if you select the below configurations: - -| FIELDS | SELECTED VALUES | -|-----------|-------------------| -| Operating System | Windows 8.1 | -| Browser | Mozilla Firefox | -| Browser Version | 62.0 | -| Resolution | 1280x1024 | -| Selenium Version | 3.11.0 | - -Then Capabilities Generator at TestMu AI will provide you with the below program: - -``` -var capabilities = { -"build" : "your build name", //You can edit this and assign a build name -"name" : "your test name", // Assign a name to your Test -"platform" : "Windows 8.1", // The operating system on which you want to test your website -"browserName" : "Firefox", // The browser on which you want to test -"version" : "62.0", // The browser version which you've selected to perform the test upon -"resolution" : "1280x1024", // The resolution in which you want to run the test as per your operating system -"selenium_version" : "3.11.0", //The version of Selenium on which the test will run -"visual" : true, -"firefox.driver" : v0.21.0 -} -``` - ->For more information, refer to our documentation on [Selenium Automation Capabilities](/docs/selenium-automation-capabilities/). - -## Testing Locally Hosted or Privately Hosted Projects - -To help you perform cross browser testing of your locally stored web pages, TestMu AI provides an SSH(Secure Shell) tunnel connection with the name Lambda Tunnel. With Lambda Tunnel, you can execute a test server inside your Semaphore CI build container to perform automated cross browser testing on browsers offered by Selenium grid on TestMu AI. So you make sure how well your changes look, even before your customers. Curious to know more about Lambda Tunnel? - ->Follow our documentation on Lambda Tunnel to know it all. OS-specific instructions to download and set up tunnel binary are at the following links. -> ->- [Documentation For Windows User](/docs/local-testing-for-windows/) ->- [Documentation For Mac User](/docs/local-testing-for-macos/) ->- [Documentation For Linux User](/docs/local-testing-for-linux/) - ->**Download the binary file of:** -> ->- [Lambda Tunnel for Windows](https://downloads.lambdatest.com/tunnel/v3/windows/64bit/LT_Windows.zip) ->- [Lambda Tunnel for Mac](https://downloads.lambdatest.com/tunnel/v3/mac/64bit/LT_Mac.zip) ->- [Lambda Tunnel for Linux](https://downloads.lambdatest.com/tunnel/v3/linux/64bit/LT_Linux.zip) - -Now, let us take an example of Semaphore.yml for Lambda Tunnel. - -```js -//Example of .semaphore/semaphore.yml for Lambda Tunnel -version: v1.0 -name: Run Protractor Test In LambdaTest -agent: -machine: -type: e1-standard-2 -os_image: ubuntu1804 -execution_time_limit: -hours: 3 - -blocks: -- name: Set Env Variables -task: -jobs: -- name: NODE_ENV -commands: -- ls -al -- echo 'export LT_USERNAME=""' >> .bashrc -- echo 'export LT_ACCESS_KEY=""' >> .bashrc - -- name: Set Tunnel -task: -jobs: -- name: Download and install tunnel -commands: -- checkout -- cd /home/semaphore/protractor-selenium-semaphore-sample -- wget https://downloads.lambdatest.com/tunnel/v3/linux/64bit/LT_Linux.zip -#Extracting tunnel binary -- sudo apt-get install unzip -- unzip LT_Linux.zip -#Executing tunnel library -- ./LT -user ${LAMBDATEST_EMAIL} -key ${LAMBDATEST_KEY} & sleep 30 - -- name: Run Tests -task: -jobs: -- name: Run Protractor -commands: -- checkout -- cd /home/semaphore/protractor-selenium-semaphore-sample -- npm install -- node node_modules/protractor/bin/protractor single.conf.js -``` - -## Parallel Testing - -Parallel Testing is one of the most demanding features of TestMu AI Selenium Grid. By parallel testing, you can run more than one test case simultaneously. This means that Parallel testing would allow you to execute numerous automation test cases altogether. You perform a single test scenario across different browsers or run different test scenarios across the same browser but with different browser versions. - -To perform automation testing in parallel on Windows, use the below command. - -``` -node ./node_modules/protractor/bin/protractor parallel.conf.js -``` - -Monitor and analyze your test result on the [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/). Reliably deploy your code at scale using Semaphore CI integration with TestMu AI, and ensure it looks robust across every browser to provide a seamless user experience to all your visitors. Happy Testing! diff --git a/static/docs/serenity-test-on-selenium-grid.md b/static/docs/serenity-test-on-selenium-grid.md deleted file mode 100644 index cfff5ad64..000000000 --- a/static/docs/serenity-test-on-selenium-grid.md +++ /dev/null @@ -1,224 +0,0 @@ -# Selenium With Serenity BDD - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Run Serenity BDD tests on the TestMu AI cloud grid. This guide covers setup, running a sample test, configuring capabilities, and testing locally hosted pages. - -**Sample repo** -All the code used in this guide is available in the sample repository. - - View on GitHub - -## Prerequisites - -Make sure you have the following set up before you start. - -1. [Create a TestMu AI account](https://www.testmuai.com/register/) if you don't have one. -2. Get your **Username** and **Access Key** from the [TestMu AI Dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard). -3. Install the **Java development environment** (Java 11 recommended). -4. Install **Maven**. Download it from [the official website](https://maven.apache.org/) or install it on **Linux/MacOS** using [**Homebrew**](https://brew.sh/). - -## Step 1: Clone the Sample Project - -Pull the sample repo to your local machine and navigate into the project directory. - -```bash -git clone https://github.com/LambdaTest/Serenity-Selenium-Sample -cd Serenity-Selenium-Sample -``` - -You may also want to run the command below to check for outdated dependencies. - -```bash -mvn versions:display-dependency-updates -``` - -## Step 2: Set Your Credentials - -Add your TestMu AI credentials as environment variables so the test can authenticate with the grid. - -Visit the [TestMu AI Dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard), navigate to the left sidebar, and select **Credentials**. Copy your **Username** and **Access Key**, then set them as environment variables: - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -## Step 3: Configure Your Test Capabilities - -Define the browser, version, and OS for your test run. - -The sample feature file checks for the word "LambdaTest" on Google and validates the title of the resultant page: - -```bash -Feature: Google's Search Functionality -Scenario: Can find search results -When I type query as "LambdaTest" -And I submit -Then I should see title "LambdaTest - Google Search" -``` - -Below is the `GooglePage.java` file for the above test case scenario: - -```java title="GooglePage.java" -package com.lambdatest.cucumber.pages; - -import static org.assertj.core.api.Assertions.assertThat; - -import org.openqa.selenium.support.FindBy; - -import net.serenitybdd.core.pages.WebElementFacade; -import net.thucydides.core.annotations.DefaultUrl; -import net.thucydides.core.pages.PageObject; - -@DefaultUrl("https://www.google.com/ncr") -public class GooglePage extends PageObject { - -@FindBy(name = "q") -WebElementFacade search; - -@FindBy(name = "btnK") -WebElementFacade searchButton; - -public void searchForString(String searchString) { -search.sendKeys(searchString); -} -public void submitForm() throws Exception { -searchButton.click(); -Thread.sleep(5000); -} - -public void titleShouldMatch(String matchTitle) { -assertThat(this.getTitle()).containsIgnoringCase(matchTitle); -} -} -``` - -Below is the `LambdaTestSerenityDriver.java` file that shows the integration of Serenity with TestMu AI: - -```java title="LambdaTestSerenityDriver.java" -package com.lambdatest; - -import java.net.URL; -import java.util.HashMap; -import java.util.Iterator; - -import org.openqa.selenium.WebDriver; -import org.openqa.selenium.chrome.ChromeOptions; -import org.openqa.selenium.remote.RemoteWebDriver; - -import net.thucydides.core.util.EnvironmentVariables; -import net.thucydides.core.util.SystemEnvironmentVariables; -import net.thucydides.core.webdriver.DriverSource; - -public class LambdaTestSerenityDriver implements DriverSource { - -public WebDriver newDriver() { -EnvironmentVariables environmentVariables = SystemEnvironmentVariables.createEnvironmentVariables(); - -String username = System.getenv("LT_USERNAME"); -if (username == null) { -username = (String) environmentVariables.getProperty("lt.user"); -} - -String accessKey = System.getenv("LT_ACCESS_KEY"); -if (accessKey == null) { -accessKey = (String) environmentVariables.getProperty("lt.key"); -} - -String environment = System.getProperty("environment"); -ChromeOptions browserOptions = new ChromeOptions(); -HashMap ltOptions = new HashMap(); -ltOptions.put("plugin", "Serenity LambdaTest Plugin"); -ltOptions.put("w3c", true); - -Iterator it = environmentVariables.getKeys().iterator(); -while (it.hasNext()) { -String key = (String) it.next(); - -if (key.equals("lt.user") || key.equals("lt.key") || key.equals("lt.grid")) { -continue; -} else if (key.startsWith("lt_")) { -ltOptions.put(key.replace("lt_", ""), environmentVariables.getProperty(key)); - -} else if (environment != null && key.startsWith("environment." + environment)) { - -ltOptions.put(key.replace("environment." + environment + ".", ""), -environmentVariables.getProperty(key)); -} -} -browserOptions.setCapability("LT:Options", ltOptions); - -try { -String url = "https://" + username + ":" + accessKey + "@" + environmentVariables.getProperty("lt.grid") -+ "/wd/hub"; -return new RemoteWebDriver(new URL(url), browserOptions); -} catch (Exception e) { -System.out.println(e); -return null; -} -} - -public boolean takesScreenshots() { -return false; -} -} -``` - -Use the [Capabilities Generator](https://www.testmuai.com/capabilities-generator/) to auto-generate capabilities for any browser, version, and OS combination. - -## Step 4: Run the Test - -Trigger the test from your terminal. - -```bash -mvn verify -P single -``` - -```bash -mvn verify -P parallel -``` - -For parallel execution, the sample project runs **single.feature** test case in four different environments: Chrome, Firefox, IE, and Safari. - -```java title="ParallelChromeTest.java" -//Running Parallel Test On Chrome - -@RunWith(CucumberWithSerenity.class) -@CucumberOptions(features = "src/test/resources/features/single.feature") -public class ParallelChromeTest extends LambdaTestSerenityTest { -} -``` - -Define similar classes for the remaining browsers. - -## Step 5: View Your Results - -Check the Automation Dashboard to see exactly what happened during your test. - -Visit the [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) to see your test results. Each session includes: - -- **Video recording** of the full test execution -- **Screenshots** captured at each step -- **Console logs** from the browser -- **Network logs** for every request and response -- **Selenium command logs** showing each driver action - -## Run Serenity BDD Tests Using Agent Skills - -Use AI coding assistants to generate and run Serenity BDD tests with the TestMu AI Agent Skill. - -The [serenity-bdd-skill](https://github.com/LambdaTest/agent-skills/tree/main/serenity-bdd-skill) is part of [TestMu AI Agent Skills](https://github.com/LambdaTest/agent-skills/) - structured packages that teach AI coding assistants how to write production-grade test automation. - -Install the skill: - -```bash -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/serenity-bdd-skill .claude/skills/ - -# For Cursor / Copilot -cp -r agent-skills/serenity-bdd-skill .cursor/skills/ -``` - -Install all available framework skills at once by cloning the repository directly into your tool's skills directory (e.g., `.claude/skills/`, `.cursor/skills/`). diff --git a/static/docs/service-accounts.md b/static/docs/service-accounts.md deleted file mode 100644 index 9708d3f56..000000000 --- a/static/docs/service-accounts.md +++ /dev/null @@ -1,46 +0,0 @@ -# Creating and Managing Service Accounts - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Service accounts are a distinct type of user that you can use with command line interfaces. They are primarily used to perform tests through pipelines like Jenkins or GitHub Actions. Service accounts are not granted login or dashboard viewing privileges. Instead, they are helpful in monitoring and reporting test execution at the user level. - -The service account has the following benefits: - -* Run tests via CI/CD pipelines like Jenkins, GitHub Actions, etc. - -* Track and report of test execution at the user level. - -* User Access Key doesn't need to be shared across the organization. - -## Creating a Service Account - -**Note** -Service accounts can only be created by Admins. - -To create a service account on TestMu AI, you can follow these steps: - -**Step 1:** Sign in to your TestMu AI account. Don't have an account, register for free. - -**Step 2:** Head to **Settings** and select **Organization Settings** from the dropdown. - -**Step 3:** Under the **Members** tab, click on **Service Accounts**. - -**Step 4:** Click on **Create**. - -**Step 5:** Enter your preferred **Service Account Name** and click **Create**. - -**Step 6:** After creating your account, you will be presented with your **Service Account Credentials**. You can use these credentials to login to your service account. - ->*You can also find service account credentials from the service account dashboard. Just click on the three dots menu and select **View Username & Access Token**.* - -Once you have created the service account, it will appear under the **Service Accounts** tab. - -## Deleting Your Service Account - -In case, you wish to delete your service account, follow the given below steps: - -1. Click on the three dots menu and click **Delete**. - -2. A pop-up box will show up. Click **Delete**. - -Your service account associated with you will be deleted. diff --git a/static/docs/servicenow-integration.md b/static/docs/servicenow-integration.md deleted file mode 100644 index 10f74713b..000000000 --- a/static/docs/servicenow-integration.md +++ /dev/null @@ -1,84 +0,0 @@ -# ServiceNow Integration with TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -ServiceNow is a cloud-based platform for IT service management and automating business processes. Integrating ServiceNow with TestMu AI enables you to log incidents directly from your testing sessions into ServiceNow, streamlining your workflow and enhancing team collaboration. - -> ServiceNow Integration with TestMu AI is available for both freemium and premium plans. - -### Create an OAuth Application for TestMu AI in ServiceNow - -> Only an administrator of your ServiceNow instance can create the OAuth application required for integration. - -To enable integration, you need to create an OAuth application in your ServiceNow instance for TestMu AI: - -1. **Log in to your ServiceNow instance** as an administrator. -2. Go to **System OAuth > Application Registry** in the left navigation pane. - -3. Click **New** and select **Create an OAuth API endpoint for external clients**. -4. Fill in the required fields: - - **Name:** Enter a name (e.g., TestMu AI Integration). - - **Client ID:** (Optional) Leave blank to auto-generate or specify your own. - - **Client Secret:** (Optional) Leave blank to auto-generate or specify your own. - - **Redirect URL:** https://integrations.lambdatest.com/servicenow/auth/callback - - **Logo URL:** https://assets.testmuai.com/resources/images/logos/logo.svg - -5. Click **Submit** to create the application. -6. After creation, copy the **Client ID** and **Client Secret**. You will need these values to complete the integration in TestMu AI. - -Once the OAuth application is set up, proceed with the integration steps below. - -## Integrate ServiceNow from your TestMu AI Account - -**Step 1:** Log in to your [TestMu AI](https://www.testmuai.com/login/) account. Ensure you have Admin or User level access to manage integrations. - -**Step 2:** Navigate to `Settings` -> `Integrations` -> `Bug Tracker`. - -**Step 3:** Click on the **Connect** button of the ServiceNow block. - -**Step 4:** In the integration setup form, enter the following details: - -- **ServiceNow Instance URL:** The URL of your ServiceNow instance (e.g., `https://your-instance.service-now.com`). -- **Client ID:** The Client ID from your ServiceNow OAuth application. -- **Client Secret:** The Client Secret from your ServiceNow OAuth application. - -After filling in the details, click on the **Install** button to establish the integration. - -## Log your First Incident - -**Step 1:** Ensure you have access to the appropriate ServiceNow instance and the Incident table is available. - -**Step 2:** While working on TestMu AI, if you encounter an incident, click on the **Mark as Bug** button available on different products. - -**Step 3:** In the incident logging form, provide the necessary details such as: - -- **User:** The user in ServiceNow to whom the incident should be assigned. -- **Short Description:** A concise summary of the incident. -- **Description:** Detailed information about the incident. -- **Urgency:** Select the urgency level for the incident. -- **Priority:** Select the priority for the incident. -- **Due Date:** The date by which the incident should be resolved. - -**Step 4:** Click on the **Create Issue** button. The incident will be logged into your ServiceNow instance, and you can view it directly from your ServiceNow dashboard. - -## Resync ServiceNow Integration - -To ensure that the latest users from your ServiceNow system are available in TestMu AI, you can resync the integration. - -**Step 1:** Navigate to `Settings` -> `Integrations` in your TestMu AI account. - -**Step 2:** Locate the ServiceNow integration and click on the **Resync** button. - -- This will fetch the latest users from your ServiceNow system. - -- If your refresh token has expired, you will be prompted to reauthorize your ServiceNow integration. - -Once resynced, the latest data will be available for use. - -## Uninstall ServiceNow Integration - -**Step 1:** Navigate to `Settings` -> `Integrations` in your TestMu AI account. - -**Step 2:** Locate the ServiceNow integration and click on the **Remove** button to uninstall the integration. - -> That was all you need to know for TestMu AI + ServiceNow Integration. Increase your productivity with our integrations. If you still have any questions for us, please feel free to let us know. Our experts are always available on **chat** to help you out with any roadblock. diff --git a/static/docs/set-date-time-hour-format-real-devices-browser.md b/static/docs/set-date-time-hour-format-real-devices-browser.md deleted file mode 100644 index 3748300b7..000000000 --- a/static/docs/set-date-time-hour-format-real-devices-browser.md +++ /dev/null @@ -1,67 +0,0 @@ -# Set Custom Date, Time & Hour Format on Real Devices (Browser) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Testing applications often requires validation across different date, time, and format scenarios. TestMu AI now supports dynamic configuration of **date, time, 12/24-hour format**, and **automatic time toggle** on real **iOS (14+)** and **Android (10+)** devices, both in **manual** sessions and via **Appium automation hooks**. - -This enables use cases like testing scheduled alerts, AM/PM behaviors, time-bound features, and regional formatting validations. - -## Access Date & Time Settings on TestMu AI Real Devices - -**Step 1:** Log into your TestMu AI dashboard and navigate to **Real Devices** > **App Testing**. - -**Step 2:** Select your desired app and a supported device (iOS 14+ or Android 10+), then click **Start** to launch your session. - -**Step 3:** Once the session starts, open the **iOS Settings** or **Android Settings** tab in the left sidebar, based on the device platform. - -**Step 4:** Click on **Set Date and Time** to open the configuration modal. - -**Step 5:** In the **modal**, configure the **date**, **time**, and **time format** as needed, then click **Update** to apply the changes to the device. - -On Android devices, certain models, particularly those from **Motorola, Xiaomi, Oppo, and other Chinese OEMs**, do not support custom date and time configuration. On such devices, the modal will display a **Not Supported** message. - -## Date & Time Configuration Options - -The modal includes four options to simulate various datetime-related behaviors: - -### 1. Set Date and Time Automatically -- **Toggle ON:** Syncs the device with network time. -- **Toggle OFF:** Unlocks manual controls for custom configuration. -- Disabling this is mandatory to manually edit date or time. - -### 2. Date -- Opens a calendar picker with selectable dates up to **7 days ahead**. -- Selecting a date updates the system date on the device. -- Past dates and dates beyond 7 days are **grayed out** and unselectable. -- An **Apply** button appears only when a valid date is selected. - -### 3. Time -- Allows custom time entry in `HH:MM:SS` format. -- Picker adjusts based on selected hour-format (12 or 24-hour). -- Supports both **manual typing** and **arrow key navigation** for time input. - -### 4. Time Format (12/24 Hour) -- Choose between **12-hour** (AM/PM toggle) and **24-hour** formats. -- 12-hour format shows AM/PM option in time picker. -- 24-hour format disables AM/PM selection automatically. - -## Supported Platforms: - -| Platform | Configuration Methods | OS Versions Supported | -| -------- | --------------------------- | --------------------- | -| iOS | Manual + Appium Executor | iOS 14 and above | -| Android | Manual + Appium Executor | Android 10 and above | - -Custom date and time configuration is not supported on certain Android device models, particularly those from **Motorola, Xiaomi, Oppo, and other Chinese OEMs**. On these devices, the modal will display a **Not Supported** message during manual sessions, and the Appium hook will return the following error during automation: - -`Custom date and time hook is not supported on this device. Please try on another device_id.` - -## Use Cases: - -- Test scheduled notifications and alerts -- Validate time-sensitive app flows -- Check 12/24-hour format compatibility -- Simulate future or custom dates -- Debug calendar and time-based modules -- Verify auto/manual time toggle behavior -- Test date/time edge cases (e.g., midnight, EOM) diff --git a/static/docs/set-date-time-hour-format-real-devices.md b/static/docs/set-date-time-hour-format-real-devices.md deleted file mode 100644 index 1e1f6f249..000000000 --- a/static/docs/set-date-time-hour-format-real-devices.md +++ /dev/null @@ -1,67 +0,0 @@ -# Set Custom Date, Time & Hour Format on Real Devices - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Testing applications often requires validation across different date, time, and format scenarios. TestMu AI now supports dynamic configuration of **date, time, 12/24-hour format**, and **automatic time toggle** on real **iOS (14+)** and **Android (10+)** devices, both in **manual** sessions and via **Appium automation hooks**. - -This enables use cases like testing scheduled alerts, AM/PM behaviors, time-bound features, and regional formatting validations. - -## Access Date & Time Settings on TestMu AI Real Devices - -**Step 1:** Log into your TestMu AI dashboard and navigate to **Real Devices** > **App Testing**. - -**Step 2:** Select your desired app and a supported device (iOS 14+ or Android 10+), then click **Start** to launch your session. - -**Step 3:** Once the session starts, open the **iOS Settings** or **Android Settings** tab in the left sidebar, based on the device platform. - -**Step 4:** Click on **Set Date and Time** to open the configuration modal. - -**Step 5:** In the **modal**, configure the **date**, **time**, and **time format** as needed, then click **Update** to apply the changes to the device. - -On Android devices, certain models, particularly those from **Motorola, Xiaomi, Oppo, and other Chinese OEMs**, do not support custom date and time configuration. On such devices, the modal will display a **Not Supported** message. - -## Date & Time Configuration Options - -The modal includes four options to simulate various datetime-related behaviors: - -### 1. Set Date and Time Automatically -- **Toggle ON:** Syncs the device with network time. -- **Toggle OFF:** Unlocks manual controls for custom configuration. -- Disabling this is mandatory to manually edit date or time. - -### 2. Date -- Opens a calendar picker with selectable dates up to **7 days ahead**. -- Selecting a date updates the system date on the device. -- Past dates and dates beyond 7 days are **grayed out** and unselectable. -- An **Apply** button appears only when a valid date is selected. - -### 3. Time -- Allows custom time entry in `HH:MM:SS` format. -- Picker adjusts based on selected hour-format (12 or 24-hour). -- Supports both **manual typing** and **arrow key navigation** for time input. - -### 4. Time Format (12/24 Hour) -- Choose between **12-hour** (AM/PM toggle) and **24-hour** formats. -- 12-hour format shows AM/PM option in time picker. -- 24-hour format disables AM/PM selection automatically. - -## Supported Platforms: - -| Platform | Configuration Methods | OS Versions Supported | -| -------- | --------------------------- | --------------------- | -| iOS | Manual + Appium Executor | iOS 14 and above | -| Android | Manual + Appium Executor | Android 10 and above | - -Custom date and time configuration is not supported on certain Android device models, particularly those from **Motorola, Xiaomi, Oppo, and other Chinese OEMs**. On these devices, the modal will display a **Not Supported** message during manual sessions, and the Appium hook will return the following error during automation: - -`Custom date and time hook is not supported on this device. Please try on another device_id.` - -## Use Cases: - -- Test scheduled notifications and alerts -- Validate time-sensitive app flows -- Check 12/24-hour format compatibility -- Simulate future or custom dates -- Debug calendar and time-based modules -- Verify auto/manual time toggle behavior -- Test date/time edge cases (e.g., midnight, EOM) diff --git a/static/docs/set-device-dark-mode.md b/static/docs/set-device-dark-mode.md deleted file mode 100644 index a359fbd0e..000000000 --- a/static/docs/set-device-dark-mode.md +++ /dev/null @@ -1,87 +0,0 @@ -# Set Device Dark Mode - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI now allows you to **enable or disable dark mode** for automation testing as well. This helps developers and testers validate UI behavior across light and dark themes, theme switching without manual intervention, and visual consistency for mobile apps and web applications. - -## Use Cases - -### Mobile App Testing - -- Ensure UI elements render correctly in dark mode -- Check text contrast and readability -- Verify icon and image visibility in dark backgrounds -- Confirm theme consistency across screens - -### Web Testing - -- Test CSS media queries for `prefers-color-scheme: dark` -- Validate dark theme implementations in your app -- Check browser-level dark mode support -- Review responsiveness across both themes - - This feature works for both Virtual and Real Devices. - -## Supported Platforms - -| Platform | Version | -|----------|----------------| -| iOS | 13 and above | -| Android | 11 and above | - -### Enabling Dark Mode via Capabilities - -You can enable the dark mode setting through `darkMode` capability before the session starts. This ensures the device starts in the dark theme without additional steps during test execution. - -```json -{ -"deviceName":"iPhone 16", -"platformName":"ios", -"platformVersion":"18", -"isRealMobile":True, -"app":"YOUR_APP_URL", -#highlight-next-line -"darkMode": true -} -``` - -```json -{ -"deviceName":"Galaxy S20", -"platformName":"Android", -"platformVersion":"10", -"isRealMobile":True, -"app":"YOUR_APP_URL", -#highlight-next-line -"darkMode": true -} -``` - -- You must add the generated **APP_URL** to the `app` capability in the config file. -- You can generate capabilities for your test requirements with the help of our inbuilt [**Capabilities Generator tool**](https://www.testmuai.com/capabilities-generator/).For more details, please refer to our guide on [**Desired Capabilities in Appium**](/support/docs/desired-capabilities-in-appium/). - -### Enabling Dark Mode via Hooks - -To change dark mode settings during test execution, use the hook with the `updateDeviceSettings` action. The example below demonstrates this using **Python**. - -```python -driver.execute_script('lambda_executor: { -"action": "updateDeviceSettings", -"arguments": { -"darkMode" : "on" -} -}') -``` - -```python -driver.execute_script('lambdatest_executor: { -"action": "updateDeviceSettings", -"arguments": { -"DarkMode": "true" -} -}') -``` - -- Dark mode changes are applied at the system level and may require app restarts for full effect -- Some legacy applications may not fully support dark mode theming -- Web applications must implement their own dark mode CSS for the setting to take effect beyond browser UI diff --git a/static/docs/settings-and-security.md b/static/docs/settings-and-security.md deleted file mode 100644 index 965f95fc2..000000000 --- a/static/docs/settings-and-security.md +++ /dev/null @@ -1,38 +0,0 @@ -# Settings and Security - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This page empowers you to manage your account's security settings, ensuring the confidentiality and integrity of your data. - -## Key Features: - -- Password Management: -- Change your account password. -- Enable two-factor authentication (2FA) for added security. -- Integration Management: -- View and manage third-party integrations connected to your account. -- Revoke access for specific integrations if needed. -- Session Management: -- View active and recent testing sessions. -- Terminate active sessions remotely if necessary. -- API Key Management: -- Generate and manage API keys for programmatic access to TestMu AI features. -- Revoke API keys to restrict access. -- Data Privacy: -- Access and download your personal data. -- Request data deletion, if applicable. - -## Additional Information: - -> Refer to TestMu AI's Terms of Service and Privacy Policy: for detailed information on data handling and security practices. - -Contact TestMu AI Support: for assistance with any security-related concerns or inquiries. - -## Best Practices: - -- Use strong passwords: that are unique to your TestMu AI account. -- Enable 2FA: for an extra layer of protection. -- Regularly review your integrations: and revoke access for those that are no longer needed. -- Monitor active sessions: and terminate any suspicious activity. -- Securely store API keys: and avoid sharing them with unauthorized individuals. -- Stay informed: about TestMu AI's security updates and best practices. diff --git a/static/docs/setup-pre-run-executable.md b/static/docs/setup-pre-run-executable.md deleted file mode 100644 index 9232c0e1d..000000000 --- a/static/docs/setup-pre-run-executable.md +++ /dev/null @@ -1,127 +0,0 @@ -# Install Custom Certificates on the VM Through Pre-run Executables - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Pre-run executables let you run custom scripts before the Selenium test starts. Use this feature to install user custom certificates that are required for test cases to execute successfully. - -To use certificates for Selenium tests, you need 3 things: - -* Certificate file. -* The script that installs the certificate in the Virtual Machine (VM). -* The script that deletes the certificate in the VM. - -## Instructions - -Upload your certificate file, installation, and uninstallation script using the below steps. - -1. Upload your certificate using the below command. - -```bash -curl --location --request POST '' -``` -2. Install the certificate on macOS using the following command. - -```bash -sudo security add-trusted-cert -d -r trustRoot -k /Library/Keychains/System.keychain /Users/ltuser/Downloads/{CERTIFICATE-FILE-NAME} -``` -3. Install the certificate on Windows using the below command. - -```bash -Import-Certificate -FilePath "C:\Users\ltuser\Downloads\{NAME-OF-THE-CERTIFICATE}" -CertStoreLocation 'Cert:\LocalMachine\Root' -Verbose -``` - -4. Use the following cURL request to upload pre and post run files altogether. - -```bash -curl --request POST \ ---url \ ---header 'Authorization: Basic {TOKEN}' \ ---header 'Content-Type: multipart/form-data' \ ---header 'cache-control: no-cache,no-cache,no-cache' \ ---header 'content-type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW' \ ---form pre_run_file=@{PATH-TO-CERTIFIFICATE-INSTALLATION-SCRIPT} \ ---form post_run_file=@{PATH-TO-CERTIFICATE-DELETION-SCRIPT} \ ---form 'name={NAME-OF-YOUR-PRERUN}' -``` - -5. Run the below capabilities to use the certificate, pre, and post run files in a test. - -```bash -"lambda:userFiles": ["certificate.p12"], -"prerun": { -"accessKey": "****", -"background": False, -"filePath": "/uploads/install_certificate.ps1", -"postAction": { -"filePath": "xyz/post/delete_certificate.ps1", -"url": "" -}, -"url": "", -"user": "****" -}, -``` - -> **Note:** Replace `{TOKEN}`, `{PATH-TO-CERTIFICATE-FILE}`, `{PATH-TO-CERTIFICATE-INSTALLATION-SCRIPT}`, `{PATH-TO-CERTIFICATE-DELETION-SCRIPT}`, `{NAME-OF-YOUR-PRERUN}` with the appropriate values. - -6. After uploading your pre-run script, reach out to us via chat support or email to get the scripts approved. - -7. Once your script is approved, pass the `prerun` key in the capability as follows: - -```bash -"prerun": { -"url": "{NAME-OF-YOUR-PRERUN}/pre/{NAME-OF-YOUR-CERTIFICATE-INSTALLATION-SCRIPT}", -}, -``` -> **Note:** Replace `{NAME-OF-YOUR-PRERUN}` as provided in **Step 1** and `{NAME-OF-YOUR-CERTIFICATE-INSTALLATION-FILE-NAME}` with the name of the certificate installation script that you uploaded in **Step 1**. - -## Leverage the Use of APIs to Pre-run Tests - -Use the following prerun APIs to manage pre-run files. - -* **GET/`files`:** Fetch all pre run files uploaded by the user. -* **POST/`files`:** Upload pre run executable file to lambda storage. -* **DELETE/`files`/`delete`:** Delete pre run from lambda storage. -* **POST/`files`/`validate`:** Check if the file is approved by TestMu AI. -* **PUT/`files`/`download`:** Download pre run executable file. - -> **Note:** To learn more about our APIs, check out our [API Documentation](https://www.testmuai.com/support/api-doc/). - -## Sample Certificate Installation and Uninstallation File (macOS) - -Replace the placeholder values in each script file before use. - -* In `install_certificate.sh` replace `{CERTIFICATE-FILE-NAME}` with the certificate file name. -* In `delete_certificate.sh` replace `{CERTIFICATE-NAME}` with the certificate name (use **Keychain Access** to find the certificate name). - -Following are the contents of the sample file: - -**For Installing:** -```bash -sudo security add-trusted-cert -d -r trustRoot -k /Library/Keychains/System.keychain /Users/ltuser/Downloads/{CERTIFICATE-FILE-NAME} -``` - -**For Deleting:** -```bash -sudo security delete-certificate -c "Local Certificate" -``` - -## Sample Certificate Installation and Uninstallation File (Windows) - -Replace the placeholder values in each script file before use. - -* In `install_certificate.ps1` replace `{CERTIFICATE-FILE-NAME}` with the certificate file name. -* In `delete_certificate.ps1` replace `{CERTIFICATE-NAME}` with the certificate name. - -Following are the contents of the sample file: - -**For Installing:** -```bash -Import-Certificate -FilePath "C:\Users\ltuser\Downloads\{NAME-OF-THE-CERTIFICATE}" -CertStoreLocation 'Cert:\LocalMachine\Root' -Verbose -``` -**For Deleting:** -```bash -Get-ChildItem Cert:\CurrentUser\Root\{THUMBPRINT-OF-THE-CERTIFICATE} | Remove-Item -``` - ->Got any questions? -Please reach out at our **24x7 Chat Support** or you could also mail us at support@testmuai.com. diff --git a/static/docs/sharding-espresso-rd-hyperexecute.md b/static/docs/sharding-espresso-rd-hyperexecute.md deleted file mode 100644 index 722cd116f..000000000 --- a/static/docs/sharding-espresso-rd-hyperexecute.md +++ /dev/null @@ -1,356 +0,0 @@ -# Sharding for Espresso - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Running **Espresso** tests sequentially can be laborious and time-intensive. This guide offers an efficient solution by introducing the concept of **sharding**. By breaking down tests into shards, they can be executed in parallel, significantly trimming down the total testing duration. - -Furthermore, this document provides insights on executing Espresso Tests on real devices using the innovative **HyperExecute** platform. Designed with precision, HyperExecute stands out as a test orchestration solution, tailored to execute end-to-end tests at breakneck speed. To streamline its setup and configuration, a user-friendly `YAML` file is employed. Dive into this guide to make your testing process more agile and efficient. - -**Note** -You can learn more about the HyperExecute portal and UI by going through our [Guided Walkthrough](/support/docs/hyperexecute-guided-walkthrough/) page. It contains all the relevant information that you need to optimize your testing process with HyperExecute. - -## Objectives - -By the end of this topic, you will be able to: -1. Speed up your Espresso tests with sharding -2. Learn more about HyperExecute -3. Explore advanced features of TestMu AI - -## Prerequisites - -Before you start performing your App automation testing with Espresso, please make sure: - -- You have access to TestMu AI username and accessKey. If you have not registered yet, you can do the same by visiting our website. You will be able to access the credentials at the [TestMu AI Profile](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile) -- Make sure you have App Automation Cloud plans on your account. - -## Uploading your App & Test suite - -### Step 1: Upload Your Application - -Upload your **android** application (.apk file) to the TestMu AI servers using our **REST API**. You need to provide your **Username** and **AccessKey** in the format ```Username:AccessKey``` in the **cURL** command for authentication. Make sure to add the path of the **appFile** in the cURL request. - -Here is an example cURL request to upload your app using our REST API: - -**Using App File:** - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" \\ ---location --request POST 'https://manual-api.lambdatest.com/app/uploadFramework' \\ ---form 'appFile=@"/Users/macuser/Downloads/proverbial.apk"' \\ ---form 'type="espresso-android"'`} - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/uploadFramework" -F "appFile=@"/Users/macuser/Downloads/proverbial.apk"" -F "type="espresso-android""`} - ->Response of above cURL will be a **JSON** object containing the `App URL` of the format - `lt://APP123456789123456789` and will be used in the last step as `appId`. - -### Step 2: Uploading Test Suite - -Upload your **test suite** (.apk file) to the TestMu AI servers using our **REST API**. You need to provide your **Username** and **AccessKey** in the format `Username:AccessKey` in the **cURL** command for authentication. Make sure to add the path of the **appFile** in the cURL request. Here is an example cURL request to upload your app using our REST API: - -**Using App File:** - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" \\ ---location --request POST 'https://manual-api.lambdatest.com/app/uploadFramework' \ ---form 'appFile=@"/Users/macuser/Downloads/ProverbialEspressoTest.apk"' \ ---form 'type="espresso-android"'`} - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" --location --request POST "https://manual-api.lambdatest.com/app/uploadFramework" --form "appFile=@"C:/Users/varunkumarb/Downloads/ProverbialEspressoTest.apk"" --form "type=\"espresso-android\""`} - ->Response of above cURL will be a **JSON** object containing the `App URL` of the format - `lt://APP123456789123456789` and will be used in the next step as ``testSuiteAppId`` - -## How to use Sharding - -1. Firstly, create a folder on your local. -2. Download the **HyperExecute CLI** file and put it under this folder based on your platform. -#### Download HyperExecute CLI - -| Platform | HyperExecute CLI download location | -| ---------| --------------------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| macOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -## Types of Sharding -Sharding can be categorized into two types: - -**Manual Sharding**: This feature enables users to manually specify shards along with their definitions, offering multiple strategies such as class, package, and more. By doing so, users can have greater flexibility and control over the sharding process. - -Refer to the sample `.yaml` file here - -```yaml title="SampleYamlFile.yaml" -version: "0.2" -concurrency: 2 -runson: android - -# Set autosplit to true to enable auto sharding. -# The system will automatically split and distribute tests across the selected devices. -#highlight-next-line -autosplit: false - -globalTimeout: 180 #MAXQUEUETIMEOUT - -framework: -name: "android/espresso" -args: -buildName: "Espresso" -video: true -deviceLog: true - -# You can use either the appId (lt://APP1234567) or provide the path of the application using appPath. -# Both examples are given below. - -appPath: Proverbial.apk -testSuitePath: ProverbialExpressoTest.apk -# We have used the appPath and testSuitePath here. - -appId: lt://APP1016025651781520733656681 -testSuiteAppId: lt://APP10160332171784881008900412 -# We have used the appId and testSuiteAppID here. - -deviceSelectionStrategy: all -devices: ["Galaxy.*", "Pixel.*"] - -shards: -mappings: -- name: shard1 -strategy: "class" -values: -- "com.lambdatest.proverbial.BrowserTest" -# The strategy for this shard is based on "class". -# This shard will run all tests from the class com.lambdatest.proverbial.BrowserTest. - -- name: shard2 -strategy: "package" -values: -- "com.lambdatest.proverbial" -# The strategy for this shard is based on "package". -# This shard will run all tests that belong to the package com.lambdatest.proverbial. - -- name: shard3 -strategy: "skipClass" -values: -- "com.lambdatest.proverbial.BrowserTest" -# The strategy for this shard is to skip a specific class. -# This shard will avoid running tests from the class com.lambdatest.proverbial.BrowserTest. -``` - -```yaml title="SampleYamlFile.yaml" -version: "0.2" -concurrency: 2 -runson: android - -# Set autosplit to true to enable auto sharding. -# The system will automatically split and distribute tests across the selected devices. -#highlight-next-line -autosplit: false - -globalTimeout: 180 #MAXQUEUETIMEOUT - -framework: -name: "android/espresso" -args: -buildName: "Espresso" -video: true -deviceLog: true - -# You can use either the appId (lt://APP1234567) or provide the path of the application using appPath. -# Both examples are given below. - -appPath: Proverbial.apk -testSuitePath: ProverbialExpressoTest.apk -# We have used the appPath and testSuitePath here. - -appId: lt://APP1016025651781520733656681 -testSuiteAppId: lt://APP10160332171784881008900412 -# We have used the appId and testSuiteAppID here. - -deviceSelectionStrategy: all -devices: ["Galaxy.*", "Pixel.*"] - -#highlight-next-line -isVirtualDevice: true - -shards: -mappings: -- name: shard1 -strategy: "class" -values: -- "com.lambdatest.proverbial.BrowserTest" -# The strategy for this shard is based on "class". -# This shard will run all tests from the class com.lambdatest.proverbial.BrowserTest. - -- name: shard2 -strategy: "package" -values: -- "com.lambdatest.proverbial" -# The strategy for this shard is based on "package". -# This shard will run all tests that belong to the package com.lambdatest.proverbial. - -- name: shard3 -strategy: "skipClass" -values: -- "com.lambdatest.proverbial.BrowserTest" -# The strategy for this shard is to skip a specific class. -# This shard will avoid running tests from the class com.lambdatest.proverbial.BrowserTest. -``` - -**When shards are added** - -If you are using the `deviceSelectionStrategy: all`,then in that case all the specified shards will be executed on every device available. -**For example:** If there are 3 shards, 2 devices in list (d1, d2) and test suites contains total 10 test cases. -Then, d1 has 3 shards, i.e., total 3 devices of d1 configuration will be used. 10 test cases distributed among these 3 shards and similarly d2 also has 3 shards, i.e., total 3 devices of d2 configuration will be used. 10 test cases will be distributed among these 3 shards. - -If you are using the `deviceSelectionStrategy: any`, then in that case all the mentioned shards will be executed on just one device from the provided list. -**For example:** If there are 2 shards mentioned in `.yaml`and 3 devices mentioned, the system will create 2 shards. These shards might use any 2 devices from the given 3 configurations. It's also possible that the same device configuration could be used for both shards. In this setup, test cases will be distributed between these shards. - -**Auto Sharding** : The system intelligently determines the distribution of tests across devices, employing specific criteria to optimize the testing process. - -Refer to the sample `.yaml` file here - -```yaml title="SampleYamlFile.yaml" -version: "0.2" -concurrency: 2 -runson: android - -# Set autosplit to true to enable auto sharding. -# The system will automatically split and distribute tests across the selected devices. -#highlight-next-line -autosplit: true - -globalTimeout: 180 #MAXQUEUETIMEOUT - -framework: -name: "android/espresso" -args: -buildName: "Espresso" -video: true -deviceLog: true - -# You can use either the appId (lt://APP1234567) or provide the path of the application using appPath. -# Both examples are given below. - -appPath: Proverbial.apk -testSuitePath: ProverbialExpressoTest.apk -# We have used the appPath and testSuitePath here. - -appId: lt://APP1016025651781520733656681 -testSuiteAppId: lt://APP10160332171784881008900412 -# We have used the appId and testSuiteAppID here. - -deviceSelectionStrategy: all -devices: ["Galaxy.*", "Pixel.*"] -``` - -```yaml title="SampleYamlFile.yaml" -version: "0.2" -concurrency: 2 -runson: android - -# Set autosplit to true to enable auto sharding. -# The system will automatically split and distribute tests across the selected devices. -#highlight-next-line -autosplit: true - -globalTimeout: 180 #MAXQUEUETIMEOUT - -framework: -name: "android/espresso" -args: -buildName: "Espresso" -video: true -deviceLog: true - -# You can use either the appId (lt://APP1234567) or provide the path of the application using appPath. -# Both examples are given below. - -appPath: Proverbial.apk -testSuitePath: ProverbialExpressoTest.apk -# We have used the appPath and testSuitePath here. - -appId: lt://APP1016025651781520733656681 -testSuiteAppId: lt://APP10160332171784881008900412 -# We have used the appId and testSuiteAppID here. - -#highlight-next-line -isVirtualDevice: true - -deviceSelectionStrategy: all -devices: ["Galaxy.*", "Pixel.*"] -``` - -**When shards aren't added** - -If you are using the `deviceSelectionStrategy: all`, then in that case the tests will be executed on all mentioned devices in `.yaml` based on the concurrency. -**For example:** If the concurrency is 3, 2 devices in list (d1, d2) and test suites contains total 10 test cases. -Then, d1 has 3 shards, i.e., total 3 devices of d1 configuration will be used. 10 test cases distributed among these 3 shards and similarly d2 also has 3 shards, i.e., total 3 devices of d2 configuration will be used. 10 test cases will be distributed among these 3 shards. - -If you are using the `deviceSelectionStrategy: any`, then in that case all the specified tests will be executed on each device from the provided list, considering the concurrency setting. -**For example:** If the concurrency is set to 2 and 3 devices mentioned, the system will create 2 shards. These shards might use any 2 devices from the given 3 configurations. It's also possible that the same device configuration could be used for both shards. In this setup, test cases will be distributed between these shards. - -4. Go to the location of the folder on the terminal and run the below commands. After the second command, you may need to give permission on **System Settings/Privacy & Security**. - -```bash -chmod u+x -./ -user -key --verbose -i .yaml -``` - -You can refer to this example and screenshot below: -``` -./hyperexecute -user my_user_name -key xyx123abc --verbose -i hyperexecute.yaml -``` - -5. After the test is started you can follow the test on the below links. -- [TestMu AI Automation](https://www.testmuai.com/login/?redirectTo=https://appautomation.lambdatest.com/build) - -- [TestMu AI HyperExecute](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) - -**globalTimeout** ​ The globalTimeout value determines the maximum duration (in minutes) of a Job . It can be set between 5 and 180 minutes, and has a default value of 10 minutes. - -## Filters in Sharding - -You can filter the Classes / Tests that you'd like to execute using filters. Here's an example of the same. - -```java -filters: -attributes: -- type: class -values: ["com.lambdatest.proverbial.BrowserTest","com.lambdatest.proverbial.MainActivityTest"] -- type: package -values: ["com.lambdatest.proverbial"] -``` - -This example will fetch all the test cases from 2 classes and 1 package as defined above and divide them with respect to concurrency given in the yaml file. - -## Annotations in Sharding - -You can also filter your Espresso tests based on annotations while using sharding. This helps you execute only the test cases marked with specific annotations. - -```yaml -filters: -attributes: -- type: annotation -values: ["com.lambdatest.proverbial.demo1", "com.lambdatest.proverbial.demo2"] -``` - -This configuration will run tests annotated with multiple annotations, i.e., only the intersection of all mentioned annotations will be considered. - -## Generate Reports and Artifacts -To generate artifacts for your Espresso tests, add the `artifacts: true`, and `report: true` flag in your YAML file: - -```yaml -report: true -framework: -name: "android/espresso" -args: -artifacts: true -...// -``` - -To download these artifacts in your local machine, you can pass the `--download-artifacts` and `--download-report` flag with the CLI command to execute the tests as shown in the next step. - -> Learn more about it in the [HyperExecute Espresso Documentation](/support/docs/hyperexecute-espresso-testing/) - -## Additional Links - -- [Advanced Configuration for Capabilities](/support/docs/desired-capabilities-in-appium/) -- [How to test locally hosted apps](/support/docs/testing-locally-hosted-pages/) -- [How to integrate TestMu AI with CI/CD](/support/docs/integrations-with-ci-cd-tools/) diff --git a/static/docs/sharding-rd-hyperexec.md b/static/docs/sharding-rd-hyperexec.md deleted file mode 100644 index ddda578e2..000000000 --- a/static/docs/sharding-rd-hyperexec.md +++ /dev/null @@ -1,253 +0,0 @@ -# Sharding for XCUI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Generally the XCUI tests are run in sequence which is a time taking process. This document explains how you can speed up this process by splitting the tests into **shards**. We can divide the various tests into shards which can run parallelly and save time while running various XCUI tests. - -This document will cover how to execute **XCUI Tests** on real devices with **HyperExecute**. HyperExecute is a smart test orchestration platform to run end-to-end tests at the fastest speed possible. HyperExecute is configured using a YAML file. - -**Note** -You can learn more about the HyperExecute portal and UI by going through our [Guided Walkthrough](/support/docs/hyperexecute-guided-walkthrough/) page. It contains all the relevant information that you need to optimise your testing process with HyperExecute. - -## Objectives - -By the end of this topic, you will be able to: -1. Speed up your XCUI tests with sharding -2. Learn more about HyperExecute -3. Explore advanced features of TestMu AI - -## Prerequisites - -Before you start performing your App automation testing with XCUI, please make sure: - -- You have access to TestMu AI username and accessKey. If you have not registered yet, you can do the same by visiting our website. You will be able to access the credentials at the [TestMu AI Profile](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile) -- Make sure you have App Automation and HyperExecute Cloud plans on your account. - -## Uploading your App & Test suite - -### Step 1: Upload Your Application - -Upload your **iOS** application (.ipa file) to the TestMu AI servers using our **REST API**. You need to provide your **Username** and **AccessKey** in the format ```Username:AccessKey``` in the **cURL** command for authentication. Make sure to add the path of the **appFile** in the cURL request. - -Here is an example cURL request to upload your app using our REST API: - -**Using App File:** - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" \\ ---location --request POST 'https://manual-api.lambdatest.com/app/uploadFramework' \\ ---form 'appFile=@"/Users/macuser/Downloads/proverbial.ipa"' \\ ---form 'type="xcuit-ios"'`} - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/uploadFramework" -F "appFile=@"/Users/macuser/Downloads/proverbial.ipa"" -F "type="xcuit-ios""`} - -Response of above cURL will be a **JSON** object containing the `App URL` of the format - `APP123456789123456789` and will be used in the last step as `appId`. - -### Step 2: Uploading Test Suite - -Upload your **test suite** (.ipa file) to the TestMu AI servers using our **REST API**. You need to provide your **Username** and **AccessKey** in the format `Username:AccessKey` in the **cURL** command for authentication. Make sure to add the path of the **appFile** in the cURL request. Here is an example cURL request to upload your app using our REST API: - -**Using App File:** - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" \\ ---location --request POST 'https://manual-api.lambdatest.com/app/uploadFramework' \ ---form 'appFile=@"/Users/macuser/Downloads/proverbial_ios_xcuitest.ipa"' \ ---form 'type="xcuit-ios"'`} - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" --location --request POST "https://manual-api.lambdatest.com/app/uploadFramework" --form "appFile=@"C:/Users/varunkumarb/Downloads/proverbial_ios_xcuitest.ipa"" --form "type=\"xcuit-ios\""`} - -Response of above cURL will be a **JSON** object containing the `App URL` of the format - `APP123456789123456789` and will be used in the next step as ``testSuiteAppId`` - -## How to use Sharding - -1. Firstly, create a folder on your local. -2. Download the HyperExecute CLI file and put it under this folder. You may download HyperExecute CLI from either of the following ways: -- Download it from our HyperExecute documentation page [here](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/), or -- "Get Started" through HyperExecute [onboarding](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/quickstart) and download the CLI. -3. Refer to the sample `.ipa` files and `.yaml` file given below. Put all of them under this folder. -- [ProverbialTest.ipa](https://prod-mobile-artefacts.lambdatest.com/assets/docs/ProverbialTest.ipa) -- [LambdaUiKitIOS.ipa](https://prod-mobile-artefacts.lambdatest.com/assets/docs/LambdaUiKitIOS.ipa) - -Refer to the sample `.yaml` file here - -```yaml title="SampleYamlFile.yaml" -version: "0.2" -concurrency: 2 -runson: ios - -# Set autosplit to true to enable auto sharding. -# The system will automatically split and distribute tests across the selected devices. -#highlight-next-line -autosplit: false - -maxRetries: 2 -retryOnFailure: true -globalTimeout: 180 #MAXQUEUETIMEOUT - -framework: -name: "ios/xcui" -args: -buildName: "XCUIT" -video: true -networkLog: true -deviceLog: true - -# You can use either the appId (lt://APP1234567) or provide the path of the application using appPath. -# Both examples are given below. - -appPath: ProverbialTest.ipa -testSuitePath: LambdaUiKitIOS.ipa -# We have used the appPath and testSuitePath here. - -appId: lt://APP1010461471690377432133206 -testSuiteAppId: lt://APP10104592261690377454846669 -# We have used the appId and testSuiteAppID here. - -deviceSelectionStrategy: all -devices: ["iPhone 12 Pro-14", "iPad Air (2019)-16"] - -shards: -mappings: -- name: shard1 -strategy: "only-testing/skip-testing" -values: -- "/" -# The strategy for this shard is based on "only-testing/skip-testing". -# This shard will either execute only the specified test(s) or skip the specified test(s), -# depending on the strategy value configured. - -- name: shard2 -strategy: "only-testing/skip-testing" -values: -- "/" -- "/" -# The strategy for this shard is based on "only-testing/skip-testing". -# This shard will either execute only the specified test(s) or skip the specified test(s), -# depending on the strategy value configured. -``` - -```yaml title="SampleYamlFile.yaml" -version: "0.2" -concurrency: 2 -runson: ios - -# Set autosplit to true to enable auto sharding. -# The system will automatically split and distribute tests across the selected devices. -#highlight-next-line -autosplit: false - -maxRetries: 2 -retryOnFailure: true -globalTimeout: 180 #MAXQUEUETIMEOUT - -framework: -name: "ios/xcui" -args: -buildName: "XCUIT" -video: true -networkLog: true -deviceLog: true - -# You can use either the appId (lt://APP1234567) or provide the path of the application using appPath. -# Both examples are given below. - -appPath: ProverbialTest.ipa -testSuitePath: LambdaUiKitIOS.ipa -# We have used the appPath and testSuitePath here. - -appId: lt://APP1010461471690377432133206 -testSuiteAppId: lt://APP10104592261690377454846669 -# We have used the appId and testSuiteAppID here. - -deviceSelectionStrategy: all -devices: ["iPhone 12 Pro-14", "iPad Air (2019)-16"] - -#highlight-next-line -isVirtualDevice: true - -shards: -mappings: -- name: shard1 -strategy: "only-testing/skip-testing" -values: -- "/" -# The strategy for this shard is based on "only-testing/skip-testing". -# This shard will either execute only the specified test(s) or skip the specified test(s), -# depending on the strategy value configured. - -- name: shard2 -strategy: "only-testing/skip-testing" -values: -- "/" -- "/" -# The strategy for this shard is based on "only-testing/skip-testing". -# This shard will either execute only the specified test(s) or skip the specified test(s), -# depending on the strategy value configured. -``` - -**When shards are added** - -If you are using the `deviceSelectionStrategy: all`, then in that case all the mentioned shards will be executed on all the devices. -**For example:** There are 2 shards and 2 devices mentioned, then the 2 shards will be executed on 2 devices (2 Shards * 2 Devices). - -If you are using the `deviceSelectionStrategy: any`, then in that case all the mentioned shards will be executed on any one device from the list provided. -**For example:** There are 2 shards and 2 devices mentioned, then the 2 shards will be executed on any device (2 shards * any device mentioned). - -**When shards aren't added** - -If you are using the `deviceSelectionStrategy: all`, then in that case all the tests will be executed on all the devices based on the concurrency. -**For example:** There are 6 tests and 10 devices mentioned, then the 6 tests will be auto distributed on the devices based on the max concurrency. - -If you are using the `deviceSelectionStrategy: any`, then in that case all the mentioned tests will be executed on any one device from the list provided. -**For example:** There are 6 tests and 10 devices mentioned, then the 6 tests will be executed on any one device. - -4. Go to the location of the folder on the terminal and run the below commands. After the second command, you may need to give permission on **System Settings/Privacy & Security**. - -```bash -chmod u+x -./ --u --k --verbose -i .yaml -``` - -You can refer to this example and screenshot below: -``` -./hyperexecute --u my_user_name --k xyx123abc --verbose -i hyperexecute.yaml -``` - -5. After the test is started you can follow the test on the below links. -- [TestMu AI Automation](https://www.testmuai.com/login/?redirectTo=https://appautomation.lambdatest.com/build) - -- [TestMu AI HyperExecute](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) - -**globalTimeout** ​ The globalTimeout value determines the maximum duration (in minutes) of a Job . It can be set between 5 and 180 minutes, and has a default value of 10 minutes. - -## Filters in Sharding -You can filter the Classes / Tests that you'd like to execute using filters. -Here's an example of the same. - -```java -filters: -attributes: -- type: className -values: ["LambdaUiKitIOSUITests","LambdaUiKitIOSUITestsLaunchTests"] -- type: testName -values: ["LambdaUiKitIOSUITests/testverifyAppLaunch"] -``` -This example will run only the 2 classes & one test as mentioned from the TestSuite. - -## XCTestPlan in Sharding -To implement the XCTestPlan in Sharding, add the `xctestplan` flag along with `app` and `testSuite` in the [framework](/support/docs/hyperexecute-yaml-version0.2/#framework) flag as shown below: - -```yaml -framework: -name: "ios/xcui" -args: -"app" : "lt://APP_ID", -"testSuite": "lt://TEST_SUITE_ID", -"xctestplan" : "lt://YOUR_XC_TEST_PLAN_ID" #only when you want to use XCTestPlan -``` - -> Refer the documentation to learn how to use [XCTestPlan feature](/support/docs/xctestplan/). - -## Additional Links -- [Advanced Configuration for Capabilities](/support/docs/desired-capabilities-in-appium/) -- [How to test locally hosted apps](/support/docs/testing-locally-hosted-pages/) -- [How to integrate TestMu AI with CI/CD](/support/docs/integrations-with-ci-cd-tools/) diff --git a/static/docs/shared-cloud-usage.md b/static/docs/shared-cloud-usage.md deleted file mode 100644 index 11fd73229..000000000 --- a/static/docs/shared-cloud-usage.md +++ /dev/null @@ -1,108 +0,0 @@ -# Daily Usage Limit Per Organization - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -To avoid over-exploitation of TestMu AI and promote best practices, we have added a capacity constraint on the total test time that can be used by an organization in a day. - -| PARALLEL SESSIONS | COMPLETED RUN TIME (HRS) | -| ---------- | ----------------- | -| 1 | 6 | -| 2 | 12 | -| 3 | 18 | -| 4 | 24 | -| 5 | 30 | -| 6 | 36 | -| 7 | 42 | -| 8 | 48 | -| 9 | 54 | -| 10 | 60 | -| 11 | 66 | -| 12 | 72 | -| 13 | 78 | -| 14 | 84 | -| 15 | 90 | -| 16 | 96 | -| 17 | 102 | -| 18 | 108 | -| 19 | 114 | -| 20 | 120 | -| 21 | 126 | -| 22 | 132 | -| 23 | 138 | -| 24 | 144 | -| 25 | 150 | -| 26 | 156 | -| 27 | 162 | -| 28 | 168 | -| 29 | 174 | -| 30 | 180 | -| 31 | 186 | -| 32 | 192 | -| 33 | 198 | -| 34 | 204 | -| 35 | 210 | -| 36 | 216 | -| 37 | 222 | -| 38 | 228 | -| 39 | 234 | -| 40 | 240 | -| 41 | 246 | -| 42 | 252 | -| 43 | 258 | -| 44 | 264 | -| 45 | 270 | -| 46 | 276 | -| 47 | 282 | -| 48 | 288 | -| 49 | 294 | -| 50 | 300 | -| 51 | 306 | -| 52 | 312 | -| 53 | 318 | -| 54 | 324 | -| 55 | 330 | -| 56 | 336 | -| 57 | 342 | -| 58 | 348 | -| 59 | 354 | -| 60 | 360 | -| 61 | 366 | -| 62 | 372 | -| 63 | 378 | -| 64 | 384 | -| 65 | 390 | -| 66 | 396 | -| 67 | 402 | -| 68 | 408 | -| 69 | 414 | -| 70 | 420 | -| 71 | 426 | -| 72 | 432 | -| 73 | 438 | -| 74 | 444 | -| 75 | 450 | -| 76 | 456 | -| 77 | 462 | -| 78 | 468 | -| 79 | 474 | -| 80 | 480 | -| 81 | 486 | -| 82 | 492 | -| 83 | 498 | -| 84 | 504 | -| 85 | 510 | -| 86 | 516 | -| 87 | 522 | -| 88 | 528 | -| 89 | 534 | -| 90 | 540 | -| 91 | 546 | -| 92 | 552 | -| 93 | 558 | -| 94 | 564 | -| 95 | 570 | -| 96 | 576 | -| 97 | 582 | -| 98 | 588 | -| 99 | 594 | -| 100 | 600 | diff --git a/static/docs/sharing-lambda-tunnel.md b/static/docs/sharing-lambda-tunnel.md deleted file mode 100644 index f6caa0baa..000000000 --- a/static/docs/sharing-lambda-tunnel.md +++ /dev/null @@ -1,50 +0,0 @@ -# Sharing TestMu AI Tunnel - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI is a [cross browser testing](https://www.testmuai.com/) tool which allows you to test your locally hosted website or web application on 3000+ real browsers. It does so with the help of a TCP with TLS 1.2 secure connection tunnel which establishes a secure and unique connection from your local system to TestMu AI cloud servers. - -Earlier with TestMu AI Tunnel, users belonging to the same organization were able to choose any tunnel they want. So if a team had 5 members and all of them had configured their own tunnels, then they were able to shuffle their TCP with TLS 1.2 secure connection tunnel with any of the 5 connections. However, that is not the case anymore! - -Now, a user can only work on a user-specific TCP with TLS 1.2 secure connection tunnel. In case you wish to keep a common TCP with TLS 1.2 secure connection tunnel across all your teammates then you will need to establish a shared TestMu AI Tunnel that would work for all of the teammates, listed under your organization at TestMu AI. - -This document will help you share your TestMu AI Tunnel with colleagues listed as team in your TestMu AI account. - -## Configuring TestMu AI Tunnel - -**Step 1:** Download the zip file of TestMu AI Tunnel for your operating system: -- [Download for Windows](https://downloads.lambdatest.com/tunnel/v3/windows/64bit/LT_Windows.zip) -- [Download for Mac OS](https://downloads.lambdatest.com/tunnel/v3/mac/64bit/LT_Mac.zip) -- [Download for Linux](https://downloads.lambdatest.com/tunnel/v3/linux/64bit/LT_Linux.zip) - -**Step 2:** Extract the zip file in your system and route your CMD/terminal to the same folder where the zip file is extracted. For demonstration purpose, we have extracted the zip file to desktop so you will find the CMD to be pointed towards the desktop. - -**Step 3:** To configure a shared TestMu AI Tunnel you would need to append an argument `--shared-tunnel` to the command used for invoking TestMu AI Tunnel. Syntax to configure a shared TCP with TLS 1.2 secure connection TestMu AI Tunnel would be: - -`LT --user {user's login email} --key {user's access key} --shared-tunnel --tunnelName {string}` - -So for example, if your user login email is example@lambdatest.com and your user key is 123asd123, the command would be: - -`LT --user example@lambdatest.com --key 123asd123 --shared-tunnel --tunnelName SharedTCP-TLS12-Example` - -**Step 4:** As you run the above command you will find the below console UI indicating with a message that the tunnel is successfully established. - -That is it, if your teammates will now open their TestMu AI account then they will find your Tunnel to be active for them by default. - -## Don't Forget To Name The Shared Tunnel - -It could be problematic if 2 or more teammates start sharing their tunnels across the organization, as a best practice we recommend you to name the Shared tunnel appropriately so your teammates could easily identify which TCP with TLS 1.2 secure connection tunnel is their own and which is the one shared by you. - -To name a shared tunnel you will have to append another argument in the command for configuring tunnel. The new argument would be: `-tunnelName {string}` - -Syntax: - -`LT --user {user's login email} --key {user's access key} --shared-tunnel --tunnelName {string}` - -Example - -`LT --user example@lambdatest.com --key 123asd123 --shared-tunnel --tunnelName SharedTCP-TLS12-example` - -**Important Note:** If you are running local host at a port which is different from the default port being used your team then make sure to tell them so they can address it in the URL while performing cross browser testing at TestMu AI. For example, if you are running Apache server at port `4200`, and want your teammates to hop over a URL that is hosted locally by you, then they will have to specify port `4200` before they hit the start testing button on TestMu AI. - -Kudos! You have successfully created a Shared TestMu AI Tunnel for your organization. If you still have any questions, feel free to share them with us through our **24/7 chat support** or by mailing to us on [support@testmuai.com](mailto:support@testmuai.com). Happy testing! 🙂 diff --git a/static/docs/sharing-test-results.md b/static/docs/sharing-test-results.md deleted file mode 100644 index d21effa2a..000000000 --- a/static/docs/sharing-test-results.md +++ /dev/null @@ -1,124 +0,0 @@ -# Sharing Test Results on TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI lets you share individual test results with team members. Retrieve the Session ID from your test script, then build a shareable URL for automation logs or execution videos. - -## Get the Session ID - -Retrieve the Selenium Session ID programmatically so you can build shareable result URLs. - -Every test session on TestMu AI has a unique Session ID. Use the code below to retrieve it in your preferred language: - -```java -import org.openqa.selenium.remote.SessionId; - -SessionId session = ((RemoteWebDriver) driver).getSessionId(); -System.out.println("Session ID: " + session.toString()); -``` - -```javascript -const session = await driver.getSession(); -console.log("Session ID: " + session.getId()); -``` - -```python -session_id = driver.session_id -print("Session ID: " + session_id) -``` - -```csharp -var sessionId = ((RemoteWebDriver)driver).SessionId; -Console.WriteLine("Session ID: " + sessionId); -``` - -```php -$sessionId = $driver->getSessionID(); -echo "Session ID: " . $sessionId; -``` - -```ruby -session_id = driver.session_id -puts "Session ID: #{session_id}" -``` - -Once you have the Session ID, share the automation logs URL with your colleague: - -``` -https://automation.lambdatest.com/logs/?sessionID=YOUR_SESSION_ID -``` - -## Share Your Test Execution Video - -Build a shareable URL using the TestID/SessionID and an MD5-generated auth token. - -You can share a video recording of any test execution. Build the URL in the following format: - -``` -https://automation.lambdatest.com/public/video?testID={testid/sessionid}&auth=AUTH_TOKEN -``` - -### Step 1 - Get Your TestID or SessionID - -Go to the **Automation Dashboard** and click on the test whose ID you need. Click the **"i"** button on the status bar to access the test metadata. - -After clicking the **"i"** button, you will see the test metadata including the SessionID. - -For example, if your SessionID is `HJKXM-RHZL1-SVPWY-AB8X6`, the URL becomes: - -``` -https://automation.lambdatest.com/public/video?testID=HJKXM-RHZL1-SVPWY-AB8X6&auth=AUTH_TOKEN -``` - -### Step 2 - Generate the AUTH_TOKEN - -Create the AUTH_TOKEN by computing an MD5 hash of your `username:access_key` string. - -```java -MessageDigest m = MessageDigest.getInstance("MD5"); -String s = "username:access_key"; -m.update(s.getBytes(), 0, s.length()); -System.out.println("MD5: " + new BigInteger(1, m.digest()).toString(16)); -``` - -```javascript -var crypto = require('crypto'); -var token = crypto.createHash('md5').update("username:access_key").digest("hex"); -console.log("AUTH_TOKEN: " + token); -``` - -```python -import hashlib -token = hashlib.md5("username:access_key".encode('utf-8')).hexdigest() -print("AUTH_TOKEN: " + token) -``` - -```csharp -byte[] inputBytes = System.Text.Encoding.ASCII.GetBytes("username:access_key"); -byte[] hashBytes = System.Security.Cryptography.MD5.Create().ComputeHash(inputBytes); -StringBuilder sb = new StringBuilder(); -for (int i = 0; i < hashBytes.Length; i++) -{ -sb.Append(hashBytes[i].ToString("X2")); -} -Console.WriteLine("AUTH_TOKEN: " + sb.ToString()); -``` - -```php -$token = md5("username:access_key"); -echo "AUTH_TOKEN: " . $token; -``` - -```ruby -require 'digest' -token = Digest::MD5.hexdigest("username:access_key") -puts "AUTH_TOKEN: #{token}" -``` - -For example, if the generated AUTH_TOKEN is `331k534uf3toef`, the final URL becomes: - -``` -https://automation.lambdatest.com/public/video?testID=HJKXM-RHZL1-SVPWY-AB8X6&auth=331k534uf3toef -``` - -You must be logged into TestMu AI to access the sharing URL. diff --git a/static/docs/shortcut-integration.md b/static/docs/shortcut-integration.md deleted file mode 100644 index 3c7484ef4..000000000 --- a/static/docs/shortcut-integration.md +++ /dev/null @@ -1,70 +0,0 @@ -# Shortcut Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -> Shortcut is a project management platform that would help everyone on your team to collaborate better for hastening the release process. Shortcut helps in finding the right balance for cross-functional collaboration. It is easy for everyone on the team to focus on their individual task, while also being able to envision the bigger picture by zooming out to get an idea about the overall progress of a project. From declaring a story to achieving huge milestones, everything is represented with an intuitive visual representation using simple drag and drop Kanban dashboard. With a simple API and robust set of integrations, Shortcut also seamlessly integrates with the tools you use every day, getting out of your way so that you can deliver quality software on time. - -The TestMu AI Shortcut integration helps you in filing issues to your project in Shortcut directly from TestMu AI platform. With one-click integration, you can push annotated issues to the project of your choice, assign it to the required teammate, and attach necessary screenshots. You can do all that while in the middle of a test session in TestMu AI platform. The fields populated by you when marking as a bug through TestMu AI are displayed as information on the Shortcut for a testing instance. - -> Shortcut Integration with TestMu AI, like all of the integrations to 3rd party applications, is available for freemium as well as premium plan. - -## How To Establish Integration With Shortcut From Your TestMu AI Account? - -**Step 1:** Login to your TestMu AI account. You would need to have Admin or User level access to see and install integrations. - -**Step 2:** Select ‘Integrations’ from the left navigation menu bar. This will guide you to a screen where you will find a list of 3rd party applications, available to integrate with your TestMu AI account. - -**Step 3:** Click on ‘ADD’ under the block that says ‘Shortcut’. - -**Step 4:** Once you click on Add icon, you will be redirected to the below screen. You will be asked to provide your Shortcut API Token credentials in the highlighted fields to establish integration with TestMu AI. - -> **What is an API token?** -When two different applications interact with each other, a server-to-server communication is established. API token is needed for authenticating user identity from one server to another, by fetching user-owned resources available on the server from where the communication is initiated. API token has been a necessity for maintaining security and data integrity of any application. They are just as vital to an end user as they are to an application. API tokens are relevant if an end user logs in an application through a 2-step verification process. - -**Step 5:** You can find your API token from your Shortcut account. If you have not generated one before, you can generate an API token by going to the settings. Hit the Profile avatar icon on the bottom left and select ‘Settings’. - -**Step 6:** As you click on Settings, select "API Tokens" from the left hand side. - -**Step 7:** Provide a token name and hit the button that says "Generate Token". - -> You can keep a note of this API token with you for future authentication. Although, in case you forget to do so, or you end up neglecting it due to some reason. Then there is no need to panic! You can create a new token by defining a new label. You can also delete a previously declared API Key, in case you feel that your API Key has been placed in some wrong hands. You can delete an API Key by hovering over the label and clicking on the bin icon. - -**Step 8:** Your API token will be derived from the value you provided as an input. Copy your API Token. - -**Step 9:** Paste the API token into the field provided at TestMu AI & hit the ‘Install’ button. You are all set to experience one-click bug logging to share your issues directly from your TestMu AI account to your project on Shortcut. - -That’s it! Go to Integrations again and you will be able to notice a green tick indicating that Shortcut is successfully installed. - -## How To Log Your First Bug Through Shortcut Integration? - -**Step 1:** Go for any of the test from the left navigation menu. For demo, we will be taking "Real Time Test" option. - -**Step 2:** Present a URL of the web-app you need to test in the dialog box. After that, select any configuration for browser and operating system of your choice & hit ‘Start‘. - -**Step 3:** After the VM is launched and operable. You can perform testing on your web-app for finding bugs. If a bug gets revealed, then you need to click on the Bug icon from the left panel for capturing a screenshot of the same. - -**Step 4:** After a screenshot is captured, you can annotate any issue or a task with an in-built image editor. Once you are done highlighting the bug, click on the button that says "Mark as Bug". - -**Step 5:** After clicking on "Mark as Bug" button a Shortcut specific form would open up. Fill the fields as per your requirement. Click on "Create Issue" & observe it being successfully marked through a single click effort. - -* You can select which **Project** should the ticket go under. -* You can set the particular **Story Type**. -* You can assign the bug to a colleague by populating the **Assignee** field. -* You can mention the **Summary** as well to easily organize your bugs. -* You also get to post a **Description** to help relate the cause of the issue or the task. - -**Step 7:** Visit your Shortcut dashboard. You will be able to notice the logged issue right away by going to stories in the particular project. - -**Step 8:** As you click on the story you will notice that all the data which you provided through TestMu AI would already be presented in it. TestMu AI automatically includes test environment details and related screenshots in the ticket as attachments. - -## How To Uninstall Shortcut Integration? - -> You can work with one integration at a time. So if you would want to integrate to a similar 3rd party application, then you would have to `uninstall` your current integration. Here is how you can do that. - -**Step 1:** Login to your TestMu AI account. - -**Step 2:** Select ‘Integration’ from the left navigation menu bar. This will guide you to a screen where you will find 3rd party applications, available to integrate with your TestMu AI account. - -**Step 3:** Click on ‘REMOVE’. You can find the remove button right under the ‘Shortcut’ block which would be highlighted with a green tick. - -> That was all you need to know for TestMu AI + Shortcut Integration. Increase your productivity with our integrations. If you still have any questions for us, please feel free to let us know. Our experts are always available on **chat** to help you out with any roadblock regarding our product. Happy testing! diff --git a/static/docs/siesta-integration.md b/static/docs/siesta-integration.md deleted file mode 100644 index 12e9c1140..000000000 --- a/static/docs/siesta-integration.md +++ /dev/null @@ -1,51 +0,0 @@ -# Siesta Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Siesta is a JavaScript and UI testing tool for web pages and Node.js processes. You can run tests in browsers, Node.js, and Deno on Linux, macOS, and Windows. - -TestMu AI and Siesta integration lets you perform automated browser testing of your locally hosted websites or web apps across 3000+ real browsers and OS combinations. - -## Prerequisites - -1. A TestMu AI account. In case, you don't have an TestMu AI account, sign up for free. - -2. Download Siesta from official Bryntum website. - -3. In order to run your Siesta tests, you will need to set your TestMu AI username and access key in the environment variables. Click the Access Key button at the top-right of the [Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) to access it. - -**Windows** - -```js -set LT_USERNAME="YOUR_LAMBDATEST_USERNAME" -set LT_ACCESS_KEY="YOUR_LAMBDATEST_ACCESS_KEY" -``` - -**macOS/Linux** - -```js -export LT_USERNAME="YOUR_LAMBDATEST_USERNAME" -export LT_ACCESS_KEY="YOUR_LAMBDATEST_ACCESS_KEY" -``` - -## Running Single Test With Siesta - -Configure your local web server "localhost" on port 80 and then run the below command in the CMD terminal - - -```js -C:\xampp\htdocs\siesta\bin\webdriver http://localhost/siesta/examples/browser/index.html --filter basic --lambdatest LT_USERNAME, LT_ACCESS_KEY --cap browserName=firefox --cap platform=windows -``` - -To generate desired capabilities for your test script, head over to our [Desired Capabilities Generator](https://www.testmuai.com/capabilities-generator/). - -## Parallel Testing With Siesta - -TestMu AI lets you to perform parallel testing with Siesta across 3000+ browsers and OS. You can run one test case across multiple browsers, or you can run multiple test case scenarios in the same browser with different browser versions. - -To run your tests in parallel, pass the below command in the CMD terminal- - -```js -C:\xampp\htdocs\siesta\bin\webdriver http://localhost/siesta/examples/browser/index.html --filter basic --lambdatest LT_USERNAME, LT_ACCESS_KEY --cap browserName=firefox --cap platform=windows --max-workers -``` - -> If you come across any issue or doubt, feel free to reach out to us via our **24/7 chat support**. Or you can even mail us at [support@testmuai.com](mailto:support@testmuai.com) Happy Testing!! diff --git a/static/docs/sim-support-realdevices.md b/static/docs/sim-support-realdevices.md deleted file mode 100644 index 112fa3d59..000000000 --- a/static/docs/sim-support-realdevices.md +++ /dev/null @@ -1,62 +0,0 @@ -# SIM Support On Public Real Devices - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -SIM & eSIM support allows testers to validate **SMS-driven workflows** on real mobile devices, including **OTP verification** and **two-factor authentication**. By using SIM-enabled devices manually, you can test SMS handling, **verify phone numbers**, and confirm secure onboarding processes without needing physical test phones in hand. - -**SMS-based authentication** is a widely used security and identity verification method. Testing these flows manually on real devices ensures accurate results, provides hands-on visibility, and helps confirm a smooth experience for end-users. - -With **TestMu AI**, you can manually test SIM and eSIM scenarios on **Real Devices**. From selecting SIM-enabled devices in the dashboard to receiving OTPs directly on-screen, TestMu AI offers a complete setup for validating SMS-driven features during live testing sessions. - -> To enable this for your organization, contact us via **24×7 chat support** or email **support@testmuai.com**. - -## Use Cases - -- **OTP & Two-Factor Testing**: Manually trigger SMS-based login or sign-up and verify OTP delivery and entry. -- **Phone Number Validation**: View the device’s number during a session and confirm its use in verification flows. -- **SMS Interrupt Testing**: Receive SMS messages during testing and check how the app responds to message interrupts. -- **Messaging Workflows**: Validate outbound and inbound SMS behavior while interacting with the app manually. - -## Configuring SIM Support for Manual Testing - -### Step 1: Enable SIM Support -On the App Testing Dashboard, click **Advanced Settings** and enable the **SIM Enabled** toggle. This will unlock additional options for configuring SIM and eSIM. - -### Step 2: Select SIM Configurations -Choose your **Country**, **Carrier**, and **SIM Type**. The availability of SIM types may vary by region and device. - -| Country | Carriers | SIM Type Availability | -|---------|-----------------------|-----------------------------| -| Ireland | Three | Physical SIM / eSIM (iOS only) | -| India | Airtel, Jio, Vodafone | Physical SIM | -| USA | T-Mobile, Verizon, AT&T | Physical SIM / eSIM (iOS only) | - -- For customers with the **region filter** enabled, the country option will not be available for selection. In such cases, the country is automatically determined by the chosen region. -- Currently, **eSIM support is limited to selected iPhone models only**. Android devices support **physical SIM** at this time. - -### Step 3: Start a Manual Session -Launch a real device session from the dashboard. - -### Step 4: Verify Device Details -From the session toolbar, open **Switch** to view details such as **Phone Number**, **Region**, **Carrier**, and **SIM Type**. Verify that the device matches your intended test configuration. - -### Step 5: Begin Testing -With the device configured and details verified, you can now proceed to test your application under real SIM conditions. Use the session to validate how your app interacts with SMS messages, confirm that authentication flows work as expected, and observe the overall user experience in a live environment. - -## Supported Devices - -| Device | OS Version | -|--------------------------|-------------| -| Samsung Galaxy S25 Ultra | Android 15 | -| Samsung Galaxy S25 | Android 15 | -| Samsung Galaxy S24 | Android 14 | -| Samsung Galaxy S23 | Android 13 | -| Google Pixel 9 | Android 15 | -| Google Pixel 7 | Android 13 | -| Google Pixel 6 | Android 12 | -| iPhone 16 Pro | iOS 18 | -| iPhone 15 Pro | iOS 17 | -| iPhone 14 | iOS 16 | -| iPhone 14 Pro | iOS 16 | -| iPhone 13 | iOS 15 | -| iPhone 13 Mini | iOS 15 | diff --git a/static/docs/sim-support.md b/static/docs/sim-support.md deleted file mode 100644 index 1d8f816ac..000000000 --- a/static/docs/sim-support.md +++ /dev/null @@ -1,127 +0,0 @@ -# SIM Support On Public Real Devices - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -SIM & eSIM support enables testing of SMS-driven use cases on real mobile devices, such as **OTP verification** and **two-factor authentication** workflows. By automating SIM interactions, teams can validate SMS handling, **verify phone numbers**, and confirm secure onboarding processes without relying on physical test phones. - -**SMS-based authentication** is one of the most widely used security and identity verification methods. Automating these flows ensures consistent validation, reduces manual effort, and helps uncover edge cases early in the testing cycle. - -With **TestMu AI**, you can automate SIM and eSIM scenarios on **Real Devices**. From requesting SIM-enabled devices to retrieving phone numbers and polling incoming SMS for OTP validation, TestMu AI provides an end-to-end setup for testing SMS-driven features in your automation suite. - -> To enable this for your organization, contact us via **24×7 chat support** or email **support@testmuai.com**. - -## Use Cases - -- **OTP & Two-Factor authentication**: Automate SMS receipt and validation to test secure login or sign-up workflows. -- **Phone Number Retrieval**: Programmatically fetch the device’s number at runtime to trigger verification flows. -- **SMS Interrupt Testing**: Inject incoming SMS messages during a test session to validate in-app handling of message interrupts. -- **Messaging Workflows**: Verify outbound and inbound SMS flows as part of automated Appium tests. - -## Supported Countries & SIM Availability -TestMu AI provides SIM and eSIM-enabled devices across **specific countries**, allowing you to test SMS-driven workflows under real carrier conditions. The **availability** of physical SIMs or eSIMs varies by region and carrier, ensuring your automation scripts can validate authentication flows, messaging, and phone number handling in environments that closely resemble real-world usage. - -| Country | Carriers | SIM Type Availability | -|---------|-----------------------|-----------------------------| -| Ireland | Three | Physical SIM / eSIM (iOS only) | -| India | Airtel, Jio, Vodafone | Physical SIM | -| USA | T-Mobile, Verizon, AT&T | Physical SIM / eSIM (iOS only) | - -## Capabilities -When automating SIM or eSIM scenarios on real devices, you can **customize** your test session using specific capabilities. These capabilities allow you to request **SIM-enabled devices** and define preferences such as region, carrier, or eSIM usage. The table below summarizes the key capabilities available for your automation scripts. - -| Capability | Type | Default | Description | -|---------------|---------|---------|-------------| -| **enableSim** | Boolean | false | Provision a SIM-enabled device. Must be set to `True` to request SIM support. | -| **simOptions**| Object | None | Define SIM preferences such as region, carrier, or eSIM request. These parameters are nested inside `simOptions`. | - -### simOptions Configurations - -| Configuration | Type | Example | Description | -|-----------|------------|--------------------------|-------------| -| region | String | "IND", "USA" ,"IRL" | ISO country code for SIM region. | -| carrier | String | ["airtel"] or ["jio","vodafone"] | Select preferred carrier(s). | -| esim | Boolean | true | Request an eSIM-enabled device (iOS only). | - -Region and carrier values are case-sensitive. Make sure to enter them exactly as shown in the example above. -:::: - -## SIM support Workflow in Automation Test - -### Step 1: Add SIM Capabilities - -To enable SIM & eSIM support, include the following capability in your automation session as highlighted in the Capabilities: - -```python title="iOS(.ipa)" -desired_caps = { -'platformName': 'iOS', -'platformVersion': '16.0', -'deviceName': 'iPhone 14', -'app': 'YOUR_APP_ID', -'enableSim': True, -'simOptions': { -'region': 'IND', -'carrier': ['airtel', 'vodafone'], -'esim': True -} -} -``` - -```python title="Android(.apk)" -desired_caps = { -'platformName': 'Android', -'platformVersion': '16', -'deviceName': 'Galaxy S25', -'app': 'YOUR_APP_ID', -'enableSim': True, -'simOptions': { -'region': 'IND', -'carrier': ['airtel', 'vodafone'] -} -} -``` - -### Step 2: Retrieve SIM Properties at Runtime - -```python -driver.execute_script( -"lambda_executor: { \"action\": \"deviceInfo\", \"arguments\": { \"deviceProperties\": [\"simOptions\"] } }" -) -``` -Sample Response: -``` JSON -{ -"simOptions": { -"phoneNumber": "+15555551234", -"region": "USA", -"esim": true, -"carrier": "TMobile" -} -} -``` -### Step 3: Validate SMS Workflows -- Trigger an SMS to the device using your backend/system. -- Poll and read incoming SMS messages via Appium script. -- Extract OTP codes or verification text to validate end-to-end flow - -### Session Validation & Error Handling -If SIM-enabled devices matching your preferences are unavailable, sessions may fail with errors such as: -- **No match**: There’s no device that fits the SIM settings you selected (for example, region or eSIM options). Try changing your preferences or broadening your criteria. -- **Busy devices**: All devices that match your settings are currently in use. You can either wait for one to become available or pick a different device - -## Supported Devices - -| Device | OS Version | -|--------------------------|-------------| -| Samsung Galaxy S25 Ultra | Android 15 | -| Samsung Galaxy S25 | Android 15 | -| Samsung Galaxy S24 | Android 14 | -| Samsung Galaxy S23 | Android 13 | -| Google Pixel 9 | Android 15 | -| Google Pixel 7 | Android 13 | -| Google Pixel 6 | Android 12 | -| iPhone 16 Pro | iOS 18 | -| iPhone 15 Pro | iOS 17 | -| iPhone 14 | iOS 16 | -| iPhone 14 Pro | iOS 16 | -| iPhone 13 | iOS 15 | -| iPhone 13 Mini | iOS 15 | diff --git a/static/docs/simplifyqa-integration.md b/static/docs/simplifyqa-integration.md deleted file mode 100644 index 59ab2906e..000000000 --- a/static/docs/simplifyqa-integration.md +++ /dev/null @@ -1,38 +0,0 @@ -# How to integrate SimplyfyQA With TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -SimplifyQA is a test automation platform designed to leverage AI for creating, executing, and managing automated tests with ease. The platform's codeless approach democratizes test automation, allowing QA teams to achieve broader test coverage and faster release cycles without the need for extensive programming skills. SimplifyQA aims to streamline the entire QA workflow with its comprehensive suite of features supporting web, mobile, and API testing. - -This document will guide you step-by-step method to integrate the SimplifyQA with TestMu AI's Real Device. - -## Prerequisites -- SimplifyQA subscription -- SimplifyQA agent installed in Local Machine -- TestMu AI Subscription -- Required .apk or .ipa files - -## Steps to integrate SimplifyQA - -### Step 1: Fetch your TestMu AI Credentials -- Login to TestMu AI account and click on the profile icon. -- Navigate to **Account Settings** > **Password and Security**. Copy the provided **Username** and **Access Key** - -### Step 2: Setup your SimplifyQA platform -- Login to SimplifyQA as non-admin user. Click on the **Settings** icon > **Mobile Settings** . -- Go to Cloud Setup and navigate to TestMu AI tab. Enter details copied from TestMu AI and save the settings: - - Username – Your TestMu AI Username - - AccessKey – Your TestMu AI Access Key - - Build – Reference build numbers - -### Step 3: Upload your application -- Go to TestMu AI's **Real Device** > **App Testing** and upload your desired **.apk** file. -- After successfully uploading the APK, click on the settings icon and copy the provided **App ID**. - -### Step 4: Trigger your tests from SimplifyQA -- Record your test cases in SimplifyQA. Go to the Test Case module > click on **+** icon and select the **Mobile** to record your test cases. -> **Note:** You should use **Cloud Start App** method instead of **Launch App** method. -- Click on test data and paste the App ID copied from Lambda Test in **APP Id** parameter. -- In the Execute Tab, choose **Cloud Automation**. Click on TestMu AI. -- Select Release, Sprint. In the Devices dropdown choose the device of your choice and click on Execute button. -> To view the TestMu AI execution, navigate to [automation](https://www.testmuai.com/login/?redirectTo=https://appautomation.lambdatest.com/build) in the left panel in TestMu AI. diff --git a/static/docs/single-sign-on.md b/static/docs/single-sign-on.md deleted file mode 100644 index c7e7219e4..000000000 --- a/static/docs/single-sign-on.md +++ /dev/null @@ -1,37 +0,0 @@ -# Getting Started With Single Sign On (SSO) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Single sign-on is an authentication feature that allows you to access multiple applications with one set of login credentials. Once you log in, you don't need to login repeatedly for every application linked to a particular application. TestMu AI supports Security Assertion Markup Language (SAML) that allows you to create a team account and authenticate to the TestMu AI dashboard with Identity Provider (IdP). - -> SSO is supported in our Enterprise Plans. Please reach out to your account manager or [support@testmuai.com](mailto:support@testmuai.com) for questions. - -## Benefits Of Authenticating Single Sign On (SSO) - -Here are the following benefits of authenticating a Single sign-on to the TestMu AI dashboard. - -* **Security Enhancements**: Your team members can access the TestMu AI dashboard directly without creating a password. Also, you can leverage several authentication decisions provided by an identity provider(IdP) like security protocols, multi-factor authentication, etc. - -* **Easy to manage**: SSO consolidates passwords and user data, making it easier to access different platforms and resources. New team members can leverage Just-in-Time account provisioning to leverage login to the TestMu AI dashboard instantly. Also, with Just-in-Time (JIT), you can revoke TestMu AI dashboard access from one central place. - -## Feature Of Single Sign On (SSO) - -TestMu AI provides the support for the below Single sign-on features. - -* **Configure SSO**: You can configure SSO through multiple options. You can set up TestMu AI accounts to enable SSO for all team members or enable sign-in using SSO or by email and password. - -* **Just-in-Time Provisioning**: Just-in-Time provisioning automates user account creation when the user first tries to perform SSO and the user doesn't have an existing TestMu AI account. - -* **Identity provider(IdP-initiated) SSO**: You can directly authenticate with identify service provider like Okta, OneLogin, Azure AD. You need to ensure that the identity provider supports the Service Provider(SP-initiated) SSO. - -## Enable TestMu AI SSO - -**Step 1:** Sign in to your TestMu AI account. Don't have an account, register for free. - -**Step 2:** Click on the **User Icon** on the top right and select **Organization Settings** from the dropdown. - -**Step 3:** Head to the **Security** tab and click the **SSO** option. - -You can contact the **Support Team**, who will help you enable the SSO feature for your TestMu AI account. - -> That's all you need to know about Single sign-on(SSO) authentication feature.In case you have any questions please feel free to reach out to us via the **24/7 chat support** or email us over [support@testmuai.com](mailto:support@testmuai.com). diff --git a/static/docs/site-to-site-ipsec-vpn-setup.md b/static/docs/site-to-site-ipsec-vpn-setup.md deleted file mode 100644 index a21ff298f..000000000 --- a/static/docs/site-to-site-ipsec-vpn-setup.md +++ /dev/null @@ -1,181 +0,0 @@ -# IPsec Site-to-Site VPN Setup Guide - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI supports connectivity via IPsec Site-to-Site VPN (S2S VPN) for clients who need to test web applications hosted on internal/private networks that are not publicly accessible. This allows TestMu AI cloud infrastructure to reach your privately hosted test environments directly over an encrypted, authenticated tunnel, just as if both networks are on the same LAN. - -**Executive Summary** -An IPsec Site-to-Site VPN creates a permanent, encrypted tunnel between your network gateway and TestMu AI cloud, enabling secure access to internal staging environments, development servers, and private applications without exposing them to the public internet. - -**Client Action Required** -Setting up an IPsec Site-to-Site VPN requires coordination with your organization's internal **Network and Security Team**. Your network/security team will need to configure your on-premises VPN gateway/firewall device, define firewall rules, and coordinate IP addressing with the TestMu AI network team. Please engage your internal IT/Security team before initiating this process. - -## What is IPsec Site-to-Site VPN? - -An IPsec Site-to-Site (S2S) VPN creates a permanent, encrypted tunnel between two network gateways, allowing devices on both networks to communicate securely as if they were on the same Local Area Network (LAN). - -### Key Concepts - -- **Gateway-to-Gateway**: Unlike remote access VPNs (user-to-device), S2S VPN connects entire networks. It does not require software installation on individual test machines. -- **Persistence**: The tunnel is "always on," automatically re-establishing connection if interrupted. -- **Technology**: It utilizes the IPsec suite, specifically IKEv2 for secure key exchange and ESP in Tunnel Mode for data encapsulation and encryption. -- **Use Case**: It enables TestMu AI cloud nodes to access private URLs (e.g., `http://192.168.10.5/myapp` or `http://internal.company.com`) that are not accessible via the public internet. - -## Network Architecture - -The diagram below illustrates the secure connection between your internal network and TestMu AI's cloud infrastructure. - -### Traffic Flow Steps - -1. TestMu AI test node initiates HTTP request to internal URL (e.g., `http://10.10.1.50`). -2. Traffic is routed to TestMu AI VPN Gateway based on routing table. -3. VPN Gateway encapsulates packet in IPsec ESP Tunnel Mode (encrypted). -4. Encrypted packet traverses the public internet to Client's Public IP. -5. Client's VPN Gateway decrypts the packet (ESP decapsulation). -6. Request is forwarded to the internal web server. -7. Response follows the reverse path back to the test node. - -## IPsec Protocol Suite - -### IPsec Overview -IPsec (RFC 4301) secures IP communications. For S2S VPNs, we use **Tunnel Mode**, which encapsulates the entire original IP packet within a new IP packet, protecting both the payload and the original header. - -### IKEv2 (RFC 7296) -Internet Key Exchange Version 2 is responsible for negotiation and authentication. - -- **Phase 1 (IKE_SA_INIT)**: Negotiates cryptography (DH group, encryption) and generates keying material. -- **Phase 2 (IKE_AUTH)**: Authenticates peers and establishes Child SAs for traffic. -- **Ports**: UDP 500 (Initial), UDP 4500 (NAT Traversal). - -### ESP (RFC 4303) -Encapsulating Security Payload provides confidentiality (encryption) and integrity. It operates as IP Protocol 50. - -### NAT Traversal (NAT-T) -Standard ESP cannot pass through NAT devices. RFC 3948 solves this by encapsulating ESP packets inside UDP packets on port 4500. This is automatically detected and negotiated during IKEv2 setup. - -## Recommended Security Parameters - -| Parameter | Recommended Value | Notes | -|-----------|-------------------|-------| -| **IKE Version** | IKEv2 | IKEv1 is deprecated | -| **Encryption (Phase 1)** | AES-256-GCM or AES-256-CBC | NIST SP 800-77r1 approved | -| **Encryption (Phase 2)** | AES-256-GCM | Preferred AEAD cipher | -| **Integrity (Phase 1)** | HMAC-SHA-384 | SHA-1 is deprecated | -| **Diffie-Hellman Group** | Group 14 (Min) / Group 20 (Preferred) | Groups < 14 are insecure | -| **PFS** | Enabled | Perfect Forward Secrecy | -| **SA Lifetime (Phase 1)** | 28800 seconds (8 hours) | Standard enterprise setting | -| **SA Lifetime (Phase 2)** | 3600 seconds (1 hour) | Renegotiate frequently | -| **DPD** | Enabled | Dead Peer Detection | - -## Firewall / Port Requirements - -The following ports must be permitted on your external firewall between the Client Gateway Public IP and the TestMu AI Gateway IP. - -| Protocol | Port | Direction | Purpose | -|----------|------|-----------|---------| -| UDP | 500 | Bidirectional | IKE/ISAKMP Negotiation | -| UDP | 4500 | Bidirectional | NAT-Traversal & ESP Encapsulation | -| IP Proto 50 | N/A | Bidirectional | ESP (If no NAT is present) | -| TCP/UDP | 80, 443 | Client → TestMu AI | Allowed Test Traffic (HTTP/HTTPS) | - -## S2S VPN vs. SSH Tunnel Comparison - -| Feature | S2S IPsec VPN | TestMu AI SSH Tunnel | -|---------|---------------|----------------------| -| Connectivity | Gateway-to-Gateway (Permanent) | Client-Initiated (Session-based) | -| Client Software | None (Router Config) | Requires LT Binary/App | -| Scope | Entire Network/Subnets | Local Machine only | -| Best For | Enterprise / Permanent Staging | Ad-hoc / Dev Testing | - -## Client Coordination - -**Network Team Involvement Mandatory** -Setting up an IPsec Site-to-Site VPN is **NOT** a self-service process. It requires configuration on your organization's edge firewalls/routers. - -### Required Actions by Client Team - -1. **Gateway Configuration**: Configure on-premise device (Cisco, Fortinet, Palo Alto, etc.) with provided parameters. -2. **Firewall Rules**: Permit UDP 500/4500 from TestMu AI IP. -3. **Routing**: Add static routes directing traffic for TestMu AI subnets into the tunnel. -4. **Security Approval**: Obtain internal change management approval. - -**Timeline**: Typical setup takes 3–10 business days depending on internal client approvals. - -## S2S VPN Requirement Gathering - -To initiate the VPN setup process, please gather the following details from your Network/Security team and share them with TestMu AI support at **support@testmu.ai**. - -### Organization Info - -| Field | Description | Example | -|-------|-------------|---------| -| **Organization Name** | Your company or organization name | Acme Corp | -| **LambdaTest Account Email** | The email associated with your TestMu AI account | admin@acme.com | -| **Network/Security Contact Email** | Email of the technical contact from your network or security team | netops@acme.com | - -### Gateway Details - -| Field | Description | Example | -|-------|-------------|---------| -| **VPN Gateway Vendor** | The vendor/make of your on-premise VPN gateway or firewall device | Cisco, Fortinet, Palo Alto, AWS, Azure, etc. | -| **Gateway Public IP** | The public IP address of your VPN gateway | 203.0.113.10 | -| **Is Gateway behind NAT?** | Whether your gateway sits behind a NAT device. If **Yes**, provide the NAT IP address | No / Yes (specify NAT IP) | - -### Network Addressing - -| Field | Description | Example | -|-------|-------------|---------| -| **Internal Subnets to Expose** | The private subnets (in CIDR format) that TestMu AI test nodes need to reach. List one per line | 10.10.1.0/24, 192.168.5.0/24 | -| **Internal Domain Names** | Internal domain names that should be routed through the VPN tunnel (for DNS routing) | uat.company.internal, app.corp | - -### IPsec Parameters Preference - -| Parameter | Options | Recommended | -|-----------|---------|-------------| -| **IKE Version** | IKEv1 / IKEv2 | IKEv2 (Recommended) | -| **Auth Method** | Pre-Shared Key (PSK) / Certificates | Pre-Shared Key (PSK) | -| **Phase 1 Encryption** | AES-256-GCM / AES-256-CBC / AES-128-GCM | AES-256-GCM (Preferred) | -| **Phase 1 DH Group** | Group 14 / Group 19 (ECP-256) / Group 20 (ECP-384) | Group 20 (ECP-384) | - -**Submit Your Request** -Send the above details to **support@testmu.ai** to submit your VPN setup request. TestMu AI Network Engineering will review and contact you within **2 business days**. - -## Implementation Checklist - -Setup involves parallel work from both teams. Use the checklists below to track progress. - -### Client Network Team - -- Identify VPN Gateway device -- Obtain InfoSec approval -- Complete Requirement Form (see [above](#s2s-vpn-requirement-gathering)) -- Configure Gateway with Parameters -- Open UDP 500/4500 on Firewall -- Configure Static Routes - -### TestMu AI Network Team - -- Provide Public IP & Parameters -- Configure Cloud VPN Gateway -- Provide Cloud Subnet Ranges -- Validate SA Establishment -- End-to-End Connectivity Test - -## Troubleshooting - -| Issue | Possible Cause | Resolution | -|-------|----------------|------------| -| Tunnel not establishing | Firewall blocking UDP 500 | Ensure UDP 500 is open bidirectionally | -| IKE Auth Failed | PSK mismatch | Verify Pre-Shared Key matches exactly | -| TS_UNACCEPTABLE | Subnet mismatch | Verify Traffic Selectors (subnets) match on both sides | -| Tunnel Up, No Traffic | Missing Route | Add route for TestMu AI subnet via tunnel interface | -| Drops Frequently | NAT Timeout | Enable NAT-T keepalives; check DPD | - -## References & Standards - -- RFC 4301: Security Architecture for the Internet Protocol (IPsec) -- RFC 7296: Internet Key Exchange Protocol Version 2 (IKEv2) -- NIST SP 800-77 Rev. 1: Guide to IPsec VPNs -- FIPS 140-3: Security Requirements for Cryptographic Modules - -For VPN setup assistance, contact: **support@testmu.ai** diff --git a/static/docs/slack-app.md b/static/docs/slack-app.md deleted file mode 100644 index d68ed2866..000000000 --- a/static/docs/slack-app.md +++ /dev/null @@ -1,75 +0,0 @@ -# Slack App Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The TestMu AI Slack App allows you to initiate real-time tests directly from Slack. Your team can collaborate more effectively by easily communicating on bugs, reducing the feedback loop, and troubleshooting web elements in real-time. - -By integrating TestMu AI with Slack App, - -* Perform Real time testing with access to 3000+ browser and operating systems on the TestMu AI cloud. -* Reuse the generated test cards and relaunch the test with the same configuration. - -*To perform the Real-time testing, you will need to login with your TestMu AI Account. If you don’t have an account on TestMu AI, visit the TestMu AI register page.* - -## Integrating Slack With Your TestMu AI Account - -> -**Note**: To push the Slack App alert notification of the logged bugs to your specified Slack channel, you will need to integrate Slack with your TestMu AI account. - -1. Login to your TestMu AI account and visit **Integrations**. - -2. From the **COMMUNICATION** category, select Slack. - -3. You'll be redirect to OAuth page. Enter the Slack channel on which you wish to receive Slack alert notifications. - -4. Click on **Allow**. - -Slack will be integrated successfully with your TestMu AI account. You'll also receive the notification and the confirmation email of the same. - -## Adding TestMu AI From Slack App - -**Step 1**: Open the Slack App. - -**Step 2**: From the left menu, navigate to **Apps** and click on Add (+) icon. - -**Step 3**: Search for TestMu AI and then click on the TestMu AI block. - -Upon successful integration, the TestMu AI App will be added to Slack. - -## Performing Real Time Testing From Slack App - -**Step 1**: From the Apps dropdown, click on the TestMu AI App and select **Run Realtime Test**. - -**Step 2**: A pop-up box will appear where you need to provide the test environment details like Test URL, Device, Browser, Version, OS and Resolution. - -**Step 3**: After entering the test environment details, click on **Submit**. - -**Step 4**: After clicking on Submit button, you'll receive a Slack notification Open the notification and you'll find a Real-time test card that includes the test environment details that you just selected. - -To launch the Realtime test, click on **Run Test**. - -**Step 6**: You'll be redirected to the TestMu AI cloud-based VM where you can perform real-time testing of your websites and web apps. You can also perform one-click bug logging, capture full page screenshots, [geolocation testing](https://www.testmuai.com/geolocation-testing), and more. - -To know more about Real-time Testing, refer to the documentation [Real Time Browser Testing](/docs/getting-started-with-desktop-browser-real-time-testing/). - -## Running Screenshot Tests From Slack App - -**Deprecated** - -Screenshot testing through the Slack App has been deprecated and is no longer available. This includes both the **Run Screenshot Test** button flow and the earlier `/screenshot` slash command. - -To capture screenshots of a URL, use [Web Scanner](/support/docs/web-scanner-getting-started/) instead, which runs Visual UI scans across browsers and viewports. - -## Removing/Uninstalling The Slack App - -**Step 1**: Navigate to **Apps** from the Slack left menu. - -**Step 2**: Once you hover on TestMu AI App, a cross icon will appear. Click on the Cross icon. - -The TestMu AI Slack App will be removed successfully from Slack. - -**Step 3**: Now visit **Integrations**. From the MY INTEGRATIONS section, click on **Remove** icon under the Slack block. - -The Slack will be uninstalled from your TestMu AI account. - -> That’s all about TestMu AI+Slack App integration. If you have any questions or require any further information, you can reach out to us at our **24/7 chat portal** or drop us a mail at support@testmuai.com. diff --git a/static/docs/slack-integration.md b/static/docs/slack-integration.md deleted file mode 100644 index de66ff205..000000000 --- a/static/docs/slack-integration.md +++ /dev/null @@ -1,43 +0,0 @@ -# Slack Integration With TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Slack is a cloud-based messaging platform designed for teams to collaborate in real time. It offers organized conversations through channels, instant messaging, file sharing, and powerful integrations with developer and productivity tools, making it ideal for DevOps and QA workflows. - -The TestMu AI Slack Integration enables users to share or push bugs to their specified channel, capture screenshots, annotations, and issue information. - -> Slack Integration with TestMu AI is available for freemium as well as premium plan. - -## Integrate Slack from your TestMu AI Account - -**Step 1:** Login to your TestMu AI account. You should have Admin or User level access to see and install integrations. - -**Step 2:** Click on Settings -> Integrations -> Communication. - -**Step 3:** Click on **Connect** button of `Slack` block. - -**Step 4:** Now click on **install** button. If you’re already logged into your Slack account, you just have to post to a channel to confirm your identity. However, if you’re not logged into your Slack account, provide the Slack URL of your workplace. - -## Lodge your First Bug - -> Note: If you are using Slack for the first time, then make sure to create a channel for yourself. It is a pre-requisite in order to push messages from your TestMu AI account. - -**Step 1:** Start with any type of testing, for the demo purpose we are going with the **Real Time Testing**. - -**Step 2:** Enter your Project URL and configure for browser and operating system of your choice & hit **Start**. - -**Step 3:** After the VM is launched and operable. You can perform testing on your web-app for finding bugs. If a bug gets revealed, then you need to click on the Bug icon from the left panel for capturing a screenshot of the same. - -**Step 4:** After clicking on "Mark as Bug" button, a specific form would open up. Fill the fields as per your requirement and check mark the option for "Send to Slack". - -**Step 5:** Click on **Mark as Bug** button. Now go to your respective slack channel and you can check a notification is sent for the same. - -## Uninstall Slack Integration - -**Step 1:** Login to your TestMu AI account. You should have Admin or User level access to see and install integrations. - -**Step 2:** Click on Settings -> Integrations -> Communication. - -**Step 3:** Click on the **Remove** button. - ->That was all you need to know for TestMu AI + Slack Integration. Increase your productivity multifold with our integrations. If you still have any questions for us, please feel free to let us know. Our experts are always **available on chat** to help you out with any roadblock regarding our product. Happy testing! diff --git a/static/docs/smart-heal-appium.md b/static/docs/smart-heal-appium.md deleted file mode 100644 index 0d0853a67..000000000 --- a/static/docs/smart-heal-appium.md +++ /dev/null @@ -1,126 +0,0 @@ -# AI-Powered Smart Heal for Automation Tests - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI’s **Smart Heal** uses **AI-powered algorithms** to automatically detect and recover from locator failures during test execution. It intelligently detects missing elements, analyzes the UI in real time, and applies the closest valid match, keeping tests running smoothly despite UI changes. Both the **original and recovered locators** are logged for full visibility through the TestMu AI dashboard. - -> Smart Heal is currently in **closed beta**. To enable it for your organization, reach out via **24×7 chat** or email **support@testmuai.com**. - -## Use Cases - -- **Frequent UI Updates**: When product teams ship fast and locator changes are common, Smart Heal prevents brittle tests from breaking on every release and make shipments fast. -- **CI/CD Reliability**: Reduce flaky build failures by automatically recovering from locator issues in pipelines. -- **Maintenance Reduction**: Spend less time fixing scripts manually and more time building new coverage.Deploy first and fix later. -- **Audit & Debugging**: Use healed locator logs and screenshots to understand changes quickly and improve your scripts over time. - -## Smart Heal Workflow - -1. **Baseline Creation** - Smart Heal requires at least one successful (**Passed**) test run to establish a baseline. On this initial run, TestMu AI captures a **baseline snapshot** of all element locators in your script. Make sure to include the [TestMu AI Hook](/support/docs/appium-testmu-hooks/#adding-custom-status--remark) to explicitly mark tests as passed during execution. - - The **project name** and **test name** must remain the same across runs for the baseline to apply. - - Since Smart Heal uses AI-driven analysis, expect executions to take slightly longer than usual. - -2. **Baseline Update** - After each successful test run, Smart Heal automatically updates your baseline to reflect the latest passed build. This ensures the most recent valid UI state is used as reference for future healing attempts. - -3. **Detection and Healing** - In subsequent runs, if an element cannot be found due to **UI or DOM changes**, Smart Heal triggers automatically, leveraging **AI-driven analysis** of element attributes, hierarchy, and **visual cues** to find the closest valid match in the updated UI. - -4. **Retry with Healed Locator** - When a likely match is found, the test step retries with the **healed locator**. These adjustments apply at runtime so the test flow continues without interruption. Both the original and healed locators are logged for full transparency. - -5. **Fallback and Suggestions** - If Smart Heal cannot confidently identify an alternative, it records **AI-driven suggestions** in the dashboard. These insights help you quickly update or strengthen your locators to avoid repeated failures in future runs. - -## Smart Heal in Automation Tests - -### 1. Upload Your App - -Before enabling Smart Heal, ensure your app is uploaded to TestMu AI. - -1. Follow the [Upload Your Application](/support/docs/upload-apps-on-real-device-cloud/) guide. -2. Once uploaded, **note the App ID** returned by the API or dashboard. -3. Use this **App ID** in the `"app"` capability in your automation script. - -### 2. Enable Smart Heal with Capabilities - -To enable Smart Heal, add `"smartHeal": true` to your desired capabilities in your Appium test script. - -```python -desired_caps = { -"deviceName": "iPhone 16", -"platformName": "iOS", -"platformVersion": "18", -"isRealMobile": True, -"app": "YOUR_APP_URL", -"build": "Smart Heal iOS", -"name": "Sample Smart Heal Test", -# highlight-next-line -"smartHeal": True -} -``` - -```python -desired_caps = { -"deviceName": "Galaxy S25", -"platformName": "Android", -"platformVersion": "16", -"isRealMobile": True, -"app": "YOUR_APP_URL", -"build": "Smart Heal Android", -"name": "Sample Smart Heal Test", -# highlight-next-line -"smartHeal": True -} -``` - -You can generate capabilities for your test requirements with the help of our inbuilt [**Capabilities Generator tool**](https://www.testmuai.com/capabilities-generator/). For more details, please refer to our guide on [**Desired Capabilities in Appium**](/support/docs/desired-capabilities-in-appium/). - -### 3. Enable Smart Heal with Runtime Hooks - -You can also control Smart Heal dynamically during test execution using runtime hooks. This is useful when you want healing active in specific phases or after major UI changes. - -```java -// Stop Smart Heal -driver.executeScript("lambda-heal-stop"); - -// Start Smart Heal -driver.executeScript("lambda-heal-start"); - -``` - -### 4. Running Your Tests - -Once your app is uploaded and Smart Heal is enabled (either via capabilities or runtime hooks), execute your test script as usual with your preferred automation framework. Smart Heal will monitor for locator failures during the run, apply healing when possible, and log all details to the TestMu AI dashboard for review. - -## Viewing Results in Dashboard - -### Accessing the Dashboard -Your test results are displayed on the [TestMu AI App Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://appautomation.lambdatest.com/build). - -### Filtering Healed Builds -To display only healed builds, click on the **Configure** option at the top of the dashboard. - -- The following image shows the **Configure box**, with the **Features** tab highlighted: - -- When you open the **Features** tab, a pop-up appears where you can enable **Auto-Heal** to filter and display only executions where Smart Heal was applied: - -- Once filtering is applied, the dashboard highlights all healed elements in your tests. In this view, healed elements are marked clearly, while those that could not be healed are highlighted in red: - -### Hovering Over Healed Builds -Each healed build has an associated icon. Hovering over this icon provides a tooltip that shows a brief summary of the session and the healing actions performed. - -### Session Details -Access detailed execution logs that clearly differentiate between **original and healed selectors**, along with AI suggestions, and compare before-and-after screenshots. These insights help you understand how the Auto-Heal mechanism worked during execution and guide you in refining locators over time. - -### AI Review on Failures -When a test case fails, the dashboard provides **AI-powered analysis and suggestions** to help you quickly identify root causes and fix issues. - -Smart Heal delivers the best results when applied to **static components** such as buttons or form fields, where locators remain relatively consistent across runs. - -## Related Docs - -- [Selenium Auto Healing](/support/docs/auto-healing/):Auto-heal for Selenium web automation tests -- [Playwright Auto Healing](/support/docs/playwright-auto-healing/):Auto-heal for Playwright web automation tests -- [Auto Healing in HyperExecute](/support/docs/hyperexecute-auto-healing/):Auto-heal for tests executed via HyperExecute -- [Auto-Heal in KaneAI](/support/docs/kaneai-auto-heal/):AI-native auto-heal with natural language understanding for KaneAI-authored tests diff --git a/static/docs/smart-ui-build-options.md b/static/docs/smart-ui-build-options.md deleted file mode 100644 index f39694b97..000000000 --- a/static/docs/smart-ui-build-options.md +++ /dev/null @@ -1,116 +0,0 @@ -# Build Configuration and Options for Smart UI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Using the TestMu AI platform, perform regression testing in just one click and explore various possible solutions for grouping your screenshots into different builds and map as per your testing suite needs. - -## Enabling build mapping for multiple screenshots - -If you have multiple screenshots running the same test suite and want to run the comparison for the same test suite. We have to enable the mapping with the `smartUI.build` to be added your capabilities. - -```js title="Please add the following to your capabilities configuration" -let capabilities = { -// highlight-next-line -"smartUI.build": // Please add your build name of choice here -} -``` - -```json title="Please make the changes in lambdatest-config.json" -"smart_ui": { -// highlight-next-line -"build": // Please add your build name of choice here -} -``` - -```js title="Please add the following in in your capabilities configuration " - -const capabilities = { -'LT:Options': { -// highlight-next-line -"smartUIBuildName": // Please add your build name of choice here - -}} -``` - -```js title="Please add the following in in your capabilities configuration " - -const capabilities = { -'LT:Options': { -// highlight-next-line -"smartUIBuildName": // Please add your build name of choice here - -}} -``` - -```js title="Please add the following in in your capabilities configuration " - -const capabilities = { -'LT:Options': { -// highlight-next-line -"smartUIBuildName": // Please add your build name of choice here - -}} -``` - -## Enabling build as a baseline - -If you want to mark your build as a baseline from your test suite while executing tests then you have to enable the below mentioned capability configuration needs to be enabled. - -```js title="Please add the following to your capabilities configuration" -let capabilities = { -// highlight-next-line -"smartUI.baseline": true / false, // Enable if you want to update to a new baseline build -}; -``` - -```json title="Please make the changes in lambdatest-config.json" -"smart_ui": { -// highlight-next-line -"baseline": true/false // Enable if you want to update to a new baseline build -} -``` - -```js title="Please add the following in in your capabilities configuration " - -const capabilities = { -'LT:Options': { -// highlight-next-line -"smartUIBaseline": true // Please add if you want to make this build as baseline - -}} -``` - -```js title="Please add the following in in your capabilities configuration " - -const capabilities = { -'LT:Options': { -// highlight-next-line -"smartUIBaseline": true // Please add if you want to make this build as baseline - -}} -``` - -```js title="Please add the following in in your capabilities configuration " - -const capabilities = { -'LT:Options': { -// highlight-next-line -"smartUIBaseline": true // Please add if you want to make this build as baseline - -}} -``` - -## Build Features - -The following are the list of features that are available for the build process in SmartUI: - -- All the screenshots are grouped together as a one build. -- The screenshots are mapped from the **baseline** to the **Comparison screenshot** in the same sequential pattern as per the baseline build. -- Add additional screenshots if needed for the existing build by using the same build name in the capabilities. This will add new screenshots to the build. -- You can also update any build of choice as a new **Baseline** build. - -We currently only support one baseline for one project in our current version. We recommend you to create a new project for multiple baseline builds - -## Build Status Information - -We are provide multiple build status options for the project owners to check the status of builds and take required actions to the screenshots by the approver(s). diff --git a/static/docs/smart-ui-cypress.md b/static/docs/smart-ui-cypress.md deleted file mode 100644 index 1f6a9d354..000000000 --- a/static/docs/smart-ui-cypress.md +++ /dev/null @@ -1,116 +0,0 @@ -# Getting started with SmartUI using Cypress On TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Using the TestMu AI platform, perform regression testing in just one click and find Visual UI Regression bugs easily with the help of Smart Testing. This documentation will act as your step-by-step guide in performing successful Visual Regression tests. - -## Prerequisites for running SmartUI with Cypress - -- Basic understanding of Cypress is required. -- Cypress version should be higher than 9.5.0. Click [here](https://docs.cypress.io/guides/references/changelog) to know more -- Login to [TestMu AI SmartUI](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) with your credentials. -- Copy `LT_USERNAME` and `LT_ACCESS_KEY` credentials from `Access Key` button on the top right of the dashboard. - -The following steps will guide you in running your first Visual Regression test on TestMu AI platform - - -## Steps to create a SmartUI Project - -The first step is to create a project with the application in which we will combine all your builds run on the project. To create a SmartUI Project, follow these steps: - -1. Go to [Projects page](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) -2. Click on the `new project` button -3. Select the platform as Web for executing your `Cypress` tests. -4. Add name of the project, approvers for the changes found, tags for any filter or easy navigation. -5. Click on the **Submit**. - -## Steps to run your first test - -**Step 1**: Install the required NPM module for `LambdaTest Cypress CLI` in your project. - -```bash -npm install -g lambdatest-cypress-cli -``` - -**Step 2:** Use `init` command to generate the sample configuration files. - -```bash -lambdatest-cypress init -``` - -**Step 3:** Define SmartUI configuration in the `lambdatest-config.json` file of your Cypress project as shown in the below code snippet. - -```json title="/cypress-project/lambdatest-config.json" -{ -"lambdatest_auth": { -"username": "", -"access_key": "" -}, -"browsers": [ -{ -"browser": "Chrome", -"platform": "Windows 10", -"versions": ["latest-1"] -}, -{ -"browser": "Firefox", -"platform": "Windows 10", -"versions": ["latest-1"] -} -], -"run_settings": { -"cypress_config_file": "cypress.json", -"build_name": "build-name", -"parallels": 2, -"specs": "./cypress/integration/examples/*.spec.js", -"ignore_files": "", -"feature_file_suppport": false, -"network": false, -"headless": false, -"reporter_config_file": "", -"npm_dependencies": { -"cypress": "9.0.0" -}, -"smart_ui": { -// highlight-start -"project": "", // Add your SmartUI Project Name here -"build": "", // Please add your build name of choice here -"baseline": false,// Enable if you want to update to a new baseline build -// highlight-end -} -}, -"tunnel_settings": { -"tunnel": false, -"tunnel_name": null -} -} -``` - -**Step 4:** Add **cy.screenshot()** method in your test script to capture the screenshots with the desired screenshot name. - -```js -describe("Cypress test with Visual Regression", () => { -it("Example test", () => { -cy.visit("https://demo.lambdatest.com/"); -// highlight-next-line -cy.screenshot(""); //Add desired Cypress screenshot name -}); -}); -``` - -**Step 5:** Run the test on the TestMu AI platform by executing the below-run command: - -```bash -lambdatest-cypress run -``` - -**Step 6:** You can now see the SmartUI dashboard to view the results. - -For additional information about Cypress framework please explore the documentation [here](/support/docs/getting-started-with-cypress-testing/) - -## Advanced Options for Screenshot Comparison - - **Build Configuration** - If you have multiple screenshots running the same test suite and want to run the comparison for the same test suite, want to add a build as a baseline from your test suite or need to access more SmartUI Build Config Options, click [here](/support/docs/smart-ui-build-options/). - - **Advanced Test Settings** - We offer multiple options for comparing the **Baseline** and the **Test Output** screenshots captured during your automation testing suites. To know more about our advanced P2P comparison options, click [here](/support/docs/test-settings-options/). - - **HTML DOM Config and Options** - In case if you have any dynamic elements that are not in the same position across test runs, you can ignore or select a specific area to be removed from the comparison. For accessing such HTML DOM Config and Options, click [here](/support/docs/html-dom-smartui-options/). diff --git a/static/docs/smart-ui-storybook.md b/static/docs/smart-ui-storybook.md deleted file mode 100644 index dc8bf1ee6..000000000 --- a/static/docs/smart-ui-storybook.md +++ /dev/null @@ -1,444 +0,0 @@ -# Getting started with SmartUI using Storybook On TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Using the TestMu AI platform, perform regression testing in just one click and find Visual UI Regression bugs easily with the help of Smart Testing. This documentation will act as your step-by-step guide in performing successful Visual Regression tests. - -## Prerequisites for running SmartUI with StoryBook - -- Basic understanding of [StoryBook](https://storybook.js.org/docs/react/get-started/introduction) is required. -- Node.js v20.3+ installed (required for SmartUI CLI v4.x.x) -- StoryBook version installed should be higher than `6.4.0.` Click [here](https://github.com/storybookjs/storybook/releases) to know more - -If you face any problems executing tests with SmartUI-CLI `versions >= v4.x.x`, upgrade your Node.js version to `v20.3` or above. - -- Login to [TestMu AI SmartUI](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) with your credentials. - -The following steps will guide you in running your first Visual Regression test on TestMu AI platform - - -## Steps to create a SmartUI Project - -The first step is to create a project with the application in which we will combine all your builds run on the project. To create a SmartUI Project, follow these steps: - -1. Go to [Projects page](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) -2. Click on the `new project` button -3. Select the platform as CLI for executing your `StoryBook` tests. -4. Add name of the project, approvers for the changes found, tags for any filter or easy navigation. -5. Click on the **Submit**. - -## Steps to run your first test - -**GitHub Sample** - -Please try our **StoryBook SmartUI** GitHub sample repository for trying an example. - -```bash -git clone https://github.com/LambdaTest/smartui-storybook-sample.git -``` - -### **Step 1**: Install the Dependencies - -Install required NPM modules for `LambdaTest SmartUI StoryBook CLI` in your **Frontend** project. - -```bash -npm install @lambdatest/smartui-storybook -g -``` - -### **Step 2:** Setup with StoryBook - -Add the following to your .storybook/main.js. You can read more about this here Storybook [Feature flags](https://storybook.js.org/docs/react/configure/overview#feature-flags) - -```js -module.exports = { -features: { -// Required for Storybook < v9 -buildStoriesJson: true, -}, -}; -``` - ->**Note**: `buildStoriesJson` is available for Storybook versions below v9. - -#### Storybook v9+ Play Function Support - -SmartUI supports Storybook's `play` function (available in Storybook v9+) for interactive component testing. The `play` function allows you to interact with components before capturing screenshots. - -**Example with Play Function:** - -```js title="Button.stories.js" - -export default { -title: 'Components/Button', -component: Button, -}; - -export const InteractiveButton = { -play: async ({ canvasElement }) => { -const canvas = within(canvasElement); -const button = canvas.getByRole('button', { name: /click me/i }); - -// Interact with the button before screenshot -await userEvent.click(button); -await expect(button).toHaveTextContent('Clicked!'); -}, -}; -``` - -**Best Practices for Play Functions:** - -- Use `play` functions to set up component states before screenshots -- Wait for async operations to complete using `waitFor` or `findBy` queries -- Avoid animations or transitions that might cause timing issues -- Use `waitForTimeout` in SmartUI config if components need additional render time after play functions - -#### Storybook Globals (Themes) Configuration - -SmartUI supports Storybook's global decorators and parameters, including theme switching. You can configure themes in your `.smartui.json` file. - -**Configuration Example:** - -```json title=".smartui.json" -{ -"storybook": { -"browsers": ["chrome", "firefox", "safari", "edge"], -"viewports": [[1920, 1080]], -"backgroundTheme": "light", // Options: "light", "dark", or "both" -"useGlobals": true, // Enable global decorators and parameters -"waitForTimeout": 0 -} -} -``` - -**Theme Options:** - -- `"light"`: Capture stories in light theme only -- `"dark"`: Capture stories in dark theme only -- `"both"`: Capture stories in both light and dark themes (creates separate screenshots) - -**Example Story with Theme Globals:** - -```js title="Card.stories.js" -export default { -title: 'Components/Card', -component: Card, -parameters: { -backgrounds: { -default: 'light', -values: [ -{ name: 'light', value: '#ffffff' }, -{ name: 'dark', value: '#1a1a1a' }, -], -}, -}, -globalTypes: { -theme: { -description: 'Global theme for components', -defaultValue: 'light', -toolbar: { -title: 'Theme', -icon: 'circlehollow', -items: ['light', 'dark'], -dynamicTitle: true, -}, -}, -}, -}; - -export const Default = { -decorators: [ -(Story, context) => { -const theme = context.globals.theme || 'light'; -return ( -
- -
-); -}, -], -}; -``` - -**Using Multiple Themes:** - -If you set `"backgroundTheme": "both"` in your SmartUI config, each story will be captured twice - once in light theme and once in dark theme. The screenshot names will be automatically suffixed (e.g., `Card-Default-light.png` and `Card-Default-dark.png`). - -**Note**: When using `"backgroundTheme": "both"`, ensure your Storybook stories properly handle theme switching via globals or decorators. - -### **Step 3:** Configure your Project Token - -Setup your project token shown in the **SmartUI** app after creating your project. - -```bash -export PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -``` - -```bash -set PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -``` - -```powershell -$env:PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -``` - -### **Step 3.1:** Configure Proxy (Optional) - -If you are behind a proxy, you can configure the `SMARTUI_API_PROXY` environment variable. - -```bash -export SMARTUI_API_PROXY=http://172.17.0.1:3128 -``` - -```bash -set SMARTUI_API_PROXY=http://172.17.0.1:3128 -``` - -> **Note**: Replace the IP address and port with the appropriate values for your environment. - -### **Step 4:** Create and Configure SmartUI Config - -You can now configure your project settings on using various available options to run your tests with the SmartUI integration. To generate the configuration file, please execute the following command: - -```bash -smartui config create .smartui.json -``` - -Once, the configuration file will be created, you will be seeing the default configuration pre-filled in the configuration file: - -```json title="/smartUi-storybook-project/smartui.json" -{ -"storybook": { -"browsers": [ -"chrome", -"firefox", -"safari", -"edge", -// Add more browser configuration here -], -"viewports": [ -[1920, 1080] // Add more view ports to capture here -], -"waitForTimeout": 0, // (Optional) Add wait time for the page to load -"include": [], // (Optional) Only compare limited stories -"exclude": [] // (Optional) Don't compare the stories // Apply exclusions/inclusions at directory, sub-directory, or individual story level -} -} -``` - -#### SmartUI StoryBook Config Options - -Please read the following table for more information about the configuration file: - -| Config Key | Description | Usage | -| -------------- | ---------------------------------------------------------------------------------------------------------------------------------- | --------- | -| browsers | You can add all the supported browsers brands here to run your tests for SmartUI. Ex: `"chrome", "firefox", "safari", "edge", etc..` | Mandatory | -| viewports | You can add all the supported browser viewpoints here to run your tests for SmartUI Ex: `[1920, 1080],[width, height] etc..` | Mandatory | -| waitForTimeout | You can add wait time for the page to load DOM of your StoryBook components. This can be added globally to your configuration and to individual stories as well. Ex: `3000` | Optional | -| include | Add the stories which should only be included in SmartUI tests Ex: `"/dashboard/","/features/"` | Optional | -| exclude | Don't compare the stories which should be excluded in SmartUI tests Ex: `"/login/","/marketing/"` | Optional | -| backgroundTheme | Theme for capturing stories. Options: `"light"`, `"dark"`, or `"both"` (captures both themes) Ex: `"light"` | Optional (default: `"light"`) | -| useGlobals | Enable Storybook global decorators and parameters (required for theme switching) Ex: `true` | Optional (default: `false`) | - -SmartUI Storybook testing now supports `Edge` browser. - -**Please Note** -The `waitForTimeout` is an optional configuration. This is available for the versions above `1.1.7` of the NPM CLI. - -If you are using any `async` components in your StoryBook, then you can add the wait time for the page to load the DOM of your StoryBook components. - -This will help you to avoid any false-positive results for your tests. You can add the wait time in milliseconds. This might increase the execution time of your tests. - -For capturing the stories in **full page** without limiting the height to the viewport then in the `viewports` array, you can change the following configuration: - -```json -"viewports": [ -[1920], // Only mention the width of the viewport -[1440] -] -``` - -#### Custom Viewport Configuration - -To facilitate the visualization of your UI components on various device screens, you can now setup custom viewport configurations. This feature is an extension of Storybook's existing Viewport toolbar item, enabling you to capture and view stories in different dimensions, such as `mobile` or `tablet`, with specific orientations like portrait or landscape. - -To configure custom viewports for your stories, you can update the `.smartui.json` file within your project repository. - -```json -{ -... -... -"customViewports": [ -{ -"stories": [ -"" -], -{ -"styles": { -"width": 322, -"height": 321 -}, -"exclude": [ // Example: Excluding stories for a specific viewport -"/directory1", -"/directory2/subdirectory1", -"/directory2/subdirectory2", -"/directory3/subdirectory3/item1", -] -}, -}, -{ -"stories": [ -"" -], -"styles": { -"width": 834, //Tablet view -"height": 1112 -}, -"waitForTimeout": 4000 //Story-level waitForTimeout (Applied to all the combinations of the mentioned stories) -} -{ -"stories": [ -"" -], -"waitForTimeout": 3000 //Story-level waitForTimeout (Applied to all the combinations of the mentioned stories) -} -} -// Additional custom viewport configurations can be added here -] -... -... -} -``` - -The `waitForTimeout` setting at the story level takes precedence over the global `waitForTimeout` configuration and only applies to the specific stories to which it is assigned. - -For instance, if `Story-1` has a story-level `waitForTimeout` value (T1) set within custom viewport settings, and there exists a global `waitForTimeout` value (T2) defined in the configuration, all browser and viewport combinations of `Story-1` will render with T1. Conversely, all other stories will be rendered with T2 across all combinations. - -### **Step 5:** Execute the Tests on SmartUI Cloud using CLI - -You can now execute your `StoryBook` components for `Visual Regression Testing` using the following options:. - -```bash -npm run storybook // Starts your local StoryBook server -smartui storybook http://localhost:6006 --config .smartui.json // Captures all the stories running on local server -``` - -```bash -npm run build-storybook // Creates a Static Build Folder of StoryBook Stories -smartui storybook ./storybook-static --config .smartui.json // Captures all the stories added in the static build folder -``` - -```bash -smartui storybook https:// --config .smartui.json // Captures all the stories running on local server -``` - -**For Continuous Integration (CI)** - -If you are using the Continuous Integration (CI) pipeline for your application and want to integrate `SmartUI StoryBook` execution then the following are the steps needs to be added to your `.yaml` file: - -```yaml -steps: -- name: Running SmartUI StoryBook Tests -- run: npm i -- run: npm install @lambdatest/smartui-storybook -g -- run: npm run build-storybook -- run: smartui storybook ./storybook-static --config .smartui.json -``` - -```yaml -steps: -- name: Running SmartUI StoryBook Tests -- run: npm i -- run: npm install @lambdatest/smartui-storybook -g -- run: smartui storybook https:// --config .smartui.json -``` - -#### CLI Options and Keys - -The following are supported `CLI (Command Line Interface)` options for Visual Regression Testing with SmartUI: - -| CLI Flag Key | Description | Usage | -| ------------ | ----------------------------------------------------------------------------------- | -------- | -| --config | This is the reference configuration file containing the SmartUI Cloud Configuration | Optional | -| --help | This will print all help information for the SmartUI CLI options | Optional | - -### **Step 6:** View SmartUI Results - -You can now see the SmartUI dashboard to view the results. Can also identify the mis-matches from the existing `Baseline` build. - -## Troubleshooting - -Verify Storybook Server - -- Ensure Storybook is running on the specified URL/port - - Check that `buildStoriesJson: true` is set in `.storybook/main.js` - ->**Note**: `buildStoriesJson` is available for Storybook versions below v9. - -Check Story Inclusion/Exclusion - -- Verify `include` and `exclude` patterns in `.smartui.json` - - Ensure story paths match your Storybook structure - -Validate Configuration - -```bash -cat .smartui.json | python -m json.tool -``` - Ensure JSON is valid and configuration is correct -**Symptoms**: -- Play functions not executing -- Components not in expected state -**Solutions**: - -Increase Wait Timeout - -```json -{ -"storybook": { -"waitForTimeout": 3000 // Increase if play functions need more time -} -} -``` - -Check Play Function Syntax - -- Ensure play functions are properly exported - - Verify async/await usage is correct - - Check for errors in browser console -**Symptoms**: -- Themes not switching -- Globals not applied -**Solutions**: - -Verify useGlobals Setting - -```json -{ -"storybook": { -"useGlobals": true // Must be true to use globals -} -} -``` - -Check Storybook Version - -- Ensure Storybook v6.4+ for globals support - - Verify decorators are properly configured - -Validate Theme Configuration - -- Check `backgroundTheme` value is correct (`light`, `dark`, or `both`) - - Ensure theme decorators are properly set up in stories -If you encounter issues not covered here: -- Review the [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) for detailed solutions -- Check [Storybook Documentation](https://storybook.js.org/docs) for Storybook-specific issues -- Visit [TestMu AI Support](https://www.testmuai.com/support) for additional resources -- Contact support at support@testmuai.com or use [24/7 Chat Support](https://www.testmuai.com/support) - -## Additional Resources - -- [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) -- [CLI Complete Reference](/support/docs/smartui-cli-complete-reference) -- [Baseline Management](/support/docs/smartui-baseline-management) -- [Running Your First Project](/support/docs/smartui-running-your-first-project) -- [Storybook Documentation](https://storybook.js.org/docs) -- [SmartUI API Documentation](https://www.testmuai.com/support/api-doc/) diff --git a/static/docs/smart-ui-testing.md b/static/docs/smart-ui-testing.md deleted file mode 100644 index bfec13155..000000000 --- a/static/docs/smart-ui-testing.md +++ /dev/null @@ -1,27 +0,0 @@ -# SmartUI Testing Using HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -SmartUI Testing is a technique used by designers and front-end testers to ensure the visual consistency of web applications by employing a sort of testing called **"Visual Regression"**. - -In Visual Regression testing, we compare screenshots and identify visual variations in the pixels. It is integrated with TestMu AI's HyperExecute, allowing you to execute **SmartUI** Tests on the platform. - -## Features of SmartUI Testing -HyperExecute's SmartUITesting offers the following features: - -- **Pixel-to-Pixel Comparison:** Perform accurate visual regression testing by comparing screenshots at the pixel level. -- **Multi-browser and Resolution Tests:** Run tests simultaneously across multiple browsers and resolutions to ensure consistent user experience. -- **Webhook Integration:** Connect SmartUI Testing with your automation test suite using webhook configurations. -- **Project Collaboration:** Add up to 5 approvers/tags to a project for streamlined collaboration and easy navigation. -- **Real-time Notifications:** Approvers are notified of test status changes, enabling prompt action and communication. - -To execute SmartUI Testing using HyperExecute, you can opt for the following methods: - - Using Selenium - HyperExecute allows you to execute your SmartUI Tests in HyperExecute using Selenium. - - Using Cypress - HyperExecute allows you to execute your SmartUI Tests in HyperExecute using Cypress. - - Using Playwright - HyperExecute allows you to execute your SmartUI Tests in HyperExecute using Playwright. diff --git a/static/docs/smart-visual-regression-testing.md b/static/docs/smart-visual-regression-testing.md deleted file mode 100644 index 378ef39dd..000000000 --- a/static/docs/smart-visual-regression-testing.md +++ /dev/null @@ -1,34 +0,0 @@ -# Explore Visual Regression tests with SmartUI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -SmartUI is a visual testing tool that helps in identifying visual regressions in web applications. It uses advanced image comparison techniques to detect differences between baseline images and those captured during subsequent test runs. - -You can perform following Visual Regression tests using SmartUI: - - SDK - Perform Visual UI Testing using CLI. - - Lambda Hooks - Perform Visual UI Testing using Hooks. - - Figma Designs - Perform Visual UI testing of Figma Designs using SmartUI - - PDFs - Perform Visual UI testing of PDFs using SmartUI. - - Upload through API - Upload Screenshots through API for Visual UI Testing - - Upload through CLI - Upload Screenshots through CLI for Visual UI Testing - - Storybook - Perform Visual UI Testing using Storybook on SmartUI. - - Git Baseline Branching - Learn how to execute SmartUI CLI with Git Commits. - - In case your favorite framework or tool isn't listed here, just give us a Shout Out Here. - In case of any other problems, feel free to Contact Us. diff --git a/static/docs/smart-visual-testing.md b/static/docs/smart-visual-testing.md deleted file mode 100644 index 2a295ed52..000000000 --- a/static/docs/smart-visual-testing.md +++ /dev/null @@ -1,53 +0,0 @@ -# Smart Visual Testing - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -> Smart Visual Testing feature allows you to test and compare two image layouts to determine the differences between them quickly. With just one click, you will now be able to view visual bugs easily, such as Icon Size, Padding, Color, Layout, Text, Element Placement, and much more. - -## Upload Baseline - -Once you login, click on the Navigation menu bar in the left *Visual UI Testing > Smart Testing*. - -Next, we have to upload a Baseline image - -**Step 1:** Upload a baseline image. Click on the + icon next to the Base Line Images header. - -**Step 2:** Select a baseline image from your computer. You can also select multiple images at a time. - -**Step 3:** Once selected, images will be uploaded, and you can view the same under the Baseline Image gallery. - -## Upload Comparison Images - -Upload Comparison Images. Please note that you cannot upload comparison images before uploading a baseline. Once you have uploaded the baseline image, below are the steps to upload a comparison image. - -**Step 1:** Click on the baseline image you want to run a comparison with. - -**Step 2:** On the right, you will find the Upload Comparison Image button. Click on it to upload images. - -**Step 3:** Select the images you want to compare with the baseline image. You can select multiple images as well. - -**Step 4:** If everything goes well, you would see the thumbnail of your comparison images along with the RUN button. - -## Run Comparison - -**Step 1:** Once you have uploaded the comparison images, running the comparison is pretty simple. Just click on the Run button on the comparison image thumbnail. - -**Step 2:** You can also click on compare all button. - -## View Comparison and Issues - -Once you have run the comparison between baseline and comparison images, you can check the issues by clicking on the View issues button on the comparison image. - -This will take you to a new window with a comparison image. You can zoom-in or Zoom out the images in this window, change the image difference color, switch to slider mode, switch to side-by-side mode, or mark the image as a bug. - -## Side by Side Mode - -There are multiple options for you to view your images. The side by side mode will open all three images, baseline image, comparison image, and comparison result image in a single window. In this mode, you can also mark the image as a bug. Or you can switch to any other mode. - -## Slider Mode - -In slider mode, you can view the baseline image and comparison image overlapped over each other and toggle them with a slider. This mode can help you view the difference between the two images very easily. - -## Change Baseline or Comparison Image - -You can change the baseline or comparison image easily. Just click on the settings button at the bottom of the image thumbnail. You would see multiple options like View Image Details, Replace Image, Rename Image, and Delete Image. diff --git a/static/docs/smart-wait.md b/static/docs/smart-wait.md deleted file mode 100644 index 875d75fe2..000000000 --- a/static/docs/smart-wait.md +++ /dev/null @@ -1,43 +0,0 @@ -# SmartWait Functionality for Selenium Automation - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -With TestMu AI's SmartWait feature, you can increase the efficiency and accuracy of automated test execution. Before performing any action on a webpage element, SmartWait conducts a series of actionability checks. It holds off on performing an action until all relevant checks pass. If the checks do not pass within a set timeframe, it returns the appropriate Selenium error message. - -You can use this feature to reduce the amount of code dedicated to explicit/implicit waits in your test suite, making it easier to read and maintain. - -> **Note:** The **SmartWait** functionality is fully supported on HyperExecute as well, allowing you to leverage intelligent wait handling during parallel and distributed test executions. - -## How SmartWait Can Help You - -SmartWait improves your test automation workflow in multiple ways. - -- **Increased Accuracy**: SmartWait ensures that actions are only executed on elements that are ready and actionable, increasing the accuracy of your test results. -- **Code Optimization**: Your test scripts will have fewer explicit and implicit wait commands, making them easier to read and maintain. -- **Enhanced Efficiency**: Your test suite becomes more efficient by eliminating unnecessary waits and performing actions as soon as the elements are ready. -- **Intelligent Error Handling**: When actionability checks fail, SmartWait returns the relevant Selenium error, helping you identify and resolve issues more quickly. - -## How to Use SmartWait Functionality - -Follow the below steps to configure SmartWait for your tests. - -1. **Set SmartWait Time Limit:** Specify the maximum amount of time you want your test script to wait until an element becomes actionable. - -```javascript -LT:Options { -... -"smartWait": 10 // It accepts integer values as second -"smartWaitRetryDelay": 1 -... -} -``` - -- Min value of `smartWait` is 5 seconds and max is 120 seconds, default being 1 seconds. -- The capability `smartWaitRetryDelay` to define the time interval of retries. If not provided, default interval of 1 second is considered. -- Min value of `smartWaitRetryDelay` is 1 second and max is 4 seconds, default being 1 second. - -2. **Execute Test Suite:** Run your test suite as usual. SmartWait automatically performs actionability checks before each action on a webpage element. - -3. **View Test Results:** Review your results after execution. SmartWait returns the relevant Selenium error message if any actionability checks failed within the set timeframe. - -TestMu AI's SmartWait feature enhances automated testing by intelligently managing wait times and ensuring actions execute at the most opportune time. diff --git a/static/docs/smartui-ab-testing-variations.md b/static/docs/smartui-ab-testing-variations.md deleted file mode 100644 index e4bd19844..000000000 --- a/static/docs/smartui-ab-testing-variations.md +++ /dev/null @@ -1,174 +0,0 @@ -# A/B Testing and Baseline Variations - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -In TestMu AI **SmartUI**, **A/B testing** lets one **screenshot or PDF page** keep **multiple approved reference images** (called **variations**). Each new capture is compared against **every active variation**; the UI shows which one **matched** (closest visual agreement). That cuts false failures when the same page can look different but still be correct (A/B tests, feature flags, themes, locales, or PDF layout variants) without splitting into separate names in automation or upload flows. - -This guide walks through the **review** flow in the SmartUI dashboard: the **A/B** panel, the **default** variation, **Add new**, **Matched with**, managing variations, and **ending** A/B by picking a single winner. - -## Where A/B testing works - -A/B variations use the **same in-product panel** whether the asset is a **standard screenshot** or a **PDF page**: - -| Capture type | Typical source | Notes | -|--------------|----------------|--------| -| **Web / app screenshots** | Automation (Selenium, Playwright, Cypress, Hooks, SDK, CLI), manual capture, Storybook, and similar | Use a stable **screenshot name** per page or state; open that screenshot in a build to manage variations. | -| **PDF pages** | [PDF comparison](/support/docs/smartui-pdf-comparison/) projects (upload, Java SDK, CLI, or API) | A/B applies **per PDF page** in the build: the same **Add new**, **Matched with**, and **End A/B testing** controls as for web screenshots. | - -You do **not** need a separate project type for A/B. Create or open the screenshot or PDF page in your existing SmartUI build, then use the **A/B** icon on that asset’s review screen. - -**PDF projects** -Set up PDF ingestion first if you are new to PDF comparison: [PDF Comparison in SmartUI](/support/docs/smartui-pdf-comparison/), [Upload PDFs via Java SDK](/support/docs/smartui-pdf-java-sdk/), or [PDF CLI upload](/support/docs/smartui-pdf-cli-upload/). - -## Key terms - -| Term | Meaning | -|------|--------| -| **Variation** | An approved reference image for this screenshot. The **default** variation tracks the platform baseline; you can add up to **19** more. | -| **Default variation** | The reference tied to the **current SmartUI baseline** for this build (dynamic, not a user-created slot you rename as “default”). | -| **Matched with** | The variation that best matches the latest capture (typically lowest diff % within threshold). | -| **End A/B testing** | Pick one variation as the **only** baseline going forward; other variations are removed. | - -## Workflow at a glance - -1. Open a screenshot in a build → click the **A/B** icon → review the **default** variation in the right pane. -2. Click **Add new** to save the **current capture** as another variation → A/B is **active** for this screenshot. -3. On **later builds**, open the pane again → check **Matched with** for each new capture. -4. **Rename**, **update**, or **delete** user-added variations as needed (up to **19** plus default). -5. When the experiment ends → **End A/B testing** → choose the **winning** variation → others are deleted. - -**UI labels** -Button names may vary slightly (**End experiment**, **End A/B testing**, **Pick winner**). Use the action that **selects one variation as the sole baseline** and removes the rest. Contact [support](mailto:support@testmuai.com) if you are unsure. - -## When to use A/B variations - -**Use this feature when:** - -- Your UI runs **A/B experiments**, **feature flags**, or **progressive rollouts** and the same `screenshotName` can look different while still being valid. -- You support **locales**, **themes**, or **segment layouts** with more than one acceptable visual outcome. -- You want a **stable screenshot name** in tests but **multiple approved references** for comparison. -- You are **iterating on design** and need several references until you commit to one winner. -- You compare **PDF documents** where the same page can legitimately differ (for example invoice layout A/B, regional template, or revised legal copy) without uploading separate files for every variant. - -**Why it helps** - -- Fewer false **changes found** when the diff is an expected variant, not a regression. -- Reviewers see **which variation matched** instead of guessing the intended baseline. -- You can **close an experiment** in one step by promoting a winner and removing the rest. - -**Examples:** - -- **Web:** A homepage runs a light/dark theme test. Run #1 matches the light **default**; run #2 flags a mismatch until you **Add new** for dark. Later runs auto-match light or dark; **End A/B testing** keeps only the winning theme. -- **PDF:** A statement PDF ships two footer layouts. Page 3 mismatches until you **Add new** from the alternate upload; later builds show **Matched with** the correct layout; **End A/B testing** promotes the winner and removes the other variation. - -For **build-level** baselines (branches, approvals, Smart Git), see [Baseline Management](/support/docs/smartui-baseline-management/) and [Approval & Baseline Workflow](/support/docs/smartui-approval-workflow-guide/). - -## Prerequisites - -- A TestMu AI account with **SmartUI** access and an existing **project** with at least one build. -- At least one **screenshot** or **PDF page** in that build to open in review. -- Permission to **review** and update baselines in that project. -- No extra automation flags are required to **start** A/B from the UI: for web tests, keep the same **`screenshotName`**; for PDFs, keep the same page identity in your upload or SDK flow. - -## Open the A/B testing panel - -1. Open your **SmartUI project** and select the **build** that contains the screenshot. -2. Open the **screenshot** (comparison / review view). -3. Click the **A/B** icon on the screenshot toolbar. - -The **variations pane** opens on the **right**. The steps below assume you stay in this pane unless noted. - -## Default variation (dynamic baseline) - -When the pane opens, you see the **default variation**: - -- It reflects the **current baseline** for this screenshot in this **build context**, per your project’s **SmartUI baseline rules** (for example the latest **approved** reference for that build strategy). -- It is **dynamic**: if the governing baseline changes (approval, move to baseline, branch rules), the default variation **follows** that reference; it is not a separate image you pin manually. - -You **cannot** assign another variation to replace this **default** slot. **Default** means the **platform baseline** for this screenshot, not a renameable card in your list. - -## Activate A/B testing: Add new - -1. With the pane open, click **Add new**. -2. SmartUI creates a variation from the **current capture** (the screenshot you are viewing). -3. From the **next build onward**, new captures for this name are compared against the **default** and **every user-added** variation. That activates multi-baseline A/B for this screenshot. - -Each added variation stays active until you **delete** it or **end A/B testing**. - -## After new builds: Matched with and limits - -### Matched with - -When a new build processes this screenshot, the pane shows **Matched with** and the **variation name** when a capture aligns with one of your references. SmartUI picks the **closest** match among active variations (typically the **lowest diff percentage** within your comparison threshold). - -Only **one** variation is shown as the match for triage at a time, the best fit for that capture. - -**Unmatched captures** -If **no** variation is within threshold, the screenshot may still show as **changes found** or unmatched. Add a new variation from that capture, adjust thresholds, or **end A/B** once you know the intended winner. See [Mismatch Thresholds](/support/docs/smartui-mismatch-thresholds/). - -### How many variations? - -| Limit | Detail | -|-------|--------| -| **Total** | Up to **20** variations: **1 default** + up to **19** user-added | -| **Add new** | Available until you reach **19** user-added variations | - -### Screenshot usage - -Every **active variation** is used when comparing **future** captures for that screenshot. Usage is counted like standard baseline comparison work across the full variation set. For plan-specific metering, contact [support](mailto:support@testmuai.com). - -## Manage user variations (rename, update, delete) - -For variations you added with **Add new** (not the dynamic **default**): - -| Action | When to use it | -|--------|----------------| -| **Rename** | Clarify the label (for example `Dark theme`, `Locale DE`). | -| **Update** | Replace the variation’s reference with the **current capture** when that alternate “good” state changed. | -| **Delete** | Remove a variation you no longer need. You cannot delete the **default** slot. | - -To return to a **single** baseline without picking a winner yet, delete user-added variations one by one. To **commit** to one outcome and remove the rest in one step, use **End A/B testing** below. - -## End A/B testing (pick a winner) - -When the experiment is finished: - -1. Start **End A/B testing** (or **End experiment** / **Pick winner**) in the pane. -2. **Select the variation** that should be the **only** baseline for this screenshot going forward. -3. Confirm. SmartUI promotes that choice and **deletes the other variations**, returning this screenshot to **single-baseline** mode until you add variations again. - -## Limits and guardrails - -- **20 variations maximum** per screenshot (1 default + 19 user-added). -- **Comparison mode** (pixel, layout, Smart Ignore, and so on) applies at the **screenshot** level for all variations on that screenshot. -- **Branch and merge** behavior for baselines still follows your project settings; see [Baseline Management](/support/docs/smartui-baseline-management/) and [Branch merging](/support/docs/smartui-branch-merging/) for Git-linked workflows. - -## Troubleshooting - -### I only see one variation and no Add new - -Confirm you are on the **screenshot detail** view with the **A/B** icon visible, and that your role can edit baselines in the project. Refresh the build after approvals if the default variation has not updated yet. - -### A new build never shows Matched with - -- Ensure A/B was **activated** (at least one **Add new** besides default, or an active multi-variation state). -- Wait for the build to finish processing the screenshot. -- Check [Mismatch Thresholds](/support/docs/smartui-mismatch-thresholds/) if diffs are above your allowed percentage. - -### I hit the variation limit - -You can have at most **20** variations (including default). **Delete** unused user-added variations or **End A/B testing** to collapse to one baseline before adding more. - -### Can I mark my custom variation as default? - -No. The **default** card tracks the **platform baseline** for the build. User-added variations are managed with rename/update/delete; **End A/B testing** is how you make one variation the **sole** baseline. - -## Related documentation - -- [Baseline Management](/support/docs/smartui-baseline-management/) -- [Approval & Baseline Workflow Guide](/support/docs/smartui-approval-workflow-guide/) -- [Mismatch Thresholds](/support/docs/smartui-mismatch-thresholds/) -- [PDF Comparison in SmartUI](/support/docs/smartui-pdf-comparison/) -- [Layout Comparison in SmartUI SDK](/support/docs/smartui-layout-testing/) -- [Draw on UI](/support/docs/smartui-draw-on-ui/) -- [Group by Test Cases](/support/docs/smartui-group-by-test-cases/) diff --git a/static/docs/smartui-agent-skills.md b/static/docs/smartui-agent-skills.md deleted file mode 100644 index f986ae4b2..000000000 --- a/static/docs/smartui-agent-skills.md +++ /dev/null @@ -1,112 +0,0 @@ -# Run Your SmartUI Visual Tests Using Agent Skills - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -SmartUI catches visual bugs by comparing screenshots of your UI against an approved baseline. Wiring it into a test suite means installing the CLI, adding snapshot calls, and tuning a config. You can hand that off to your AI coding agent instead. - -The [`smartui-skill`](https://github.com/LambdaTest/agent-skills/tree/main/smartui-skill) is part of [TestMu AI Agent Skills](https://github.com/LambdaTest/agent-skills), a library of instruction packs that give AI coding agents the know-how to build working test automation on TestMu AI. Describe the pages you want covered, and it wires SmartUI into your existing framework and runs the comparison on the TestMu AI cloud. - -## Prerequisites - -- A [TestMu AI account](https://accounts.lambdatest.com/register) -- Your TestMu AI **Username** and **Access Key** from the [Automation Dashboard](https://accounts.lambdatest.com/detail/profile) -- Your SmartUI **Project Token**, available from the [SmartUI app](https://smartui.lambdatest.com/) after creating a project -- A compatible AI coding agent (Claude Code, GitHub Copilot, Cursor, Gemini CLI, or similar) - -## Installing the SmartUI Agent Skill - -Clone the Agent Skills repository and copy the SmartUI skill into your agent's skills directory: - -```bash -git clone https://github.com/LambdaTest/agent-skills.git - -# For Claude Code -cp -r agent-skills/smartui-skill .claude/skills/ - -# For Cursor -cp -r agent-skills/smartui-skill .cursor/skills/ - -# For GitHub Copilot -cp -r agent-skills/smartui-skill .github/skills/ - -# For Gemini CLI -cp -r agent-skills/smartui-skill .gemini/skills/ -``` - -If you prefer installing all available framework skills, clone the repository directly into your tool's skills directory (e.g., `.claude/skills/`, `.cursor/skills/`). - -## Set Up Your Authentication - -Export your TestMu AI credentials and SmartUI project token as environment variables so the agent can run tests on the cloud: - -```bash -export LT_USERNAME="YOUR_LAMBDATEST_USERNAME" -export LT_ACCESS_KEY="YOUR_LAMBDATEST_ACCESS_KEY" -export PROJECT_TOKEN="YOUR_SMARTUI_PROJECT_TOKEN" -``` - -```powershell -$env:LT_USERNAME="YOUR_LAMBDATEST_USERNAME" -$env:LT_ACCESS_KEY="YOUR_LAMBDATEST_ACCESS_KEY" -$env:PROJECT_TOKEN="YOUR_SMARTUI_PROJECT_TOKEN" -``` - -## How to Run Visual Tests Using the SmartUI Skill - -Once the skill is installed and your credentials are set, you drive everything through natural-language prompts. Your agent reads the skill, applies the right pattern for your framework, and runs the test on the TestMu AI cloud. You do not write the configuration or the snapshot calls yourself. - -### Step 1: Ask the agent to set up SmartUI - -Describe the visual test you want. Be specific about the pages or components and, if it matters, the browsers and viewports: - -> "Add SmartUI visual regression tests to my Playwright suite. Capture the homepage and the login page across Chrome, Firefox, and Safari at desktop and mobile viewports." - -From this, the agent will: - -- Install the SmartUI CLI and the driver for your framework -- Create a `smartui.config.json` with the browsers, viewports, and wait conditions you asked for -- Add the correct snapshot call at the right points in your test (for example, `smartuiSnapshot(page, 'Homepage')` for Playwright, or `smartui.takeScreenshot` for Selenium) -- Wire up the `npx smartui exec` command that runs the whole thing on the cloud - -### Step 2: Run the visual test - -Ask the agent to run it, or run the command it generated yourself: - -```bash -npx smartui exec -- npx playwright test -``` - -The agent executes your suite through the SmartUI CLI, which captures a screenshot at every snapshot call and uploads it to your project for comparison. - -### Step 3: Review and iterate - -The agent reports the outcome and links you to the build in your [SmartUI dashboard](https://smartui.lambdatest.com/), where the visual comparisons live (see [Understanding the Results](#understanding-the-results)). From there you can keep iterating in plain language: - -> "The header changed on purpose, approve the new baseline." -> -> "Add a snapshot of the checkout page and re-run." - -### Example prompts to try - -- "Set up SmartUI screenshot comparison for my Cypress tests and run it on TestMu AI" -- "Capture visual snapshots of my Storybook components with SmartUI" -- "Add a full-page snapshot after the cart loads and compare across three viewports" -- "My login page snapshot is failing, show me what changed" - -## Understanding the Results - -The first run of a SmartUI test establishes the **baseline**, the approved reference set. No comparison happens on this run; there is nothing to compare against yet. Every later run captures fresh screenshots and compares them against the baseline, reporting a **mismatch percentage** per screenshot for each browser and viewport. - -Review the comparisons in your [SmartUI dashboard](https://smartui.lambdatest.com/), where you approve intended changes to update the baseline or flag unintended differences as regressions. - -## Supported AI Tools - -| Tool | Type | Installation Path | -| -------------- | --------- | ---------------------------- | -| Claude Code | CLI | `.claude/skills/` | -| GitHub Copilot | Extension | `.github/skills/` | -| Cursor | IDE | `.cursor/skills/` | -| Gemini CLI | CLI | `.gemini/skills/` | -| Codex CLI | CLI | `.codex/skills/` | -| OpenCode | CLI | `.opencode/skills/` | -| Claude.ai | Web | Settings > Features > Skills | diff --git a/static/docs/smartui-appium-hooks.md b/static/docs/smartui-appium-hooks.md deleted file mode 100644 index 22f19e399..000000000 --- a/static/docs/smartui-appium-hooks.md +++ /dev/null @@ -1,569 +0,0 @@ -# Getting Started With Visual Regression Testing Using Appium On SmartUI Real Devices (NodeJS) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Using the TestMu AI platform, perform regression testing in just one click and find Visual UI Regression bugs easily with the help of Smart Testing. This documentation will act as your step-by-step guide in performing successful Visual Regression tests. - -## Prerequisites for SmartUI with App Automation - -- Basic understanding of [appium](https://appium.io/docs/en/2.0/intro/) and remote [WebDriver](https://www.selenium.dev/documentation/webdriver/drivers/remote_webdriver/) is required. -- Go to [`TestMu AI SmartUI`](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) and login along with your credentials. -- Access your Hub for your appium **remote connection** instance at `@mobile-hub.lambdatest.com/wd/hub`. -- Copy `LT_USERNAME` and `LT_ACCESS_KEY` credentials from `Access Key` button on the top right of the dashboard. - -Below are code examples for taking viewport screenshots using SmartUI with Appium across different programming languages and frameworks: - -```bash -export LT_USERNAME="YOUR_USERNAME" -``` - -```bash -set LT_USERNAME="YOUR_USERNAME" -``` - -```powershell -$env:LT_USERNAME="YOUR_USERNAME" -``` - -The following steps will guide you in running your first Visual Regression test on TestMu AI platform - - -## Step 1: Create a SmartUI Project - -The first step is to create a project with the application in which we will combine all your **builds** run on the project. -To create a SmartUI Project, follow these steps: - -1. Go to [Projects page](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) -2. Click on the `new project` button -3. Select the platform as Real Device for executing your `Appium` tests. -4. Add name of the project, approvers for the changes found, tags for any filter or easy navigation. -5. Click on the **Submit**. - -## Step 2: Upload your application - -Upload your **_iOS_** application (.ipa file) or **_android_** application (.apk file) to the TestMu AI servers using our **REST API**. You need to provide your **Username** and **AccessKey** in the format `Username:AccessKey` in the **cURL** command for authentication. Make sure to add the path of the **appFile** in the cURL request. Here is an example cURL request to upload your app using our REST API: - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "appFile=@"/Users/macuser/Downloads/proverbial_android.apk"" -F "name="proverbial_app"" -`} - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "url=:https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk" -F "name=Proverbial_App" -`} - -- If you do not have any **.apk** or **.ipa** file, you can run your sample tests on TestMu AI by using our sample :link: [Android app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk) or sample :link: [iOS app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_ios.ipa). -- Response of above cURL will be a **JSON** object containing the `APP_URL` of the format - ```lt://APP123456789123456789``` and will be used in the next step. - -## Step 3: Clone the sample project - -Clone the TestMu AI’s :link: [LT-appium-nodejs](https://github.com/LambdaTest/LT-appium-nodejs) repository and navigate to the code directory as shown below: - -```bash -git clone https://github.com/LambdaTest/LT-appium-nodejs -cd LT-appium-nodejs -``` - -## Step 4: Set up your authentication - -Make sure you have your TestMu AI credentials with you to run test automation scripts on TestMu AI. To obtain your access credentials, [purchase a plan](https://billing.lambdatest.com/billing/plans) or access the [Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://appautomation.lambdatest.com/). Then, set TestMu AI `Username` and `Access Key` in environment variables with following commands. - - {`export LT_USERNAME=${ YOUR_LAMBDATEST_USERNAME()} \\ -export LT_ACCESS_KEY=${ YOUR_LAMBDATEST_ACCESS_KEY()}`} - - {`set LT_USERNAME=${ YOUR_LAMBDATEST_USERNAME()} \` -set LT_ACCESS_KEY=${ YOUR_LAMBDATEST_ACCESS_KEY()}`} - -## Step 5: Configure your test with Appium Desired Capabilities - -**Explore more capabilities** - -To view and generate more capabilities for your appium application test suite, please see the capability generator here: https://www.testmuai.com/capabilities-generator/ - -Once you have created a SmartUI Project, you can generate screenshots by running automation scripts. Edit the required capabilities for your test suite as shown in the below sample code snippet: - -```javascript title="Please replace your capabilities in your Remote Webdriver DesiredCapabilities configuration. This sample is for NodeJS" -let capabilities = { -deviceName: "iPhone 12", // Configure your Device for appium test (Mandatory) -platformName: "ios", // Configure your OS for appium test (Mandatory) -platformVersion: "14", // Configure your OS Version for appium test (Mandatory) -isRealMobile: true, // Configure if you want to test in a real-device (Mandatory) -app: "APP_URL", // Set your uploaded App URL for testing (Mandatory) -//highlight-next-line -visual: true, // Configure your Capture screenshot for appium test (Mandatory) -name: "testing app session ", // Name of the test for appium (Recommended) -build: "Real Device App Build", // Name of the build for appium (Recommended) -//highlight-start -"smartUI.project": "", // Replace the name of project with the new project name (Mandatory) -"smartUI.build": "", // Replace the name of Build with the new Build name (Optional) -"smartUI.baseline": false, // Enable if you want to update to a new baseline build (Optional) -"smartUI.cropStatusBar" : true, // Optional (By default true) -"smartUI.cropFooter" : true, // Optional (By default false) -//highlight-end -}; - -// Connecting to the LambdaTest appium Cloud Grid with SmartUI -let gridUrl = -"https://" + -"" + -":" + -"" + -`@mobile-hub.lambdatest.com/wd/hub`; - -// Here is your Remote WebDrive Connection -let driver = await new webdriver.Builder() -.usingServer(gridUrl) -.withCapabilities(capabilities) -.build(); -``` -**Important** - -It is important that the `visual:true` is set your capabilities configuration for capturing the screenshots to SmartUI - Visual Regression tests and add into the build for comparison. If this capability is not added then the build status will be shown as `Error`. - -- Webhook for taking the screenshot - This part of the code needs to be attached below the required segment of appium script of which we would like to take the screenshot to test on. - -```javascript title="For Example if your are using with NodeJS" -driver.execute("smartui.takeScreenshot="); -``` - -Appium with SmartUI is currently only supports the viewport based screenshot comparisons. - -- Execute your test suite as per the execution command depending on your framework or language. - -```bash title="For Example if your are using with NodeJS" -npm i && node your_test_script.js -``` - -- You can check the executed builds over at [TestMu AI SmartUI](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/). - -## Smart Crop With SmartUI - -The all-new **Real Device mobile notification status bar and navigation bar crop** feature in SmartUI allows you to take your visual regression testing workflows to the next level. With Smart Crop, you can crop the status bar and navigation bar or footer from screenshots, enabling them to focus solely on the core UI elements during visual comparisons. - -By leveraging machine learning algorithms, it accurately detects and crops the status bar and navigation bar from screenshots. With precise image processing techniques, SmartUI precisely identifies the location of status bar elements. By excluding it from visual comparisons, the focus is solely on critical UI elements. - -### Original Screenshot: - -### Cropped Screenshot - -## Region-Based Ignore/Select for Dynamic Content (Advanced) - -To handle dynamic content like timestamps, user names, ads, or banners that cause false positives in visual comparisons, SmartUI supports region-based ignore and select functionality using **XPath locators**. - -You can either: -- **Ignore specific regions** during comparison using `ignoreBoxes` -- **Compare only specific regions** using `selectBoxes` - -This is especially useful for enterprise applications where certain UI elements change dynamically between test runs. - -### Usage in Node.js (Primary Example) - -```javascript title="Example: Ignoring Dynamic Elements in Node.js" -let config = { -screenshotName: 'Home Screen', -ignoreBoxes: JSON.stringify({ -xpath: [ -"//*[@text='Backpack']", -"//*[@text='Onesie']", -"//*[@text='PRODUCTS']", -"//*[@text='Terms of Service | Privacy Policy']" -] -}) -}; - -await driver.execute("smartui.takeScreenshot", config); -``` - -```javascript title="Example: Selecting Only Critical Regions in Node.js" -let config = { -screenshotName: 'Checkout Form', -selectBoxes: JSON.stringify({ -xpath: [ -"//*[@resource-id='checkout-form']", -"//*[@resource-id='total-amount']" -] -}) -}; - -await driver.execute("smartui.takeScreenshot", config); -``` - -### Cross-Framework Examples - -```javascript -let config = { -screenshotName: '', -ignoreBoxes: JSON.stringify({ -xpath: ["//*[@text='Dynamic Ad']", "//*[@id='timestamp']"] -}) -}; -await driver.execute("smartui.takeScreenshot", config); -``` - -```python -config = { -'screenshotName': '', -'ignoreBoxes': '{"xpath": ["//*[@text=\'Dynamic Ad\']", "//*[@id=\'timestamp\']"]}' -} -driver.execute_script("smartui.takeScreenshot", config) -``` - -```java -Map configIgnore = new HashMap<>(); -configIgnore.put("screenshotName", "Screenshot"); - -Map ignoreBoxes = new HashMap<>(); -ignoreBoxes.put("xpath", new String[]{ -"//*[@name='Toast']" -}); - -configIgnore.put("ignoreBoxes", ignoreBoxes); -((JavaScriptExecutor) driver).executeScript("smartui.takeScreenshot", configIgnore); - -``` - -```csharp -var config = new Dictionary { -{"screenshotName", ""}, -{"ignoreBoxes", "{\"xpath\": [\"//*[@text='Dynamic Ad']\", \"//*[@id='timestamp']\"]}"} -}; -driver.ExecuteScript("smartui.takeScreenshot", config); -``` - -```ruby -config = { -'screenshotName' => '', -'ignoreBoxes' => '{"xpath": ["//*[@text=\'Dynamic Ad\']", "//*[@id=\'timestamp\']"]}' -} -driver.execute_script("smartui.takeScreenshot", config) -``` - -### Configuration Keys - -| Key | Type | Description | Required | -|-----|------|-------------|----------| -| `ignoreBoxes` | JSON String | Defines regions to ignore during visual comparison. Accepts XPath locators. | No | -| `selectBoxes` | JSON String | Defines regions to include in visual comparison. Accepts XPath locators. | No | - -**Best Practices** -- Use `ignoreBoxes` for elements that change frequently (e.g., ads, timestamps, user avatars). -- Use `selectBoxes` when you want to focus comparison only on critical UI sections. -- Avoid using both `ignoreBoxes` and `selectBoxes` in the same config; they are mutually exclusive. -- Ensure XPath expressions are unique and stable across test runs. -- Test your XPath locators using Appium Inspector or similar tools before integrating. - -## Running Tests on Other Languages and Frameworks - -In this module we discussed about running smart visual tests on **NodeJS**, here we will know more about running those tests for any language or framework with appium. - -In a similar way, we can run visual tests for other languages and frameworks using their corresponding script executing commands. To understand better, we provided the commands for some of the popular languages and frameworks: - -### For taking viewport screenshot - -This part of the code needs to be attached below the required segment of selenium script of which we would like to take the screenshot to test on. - -```javascript -driver.execute("smartui.takeScreenshot="); -``` - -```python -driver.execute("smartui.takeScreenshot=") -``` - -```ruby -driver.execute("smartui.takeScreenshot=") -``` - -```csharp -driver.Execute("smartui.takeScreenshot="); -``` - -```java -((JavaScriptExecutor)driver).executeScript("smartui.takeScreenshot="); -``` - -### For capturing full page screenshot in Native Apps - -You can capture the full page screenshot for apps which have a scrolling functionality on their application user interface for your **Appium** functional testing. - -Add the following **Webhook** to your test cases where we need to capture the screenshot of your application. - -```javascript -let config = { -screenshotName: '', -fullPage: true, -pageCount: 15 // Enter the number of pages for the Full Page screenshot (Minimum 1, Maximum 20) -}; -await driver.execute("smartui.takeScreenshot", config); -``` - -```python -config = { -'screenshotName': '', -'fullPage': True, -'pageCount': 15 # Enter the number of pages for the Full Page screenshot (Minimum 1, Maximum 20) -} -driver.execute("smartui.takeScreenshot", config) - - -```ruby -config = { - 'screenshotName' => '', - 'fullPage': true, - 'pageCount': 15 # Enter the number of pages for the Full Page screenshot (Minimum 1, Maximum 20) -} -driver.execute("smartui.takeScreenshot", config) -``` - - - - -```csharp -var config = new Dictionary { - {"screenshotName", ""}, - {"fullPage", true}, - {"pageCount", 15} // Enter the number of pages for the Full Page screenshot (Minimum 1, Maximum 20) -}; -driver.Execute("smartui.takeScreenshot", config); -``` - - - - -```java -Map config = new HashMap<>(); -config.put("screenshotName", ""); -config.put("fullPage", true); -config.put("pageCount", 15); // Enter the number of pages for the Full Page screenshot (Minimum 1, Maximum 20) -((JavaScriptExecutor)driver).executeScript("smartui.takeScreenshot", config); -``` - - - - -:::note -Please note that this webhook is only applicable to native app screenshots and has known limitations. You can use an optimized value of page count (between 1 and 20) to get the best results of your full page screenshots, according to your use case. -::: - -## Best Practices - - - - -**Screenshot Naming** - -- Use descriptive, consistent names for screenshots -- Include screen/component name in screenshot names -- Avoid special characters that might cause issues -- Use consistent naming conventions across your test suite - -**Example:** -```javascript -await driver.execute("smartui.takeScreenshot=HomeScreen-Header"); -await driver.execute("smartui.takeScreenshot=CheckoutScreen-PaymentForm"); -``` - - - - -**Wait for Screen Load** - -- Always wait for screens to fully load before taking screenshots -- Use Appium's wait methods for dynamic content -- Consider device-specific loading times - -**Example:** -```javascript -const { until, By } = require('selenium-webdriver'); -await driver.wait(until.elementLocated(By.id('main-content')), 10000); -await driver.execute("smartui.takeScreenshot=Screen Loaded"); -``` - - - - -**Handle Dynamic Content** - -- Use `ignoreBoxes` for elements that change between runs (ads, timestamps, user avatars) -- Use `selectBoxes` when you only need to compare specific regions -- Test XPath locators using Appium Inspector before integrating - - - - -**Smart Crop Configuration** - -- Enable `cropStatusBar` to focus on core UI elements -- Enable `cropNavigationBar` for Android devices -- Test cropped screenshots to ensure important content isn't removed - - - - -**Test Organization** - -- Group related screenshots in the same build -- Use meaningful build names -- Run tests on consistent device configurations - - - - -**Test Organization** - -- Group related screenshots in the same build -- Use meaningful build names -- Run tests on consistent device configurations - - - - -## Troubleshooting - - - - -**Issue: Screenshots Not Captured** - -**Symptoms**: Tests run but no screenshots appear in SmartUI dashboard - -**Possible Causes**: -- `visual: true` not set in capabilities -- Incorrect project name -- Network connectivity issues -- Credentials not set correctly - -**Solutions**: -1. Verify `visual: true` is set in capabilities: -```javascript - let capabilities = { - visual: true, // Must be set - // ... other capabilities - }; -``` - -2. Check project name matches exactly (case-sensitive): -```javascript - "smartUI.project": "ExactProjectName" -``` - -3. Verify credentials are set: -```bash - echo $LT_USERNAME - echo $LT_ACCESS_KEY -``` - -4. Check network connectivity to LambdaTest - - - - -**Issue: "Project Not Found" Error** - -**Symptoms**: Error indicating SmartUI project cannot be found - -**Possible Causes**: -- Project name typo or mismatch -- Project deleted -- Wrong account credentials - -**Solutions**: -1. Verify project exists in SmartUI dashboard -2. Copy project name directly from dashboard -3. Check credentials match the account with the project -4. Ensure project name is in capabilities, not just in dashboard - - - - -**Issue: Screenshots Show Blank or Incorrect Content** - -**Symptoms**: Screenshots captured but show blank screens or incorrect content - -**Possible Causes**: -- Screen not fully loaded -- App state issues -- Timing issues -- Device-specific rendering - -**Solutions**: -1. Add explicit waits before screenshots: -```javascript - await driver.wait(until.elementLocated(By.id('content')), 10000); -``` - -2. Wait for specific elements to be visible: -```javascript - await driver.wait(until.elementIsVisible(By.id('main-content')), 10000); -``` - -3. Increase wait time for slow-loading screens - -4. Verify app is in correct state before screenshot - - - - -**Issue: Full Page Screenshot Issues** - -**Symptoms**: Full page screenshots incomplete or incorrect - -**Possible Causes**: -- `pageCount` value too low -- App doesn't support scrolling -- Content not loading properly - -**Solutions**: -1. Increase `pageCount` value (between 1-20): -```javascript - let config = { - screenshotName: 'Full Page', - fullPage: true, - pageCount: 15 // Increase if needed - }; -``` - -2. Verify app supports scrolling functionality - -3. Test with different `pageCount` values to find optimal setting - - - - -**Issue: ignoreBoxes/selectBoxes Not Working** - -**Symptoms**: Dynamic content still causing false positives - -**Possible Causes**: -- XPath locators incorrect -- Elements not found at screenshot time -- JSON string formatting issues - -**Solutions**: -1. Verify XPath locators using Appium Inspector -2. Ensure elements exist at screenshot time -3. Check JSON string formatting: -```javascript - ignoreBoxes: JSON.stringify({ - xpath: ["//*[@text='Dynamic Ad']"] - }) - ``` - -4. Test XPath locators in isolation before using in config - -**Getting Help** - -If you encounter issues not covered here: - -- Review the [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) for detailed solutions -- Check [SmartUI Configuration Options](/support/docs/smartui-sdk-config-options) documentation -- See [Handling Dynamic Data](/support/docs/smartui-handle-dynamic-data) for dynamic content issues -- Visit [TestMu AI Support](https://www.testmuai.com/support) for additional resources -- Contact support at support@testmuai.com or use [24/7 Chat Support](https://www.testmuai.com/support) - -## Additional Resources - -- [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) -- [SmartUI Configuration Options](/support/docs/smartui-sdk-config-options) -- [Handling Dynamic Data](/support/docs/smartui-handle-dynamic-data) -- [Baseline Management](/support/docs/smartui-baseline-management) -- [Running Your First Project](/support/docs/smartui-running-your-first-project) -- [Appium Documentation](/support/docs/appium-nodejs/) - -For additional information about appium framework please explore the documentation [here](/support/docs/appium-nodejs/) diff --git a/static/docs/smartui-appium-java-sdk.md b/static/docs/smartui-appium-java-sdk.md deleted file mode 100644 index 0f2404e90..000000000 --- a/static/docs/smartui-appium-java-sdk.md +++ /dev/null @@ -1,434 +0,0 @@ -# SmartUI App SDK Integration Guide - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -SmartUI App SDK enables you to perform visual regression testing on your mobile applications using any cloud testing provider. This guide will help you integrate SmartUI App SDK with your existing mobile app testing framework. - -## Prerequisites - -- Basic understanding of mobile app testing and Appium -- Login to [TestMu AI SmartUI](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) with your credentials -- An active [subscription](https://www.testmuai.com/pricing) plan with valid screenshots limit - -## Create a SmartUI Project - -The first step is to create a project that will contain all your builds. To create a SmartUI Project: - -1. Go to [Projects page](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) -2. Click on the `new project` button -3. Select the platform as CLI for executing your `SDK` tests -4. Add name of the project, approvers for the changes found, and tags for filtering -5. Click on **Submit** - -## Steps to Run Your First Test - -### Step 1: Add the SmartUI SDK Dependency - -Add the following dependency to your `pom.xml` file: - -```xml - -io.github.lambdatest -lambdatest-java-sdk -1.0.8 - -``` - -### Step 2: Configure Your Project Token - -You can configure your project token in one of two ways: - -1. **Using Environment Variables**: - -```bash -export PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -``` - -```bash -set PROJECT_TOKEN=123456#1234abcd-****-****-****-************" -``` - -```powershell -$env:PROJECT_TOKEN=123456#1234abcd-****-****-****-************" -``` - -2. **Directly in the Configuration**: -You can pass the project token directly in your test configuration as shown in Step 3. - -### Step 3: Integrate SmartUI in Your Test Script - -Import the required SmartUI class and add the screenshot capture code where needed: - -```java - -public class YourTestClass { -@Test -public void testMethod() { -// Initialize SmartUI -SmartUIAppSnapshot SmartUI = new SmartUIAppSnapshot(); - -// Configure screenshot settings -Map ssConfig = new HashMap<>(); -// Either use environment variable -ssConfig.put("projectToken", "your-project-token-here"); // Use this if you are not setting the project token in environment variable -// ssConfig.put("buildName", "First Build"); // Optional -ssConfig.put("deviceName", "iPhone 15"); // Required, you can use the variables that you are setting in the cloud capabilities - -ssConfig.put("platform", "iOS"); // Optional,you can use the variables that you are setting in the cloud capabilities - -try { -// Start SmartUI session -SmartUI.start(ssConfig); - -// Your test code here -// Your test code here - Example of native app interactions -driver.findElement(MobileBy.AccessibilityId("username-input")).sendKeys("test@example.com"); -driver.findElement(MobileBy.AccessibilityId("password-input")).sendKeys("password123"); - -// Take screenshot of login form -SmartUI.smartuiAppSnapshot(driver, "Login Form", ssConfig); - -driver.findElement(MobileBy.AccessibilityId("login-button")).click(); - -// Wait for home screen to load -WebDriverWait wait = new WebDriverWait(driver, 10); -wait.until(ExpectedConditions.presenceOfElementLocated(MobileBy.AccessibilityId("home-screen"))); - -// Take screenshot of home screen -SmartUI.smartuiAppSnapshot(driver, "Home Screen", ssConfig); - -// More test steps... - -} finally { -// Stop SmartUI session -SmartUI.stop(); -} -} -} -``` - -### Step 4: Execute the Tests - -Run your tests as you normally would with your cloud provider. SmartUI will automatically capture and process the screenshots for visual regression testing. - -```bash -mvn test -``` - -## Configuration Options - -### Screenshot Configuration Options - -| Key | Description | Required | -|-----|-------------|----------| -| projectToken | Your SmartUI project token | Yes | -| deviceName | Name of the device being tested | Yes | -| buildName | Custom name for your build | No | -| platform | Platform being tested (iOS/Android) | No | - -### Supported Platforms and Devices - -The `deviceName` and `platform` parameters in SmartUI App SDK are used as metadata to ensure consistent screenshot comparison across builds. You can use any device name and platform that matches your cloud provider's capabilities: - -```java -ssConfig.put("deviceName", "iPhone 15"); -ssConfig.put("platform", "iOS"); -``` - -#### Important Notes: -- It is advised to use the same `deviceName` and `platform` combination across builds to compare screenshots of the same device -- These parameters are metadata tags and don't affect the actual device selection on your cloud provider - -Example configurations for different cloud providers: - -```java -// For an iOS test on LambdaTest -ssConfig.put("deviceName", "iPhone 12"); -ssConfig.put("platform", "iOS"); - -// For an Android test on BrowserStack -ssConfig.put("deviceName", "Samsung Galaxy S22"); -ssConfig.put("platform", "Android"); - -// For a custom device on any cloud provider -ssConfig.put("deviceName", "Custom Device Name"); // Use the same name consistently -ssConfig.put("platform", "iOS/Android"); // Use the actual platform -``` - -The device name and platform you specify here are used only for screenshot organization and comparison. The actual device selection is handled by your cloud provider's capabilities configuration. - -## Region-Based Ignore/Select Functionality for Dynamic Content - -To handle dynamic content like timestamps, usernames, or ads that cause false positives in visual comparisons, SmartUI App SDK supports region-based ignore and select functionality using XPath locators. - -You can either: -- **Ignore specific regions** during comparison using `ignoreBoxes` -- **Compare only specific regions** using `selectBoxes` - -This feature requires the `Gson` library for JSON serialization. Add it to your `pom.xml` if not already present: - -```xml - -com.google.code.gson -gson -2.10.1 - -``` - -### Usage Examples - -#### 1. Ignoring Regions (Recommended for Dynamic Content) - -```java -SmartUIAppSnapshot smartUI = new SmartUIAppSnapshot(); - -Map config = new HashMap<>(); -config.put("projectToken", "your-project-token-here"); -config.put("deviceName", "Pixel 6"); -config.put("fullPage", "true"); - -// Define XPaths of elements to ignore -List ignoreXpath = Arrays.asList( -"//*[@text=\"Backpack\"]", -"//*[@text=\"Onesie\"]", -"//*[@text=\"PRODUCTS\"]", -"//*[@text=\"Terms of Service | Privacy Policy\"]" -); - -Map ignoreBoxesMap = new HashMap<>(); -ignoreBoxesMap.put("xpath", ignoreXpath); - -Gson gson = new Gson(); -config.put("ignoreBoxes", gson.toJson(ignoreBoxesMap)); - -smartUI.start(config); -smartUI.smartuiAppSnapshot(driver, "SmartUIAndroid", config); -smartUI.stop(); -``` - -#### 2. Selecting Specific Regions for Comparison - -```java -// Replace "ignoreBoxes" with "selectBoxes" to compare only specified regions -Map selectBoxesMap = new HashMap<>(); -selectBoxesMap.put("xpath", Arrays.asList("//*[@resource-id='primary-content']")); - -config.put("selectBoxes", gson.toJson(selectBoxesMap)); -``` - -### Configuration Keys - -| Key | Type | Description | Required | -|-----|------|-------------|----------| -| `ignoreBoxes` | JSON String | Defines regions to ignore during visual comparison. Accepts XPath locators. | No | -| `selectBoxes` | JSON String | Defines regions to include in visual comparison. Accepts XPath locators. | No | - -**Best Practices** -- Use `ignoreBoxes` for elements that change frequently (e.g., ads, timestamps, user avatars). -- Use `selectBoxes` when you want to focus comparison only on critical UI sections. -- Avoid using both `ignoreBoxes` and `selectBoxes` in the same config; they are mutually exclusive. -- Ensure XPath expressions are unique and stable across test runs. - -## View SmartUI Results - -After test execution, visit your SmartUI project dashboard to: - -1. View all captured screenshots -2. Compare against baseline images -3. Identify visual regressions -4. Approve or reject changes -5. Manage baseline images - -## Best Practices - -**Screenshot Naming** - -- Use descriptive, consistent names for screenshots -- Include screen/component name in screenshot names -- Avoid special characters that might cause issues -- Use consistent naming conventions across your test suite - -**Example:** -```java -SmartUISnapshot.smartuiSnapshot(driver, HomeScreen-Header"); -SmartUISnapshot.smartuiSnapshot(driver, "CheckoutScreen-PaymentForm"); -``` - -**Wait for Screen Load** - -- Always wait for screens to fully load before taking screenshots -- Use Appium's wait methods for dynamic content -- Consider device-specific loading times - -**Example:** -```java -WebDriverWait wait = new WebDriverWait(driver, Duration.ofSeconds(10)); -wait.until(ExpectedConditions.presenceOfElementLocated(By.id(main-content"))); -SmartUISnapshot.smartuiSnapshot(driver, "Screen Loaded"); -``` - -**Handle Dynamic Content** - -- Use `ignoreBoxes` for elements that change between runs (ads, timestamps, user avatars) -- Use `selectBoxes` when you only need to compare specific regions -- Test XPath locators using Appium Inspector before integrating - -**Device Configuration** - -- Use consistent `deviceName` and `platform` combinations across builds -- Document device configurations for reference -- Test on devices that match your user base - -**Test Organization** - -- Group related screenshots in the same build -- Use meaningful build names -- Run tests on consistent device configurations - -**Troubleshooting** - -**Issue: Screenshots Not Captured** - -**Symptoms**: Tests run but no screenshots appear in SmartUI dashboard - -**Possible Causes**: -- Project token not set or incorrect -- Incorrect project name -- Network connectivity issues -- SDK not properly integrated - -**Solutions**: -1. Verify `PROJECT_TOKEN` is set correctly: -```bash -echo $PROJECT_TOKEN -``` - -2. Check project name matches exactly (case-sensitive) - -3. Verify SDK dependency is added to pom.xml: -```xml - -io.github.lambdatest -lambdatest-java-sdk -1.0.2 - -``` - -4. Check network connectivity to TestMu AI servers - -5. Review test execution logs for error messages - -**Issue: Project Not Found" Error** - -**Symptoms**: Error indicating SmartUI project cannot be found - -**Possible Causes**: -- Project name typo or mismatch -- Project deleted -- Wrong account credentials -- Token from wrong project - -**Solutions**: -1. Verify project exists in SmartUI dashboard -2. Copy project token directly from Project Settings -3. Ensure token includes the project ID prefix (e.g., `123456#...`) -4. Check credentials match the account with the project - -**Issue: Screenshots Show Blank or Incorrect Content** - -**Symptoms**: Screenshots captured but show blank screens or incorrect content - -**Possible Causes**: -- Screen not fully loaded -- App state issues -- Timing issues -- Device-specific rendering - -**Solutions**: -1. Add explicit waits before screenshots: -```java -WebDriverWait wait = new WebDriverWait(driver, Duration.ofSeconds(10)); -wait.until(ExpectedConditions.presenceOfElementLocated(By.id(content"))); -``` - -2. Wait for specific elements to be visible: -```java -wait.until(ExpectedConditions.visibilityOfElementLocated(By.id("main-content"))); -``` - -3. Increase wait time for slow-loading screens - -4. Verify app is in correct state before screenshot - -**Issue: ignoreBoxes/selectBoxes Not Working** - -**Symptoms**: Dynamic content still causing false positives - -**Possible Causes**: -- XPath locators incorrect -- Elements not found at screenshot time -- JSON formatting issues - -**Solutions**: -1. Verify XPath locators using Appium Inspector -2. Ensure elements exist at screenshot time -3. Check JSON formatting in configuration: -```java -Map ignoreBoxes = new HashMap<">(); -ignoreBoxes.put(xpath", new String[]{"//*[@text="'Dynamic" Ad']"}); -``` - -4. Test XPath locators in isolation before using in config - -**Issue: Maven Dependencies Not Resolving** - -**Symptoms**: Maven cannot find `lambdatest-java-sdk` or dependencies fail - -**Possible Causes**: -- Incorrect dependency version -- Maven repository access issues -- Network connectivity problems - -**Solutions**: -1. Check latest version on [Maven Central](https://mvnrepository.com/artifact/io.github.lambdatest/lambdatest-java-sdk) -2. Clear Maven cache: -```bash -mvn clean -``` -3. Verify internet connectivity for Maven repository access -4. Check pom.xml for version conflicts - -**Issue: Screenshot Names Not Matching Baseline** - -**Symptoms**: Screenshots appear as New" instead of comparing with baseline - -**Possible Causes**: -- Screenshot name changed -- Baseline doesn't exist -- Name contains special characters - -**Solutions**: -1. Ensure screenshot names are consistent across test runs -2. Verify baseline exists in project -3. Avoid special characters in screenshot names -4. Check for case sensitivity issues - -**Getting Help** - -If you encounter issues not covered here: - -- Review the [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) for detailed solutions -- Check [SmartUI Configuration Options](/support/docs/smartui-sdk-config-options) documentation -- See [Handling Dynamic Data](/support/docs/smartui-handle-dynamic-data) for dynamic content issues -- Visit [TestMu AI Support](https://www.testmuai.com/support) for additional resources -- Contact support at support@testmuai.com or use [24/7 Chat Support](https://www.testmuai.com/support) - -## Additional Resources - -- [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) -- [SmartUI Configuration Options](/support/docs/smartui-sdk-config-options) -- [Handling Dynamic Data](/support/docs/smartui-handle-dynamic-data) -- [Baseline Management](/support/docs/smartui-baseline-management) -- [Running Your First Project](/support/docs/smartui-running-your-first-project) -- [Appium Hooks Documentation](/support/docs/smartui-appium-hooks) -- [SmartUI API Documentation](https://www.testmuai.com/support/api-doc/) diff --git a/static/docs/smartui-appium-sdk.md b/static/docs/smartui-appium-sdk.md deleted file mode 100644 index d8d932a88..000000000 --- a/static/docs/smartui-appium-sdk.md +++ /dev/null @@ -1,143 +0,0 @@ -# Integrate SmartUI with Appium Tests - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Appium is an open-source tool for automating mobile application testing. It allows developers and testers to write tests for native, hybrid, and mobile web applications across multiple platforms, such as iOS and Android, using a single codebase. - -TestMu AI SmartUI supports Appium automation tests for mobile applications through two integration methods: - -- **SDK Integration**: Available for Java -- **Hooks Integration**: Available for JavaScript, Python, Ruby, and C# - -Below is the list of supported languages and their integration methods: - - Java SDK - - JavaScript Hooks - - Python Hooks - - Ruby Hooks - - C# Hooks - - We support all languages and frameworks that are compatible with Appium, so in case your favorite isn't in the table.Don't worry, you can still run the test. Contact Us for any help. - -## Integration Methods - -### SDK Integration (Java) - -The SmartUI App SDK provides a native Java library for integrating visual regression testing into your Appium Java test suite. This method offers: - -- Native Java API for screenshot capture -- Type-safe integration with your existing test framework -- Comprehensive error handling and logging - -**Documentation**: [Appium Java SDK](/support/docs/smartui-appium-java-sdk) - Complete guide with best practices and troubleshooting - -### Hooks Integration (JavaScript, Python, Ruby, C#) - -For languages other than Java, SmartUI uses Hooks integration, which leverages Appium's execute command to capture screenshots. This method: - -- Works with any Appium-compatible language -- Uses simple execute commands for screenshot capture -- Supports both viewport and full-page screenshots - -**Documentation**: [Appium Hooks](/support/docs/smartui-appium-hooks) - Complete guide for JavaScript, Python, Ruby, and C# implementations - -#### Quick Start with Hooks - -**Prerequisites:** -- Appium driver instance initialized -- SmartUI project created with Project Token -- TestMu AI credentials configured - -**Taking Viewport Screenshots:** - -Add the following code snippet in your test where you want to capture a screenshot: - -```javascript -driver.execute("smartui.takeScreenshot="); -``` - -```python -driver.execute("smartui.takeScreenshot=") -``` - -```ruby -driver.execute("smartui.takeScreenshot=") -``` - -```csharp -driver.Execute("smartui.takeScreenshot="); -``` - -**Taking Full Page Screenshots (Native Apps):** - -For apps with scrolling functionality, you can capture full-page screenshots: - -```javascript -let config = { -screenshotName: '', -fullPage: true, -pageCount: 15 // Minimum 1, Maximum 20 -}; -await driver.execute("smartui.takeScreenshot", config); -``` - -```python -config = { -'screenshotName': '', -'fullPage': True, -'pageCount': 15 # Minimum 1, Maximum 20 -} -driver.execute("smartui.takeScreenshot", config) -``` - -```ruby -config = { -'screenshotName' => '', -'fullPage' => true, -'pageCount' => 15 # Minimum 1, Maximum 20 -} -driver.execute("smartui.takeScreenshot", config) -``` - -```csharp -var config = new Dictionary { -{"screenshotName", ""}, -{"fullPage", true}, -{"pageCount", 15} // Minimum 1, Maximum 20 -}; -driver.Execute("smartui.takeScreenshot", config); -``` - -**Configuration:** - -Set your Project Token as an environment variable before running tests: - -```bash -export PROJECT_TOKEN="your_project_token" -``` - -```bash -set PROJECT_TOKEN=your_project_token -``` - -```powershell -$env:PROJECT_TOKEN="your_project_token" -``` - -## Next Steps - -After selecting your preferred language, refer to the specific documentation for detailed integration steps: - -- **[Java SDK](/support/docs/smartui-appium-java-sdk)** - Complete guide with best practices and troubleshooting -- **[Appium Hooks](/support/docs/smartui-appium-hooks)** - For JavaScript, Python, Ruby, and C# implementations - -## Additional Resources - -- [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) - Solutions for common issues -- [Handling Dynamic Data](/support/docs/smartui-handle-dynamic-data) - Handle dynamic content in mobile apps -- [Baseline Management](/support/docs/smartui-baseline-management) - Manage your test baselines -- [Running Your First Project](/support/docs/smartui-running-your-first-project) - Get started with SmartUI diff --git a/static/docs/smartui-approval-workflow-guide.md b/static/docs/smartui-approval-workflow-guide.md deleted file mode 100644 index a1f338d8e..000000000 --- a/static/docs/smartui-approval-workflow-guide.md +++ /dev/null @@ -1,324 +0,0 @@ -# SmartUI Approval, Reject, Move, and Merge Functions Guide - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -SmartUI provides powerful baseline management functions that allow you to control how visual regression tests are handled. Understanding these functions is crucial for maintaining accurate baselines and avoiding confusion about which screenshots serve as reference points for future comparisons. - -****Critical Concept**** -Every baseline change in SmartUI requires **explicit user action**. There is no automatic baseline updating - all baseline modifications must be intentionally triggered by users through approval workflows, CLI commands, or dashboard actions. - -## Core Functions Explained - -### Approve Function -**What it does**: Accepts the current screenshot as the new expected state and updates the baseline for future comparisons. - -**When to use**: -- When visual changes are intentional and expected -- After UI updates, design changes, or feature implementations -- When screenshots represent the correct current state - -**Behavior**: -- Moves screenshot to "Approved" tab -- Updates baseline automatically -- Creates audit trail with approver and timestamp -- Future builds will compare against this approved version - -In CLI-based workflows, approval alone may not be sufficient for baseline persistence across different build contexts. - -### Reject Function -**What it does**: Marks screenshots as unacceptable without updating the baseline. - -**When to use**: -- When visual differences are unexpected or represent bugs -- For temporary issues (loading states, animations) -- When changes are not ready for production - -**Behavior**: -- Moves screenshot to "Rejected" tab -- Preserves original baseline -- Keeps screenshot available for future re-evaluation -- Does not affect future comparisons - -### Move to Baseline Function -**What it does**: Explicitly promotes approved screenshots to become the new baseline reference. - -**When to use**: -- When you want to permanently establish a new reference point -- After major releases or design system updates -- When consolidating changes from multiple builds - -**Behavior**: -- Takes approved screenshots and makes them the new baseline -- Overwrites existing baseline for the same test name + viewport -- Creates permanent reference for all future comparisons -- Essential for CLI workflows where approval alone isn't enough - -**Key Difference from Approve**: Approve updates baseline for that specific build context, while Move to Baseline establishes a global reference. - -### Merge Function -**What it does**: Combines visual regression results from different branches or builds. - -**When to use**: -- When merging feature branches into main -- Consolidating results from multiple environments -- Creating unified baselines from distributed testing - -**CLI Usage**: -```bash -# Merge branches -npx smartui merge branch --source feature-branch --target main - -# Merge builds -npx smartui merge build --source build-name --target baseline-build -``` - -## Workflow Types and Baseline Management - -### CLI-Based Workflows (Explicit User Control) -**Characteristics**: -- Each screenshot must be explicitly approved or moved -- Baseline management is manual and granular -- Greater control but requires more steps -- Best for complex, multi-environment setups - -**Required Actions**: -1. Run build → Compare against baseline -2. Review differences → Approve valid changes -3. **Move approved screenshots to baseline** (critical step) -4. Next build uses updated baseline - -**Baseline Update Mechanism**: -- Approval marks screenshots as accepted within current build -- Move to Baseline explicitly updates the global baseline reference -- Without Move to Baseline, subsequent builds may still compare against old baseline -- All baseline changes require explicit user intervention - -### Hooks/API/PDF Workflows (Capability-Driven) -**Characteristics**: -- Builds can be automatically marked as baseline via capabilities -- Less granular control over individual screenshots -- Simpler workflow for straightforward cases -- Best for continuous integration scenarios - -**Baseline Update Mechanism**: -```javascript -// SDK capability to auto-mark baseline -capabilities.setCapability("smartUI.baseline", true); -``` -- When `smartUI.baseline: true` is set, the entire build becomes the new baseline -- No individual screenshot approval needed -- Automatic progression of reference points -- Still requires explicit capability setting by user - -### Git-Integrated Projects (Branch-Based Automation) -**Characteristics**: -- Baseline automatically managed via Git branches -- Branch-specific baselines maintained -- Automatic baseline updates for baseline branch -- System-approved status for first builds - -**Baseline Update Mechanism**: -- Baseline branch builds auto-update baseline -- Feature branches compare against baseline branch -- No manual Move to Baseline required for baseline branch -- Still requires user to configure baseline branch settings - -## Deep Dive: How Baseline Updates Actually Work - -### The Baseline Reference System -SmartUI maintains baseline references at multiple levels: - -1. **Global Project Baseline**: Default comparison point for all builds -2. **Branch-Specific Baselines**: Different baselines per Git branch -3. **Build-Level Baselines**: Specific builds marked as baseline -4. **Screenshot-Level References**: Individual screenshot comparisons - -### Explicit User Action Requirements -Every baseline modification requires deliberate user intervention: - -**CLI Workflows**: -```bash -# 1. Run tests (no baseline change) -npx smartui exec -- npm test - -# 2. Approve screenshots (marks as accepted, no baseline change) -# Manual dashboard action required - -# 3. EXPLICIT: Move to baseline (updates global baseline) -npx smartui move-to-baseline --build current-build -``` - -**SDK with Capabilities**: -```javascript -// Explicit capability setting required -capabilities.setCapability("smartUI.baseline", true); // User must set this -``` - -**Git Projects**: -```bash -# Explicit baseline branch configuration required -export BASELINE_BRANCH="main" # User must configure -``` - -## Common Customer Confusion Scenarios - -### Scenario 1: "I approved B1, why isn't it the baseline for B5?" - -**Problem**: Customer approved screenshots in Build 1 (B1), ran Build 2 (B2) which became baseline, then when running Build 5 (B5), expected B1 screenshots to be the reference. - -**Root Cause**: In CLI workflows, approval updates baseline within that build's context, but doesn't necessarily persist as the global baseline for future builds unless explicitly moved. - -**Solution**: -```bash -# After approving in B1 -npx smartui move-to-baseline --build B1 - -# Or use the dashboard "Move to Baseline" function -``` - -### Scenario 2: "I approved and moved SS3 from B3 and SS4 from B4 to B2, but B5 shows SS1 as new" - -**Problem**: Customer performed "approve and update" (move) operations on screenshots from different builds, expecting them to consolidate into a single baseline. - -**Root Cause**: Move operations work on individual screenshots, not entire builds. If B2 doesn't contain SS1 (which was in B1), it won't be in the baseline. - -**Solution**: -1. Ensure all required screenshots are present in the target baseline build -2. Use merge functions to combine results from multiple builds -3. Consider using branch-based baselines for complex scenarios - -### Scenario 3: "I set smartUI.baseline=true, but baseline didn't update" - -**Problem**: Customer expected automatic baseline updates but saw no changes. - -**Root Cause**: The capability must be set correctly and the build must complete successfully for baseline marking to occur. - -**Solution**: -```javascript -// Ensure correct capability syntax -capabilities.setCapability("smartUI.baseline", true); - -// Verify build completion -// Check SmartUI dashboard for baseline status -``` - -## Ideal Usage Guidelines - -### For CLI Workflows -```bash -# 1. Run initial build to establish baseline -npx smartui exec -- npm test - -# 2. Make necessary UI changes - -# 3. Run comparison build -npx smartui exec -- npm test - -# 4. Review and approve valid changes in dashboard - -# 5. CRITICAL: Move approved changes to baseline -# Use dashboard "Move to Baseline" or CLI equivalent - -# 6. Future builds will now use updated baseline -``` - -### For SDK Integration -```javascript -// Set capability explicitly for baseline marking -DesiredCapabilities capabilities = new DesiredCapabilities(); -capabilities.setCapability("smartUI.baseline", true); - -// Or use individual screenshot approval -SmartUI.takeScreenshot("screenshot-name"); -// Then manually move to baseline via dashboard -``` - -### Branch-Based Development -```bash -# Feature branch development -npx smartui --baselineBranch main exec -- npm test - -# After approval, merge to main -npx smartui merge branch --source feature-branch --target main -``` -## Advanced Baseline Management - -### Multi-Level Baseline Strategy -```bash -# Different baselines for different environments -npx smartui --baselineBranch staging exec -- npm test # Staging baseline -npx smartui --baselineBranch production exec -- npm test # Production baseline - -# Build-specific comparisons -npx smartui --baselineBuild "v2.1.0" exec -- npm test -``` - -### Baseline Rollback -```bash -# Rollback to previous baseline -npx smartui --baselineBuild "previous-stable-build" exec -- npm test - -# Mark old build as baseline again -npx smartui --markBaseline --baselineBuild "rollback-target" -``` -## Best Practices - -### 1. **Understand Your Workflow Type** -- **CLI**: Requires explicit baseline management -- **Hooks/API**: Capability-driven baseline marking -- **Git**: Branch-based automatic baseline management -- **PDF**: Build-level baseline marking - -### 2. **Always Verify Baseline Changes** -- Check dashboard after baseline operations -- Confirm next build uses updated baseline -- Maintain audit trail of baseline changes -- Only move validated, production-ready screenshots -- Document baseline update reasons -- Maintain separate baselines for different environments - -### 3. **Leverage Branch Merging** -- Use for complex multi-branch workflows -- Maintain audit trails of merges -- Plan baseline consolidation carefully - -### 4. **Monitor Baseline Health** -- Regularly review baseline age and completeness -- Update baselines after major releases -- Clean up outdated baselines - -### 5. **Document Decisions** -- Add comments when approving/rejecting/moving -- Maintain change logs for baseline updates -- Train team on workflow differences - -## Troubleshooting Common Issues - -### "Baseline not updating after approval" -**Cause**: Using CLI workflow without moving to baseline -**Solution**: Explicitly use "Move to Baseline" function - -### "Screenshots missing from baseline" -**Cause**: Moved screenshots from builds that don't contain all required images -**Solution**: Ensure target baseline build has complete screenshot set - -### "Unexpected rejections in new builds" -**Cause**: Baseline updated with incomplete or incorrect screenshots -**Solution**: Review baseline contents before moving - -### "Merge conflicts" -**Cause**: Conflicting screenshots between source and target -**Solution**: Resolve conflicts manually before merging - -### "Capability not working" -**Cause**: Incorrect capability syntax or build failure -**Solution**: Verify capability format and build status - -## Getting Help - -If you encounter issues with baseline management or need clarification on approval workflows: -- Check the [SmartUI Baseline Management Documentation](/support/docs/smartui-baseline-management/) -- Review [Multiselect & Bulk Operations Guide](/support/docs/smartui-multiselect-bulkops/) -- Contact support at support@testmuai.com - -Understanding these functions and their workflow differences will help you maintain accurate visual regression testing and avoid baseline-related confusion. diff --git a/static/docs/smartui-audit-logs-annotations.md b/static/docs/smartui-audit-logs-annotations.md deleted file mode 100644 index 491d1db11..000000000 --- a/static/docs/smartui-audit-logs-annotations.md +++ /dev/null @@ -1,51 +0,0 @@ -# Annotations in SmartUI Audit Logs - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -SmartUI audit logs now surface the **annotations** applied to a screenshot. When someone draws an ignore region on a screenshot, the audit log records the change and shows a preview of the annotated screenshot, so you can see exactly what was marked, on which screenshot, and by whom. - -This is the first phase of annotations in audit logs. More annotation detail and controls will be added to this view soon. - -## The annotation - -An annotation is a region you draw on a screenshot to mark it as ignored during comparison. In the captured view below, the highlighted rectangles are drawn ignore areas. Hovering an area shows its details: the label **Ignored Area (Drawn)**, the source (`Captured View, #5`), and the exact coordinates and size (`X: 406, Y: 38, W: 488, H: 434`). - -Every time these annotations are added or changed, SmartUI records the edit in the audit log. - -## The audit log list - -Open the **Audit Logs** panel to see annotation edits alongside every other tracked event. Each **Screenshot annotation updated** row shows who made the change, when, a plain-language description, and the build and baseline it belongs to. - -| Column | What it shows | -| --- | --- | -| **Date & Time** | When the annotation was updated (e.g. Jul 13, 2026, 6:30 PM). | -| **Users** | The team member who made the change (`chaitanyas`). | -| **Event** | The event type: *Screenshot annotation updated*. | -| **Description** | A readable summary, e.g. *chaitanyas updated annotations for screenshot '1' in build 'm2-phase4-live-2'*. | -| **Build Name / Build # / Baseline #** | The build and baseline the screenshot belongs to. | - -Use the **Events**, **Users**, **Branches**, and **Build No.** filters at the top to narrow the list, or toggle **System Logs** to include automated events. - -## The annotation preview - -Open an entry to see a preview of the screenshot exactly as it was annotated, so you can confirm the change without leaving the log. - -| Element | What it shows | -| --- | --- | -| **Event header** | The event line reads *Screenshot annotation updated*, with the date and time (Jul 13, 2026, 6:26 PM), the build and number (`m2-phase4-live-2 - #5`), the user who made the change (`chaitanyas`), and the count of affected screenshots (1 Screenshots). | -| **Screenshot preview** | A thumbnail of the screenshot with its annotations, captured at the time of the change. | -| **Ignore region** | The highlighted rectangle marks the area set to be ignored during comparison. Its **Ignore** label identifies the annotation type. | -| **Environment** | The browser and resolution the screenshot was captured on (Chrome, 1920x1080). | - -When a screenshot carries more than one annotation, the preview shows every drawn ignore region together. - -## Why it helps - -- **Traceability**: Every annotation change is attributed to a user with a timestamp, so you always know who marked a region and when. -- **Context at a glance**: The inline preview shows the exact region that was ignored, so you do not have to open the build to understand the change. -- **Accountability**: Annotation edits sit alongside approvals, rejections, and baseline changes in the same audit trail. - -## Related documentation - -- [SmartUI Audit & Activity Logs](/support/docs/smartui-audit-logs/): The full audit trail across projects, builds, and screenshots. -- [Ignore Regions](/support/docs/smartui-hooks-region-ignore/): How to draw and manage ignore regions on your screenshots. diff --git a/static/docs/smartui-audit-logs.md b/static/docs/smartui-audit-logs.md deleted file mode 100644 index 52959592a..000000000 --- a/static/docs/smartui-audit-logs.md +++ /dev/null @@ -1,117 +0,0 @@ -# SmartUI Audit & Activity Logs - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -**Audit Logs** give you a permanent, searchable record of every important action in your SmartUI projects: who did it, when, and what changed. Use them for compliance, accountability, and tracing baseline history across projects, builds, and individual screenshots. - -## Why use Audit Logs - -| Need | How Audit Logs help | -| --- | --- | -| **Compliance & audits** | Immutable log of project and baseline changes with timestamps and actors. | -| **Accountability** | Every action is attributed to a user or the system; filter by person to review their impact. | -| **Baseline traceability** | See when a screenshot became the baseline, who approved it, and whether it was manual or automatic. | -| **Human vs system** | Clearly separate manual approvals/rejections from auto-approvals and system-driven changes. | -| **Bulk action clarity** | One log entry per bulk approve/reject, with a visual preview of which screenshots were included. | - -## Where to look: project, build, or screenshot - -Logs are available at three levels. Choose the level that matches what you need to prove or investigate. - -| Level | Use it to | Access | -| --- | --- | --- | -| **Project** | Audit project creation/deletion, settings changes, and baseline merges. | | -| **Build** | Audit build lifecycle, bulk approvals/rejections, and who changed what in a test run. | | -| **Screenshot** | See full baseline and status history for a single asset (approvals, rejections, annotations). | | - -## Tracking baseline and screenshot history - -Use this table to go straight to the right log and filters. - -| What you need | Where | What to do | -| --- | --- | --- | -| When the project baseline was merged or updated | Project-level log | Filter by event type: *Merged to Baseline*, *Project Updated*. Check timestamp and actor. | -| Who approved/rejected screenshots in a build | Build-level log | Filter by event type (*Screenshot Approved*, *Screenshot Rejected*) and optionally by User. | -| Full history of one screenshot (approvals, rejections, annotations) | Screenshot-level log | Open the screenshot → open its Audit Log. Timeline shows every change. | -| Which screenshots were in a bulk approve/reject | Build-level log | Find the bulk event; use the visual preview (hover/click) to see affected screenshots. | -| Whether a change was human or automatic | Any level | Filter by **Actor**: User = human, System = automated. | -| Activity for a specific branch | Project- or build-level log | Use **Branch** filter; add event-type filter for merges/approvals. | -| All actions by one team member | Any level | Use **Users** (Actor) filter. | - -****Baseline history in one place**** -For a **single screenshot**, the **screenshot-level** log is that asset’s baseline history. For **project-wide** baseline changes (e.g. merge to baseline), use **project-level** logs. For **bulk** updates in a build, use **build-level** logs and the bulk-action preview. - -## What each level shows - -**Project history**: Project created/deleted; settings updated; builds merged to baseline. - -**Build history**: Build created/completed/failed; re-runs (re-execute vs re-run comparison); bulk approve/reject; build deleted. - -**Screenshot history**: Approvals and rejections (single and bulk); live comparisons; ignore zones/annotations; screenshot added/deleted. - -## What each log entry contains - -Every event shows **who**, **when**, and **what**: - -| Field | Meaning | -| --- | --- | -| **Date & time** | When the action occurred. | -| **User (actor)** | Who did it: a **User** (team member) or **System** (automated). | -| **Event name** | Action type (e.g. `SS_APPROVE`, `BUILD_UPDATE`). | -| **Description** | Short summary of the action. | -| **Metadata (visuals)** | For bulk actions: list and preview of affected screenshots. | - -## Filters: find the right events quickly - -Use filters to narrow by **who** (user or system), **what** (event type), **branch**, or **screenshot name**. - -**By user or system**: Filter by specific users for accountability, or by **System** to see only automated events (e.g. auto-approvals). - -**By event type**: e.g. Approve, Reject, Merged to Baseline, Build Re-executed. - -**By branch**: Restrict to a Git branch. - -**By screenshot name**: Search for a specific snapshot. - -**Bulk actions**: One event per bulk approve/reject; hover or click to see a visual preview of the screenshots included. - -## Common scenarios: how to get the answer - -### Who approved this screenshot as the new baseline? - -Open the **screenshot** → **Audit Log** (screenshot-level). Filter by *Screenshot Approved* (or similar). The **User** and **Date & time** columns show who and when. - -### Did we merge this branch into baseline, and who did it? - -Open **Project** → **Audit Log** (project-level). Filter by **Branch**, then by event type *Merged to Baseline* (or *Project Updated*). Check **User** and timestamp. - -### Was this build’s baseline changed by a bulk approve? Which screenshots? - -Open the **build** → **Audit Log** (build-level). Filter by *Screenshot Approved* / *Bulk Approve*. Open the event and use the **visual preview** to see the list of screenshots. **User** and **Date & time** show who and when. - -### Was this change manual or auto-approved by the system? - -Open the Audit Log at the level you need. Use **Actor**: **System** = automated; **User(s)** = manual. Use **Event type** to focus on approvals if needed. - -### What did a specific team member do in this project? - -Open **Audit Log** (project- or build-level). Filter by **Users** → select that person. Optionally add **Event type** or **Branch**. You see every approval, rejection, merge, or config change by them. - -### When was this snapshot (by name) approved or rejected? - -Open **Audit Log** (build or project). Use the **Screenshot** or **Search** filter and enter the snapshot name. Review the filtered events for that snapshot’s history. - -## Events that are logged - -| Scope | Events | -| --- | --- | -| **Project** | Created, Updated, Deleted, Merged to Baseline. | -| **Build** | Created, Completed, Updated by User, Re-run Comparison, Re-executed, Deleted, Merged. | -| **Screenshot** | Approved, Rejected, Annotation Updated, Status Changed, Live Comparison Triggered, Added, Deleted. | - -Use the **Event type** filter in the UI to restrict to these categories. - -## Related documentation - -- [Approval & Baseline Management](/support/docs/smartui-approval-workflow-guide/): How approve, reject, move, and merge affect baselines. -- [Multiselect & Bulk Operations](/support/docs/smartui-multiselect-bulkops/): How bulk actions work and how they appear in audit logs. diff --git a/static/docs/smartui-automation-dashboard.md b/static/docs/smartui-automation-dashboard.md deleted file mode 100644 index 4f7a2b6d2..000000000 --- a/static/docs/smartui-automation-dashboard.md +++ /dev/null @@ -1,85 +0,0 @@ -# SmartUI in the Automation Dashboard - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## What this feature is - -Users can now view, group, and review their SmartUI visual regression results directly inside the Automation Dashboard, without switching products. - -Every automation test run that captures SmartUI screenshots now has a dedicated **SmartUI tab** alongside the existing All Commands, Logs, Networks, and Meta Data tabs. From that tab users can see the build's screenshots, their comparison status, group them by test, and jump straight into SmartUI for a full review when they need to. - -This works across **Web Automation** and **App Automation**. - -## The problem it solves - -Until now, understanding why a visual test failed meant working across two separate places: - -1. Users would see a test fail in the Automation Dashboard, then -2. Open SmartUI separately, find the matching build, and triage the screenshots there. - -For teams running large suites, that context-switching added up to significant manual triage overhead: repetitive, slow, and disconnected from the automation run that produced the results. - -This feature closes that gap. The visual results live next to the test that generated them, so triage happens in one flow. - -## What users gain - -| Before | Now | -|---|---| -| Switch between Automation and SmartUI to investigate a failure | Review screenshots in the SmartUI tab on the test page | -| Screenshots listed flat, hard to scan in big runs | Group screenshots by test name / suite | -| Approve or reject one screenshot at a time | Review and approve at the test-case level | -| No direct path between the two products | One-click deep link from a build into full SmartUI | - -## Key capabilities - -### 1. SmartUI tab in the test view - -Open any test in the Automation Dashboard and select the **SmartUI tab**. Users will see the associated Project and Build, a total screenshot count, and a status roll-up showing how many screenshots are new, changed, approved, or rejected. - -### 2. Grid and List views - -Toggle between a **Grid view** (screenshot thumbnails, ideal for a quick visual scan) and a **List view** (compact rows with per-screenshot status counts, ideal for dense builds). Use whichever suits the run. - -### 3. Group by test - -Organize screenshots by automation suite or test name so related screenshots stay together, far easier to navigate than a flat list when a run produces dozens of screenshots. - -### 4. Deep link into SmartUI - -The **View in SmartUI** link on a build takes users straight to the full SmartUI experience for that build. - -## How to use it - -1. Open the **Automation Dashboard** and select the test the user wants to inspect. -2. In the test detail view, click the **SmartUI tab** (next to Meta Data). -3. Review the build summary at the top: Project, Build, screenshot count, and status counts. -4. Choose **Grid** or **List** view using the toggle on the screenshots panel. -5. Optionally, use **Group by** to cluster screenshots by test, and **Status** to filter to a specific state. - -## Availability and access - -- The feature appears as a tab in the test view for any test that has associated SmartUI screenshots; tests without SmartUI data won't show SmartUI content. -- It is rolled out progressively. If the user does not see the SmartUI tab and expects to, their account may not yet be enabled; contact the account team or support. -- Requires an active SmartUI project linked to the automation run. - -## Tips and known behavior - -- **Real Device sessions:** Screenshots are matched to the user's test even when a session identifier isn't present, so results display reliably for Real Device runs. -- **Mixed runs:** App Automation and Web-Automation-with-Real-Device tests each route to their correct dashboard when the user follows a deep link. -- For the cleanest grouping, use consistent, descriptive test names in the automation suite; these are what the **Group by test** view uses to cluster screenshots. - -## FAQ - -**Do I need to change my test scripts to use this?** -No. If the user's automation run already captures SmartUI screenshots, they'll appear in the SmartUI tab automatically. - -**Is this a replacement for the SmartUI product?** -No, it's an embedded view for fast, in-context triage. For the full diff viewer, baseline management, and ignore-region tools, use **View in SmartUI** to open the complete experience. - -**Will it work for my mobile / app tests?** -Yes. The integration supports Web Automation, App Automation, and Real Device sessions. - -**Why don't I see the SmartUI tab on a particular test?** -That test likely has no associated SmartUI screenshots, or the feature isn't yet enabled for the user's account. Check that a SmartUI project is linked to the run, and reach out to support if needed. - -Need help getting this enabled for an account? Contact the TestMu AI account team or support. diff --git a/static/docs/smartui-baseline-management.md b/static/docs/smartui-baseline-management.md deleted file mode 100644 index 67c6916fa..000000000 --- a/static/docs/smartui-baseline-management.md +++ /dev/null @@ -1,278 +0,0 @@ -# Baseline Management - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -In SmartUI, a **Baseline** is the reference image used for comparison against future test runs. Effective baseline management is crucial for accurate visual regression testing. This guide explains how baselines are created, updated, and maintained. - -## What is a Baseline? -A baseline represents the "expected" state of your application's UI. When you run a SmartUI test, the new screenshots are compared against this baseline to detect unauthorized visual changes. - -A baseline in SmartUI represents the reference point for visual regression testing. It's the set of screenshots against which all future builds are compared. - -For detailed guidance on approval, reject, move, and merge functions, see our comprehensive [Approval & Baseline Management Guide](/support/docs/smartui-approval-workflow-guide/). - -### Types of Baselines - -1. **Project Baseline**: Set in project settings, used as the default comparison point -2. **Branch Baseline**: Specific to a branch, used when Smart Git is enabled -3. **Dynamic Baseline**: Specified during test execution -4. **Build Baseline**: A specific build marked as baseline - -## Dynamic Baseline Specification - -SmartUI allows you to specify baselines dynamically during test execution, providing flexibility in your testing workflow. - -### Branch-based Baseline - -```bash -npx smartui --baselineBranch "branch-name" exec -- -``` - -**Use Cases**: -- Testing against a specific branch's latest approved build -- Comparing feature branch against development branch -- Testing hotfix against production branch - -### Build-based Baseline - -```bash -npx smartui --baselineBuild "build-name" exec -- -``` - -**Use Cases**: -- Testing against a specific build version -- Comparing against a known good build -- Testing against a production build - -## Mark as Baseline - -SmartUI provides the ability to mark builds as baseline directly through the CLI. This overwrites the global baseline branch present in the Project Settings. - -### Basic Usage - -```bash -npx smartui --markBaseline exec -- -``` - -## Baseline Management Strategies - -### 1. Feature Development - -**Scenario**: Managing baselines during feature development - -```bash -# 1. Set feature branch baseline -npx smartui --baselineBranch "feature/new-login" exec -- -``` - -### 2. Hotfix Management - -**Scenario**: Managing baselines for hotfixes - -```bash -# 1. Set production as baseline -npx smartui --baselineBranch "production" exec -- -``` - -### 3. Release Management - -**Scenario**: Managing baselines for releases - -```bash -# 1. Set staging as baseline -npx smartui --baselineBranch "staging" upload - -# 2. Mark release as new baseline -npx smartui --markBaseline capture urls.json -``` - -## Best Practices - -**Baseline Selection** - -- Choose stable builds as baselines -- Document baseline selection criteria -- Regular baseline updates - -**Branch Management** - -- Clear branch naming conventions -- Document branch purposes -- Regular branch cleanup - -**Build Management** - -- Meaningful build names -- Document build purposes -- Regular build cleanup - -**Approval Process** - -- Clear approval criteria -- Document approval decisions -- Maintain audit trail - -## Troubleshooting - -**Common Issues** - -1. **Baseline Issues**: - - Verify baseline exists - - Check baseline status - - Review baseline history - -2. **Comparison Issues**: - - Verify build compatibility - - Check branch status - - Review comparison settings - -3. **Permission Issues**: - - Verify user permissions - - Check branch protection - - Review access settings - -**Baseline Not Found** - -**Error**: Baseline branch or build does not exist - -**Solutions**: -1. Verify the baseline branch/build name is correct -2. Ensure the baseline has at least one approved build -3. Check branch/build exists in your project - -```bash -# Verify baseline exists before using -npx smartui --baselineBranch "main" exec -- -``` - -**Permission Denied** - -**Error**: Insufficient permissions to set baseline - -**Solutions**: -1. Verify you have admin or approver permissions -2. Check project settings for baseline management permissions -3. Contact project admin for access - -**Baseline Mismatch** - -**Error**: Screenshot names don't match between baseline and current build - -**Solutions**: -1. Ensure screenshot names are consistent -2. Check if screenshot names changed between builds -3. Review screenshot naming conventions - -## Visual Examples - -### Setting Baseline via Dashboard - -1. Navigate to your project in SmartUI dashboard -2. Open the build you want to set as baseline -3. Click on the "Mark as Baseline" button -4. Confirm the action - -### Baseline Branch Configuration - -In Project Settings → Build Settings → Git Settings: -- Set your baseline branch name (e.g., "main", "production") -- Configure auto-approval branches -- Link your Git repository - -### Dynamic Baseline in CLI - -```bash -# Use specific branch as baseline -npx smartui --baselineBranch "staging" exec -- npm test - -# Use specific build as baseline -npx smartui --baselineBuild "Release-1.0" exec -- npm test - -# Mark current build as baseline -npx smartui --markBaseline exec -- npm test -``` - -## Advanced Scenarios - -### Scenario 1: Multi-Branch Baseline Strategy - -**Use Case**: Different baselines for different branches - -```bash -# Feature branch compares against develop -npx smartui --baselineBranch "develop" exec -- npm test - -# Release branch compares against staging -npx smartui --baselineBranch "staging" exec -- npm test -``` - -### Scenario 2: Version-Based Baselines - -**Use Case**: Compare against specific version builds - -```bash -# Compare against v1.0.0 build -npx smartui --baselineBuild "v1.0.0" exec -- npm test - -# Compare against v2.0.0 build -npx smartui --baselineBuild "v2.0.0" exec -- npm test -``` - -### Scenario 3: Hotfix Baseline Management - -**Use Case**: Hotfix needs to compare against production - -```bash -# Hotfix branch compares against production -npx smartui --baselineBranch "production" exec -- npm test - -# After approval, mark hotfix as new baseline -npx smartui --markBaseline exec -- npm test -``` - -## Integration with Git - -### Git-Based Projects - -For projects integrated with Git: - -1. Baseline is automatically managed via Git branches -2. Baseline branch is set in Project Settings → Git Settings -3. Non-baseline branches compare against baseline branch -4. Smart Baseline feature is not available for Git projects - -### Git Branching Strategy - -```bash -# Main branch is baseline -Baseline: main - -# Feature branches compare against main -Feature: feature/login → compares against main - -# Release branches compare against main -Release: release/1.0 → compares against main -``` - -## Monitoring and Maintenance - -### Baseline Health Checks - -Regularly monitor: -- Baseline build age (update stale baselines) -- Baseline screenshot count (ensure completeness) -- Baseline approval status (verify all screenshots approved) - -### Baseline Updates - -Best practices for updating baselines: -1. Update after major releases -2. Update after UI framework changes -3. Update after design system updates -4. Document baseline update reason - -For detailed guidance on approval, reject, move, and merge functions across different workflows, see our [Approval & Baseline Management Guide](/support/docs/smartui-approval-workflow-guide/). - -### Getting Help - -If you encounter any issues with baseline management in SmartUI, please contact our support team at support@testmuai.com or use the [24/7 Chat Support](https://www.testmuai.com/support). diff --git a/static/docs/smartui-branch-merging.md b/static/docs/smartui-branch-merging.md deleted file mode 100644 index a4eee7ab0..000000000 --- a/static/docs/smartui-branch-merging.md +++ /dev/null @@ -1,234 +0,0 @@ -# Branch-Level Merging in SmartUI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This guide explains how to effectively merge branches in SmartUI for managing your visual regression testing workflow. - -## Branch-Level Merging - -Branch-level merging allows you to merge visual regression test results from one branch to another, maintaining a clear history of changes and approvals. - -### Merge Command - -```bash -npx smartui merge branch --source --target -``` - -### Merge Process - -1. **Source Branch Selection**: Identifies the latest approved build in the source branch -2. **Target Branch Update**: Updates the target branch with the approved changes -3. **Status Updates**: Updates build statuses in both branches -4. **Audit Trail**: Creates a merge commit with detailed information -5. **Build Naming**: - - For branch merges: `merged-branch/-` - - For build merges: `merged-build/-` - -### Example Workflow - -```bash -# Merge feature branch into main -npx smartui merge branch --source feature/new-login --target main -# Creates: merged-branch/feature-new-login-main - -# Merge hotfix into staging -npx smartui merge branch --source hotfix/security-patch --target staging -# Creates: merged-branch/hotfix-security-patch-staging -``` - -### Merge Behavior - -1. **Latest Version Merge**: Merges the latest approved version of the source branch -2. **Status Updates**: Updates build statuses in both branches -3. **Audit Trail**: Maintains a detailed merge history -4. **Merged Build Status**: The newly created merged build is always `approved` -5. **Naming Conventions**: - - Branch merges: `merged-branch/-` - - Build merges: `merged-build/-` - -## Branch Merge Strategies - -### 1. Feature Branch Strategy - -**Scenario**: Merging a feature branch into main after approval - -```bash -# 1. Ensure feature branch is approved -npx smartui merge branch --source feature/new-login --target main -``` - -### 2. Hotfix Strategy - -**Scenario**: Merging a hotfix into multiple branches - -```bash -# 1. Merge to staging -npx smartui merge branch --source hotfix/security-patch --target staging - -# 2. Merge to production -npx smartui merge branch --source hotfix/security-patch --target production -``` - -### 3. Release Branch Strategy - -**Scenario**: Managing release branches - -```bash -# 1. Create release branch -git checkout -b release/v1.0.0 - -# 2. Merge development into release -npx smartui merge branch --source development --target release/v1.0.0 - -# 3. After testing, merge to main -npx smartui merge branch --source release/v1.0.0 --target main -``` - -## Best Practices - -**Merge Planning** - -- Plan merges in advance - - Document merge strategies - - Establish approval processes - -**Branch Management** - -- Keep branches up to date - - Clean up merged branches - - Maintain clear branch naming - -**Approval Process** - -- Establish clear approval criteria - - Document approval decisions - - Maintain audit trail - -## Troubleshooting - -Merge Conflicts - -- Check build compatibility - - Verify branch status - - Review merge history - -Status Issues - -- Verify build status - - Check permissions - - Review approval history -If you encounter any issues with branch merging in SmartUI, please contact our support team at support@testmuai.com. - -## Pull Request Workflow with Branch Merging - -### Scenario: Feature Development with Multiple Iterations - -Let's walk through a complete workflow where a team is developing a new feature with multiple iterations and PRs. - -```bash -# Initial feature development -git checkout -b feature/new-dashboard - -# First iteration of changes -npx smartui --config .smartui.json exec -- --buildName "dashboard-v1-tests" - -# Create first PR -git push origin feature/new-dashboard -# Create PR on GitHub/GitLab/etc. - -# After review feedback, make changes -git commit -m "Address review comments" -npx smartui exec -- --buildName "dashboard-v2-capture" - -# PR is approved and ready to merge -# First, merge the visual changes -npx smartui merge branch --source feature/new-dashboard --target main - -# Then merge the code -git checkout main -git merge feature/new-dashboard - -# Run final tests on main -npx smartui --config .smartui.json exec -- --buildName "main-dashboard-merge" -``` - -### CI/CD Integration with Branch Merging - -Here's a GitHub Actions workflow that automates the branch merging process in your CI/CD pipeline: - -```yaml -name: Visual Regression Tests with Branch Merging - -on: -pull_request: -types: [opened, synchronize, reopened] -branches: [ main, develop ] - -jobs: -visual-tests: -runs-on: ubuntu-latest -steps: -- uses: actions/checkout@v2 - -- name: Setup Node.js -uses: actions/setup-node@v2 -with: -node-version: '16' - -- name: Install dependencies -run: npm install - -- name: Run SmartUI tests -env: -PROJECT_TOKEN: ${{ secrets.PROJECT_TOKEN }} -run: | -# Run different types of tests -npx smartui --config .smartui.json exec -- --buildName "pr-${{ github.event.pull_request.number }}-tests" - -merge-visual-changes: -needs: visual-tests -if: github.event.pull_request.merged == true -runs-on: ubuntu-latest -steps: -- uses: actions/checkout@v2 - -- name: Setup Node.js -uses: actions/setup-node@v2 -with: -node-version: '16' - -- name: Install dependencies -run: npm install - -- name: Merge visual changes -env: -PROJECT_TOKEN: ${{ secrets.PROJECT_TOKEN }} -run: | -# Merge visual changes from PR branch to target branch -# Creates: merged-branch/- -npx smartui merge branch --source ${{ github.event.pull_request.head.ref }} --target ${{ github.event.pull_request.base.ref }} -``` - -### Best Practices for PR Branch Merging - -1. **Before Merging**: - - Ensure all visual changes are approved - - Run final tests on the feature branch - - Document any special merge considerations - - Note the expected merge naming conventions: - - Branch merges: `merged-branch/-` - - Build merges: `merged-build/-` - -2. **During Merge**: - - Use descriptive build names - - Include PR number in build names - - Run tests after merge - - Verify merge naming follows conventions - -3. **After Merge**: - - Verify merged changes - - Clean up old builds - - Update documentation - - Check merge naming in SmartUI dashboard - -The branch merging process works consistently across all SmartUI CLI commands (`exec`, `capture`, `upload`, `upload-figma-web`, `upload-figma-app`), ensuring a unified experience whether you're executing tests, capturing screenshots, uploading images, or comparing Figma designs with web pages or mobile apps. Each command supports the `--buildName` flag to provide custom names for your builds. diff --git a/static/docs/smartui-build-merging.md b/static/docs/smartui-build-merging.md deleted file mode 100644 index cd1f18d53..000000000 --- a/static/docs/smartui-build-merging.md +++ /dev/null @@ -1,104 +0,0 @@ -# Build-Level Merging in SmartUI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This guide explains how to effectively merge builds in SmartUI for granular control over your visual regression testing workflow. - -## Build-Level Merging - -Build-level merging provides granular control over specific builds, allowing you to merge individual build results and manage your visual regression testing at a more detailed level. - -### Merge Command - -```bash -npx smartui merge build --source --target -``` - -### Merge Process - -1. **Build Selection**: Identifies the source and target builds -2. **Content Merge**: Merges the visual regression results -3. **Status Update**: Updates build statuses -4. **Confirmation**: Provides merge confirmation - -### Example Workflow - -```bash -# Merge specific builds -npx smartui merge build --source build-123 --target build-456 - -# Merge with status update -npx smartui merge build --source build-123 --target build-456 -``` - -### Merge Behavior - -1. **Build Merging**: Merges the source build into the target build -2. **Status Updates**: Updates the new merged build status to "approved" -3. **Content Updates**: Updates target build with merged content -4. **Confirmation**: Provides detailed merge confirmation -5. **Build Naming**: - - For branch merges: `merged-branch/-` - - For build merges: `merged-build/-` - -## Build Merge Strategies - -### 1. Build Promotion Strategy - -**Scenario**: Promoting specific builds across environments - -```bash -# 1. Merge staging build to production -npx smartui merge build --source staging-build-123 --target prod-build-456 - -### 2. Feature Build Strategy - -**Scenario**: Managing feature-specific builds - -```bash -# 1. Merge feature build into main build -npx smartui merge build --source feature-build-789 --target main-build-101 - -### 3. Hotfix Build Strategy - -**Scenario**: Managing hotfix builds - -```bash -# 1. Create hotfix build -npx smartui capture --name hotfix-build-202 - -# 2. Merge into production build -npx smartui merge build --source hotfix-build-202 --target prod-build-456 -``` - -## Best Practices - -**Build Management** - -- Use meaningful build names - - Document build purposes - - Regular cleanup of old builds - -**Merge Planning** - -- Plan merges in advance - - Document merge strategies - - Establish approval processes - -**Getting Help** - -If you encounter any issues with build merging in SmartUI, please contact our support team at support@testmuai.com. - -1. **Build Management**: - - Use meaningful build names - - Document build purposes - - Regular cleanup of old builds - -2. **Merge Planning**: - - Plan merges in advance - - Document merge strategies - - Establish approval processes - -### Getting Help - -If you encounter any issues with build merging in SmartUI, please contact our support team at support@testmuai.com. diff --git a/static/docs/smartui-cli-basic-auth.md b/static/docs/smartui-cli-basic-auth.md deleted file mode 100644 index 095f02ada..000000000 --- a/static/docs/smartui-cli-basic-auth.md +++ /dev/null @@ -1,94 +0,0 @@ -# Basic Authentication in SmartUI CLI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -When capturing screenshots of applications or websites that are protected with authentication, you need to configure SmartUI to pass the necessary authentication headers. This ensures that SmartUI can access and properly render all assets on the page. - -## Why Basic Authentication is Needed - -Without proper authentication configuration: -- Protected assets may fail to load -- Screenshots might be incomplete or broken -- CSS and other resources behind authentication may not be accessible - -## Configuration - -Add the following configuration to your `.smartui.json` file to enable basic authentication: - -```json -{ -"basicAuthorization": { -"username": "username", -"password": "password" -} -} -``` - -### Configuration Parameters - -- **username**: Your authentication username -- **password**: Your authentication password - -## Example Usage - -Here's a complete example of a SmartUI configuration file with basic authentication: - -```json -{ -"web": { -"browsers": ["chrome", "firefox"], -"viewports": [ -[1920, 1080], -[1366, 768] -] -}, -"basicAuthorization": { -"username": "your-username", -"password": "your-password" -}, -"waitForTimeout": 1000 -} -``` - -## Using Environment Variables - -For better security, you can use environment variables for your authentication credentials: - -```json -{ -"basicAuthorization": { -"username": "${AUTH_USERNAME}", -"password": "${AUTH_PASSWORD}" -} -} -``` - -Then set your environment variables: - -```bash -export AUTH_USERNAME="your-username" -``` - -```bash -set AUTH_USERNAME="your-username" -``` - -```powershell -$env:AUTH_USERNAME="your-username" -``` - -## Common Issues and Solutions - -1. **Assets Not Loading** - - Verify that the provided credentials have access to all required resources - - Check if any assets are served from different domains requiring separate authentication - -2. **Authentication Failures** - - Ensure credentials are correct and active - - Verify that the authentication endpoint is accessible from TestMu AI's infrastructure - -When using basic authentication: -1. Test your credentials manually before running SmartUI tests -2. Ensure all required assets are accessible with the provided credentials -3. Use secure methods to manage your authentication credentials -4. Consider implementing a test user specifically for visual testing diff --git a/static/docs/smartui-cli-build-name.md b/static/docs/smartui-cli-build-name.md deleted file mode 100644 index 617222e7e..000000000 --- a/static/docs/smartui-cli-build-name.md +++ /dev/null @@ -1,98 +0,0 @@ -# Grouping Screenshots through Build Names - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -SmartUI CLI allows you to group screenshots in a build name and append new screenshots to an existing build. This feature enables you to assign build names, add screenshots to existing builds and manage your visual tests efficiently. - -## Prerequisites - -- Basic understanding of Command Line Interface -- Login to [TestMu AI SmartUI](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) with your credentials. -- Ensure you are using `@lambdatest/smartui-cli` version 4.1.43 or higher -- A properly configured SmartUI CLI project - -If you face any problems executing tests with SmartUI-CLI `versions >= v4.x.x`, upgrade your Node.js version to `v20.3` or above. - -## Step 1: Install SmartUI CLI - -If you haven't already installed SmartUI CLI, install it using npm: - -**Global Installation (Recommended):** -```bash -npm install -g @lambdatest/smartui-cli -``` - -**Local Installation:** -```bash -npm install @lambdatest/smartui-cli -``` - -## Step 2: Configure your Project Token - -Setup your project token shown in the **SmartUI** app after creating your project. - -```bash -export PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -``` - -```bash -set PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -``` - -```powershell -$env:PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -``` - -## Step 3: Execute Tests with custom Build Names - -You can add a custom build name by adding the `--buildName` flag to your test execution command. Here are different ways to use this feature: - -#### Custom Build Name (Expected Usage) -Specify a custom build name to group your screenshots in the following way: - -```bash -npx smartui --config .smartui.json exec --buildName "Sample Build Name" -- -``` ->**Example:** - For a `Node.js` test script : ->```bash ->npx smartui --config .smartui.json exec --buildName "Sample Build Name" -- node test.js ->``` - -### Default Usage -If no buildname is specified, a random build name is added to the builds: - -```bash -npx smartui --config .smartui.json exec -- -``` - -## Step 4: Advanced Use Case - -If you are running multiple test cases in a single job (e.g., using HyperExecute) and want to club all screenshots under a single build while maintaining consistent Git baseline management, you can do the following: - -This can be done as follows: - -```bash -npx smartui --config .smartui.json exec --buildName $env:JOB_ID -- node test.js -``` - -## Key Benefits -- **Efficient Grouping:** Screenshots are grouped by build names, improving organization and traceability. -- **Seamless Git Integration:** Integrates smoothly with Git to maintain baseline integrity across branches. -- **Flexible Updates:** Allows appending screenshots to existing builds, enhancing adaptability. -- **Optimized Workflows:** Supports streamlined operations for both parallel and single-job executions. - -## Conclusion -This approach provides a structured and efficient way to manage visual tests: -- **New Build for Each HyperExecute Job:** A dedicated build is created for every HyperExecute job, ensuring clear separation and traceability. -- **Consolidated Screenshot Grouping:** Screenshots from all tests within the same job are appended to a single build for seamless organization. -- **Uninterrupted Git Workflow:** Git branching strategies remain unaffected, enabling robust baseline management across branches. - -## Additional Resources - -- [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) -- [CLI Documentation](/support/docs/smartui-cli) -- [CLI Exec Commands](/support/docs/smartui-cli-exec) -- [Baseline Management](/support/docs/smartui-baseline-management) -- [Running Your First Project](/support/docs/smartui-running-your-first-project) -- [SmartUI API Documentation](https://www.testmuai.com/support/api-doc/) diff --git a/static/docs/smartui-cli-complete-reference.md b/static/docs/smartui-cli-complete-reference.md deleted file mode 100644 index 577e40175..000000000 --- a/static/docs/smartui-cli-complete-reference.md +++ /dev/null @@ -1,839 +0,0 @@ -# SmartUI CLI Complete Reference Guide - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This comprehensive reference guide covers all SmartUI CLI and Storybook CLI commands, options, advanced features, and end-to-end workflows for visual regression testing. - -## Prerequisites - -- Node.js v20.3+ installed (required for SmartUI CLI v4.x.x) -- npm or yarn package manager -- TestMu AI account credentials -- SmartUI project created - -If you face any problems executing tests with SmartUI-CLI `versions >= v4.x.x`, upgrade your Node.js version to `v20.3` or above. - -## Installation - -```bash -npm install -g @lambdatest/smartui-cli@latest -``` -**Current Version**: v4.1.43+ - -```bash -npm install -g @lambdatest/smartui-storybook@latest -``` -**Current Version**: v1.1.29+ - -## SmartUI CLI Commands - -### Main Commands Overview - -| Command | Description | Usage | -|---------|------------|-------| -| `exec` | Execute test commands with SmartUI server | `smartui exec [options] -- ` | -| `capture` | Capture screenshots of static URLs | `smartui capture [options] ` | -| `upload` | Upload screenshots from directory | `smartui upload [options] ` | -| `upload-pdf` | Upload PDFs for visual comparison | `smartui upload-pdf [options] ` | - -| Command | Description | Usage | -|---------|------------|-------| -| `exec:start` | Start SmartUI snapshot server | `smartui exec:start [options]` | -| `exec:stop` | Stop SmartUI snapshot server | `smartui exec:stop` | -| `exec:ping` | Check if server is running | `smartui exec:ping` | -| `exec:pingTest` | Ping server using default HTTP client | `smartui exec:pingTest` | - -| Command | Description | Usage | -|---------|------------|-------| -| `config:create` | Create SmartUI config file | `smartui config:create [filepath]` | -| `config:create-web-static` | Create Web Static config file | `smartui config:create-web-static [filepath]` | -| `config:create-figma` | Create Figma designs config file | `smartui config:create-figma [filepath]` | -| `config:create-figma-web` | Create Figma config with browsers | `smartui config:create-figma-web [filepath]` | -| `config:create-figma-app` | Create Figma config for mobile apps | `smartui config:create-figma-app [filepath]` | - -| Command | Description | Usage | -|---------|------------|-------| -| `upload-figma` | Capture Figma screenshots | `smartui upload-figma [options] ` | -| `upload-figma-web` | Capture Figma into CLI build | `smartui upload-figma-web [options] ` | -| `upload-figma-app` | Capture Figma into App build | `smartui upload-figma-app [options] ` | - -| Command | Description | Usage | -|---------|------------|-------| -| `merge branch` | Merge source branch into target | `smartui merge branch [options]` | -| `merge build` | Merge source build into target | `smartui merge build [options]` | - -## Command Details - -**Execute Tests** - -Execute test commands with SmartUI server running. - -**Syntax:** -```bash -smartui exec [options] -- -``` -**Arguments:** - -| Argument | Description | Example | -|----------|-------------|---------| -| `` | Test command to execute | `npm test`, `mvn test`, `python test.py` | - -**Options:** - -| Option | Short | Description | Default | -|--------|-------|-------------|---------| -| `--port ` | `-P` | Port number for the server | `49152` | -| `--fetch-results [filename]` | | Fetch results and save to JSON file | `results.json` | -| `--buildName ` | | Custom build name for test run | Random | -| `--scheduled ` | | Specify schedule ID | - | -| `--userName ` | | TestMu AI username (overrides env) | - | -| `--accessKey ` | | TestMu AI access key (overrides env) | - | -| `--config ` | `-c` | Configuration file path | `.smartui.json` | - -**Examples:** -```bash -# Basic execution -smartui exec -- npm test - -# Custom port -smartui exec -P 5000 -- npm test - -# With build name and results -smartui exec --buildName "Release-1.0" --fetch-results results.json -- npm test - -# With custom config -smartui exec --config custom-config.json -- npm test - -# Override credentials -smartui exec --userName "user" --accessKey "key" -- npm test -``` - -**Capture Static URLs** - -Capture screenshots of static URLs from a configuration file. - -**Syntax:** -```bash -smartui capture [options] -``` -**Arguments:** - -| Argument | Description | Example | -|----------|-------------|---------| -| `` | Web static config file | `urls.json` | - -**Options:** - -| Option | Short | Description | Default | -|--------|-------|-------------|---------| -| `--parallel [number]` | `-C` | Number of parallel instances per browser | Auto-calculated | -| `--force` | `-F` | Forcefully apply parallel instances | `false` | -| `--fetch-results [filename]` | | Fetch results and save to JSON | `results.json` | -| `--buildName ` | | Custom build name | Random | -| `--scheduled ` | | Schedule ID | - | -| `--userName ` | | TestMu AI username | - | -| `--accessKey ` | | TestMu AI access key | - | -| `--config ` | `-c` | Configuration file path | `.smartui.json` | - -**Examples:** -```bash -# Basic capture -smartui capture urls.json --config .smartui.json - -# With parallel execution -smartui capture urls.json --parallel 3 --config .smartui.json - -# Force parallel execution -smartui capture urls.json --parallel 5 --force --config .smartui.json - -# With build name and results -smartui capture urls.json --buildName "Daily-Run" --fetch-results daily-results.json -``` -**Parallel Execution:** -- Maximum parallel threads: `log₂(N)` where N = total URLs -- Example: 100 URLs = max 6 threads, 50 URLs = max 5 threads - -**Upload Screenshots** - -Upload screenshots from a directory for comparison. - -**Syntax:** -```bash -smartui upload [options] -``` -**Arguments:** - -| Argument | Description | Example | -|----------|-------------|---------| -| `` | Path to directory containing screenshots | `./screenshots` | - -**Options:** - -| Option | Short | Description | Default | -|--------|-------|-------------|---------| -| `--ignoreResolutions` | `-R` | Ignore resolutions, compare by name only | `false` | -| `--files ` | `-F` | Comma-separated allowed extensions | All image types | -| `--removeExtensions` | `-E` | Strip extensions from snapshot names | `false` | -| `--ignoreDir ` | `-i` | Comma-separated directories to ignore | - | -| `--fetch-results [filename]` | | Fetch results and save to JSON | `results.json` | -| `--buildName ` | | Custom build name | Random | -| `--userName ` | | TestMu AI username | - | -| `--accessKey ` | | TestMu AI access key | - | - -**Examples:** -```bash -# Basic upload -smartui upload ./screenshots - -# Ignore resolutions -smartui upload ./screenshots --ignoreResolutions - -# Filter file types -smartui upload ./screenshots --files jpg,png - -# Remove extensions from names -smartui upload ./screenshots --removeExtensions - -# Ignore specific directories -smartui upload ./screenshots --ignoreDir temp,old - -# Combined options -smartui upload ./screenshots -R -E --buildName "Test-Run" -``` - -**Upload PDFs** - -Upload PDF files for visual comparison. - -**Syntax:** -```bash -smartui upload-pdf [options] -``` -**Arguments:** - -| Argument | Description | Example | -|----------|-------------|---------| -| `` | Path to directory or single PDF file | `./pdfs` or `./document.pdf` | - -**Options:** - -| Option | Description | Default | -|--------|-------------|---------| -| `--fetch-results [filename]` | Fetch results and save to JSON | `results.json` | -| `--buildName ` | Custom build name | Random | -| `--markBaseline` | Mark this build as baseline | `false` | -| `--projectToken ` | Project token (if not in env) | - | - -**Examples:** -```bash -# Upload directory of PDFs -smartui upload-pdf ./pdfs --buildName "Release-v2.1" - -# Upload single PDF -smartui upload-pdf ./document.pdf --buildName "Single-PDF" - -# Mark as baseline -smartui upload-pdf ./pdfs --markBaseline --buildName "Baseline-v1.0" - -# With results -smartui upload-pdf ./pdfs --fetch-results pdf-results.json -``` - -**Start Server** - -Start the SmartUI snapshot server. - -**Syntax:** -```bash -smartui exec:start [options] -``` -**Options:** - -| Option | Short | Description | Default | -|--------|-------|-------------|---------| -| `--port ` | `-P` | Port number for server | `49152` | -| `--fetch-results [filename]` | | Fetch results and save to JSON | `results.json` | -| `--buildName ` | | Custom build name | Random | - -**Examples:** -```bash -# Start on default port -smartui exec:start - -# Start on custom port -smartui exec:start -P 5000 - -# With build name -smartui exec:start --buildName "Server-Build" -``` -**Usage Workflow:** - -1. Start server: -```bash -smartui exec:start -``` -2. Set server address (for non-Selenium SDKs): -```bash -export SMARTUI_SERVER_ADDRESS='http://localhost:49152' -``` -3. Run tests: -```bash -npm test -``` -4. Stop server: -```bash -smartui exec:stop -``` - -**Stop Server** - -Stop the SmartUI snapshot server. - -**Syntax:** -```bash -smartui exec:stop -``` -**Important:** Always use `exec:stop` instead of Ctrl+C. Using Ctrl+C will cause the build to stop after 12 minutes. - -**Check Server Status** - -Check if the SmartUI server is running. - -**Syntax:** -```bash -smartui exec:ping -# or -smartui exec:pingTest -``` -**Command Comparison:** - -| Command | Description | HTTP Client | -|---------|-------------|-------------| -| `exec:ping` | Check server status | Custom HTTP client | -| `exec:pingTest` | Check server status | Default HTTP client | - -**Merge Branches/Builds** - -Merge source branch or build into target. - -**Merge Branch Syntax:** -```bash -smartui merge branch [options] -``` -**Merge Branch Options:** - -| Option | Description | Required | -|--------|-------------|----------| -| `--source ` | Source branch to merge | Yes | -| `--target ` | Target branch to merge into | Yes | - -**Merge Build Syntax:** -```bash -smartui merge build [options] -``` -**Merge Build Options:** - -| Option | Description | Required | -|--------|-------------|----------| -| `--source ` | Source build to merge | Yes | -| `--target ` | Target build to merge into | Yes | - -**Examples:** -```bash -# Merge feature branch into main -smartui merge branch --source feature/new-ui --target main - -# Merge build -smartui merge build --source "Build-123" --target "Baseline-Build" -``` - -**Figma Integration Commands** - -Upload Figma designs for visual comparison. - -**Syntax:** -```bash -smartui upload-figma [options] -``` -**Arguments:** - -| Argument | Description | Example | -|----------|-------------|---------| -| `` | Figma design config file | `figma-config.json` | - -**Options:** - -| Option | Description | Default | -|--------|-------------|---------| -| `--markBaseline` | Mark uploaded images as baseline | `false` | -| `--buildName ` | Name of the build | Random | - -Capture Figma screenshots into CLI build. - -**Syntax:** -```bash -smartui upload-figma-web [options] -``` -**Arguments:** - -| Argument | Description | Example | -|----------|-------------|---------| -| `` | Figma config file | `figma-config.json` | - -**Options:** - -| Option | Description | Default | -|--------|-------------|---------| -| `--markBaseline` | Mark as baseline | `false` | -| `--buildName ` | Build name | Random | -| `--fetch-results [filename]` | Fetch results and save to JSON | `results.json` | - -Capture Figma screenshots into App build. - -**Syntax:** -```bash -smartui upload-figma-app [options] -``` -**Arguments:** - -| Argument | Description | Example | -|----------|-------------|---------| -| `` | Figma config file | `figma-config.json` | - -**Options:** - -| Option | Description | Default | -|--------|-------------|---------| -| `--markBaseline` | Mark as baseline | `false` | -| `--buildName ` | Build name | Random | -| `--fetch-results [filename]` | Fetch results and save to JSON | `results.json` | - -## Global Options - -These options can be used with most commands: - -| Option | Short | Description | -|--------|-------|-------------| -| `--version` | `-V` | Output version number | -| `--config ` | `-c` | Config file path | -| `--markBaseline` | | Mark this build as baseline | -| `--baselineBranch ` | | Mark build baseline for branch | -| `--baselineBuild ` | | Mark build baseline | -| `--githubURL ` | | GitHub URL including commit ID | -| `--help` | `-h` | Display help | - -## Storybook CLI Commands - -**Main Command** - -**Syntax:** -```bash -smartui storybook [options] -``` -**Arguments:** - -| Argument | Description | Example | -|----------|-------------|---------| -| `` | Storybook URL or static build directory | `http://localhost:6006` or `./storybook-static` | - -**Options:** - -| Option | Short | Description | Default | -|--------|-------|-------------|---------| -| `--config ` | `-c` | Config file path | `.smartui.json` | -| `--force-rebuild` | | Force rebuild of existing build | `false` | -| `--buildName ` | | Build name for pipeline | Random | -| `--env ` | | Runtime environment | `prod` | - -**Examples:** -```bash -# Local Storybook server -smartui storybook http://localhost:6006 --config .smartui.json - -# Static build -smartui storybook ./storybook-static --config .smartui.json - -# Public hosted URL -smartui storybook https://your-storybook-url.com --config .smartui.json - -# Force rebuild -smartui storybook ./storybook-static --force-rebuild - -# Stage environment -smartui storybook http://localhost:6006 --env stage -``` - -**Config Command** - -**Syntax:** -```bash -smartui config create [filepath] -``` -**Arguments:** - -| Argument | Description | Default | -|----------|-------------|---------| -| `[filepath]` | Optional config file path | `.smartui.json` | - -**Options:** - -| Option | Description | -|--------|-------------| -| `--help`, `-h` | Display help information | - -**Description:** -Create a SmartUI Storybook configuration file with default settings. - -## Advanced Options - -### Hidden/Advanced Options - -These options are available but may not be prominently documented: - -| Option | Commands | Description | Example | -|--------|----------|-------------|---------| -| `--scheduled ` | `capture`, `exec` | Specify schedule ID for scheduled test runs | `smartui capture urls.json --scheduled "schedule-123"` | -| `--userName ` | `capture`, `exec`, `upload` | Override TestMu AI username (overrides env) | `smartui capture urls.json --userName "user"` | -| `--accessKey ` | `capture`, `exec`, `upload` | Override TestMu AI access key (overrides env) | `smartui exec --accessKey "key" -- npm test` | -| `--env ` | `storybook` | Switch between production and staging environments | `smartui storybook http://localhost:6006 --env stage` | -| `--force` | `capture` | Forcefully apply parallel instances even if not optimal | `smartui capture urls.json --parallel 10 --force` | -| `--force-rebuild` | `storybook` | Force rebuild of an already existing Storybook build | `smartui storybook ./storybook-static --force-rebuild` | - -**Usage Examples:** -```bash -# Scheduled test runs -smartui capture urls.json --scheduled "schedule-123" -smartui exec --scheduled "schedule-123" -- npm test - -# Override credentials -smartui capture urls.json --userName "user" --accessKey "key" -smartui exec --userName "user" --accessKey "key" -- npm test - -# Environment switching (Storybook) -smartui storybook http://localhost:6006 --env stage -smartui storybook http://localhost:6006 --env prod - -# Force parallel execution -smartui capture urls.json --parallel 10 --force - -# Force rebuild (Storybook) -smartui storybook ./storybook-static --force-rebuild -``` - -## Environment Variables - -| Variable | Description | Required | -|----------|-------------|----------| -| `PROJECT_TOKEN` | SmartUI project token | Yes (for CLI projects) | -| `LT_USERNAME` | TestMu AI username | Yes (for Hooks) | -| `LT_ACCESS_KEY` | TestMu AI access key | Yes (for Hooks) | - -| Variable | Description | Default | -|----------|-------------|---------| -| `PROJECT_NAME` | Project name (creates if doesn't exist) | - | -| `BASELINE_BRANCH` | Baseline branch for CLI projects | - | -| `CURRENT_BRANCH` | Current branch for CLI projects | - | - -| Variable | Description | Format | -|----------|-------------|--------| -| `HTTP_PROXY` | HTTP proxy URL | `http://[user:pass@]host:port/` | -| `HTTPS_PROXY` | HTTPS proxy URL | `https://[user:pass@]host:port/` | -| `NO_PROXY` | Comma-separated hosts to bypass | `localhost,127.0.0.1` | - -| Variable | Description | Default | -|----------|-------------|---------| -| `SMARTUI_SERVER_ADDRESS` | Server address for SDKs | `http://localhost:49152` | - -| Variable | Description | Default | -|----------|-------------|---------| -| `LT_SDK_DEBUG` | Enable debug mode | `false` | -| `SMARTUI_DO_NOT_USE_CAPTURED_COOKIES` | Disable captured cookies | `false` | - -**Setting Environment Variables:** - -```bash -export PROJECT_TOKEN="123456#token" -export LT_USERNAME="username" -export LT_ACCESS_KEY="access_key" -export HTTP_PROXY="http://proxy:8080" -export LT_SDK_DEBUG=true -``` - -```bash -set PROJECT_TOKEN="123456#token" -set LT_USERNAME="username" -set LT_ACCESS_KEY="access_key" -set HTTP_PROXY="http://proxy:8080" -set LT_SDK_DEBUG=true -``` - -```powershell -$env:PROJECT_TOKEN="123456#token" -$env:LT_USERNAME="username" -$env:LT_ACCESS_KEY="access_key" -$env:HTTP_PROXY="http://proxy:8080" -$env:LT_SDK_DEBUG="true" -``` - -## End-to-End Workflows - -**Basic Static URL Capture** -```bash -# 1. Install CLI -npm install -g @lambdatest/smartui-cli - -# 2. Set project token -export PROJECT_TOKEN="123456#token" - -# 3. Create config -npx smartui config:create .smartui.json - -# 4. Create URLs file (urls.json) -# 5. Capture screenshots -npx smartui capture urls.json --config .smartui.json - -# 6. Fetch results -npx smartui capture urls.json --fetch-results results.json -``` - -**SDK Test Execution** -```bash -# 1. Set project token -export PROJECT_TOKEN="123456#token" - -# 2. Start server -npx smartui exec:start - -# 3. Set server address (for non-Selenium) -export SMARTUI_SERVER_ADDRESS='http://localhost:49152' - -# 4. Run tests -npm test - -# 5. Stop server -npx smartui exec:stop -``` - -**Upload Existing Screenshots** -```bash -# 1. Set project token -export PROJECT_TOKEN="123456#token" - -# 2. Upload directory -npx smartui upload ./screenshots --buildName "Manual-Test" - -# 3. With options -npx smartui upload ./screenshots \ ---ignoreResolutions \ ---removeExtensions \ ---buildName "Test-Run" \ ---fetch-results results.json -``` - -**Storybook Testing** -```bash -# 1. Install Storybook CLI -npm install -g @lambdatest/smartui-storybook - -# 2. Set project token -export PROJECT_TOKEN="123456#token" - -# 3. Start Storybook -npm run storybook - -# 4. Capture stories -smartui storybook http://localhost:6006 --config .smartui.json - -# 5. Or use static build -npm run build-storybook -smartui storybook ./storybook-static --config .smartui.json -``` - -**PDF Testing** -```bash -# 1. Set project token -export PROJECT_TOKEN="123456#token" - -# 2. Upload PDFs -npx smartui upload-pdf ./pdfs --buildName "PDF-Release-v1.0" - -# 3. Mark as baseline -npx smartui upload-pdf ./pdfs --markBaseline --buildName "PDF-Baseline" -``` - -**CI/CD Integration** - -**GitHub Actions Example:** -```yaml -- name: Run SmartUI Tests -env: -PROJECT_TOKEN: ${{ secrets.PROJECT_TOKEN }} -run: | -npm install -g @lambdatest/smartui-cli -npx smartui exec --buildName "${{ github.sha }}" -- npm test -``` -**GitLab CI Example:** -```yaml -test: -variables: -PROJECT_TOKEN: $PROJECT_TOKEN -script: -- npm install -g @lambdatest/smartui-cli -- npx smartui exec --buildName "$CI_COMMIT_SHA" -- npm test -``` - -## Best Practices - -**Build Naming** - -- Use meaningful, consistent names -- Include version or commit info -- Avoid special characters -- Use environment variables for dynamic names -```bash -# Good ---buildName "Release-v1.0.0" ---buildName "$(git rev-parse --short HEAD)" ---buildName "${{ github.sha }}" - -# Bad ---buildName "test" ---buildName "build-$(date)" -``` - -**Configuration Management** - -- Keep config files in version control -- Use environment variables for sensitive data -- Document custom configurations -- Use separate configs for different environments - -**Parallel Execution** - -- Start with lower thread counts -- Monitor system resources -- Use `--force` only when necessary -- Calculate optimal threads: `log₂(N)` - -**Server Management** - -- Always use `exec:stop` to terminate server -- Check server status with `exec:ping` -- Use custom ports to avoid conflicts -- Set `SMARTUI_SERVER_ADDRESS` before running tests - -**Error Handling** - -- Always use `--fetch-results` for CI/CD -- Check exit codes in scripts -- Implement retry logic for transient failures -- Log errors for debugging - -**Results Reporting** - -Use the [SmartUI Reporter](https://smartui-reporter.netlify.app/) to visualize and analyze test results: - -1. **Export Results**: Use `--fetch-results` flag to generate JSON -```bash -npx smartui capture urls.json --config config.json --fetch-results results.json -``` - -2. **Upload to Reporter**: Visit [SmartUI Reporter](https://smartui-reporter.netlify.app/) and upload `results.json` - -3. **View & Export**: Access tabular view with statistics, screenshot links, and export to PDF/CSV - -**Alternative**: Fetch results via [SmartUI API](https://swagger-api-support.lambdatest.com/smartui/index.html#/Fetch%20Build%20Screenshots) - -For detailed steps, see [Fetch Results Documentation](/support/docs/smartui-sdk-fetch-results/#using-smartui-reporter-tool). - -**URL Grouping for Large Test Suites** - -When working with large numbers of URLs (1000+) that can be divided into sections/groups, you have two strategies: - -**Strategy 1: Single Project with Branching** -- Use one SmartUI project with Smart Git branching -- All sections in a single project dashboard -- Each section can have its own branch -- Unified tracking and reporting - -```bash -# Enable Smart Git -export SMART_GIT=true - -# Run different sections in different branches -git checkout -b section1-bathroom -npx smartui capture sections/section1.json --config config.json --buildName "Section1-Bathroom" -``` - -**Strategy 2: Separate Projects** -- Create individual projects for each section -- Complete isolation between sections -- Independent baselines and approvals -- Minimal projects with single branch - -```bash -# Section 1 - Bathroom Project -export PROJECT_TOKEN="${PROJECT_TOKEN_BATHROOM}" -npx smartui capture bathroom/urls.json --config config.json - -# Section 2 - Kitchens Project -export PROJECT_TOKEN="${PROJECT_TOKEN_KITCHENS}" -npx smartui capture kitchens/urls.json --config config.json -``` - -**When to Use Each:** -- **Single Project**: Unified dashboard, cross-section analysis, shared configs -- **Separate Projects**: Complete isolation, independent workflows, different configs - -For detailed guidance, see [SmartUI Capture Onboarding with HyperExecute](/support/docs/hyperexecute-smart-ui-capture-onboarding/#11-url-grouping-strategies-for-large-test-suites). - -## Troubleshooting - -**Server Not Running** -```bash -# Check status -npx smartui exec:ping - -# Start server -npx smartui exec:start - -# Verify address -echo $SMARTUI_SERVER_ADDRESS -``` - -**Port Conflicts** -```bash -# Use custom port -npx smartui exec:start -P 5000 -npx smartui exec -P 5000 -- npm test -``` - -**Authentication Issues** -```bash -# Verify credentials -echo $PROJECT_TOKEN -echo $LT_USERNAME -echo $LT_ACCESS_KEY - -# Override with command options -npx smartui capture urls.json --userName "user" --accessKey "key" -``` - -**Configuration Errors** -```bash -# Validate JSON -cat .smartui.json | python -m json.tool - -# Check config path -npx smartui capture urls.json --config .smartui.json -``` - -## Additional Resources - -- [SmartUI CLI Documentation](/support/docs/smartui-cli) -- [CLI Exec Commands](/support/docs/smartui-cli-exec) -- [CLI Upload](/support/docs/smartui-cli-upload) -- [Environment Variables](/support/docs/smartui-cli-env-variables) -- [Storybook Integration](/support/docs/smart-ui-storybook) -- [Configuration Options](/support/docs/smartui-sdk-config-options) -- [Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) diff --git a/static/docs/smartui-cli-env-variables.md b/static/docs/smartui-cli-env-variables.md deleted file mode 100644 index 7fe6a6bc6..000000000 --- a/static/docs/smartui-cli-env-variables.md +++ /dev/null @@ -1,249 +0,0 @@ -# SmartUI SDK Environment Variables - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Welcome to the world of simplified visual testing with the SmartUI SDK. - -This guide is designed to provide you with comprehensive information about the various environment variables options available within the SmartUI SDK. SmartUI retrieves additional details from the environment it is running in, like the branch name, baseline branch,proxies etc. You can modify certain aspects of SmartUI behavior by configuring these environment variables within your CI environment: - -## 1. Setting the Project Name - -Set the Project Name you want to add the current build run to by exporting these enviroment variables: - -```bash -export LT_USERNAME="YOUR USERNAME" -export LT_ACCESS_KEY="YOUR ACCESS KEY" -export PROJECT_NAME="Required Project Name" -``` - -```bash -set LT_USERNAME="YOUR USERNAME" -set LT_ACCESS_KEY="YOUR ACCESS KEY" -set PROJECT_NAME="Required Project Name" -``` - -```powershell -$env:LT_USERNAME="YOUR USERNAME" -$env:LT_ACCESS_KEY="YOUR ACCESS KEY" -$env:PROJECT_NAME="Required Project Name" -``` - -> If you specify a project name that doesn't already exist, a new project will be created by the user whose authentication is added in the environment. - -## 2: Set your Project Token - -Setup your project token shown in the **SmartUI** app after creating your project. - -```bash -export PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -``` - -```bash -set PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -``` - -```powershell -$env:PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -``` - -> Setting the project token environment variable does not require any user level authentication. - -## 3. Setting the Baseline Branch - -Set the baseline branch for `CLI` projects: - -```bash -export BASELINE_BRANCH="Required branch" -``` - -```bash -set BASELINE_BRANCH="Required branch" -``` - -```powershell -$env:BASELINE_BRANCH="Required branch" -``` - -## 4. Setting the current branch - -Set the current branch for `CLI` projects: - -```bash -export CURRENT_BRANCH="Required branch" -``` - -```bash -set CURRENT_BRANCH="Required branch" -``` - -```powershell -$env:CURRENT_BRANCH="Required branch" -``` - -## 5. Setting proxies - -In case you are accessing your network using corporate proxies, set the proxies in the environment variables as follows - -#### HTTP_PROXY: - -```bash -export HTTP_PROXY="http://:@:/" - - -```bash -set HTTP_PROXY="http://:@:" -``` - - - - -```powershell -$env:HTTP_PROXY="http://:@:/" -``` - - - - -#### HTTPS_PROXY: - - - - -```bash -export HTTPS_PROXY="https://:@:" -``` - - - - -```bash -set HTTPS_PROXY="https://:@:" -``` - - - - -```powershell -$env:HTTPS_PROXY="https://:@:/" -``` - - - - -> **NOTE :** In most cases setting only HTTP_PROXY should be enough, but if you have different proxies for HTTP and HTTPS, you can set both.The format for proxy’s is `http[s]://:@:/` [username and password is optional]. - -## 6. Enabling Debug mode - -To enable SmartUI CLI Debug mode, use the following environment variable. - - - - -```bash -export LT_SDK_DEBUG=true -``` - - - - -```bash -set LT_SDK_DEBUG=true -``` - - - - -```powershell -$env:LT_SDK_DEBUG="true" -``` - - - - - -## 7. Ignoring captured cookies - -Set the following variable to true, to stop the use of cookies captured automatically. The default is set to `false`. - - - - -```shell -export SMARTUI_DO_NOT_USE_CAPTURED_COOKIES=true -``` - - - - -```shell -set SMARTUI_DO_NOT_USE_CAPTURED_COOKIES=true -``` - - - - -```powershell -$env:SMARTUI_DO_NOT_USE_CAPTURED_COOKIES="true" -``` - - - - -## 8. Setting Server Address - -For SDK integrations (non-Selenium), set the SmartUI server address: - - - - -```bash -export SMARTUI_SERVER_ADDRESS="http://localhost:8080" -``` - - - - -```bash -set SMARTUI_SERVER_ADDRESS="http://localhost:8080" -``` - - - - -```powershell -$env:SMARTUI_SERVER_ADDRESS="http://localhost:49152" -``` - -## 9. Additional Environment Variables - -The following environment variables are also available for advanced configuration: - -| Variable | Description | Default | -|----------|-------------|---------| -| `FIGMA_TOKEN` | Figma API token for Figma integration | - | -| `NO_PROXY` | Comma-separated hosts to bypass proxy | - | -| `SMARTUI_HTTP_PROXY` | SmartUI-specific HTTP proxy | - | -| `SMARTUI_HTTPS_PROXY` | SmartUI-specific HTTPS proxy | - | -| `SMARTUI_API_PROXY` | API proxy configuration | - | -| `SMARTUI_CLIENT_API_URL` | Override API URL | `https://api.lambdatest.com/visualui/1.0` | -| `SMARTUI_UPLOAD_URL` | Override upload URL | `https://api.lambdatest.com` | -| `SMARTUI_GIT_INFO_FILEPATH` | Custom git info file path | - | -| `GITHUB_ACTIONS` | GitHub Actions environment flag | - | -| `SMARTUI_API_SKIP_CERTIFICATES` | Skip SSL certificate validation | `false` | -| `USE_REMOTE_DISCOVERY` | Use remote discovery mode | `false` | -| `SMART_GIT` | Enable Smart Git feature | `false` | -| `SHOW_RENDER_ERRORS` | Show render errors | `false` | -| `SMARTUI_SSE_URL` | Server-sent events URL | `https://server-events.lambdatest.com` | -| `LT_SDK_SKIP_EXECUTION_LOGS` | Skip execution logs | `false` | -| `MAX_CONCURRENT_PROCESSING` | Max concurrent processing threads | `0` (auto) | -| `DO_NOT_USE_USER_AGENT` | Disable user agent | `false` | - -## Additional Resources - -- [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) -- [CLI Documentation](/support/docs/smartui-cli) -- [CLI Exec Commands](/support/docs/smartui-cli-exec) -- [CLI Upload](/support/docs/smartui-cli-upload) -- [Baseline Management](/support/docs/smartui-baseline-management) -- [Running Your First Project](/support/docs/smartui-running-your-first-project) -- [SmartUI API Documentation](https://www.testmuai.com/support/api-doc/) diff --git a/static/docs/smartui-cli-exec.md b/static/docs/smartui-cli-exec.md deleted file mode 100644 index c437a0bb3..000000000 --- a/static/docs/smartui-cli-exec.md +++ /dev/null @@ -1,217 +0,0 @@ -# SmartUI CLI Exec Command - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -SmartUI CLI exec command offers you various options to manage snapshot server and execute your visual testing scripts. This guide will walk you through the available commands and their usage. - -## Prerequisites - -- Basic understanding of Command Line Interface -- SmartUI CLI version 4.1.43 or higher installed for the start, stop and ping commands -- A properly configured SmartUI CLI project - -## Available Commands - -SmartUI CLI offers the following exec commands: - -- `npx smartui exec` - Execute a command with SmartUI server running -- `npx smartui exec:start` - Start the SmartUI snapshot server -- `npx smartui exec:stop` - Stop the SmartUI snapshot server -- `npx smartui exec:ping` - Check if the SmartUI server is running (uses custom HTTP client) -- `npx smartui exec:pingTest` - Check if the SmartUI server is running (uses default HTTP client) - -## Using the Exec Command - -The `npx smartui exec` command allows you to run your tests with various configuration options. Here's the basic syntax: - -```bash -npx smartui exec [options] -- -``` - -### Available Options - -| Option | Description | -|--------|-------------| -| `-P, --port ` | Specify a custom port number for the server (default: 49152) | -| `--fetch-results [filename]` | Fetch test results and save to a JSON file (default: results.json) | -| `--buildName ` | Specify a custom build name for the test run | -| `--scheduled ` | Specify the schedule ID for scheduled test runs | -| `--show-render-errors` | Show render errors from SmartUI build | -| `--userName ` | Override TestMu AI username (overrides environment variable) | -| `--accessKey ` | Override TestMu AI access key (overrides environment variable) | -| `--config ` | Specify a configuration file to use | -| `-h, --help` | Display help information | - -### Examples - -1. Running with a custom port: -```bash -npx smartui exec -P 5000 -- npm test -``` - -2. Fetching results with custom filename: -```bash -npx smartui exec --fetch-results custom-results.json -- npm test -``` - -3. Specifying a build name: -```bash -npx smartui exec --buildName "Release-1.0" -- npm test -``` - -4. Using a configuration file: -```bash -npx smartui exec --config smartui-config.json -- npm test -``` - -5. Combining multiple options: -```bash -npx smartui exec -P 5000 --buildName "Release-1.0" --config smartui-config.json --fetch-results -- npm test -``` - -6. Running scheduled tests: -```bash -npx smartui exec --scheduled "schedule-123" -- npm test -``` - -7. Showing render errors: -```bash -npx smartui exec --show-render-errors -- npm test -``` - -8. Overriding credentials: -```bash -npx smartui exec --userName "user" --accessKey "key" -- npm test -``` - -## Starting the Server - -To start the SmartUI snapshot server: - -```bash -npx smartui exec:start -``` - -This will start a local server that handles snapshot requests. By default, the server runs on port 49152. - -## Stopping the Server - -To properly stop the SmartUI snapshot server: - -```bash -npx smartui exec:stop -``` - -> **Important:** It's recommended to use the `exec:stop` command rather than using Ctrl+C to terminate the server. Using Ctrl+C will cause the build to stop after 12 minutes of running. - -## Checking Server Status - -To verify if the SmartUI server is running, you can use either of these commands: - -**Using exec:ping (custom HTTP client):** -```bash -npx smartui exec:ping -``` - -**Using exec:pingTest (default HTTP client):** -```bash -npx smartui exec:pingTest -``` - -Both commands check if the server is running at the address specified in `SMARTUI_SERVER_ADDRESS` environment variable (default: `http://localhost:49152`). - -## Running Tests with the Server - -### Step 1: Configure Server Address - -For most test frameworks (except Selenium Java and JavaScript), you'll need to set the server address: - -```bash -export SMARTUI_SERVER_ADDRESS='http://localhost:49152' -``` - -```bash -set SMARTUI_SERVER_ADDRESS='http://localhost:49152' -``` - -```powershell -$env:SMARTUI_SERVER_ADDRESS="http://localhost:49152" -``` - -### Step 2: Execute Your Tests - -You can run your tests using your preferred test runner: - -```bash -# Using npm -npm test - -# Using Maven -mvn test - -# Using other test runners -your-test-command -``` - -### IDE Configuration - -When running tests from IDEs like IntelliJ: -1. Ensure the `SMARTUI_SERVER_ADDRESS` environment variable is set in your IDE's run configuration -2. Configure the run configuration to use the appropriate test command - -## Best Practices - -**Server Termination** - -Always use `exec:stop` to properly terminate the server. - -**Timeouts** - -Set appropriate timeouts for your tests. - -**Snapshot Naming** - -Use meaningful names for your snapshots. - -**Environment Variables** - -Configure environment variables before starting your IDE. - -**Configuration Files** - -Consider using configuration files for complex setups. - -## Troubleshooting - -**Server Status** - -Verify the server is running using `exec:ping`. - -**Server Address** - -Check if the server address is correctly configured. - -**Port Conflicts** - -Ensure no other process is using the default port. - -**Server Logs** - -Review the server logs for error messages. - -### Getting Help - -If you encounter issues not covered here: -- Review the [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) for detailed solutions -- Check [CLI Documentation](/support/docs/smartui-cli) for general CLI usage -- Visit [TestMu AI Support](https://www.testmuai.com/support) for additional resources -- Contact support at support@testmuai.com or use [24/7 Chat Support](https://www.testmuai.com/support) - -## Additional Resources - -- [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) -- [CLI Documentation](/support/docs/smartui-cli) -- [CLI Upload](/support/docs/smartui-cli-upload) -- [Environment Variables](/support/docs/smartui-cli-env-variables) -- [Baseline Management](/support/docs/smartui-baseline-management) -- [Running Your First Project](/support/docs/smartui-running-your-first-project) diff --git a/static/docs/smartui-cli-figma-app.md b/static/docs/smartui-cli-figma-app.md deleted file mode 100644 index 93fb94cce..000000000 --- a/static/docs/smartui-cli-figma-app.md +++ /dev/null @@ -1,264 +0,0 @@ -# Getting Started with TestMu AI's SmartUI Figma-App CLI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -SmartUI Figma-App CLI lets you compare **mobile app screenshots captured on real devices** with your **Figma design frames** to detect visual mismatches and ensure accurate implementation of mobile UI. - -## Prerequisites - -- Node.js and npm installed -- TestMu AI SmartUI account with App Automation plan -- Real device screenshots captured via Appium, SDK, or SmartUI platform -- Figma Personal Access Token ([how to get one](https://help.figma.com/hc/en-us/articles/8085703771159-Manage-personal-access-tokens)) - -## Understanding Figma Tokens - -| Token | Where It’s Used | Description | -|----------------------|------------------|-----------------------------------------------------------------------------| -| `FIGMA_TOKEN` | Env Variable | Your Figma **Personal Access Token** to authenticate with the Figma API | -| `figma_file_token` | `designs.json` | Figma **file ID**, extracted from the Figma file URL | -| `figma_ids` | `designs.json` | List of **frame or node IDs** you want to compare visually | - -> Example Figma URL: -> `https://www.figma.com/file/abc12345/file-name?node-id=2417-58969` -> - `figma_file_token`: `abc12345` -> - `figma_ids`: `2417-58969` - -## Step-by-Step Guide - -### 1. Create a SmartUI Project - -1. Visit [smartui.lambdatest.com](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com) -2. Click **New Project** -3. Select **Real Devices** as the platform -4. Enter: - - Project Name - - Approvers (optional) - - Tags (optional) -5. Click **Submit** - -### 2. Install SmartUI CLI - -```bash -npm install @lambdatest/smartui-cli -```` - -### 3. Generate and Edit Configuration - -Run the following to create your initial design file: - -```bash -npx smartui config:create-figma-app designs.json -``` - -#### Sample `designs.json` - -```json title="designs.json" -{ -"mobile": [ -{ -"name": "Pixel 8", -"platform": ["Android 14"], -"orientation": "portrait" -} -], -"figma": { -"depth": 2, -"configs": [ -{ -"figma_file_token": "abc12345", -"figma_ids": ["2417-58969"], -"screenshot_names": ["homepage"] -} -] -} -} -``` - -### 4. Set Environment Variables - -```bash -export PROJECT_TOKEN="your_smartui_project_token" -export FIGMA_TOKEN="your_figma_personal_token" -``` - -### 5. Run the Comparison - -```bash -npx smartui upload-figma-app designs.json -``` - -#### Optional Flags - -| Flag | Description | -| ---------------- | ------------------------------------------------- | -| `--markBaseline` | Mark this build as a new baseline for future runs | -| `--buildName` | Assign a custom name to this comparison build | - -#### Example - -```bash -npx smartui upload-figma-app designs.json --buildName "v1.0.0" --markBaseline -``` - -### View SmartUI Results - -You can see the SmartUI dashboard to view the results. This will help you identify the Mismatches from the existing `Baseline` build and do the required visual testing. - -## Best Practices - -**Build Names** - -```bash -npx smartui upload-figma-app designs.json --buildName "v1.0.0" -``` - -**Screenshot Names** - -- Good: `homepage-screen`, `login-form`, `dashboard-tab` - - Avoid: `test1`, `screenshot`, `design-1` - - Ensure `screenshot_names` in your config match the order of `figma_ids` - -**Device Names** - -**Screenshot Naming for SDK Comparisons** - -**Critical**: When comparing Figma designs with app screenshots captured via SDKs, add `.png` extension to your SDK screenshot names. - -Figma-uploaded screenshots automatically have `.png` appended (e.g., `homepage.png`), so your SDK screenshots must match: - -**In your Appium/SDK code:** -```javascript -// ❌ Wrong - will not match Figma screenshot -driver.execute("smartui.takeScreenshot", {name: "homepage"}); - -// ✅ Correct - matches Figma screenshot name -driver.execute("smartui.takeScreenshot", {name: "homepage.png"}); -``` - -**Example for different frameworks:** - -```javascript -// JavaScript -await driver.execute("smartui.takeScreenshot", {name: "homepage.png"}); -``` - -```java -// Java -driver.execute("smartui.takeScreenshot", Map.of("name", "homepage.png")); -``` - -This ensures that Figma screenshots (e.g., `homepage.png`) match app screenshots (e.g., `homepage.png`) in the same build. - -**Device Names** - -**Screenshot Naming for SDK Comparisons** - -**Critical**: When comparing Figma designs with app screenshots captured via SDKs, add `.png` extension to your SDK screenshot names. - -Figma-uploaded screenshots automatically have `.png` appended (e.g., `homepage.png`), so your SDK screenshots must match: - -**In your Appium/SDK code:** -```javascript -// ❌ Wrong - will not match Figma screenshot -driver.execute("smartui.takeScreenshot", {name: "homepage"}); - -// ✅ Correct - matches Figma screenshot name -driver.execute("smartui.takeScreenshot", {name: "homepage.png"}); -``` - -**Example for different frameworks:** - -```javascript -// JavaScript -await driver.execute("smartui.takeScreenshot", {name: "homepage.png"}); -``` - -```java -// Java -driver.execute("smartui.takeScreenshot", Map.of("name", "homepage.png")); -``` - -This ensures that Figma screenshots (e.g., `homepage.png`) match app screenshots (e.g., `homepage.png`) in the same build. - -## Troubleshooting - -Verify Figma Token - -```bash -echo $FIGMA_TOKEN -``` - Ensure the token is valid and has not expired. Generate a new token from [Figma Settings](https://www.figma.com/settings). - -Check File Token - -- Verify the `figma_file_token` in your `designs.json` matches the file ID from the Figma URL - - Ensure you have access to the Figma file - -Validate Node IDs - -- Confirm `figma_ids` in your configuration are valid - - Check that the nodes exist in the Figma file -**Symptoms**: -- Figma screenshots don't match app screenshots -- Comparison shows mismatches even when designs are identical -**Solutions**: - -Check Screenshot Names - -- Ensure SDK screenshots include `.png` extension (e.g., `homepage.png`) - - Verify screenshot names match exactly between Figma config and SDK code - - Ensure `screenshot_names` array matches the order of `figma_ids` - -Verify Device Sizes - -- Ensure device dimensions match Figma frame sizes - - Check that device names in config match actual device capabilities - - Verify orientation (portrait/landscape) matches between Figma and device - -Check Build Names - -- Ensure both Figma and SDK uploads use the same `--buildName` - - Verify builds are in the same project - -Project Type - -- Ensure project is created as **Real Devices** type (not CLI) - - Verify project exists in SmartUI dashboard -**Symptoms**: -- "Invalid project token" error -- Uploads fail with authentication errors -**Solutions**: - -Verify Project Token - -```bash -echo $PROJECT_TOKEN -``` - Ensure the token is set correctly and matches your SmartUI project. - -Check Project Type - -- Ensure project is created as **Real Devices** type - - Verify project exists in SmartUI dashboard -If you encounter issues not covered here: -- Review the [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) for detailed solutions -- Check [Figma CLI Documentation](/support/docs/smartui-cli-figma) for basic Figma workflows -- Check [Figma-Web CLI Documentation](/support/docs/smartui-cli-figma-web) for web comparison workflows -- Visit [TestMu AI Support](https://www.testmuai.com/support) for additional resources -- Contact support at support@testmuai.com or use [24/7 Chat Support](https://www.testmuai.com/support) - -## Additional Resources - -- [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) -- [Figma CLI Documentation](/support/docs/smartui-cli-figma) -- [Figma-Web CLI Documentation](/support/docs/smartui-cli-figma-web) -- [Appium Hooks Documentation](/support/docs/smartui-appium-hooks) -- [Baseline Management](/support/docs/smartui-baseline-management) -- [Running Your First Project](/support/docs/smartui-running-your-first-project) -- [SmartUI API Documentation](https://www.testmuai.com/support/api-doc/) -- [How to generate a Figma token](https://help.figma.com/hc/en-us/articles/8085703771159-Manage-personal-access-tokens) -- [SmartUI CLI Docs](/support/docs/smartui-cli/) -- [Appium + SmartUI Node Sample](https://github.com/LambdaTest/smartui-appium-nodejs) - -**Run seamless visual comparisons between real device screenshots and Figma designs with SmartUI CLI. Start validating your mobile UI today!** diff --git a/static/docs/smartui-cli-figma-web.md b/static/docs/smartui-cli-figma-web.md deleted file mode 100644 index 5c3c11402..000000000 --- a/static/docs/smartui-cli-figma-web.md +++ /dev/null @@ -1,418 +0,0 @@ -# Getting started with TestMu AI's SmartUI Figma-Web CLI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Welcome to the TestMu AI SmartUI Figma Web CLI documentation! - -With SmartUI Figma-Web CLI, you can seamlessly perform visual regression testing of your Figma designs on SmartUI using your command line, identifying discrepancies between your designs and live websites. This guide will walk you through the process of running successful Figma-Web Visual tests using SmartUI CLI. - -## Prerequisites for running SmartUI Figma CLI - -- Basic understanding of Command Line Interface is required. -- Basic understanding of Figma file structuring is required. -- Login to [TestMu AI SmartUI](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) with your credentials. -- Ensure you are using `@lambdatest/smartui-cli` version 4.1.43 or higher - -If you face any problems executing tests with SmartUI-CLI `versions >= v4.x.x`, upgrade your Node.js version to `v20.3` or above. - -The following steps will guide you in running your visual tests for Figma files on TestMu AI platform using SmartUI CLI. - -## Understanding Figma Tokens - -| Token | Where It’s Used | Description | -|----------------------|------------------|-----------------------------------------------------------------------------| -| `FIGMA_TOKEN` | Env Variable | Your Figma **Personal Access Token** to authenticate with the Figma API | -| `figma_file_token` | `designs.json` | Figma **file ID**, extracted from the Figma file URL | -| `figma_ids` | `designs.json` | List of **frame or node IDs** you want to compare visually | - -> Example Figma URL: -> `https://www.figma.com/file/abc12345/file-name?node-id=2417-58969` -> - `figma_file_token`: `abc12345` -> - `figma_ids`: `2417-58969` - -## Create a SmartUI CLI Project - -The first step is to create a project with the application in which we will combine all your builds run on the project. To create a SmartUI Project, follow these steps: - -1. Go to [Projects page](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) -2. Click on the `new project` button -3. Select the platform as CLI for executing your `CLI` tests. -4. Add name of the project, approvers for the changes found, tags for any filter or easy navigation. -5. Click on the **Submit**. - -## Steps to run your first test - -### **Step 1**: Install the Dependencies - -Install required NPM modules for `TestMu AI SmartUI CLI` globally or in your project: - -**Global Installation (Recommended):** -```bash -npm install -g @lambdatest/smartui-cli -``` - -**Local Installation:** -```bash -npm install @lambdatest/smartui-cli -``` - -### **Step 2:** Create the design configuration file - -``` -smartui config:create-figma-web designs.json -``` -Once, the `designs` file will be created, you will be seeing the sample pre-filled configuration in the `designs.json` file: - -```json title="/smartui-cli-figma-project/designs.json" -{ -"web": { -"browsers": [ -"chrome", -"firefox", -"safari", -"edge" -] -}, -"figma": { -"depth": 2, //Figma Tree depth - (Optional), change the value according to the your file structure -"configs": [ -{ -"figma_file_token": "", -"figma_ids": [ -"id-1", -"id-2" -], -"screenshot_names": [ -"homepage", -"about" -] -}, -{ -"figma_file_token": "", -"figma_ids": [ -"id-3", -"id-4" -], -"screenshot_names": [ -"xyz", -"abc" -] -} -] -} -} -``` ->Note: Unlike vanilla Figma CLI, designs.json includes browser parameters and auto-fetches viewports of Figma frames for efficient comparison. - -### **Step 3:** Configure your Project Token and Figma Token - -1. Setup your project token shown in the **SmartUI** app after creating your project. - -```bash -export PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -``` - -```bash -set PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -``` - -```powershell -$env:PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -``` - -2. Setup your [personal access token for Figma](https://help.figma.com/hc/en-us/articles/8085703771159-Manage-personal-access-tokens) to authenticate Figma with SmartUI. - -```bash -export FIGMA_TOKEN="123456#1234abcd-****-****-****-************" -``` - -```bash -set FIGMA_TOKEN="123456#1234abcd-****-****-****-************" -``` - -```powershell -$env:FIGMA_TOKEN="123456#1234abcd-****-****-****-************" -``` - -### **Step 4:** Configure your TestMu AI Credentials - -```bash -export LT_USERNAME="YOUR_USERNAME" -``` - -```bash -set LT_USERNAME="YOUR_USERNAME" -``` - -```powershell -$env:LT_USERNAME="YOUR_USERNAME" -``` - -#### SmartUI CLI Design Config Options - -Please read the following table for more information about the configuration file: - -| Config Key | Description | Usage | -| -------------- | ---------------------------------------------------------------------------------------------------------------------------------- | --------- | -| figma_file_token | File token for your required Figma file. You can use multiple figma files in the same configuration.| Mandatory | -| figma_ids | Comma separated list of nodes that you care about in the document. If specified, only a subset of the document will be returned corresponding to the nodes listed, their children, and everything between the root node and the listed nodes. | Optional | -| depth (int) | Positive integer (>1) representing how deep into the document tree to traverse. For example, setting it to 2 returns Pages and all top level objects on each page. Not setting this parameter returns all nodes | Optional | - -### **Step 5:** Uploading the Figma files on SmartUI Cloud using CLI - -You can now execute tests for `Visual Testing for Figma objects` using the following options:. - -```powershell -npx smartui upload-figma-web designs.json -``` - -### Working example with actual web comparisons - -1. Please clone the following sample GitHub repo (`https://github.com/LambdaTest/smartui-figma-web-cli-sample`). - -```bash -git clone https://github.com/LambdaTest/smartui-figma-web-cli-sample -``` -2. Install the node modules using the command: - -```bash -npm i -``` -3. Configure your project token and Figma token - -- Setup your project token shown in the **SmartUI** app after creating your project. - -```bash -export PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -``` - -```bash -set PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -``` - -```powershell -$env:PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -``` - -- Setup your [personal access token for Figma](https://help.figma.com/hc/en-us/articles/8085703771159-Manage-personal-access-tokens) to authenticate Figma with SmartUI. - -```bash -export FIGMA_TOKEN="123456#1234abcd-****-****-****-************" -``` - -```bash -set FIGMA_TOKEN="123456#1234abcd-****-****-****-************" -``` - -```powershell -$env:FIGMA_TOKEN="123456#1234abcd-****-****-****-************" -``` - -4. Create your figma baseline -```bash -npx smartui upload-figma-web designs.json --buildName=FigmaBaseline2 -``` -5. Execute your functional test script (Selenium Nodejs in this sample) -``` -npx smartui --config web-config.json exec --buildName=web-build -- node figma-web-local.js -``` - ->**Points to Consider** -> - The browsers listed in the Figma configuration are used solely to enable comparisons between your designs and their corresponding live webpages. -> - Websites may exhibit browser-specific behaviors. To account for these variations, it is recommended to create distinct designs tailored for each browser, if necessary. - -## Best Practices - -**Build Names** - -```bash -npx smartui upload-figma-web designs.json --buildName "v1.0.0" -``` - -**Screenshot Names** - -- Good: `homepage-hero`, `login-form`, `dashboard-sidebar` - - Avoid: `test1`, `screenshot`, `design-1` - - Ensure `screenshot_names` in your config match the order of `figma_ids` - -**Branch Names** - -**Screenshot Naming for SDK Comparisons** - -**Critical**: When comparing Figma designs with live implementations captured via SDKs, add `.png` extension to your SDK screenshot names. - -Figma-uploaded screenshots automatically have `.png` appended (e.g., `homepage.png`), so your SDK screenshots must match: - -**In your SDK code:** -```javascript -// ❌ Wrong - will not match Figma screenshot -smartuiSnapshot(driver, "homepage"); - -// ✅ Correct - matches Figma screenshot name -smartuiSnapshot(driver, "homepage.png"); -``` - -**Example for different frameworks:** - -```java -// Java -SmartUISnapshot.smartuiSnapshot(driver, "homepage.png"); -``` - -```javascript -// JavaScript -await smartuiSnapshot(page, "homepage.png"); -``` - -```javascript -// JavaScript -cy.smartuiSnapshot("homepage.png"); -``` - -**Configuration Example:** -```json -{ -"figma": { -"configs": [ -{ -"figma_file_token": "abc12345", -"figma_ids": ["id-1", "id-2"], -"screenshot_names": ["homepage.png", "about.png"] // Include .png extension -} -] -} -} -``` - -This ensures that Figma screenshots (e.g., `homepage.png`) match SDK screenshots (e.g., `homepage.png`) in the dashboard. - -### Branch Names - -### Screenshot Naming for SDK Comparisons - -**Critical**: When comparing Figma designs with live implementations captured via SDKs, add `.png` extension to your SDK screenshot names. - -Figma-uploaded screenshots automatically have `.png` appended (e.g., `homepage.png`), so your SDK screenshots must match: - -**In your SDK code:** -```javascript -// ❌ Wrong - will not match Figma screenshot -smartuiSnapshot(driver, "homepage"); - -// ✅ Correct - matches Figma screenshot name -smartuiSnapshot(driver, "homepage.png"); -``` - -**Example for different frameworks:** - -```java -// Java -SmartUISnapshot.smartuiSnapshot(driver, "homepage.png"); -``` - -```javascript -// JavaScript -await smartuiSnapshot(page, "homepage.png"); -``` - -```javascript -// JavaScript -cy.smartuiSnapshot("homepage.png"); -``` - -**Configuration Example:** -```json -{ -"figma": { -"configs": [ -{ -"figma_file_token": "abc12345", -"figma_ids": ["id-1", "id-2"], -"screenshot_names": ["homepage.png", "about.png"] // Include .png extension -} -] -} -} -``` - -This ensures that Figma screenshots (e.g., `homepage.png`) match SDK screenshots (e.g., `homepage.png`) in the same build. - -## Troubleshooting - -Verify Figma Token - -```bash -echo $FIGMA_TOKEN -``` - Ensure the token is valid and has not expired. Generate a new token from [Figma Settings](https://www.figma.com/settings). - -Check File Token - -- Verify the `figma_file_token` in your `designs.json` matches the file ID from the Figma URL - - Ensure you have access to the Figma file - -Validate Node IDs - -- Confirm `figma_ids` in your configuration are valid - - Check that the nodes exist in the Figma file -**Symptoms**: -- Figma screenshots don't match web screenshots -- Comparison shows mismatches even when designs are identical -**Solutions**: - -Check Screenshot Names - -- Ensure SDK screenshots include `.png` extension (e.g., `homepage.png`) - - Verify screenshot names match exactly between Figma config and SDK code - - Ensure `screenshot_names` array matches the order of `figma_ids` - -Verify Frame Sizes - -- Ensure frame dimensions are consistent across uploads - - Check that viewport sizes match between Figma frames and browser captures - - Use the same browser viewport sizes in both Figma config and SDK config - -Check Build Names - -- Ensure both Figma and SDK uploads use the same `--buildName` - - Verify builds are in the same project - -Browser-Specific Issues - -- Different browsers may render differently - - Consider creating browser-specific Figma designs if needed - - Ensure browser list in Figma config matches browsers used in SDK tests -**Symptoms**: -- "Invalid project token" error -- Uploads fail with authentication errors -**Solutions**: - -Verify Project Token - -```bash -echo $PROJECT_TOKEN -``` - Ensure the token is set correctly and matches your SmartUI project. - -Check Project Type - -- Ensure project is created as **CLI** type - - Verify project exists in SmartUI dashboard -If you encounter issues not covered here: -- Review the [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) for detailed solutions -- Check [Figma CLI Documentation](/support/docs/smartui-cli-figma) for basic Figma workflows -- Visit [TestMu AI Support](https://www.testmuai.com/support) for additional resources -- Contact support at support@testmuai.com or use [24/7 Chat Support](https://www.testmuai.com/support) - -## Additional Resources - -- [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) -- [Figma CLI Documentation](/support/docs/smartui-cli-figma) -- [Figma-App CLI Documentation](/support/docs/smartui-cli-figma-app) -- [Baseline Management](/support/docs/smartui-baseline-management) -- [Running Your First Project](/support/docs/smartui-running-your-first-project) -- [SmartUI API Documentation](https://www.testmuai.com/support/api-doc/) - -### View SmartUI Results - -You can see the SmartUI dashboard to view the results. This will help you identify the Mismatches from the existing `Baseline` build and do the required visual testing. diff --git a/static/docs/smartui-cli-figma.md b/static/docs/smartui-cli-figma.md deleted file mode 100644 index a461e97ca..000000000 --- a/static/docs/smartui-cli-figma.md +++ /dev/null @@ -1,355 +0,0 @@ -# Getting started with TestMu AI's SmartUI Figma CLI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Welcome to the TestMu AI SmartUI Figma CLI documentation! - -With SmartUI Figma CLI, you can seamlessly perform visual regression testing of your Figma designs on SmartUI using your command line, identifying Visual UI Regression bugs effortlessly. This guide will walk you through the process of running successful Figma Visual tests using SmartUI CLI. - -## Prerequisites for running SmartUI Figma CLI - -- Basic understanding of Command Line Interface is required. -- Basic understanding of Figma file structuring is required. -- Login to [TestMu AI SmartUI](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) with your credentials. - -The following steps will guide you in running your visual tests for Figma files on TestMu AI platform using SmartUI CLI. - -## Understanding Figma Tokens - -| Token | Where It’s Used | Description | -|----------------------|------------------|-----------------------------------------------------------------------------| -| `FIGMA_TOKEN` | Env Variable | Your Figma **Personal Access Token** to authenticate with the Figma API | -| `figma_file_token` | `designs.json` | Figma **file ID**, extracted from the Figma file URL | -| `figma_ids` | `designs.json` | List of **frame or node IDs** you want to compare visually | - -> Example Figma URL: -> `https://www.figma.com/file/abc12345/file-name?node-id=2417-58969` -> - `figma_file_token`: `abc12345` -> - `figma_ids`: `2417-58969` - -## Create a SmartUI CLI Project - -The first step is to create a project with the application in which we will combine all your builds run on the project. To create a SmartUI Project, follow these steps: - -1. Go to [Projects page](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) -2. Click on the `new project` button -3. Select the platform as CLI for executing your `CLI` tests. -4. Add name of the project, approvers for the changes found, tags for any filter or easy navigation. -5. Click on the **Submit**. - -## Steps to run your first test - -### **Step 1**: Install the Dependencies - -Install required NPM modules for `TestMu AI SmartUI CLI` globally or in your project: - -**Global Installation (Recommended):** -```bash -npm install -g @lambdatest/smartui-cli -``` - -**Local Installation:** -```bash -npm install @lambdatest/smartui-cli -``` - -If you face any problems executing tests with SmartUI-CLI `versions >= v4.x.x`, upgrade your Node.js version to `v20.3` or above. - -### **Step 2:** Create the design info file - -``` -smartui config:create-figma designs.json -``` -Once, the `designs` file will be created, you will be seeing the sample pre-filled configuration in the `designs.json` file: - -```json title="/smartui-cli-figma-project/designs.json" -{ -"depth":2, //Figma Tree depth - (Optional), change the value according to the your file structure -"figma_config": [ -{ -"figma_file_token": "", -"figma_ids":[ -"Id-1", "Id-2" -] -}, -{ -"figma_file_token": "", -"figma_ids":[ -"Id-3", "Id-4" -] -} -] -} -``` - -### **Step 3:** Configure your Project Token and Figma Token - -1. Setup your project token shown in the **SmartUI** app after creating your project. - -```bash -export PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -``` - -```bash -set PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -``` - -```powershell -$env:PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -``` - -2. Setup your [personal access token for Figma](https://help.figma.com/hc/en-us/articles/8085703771159-Manage-personal-access-tokens) to authenticate Figma with SmartUI. - -```bash -export FIGMA_TOKEN="123456#1234abcd-****-****-****-************" -``` - -```bash -set FIGMA_TOKEN="123456#1234abcd-****-****-****-************" -``` - -```powershell -$env:FIGMA_TOKEN="123456#1234abcd-****-****-****-************" -``` - -### **Step 4:** Configure your TestMu AI Credentials - -```bash -export LT_USERNAME="YOUR_USERNAME" -``` - -```bash -set LT_USERNAME="YOUR_USERNAME" -``` - -```powershell -$env:LT_USERNAME="YOUR_USERNAME" -``` - -#### SmartUI CLI Design Config Options - -Please read the following table for more information about the configuration file: - -| Config Key | Description | Usage | -| -------------- | ---------------------------------------------------------------------------------------------------------------------------------- | --------- | -| figma_file_token | File token for your required Figma file. You can use multiple figma files in the same configuration.| Mandatory | -| figma_ids | Comma separated list of nodes that you care about in the document. If specified, only a subset of the document will be returned corresponding to the nodes listed, their children, and everything between the root node and the listed nodes. | Optional | -| depth (int) | Positive integer (>1) representing how deep into the document tree to traverse. For example, setting it to 2 returns Pages and all top level objects on each page. Not setting this parameter returns all nodes | Optional | - -### **Step 5:** Uploading the Figma files on SmartUI Cloud using CLI - -You can now execute tests for `Visual Testing for Figma objects` using the following options:. - -```bash -npx smartui upload-figma designs.json -``` - -### Advanced options for baseline management - -1. `markBaseline` - You can mark a specific build as a baseline through the runner command - -```bash -npx smartui upload-figma designs.json --markBaseline -``` -2. `buildName` - You can add your custom build name to a build by running the following command - -```bash -npx smartui upload-figma designs.json --buildName "" -``` - - You can use these options in a nested way as well, as shown below - -```bash -npx smartui upload-figma designs.json --buildName "" --markBaseline -``` - -If `buildName` is not specified, a random build name is generated for every run. - -You can add more screenshots in a build by specifying the particular build name in the runner command - -## Best Practices - -**Build Names** - -```bash -npx smartui upload-figma designs.json --buildName "v1.0.0" -``` - -**Screenshot Names** - -- Good: `homepage-hero`, `login-form`, `dashboard-sidebar` - - Avoid: `test1`, `screenshot`, `design-1` - -**Branch Names** - -**Screenshot Naming for SDK Comparisons** - -**Important**: When comparing Figma designs with live implementations captured via SDKs, add `.png` extension to your SDK screenshot names. - -Figma-uploaded screenshots automatically have `.png` appended (e.g., `homepage.png`), so your SDK screenshots must match: - -**In your SDK code:** -```javascript -// ❌ Wrong - will not match Figma screenshot -smartuiSnapshot(driver, "homepage"); - -// ✅ Correct - matches Figma screenshot name -smartuiSnapshot(driver, "homepage.png"); -``` - -**Example for different frameworks:** - -```java -// Java -SmartUISnapshot.smartuiSnapshot(driver, "homepage.png"); -``` - -```javascript -// JavaScript -await smartuiSnapshot(page, "homepage.png"); -``` - -```javascript -// JavaScript -cy.smartuiSnapshot("homepage.png"); -``` - -This ensures that Figma screenshots (e.g., `homepage.png`) match SDK screenshots (e.g., `homepage.png`) in the same build. - -### Setup with Continuous Integration (CI) - -If you are using the Continuous Integration (CI) pipeline for your application and want to integrate `SmartUI Figma CLI` execution then the following are the steps needs to be added to your `.yaml` file: - -```yaml -steps: -- name: Running SmartUI Figma CLI Tests -- run: | -npm install @lambdatest/smartui-cli -npx playwright install-deps -npx smartui upload-figma designs.json -``` - -### View SmartUI Results - -You can see the SmartUI dashboard to view the results. This will help you identify the Mismatches from the existing `Baseline` build and do the required visual testing. - -**Branch Names** - -**Screenshot Naming for SDK Comparisons** - -**Important**: When comparing Figma designs with live implementations captured via SDKs, add `.png` extension to your SDK screenshot names. - -Figma-uploaded screenshots automatically have `.png` appended (e.g., `homepage.png`), so your SDK screenshots must match: - -**In your SDK code:** -```javascript -// ❌ Wrong - will not match Figma screenshot -smartuiSnapshot(driver, "homepage"); - -// ✅ Correct - matches Figma screenshot name -smartuiSnapshot(driver, "homepage.png"); -``` - -**Example for different frameworks:** - -```java -// Java -SmartUISnapshot.smartuiSnapshot(driver, "homepage.png"); -``` - -```javascript -// JavaScript -await smartuiSnapshot(page, "homepage.png"); -``` - -```javascript -// JavaScript -cy.smartuiSnapshot("homepage.png"); -``` - -This ensures that Figma screenshots (e.g., `homepage.png`) match SDK screenshots (e.g., `homepage.png`) in the same build. - -### Setup with Continuous Integration (CI) - -If you are using the Continuous Integration (CI) pipeline for your application and want to integrate `SmartUI Figma CLI` execution then the following are the steps needs to be added to your `.yaml` file: - -```yaml -steps: -- name: Running SmartUI Figma CLI Tests -- run: | -npm install @lambdatest/smartui-cli -npx playwright install-deps -npx smartui upload-figma designs.json -``` - -### View SmartUI Results - -You can see the SmartUI dashboard to view the results. This will help you identify the Mismatches from the existing `Baseline` build and do the required visual testing. - -## Troubleshooting - -Verify Figma Token - -```bash -echo $FIGMA_TOKEN -``` - Ensure the token is valid and has not expired. Generate a new token from [Figma Settings](https://www.figma.com/settings). - -Check File Token - -- Verify the `figma_file_token` in your `designs.json` matches the file ID from the Figma URL - - Ensure you have access to the Figma file - -Validate Node IDs - -- Confirm `figma_ids` in your configuration are valid - - Check that the nodes exist in the Figma file -**Symptoms**: -- Figma screenshots don't match SDK screenshots -- Comparison shows mismatches even when designs are identical -**Solutions**: - -Check Screenshot Names - -- Ensure SDK screenshots include `.png` extension (e.g., `homepage.png`) - - Verify screenshot names match exactly between Figma and SDK - -Verify Frame Sizes - -- Ensure frame dimensions are consistent across uploads - - Check that viewport sizes match between Figma frames and SDK captures - -Check Build Names - -- Ensure both Figma and SDK uploads use the same `--buildName` - - Verify builds are in the same project -**Symptoms**: -- "Invalid project token" error -- Uploads fail with authentication errors -**Solutions**: - -Verify Project Token - -```bash -echo $PROJECT_TOKEN -``` - Ensure the token is set correctly and matches your SmartUI project. - -Check Project Type - -- Ensure project is created as **CLI** type - - Verify project exists in SmartUI dashboard -If you encounter issues not covered here: -- Review the [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) for detailed solutions -- Check [Figma-Web CLI Documentation](/support/docs/smartui-cli-figma-web) for web comparison workflows -- Visit [TestMu AI Support](https://www.testmuai.com/support) for additional resources -- Contact support at support@testmuai.com or use [24/7 Chat Support](https://www.testmuai.com/support) - -## Additional Resources - -- [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) -- [Figma-Web CLI Documentation](/support/docs/smartui-cli-figma-web) -- [Figma-App CLI Documentation](/support/docs/smartui-cli-figma-app) -- [Baseline Management](/support/docs/smartui-baseline-management) -- [Running Your First Project](/support/docs/smartui-running-your-first-project) -- [SmartUI API Documentation](https://www.testmuai.com/support/api-doc/) diff --git a/static/docs/smartui-cli-git-branching-strategy.md b/static/docs/smartui-cli-git-branching-strategy.md deleted file mode 100644 index fc7696b05..000000000 --- a/static/docs/smartui-cli-git-branching-strategy.md +++ /dev/null @@ -1,232 +0,0 @@ -# Branching Strategy in SmartUI SDK - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This is the guide to understand our Git Branching with SmartUI projects which can detect the commit history and execute the appropriate actions for your visual regression testing. - -### Prerequisites for running Git with SmartUI - -- You have an account with [TestMu AI](https://www.testmuai.com/login/). -- You must have an active [subscription](https://www.testmuai.com/pricing) plan with valid screenshots limit. -- You must have created an Project on the SmartUI web app. - -**Warning** -This feature is supported for tests ran through CLI project type ([SDK](/support/docs/smartui-selenium-java-sdk/), [Static CLI](/support/docs/smartui-cli/), [Storybook](/support/docs/smart-ui-storybook/)) projects on the SmartUI platform. Please contact support@testmuai.com for further information or requests. - -The following steps will guide you in running your Git branching Visual Regression test on TestMu AI SmartUI platform: - -**Baseline Branch**: A baseline branch build is a set of screenshots which are captured using SmartUI where all the screenshots are compared against for the non-baseline branch builds. - -**Non-Baseline Branches**: A non-baseline branch(s) build is a set of screenshots which are captured using SmartUI where all the screenshots are compared against with the **Baseline** build which are matched with the screenshot names. - -### **Step 1**: Setup your Git Baseline branch in the Project Settings - -The following are the steps to add **Baseline** branch to your `CLI projects` with Git commits: - -1. Go to **Project Settings** page for the created project. -2. Search for **Git Settings** and add your required **Baseline** branch in the input box. -3. Click on **Update Settings** button to update the project settings and set your **Baseline** Git branch. - -Now, after the successful setup of your **Baseline** branch of your project. - -### **Step 2**: Execute your SmartUI CLI with Git commits - -The following are the steps to execute your SmartUI CLI with Git commits on CLI projects: - -1. Go to your current **project repository** and run the following command: - -```bash -git init --y -``` - -You can ignore the above step if you already have a repository with `Git` initialized - -2. Now, commit the changes (if any) to your `Git`: - -```bash -git commit -m "Your commit message" -``` - -3. Execute your **SmartUI CLI** command to execute the Visual Regression tests for your files: - -```bash -smartui capture urls.json --config .smartui.json -``` - -```bash -npx smartui --config .smartui.json exec -- -``` - -```bash -npm run build-storybook // Creates a Static Build Folder of StoryBook Stories -smartui storybook ./storybook-static --config .smartui.json // Captures all the stories added in the static build folder -``` - -## Git Branching Workflows and Strategies - -### Ability to set the **Baseline Branch** - -You can set any `branch` name as your **Baseline** branch in the SmartUI project settings page. You can follow the steps below to set your baseline branch: - -1. Go to the SmartUI Web App. -2. Select the project in which you want to configure the `Git` baseline branch for. -3. Go to the project settings section by clicking on the top right icon on the `Builds` list page. -4. In the project settings section you can provide the input of the `branch` name. -5. Click on the `Update Settings` button to update the settings. - -You can also set the Baseline branch name through your terminal by setting the following environment variable which will automatically update your project settings as well once the test is run. - -```bash -export BASELINE_BRANCH="Required baseline branch" -``` - -```bash -set BASELINE_BRANCH="Required baseline branch" -``` - -```powershell -$env:BASELINE_BRANCH="Required baseline branch" -``` - -**Default Selection of Baseline Branch** - -In case, if there is no branch is set in your new or existing project created. Then, the `repository's` where SmartUI CLI command is executed to run the tests, **HEAD Branch** will be updated as the **Baseline** branch for that project by default. - -### Displaying the Git Commit Meta data - -If you are using the `Git` based workflow with `SmartUI CLI` then the following information is displayed on the UI of the SmartUI Web App - Builds list page: - -- **Branch Name** is set as the `Build Name` in the SmartUI project. -- The latest **commit ID**of the Git commit added to the current repository. -- The **commit message** of the Git commit of the `commit ID` to the current repository. -- The commit author **username** of the Git commit to the current repository. - -The above metadata is displayed only for CLI **Git** repositories. - -### Git Baseline Branch Selection and Comparison Strategy: - -### Working on the same branch name - -In this workflow, if you are working on the same branch name and want to execute the `SmartUI CLI` command to run the Visual Regression tests for your test cases. - -And, if you have created a project and set your **Baseline** branch name to `master` then executed the following commands: - -```powershell -// Your current branch name -$ git branch -* -<--Other branches--> - -// Adds a new commit to your current active branch -$ git commit -m "Your commit message" - -// Executing the SmartUI SDK/Static CLI/Storybook tests -$ npx smartui --config .smartui.json exec -- -``` - -Then, a new build will be created with your `Current Active Branch` on the SmartUI Web App - Build Lists page for the project in which the `SmartUI CLI` command is executed. - -If the current Baseline build branch is set to `Current Active Branch` in the SmartUI project settings, then SmartUI will compare the screenshots captured to the latest `APPROVED` status build of the `Current Active Branch` branch available in the same project. - -Here is an example of the above workflow: - -```bash -$ git branch -* master - -$ git commit -m "Second Build Changes" - -$ npx smartui --config .smartui.json exec -- -``` - -You can streamline your workflow by setting the current branch name directly through your terminal. By configuring the following environment variable, you can automatically designate the input branch name as the branch for that build. This approach effectively overwrites the current Git branch from which you are running the test, providing a convenient method to manage new branches according to specific use cases without altering the Git configuration. - -```bash -export CURRENT_BRANCH="Required branch" -``` - -```bash -set CURRENT_BRANCH="Required branch" -``` - -```powershell -$env:CURRENT_BRANCH="Required branch" -``` - -### Updating the Baseline Branch Name - -In this workflow, if you have created a SmartUI project on the Web App and have executed few builds with a `Branch Name: Master` and want to switch your **Baseline** branch to `development` branch in your `Git`, then following will be the workflow that will help you in switching the **Baseline Branch** for your project: - -1. You can go to the `SmartUI Project Settings` page to update your **Baseline Branch** in the project settings from `master` to `development` or update the `BASELINE_BRANCH` variable in the environment to update the baseline branch. -2. Now, execute the `SmartUI CLI` command to run the Visual Regression tests. -3. Then, SmartUI algorithm will check for existing latest `build` with `development` run on the SmartUI project in Builds history. - -The following are the cases that SmartUI will update the **Baseline** build for the branch set in your SmartUI project settings: - -##### **Case 1**: If a build with development branch exists in build history - -In this case, the latest build run on the SmartUI project build history will be updated as a **New Baseline Build** in which the new **Non-Baseline** builds will be compared against. - -##### **Case 2**: If a build with development branch does not exist in build history - -In this case, the new build run with the `development` git branch name will be set to the current **Baseline** build for the SmartUI project. - -**Note** -For the first build generated for the **Baseline branch build** in the SmartUI Projects will be automatically set to **SYSTEM APPROVED** status by default. - -### Detect changes for Git commit added to **Baseline Branch** - -In this workflow, if I make changes in the set Baseline branch in the SmartUI project that is master and make an commit to the Git then, - -Execute `SmartUI CLI` commands to execute the test cases. Tests will compare the results with the latest **APPROVED** build run for **Baseline Branch** build in the SmartUI project. - -On approval of the all the screenshots then the new build will be updated as Baseline build for the baseline branch. - -Here is an example of the above workflow: - -```bash -// Step 1 - Commit changes to Git -$ git commit -am "Changes in login Flow" -(git) Changes added to your branch - -// Step 2 - Execute SmartUI CLI to trigger Build in the SmartUI project -$ npx smartui --config .smartui.json exec -- -``` - -### Comparing Non-Baseline Branch builds with **Baseline** branch builds - -In this workflow, if you want to compare a **non-baseline branch build** to a **baseline build** the following are steps to be followed: - -Step 1: Checkout to a non-baseline branch in your `Git` repository. - -Step 2: Then run the SmartUI CLI command to execute the tests should compare the screenshots captured of **Non-Baseline** branch build with the **Baseline branch** latest **APPROVED** build in the SmartUI project. - -Here is an example of the above workflow: - -```bash -// Baseline Branch -$ git branch -* master - -// Non-Baseline branch -$ git checkout -b develop -* develop -master - -// Executing SmartUI Build with Non-Baseline Git branch -$ npx smartui --config .smartui.json exec -- -``` - -### Handling Builds with Missing Screenshots - -When a build is executed in the same branch as the **Baseline Branch**, SmartUI will compare it against the latest approved baseline build. If some screenshots (or variants) that were present in the baseline are missing from the current build, a `Missing Screenshots` warning will be displayed. - -If this build with missing screenshots shows no visual changes compared to the baseline, it will be automatically approved. However, SmartUI will mark it as an `Incomplete` build and will not set it as a new baseline. - -For builds with missing screenshots that do contain visual changes, you can manually approve them to set them as a new baseline if desired. - -> **Note**: If a build contains any new screenshots that weren't present in the baseline, SmartUI interprets this as an intentional test case modification. In such cases, the `Missing Screenshots` tooltip will not be displayed, even if some screenshots from the baseline are missing. - -### Auto Updating **Baseline** build for the **Baseline** branch - -In this workflow, once all the `Changes Found` are approved by the `approver` for the screenshots in the **Baseline** branch's **Non-Baseline** build then the `latest` approved **Baseline Branch** build will be updated to the `Baseline`. diff --git a/static/docs/smartui-cli-request-headers.md b/static/docs/smartui-cli-request-headers.md deleted file mode 100644 index cfc541c22..000000000 --- a/static/docs/smartui-cli-request-headers.md +++ /dev/null @@ -1,53 +0,0 @@ -# Adding Custom Request Headers in SmartUI CLI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The SmartUI CLI includes a `requestHeaders` configuration option, allowing you to specify custom HTTP headers for fetching web assets during your visual testing workflow. This feature is essential for scenarios where resources like images, CSS, or scripts are located behind a protected environment that requires authentication via custom headers (e.g., a staging environment). - -By including these headers in the SmartUI CLI configuration, you can seamlessly access protected resources, enhancing the flexibility and security of your automated visual testing process. - -### Steps to Implement - -1. **Locate Your Configuration File**: Open your `smartui.json` file, which should be in the root directory of your project. -2. **Add the `requestHeaders` Array**: If the array does not already exist, add it to the JSON structure. -3. **Define Your Headers**: Inside the `requestHeaders` array, add a new object for each header you need to send. Each object must contain a single key-value pair representing the header's name and its value. -4. **Save the File**: Once you save the changes, the SmartUI CLI will use this configuration for the next test run. - -## Configuration in `smartui.json` - -To use this feature, you need to add the `requestHeaders` array to your `smartui.json` configuration file. This array contains objects, where each object represents a key-value pair for a custom header. The CLI will automatically include these headers in all HTTP requests it makes to fetch assets for rendering the webpage. - -### Example Configuration - -Below is an example of how to configure custom headers in your `smartui.json` file. - -```json title="smartui.json" -{ -"smartUI": { -"project": "Your Project Name", -"build": "Build Name" -}, -"requestHeaders": [ -{ -"X-Custom-Header-1": "custom-value-1" -}, -{ -"Authorization": "Bearer your-access-token" -}, -{ -"X-Custom-Header-2": "custom-value-2" -} -] -} -```` - -### Explanation - - * **`requestHeaders`**: An array of objects. - * **Header Object**: Each object in the array defines a single HTTP header. - * **Key**: The name of the HTTP header (e.g., `Authorization`, `X-Custom-Header-1`). - * **Value**: The corresponding value for the header (e.g., `Bearer your-access-token`, `custom-value-1`). - - * You can define multiple headers by adding more objects to the `requestHeaders` array. - * The headers defined in this configuration will be sent with every page load and asset request made by the SmartUI CLI during the test. - * This feature is particularly useful for testing web applications in staging or development environments that are protected from public access. diff --git a/static/docs/smartui-cli-responsive-dom.md b/static/docs/smartui-cli-responsive-dom.md deleted file mode 100644 index 6c92523a3..000000000 --- a/static/docs/smartui-cli-responsive-dom.md +++ /dev/null @@ -1,73 +0,0 @@ -# Capturing Responsive DOM with SmartUI SDK - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -If your application dynamically changes the DOM based on viewport size or specific breakpoints, SmartUI SDKs can be configured to adapt accordingly. When using the SmartUI CLI, the SDK captures the DOM state at the specified viewport size within the test browser, then renders it across the browsers and resolutions defined in your [configuration file](/support/docs/smartui-sdk-config-options/). - -Therefore, if your application adjusts DOM elements between different screen sizes, ensure that you resize the viewport of your test browser and capture a snapshot at each required width. - -## Prerequisites - -1. **SmartUI CLI Version:** Ensure you are using `@lambdatest/smartui-cli` version 4.1.43 or higher -2. **SDK Installation:** Install the **SDK** with the specified compatible version. -3. **Recommended Frameworks:** For optimal performance, use either Playwright or Selenium frameworks. - -If you face any problems executing tests with SmartUI-CLI `versions >= v4.x.x`, upgrade your Node.js version to `v20.3` or above. - -## Steps to Use - -### Step-1 -Specify `delayedUpload` to true in your configuration file. - -```json -"delayedUpload" : true -``` - -### Step-2 - -Resize your viewport according to your framework and take the screenshot. For example in **Selenium NodeJs**, you can do it as follows: - -```js - -let options = -{ -"web": { -"browsers": [ -"chrome", -"firefox", -"safari" -], -"viewports": [[768]], //required resized dom viewport -}, -ignoreDOM: { -class: ["required-class"], -} -}; -await driver.manage().window().setSize(768, 1024); -await smartuiSnapshot(driver, 'Screenshot Name', options); - -let mobileOptions= -{ -"mobile": { -"devices": [ -"iPhone 14", -"Galaxy S23" -], -"fullPage": true, -"orientation": "portrait" -}, -ignoreDOM: { -class: ["nextAvailDate"], -} -}; -await driver.manage().window().setSize(360, 1024); -await smartuiSnapshot(driver, 'Screenshot Name', mobileOptions); -``` - ->Note :: Any existing screenshots with same screenshot name,browser and resolution will get over-written. - -## Things to keep in mind: - -1. **Viewport and Browser Configuration:** You can set the viewport size or specify browsers for individual snapshots either globally in the configuration file or at the snapshot level based on your needs. -2. **Firefox Limitation:** Firefox can only capture snapshots at DOM widths of `450px` or larger due to resizing limitations. -3. **Chrome Resizing in Selenium:** In Selenium, Chrome’s viewport cannot be resized below `500px` due to a limitation in Selenium. diff --git a/static/docs/smartui-cli-upload.md b/static/docs/smartui-cli-upload.md deleted file mode 100644 index b19611e59..000000000 --- a/static/docs/smartui-cli-upload.md +++ /dev/null @@ -1,154 +0,0 @@ -# Upload your Screenshots via CLI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -With SmartUI CLI, you can seamlessly perform visual regression testing on the TestMu AI platform using your command line, identifying Visual UI Regression bugs effortlessly. This guide will walk you through the process of uploading and comparing images using the SmartUI CLI. - -## Prerequisites for running SmartUI CLI - -- Basic understanding of Command Line Interface is required. -- Login to [TestMu AI SmartUI](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) with your credentials. - -The following steps will guide you in running your first Visual Regression test on TestMu AI platform using SmartUI CLI. - -## Create a SmartUI Web-Project - -The first step is to create a project with the application in which we will combine all your builds run on the project. To create a SmartUI Project, follow these steps: - -1. Go to [Projects page](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) -2. Click on the `new project` button -3. Select the platform as CLI for executing your `CLI` tests. -4. Add name of the project, approvers for the changes found, tags for any filter or easy navigation. -5. Click on the **Submit**. - -## Step 1: Install the Dependencies - -Install required NPM modules for `TestMu AI SmartUI CLI` globally or in your project: - -**Global Installation (Recommended):** -```bash -npm install -g @lambdatest/smartui-cli -``` - -**Local Installation:** -```bash -npm install @lambdatest/smartui-cli -``` - -If you face any problems executing tests with SmartUI-CLI `versions >= v4.x.x`, upgrade your Node.js version to `v20.3` or above. - -## Step 2: Configure your Project Token - -Setup your project token shown in the **SmartUI** app after creating your project. - -```bash -export PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -``` - -```bash -set PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -``` - -```powershell -$env:PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -``` - -## Step 3: Upload the required directory of images - -``` -npx smartui upload -``` -The screenshots in the directory name will be uploaded to SmartUI. - -### For ignoring the resolutions of the images - -By default, SmartUI captures and considers image resolutions. If you prefer to ignore resolutions and compare images solely based on their names, use the following flag: - -``` -npx smartui upload --ignoreResolutions -``` - -### SmartUI CLI Upload Options - -Please read the following table for more information about the options available to upload a directory of static images to SmartUI. - -| Config Key Shortcut | Configuration Key     | Description| Usage     | -| ---------------------| ---------------------------------------------------------------------------|--------------| -------------------- | -| -R | --ignoreResolutions | Ignores resolutions to compare only based on screenshot names | `npx smartui upload -R` | -| -F | --files `extensions` | Comma-separated list of allowed file extensions| `npx smartui upload -F jpg,png`| -| -E | --removeExtensions | Strips file extensions from snapshot names| `npx smartui upload -E` | -| -i | --ignoreDir `patterns` | Comma-separated list of directories to ignore | `npx smartui upload -i dir1/dir2,dir3` | -| | --fetch-results [filename] | Live fetched comparison results in a json file | `npx smartui upload --fetch-results [custom-results.json]`| -| | --buildName <string> | Specify a custom build name | `npx smartui upload --buildName "Build Name"`| -| | --userName <string> | Override TestMu AI username | `npx smartui upload --userName "user"`| -| | --accessKey <string> | Override TestMu AI access key | `npx smartui upload --accessKey "key"`| -| | --config <filepath> | Configuration file path | `npx smartui upload --config .smartui.json`| - -You may use the `smartui upload --help` command in case you are facing issues during the execution of SmartUI Upload options in the CLI. - -## Setup with Continuous Integration (CI) - -If you are using the Continuous Integration (CI) pipeline for your application and want to integrate `SmartUI CLI` execution then the following are the steps needs to be added to your `.yaml` file: - -```yaml -steps: -- name: Running SmartUI CLI Tests -- run: | -npm install @lambdatest/smartui-cli -npx playwright install-deps -npx smartui upload --removeExtensions -``` - -## SmartUI CLI Options and Keys - -The following are supported `CLI (Command Line Interface)` options for Visual Regression Testing with SmartUI: - -| CLI Flag Key | Description | Usage | -| ------------ | ----------------------------------------------------------------------------------- | -------- | -| --config | This is the reference configuration file containing the SmartUI Cloud Configuration | Optional | -| --help | This will print all help information for the SmartUI CLI options | Optional | - -## View SmartUI Results - -You can see the SmartUI dashboard to view the results. This will help you identify the Mismatches from the existing `Baseline` build and do the required visual testing. - -## Fetch results - -You can fetch build results by adding the `--fetch-results` flag to your test execution command. Here are different ways to use this feature: - -### Default Usage -If no filename is specified, results will be stored in `results.json`: - -```bash -npx smartui upload --fetch-results -``` - -### Custom Filename -Specify a custom filename for your results: - -```bash -npx smartui upload --fetch-results custom-results.json -``` -## Adding a custom build name -You can add a custom build name by adding the `--buildName` flag to your test execution command. Here is how you can utilise this feature: - -Specify a custom build name to group your screenshots in the following way: - -```bash -npx smartui upload --buildName "Sample Build Name" -``` - -> Note: -> - If no build name is provided, a random name will be automatically assigned to the build. -> - Specifying the name of an existing build within the project will append the screenshots to that build. -> - Existing screenshots with the same name and configuration in the build will be overwritten during a re-run. - -## Additional Resources - -- [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) -- [CLI Documentation](/support/docs/smartui-cli) -- [CLI Exec Commands](/support/docs/smartui-cli-exec) -- [Environment Variables](/support/docs/smartui-cli-env-variables) -- [Baseline Management](/support/docs/smartui-baseline-management) -- [Running Your First Project](/support/docs/smartui-running-your-first-project) -- [SmartUI API Documentation](https://www.testmuai.com/support/api-doc/) diff --git a/static/docs/smartui-cli.md b/static/docs/smartui-cli.md deleted file mode 100644 index 1c82e50ea..000000000 --- a/static/docs/smartui-cli.md +++ /dev/null @@ -1,322 +0,0 @@ -# Capture Static URLs via CLI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -With SmartUI CLI, you can seamlessly perform visual regression testing on the TestMu AI platform using your command line, identifying Visual UI Regression bugs effortlessly. This guide will walk you through the process of running successful Visual Regression tests using SmartUI CLI. - -## Prerequisites for running SmartUI CLI - -- Basic understanding of Command Line Interface is required. -- Login to [TestMu AI SmartUI](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) with your credentials. - -The following steps will guide you in running your first Visual Regression test on TestMu AI platform using SmartUI CLI. - -## Create a SmartUI Web-Project - -The first step is to create a project with the application in which we will combine all your builds run on the project. To create a SmartUI Project, follow these steps: - -1. Go to [Projects page](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) -2. Click on the `new project` button -3. Select the platform as CLI for executing your `CLI` tests. -4. Add name of the project, approvers for the changes found, tags for any filter or easy navigation. -5. Click on the **Submit**. - -## Step 1: Install the Dependencies - -Install required NPM modules for `TestMu AI SmartUI CLI` globally or in your project: - -**Global Installation (Recommended):** -```bash -npm install -g @lambdatest/smartui-cli -``` - -**Local Installation:** -```bash -npm install @lambdatest/smartui-cli -``` - -If you face any problems executing tests with SmartUI-CLI `versions >= v4.x.x`, upgrade your Node.js version to `v20.3` or above. - -## Step 2: Create URL file - -``` -smartui config:create-web-static urls.json -``` -Once, the `URLs` file will be created, you will be seeing the sample pre-filled URLs in the `urls.json` file: - -```json title="/smartui-cli-project/urls.json" -[ -{ -"name": "lambdatest-home-page", -"url": "https://www.testmuai.com", -"waitForTimeout": 1000 //Optional -}, -{ -"name": "example-page", -"url": "https://example.com/" -} -] - -``` -**Please Note** -The `waitForTimeout` is an optional configuration. - -If you are using any async components, you can add wait time for the page to load the DOM of your components. This can help avoid false-positive results for your tests. You can add the wait time in milliseconds, which might increase the execution time of your tests. - -## Step 3: Configure your Project Token - -Setup your project token shown in the **SmartUI** app after creating your project. - -```bash -export PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -``` - -```bash -set PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -``` - -```powershell -$env:PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -``` - -## Step 4: Create and Configure SmartUI Config - -You can now configure your project settings on using various available options to run your tests with the SmartUI integration. To generate the configuration file, please execute the following command: - -```bash -npx smartui config:create .smartui.json -``` - -Once, the configuration file will be created, you will be seeing the default configuration pre-filled in the configuration file: - -```json title="/smartui-sdk-project/.smartui.json" -{ -"web": { -"browsers": [ -"chrome", -"firefox", -"safari", -"edge" -], -"viewports": [ -[ -1920 -], -[ -1366 -], -[ -1028 -] -] // Full Page screenshots are captured by default for web viewports -}, -"mobile": { -"devices": [ -"iPhone 14", //iPhone 14 viewport -"Galaxy S24" //Galaxy S24 viewport -], -"fullPage": true, //Full Page is true by default for mobile viewports -"orientation": "portrait" //Change to "landscape" for landscape snapshot -} -} -``` - -### For capturing viewport screenshots - -To capture a screenshot of the content currently visible in your viewport, rather than the entire page, it's important to define the viewport's width and height in your configuration settings. Specify the desired width and height parameters as demonstrated in the following example to ensure that the screenshot encompasses only the viewport area. - -```json -"viewports": [ -[ -1920, -1080 -], -[ -1366, -768 -], -[ -360, -640 -] -], -``` - -You may use the `smartui --help` command in case you are facing issues during the execution of SmartUI commands in the CLI. - -### SmartUI CLI Config Options - -Please read the following table for more information about the configuration file: - -| Config Key | Description | Usage | -| -------------- | ---------------------------------------------------------------------------------------------------------------------------------- | --------- | -| browsers | You can add all the supported browsers brands here to run your tests for SmartUI. Ex: `"chrome", "firefox", "safari", "edge", etc..` | Mandatory | -| viewports | You can add all the supported browser viewpoints here to run your tests for SmartUI Ex: `[1920, 1080],[width, height] etc..` | Mandatory | - -## **Step 5:** Execute the Tests on SmartUI Cloud using CLI - -You can now execute tests for `Visual Regression Testing` using the following options:. - -```bash -npx smartui capture urls.json --config .smartui.json -``` - -### Additional Options - -You can also use the following options with the capture command: - -- `--scheduled ` - Specify schedule ID for scheduled test runs -- `--userName ` - Override TestMu AI username -- `--accessKey ` - Override TestMu AI access key -- `--buildName ` - Specify a custom build name -- `--fetch-results [filename]` - Fetch and save results to JSON file - -**Example with additional options:** -```bash -npx smartui capture urls.json --config .smartui.json --buildName "Release-v1.0" --fetch-results results.json -``` - -## Executing browser scripts using the Capture Command - -SmartUI CLI provides the flexibility to execute custom JavaScript code on target URLs at specific stages of the snapshot process. This feature is crucial for: - -- Interacting with dynamic elements -- Managing loading states -- Modifying page content -- Validating page states -- Performing actions like accepting cookies, clicking on buttons, etc. -These capabilities ensure that your visual tests accurately capture the true representation of your web pages. - -You can execute any valid JavaScript code that you would typically run inside a browser. For example, to simulate a button click, you might use: - -```js -document.querySelector('.my-button-example').click(); -``` - -### Execute Option Keys - -The `execute` option accepts an object with the following keys: - -- **afterNavigation**: This function is called after the page navigates to the specified URL. It is useful for actions that need to occur once the page has loaded. - -- **beforeSnapshot**: This function is called just before SmartUI captures a snapshot. It is ideal for final adjustments or interactions needed to prepare the page for capture. - -Example usage in a configuration: - -```json -{ -"name": "Example Page", -"url": "https://example.com/", -"waitForTimeout": 3000, -"execute": { -"afterNavigation": "await page.waitForSelector('.loading', { hidden: true })", -"beforeSnapshot": "document.querySelector('.cookie-banner').click()" -} -} -``` - -This example waits for a loading element to disappear after navigation and clicks a cookie banner before taking a snapshot. - -## Fetch results - -You can fetch build results by adding the `--fetch-results` flag to your test execution command. Here are different ways to use this feature: - -### Default Usage -If no filename is specified, results will be stored in `results.json`: - -```bash -npx smartui capture urls.json --config .smartui.json --fetch-results -``` - -### Custom Filename -Specify a custom filename for your results: - -```bash -npx smartui capture urls.json --config .smartui.json --fetch-results custom-results.json -``` - -## Adding a custom build name -You can add a custom build name by adding the `--buildName` flag to your test execution command. Here is how you can utilise this feature: - -Specify a custom build name to group your screenshots in the following way: - -```bash -npx smartui capture urls.json --buildName "Sample Build Name" --config .smartui.json -``` - -> Note: -> - If no build name is provided, a random name will be automatically assigned to the build. -> - Specifying the name of an existing build within the project will append the screenshots to that build. -> - Existing screenshots with the same name and configuration in the build will be overwritten during a re-run. - -## Setup with Continuous Integration (CI) - -If you are using the Continuous Integration (CI) pipeline for your application and want to integrate `SmartUI CLI` execution then the following are the steps needs to be added to your `.yaml` file: - -```yaml -steps: -- name: Running SmartUI CLI Tests -- run: | -npm install -g @lambdatest/smartui-cli -npx playwright install-deps -smartui capture urls.json --config smartui-web.json -``` - -## SmartUI CLI Options and Keys - -The following are supported `CLI (Command Line Interface)` options for Visual Regression Testing with SmartUI: - -| CLI Flag Key | Description | Usage | -| ------------ | ----------------------------------------------------------------------------------- | -------- | -| --config | This is the reference configuration file containing the SmartUI Cloud Configuration | Optional | -| --help | This will print all help information for the SmartUI CLI options | Optional | - -### View SmartUI Results - -You can see the SmartUI dashboard to view the results. This will help you identify the Mismatches from the existing `Baseline` build and do the required visual testing. - -## Parallel execution of static URLs - -You can reduce the build time by executing parallel URLs in the following way. - -```bash -npx smartui capture urls.json --config .smartui.json --parallel --fetch-results -``` - -- The `--parallel` flag determines how many URLs will be processed simultaneously per browser -- Each thread captures screenshots independently, maximizing throughput -- Use `--force` flag to forcefully apply parallel instances even if not optimal - ->**Example:** ->```bash ->npx smartui capture urls.json --config .smartui.json --parallel 3 --fetch-results ->``` - ->**Example with force flag:** ->```bash ->npx smartui capture urls.json --config .smartui.json --parallel 5 --force --fetch-results ->``` - -### Determining Optimal Thread Count -The maximum number of parallel threads is calculated using the formula: log2(N) where N is the total number of URLs. - ->**For example:** ->- For 100 URLs: Maximum parallel threads = log2(100) = 6 threads ->- For 50 URLs: Maximum parallel threads = log2(50) = 5 threads ->- For 25 URLs: Maximum parallel threads = log2(25) = 4 threads - -### Best Practices for parallel execution - -- Start with a lower thread count and gradually increase based on your system's performance -- Monitor system resources during execution -- Ensure stable internet connection for reliable parallel processing - -## Additional Resources - -- [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) -- [CLI Exec Commands](/support/docs/smartui-cli-exec) -- [CLI Upload](/support/docs/smartui-cli-upload) -- [Environment Variables](/support/docs/smartui-cli-env-variables) -- [Baseline Management](/support/docs/smartui-baseline-management) -- [Running Your First Project](/support/docs/smartui-running-your-first-project) -- [SmartUI API Documentation](https://www.testmuai.com/support/api-doc/) diff --git a/static/docs/smartui-comparison-capabilities.md b/static/docs/smartui-comparison-capabilities.md deleted file mode 100644 index 167227afb..000000000 --- a/static/docs/smartui-comparison-capabilities.md +++ /dev/null @@ -1,164 +0,0 @@ -# Set SmartUI Comparison Modes at the Session Level - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -SmartUI now lets you declare a comparison mode **once at the session level** and have it apply to every screenshot in that run. Set the mode in `LT:Options` alongside your other capabilities, and SmartUI uses it as the default for each `smartui.takeScreenshot` call, no need to repeat the option on every screenshot. - -This is ideal when a whole project or test suite should share the same comparison behaviour. A team can switch a project between comparison modes by changing a single capability line, while individual screenshots can still opt into a different mode when they need to. - -## How it works - -SmartUI resolves the comparison mode for each screenshot in two layers: - -| Layer | Where you set it | Scope | -|-------|------------------|-------| -| **Session default** | A `smartUI.*` capability in `LT:Options` | Applies to every screenshot in the session. | -| **Per-screenshot override** | The options map passed to `smartui.takeScreenshot` | Applies to that one screenshot and wins over the session default. | - -If a screenshot does not specify a comparison option, it inherits the session default from `LT:Options`. If it does specify one, the screenshot-level value takes precedence for that screenshot only. This means you can set a baseline behaviour for the run and still fine-tune individual screenshots. - -## Supported comparison capabilities - -Set any of these in the `LT:Options` block for the session: - -| Capability | Type | Effect | -|------------|------|--------| -| `smartUI.ignoreType` | array | Comparison type applied to every screenshot, for example `["layout"]` for layout comparison. | -| `smartUI.smartIgnore` | boolean | Enables [Smart Ignore](/support/docs/smartui-smartignore/) for the whole session. | - -Over the CDP-based drivers (Puppeteer and Playwright via CDP), `smartUI.ignoreType` is also accepted under its alias **`smartUIIgnoreType`**. Both keys are equivalent; use whichever your driver expects. - -Set the same comparison capabilities on both the baseline and the comparison runs so the two builds are compared in the same mode. - -## 1. Layout comparison for the whole session - -Set `smartUI.ignoreType` to `["layout"]` in `LT:Options`. Every screenshot in the session is then compared in layout mode, the same as passing `ignoreType: ["layout"]` on each individual screenshot. - -```java -import java.util.Arrays; -import java.util.HashMap; -import org.openqa.selenium.chrome.ChromeOptions; - -ChromeOptions browserOptions = new ChromeOptions(); -HashMap ltOptions = new HashMap<>(); -ltOptions.put("username", System.getenv("LT_USERNAME")); -ltOptions.put("accessKey", System.getenv("LT_ACCESS_KEY")); -ltOptions.put("visual", true); -ltOptions.put("smartUI.project", "Your_Project_Name"); -ltOptions.put("smartUI.ignoreType", Arrays.asList("layout")); // applies to every screenshot - -browserOptions.setCapability("LT:Options", ltOptions); -``` - -```javascript -'LT:Options': { -user: process.env.LT_USERNAME, -accessKey: process.env.LT_ACCESS_KEY, -visual: true, -'smartUI.project': 'Your_Project_Name', -'smartUI.ignoreType': ['layout'], // applies to every screenshot -}, -``` - -```python -lt_options = { -"username": os.getenv("LT_USERNAME"), -"accessKey": os.getenv("LT_ACCESS_KEY"), -"visual": True, -"smartUI.project": "Your_Project_Name", -"smartUI.ignoreType": ["layout"], # applies to every screenshot -} -options.set_capability("LT:Options", lt_options) -``` - -```csharp -capabilities.SetCapability("visual", true); -capabilities.SetCapability("smartUI.project", "Your_Project_Name"); -capabilities.SetCapability("smartUI.ignoreType", new[] { "layout" }); // applies to every screenshot -``` - -With this set, your screenshot calls stay clean, no comparison option is needed on each one: - -```java -Map options = new HashMap<>(); -options.put("screenshotName", "home-page"); -((JavascriptExecutor) driver).executeScript("smartui.takeScreenshot", options); -// compared in layout mode because of the session default -``` - -## 2. Smart Ignore for the whole session - -Set `smartUI.smartIgnore` to `true` in `LT:Options` to apply [Smart Ignore](/support/docs/smartui-smartignore/) across the session. - -```java -ltOptions.put("smartUI.project", "Your_Project_Name"); -ltOptions.put("smartUI.smartIgnore", true); // applies to every screenshot -``` - -```javascript -'LT:Options': { -'smartUI.project': 'Your_Project_Name', -'smartUI.smartIgnore': true, // applies to every screenshot -}, -``` - -```python -lt_options = { -"smartUI.project": "Your_Project_Name", -"smartUI.smartIgnore": True, # applies to every screenshot -} -``` - -```csharp -capabilities.SetCapability("smartUI.project", "Your_Project_Name"); -capabilities.SetCapability("smartUI.smartIgnore", true); // applies to every screenshot -``` - -## 3. Ignore regions are set per screenshot, not per session - -There is no session-level ignore-regions capability. Ignore regions are configured **per screenshot** through the `ignoreDOM` (and `selectDOM`) argument on each `smartui.takeScreenshot` call, where you can pass DOM selectors or pixel coordinates: - -```java -HashMap ignoreByCoord = new HashMap<>(); -ignoreByCoord.put("coordinates", new String[]{"847,185,1571,734"}); - -Map options = new HashMap<>(); -options.put("screenshotName", "home-page"); -options.put("ignoreDOM", ignoreByCoord); // applies to this screenshot only -((JavascriptExecutor) driver).executeScript("smartui.takeScreenshot", options); -``` - -For the full set of coordinate and selector options on the Web Hooks path, see [Ignore and Select Regions on Web Hooks](/support/docs/smartui-hooks-region-ignore/). - -## 4. Override the session default for one screenshot - -Because the screenshot-level option always wins, you can keep a session default and still change the mode for a specific screenshot. For example, with a layout default set in `LT:Options`, you can compare one screenshot in the standard pixel mode by passing a different `ignoreType` on that call: - -```java -Map options = new HashMap<>(); -options.put("screenshotName", "pricing-table"); -options.put("ignoreType", Arrays.asList()); // override: compare this one in full detail -((JavascriptExecutor) driver).executeScript("smartui.takeScreenshot", options); -``` - -## Behaviour summary - -| You set | Where | Result | -|---------|-------|--------| -| `smartUI.ignoreType: ["layout"]` | `LT:Options` | Every screenshot uses layout comparison by default. | -| `smartUI.smartIgnore: true` | `LT:Options` | Smart Ignore applied across the session. | -| A comparison option on a screenshot | `smartui.takeScreenshot` | Overrides the session default for that screenshot only. | - -## Baseline and comparison notes - -- Use the same `smartUI.project` and screenshot names across runs. -- Apply the same comparison capabilities to both the baseline and comparison sessions. -- Changing the comparison mode generally requires a fresh baseline. - -## Related Docs - -- [SmartUI Hooks: Layout, Full-Page Screenshots, and Smart Ignore](/support/docs/smartui-hooks-layout-fullpage-smartignore/) -- [Ignore and Select Regions on Web Hooks](/support/docs/smartui-hooks-region-ignore/) -- [Smart Ignore](/support/docs/smartui-smartignore/) -- [Handling Dynamic Data with DOM Configuration and Options](/support/docs/html-dom-smartui-options/) -- [Selenium Visual Regression](/support/docs/selenium-visual-regression/) diff --git a/static/docs/smartui-custom-css.md b/static/docs/smartui-custom-css.md deleted file mode 100644 index 3d65131d7..000000000 --- a/static/docs/smartui-custom-css.md +++ /dev/null @@ -1,220 +0,0 @@ -# Custom CSS Injection in SmartUI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Custom CSS injection is a specialized feature in SmartUI that allows you to apply test-only styles during snapshot capture without modifying your application code. This feature enables you to stabilize visual tests by normalizing dynamic content, enforcing consistent styling across environments, and masking sensitive information, all while keeping your visual testing logic centralized and maintainable. - -### Why Custom CSS Matters - -1. **Stabilize Visual Diffs**: Hide or normalize volatile UI elements (ads, rotating banners, dynamic counters, time-based content) to reduce false positives in your visual regression tests. - -2. **Environment Parity**: Enforce consistent fonts, themes, and spacing across different operating systems, browsers, and devices for predictable rendering and accurate comparisons. - -3. **Non-Intrusive Testing**: Avoid modifying application code or injecting styles directly in test scripts. Keep all visual adjustments in a centralized configuration, making it easier to maintain and review. - -4. **Enhanced Test Reliability**: Reduce test flakiness by masking dynamic elements that change between test runs, ensuring your visual tests focus on meaningful UI changes rather than transient content. - -## Prerequisites - -Before using the Custom CSS feature, ensure you meet the following requirements: - -- Node.js v20.3+ (recommended) -- SmartUI CLI v4.1.40+ (supports both `exec` and `capture` commands) -- Valid PROJECT_TOKEN configured in your environment - -# Custom CSS Configuration in SmartUI - -SmartUI supports two methods for providing custom CSS: file path (recommended for maintainability) and embedded string (quick and portable). Choose the method that best fits your workflow. - -## Method 1: File Path (Recommended) - -The file path method is recommended for larger stylesheets and team collaboration. Create a separate CSS file in your project and reference it in your SmartUI configuration. - -**Steps:** - -1. Create a CSS file (e.g., `visual-test-styles.css` or `code.css`) in your project directory. - -2. Add your CSS rules to the file: - -```css -/* General samples: pick what suits your use case */ - -/* 1) Normalize fonts for consistent rendering */ -body { font-family: system-ui, -apple-system, Segoe UI, Roboto, sans-serif; } - -/* 2) Hide flaky, time-based banners or rotating promos */ -.promo-banner, [data-testid="rotating-banner"] { display: none !important; } - -/* 3) Freeze dynamic badges/counters that change every run */ -[data-badge], .cart-count { visibility: hidden; } -``` - -3. Reference the file path in your SmartUI configuration: - -```json -{ -"web": { -"browsers": ["chrome"], -"viewports": [[1440, 900]] -}, -"enableJavaScript": true, -"customCSS": "./visual-test-styles.css" -} -``` - -## Method 2: Embedded String - -The embedded string method is useful for quick edits and single-use CSS rules. Provide the CSS directly as a JSON string value. - -**Important Notes:** - -- Keep CSS single-line to avoid JSON parsing issues -- Escape quotes properly (`'` or `\"`) -- Use `\n` for newlines if needed (though single-line is preferred) - -```json -{ -"web": { -"browsers": ["chrome"], -"viewports": [[1440, 900]] -}, -"enableJavaScript": true, -"customCSS": "body{font-family:'Inter',sans-serif!important;} .banner,.ad{display:none!important;}" -} -``` - -## Configuration Guidelines - -- **Placement**: The `customCSS` property must be placed at the top level of your configuration file (not inside the `web` object). Placing it inside `web` will result in a "must NOT have additional properties" error. - -- **Path Resolution**: File paths are relative to your project root directory. Ensure the CSS file exists at the specified path. Files outside the project directory may not be accessible. - -- **CSS Specificity**: Your custom CSS will be injected at snapshot time. Use `!important` declarations if you need to override existing styles. If CSS is overridden by inline styles, increase selector specificity (e.g., `.target-class` → `#specific-id .target-class`). - -- **Multi-line CSS**: For multi-line CSS, prefer the file path method to avoid JSON escaping complexity. If using embedded strings, keep rules compact and single-line. - -- **Method Selection**: Choose the file path method for larger stylesheets and team collaboration. Use embedded strings only for quick, single-use CSS rules. - -- **CSS Organization**: Keep your CSS snapshot-specific: target only elements that need stabilization or normalization. Avoid broad selectors that might affect unintended elements. - -- **JSON Escaping**: When using embedded strings, escape quotes properly: - - Use single quotes within CSS strings: `"body{font-family:'Inter',sans-serif;}"` - - Or escape double quotes: `"body{font-family:\"Inter\",sans-serif;}"` - -- **Selector Verification**: If the CSS Injection Report shows "no elements found" or "invalid selector" errors: - - Verify the element exists at snapshot time (use browser dev tools) - - Check selector specificity: it may need to be more specific or less specific - - Consider timing issues: ensure the element is rendered before snapshot capture - - Check the CSS Injection Report in logs for detailed feedback - -- **CLI Version**: Ensure SmartUI CLI v4.1.40+ is installed. You can verify this by running `npx smartui --version`. Older versions may not support the `customCSS` feature. - -- **Waiting for UI Readiness**: If you need to wait for UI elements to be ready before snapshots, add these options to your configuration: -```json -{ -"waitForTimeout": 2000, -"waitForPageRender": 5000, -"customCSS": "./visual-test-styles.css" -} -``` - -## Known Limitations - -The Custom CSS feature has the following limitations: - -- **Specificity Constraints**: Custom CSS can be overridden by higher-specificity inline styles. Increase your selector specificity or use `!important` declarations if needed. - -- **Snapshot-Only Application**: CSS is only injected during snapshot capture and does not affect your application's runtime behavior. - -- **File Path Resolution**: Ensure CSS file paths are correctly specified relative to your project root. Files outside the project directory may not be accessible. - -## Use Cases for Custom CSS - -The custom CSS feature is particularly valuable in the following scenarios: - -```css -/* Hide elements that change between runs */ -.ad, .banner, #cookie-consent { display: none !important; } - -/* Replace volatile text with a constant */ -[data-testid="rotating-copy"] { font-size: 0 !important; } -[data-testid="rotating-copy"]::after { content: "Stable text for snapshots"; } -``` - -```css -/* Force consistent light theme */ -:root { color-scheme: light; } -body { font-family: "Inter", system-ui, sans-serif !important; color: #111827; background: #ffffff; } - -/* Optional: Dark mode */ -/* :root { color-scheme: dark; } -body { background: #0f172a !important; color: #e5e7eb !important; } */ -``` - -```css -/* Center content and unify spacing (use carefully) */ -#root, main, section { display: flex !important; flex-direction: column !important; align-items: center !important; gap: 12px !important; } -* { text-align: center !important; } -``` - -```css -/* Hide IPs, locations, or IDs */ -#ip-value, #location-value { font-size: 0 !important; } -#ip-value::after { content: "0.0.0.0" !important; } -#location-value::after { content: "Unknown" !important; } -``` - -```css -/* Deliberately apply a very different theme */ -body { -font-family: "Century Gothic","URW Gothic","Apple Gothic",system-ui,Helvetica,Arial,sans-serif !important; -background-image: linear-gradient(180deg,rgba(0,0,0,.55),rgba(0,0,0,.75)), -url('https://images.unsplash.com/photo-1500530855697-b586d89ba3ee?q=80&w=1920&auto=format&fit=crop'); -background-size: cover; background-attachment: fixed; background-position: center; -color: #e6e6e6 !important; -} -``` - -```css -/* Disable transitions/animations */ -*, *::before, *::after { transition: none !important; animation: none !important; } -``` - -```css -/* Override brand cues (colors, shadows, shapes) */ -header, footer, nav { background: rgba(0,0,0,.45) !important; box-shadow: none !important; } -.btn, a { border-radius: 10px !important; border: 1px solid rgba(255,255,255,.25) !important; } -``` - -## Minimal Configuration Templates - -### File Path Template - -```json -{ -"web": { -"browsers": ["chrome"], -"viewports": [[1440, 900]] -}, -"customCSS": "./path/to/visual-test-styles.css" -} -``` - -### Embedded String Template - -```json -{ -"web": { -"browsers": ["chrome"], -"viewports": [[1440, 900]] -}, -"customCSS": "body{font-family:'Inter',sans-serif!important;} .ad,.banner{display:none!important;}" -} -``` - -## Additional Resources - -- [SmartUI CLI Documentation](/support/docs/smartui-cli/) -- [SmartUI Configuration Options](/support/docs/smartui-cli/) -- [Visual Regression Testing Guide](/support/docs/smart-visual-testing/) -- [Layout Comparison Documentation](/support/docs/smartui-layout-testing/) diff --git a/static/docs/smartui-customscroll.md b/static/docs/smartui-customscroll.md deleted file mode 100644 index 51e0b89dc..000000000 --- a/static/docs/smartui-customscroll.md +++ /dev/null @@ -1,213 +0,0 @@ -# Custom Viewport Capture in SmartUI Screenshots - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -In TestMu AI **SmartUI**, **custom viewport capture screenshots** preserve the scroll position that exists when you take a snapshot. This helps SmartUI capture the exact visual state of pages that use nested scroll containers, PDF viewers, document viewers, virtualized lists, infinite-scroll pages, data grids, modals, drawers, and embedded scrollable components. - -Use CustomScroll when the content you want to validate is not fully represented by the top-level page viewport. SmartUI records the active scroll state during snapshot capture, restores it before rendering the screenshot, and compares the resulting image against your baseline. - -## Where CustomScroll helps - -| Scenario | Recommended option | Why it helps | -|----------|--------------------|--------------| -| Main page, `body`, or `html` scrolls | `pageCustomScroll: true` | Captures the page-level scroll position before the screenshot is rendered. | -| Content is inside a PDF viewer, document viewer, data grid, modal, drawer, table, or scrollable `div` | `elementsCustomScroll: true` | Captures scroll positions for nested scrollable elements. | -| Both the page and an inner container can scroll | Enable both options | Preserves page-level and element-level scroll state together. | -| Static page with no scroll-dependent state | No CustomScroll option required | Standard snapshots are enough. | - -## Supported options - -CustomScroll is opt-in. Existing snapshots continue to behave the same unless you enable one or both options in the snapshot call. - -| Option | Type | Default | Use case | -|--------|------|---------|----------| -| `pageCustomScroll` | Boolean (`true`/`false`) | `false` | Use when the main page, `body`, or `html` document is the scroll container. | -| `elementsCustomScroll` | Boolean (`true`/`false`) | `false` | Use when important content is inside a nested scrollable element. | - -## Prerequisites - -- SmartUI CLI version `4.1.71` or later. -- A SmartUI project token configured in your test environment. -- A page state where the target content is rendered before the snapshot is taken. - -Install or update the SmartUI CLI in the repository where your tests run: - -```bash -npm install @lambdatest/smartui-cli@latest -``` - -You can verify the installed version with: - -```bash -npx smartui --version -``` - -## How CustomScroll works - -CustomScroll works in two steps: - -1. **Capture scroll state**: SmartUI records the active scroll position from the live browser session when the snapshot is triggered. -2. **Restore before screenshot**: SmartUI restores that scroll state during rendering, before the screenshot is captured. - -This ensures the screenshot reflects the page or component state your test actually reached. - -CustomScroll does not replace waits. For virtualized, lazy-loaded, or document-heavy pages, wait until the target page, row, or element is visible before taking the snapshot. - -## Java And Playwright Examples - -Use `pageCustomScroll` and `elementsCustomScroll` in the snapshot options when the page and nested content can both scroll. - -```java -Map options = new HashMap<>(); -options.put("pageCustomScroll", true); -options.put("elementsCustomScroll", true); - -SmartUISnapshot.smartuiSnapshot(driver, "PDF-Viewer-Page-25", options); -``` - -```javascript -await page.goto('https://vault.example.com/ui/#doc_info/2/0/1?anQS=page25'); - -await page.waitForSelector('.pageContent-scrollbar-content'); -await page.waitForTimeout(1500); - -await smartuiSnapshot(page, 'Vault-PDF-Page-25', { -elementsCustomScroll: true, -pageCustomScroll: false, -}); -``` - -In this example: - -- The test navigates to a specific document page. -- The viewer is allowed to render the target content. -- SmartUI captures the scroll position inside the document viewer. -- The screenshot reflects the intended document state. - -## Enable CustomScroll Only Where Needed - -You can keep standard screenshots unchanged and enable CustomScroll only for scroll-dependent states. - -```javascript -await smartuiSnapshot(page, 'Header'); - -await smartuiSnapshot(page, 'Doc-Viewer-Page-25', { -pageCustomScroll: true, -elementsCustomScroll: true, -}); - -await smartuiSnapshot(page, 'Footer'); -``` - -## Recommended Use Cases - -### PDF and Document Viewers - -Use CustomScroll when you need to capture a specific page or section inside a document viewer. - -Best for: - -- PDF viewers -- Vault-style viewers -- Contract viewers -- Compliance documents -- Reports -- Multi-page documents - -Recommended option: - -```javascript -{ -elementsCustomScroll: true -} -``` - -Use both options if the page and viewer can both scroll: - -```javascript -{ -pageCustomScroll: true, -elementsCustomScroll: true -} -``` - -### Virtualized tables and data grids - -Use CustomScroll when validating rows, columns, sticky headers, or mid-table states inside a scrollable grid. - -Best for: - -- MUI DataGrid -- React Virtualized -- React Window -- Enterprise data tables -- Admin dashboards -- Large reports - -Recommended option: - -```javascript -{ -elementsCustomScroll: true -} -``` - -### Modals, drawers, and side panels - -Use CustomScroll when important content is inside a modal, drawer, side panel, or embedded workflow. - -Best for: - -- Settings modals -- Audit log drawers -- User detail panels -- Configuration sidebars -- Long forms - -Recommended option: - -```javascript -{ -elementsCustomScroll: true -} -``` - -## Best practices - -- **Use CustomScroll only where needed**: Keep simple static screenshots unchanged and enable CustomScroll for scroll-dependent snapshots. -- **Wait for the UI to settle**: Wait for the target selector, row, document page, or modal content before taking the snapshot. -- **Use stable snapshot names**: Name snapshots based on the state being captured, such as `Vault-PDF-Page-25`, `DataGrid-Midpoint`, or `AuditLog-Drawer-Expanded`. -- **Match browser and viewport configuration**: Use the same browser and viewport setup you expect for baseline comparison. -- **Enable both options for complex pages**: If both the main page and an inner container can scroll, enable `pageCustomScroll` and `elementsCustomScroll` together. - -## Troubleshooting - -### The screenshot does not show the expected scrolled content - -- Confirm the target content is visible before calling `smartuiSnapshot`. -- Add a wait for the scrollable selector or target row/page. -- Use `elementsCustomScroll: true` for nested scrollable containers. -- Use `pageCustomScroll: true` if the main page itself is scrolled. - -### The PDF viewer or grid still captures the wrong state - -- Ensure the viewer has finished rendering after navigation or scrolling. -- Wait for a stable selector inside the viewer or grid. -- Use both CustomScroll options if the page and the viewer both scroll. - -### Do I need CustomScroll for every snapshot? - -No. CustomScroll is opt-in. Use it only for snapshots where scroll position affects the visual state. - -### Does CustomScroll work with virtualized lists? - -Yes. CustomScroll helps preserve the visible scroll state, but your test should still wait until the virtualized content has rendered before taking the snapshot. - -## Related documentation - -- [SmartUI Configuration Options](/support/docs/smartui-sdk-config-options/) -- [SmartUI CLI Documentation](/support/docs/smartui-cli/) -- [Handle Lazy Loading](/support/docs/smartui-handle-lazy-loading/) -- [Handle Sticky Elements](/support/docs/smartui-handle-sticky-elements/) -- [Custom CSS Injection in SmartUI](/support/docs/smartui-custom-css/) -- [PDF Comparison in SmartUI](/support/docs/smartui-pdf-comparison/) diff --git a/static/docs/smartui-cypress-sdk.md b/static/docs/smartui-cypress-sdk.md deleted file mode 100644 index 0a00c8e90..000000000 --- a/static/docs/smartui-cypress-sdk.md +++ /dev/null @@ -1,494 +0,0 @@ -# Integrate SmartUI SDK with Cypress Tests - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Welcome to the world of simplified visual testing with the SmartUI SDK. - -Integrating seamlessly into your existing Cypress testing suite, SmartUI SDK revolutionizes the way you approach visual regression testing. Our robust solution empowers you to effortlessly capture, compare, and analyze screenshots across a multitude of browsers and resolutions, ensuring comprehensive coverage and accuracy in your visual testing endeavors. - -## Prerequisites - -- Basic understanding of Command Line Interface and Cypress is required. -- Login to [TestMu AI SmartUI](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) with your credentials. - -The following steps will guide you in running your first Visual Regression test on TestMu AI platform using SmartUI Cypress SDK integration. - -## Create a SmartUI Project - -The first step is to create a project with the application in which we will combine all your builds run on the project. To create a SmartUI Project, follow these steps: - -1. Go to [Projects page](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) -2. Click on the `new project` button -3. Select the platform as **CLI** for executing your `SDK` tests. -4. Add name of the project, approvers for the changes found, tags for any filter or easy navigation. -5. Click on the **Submit**. - -## Steps to run your first test - -Once you have created a SmartUI Project, you can generate screenshots by running automation scripts. Follow the below steps to successfully generate screenshots - -### **Step 1:** Create/Update your test - -You can clone the sample repository to run `Cyoress` tests with `SmartUI` and use the `cypress/e2e/smartuiSDKLocal.cy.js` file. - -```bash -git clone https://github.com/LambdaTest/smartui-cypress-sdk-sample -cd smartui-cypress-sdk-sample -``` -### **Step 2**: Install the Dependencies - -1. Install required NPM modules for `TestMu AI SmartUI Cypress SDK` in your **Frontend** project. - -```bash -npm install @lambdatest/smartui-cli @lambdatest/cypress-driver cypress@v13 -``` - -If you face any problems executing tests with SmartUI-CLI `versions >= v4.x.x`, upgrade your Node.js version to `v20.3` or above. - -2. Import the TestMu AI Cypress driver in `support/e2e.js`. - -```js -import '@lambdatest/cypress-driver' -``` - -3. Update your config `(cypress.config.js)` or `plugins/index.js` with the following snippet - -```js -on('task', { -log(message) { -console.log(message); -return null; -}, -}); -``` - -SmartUI SDK only supports Cypress versions >= 10.0.0 - -### **Step 3:** Configure your Project Token - -Setup your project token shown in the **SmartUI** app after creating your project. - -```bash -export PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -``` - -```bash -set PROJECT_TOKEN=123456#1234abcd-****-****-****-************" -``` - -```powershell -$env:PROJECT_TOKEN=123456#1234abcd-****-****-****-************" -``` - -### **Step 4:** Create and Configure SmartUI Config - -You can now configure your project configurations on using various available options to run your tests with the SmartUI integration. To generate the configuration file, please execute the following command: - -```bash -npx smartui config:create .smartui.json -``` - -Once, the configuration file will be created, you will be seeing the default configuration pre-filled in the configuration file: - -```json title="/smartui-sdk-project/.smartui.json" -{ -"web": { -"browsers": [ -"chrome", -"firefox", -"safari", -"edge" -], -"viewports": [ -[ -1920 -], -[ -1366 -], -[ -1028 -] -] // Full Page screenshots are captured by default for web viewports -}, -"mobile": { -"devices": [ -"iPhone 14", //iPhone 14 viewport -"Galaxy S24" //Galaxy S24 viewport -], -"fullPage": true, //Full Page is true by default for mobile viewports -"orientation": "portrait" //Change to "landscape" for landscape snapshot -}, -"waitForTimeout": 1000, //Optional (Should only be used in case lazy-loading/async components are present) -"waitForPageRender": 50000, //Optional (Should only be used in case of websites which take more than 30s to load) -"enableJavaScript": false, //Enable javascript for all the screenshots of the project -"allowedHostnames": [] //Additional hostnames to capture assets from -} -``` -**Advanced options in SmartUI configuration** -- For capturing fullpage or viewport screenshots, please refer to this [documentation](/docs/smartui-sdk-config-options/#12-viewports) -- For the list of available mobile viewports, please refer to this [documentation](/docs/smartui-sdk-config-options/#list-of-supported-device-viewports) -- For more information about SmartUI config global options, please refer to this [documentation](/docs/smartui-sdk-config-options/#3-global-options-optional). - -### **Step 5:** Adding SmartUI function to take screenshot - -- You can incorporate SmartUI into your custom `Cypress` test script, as shown below: - -```js -/// - -describe('Test Case name', () => { -beforeEach(() => { - -cy.visit('Required URL') -}) - -it('SmartUI Snapshot', () => { -cy.smartuiSnapshot('Screenshot Name'); -}) -}) - -``` - -### **Step 6:** Execute the Tests on SmartUI Cloud - -Execute `visual regression tests` on SmartUI using the following commands - -```bash -npx smartui --config .smartui.json exec -- npx cypress run --spec cypress/e2e/smartuiSDKLocal.cy.js --browser chrome --headed -``` - -You may use the `npx smartui --help` command in case you are facing issues during the execution of SmartUI commands in the CLI. - -## View SmartUI Results - -You have successfully integrated SmartUI SDK with your Cypress tests. Visit your SmartUI project to view builds and compare snapshots between different test runs. - -You can see the SmartUI dashboard to view the results. This will help you identify the Mismatches from the existing `Baseline` build and do the required visual testing. - -## Arguments supported in the `cy.smartUISnapshot` function - -The following are the different options which are currently supported: - -| Key | Description | -| ------------------------- | ------------------------------------------------------------------------------------------------------------------------- | -| `t` (test controller) | This object allows you to control the test execution and interact with the web page under test. | -| `"Screenshot Name"` (string) | Specify a name for the screenshot in your tests to match the same screenshot with the name from your baseline. | -| `options` (object) | Specify one or a combination of selectors in the `ignoreDOM` or `selectDOM` objects. These selectors can be based on `HTML DOM IDs, CSS classes, CSS selectors, or XPaths` used by your webpage. They define elements that should be excluded from or included in the visual comparison.| - -## Handling Dynamic Data in SmartUI SDK **** - -When conducting visual tests, you may encounter scenarios where certain elements within your application change between test runs. These changes might introduce inconsistencies in your test results.You can ignore / select specific element(s) to be removed from the comparison by parsing the options in the `cy.smartuiSnapshot` function in the following way - -```js title="This is a sample for your configuration for Cypress to ignore by" ID" -let options = { -ignoreDOM: { -id: ["ID-1", "ID-2"], -} -} -cy.smartuiSnapshot(‘Screenshot Name’, options); -``` - -```js title="This is a sample for your configuration for Cypress to ignore by" Class" -let options = { -ignoreDOM: { -class: ["Class-1", "Class-2"], -} -} -cy.smartuiSnapshot(‘Screenshot Name’, options); -``` - -```js title="This is a sample for your configuration for Cypress to ignore by" XPath" -let options = { -ignoreDOM: { -xpath: ["Xpath-1", "Xpath-2"], -} -} -cy.smartuiSnapshot(‘Screenshot Name’, options); -``` - -```js title="This is a sample for your configuration for Cypress to ignore by CSS" Selector" -let options = { -ignoreDOM: { -cssSelector: ["CSS-Selector-1", "CSS-Selector-2"], -} -} -cy.smartuiSnapshot(‘Screenshot Name’, options); -``` - -```js title="This is a sample for your configuration for Cypress to select by" ID." -let options = { -selectDOM: { -id: ["ID-1", "ID-2"], -} -} -cy.smartuiSnapshot(‘Screenshot Name’, options); -``` - -```js title="This is a sample for your configuration for Cypress to select by" Class" -let options = { -selectDOM: { -class: ["Class-1", "Class-2"], -} -} -cy.smartuiSnapshot(‘Screenshot Name’, options); -``` - -```js title="This is a sample for your configuration for Cypress to select by" XPath" -let options = { -selectDOM: { -xpath: ["Xpath-1", "Xpath-2"], -} -} -cy.smartuiSnapshot(‘Screenshot Name’, options); -``` - -```js title="This is a sample for your webhook configuration for Cypress to select by CSS" Selector" -let options = { -selectDOM: { -cssSelector: ["CSS-Selector-1", "CSS-Selector-2"], -} -} -cy.smartuiSnapshot(‘Screenshot Name’, options); -``` - -## For capturing the screenshot of a specific element - -You can capture screenshots of targeted elements by leveraging various locator mechanisms such as XPath, CSS ID, class, and selectors. This precision-driven approach ensures accurate and specific visual regression testing for your web application's components. - -```js title="This is a sample for your configuration for Cypress to capture an element by" ID." -let options = { -element: { -id: 'Required ID', -} -}; -cy.smartuiSnapshot(‘Screenshot Name’, options); -``` - -```js title="This is a sample for your configuration for Cypress to capture an element by" Class" -let options = { -element: { -class: 'Required Class', -} -}; -cy.smartuiSnapshot(‘Screenshot Name’, options); -``` - -```js title="This is a sample for your configuration for Cypress to capture an element by" XPath" -let options = { -element: { -xpath: 'Required Xpath', -} -}; -cy.smartuiSnapshot(‘Screenshot Name’, options); - -``` - -```js title="This is a sample for your webhook configuration for Cypress to capture an element by CSS" Selector" -let options = { -element: { -cssSelector: 'Required CSS Selector', -} -}; -cy.smartuiSnapshot(‘Screenshot Name’, options); -``` - -## Best Practices - -**Screenshot Naming** - -- Use descriptive, consistent names for screenshots -- Include page/component name in screenshot names -- Avoid special characters that might cause issues -- Use consistent naming conventions across your test suite - -**Example:** -```javascript -cy.smartuiSnapshot('HomePage-Header'); -cy.smartuiSnapshot('ProductPage-MainContent'); -``` - -**Wait for Page Load** - -- Always wait for pages to fully load before taking screenshots -- Use Cypress's built-in wait methods for dynamic content -- Consider using `waitForTimeout` in configuration for lazy-loaded content - -**Example:** -```javascript -cy.visit('https://example.com'); -cy.get('#main-content').should('be.visible'); -cy.wait(1000); // Wait for animations -cy.smartuiSnapshot('Page Loaded'); -``` - -**Handle Dynamic Content** - -- Use `ignoreDOM` for elements that change between runs -- Use `selectDOM` when you only need to compare specific areas -- Document why elements are ignored for future reference - -**Configuration Management** - -- Keep `.smartui.json` in version control -- Use environment variables for sensitive data -- Document custom configuration choices - -**Test Organization** - -- Group related screenshots in the same build -- Use meaningful build names -- Run tests in consistent environments - -## Troubleshooting - -**Issue: Screenshots Not Appearing in Dashboard** - -**Symptoms**: Tests run successfully but no screenshots appear in SmartUI dashboard - -**Possible Causes**: -- Project token not set or incorrect -- Project name mismatch -- Network connectivity issues -- CLI not installed or outdated - -**Solutions**: -1. Verify `PROJECT_TOKEN` is set correctly: -```bash -echo $PROJECT_TOKEN -``` - -2. Check project name matches exactly (case-sensitive) - -3. Verify SmartUI CLI is installed: -```bash -npx smartui --version -``` - -4. Check network connectivity to TestMu AI servers - -5. Review test execution logs for error messages - -**Issue: Project Not Found" Error** - -**Symptoms**: Error message indicating project cannot be found - -**Possible Causes**: -- Incorrect project token -- Project deleted or renamed -- Token from wrong project - -**Solutions**: -1. Verify project exists in SmartUI dashboard -2. Copy project token directly from Project Settings -3. Ensure token includes the project ID prefix (e.g., `123456#...`) -4. Check for extra spaces or quotes in token - -**Issue: Screenshots Show Blank or Incorrect Content** - -**Symptoms**: Screenshots captured but show blank pages or incorrect content - -**Possible Causes**: -- Page not fully loaded -- JavaScript not executed -- Viewport size issues -- Timing issues - -**Solutions**: -1. Add explicit waits before screenshots: -```javascript -cy.visit('https://example.com'); -cy.get('#content').should('be.visible'); -cy.wait(2000); // Wait for content to load -cy.smartuiSnapshot('Page'); -``` - -2. Enable JavaScript in configuration: -```json -{ -enableJavaScript": true -} -``` - -3. Increase `waitForTimeout` in configuration - -4. Verify viewport size matches expected dimensions - -**Issue: Build Execution Fails** - -**Symptoms**: `npx smartui exec` command fails - -**Possible Causes**: -- Missing or incorrect configuration file -- Invalid JSON in configuration -- Port conflicts -- Permission issues - -**Solutions**: -1. Verify `.smartui.json` exists and is valid JSON -2. Check configuration file syntax -3. Try different port if default is in use: -```bash -npx smartui exec -P 5000 -- -``` -4. Check file permissions for configuration and project files - -**Issue: npm Dependencies Not Resolving** - -**Symptoms**: npm cannot find `@lambdatest/cypress-driver` or `@lambdatest/smartui-cli` - -**Possible Causes**: -- Incorrect package version -- npm registry access issues -- Network connectivity problems - -**Solutions**: -1. Check latest versions on npm: -```bash -npm view @lambdatest/cypress-driver version -npm view @lambdatest/smartui-cli version -``` - -2. Clear npm cache: -```bash -npm cache clean --force -``` - -3. Verify internet connectivity for npm registry access - -4. Check package.json for version conflicts - -**Issue: Screenshot Names Not Matching Baseline** - -**Symptoms**: Screenshots appear as New" instead of comparing with baseline - -**Possible Causes**: -- Screenshot name changed -- Baseline doesn't exist -- Name contains special characters - -**Solutions**: -1. Ensure screenshot names are consistent across test runs -2. Verify baseline exists in project -3. Avoid special characters in screenshot names -4. Check for case sensitivity issues - -**Getting Help** - -If you encounter issues not covered here: - -- Review the [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) for detailed solutions -- Check [SmartUI Configuration Options](/support/docs/smartui-sdk-config-options) documentation -- See [Handling Dynamic Data](/support/docs/smartui-handle-dynamic-data) for dynamic content issues -- Visit [TestMu AI Support](https://www.testmuai.com/support) for additional resources -- Contact support at support@testmuai.com or use [24/7 Chat Support](https://www.testmuai.com/support) - -## Additional Resources - -- [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) -- [SmartUI Configuration Options](/support/docs/smartui-sdk-config-options) -- [Handling Dynamic Data](/support/docs/smartui-handle-dynamic-data) -- [Handling Lazy Loading](/support/docs/smartui-handle-lazy-loading) -- [Baseline Management](/support/docs/smartui-baseline-management) -- [Running Your First Project](/support/docs/smartui-running-your-first-project) -- [SmartUI API Documentation](https://www.testmuai.com/support/api-doc/) diff --git a/static/docs/smartui-diff-highlighter.md b/static/docs/smartui-diff-highlighter.md deleted file mode 100644 index 6f94e6d68..000000000 --- a/static/docs/smartui-diff-highlighter.md +++ /dev/null @@ -1,83 +0,0 @@ -# Diff Highlighter & Navigation - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -When reviewing SmartUI visual comparisons, small differences can be easy to miss on dense or zoomed-out screens. The **Diff Highlighter** feature helps you **discover every change** quickly using in-view navigation and an optional highlight animation, so you spend less time scanning and more time deciding what to accept or reject. - -## What You Get - -| Feature | Description | -|--------|-------------| -| **Previous / Next** | Move focus to the previous or next diff region in order (top to bottom). | -| **Highlight** | Run an animation that traverses each diff from top to bottom with popping circles, making every change location obvious at a glance. | -| **Actions menu** | All controls live in the comparison view’s actions menu (e.g. 3-dot or actions dropdown). | - -No extra setup is required: if your screenshot uses **Smart Ignore** comparison, diff navigation and highlight are available as soon as you open the comparison. - -## How to Use Diff Navigation - -### Step 1: Open the comparison view - -1. In your [SmartUI dashboard](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/), open the **build** that contains the screenshot you want to review. -2. Open the **screenshot/comparison view** (e.g. by clicking the screenshot or “View issues”). - -### Step 2: Open the actions menu - -On the comparison view, open the **actions menu** (e.g. the 3-dot or actions dropdown). - -### Step 3: Use Previous, Highlight, and Next - -In the actions menu you’ll see: - -- **Previous**: moves focus to the **previous** diff (above the current one). -- **Highlight**: runs the **highlight animation**: popping circles traverse each diff from **top to bottom**, so you can see every change location in sequence. -- **Next**: moves focus to the **next** diff (below the current one). - -Use **Highlight** when you want to quickly see all diffs without clicking through. Use **Previous** and **Next** when you want to step through differences one by one. - -### Step 4: Review each difference - -- On clicking the **Previous** or **Next** buttons, the view will zoom into that specific diff and highlight it with a bounded box. - -- You can continue to use overlay toggles, zoom, slider/side-by-side, and **Mark as bug** or approval actions as usual. -- Diff navigation and highlight do **not** replace existing comparison tools, they only make it easier to find where the changes are. - -## Highlight animation - -When you click **Highlight**: - -- It will show circles around the areas where the differences are present. -- They **traverse each diff region from top to bottom** in sequence. -- This makes it easy to see how many changes there are and where they sit, without manually scanning the whole screenshot. - -## Comparison mode support - -| Comparison mode | Diff navigation & highlight | -|----------------|-----------------------------| -| **Smart Ignore** | Supported | -| **Strict** | Not yet supported | -| **Layout** | Not yet supported | - -Diff navigation and highlight use **precomputed diff regions** from the comparison. Today these are available for **Smart Ignore** only. Support for **Strict** and **Layout** modes is planned for a future release. - -To use diff navigation and highlight, ensure the screenshot is compared with **Smart Ignore**. You can enable Smart Ignore in [Project Settings](/support/docs/smartui-project-settings) or choose it per screenshot from the **Diff options** dropdown on the comparison page. - -## Tips - -- **Use Highlight first** on a new comparison to see how many diffs there are and where they are, then use **Previous/Next** to inspect each one. -- **Zoom and overlays** still work as before; combine them with diff navigation for a faster review. -- **Commenting and Mark as bug** are unchanged; use them as usual while stepping through diffs. - -## Troubleshooting - -| Issue | What to check | -|-------|----------------| -| **No Previous/Next/Highlight in the menu** | Confirm the screenshot is using **Smart Ignore** comparison. These controls are not shown for Strict or Layout mode yet. | -| **Highlight doesn’t show anything** | Ensure there is at least one diff; if the comparison is identical, there’s nothing to highlight. | -| **Order of diffs** | Diffs are always traversed **top to bottom**; Previous goes up, Next goes down. | - -## Related docs - -- [Smart Ignore](/support/docs/smartui-smartignore): Reduce noise and use diff regions for navigation. -- [Layout Testing](/support/docs/smartui-layout-testing): Compare layout structure (diff navigation for this mode coming later). -- [View comparison and issues](/support/docs/smart-visual-testing#view-comparison-and-issues): Basics of the comparison view. diff --git a/static/docs/smartui-draw-on-ui.md b/static/docs/smartui-draw-on-ui.md deleted file mode 100644 index 917ffc8e1..000000000 --- a/static/docs/smartui-draw-on-ui.md +++ /dev/null @@ -1,373 +0,0 @@ -# Ignore or Select Annotated Regions - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Web applications often have dynamic elements that can cause unnecessary noise in your visual testing. Take a social media platform, for instance. The number of unread notifications displayed might change with each test run. While these variations are expected, you don't necessarily want them to trigger alerts as potential regressions. - -The SmartUI Annotation tool allows you to interact directly with your screenshots through detailed annotations. You can draw over screenshots, define regions with boxes, and choose to ignore or select these regions for current and future comparisons. With advanced features like **Ignore Colors**, **Floating Regions**, and **Select Ignore**, you can handle even the most complex dynamic content scenarios. - -By utilizing ignored/selected regions, you can keep your test results focused on the truly important changes, streamlining your workflow and saving you time from chasing irrelevant discrepancies. - -## Quick Start - -### Accessing the Annotation Tool - -1. Navigate to your SmartUI project dashboard -2. Open a screenshot comparison that shows differences -3. Click on the **Actions** button (annotation icon) in the comparison view - -### Basic Workflow - -1. **Click the annotation icon** to open the annotation tool -2. **Click "Add Region"** and draw a box around the area you want to annotate -3. **Select the annotation type** (Ignore Region, Select Region, Floating Region, or Ignore Colors) -4. **Click "Save"** to apply the annotation -5. **Choose application scope**: Apply to current screenshot only or all browser variants - -> **Tip:** Start with **Ignore Region** for simple dynamic content like timestamps or notification counters. It's the most straightforward annotation type. - -## Annotation Methods - -All four annotation methods are accessible from the same **Actions** button (annotation icon). Click on the annotation icon to open the annotation tool, then select your desired annotation type from the available options. - -**What is Ignore Region?** - -**Ignore Region** excludes specific areas from visual comparison. Ideal for dynamic content that changes frequently but isn't critical to your visual tests, such as timestamps, notification badges, or advertisements. - -**When to Use** - -- Ignoring notification counters that change frequently -- Excluding advertisement banners -- Ignoring timestamps or date displays -- Hiding dynamic user-generated content -- Excluding non-critical UI elements that vary between builds - -**How to Use** - -**Step 1:** Click on the **Actions** button (annotation icon) to open the annotation tool. - -**Step 2:** Click on the **Add Region** button and draw a box around the area you want to ignore. - -**Step 3:** Select **Ignore Region** from the annotation type dropdown and click **Save**. - -**Step 4:** Choose whether to apply to the current screenshot only or all browser variants. - -**What Happens:** The configured area will be ignored in every consecutive build for the same screenshot test case. - -**Example** - -Ignoring a "Last updated" timestamp in a dashboard that changes with each page load. - -**What is Select Region?** - -**Select Region** focuses visual comparison on specific areas of your screenshot. Only the selected regions will be compared, while everything else is automatically ignored. This is useful when you want to test only critical UI components. - -**When to Use** - -- Testing only the main content area -- Focusing on specific form fields -- Comparing only critical UI components -- Isolating header or footer sections -- Testing specific widgets or components in isolation - -**How to Use** - -**Step 1:** Click on the **Actions** button (annotation icon) to open the annotation tool. - -**Step 2:** Click on the **Add Region** button and draw a box around the area you want to select for comparison. - -**Step 3:** Select **Select Region** from the annotation type dropdown and click **Save**. - -**Step 4:** Choose whether to apply to the current screenshot only or all browser variants. - -**What Happens:** Only the selected regions will be compared in future builds, while all other areas are automatically ignored. - -> **Note:** You can define multiple select regions. All selected regions will be compared, and everything outside these regions will be ignored. - -**Example** - -Testing only the checkout form while ignoring the rest of the page, including header, footer, and sidebar elements. - -**What is Floating Region?** - -**Floating Region** handles elements that move or shift position within a defined boundary. This feature uses two regions: -- **Blue Region:** Defines the boundary where the element might move to (the floating area) -- **Yellow Region:** Defines which element to test within the floating region - -Perfect for handling dynamic elements that may appear in different positions but should be tested for content consistency. - -**When to Use** - -- Testing notification badges that may appear in different positions -- Handling tooltips or popovers that can move -- Testing floating action buttons -- Comparing elements that shift based on content length -- Handling dropdown menus or modals that may appear in different locations - -**How to Use** - -**Step 1:** Click on the **Actions** button (annotation icon) to open the annotation tool. - -**Step 2:** First, draw a **Blue Region** (Floating Area) that defines the boundary where the element might move to. Make it large enough to encompass all possible positions. - -**Step 3:** Then, within the blue region, draw a **Yellow Region** (Element to Test) that defines the specific element you want to test. - -**Step 4:** Select **Floating Region** from the annotation type dropdown and click **Save**. - -**Step 5:** Choose whether to apply to the current screenshot only or all browser variants. - -**What Happens:** SmartUI will compare the yellow region element, allowing it to move anywhere within the blue floating area without triggering false positives. - -> **Tip:** Make the blue region large enough to encompass all possible positions of the element, and keep the yellow region focused on the specific element you want to test. - -**Example** - -Testing a notification badge that may appear in different positions based on content length or screen size. - -**What is Ignore Colors?** - -**Ignore Colors** ignores color differences in specific regions while still comparing the structure and layout. Particularly useful when colors may vary due to themes, user preferences, or A/B testing, but the structure should remain consistent. - -**When to Use** - -- Ignoring color changes in themed applications -- Handling A/B testing color variations -- Ignoring user-customizable color schemes -- Testing structure while ignoring color differences in specific regions -- Comparing layouts across different theme configurations - -**How to Use** - -**Step 1:** Click on the **Actions** button (annotation icon) to open the annotation tool. - -**Step 2:** Click on the **Add Region** button and draw a box around the area where you want to ignore color differences. - -**Step 3:** Select **Ignore Colors** from the annotation type dropdown and click **Save**. - -**Step 4:** Choose whether to apply to the current screenshot only or all browser variants. - -**What Happens:** The region will be compared for structure and layout, but color differences will be ignored, reducing false positives from color variations. - -> **Note:** Ignore Colors is particularly effective when combined with other annotation types for comprehensive visual testing. - -**Example** - -Ignoring color differences in a themed navigation bar while testing its structure and layout across different theme configurations. - -## Advanced: Select Ignore - -**Select Ignore** is an advanced feature that combines selection and ignoring. You first select a region to focus on, then within that selected region, you can ignore specific sub-regions. This is perfect for scenarios where you want to test most of a component but ignore certain dynamic elements within it. - -### When to Use - -- Testing a product card but ignoring the price (which may change) -- Comparing a user profile section but ignoring the avatar (which may be different) -- Testing a dashboard widget but ignoring specific metrics that change frequently -- Focusing on a form while ignoring dynamic validation messages -- Testing a navigation menu but ignoring the active state indicator - -### How to Use - -**Step 1:** Click on the **Actions** button (annotation icon) to open the annotation tool. - -**Step 2:** First, draw a **Select Region** box around the main area you want to test. - -**Step 3:** Then, within that selected region, draw an **Ignore Region** box around the sub-area you want to ignore. - -**Step 4:** Select **Select Ignore** from the annotation type dropdown and click **Save**. - -**Step 5:** Choose whether to apply to the current screenshot only or all browser variants. - -**What Happens:** The outer selected region will be compared, but the inner ignored sub-region will be excluded from comparison, giving you precise control over what gets tested. - -> **Tip:** Use Select Ignore when you need granular control over comparison areas within a larger component. Ensure the ignore region is completely within the select region. - -### Example - -Testing a product card but ignoring the price and rating that may change, while still comparing the product image, title, and description. - -## Baseline Regions - -A **baseline** is the reference image that every later build is compared against. Normally you draw regions on a comparison build, so the region belongs to that build's comparison. **Baseline Regions** let you draw an ignore or select region directly on the **#1 Baseline Build**, so the region is owned by the baseline itself and is honored for comparisons from the baseline forward. - -This is useful when you already know an area is dynamic at the moment you establish the baseline. Instead of waiting for a comparison build and re-applying the region there, you can annotate the baseline once and have it apply to every build that uses that image as its baseline. - -### When to Use - -- Marking a known dynamic area (a timestamp, a session banner, a live counter) as ignored right when the baseline is created -- Keeping the baseline and what is actually compared in sync, without waiting for the next comparison build -- Establishing a select region on the baseline so every downstream build compares only the area you care about - -### How to Use - -**Step 1:** Open the **#1 Baseline Build** for your screenshot in the SmartUI dashboard. - -**Step 2:** Click on the **Actions** button (annotation icon) to open the annotation tool. - -**Step 3:** Click on the **Add Region** button and draw a box around the area you want to ignore or select, then choose the annotation type (for example, **Ignore Region** or **Select Region**). - -**Step 4:** Mark the region as a **baseline region** in the apply dialog and click **Save**. - -**What Happens:** The region is stored against the baseline screenshot and is applied in every consecutive build that is compared against that baseline. Regions that are not marked as baseline regions continue to behave exactly as before and apply only from the build on which you drew them. - -> **Note:** Baseline Regions and the [per-region **Apply to all variants**](/support/docs/smartui-draw-on-ui/#applying-annotations) scope are independent. A baseline region controls the builds a region applies to (from the baseline forward), while the variant scope controls the browser and viewport combinations a region is copied to. - -> **Tip:** Use a baseline region for content you already know is dynamic before the first comparison ever runs. For areas you discover later while reviewing a comparison, a normal region on that build is the simpler choice. - -## Managing Annotations - -Once you've created annotations, you can view, edit, and delete them as needed. - -### Viewing Annotations - -**Step 1:** Toggle the **Annotations** switch to view/analyze your ignored/selected areas. - -**Step 2:** When the toggle is on, you'll see all your annotation boxes with color coding (see [Annotation Color Coding](#annotation-color-coding) below). - -**Step 3:** Hover over any annotation box to see a tooltip with detailed information, including: -- Box dimensions (width × height) -- Annotation type (Ignore, Select, Floating, etc.) -- Coordinates - -### Editing Annotations - -You can always edit or delete pre-configured areas or add new ones according to your requirements. - -**To Edit an Annotation:** - -**Step 1:** Click on the **Actions** button (annotation icon). - -**Step 2:** Click on the annotation box you want to edit. - -**Step 3:** Resize or reposition the box as needed. - -**Step 4:** Click **Save** to apply changes. - -### Deleting Annotations - -**Step 1:** Click on the **Actions** button (annotation icon). - -**Step 2:** Click on the annotation box you want to delete, or click **Delete All** to remove all annotations. - -**Step 3:** Click on the **Save** button to confirm deletion. - -> **Note:** Deleting annotations will trigger a re-run of the comparison, and the previously ignored/selected areas will be included in future comparisons. - -### Applying Annotations - -After drawing a region, you'll see a modal dialog that controls where that specific region is applied: - -**Options:** - -1. **Apply to Current Screenshot Only:** The region will only affect the current screenshot variant (for example, Chrome on desktop). - -2. **Apply to All Browser Variants:** The region will be applied to every browser and viewport combination for this screenshot (for example, Chrome, Firefox, and Safari on desktop). - -Each region has its own scope, so **Apply to all variants** is a per-region action. Every region you draw is tracked with a stable identity of its own, which means the scope you pick applies to that one region and to no others. - -> **Note:** **Apply to all variants** propagates only the region you currently have selected or have just drawn. Any other annotations that already exist on the screenshot are left untouched and are not copied to the other variants unless you select them and apply them too. This keeps regions you never intended to share from spreading across the variant matrix. - -To propagate more than one region, select each region and choose **Apply to all variants** for it in turn. - -> **Best Practice:** While you can apply a region to all browser variants, it's recommended to manually verify that the area is configured correctly for each browser. Certain browsers may render elements differently, which can cause inconsistencies in your regions. - -### Annotation Color Coding - -When viewing annotations, different colors indicate their type: -- **Red boxes:** Ignore regions -- **Green boxes:** Select regions -- **Blue boxes:** Floating area boundaries -- **Yellow boxes:** Elements within floating regions -- **Purple boxes:** Ignore colors regions - -## Keyboard Shortcuts - -Speed up your annotation workflow with these keyboard shortcuts: - -- `Shift + A:` Resize annotation box left -- `Shift + S:` Resize annotation box down -- `Shift + D:` Resize annotation box right -- `Shift + W:` Resize annotation box up -- `Shift + T:` Define a region encompassing the entire screenshot -- `Delete or Backspace:` Remove the currently selected annotation - -> **Tip:** Use keyboard shortcuts for precise control when resizing annotation boxes, especially for small or complex regions. - -## Best Practices - -Follow these best practices to get the most out of the annotation tool: - -### Choosing the Right Annotation Type - -- **Start with Ignore Region** for simple dynamic content (timestamps, counters, ads) -- **Use Select Region** when you only care about specific UI components -- **Use Floating Region** for elements that move within a boundary -- **Use Ignore Colors** when structure matters more than color variations -- **Use Select Ignore** for granular control within larger components - -### General Guidelines - -- **Verify across browsers:** Test annotations on different browsers to ensure consistency -- **Document your decisions:** Note why you ignored/selected certain regions for future reference -- **Review periodically:** Regularly review annotations to ensure they're still relevant -- **Start simple:** Begin with basic Ignore Region before moving to advanced features -- **Test thoroughly:** Verify annotations work correctly before applying to all browser variants - -### Workflow Tips - -- Create annotations after reviewing initial comparison results -- Use the annotation toggle to hide/show annotations for a cleaner view -- Leverage keyboard shortcuts for faster annotation creation -- Apply to current screenshot first, then verify before applying to all variants -- Combine multiple annotation types for comprehensive coverage - -## Troubleshooting - -**Annotations Not Applying** - -**Issue:** Annotations are not being applied to future builds. - -**Solutions:** -- Ensure you clicked **Save** after creating annotations -- Verify you selected the correct annotation type -- Check that you applied annotations to the correct screenshot variant -- Refresh the page and verify annotations are visible - -**Floating Region Not Working** - -**Issue:** Floating region is still showing false positives. - -**Solutions:** -- Ensure the blue region (floating area) is large enough to encompass all possible positions -- Verify the yellow region (element to test) is correctly positioned within the blue region -- Check that the element structure hasn't changed significantly -- Try adjusting the size of both regions - -**Ignore Colors Still Showing Differences** - -**Issue:** Color differences are still being detected in ignore colors regions. - -**Solutions:** -- Verify you selected "Ignore Colors" as the annotation type -- Ensure the region covers the entire area where colors may vary -- Check that structural differences aren't being mistaken for color differences -- Review the comparison settings in project configuration - -**Annotations Disappearing** - -**Issue:** Annotations are not visible or have disappeared. - -**Solutions:** -- Toggle the **Annotations** switch to ensure it's enabled -- Refresh the page -- Check if annotations were deleted accidentally -- Verify you're viewing the correct screenshot variant - -For more comprehensive troubleshooting, refer to the [SmartUI Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide). - -## Additional Resources - -- [SmartUI Project Settings](/support/docs/smartui-project-settings) - Configure comparison settings -- [Handling Dynamic Data](/support/docs/smartui-handle-dynamic-data) - Code-based solutions for dynamic content -- [Smart Ignore](/support/docs/smartui-smartignore) - Automatic layout shift handling -- [Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) - Comprehensive issue resolution diff --git a/static/docs/smartui-export-build-data.md b/static/docs/smartui-export-build-data.md deleted file mode 100644 index 72fddf2ab..000000000 --- a/static/docs/smartui-export-build-data.md +++ /dev/null @@ -1,43 +0,0 @@ -# Export SmartUI Build Data - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -From the TestMu AI dashboard you can export **build-level** data for a SmartUI run (comparison outcomes, statuses, and related metadata) without using the CLI or API. Use it for stakeholder summaries, ticket attachments, spreadsheets, or downstream automation. - -## Prerequisites - -1. Open **SmartUI** and select the **project** that contains your build. -2. Open the **build** you want to export (the completed run you need a snapshot of). - -## Export from the build view - -### Step 1: Open Export from the build menu - -With the build page open, select the **three-dot menu** (⋮) in the **upper-right** corner, then choose **Export**. - -### Step 2: Wait for export data to load - -The product gathers build data for export. Stay on the view until loading **finishes** (spinner or progress clears). Large builds can take longer; avoid closing the tab during this step. - -### Step 3: Choose a format and download - -When loading completes, select **PDF**, **CSV**, or **JSON**. Your browser will download the file (exact name depends on the product version). Use your browser’s download bar or downloads folder if the save dialog does not appear. - -If a download does not start, check pop-up or download permissions for your TestMu AI domain, then try the same format again after a short wait. - -## Choose a format - -| Format | Best for | -|--------|----------| -| **PDF** | Readable summaries for reviews, email, or release records. | -| **CSV** | Sorting, filtering, and charts in Excel, Google Sheets, or BI tools. | -| **JSON** | Scripts, custom dashboards, or merging with CI metadata. | - -Column and field names in CSV/JSON follow the current SmartUI release; if you automate parsing, pin expectations to a tested export sample after upgrades. - -## Related docs - -- [Running Your First SmartUI Project](/support/docs/smartui-running-your-first-project/) -- [Multiselect & Bulk Operations](/support/docs/smartui-multiselect-bulkops/) -- [Audit & Activity Logs](/support/docs/smartui-audit-logs/) -- [Fetching Results through SmartUI SDK](/support/docs/smartui-sdk-fetch-results/) diff --git a/static/docs/smartui-freeze-animations.md b/static/docs/smartui-freeze-animations.md deleted file mode 100644 index 39d9acc7a..000000000 --- a/static/docs/smartui-freeze-animations.md +++ /dev/null @@ -1,27 +0,0 @@ -# Capture Screenshots by Freezing Animations - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Animations, while visually appealing, can pose challenges during visual testing by causing false positives in visual diffs. These false positives often arise because animations can create minor variations between screenshots, even when there are no actual changes to the content. To address this, TestMu AI SmartUI provides the capability to automatically freeze animations during visual testing, ensuring consistent and reliable test results. - -## Freezing SVG Animations -Consider an SVG element on your page with a loading animation that spins using the transform attribute. To freeze this animation, you can add the following custom JavaScript to your page before taking a screenshot: - -```javascript -const freezeAnimations = () => { -// Freeze all SVGs animated using animate and animateTransform tag -// SVGs animated through CSS or JS will require custom solutions -const allSVGs = document.getElementsByTagName('svg'); -let allSVGAnimations = []; -for (let svg of allSVGs) { -const svgAnimation = [...svg.getElementsByTagName('animate'), ...svg.getElementsByTagName('animateTransform')]; -allSVGAnimations = allSVGAnimations.concat(svgAnimation); -} - -allSVGAnimations.forEach(animation => { -const duration = animation.getAttribute('dur'); -animation.setAttribute('begin', '0s'); -animation.setAttribute('dur', '0s'); -}); -} -``` diff --git a/static/docs/smartui-git-branching-strategy.md b/static/docs/smartui-git-branching-strategy.md deleted file mode 100644 index 356157bb2..000000000 --- a/static/docs/smartui-git-branching-strategy.md +++ /dev/null @@ -1,205 +0,0 @@ -# Git Branching Strategy in SmartUI Storybook Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This is the guide to understand our Git Branching with SmartUI projects which can detect the commit history and execute the appropriate actions for your visual regression testing. - -### Prerequisites for running Git with SmartUI - -- You have an account with [TestMu AI](https://www.testmuai.com/login/). -- You must have an active [subscription](https://www.testmuai.com/pricing) plan with valid screenshots limit. -- You must have created an Project on the SmartUI web app. - -**Warning** -This feature is currently supported for only [**StoryBook**](/support/docs/smart-ui-storybook/) projects on the SmartUI platform. Please contact support@testmuai.com for further information or requests. - -The following steps will guide you in running your Git branching Visual Regression test on TestMu AI SmartUI platform: - -**Baseline Branch**: A baseline branch build is a set of screenshots which are captured using SmartUI where all the screenshots are compared against for the non-baseline branch builds. - -**Non-Baseline Branches**: A non-baseline branch(s) build is a set of screenshots which are captured using SmartUI where all the screenshots are compared against with the **Baseline** build which are matched with the screenshot names. - -### **Step 1**: Setup your Git Baseline branch in the Project Settings - -The following are the steps to add **Baseline** branch to your `StoryBook projects` with Git commits: - -1. Go to **Project Settings** page for the created project. -2. Search for **Git Settings** and add your required **Baseline** branch in the input box. -3. Click on **Update Settings** button to update the project settings and set your **Baseline** Git branch. - -Now, after the successful setup of your **Baseline** branch of your project. - -### **Step 2**: Execute your SmartUI CLI with Git commits - -The following are the steps to execute your SmartUI CLI with Git commits on StoryBook projects: - -1. Go to your current **StoryBook** repository and run the following command: - -```bash -git init --y -``` - -You can ignore the above step if you already have a repository with `Git` initialized - -2. Now, make your changes in your `.stories` files and then commit the changes to your `Git`: - -```bash -git commit -m "Your commit message" -``` - -3. Execute your **SmartUI StoryBook CLI** command to execute the Visual Regression tests for your `.stories` files: - -```bash -npm run storybook // Starts your local StoryBook server -smartui storybook http://localhost:6006 --config .smartui.json // Captures all the stories running on local server -``` - -```bash -npm run build-storybook // Creates a Static Build Folder of StoryBook Stories -smartui storybook ./storybook-static --config .smartui.json // Captures all the stories added in the static build folder -``` - -## Git Branching Workflows and Strategies - -### Ability to set the **Baseline Branch** - -You can set any `branch` name as your **Baseline** branch in the SmartUI project settings page. You can follow the steps below to set your baseline branch: - -1. Go to the SmartUI Web App. -2. Select the project in which you want to configure the `Git` baseline branch for. -3. Go to the project settings section by clicking on the top right icon on the `Builds` list page. -4. In the project settings section you can provide the input of the `branch` name. -5. Click on the `Update Settings` button to update the settings. - -**Default Selection of Baseline Branch** - -In case, if there is no branch is set in your new or existing project created. Then, the `repository's` where SmartUI CLI command is executed to run the tests, **HEAD Branch** will be updated as the **Baseline** branch for that project by default. - -### Displaying the Git Commit Meta data - -If you are using the `Git` based workflow with `SmartUI StoryBook CLI` then the following information is displayed on the UI of the SmartUI Web App - Builds list page: - -- **Branch Name** is set as the `Build Name` in the SmartUI project. -- The latest **commit ID**of the Git commit added to the current repository. -- The **commit message** of the Git commit of the `commit ID` to the current repository. -- The commit author **username** of the Git commit to the current repository. - -The above metadata is displayed only for StoryBook **Git** repositories. - -### Git Baseline Branch Selection and Comparison Strategy: - -### Working on the same branch name - -In this workflow, if you are working on the same branch name and want to execute the `SmartUI StoryBook CLI` command to run the Visual Regression tests for your `.stories`. - -And, if you have created a project and set your **Baseline** branch name to `master` then executed the following commands: - -```bash -// Your current branch name -$ git branch -* -<--Other branches--> - -// Adds a new commit to your current active branch -$ git commit -m "Your commit message" - -// Executing the SmartUI StoryBook tests -$ smartui storybook --config .smartui.json -``` - -Then, a new build will be created with your `Current Active Branch` on the SmartUI Web App - Build Lists page for the project in which the `SmartUI CLI` command is executed. - -If the current Baseline build branch is set to `Current Active Branch` in the SmartUI project settings, then SmartUI will compare the screenshots captured to the latest `APPROVED` status build of the `Current Active Branch` branch available in the same project. - -Here is an example of the above workflow: - -```bash -$ git branch -* master - -$ git commit -m "Second Build Changes" - -$ smartui storybook http://localhost:6006 --config .smartui.json -``` - -### Updating the Baseline Branch Name - -In this workflow, if you have created a SmartUI project on the Web App and have executed few builds with a `Branch Name: Master` and want to switch your **Baseline** branch to `development` branch in your `Git`, then following will be the workflow that will help you in switching the **Baseline Branch** for your project: - -1. You have to go to the `SmartUI Project Settings` page to update your **Baseline Branch** in the project settings from `master` to `development`. -2. Make changes in your `.stories` folder files in your repository. -3. Now, execute the `SmartUI StoryBook` CLI command to run the Visual Regression tests. -4. Then, SmartUI algorithm will check for existing latest `build` with `development` run on the SmartUI project in Builds history. - -The following are the cases that SmartUI will update the **Baseline** build for the branch set in your SmartUI project settings: - -##### **Case 1**: If a build with development branch exists in build history - -In this case, the latest build run on the SmartUI project build history will be updated as a **New Baseline Build** in which the new **Non-Baseline** builds will be compared against. - -##### **Case 2**: If a build with development branch does not exist in build history - -In this case, the new build run with the `development` git branch name will be set to the current **Baseline** build for the SmartUI project. - -**Note** -For the first build generated for the **Baseline branch build** in the SmartUI Projects will be automatically set to **SYSTEM APPROVED** status by default. - -### Detect changes for Git commit added to **Baseline Branch** - -In this workflow, if I make changes in the set Baseline branch in the SmartUI project that is master and make an commit to the Git then, - -Execute `SmartUI Storybook CLI` commands to execute the `.stories` tests will compare the results with the latest **APPROVED** build run for **Baseline Branch** build in the SmartUI project. - -On approval of the all the screenshots then the new build will be updated as Baseline build for the baseline branch. - -Here is an example of the above workflow: - -```bash -// Step 1 - Commit changes to Git -$ git commit -am "Changes in login stories" -(git) Changes added to your branch - -// Step 2 - Execute SmartUI CLI to trigger Build in the SmartUI project -$ smartui storybook http://localhost:6006 --config .smatui.json -``` - -### Comparing Non-Baseline Branch builds with **Baseline** branch builds - -In this workflow, if you want to compare a **non-baseline branch build** to a **baseline build** the following are steps to be followed: - -Step 1: Checkout to a non-baseline branch in your `Git` repository. - -Step 2: Then run the SmartUI CLI command to execute the tests should compare the screenshots captured of **Non-Baseline** branch build with the **Baseline branch** latest **APPROVED** build in the SmartUI project. - -Here is an example of the above workflow: - -```bash -// Baseline Branch -$ git branch -* master - -// Non-Baseline branch -$ git checkout -b develop -* develop -master - -// Executing SmartUI Build with Non-Baseline Git branch -$ smartui storybook http://localhost:6006 --config .smartui.json -``` - -### Auto Updating **Baseline** build for the **Baseline** branch - -In this workflow, once all the `Changes Found` are approved by the `approver` for the screenshots in the **Baseline** branch's **Non-Baseline** build then the `latest` approved **Baseline Branch** build will be updated to the `Baseline`. - -In case if the **Baseline** branch does not have an existing build then by default the new build run will be set as Baseline build. - -### Running Force Builds - -In this workflow, If you have not added any commit to your current `Git` in the repository after making the changes then SmartUI will consider that no changes are made to your `.stories` files, then you will need to use the following command to execute the build: - -```bash -$ smartui storybook --config .smartui.json --force-rebuild -``` - -**Warning** -In the above workflow, if there are no changes which are committed to your `Git` repository and if a build with the same commit number already exists in the build history of the SmartUI project. diff --git a/static/docs/smartui-github-app-integration.md b/static/docs/smartui-github-app-integration.md deleted file mode 100644 index b72e93efb..000000000 --- a/static/docs/smartui-github-app-integration.md +++ /dev/null @@ -1,120 +0,0 @@ -# GitHub App Integration with SmartUI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This is the guide to setup your GitHub Repos with SmartUI projects and run your CI along with visual regression testing. - -## Prerequisites - -- An account with GitHub with valid permission to install new applications to your repositories. -- Basic understanding of Continuous Integration tools (CI) is required. -- Should have setup the SmartUI suite, else please read [this](/docs/selenium-visual-regression/) - -The following steps will guide you in running your first Visual Regression test on TestMu AI SmartUI platform using GitHub App- - -## Step 1: Integrate the your TestMu AI Account with GitHub App - -The following are the steps to integrate your account with GitHub App: - -1. Go to [Integrations page](https://integrations.lambdatest.com/) -2. Search for GitHub App and select the integration. - -3. Click on the `OAuth` as your preferred authentication. -4. Click on **Install**. - -5. You will be redirected to the GitHub Authentication page to confirm the permissions required for the application to be installed. -6. Click on **Confirm** button to all the authentication requirements from your GitHub Account. -7. After successful authentication, you can refresh the [Integrations page](https://integrations.lambdatest.com/) to view the GitHub App installed. - -Now, after the successful installation of the GitHub Integration, please follow the below steps to configure to your project repo: - -## Step 2: Select your GitHub repository - -Go to your GitHub repository where you want to configure your SmartUI project. - -**GitHub Sample** - -Please check out GitHub sample here: https://github.com/LambdaTest/smartui-node-sample - -## Step 3: Configure your test suite - - Add the `GitHub` capability to your current test configuration: - -```js -const capabilities: { -platform: "Windows 10", -browserName: "chrome", -version: "latest", -"smartUI.project": "SmartUI sample test", -// highlight-start -github: { - -"url": process.env.GITHUB_URL // Mandatory -//GitHub URL format-https://api.github.com/repos/OWNER/REPO/statuses/commitId - -} -// highlight-end -} -``` - -## Step 4: Setting up your CI configuration - -Setting up your **CI** workflow to execute on **GitHub**. Here is an example setup with `GitHub Actions`: - -Go to `.github/workflows/.yml` - -```yaml -name: Execute SmartUI Test with GitHub App Integration -runs-on: ubuntu-latest -steps: -- uses: actions/checkout@v1 -with: -fetch-depth: 10 - -- name: Step for push event -run: | -echo "This is a push event!" -echo "The latest commitId $(git log -1 --format='%H')" -echo "COMMIT_ID=$(git log -1 --format='%H')" >> $GITHUB_ENV -if: github.event_name == 'push' - -- name: Step for pull_request event -run: | -echo "This is a pull_request event!" -git log -n 5 --format="%H %an %s" | while read line; do echo "$line"; done -echo "The latest commitId $(git log -n 2 --format='%H' | tail -n 1)" -echo "COMMIT_ID=$(git log -n 2 --format='%H' | tail -n 1)" >> $GITHUB_ENV -if: github.event_name == 'pull_request' - -- name: Create GitHub URL -run: | -API_HOST=https://api.github.com -echo "The latest commitId is $COMMIT_ID" -GITHUB_URL=$API_HOST/repos/$GITHUB_REPOSITORY/statuses/$COMMIT_ID -echo "GITHUB_URL: $GITHUB_URL" -echo "GITHUB_URL=$GITHUB_URL" >> $GITHUB_ENV -``` - -**Note** - -We also support other Continuous Integrations (CI) tools to execute the similar process as well. - -## Step 5: Execute your test suite with CI - -After the setup is completed, you can now execute your test suite with the Continuous Integration (CI) pipeline with any tool of your choice. - -**GitHub Actions - Example** - -If your current pipeline is set to `GitHub Actions` then you can configure your `workflow` to trigger the test suite by committing the changes to the `GitHub Repo` or on raising a `Pull Request` to merge any changes to the branch as per your branch rules. - -## GitHub App Feedback State - -Here is an example of **Successful** feedback app state: - -Here is an example of **Failed** feedback app state: - -**Note** - -You can click on `Details` link which will redirect the user to the build of the SmartUI test to view the results - -To understand more about the GitHub Actions, setup please read this article : https://docs.github.com/en/actions/quickstart diff --git a/static/docs/smartui-gitlab-pr-checks-exec.md b/static/docs/smartui-gitlab-pr-checks-exec.md deleted file mode 100644 index 195a28bf9..000000000 --- a/static/docs/smartui-gitlab-pr-checks-exec.md +++ /dev/null @@ -1,569 +0,0 @@ -# GitLab PR Checks with SmartUI Exec (SDK) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This guide shows you how to integrate SmartUI visual regression testing with GitLab merge requests using the **SmartUI SDK/Exec method**. This works for both **web testing** (Selenium, Playwright, Cypress, Puppeteer) and **mobile app testing** (Appium, WebdriverIO) across all supported languages. - -**SmartUI SDK/Exec vs Hooks** - -This guide covers the **SmartUI SDK/Exec approach**, where you use `npx smartui exec -- ` to run your tests. This is different from the Hooks approach: - -**SmartUI SDK/Exec (This Guide):** -- ✅ Uses `npx smartui exec -- ` -- ✅ Works with Java SDK and CLI-based projects -- ✅ Requires SmartUI CLI installation -- ✅ Suitable for Selenium/Playwright/Cypress/Puppeteer/Appium -- ✅ Works with **web testing** (Selenium, Playwright, Cypress, Puppeteer) -- ✅ Works with **mobile app testing** (Appium, iOS/Android) - -**SmartUI Hooks:** -- No `npx smartui exec` command needed -- Tests run normally (e.g., `npm test`, `mvn test`, `pytest`) -- SmartUI integration happens automatically through capabilities -- See [GitLab PR Checks with SmartUI Hooks](/support/docs/smartui-gitlab-pr-checks-hooks) for Hooks approach - -## Prerequisites - -Before you begin, ensure you have: - -- TestMu AI account with active subscription -- GitLab repository with CI/CD enabled -- SmartUI project created in [TestMu AI SmartUI Dashboard](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) -- Test suite configured (Selenium/Playwright/Cypress/Puppeteer/Appium/WebdriverIO) -- Test framework configured in your preferred language (TypeScript/JavaScript/Java/Python/Ruby/C#) -- TestMu AI credentials (`LT_USERNAME` and `LT_ACCESS_KEY`) -- SmartUI CLI installed (or use `npx @lambdatest/smartui-cli`) - -## Step 1: Integrate GitLab with TestMu AI - -1. Go to [TestMu AI Integrations page](https://integrations.lambdatest.com/) -2. Search for **GitLab** and select the integration -3. Click on **OAuth** as your preferred authentication method -4. Click **Install** and authorize the integration -5. After successful authentication, refresh the Integrations page to verify GitLab is installed - -**Integration Status** - -You can verify your GitLab integration is active by checking the [Integrations page](https://integrations.lambdatest.com/). The GitLab integration should show as "Installed" or "Active". - -## Step 2: Configure SmartUI Project - -Create or update your SmartUI configuration file (`.smartui.json`): - -```json title=".smartui.json" -{ -"projectName": "your-smartui-project-name", -"buildName": "smartui-build-${CI_PIPELINE_ID}", -"baseline": false -} -``` - -**Configuration File** - -The SmartUI configuration file is used to specify project settings. You can also use environment variables or command-line options to override these settings. - -## Step 3: Configure GitLab CI/CD Pipeline - -Create or update your `.gitlab-ci.yml` file. The key difference with the Exec method is that you use `npx smartui exec --gitURL` to run your tests. - -### Complete GitLab CI/CD Configuration - -```yaml title=".gitlab-ci.yml - TypeScript/JavaScript Example" -stages: -- test - -variables: -NODE_VERSION: "18" -LT_USERNAME: $LT_USERNAME -LT_ACCESS_KEY: $LT_ACCESS_KEY -PROJECT_TOKEN: $PROJECT_TOKEN - -visual_regression_tests: -stage: test -image: node:${NODE_VERSION} - -before_script: -- npm ci -- npm install -g @lambdatest/smartui-cli - -script: -# Get GitLab project ID and commit SHA -- | -PROJECT_ID=${CI_PROJECT_ID} -COMMIT_SHA=${CI_COMMIT_SHA} - -# For merge requests, use the merge request commit SHA -if [ -n "$CI_MERGE_REQUEST_IID" ]; then -COMMIT_SHA=${CI_MERGE_REQUEST_SHA:-${CI_COMMIT_SHA}} -fi - -# Construct GitLab API URL for status updates -GIT_URL="https://gitlab.com/api/v4/projects/${PROJECT_ID}/statuses/${COMMIT_SHA}" - -echo "GitLab Project ID: ${PROJECT_ID}" -echo "Commit SHA: ${COMMIT_SHA}" -echo "GitLab Status URL: ${GIT_URL}" - -# Run tests with SmartUI Exec and GitLab integration -npx smartui exec --gitURL "${GIT_URL}" -- npm test -# Or: npx smartui exec --gitURL "${GIT_URL}" -- npx wdio run wdio.conf.ts -# Or: npx smartui exec --gitURL "${GIT_URL}" -- npm run test:mobile - -only: -- merge_requests -- main -- develop - -environment: -name: visual-regression/$CI_COMMIT_REF_NAME -``` - -```yaml title=".gitlab-ci.yml - Java Example" -stages: -- test - -variables: -MAVEN_OPTS: "-Dmaven.repo.local=.m2/repository" -LT_USERNAME: $LT_USERNAME -LT_ACCESS_KEY: $LT_ACCESS_KEY -PROJECT_TOKEN: $PROJECT_TOKEN - -visual_regression_tests: -stage: test -image: maven:3.8-openjdk-11 - -cache: -paths: -- .m2/repository/ - -before_script: -- mvn clean install -DskipTests -- npm install -g @lambdatest/smartui-cli - -script: -# Get GitLab project ID and commit SHA -- | -PROJECT_ID=${CI_PROJECT_ID} -COMMIT_SHA=${CI_COMMIT_SHA} - -# For merge requests, use the merge request commit SHA -if [ -n "$CI_MERGE_REQUEST_IID" ]; then -COMMIT_SHA=${CI_MERGE_REQUEST_SHA:-${CI_COMMIT_SHA}} -fi - -# Construct GitLab API URL for status updates -GIT_URL="https://gitlab.com/api/v4/projects/${PROJECT_ID}/statuses/${COMMIT_SHA}" - -echo "GitLab Project ID: ${PROJECT_ID}" -echo "Commit SHA: ${COMMIT_SHA}" -echo "GitLab Status URL: ${GIT_URL}" - -# Run tests with SmartUI Exec and GitLab integration -npx smartui exec --gitURL "${GIT_URL}" -- mvn test -# Or: npx smartui exec --gitURL "${GIT_URL}" -- ./gradlew test (for Gradle) - -only: -- merge_requests -- main -- develop - -environment: -name: visual-regression/$CI_COMMIT_REF_NAME -``` - -```yaml title=".gitlab-ci.yml - Python Example" -stages: -- test - -variables: -PYTHON_VERSION: "3.9" -LT_USERNAME: $LT_USERNAME -LT_ACCESS_KEY: $LT_ACCESS_KEY -PROJECT_TOKEN: $PROJECT_TOKEN - -visual_regression_tests: -stage: test -image: python:${PYTHON_VERSION} - -before_script: -- pip install -r requirements.txt -- npm install -g @lambdatest/smartui-cli - -script: -# Get GitLab project ID and commit SHA -- | -PROJECT_ID=${CI_PROJECT_ID} -COMMIT_SHA=${CI_COMMIT_SHA} - -# For merge requests, use the merge request commit SHA -if [ -n "$CI_MERGE_REQUEST_IID" ]; then -COMMIT_SHA=${CI_MERGE_REQUEST_SHA:-${CI_COMMIT_SHA}} -fi - -# Construct GitLab API URL for status updates -GIT_URL="https://gitlab.com/api/v4/projects/${PROJECT_ID}/statuses/${COMMIT_SHA}" - -echo "GitLab Project ID: ${PROJECT_ID}" -echo "Commit SHA: ${COMMIT_SHA}" -echo "GitLab Status URL: ${GIT_URL}" - -# Run tests with SmartUI Exec and GitLab integration -npx smartui exec --gitURL "${GIT_URL}" -- pytest -# Or: npx smartui exec --gitURL "${GIT_URL}" -- python -m unittest discover -# Or: npx smartui exec --gitURL "${GIT_URL}" -- behave - -only: -- merge_requests -- main -- develop - -environment: -name: visual-regression/$CI_COMMIT_REF_NAME -``` - -```yaml title=".gitlab-ci.yml - Ruby Example" -stages: -- test - -variables: -RUBY_VERSION: "3.1" -LT_USERNAME: $LT_USERNAME -LT_ACCESS_KEY: $LT_ACCESS_KEY -PROJECT_TOKEN: $PROJECT_TOKEN - -visual_regression_tests: -stage: test -image: ruby:${RUBY_VERSION} - -before_script: -- bundle install -- npm install -g @lambdatest/smartui-cli - -script: -# Get GitLab project ID and commit SHA -- | -PROJECT_ID=${CI_PROJECT_ID} -COMMIT_SHA=${CI_COMMIT_SHA} - -# For merge requests, use the merge request commit SHA -if [ -n "$CI_MERGE_REQUEST_IID" ]; then -COMMIT_SHA=${CI_MERGE_REQUEST_SHA:-${CI_COMMIT_SHA}} -fi - -# Construct GitLab API URL for status updates -GIT_URL="https://gitlab.com/api/v4/projects/${PROJECT_ID}/statuses/${COMMIT_SHA}" - -echo "GitLab Project ID: ${PROJECT_ID}" -echo "Commit SHA: ${COMMIT_SHA}" -echo "GitLab Status URL: ${GIT_URL}" - -# Run tests with SmartUI Exec and GitLab integration -npx smartui exec --gitURL "${GIT_URL}" -- bundle exec rspec -# Or: npx smartui exec --gitURL "${GIT_URL}" -- bundle exec cucumber - -only: -- merge_requests -- main -- develop - -environment: -name: visual-regression/$CI_COMMIT_REF_NAME -``` - -### Key Configuration Points - -1. **SmartUI CLI Installation**: Install SmartUI CLI globally or use `npx @lambdatest/smartui-cli` -2. **GitLab Project ID**: Automatically available as `CI_PROJECT_ID` in GitLab CI/CD -3. **Commit SHA**: Use `CI_COMMIT_SHA` for regular commits, or `CI_MERGE_REQUEST_SHA` for merge requests -4. **GitLab API URL**: Construct as `https://gitlab.com/api/v4/projects/{projectId}/statuses/{commitId}` -5. **Exec Command**: Use `npx smartui exec --gitURL "${GIT_URL}" -- ` - -**Understanding the SmartUI Exec Command** - -The `npx smartui exec` command wraps your test execution and provides SmartUI integration: - -```bash -npx smartui exec --gitURL "" -- -``` - -- `--gitURL`: GitLab API URL for status updates (legacy name, works with GitLab) -- `--`: Separator before your test command -- ``: Your normal test command (e.g., `npm test`, `mvn test`, `pytest`) - -## Step 4: Set Up GitLab CI/CD Variables - -Configure the following variables in your GitLab project: - -1. Go to your GitLab project → **Settings** → **CI/CD** → **Variables** -2. Add the following variables: - -| Variable Name | Description | -|--------------|-------------| -| `LT_USERNAME` | Your TestMu AI username | -| `LT_ACCESS_KEY` | Your TestMu AI access key | -| `PROJECT_TOKEN` | Your SmartUI project token (found in SmartUI project settings) | - -**Project Token** - -The `PROJECT_TOKEN` is different from `LT_USERNAME` and `LT_ACCESS_KEY`. You can find it in your SmartUI project settings in the [SmartUI Dashboard](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/). - -## Step 5: View Pipeline Results in GitLab - -After your pipeline runs, you can view the results in the GitLab Pipelines page: - -The pipeline will show: -- **Pipeline status** (Success/Failed) -- **Job status** for SmartUI tests -- **Screenshot statistics** (Total, Approved, Changes Found) in the job tooltip - -## Step 6: View PR Check Results in GitLab Merge Request - -After your pipeline runs, you'll see SmartUI status checks in your GitLab merge request: - -### Successful Status - -When all visual tests pass: -- ✅ **Status**: Success -- **Details**: Click "Details" to view the SmartUI build in the dashboard -- **Screenshot**: All screenshots match baseline or are approved - -### Failed Status - -When visual differences are detected: -- ❌ **Status**: Failed -- **Details**: Click "Details" to review differences in SmartUI dashboard -- **Action Required**: Review and approve/reject changes in SmartUI dashboard - -## Complete Working Examples - -```yaml title=".gitlab-ci.yml - Complete Web Testing Example" -stages: -- test - -variables: -NODE_VERSION: "18" -LT_USERNAME: $LT_USERNAME -LT_ACCESS_KEY: $LT_ACCESS_KEY -PROJECT_TOKEN: $PROJECT_TOKEN - -visual_regression_tests: -stage: test -image: node:${NODE_VERSION} - -before_script: -- npm ci -- npm install -g @lambdatest/smartui-cli - -script: -- | -PROJECT_ID=${CI_PROJECT_ID} -COMMIT_SHA=${CI_COMMIT_SHA} - -if [ -n "$CI_MERGE_REQUEST_IID" ]; then -COMMIT_SHA=${CI_MERGE_REQUEST_SHA:-${CI_COMMIT_SHA}} -fi - -GIT_URL="https://gitlab.com/api/v4/projects/${PROJECT_ID}/statuses/${COMMIT_SHA}" - -echo "GitLab Status URL: ${GIT_URL}" - -# Run web tests with SmartUI Exec -npx smartui exec --gitURL "${GIT_URL}" -- npm test - -only: -- merge_requests -- main -``` - -```yaml title=".gitlab-ci.yml - Complete Java Web Testing Example" -stages: -- test - -variables: -MAVEN_OPTS: "-Dmaven.repo.local=.m2/repository" -LT_USERNAME: $LT_USERNAME -LT_ACCESS_KEY: $LT_ACCESS_KEY -PROJECT_TOKEN: $PROJECT_TOKEN - -visual_regression_tests: -stage: test -image: maven:3.8-openjdk-11 - -cache: -paths: -- .m2/repository/ - -before_script: -- mvn clean install -DskipTests -- npm install -g @lambdatest/smartui-cli - -script: -- | -PROJECT_ID=${CI_PROJECT_ID} -COMMIT_SHA=${CI_COMMIT_SHA} - -if [ -n "$CI_MERGE_REQUEST_IID" ]; then -COMMIT_SHA=${CI_MERGE_REQUEST_SHA:-${CI_COMMIT_SHA}} -fi - -GIT_URL="https://gitlab.com/api/v4/projects/${PROJECT_ID}/statuses/${COMMIT_SHA}" - -echo "GitLab Status URL: ${GIT_URL}" - -# Run Java tests with SmartUI Exec -npx smartui exec --gitURL "${GIT_URL}" -- mvn test - -only: -- merge_requests -- main -``` - -```yaml title=".gitlab-ci.yml - Complete Mobile Testing Example" -stages: -- test - -variables: -NODE_VERSION: "18" -LT_USERNAME: $LT_USERNAME -LT_ACCESS_KEY: $LT_ACCESS_KEY -PROJECT_TOKEN: $PROJECT_TOKEN - -visual_regression_tests: -stage: test -image: node:${NODE_VERSION} - -before_script: -- npm ci -- npm install -g @lambdatest/smartui-cli - -script: -- | -PROJECT_ID=${CI_PROJECT_ID} -COMMIT_SHA=${CI_COMMIT_SHA} - -if [ -n "$CI_MERGE_REQUEST_IID" ]; then -COMMIT_SHA=${CI_MERGE_REQUEST_SHA:-${CI_COMMIT_SHA}} -fi - -GIT_URL="https://gitlab.com/api/v4/projects/${PROJECT_ID}/statuses/${COMMIT_SHA}" - -echo "GitLab Status URL: ${GIT_URL}" - -# Run mobile tests with SmartUI Exec -npx smartui exec --gitURL "${GIT_URL}" -- npm run test:mobile -# Or: npx smartui exec --gitURL "${GIT_URL}" -- npx wdio run wdio.conf.ts - -only: -- merge_requests -- main -``` - -```yaml title=".gitlab-ci.yml - Complete Java Mobile Testing Example" -stages: -- test - -variables: -MAVEN_OPTS: "-Dmaven.repo.local=.m2/repository" -LT_USERNAME: $LT_USERNAME -LT_ACCESS_KEY: $LT_ACCESS_KEY -PROJECT_TOKEN: $PROJECT_TOKEN - -visual_regression_tests: -stage: test -image: maven:3.8-openjdk-11 - -cache: -paths: -- .m2/repository/ - -before_script: -- mvn clean install -DskipTests -- npm install -g @lambdatest/smartui-cli - -script: -- | -PROJECT_ID=${CI_PROJECT_ID} -COMMIT_SHA=${CI_COMMIT_SHA} - -if [ -n "$CI_MERGE_REQUEST_IID" ]; then -COMMIT_SHA=${CI_MERGE_REQUEST_SHA:-${CI_COMMIT_SHA}} -fi - -GIT_URL="https://gitlab.com/api/v4/projects/${PROJECT_ID}/statuses/${COMMIT_SHA}" - -echo "GitLab Status URL: ${GIT_URL}" - -# Run Java mobile tests with SmartUI Exec -npx smartui exec --gitURL "${GIT_URL}" -- mvn test -D suite=mobile-tests.xml - -only: -- merge_requests -- main -``` - -## Troubleshooting - -**Issue: PR Check Not Appearing in GitLab** - -**Symptoms**: Pipeline runs but no SmartUI status check appears in merge request. - -**Solutions**: -1. Verify GitLab integration is active in [TestMu AI Integrations](https://integrations.lambdatest.com/) -2. Check that `--gitURL` parameter is correctly set in the exec command -3. Verify GitLab API URL format: `https://gitlab.com/api/v4/projects/{projectId}/statuses/{commitId}` -4. Ensure `CI_PROJECT_ID` and `CI_COMMIT_SHA` are correctly set -5. For merge requests, use `CI_MERGE_REQUEST_SHA` instead of `CI_COMMIT_SHA` -6. Check pipeline logs to ensure tests completed successfully -7. Verify SmartUI CLI is installed and accessible - -**Issue: Tests Run But No Screenshots in SmartUI** - -**Symptoms**: Pipeline completes successfully but no screenshots appear in SmartUI dashboard. - -**Solutions**: -1. Verify `PROJECT_TOKEN` is correctly set in GitLab CI/CD variables -2. Check that SmartUI configuration file (`.smartui.json`) exists and is valid -3. Ensure `LT_USERNAME` and `LT_ACCESS_KEY` are correctly set -4. Verify SmartUI project exists and is accessible -5. Check test logs for SmartUI execute command errors -6. Ensure screenshot commands are being called in your tests -7. Verify `SMARTUI_SERVER_ADDRESS` is set correctly (if using non-Selenium frameworks) - -**Issue: SmartUI Exec Command Fails** - -**Symptoms**: `npx smartui exec` command fails or doesn't run tests. - -**Solutions**: -1. Verify SmartUI CLI is installed: `npm install -g @lambdatest/smartui-cli` -2. Check that `PROJECT_TOKEN` environment variable is set -3. Verify `.smartui.json` configuration file exists and is valid -4. Ensure test command after `--` is correct -5. Check for port conflicts (default port: 49152) -6. Review pipeline logs for detailed error messages -7. Try running the command locally first to debug - -## Key Differences: Exec vs Hooks - -| Aspect | SmartUI Exec (This Guide) | SmartUI Hooks | -|--------|---------------------------|---------------| -| **Command** | Use `npx smartui exec --gitURL -- ` | Run tests normally (`npm test`, `mvn test`, `pytest`) | -| **Integration** | Requires CLI wrapper | Automatic via capabilities | -| **Setup** | Install SmartUI CLI, configure `.smartui.json` | Add capabilities to test config | -| **GitLab Integration** | Use `--gitURL` parameter with exec | Add `github.url` capability | -| **Languages** | Java SDK, CLI projects, all frameworks | TypeScript/JS/Java/Python/Ruby/C#/WebdriverIO/Appium | -| **Project Token** | Required (`PROJECT_TOKEN`) | Not required (uses `LT_USERNAME`/`LT_ACCESS_KEY`) | -| **Server Address** | May need `SMARTUI_SERVER_ADDRESS` for non-Selenium | Not required | - -## Next Steps - -- Learn about [SmartUI CLI Exec Commands](/support/docs/smartui-cli-exec) for detailed exec usage -- Check the [SmartUI Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide/) for common issues -- Review [GitLab CI/CD Documentation](https://docs.gitlab.com/ee/ci/) for advanced pipeline configuration - -## Related Documentation - -- [GitLab PR Checks with SmartUI Hooks](/support/docs/smartui-gitlab-pr-checks-hooks) - Hooks approach for GitLab integration -- [SmartUI with GitLab](/support/docs/smartui-with-gitlab) - General GitLab integration guide -- [SmartUI CLI Exec Commands](/support/docs/smartui-cli-exec) - Detailed exec command reference -- [SmartUI Project Settings](/support/docs/smartui-project-settings) - Configure SmartUI projects -- [SmartUI Appium Java SDK](/support/docs/smartui-appium-java-sdk) - Java SDK documentation diff --git a/static/docs/smartui-gitlab-pr-checks-hooks.md b/static/docs/smartui-gitlab-pr-checks-hooks.md deleted file mode 100644 index 7dac74f04..000000000 --- a/static/docs/smartui-gitlab-pr-checks-hooks.md +++ /dev/null @@ -1,1063 +0,0 @@ -# GitLab PR Checks with SmartUI Hooks - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This guide shows you how to integrate SmartUI visual regression testing with GitLab merge requests using the **SmartUI Hooks** approach. This works for both **web testing** (Selenium, Playwright, Cypress, Puppeteer) and **mobile app testing** (Appium, WebdriverIO) across all supported languages. - -**SmartUI Hooks vs SDK** - -This guide covers the **SmartUI Hooks** approach, where you pass SmartUI capabilities directly in your test configuration. This is different from the SDK approach: - -**SmartUI Hooks (This Guide):** -- ✅ No `npx smartui exec` command needed -- ✅ Tests run normally (e.g., `npm test`, `mvn test`, `pytest`) -- ✅ SmartUI integration happens automatically through capabilities -- ✅ Suitable for TypeScript/JavaScript/Java/Python/Ruby/C#/WebdriverIO/Appium -- ✅ Works with **web testing** (Selenium, Playwright, Cypress, Puppeteer) -- ✅ Works with **mobile app testing** (Appium, iOS/Android) - -**SmartUI SDK:** -- Requires `npx smartui exec -- ` -- Used for Java SDK and CLI-based projects -- See [SmartUI Appium Java SDK](/support/docs/smartui-appium-java-sdk) for SDK approach - -## Prerequisites - -Before you begin, ensure you have: - -- TestMu AI account with active subscription -- GitLab repository with CI/CD enabled -- SmartUI project created in [TestMu AI SmartUI Dashboard](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) -- Test suite configured (Selenium/Playwright/Cypress/Puppeteer/Appium/WebdriverIO) -- Test framework configured in your preferred language (TypeScript/JavaScript/Java/Python/Ruby/C#) -- TestMu AI credentials (`LT_USERNAME` and `LT_ACCESS_KEY`) - -## Step 1: Integrate GitLab with TestMu AI - -1. Go to [TestMu AI Integrations page](https://integrations.lambdatest.com/) -2. Search for **GitLab** and select the integration -3. Click on **OAuth** as your preferred authentication method -4. Click **Install** and authorize the integration -5. After successful authentication, refresh the Integrations page to verify GitLab is installed - -**Integration Status** - -You can verify your GitLab integration is active by checking the [Integrations page](https://integrations.lambdatest.com/). The GitLab integration should show as "Installed" or "Active". - -## Step 2: Configure SmartUI Capabilities with GitLab Integration - -Configure your test suite with SmartUI capabilities. Since you're using the **Hooks approach**, you'll pass SmartUI capabilities directly in your test configuration, including the GitLab integration capability. - -```typescript title="Example: TypeScript Selenium Configuration with SmartUI Hooks and GitLab" -import { Builder, Capabilities } from 'selenium-webdriver'; - -const capabilities = { -browserName: 'Chrome', -browserVersion: 'latest', -platformName: 'Windows 10', -'LT:Options': { -username: process.env.LT_USERNAME, -accessKey: process.env.LT_ACCESS_KEY, -project: 'Your Project Name', -w3c: true, -name: 'Web Test Session', -build: process.env.CI -? `${process.env.CI_PROJECT_NAME}-${process.env.CI_PIPELINE_ID}` -: `smartui-local-build-${new Date().toISOString().split('T')[0]}`, - -// SmartUI Hooks Configuration -"smartUI.project": `${process.env.SMARTUI_PROJECT_NAME}-visual`, -"smartUI.build": process.env.CI -? `${process.env.CI_PROJECT_NAME}-${process.env.CI_PIPELINE_ID}` -: `smartui-local-build-${new Date().toISOString().split('T')[0]}`, -"smartUI.baseline": false, - -// GitLab Integration Capability -github: { -url: process.env.GIT_URL // GitLab API URL for status updates -} -} -}; - -const driver = await new Builder() -.usingServer(`https://${process.env.LT_USERNAME}:${process.env.LT_ACCESS_KEY}@hub.lambdatest.com/wd/hub`) -.withCapabilities(capabilities) -.build(); -``` - -```java title="Example: Java Selenium Configuration with SmartUI Hooks and GitLab" -package webhook; - -import org.openqa.selenium.chrome.ChromeOptions; -import org.openqa.selenium.remote.RemoteWebDriver; -import org.testng.annotations.BeforeClass; - -import java.net.MalformedURLException; -import java.net.URL; -import java.util.HashMap; -import java.util.Map; - -public class BaseClassWebhook { - -public RemoteWebDriver driver; -public String githubURL = System.getenv("GITHUB_URL"); // GitLab URL from CI/CD - -@BeforeClass -public void setup() throws MalformedURLException { -String username = System.getenv("LT_USERNAME") == null -? "Your LT Username" -: System.getenv("LT_USERNAME"); -String authkey = System.getenv("LT_ACCESS_KEY") == null -? "Your LT AccessKey" -: System.getenv("LT_ACCESS_KEY"); - -ChromeOptions browserOptions = new ChromeOptions(); -HashMap ltOptions = new HashMap(); - -// LambdaTest Options -ltOptions.put("username", username); -ltOptions.put("accessKey", authkey); -ltOptions.put("project", "Your Project Name"); -ltOptions.put("w3c", true); -ltOptions.put("browserName", "Chrome"); -ltOptions.put("browserVersion", "latest"); -ltOptions.put("platformName", "Windows 10"); - -// SmartUI Hooks Configuration -String projectName = System.getenv("SMARTUI_PROJECT_NAME") != null -? System.getenv("SMARTUI_PROJECT_NAME") + "-visual" -: "Your SmartUI Project Name"; -ltOptions.put("smartUI.project", projectName); - -String buildName = System.getenv("CI") != null -? System.getenv("CI_PROJECT_NAME") + "-" + System.getenv("CI_PIPELINE_ID") -: "smartui-local-build"; -ltOptions.put("smartUI.build", buildName); -ltOptions.put("smartUI.baseline", false); - -browserOptions.setCapability("LT:Options", ltOptions); - -// GitLab Integration Capability -if (githubURL != null) { -Map github = new HashMap(); -github.put("url", githubURL); -browserOptions.setCapability("github", github); -System.out.println("GitLab URL received successfully: " + githubURL); -} - -String remoteUrl = "https://" + username + ":" + authkey + "@hub.lambdatest.com/wd/hub"; -driver = new RemoteWebDriver(new URL(remoteUrl), browserOptions); -} -} -``` - -```python title="Example: Python Selenium Configuration with SmartUI Hooks and GitLab" -from selenium import webdriver -from selenium.webdriver.common.desired_capabilities import DesiredCapabilities -import os - -# Get GitLab URL from environment -github_url = os.getenv("GITHUB_URL") - -capabilities = { -"browserName": "Chrome", -"browserVersion": "latest", -"platformName": "Windows 10", -"LT:Options": { -"username": os.getenv("LT_USERNAME"), -"accessKey": os.getenv("LT_ACCESS_KEY"), -"project": "Your Project Name", -"w3c": True, -"name": "Web Test Session", -"build": f"{os.getenv('CI_PROJECT_NAME')}-{os.getenv('CI_PIPELINE_ID')}" if os.getenv("CI") else "smartui-local-build", - -# SmartUI Hooks Configuration -"smartUI.project": f"{os.getenv('SMARTUI_PROJECT_NAME')}-visual", -"smartUI.build": f"{os.getenv('CI_PROJECT_NAME')}-{os.getenv('CI_PIPELINE_ID')}" if os.getenv("CI") else "smartui-local-build", -"smartUI.baseline": False, - -# GitLab Integration Capability -"github": { -"url": github_url -} -} -} - -driver = webdriver.Remote( -command_executor=f"https://{os.getenv('LT_USERNAME')}:{os.getenv('LT_ACCESS_KEY')}@hub.lambdatest.com/wd/hub", -desired_capabilities=capabilities -) -``` - -```typescript title="Example: TypeScript/WebdriverIO Mobile Configuration with SmartUI Hooks and GitLab" -import { remote, RemoteOptions } from 'webdriverio'; - -const capabilities: RemoteOptions['capabilities'] = { -deviceName: "iPhone 12", -platformName: "ios", -platformVersion: "14", -isRealMobile: true, -app: "APP_URL", // Your uploaded app URL -visual: true, // Mandatory for SmartUI -name: "Mobile App Test Session", -build: process.env.CI -? `${process.env.CI_PROJECT_NAME}-${process.env.CI_PIPELINE_ID}` -: `smartui-local-build-${new Date().toISOString().split('T')[0]}`, - -// SmartUI Hooks Configuration -"smartUI.project": `${process.env.SMARTUI_PROJECT_NAME}-visual`, -"smartUI.build": process.env.CI -? `${process.env.CI_PROJECT_NAME}-${process.env.CI_PIPELINE_ID}` -: `smartui-local-build-${new Date().toISOString().split('T')[0]}`, -"smartUI.baseline": false, -"smartUI.cropStatusBar": true, -"smartUI.cropFooter": true, - -// GitLab Integration Capability -github: { -url: process.env.GITHUB_URL // GitLab API URL for status updates -// GitLab URL format: https://gitlab.com/api/v4/projects/{projectId}/statuses/{commitId} -} -}; - -const driver = await remote({ -hostname: 'mobile-hub.lambdatest.com', -port: 443, -path: '/wd/hub', -protocol: 'https', -user: process.env.LT_USERNAME, -key: process.env.LT_ACCESS_KEY, -capabilities: capabilities as any, -}); -``` - -```java title="Example: Java Appium Configuration with SmartUI Hooks and GitLab" -package webhook; - -import org.openqa.selenium.chrome.ChromeOptions; -import org.openqa.selenium.remote.RemoteWebDriver; -import org.testng.annotations.BeforeClass; - -import java.net.MalformedURLException; -import java.net.URL; -import java.util.HashMap; -import java.util.Map; - -public class BaseClassWebhook { - -public RemoteWebDriver driver; -public String githubURL = System.getenv("GITHUB_URL"); // GitLab URL from CI/CD - -@BeforeClass -public void setup() throws MalformedURLException { -String username = System.getenv("LT_USERNAME") == null -? "Your LT Username" -: System.getenv("LT_USERNAME"); -String authkey = System.getenv("LT_ACCESS_KEY") == null -? "Your LT AccessKey" -: System.getenv("LT_ACCESS_KEY"); - -ChromeOptions browserOptions = new ChromeOptions(); -HashMap ltOptions = new HashMap(); - -// LambdaTest Options -ltOptions.put("username", username); -ltOptions.put("accessKey", authkey); -ltOptions.put("project", "Your Project Name"); -ltOptions.put("w3c", true); -ltOptions.put("deviceName", "iPhone 12"); -ltOptions.put("platformName", "ios"); -ltOptions.put("platformVersion", "14"); -ltOptions.put("isRealMobile", true); -ltOptions.put("app", "APP_URL"); -ltOptions.put("visual", true); // Mandatory for SmartUI - -// SmartUI Hooks Configuration -String projectName = System.getenv("SMARTUI_PROJECT_NAME") != null -? System.getenv("SMARTUI_PROJECT_NAME") + "-visual" -: "Your SmartUI Project Name"; -ltOptions.put("smartUI.project", projectName); - -String buildName = System.getenv("CI") != null -? System.getenv("CI_PROJECT_NAME") + "-" + System.getenv("CI_PIPELINE_ID") -: "smartui-local-build"; -ltOptions.put("smartUI.build", buildName); -ltOptions.put("smartUI.baseline", false); -ltOptions.put("smartUI.cropStatusBar", true); - -browserOptions.setCapability("LT:Options", ltOptions); - -// GitLab Integration Capability -if (githubURL != null) { -Map github = new HashMap(); -github.put("url", githubURL); -browserOptions.setCapability("github", github); -System.out.println("GitLab URL received successfully: " + githubURL); -} - -String remoteUrl = "https://" + username + ":" + authkey + "@mobile-hub.lambdatest.com/wd/hub"; -driver = new RemoteWebDriver(new URL(remoteUrl), browserOptions); -} -} -``` - -```python title="Example: Python Appium Configuration with SmartUI Hooks and GitLab" -from appium import webdriver -import os - -# Get GitLab URL from environment -github_url = os.getenv("GITHUB_URL") - -capabilities = { -"deviceName": "iPhone 12", -"platformName": "ios", -"platformVersion": "14", -"isRealMobile": True, -"app": "APP_URL", # Your uploaded app URL -"visual": True, # Mandatory for SmartUI -"name": "Mobile App Test Session", -"build": f"{os.getenv('CI_PROJECT_NAME')}-{os.getenv('CI_PIPELINE_ID')}" if os.getenv("CI") else "smartui-local-build", - -# SmartUI Hooks Configuration -"smartUI.project": f"{os.getenv('SMARTUI_PROJECT_NAME')}-visual", -"smartUI.build": f"{os.getenv('CI_PROJECT_NAME')}-{os.getenv('CI_PIPELINE_ID')}" if os.getenv("CI") else "smartui-local-build", -"smartUI.baseline": False, -"smartUI.cropStatusBar": True, - -# GitLab Integration Capability -"github": { -"url": github_url # GitLab API URL for status updates -} -} - -driver = webdriver.Remote( -command_executor=f"https://{os.getenv('LT_USERNAME')}:{os.getenv('LT_ACCESS_KEY')}@mobile-hub.lambdatest.com/wd/hub", -desired_capabilities=capabilities -) -``` - -**GitLab Capability Note** - -The capability is named `github` (legacy name) but works with GitLab's API endpoint. Use the `GIT_URL` environment variable to pass the GitLab API URL. This is the same capability used for GitHub integration and supports both GitHub and GitLab status APIs. - -### Taking Screenshots with SmartUI Hooks - -In your test code, use the SmartUI execute command to capture screenshots: - -```typescript title="Taking Screenshots with SmartUI Hooks" -// Viewport screenshot -await driver.execute("smartui.takeScreenshot=Homepage"); - -// Full page screenshot (if supported) -const config = { -screenshotName: 'Homepage', -fullPage: true, -pageCount: 15 // Minimum 1, Maximum 20 -}; -await driver.execute("smartui.takeScreenshot", config); -``` - -```java title="Taking Screenshots with SmartUI Hooks in Java" -import org.openqa.selenium.JavaScriptExecutor; -import java.util.HashMap; -import java.util.Map; - -// Viewport screenshot -((JavaScriptExecutor) driver).executeScript("smartui.takeScreenshot=Homepage"); - -// Full page screenshot (if supported) -Map config = new HashMap<>(); -config.put("screenshotName", "Homepage"); -config.put("fullPage", true); -config.put("pageCount", 15); // Minimum 1, Maximum 20 -((JavaScriptExecutor) driver).executeScript("smartui.takeScreenshot", config); -``` - -```python title="Taking Screenshots with SmartUI Hooks in Python" -# Viewport screenshot -driver.execute_script("smartui.takeScreenshot=Homepage") - -# Full page screenshot (if supported) -config = { -"screenshotName": "Homepage", -"fullPage": True, -"pageCount": 15 # Minimum 1, Maximum 20 -} -driver.execute_script("smartui.takeScreenshot", config) -``` - -```ruby title="Taking Screenshots with SmartUI Hooks in Ruby" -# Viewport screenshot -driver.execute_script("smartui.takeScreenshot=Homepage") - -# Full page screenshot (if supported) -config = { -'screenshotName' => 'Homepage', -'fullPage' => true, -'pageCount' => 15 # Minimum 1, Maximum 20 -} -driver.execute_script("smartui.takeScreenshot", config) -``` - -```csharp title="Taking Screenshots with SmartUI Hooks in C#" -using OpenQA.Selenium; - -// Viewport screenshot -((IJavaScriptExecutor)driver).ExecuteScript("smartui.takeScreenshot=Homepage"); - -// Full page screenshot (if supported) -var config = new Dictionary -{ -{ "screenshotName", "Homepage" }, -{ "fullPage", true }, -{ "pageCount", 15 } // Minimum 1, Maximum 20 -}; -((IJavaScriptExecutor)driver).ExecuteScript("smartui.takeScreenshot", config); -``` - -**Important** - -The `visual: true` capability is **mandatory** for SmartUI visual regression testing. Without this capability, screenshots will not be captured and the build status will show as `Error`. - -## Step 3: Configure GitLab CI/CD Pipeline - -Create or update your `.gitlab-ci.yml` file. Since you're using **Hooks**, you just need to run your tests normally - no SmartUI CLI exec command required. - -### Complete GitLab CI/CD Configuration - -```yaml title=".gitlab-ci.yml - TypeScript/JavaScript Example" -stages: -- test - -variables: -NODE_VERSION: "18" -LT_USERNAME: $LT_USERNAME -LT_ACCESS_KEY: $LT_ACCESS_KEY -SMARTUI_PROJECT_NAME: $SMARTUI_PROJECT_NAME - -visual_regression_tests: -stage: test -image: node:${NODE_VERSION} - -before_script: -- npm ci - -script: -# Get GitLab project ID and commit SHA -- | -PROJECT_ID=${CI_PROJECT_ID} -COMMIT_SHA=${CI_COMMIT_SHA} - -# For merge requests, use the merge request commit SHA -if [ -n "$CI_MERGE_REQUEST_IID" ]; then -COMMIT_SHA=${CI_MERGE_REQUEST_SHA:-${CI_COMMIT_SHA}} -fi - -# Construct GitLab API URL for status updates -GITHUB_URL="https://gitlab.com/api/v4/projects/${PROJECT_ID}/statuses/${COMMIT_SHA}" - -echo "GitLab Project ID: ${PROJECT_ID}" -echo "Commit SHA: ${COMMIT_SHA}" -echo "GitLab Status URL: ${GITHUB_URL}" - -# Export GITHUB_URL as environment variable for use in test capabilities -export GITHUB_URL="${GITHUB_URL}" - -# Run your tests normally - SmartUI Hooks work automatically through capabilities -npm test -# Or: npx wdio run wdio.conf.ts -# Or: npm run test:mobile - -only: -- merge_requests -- main -- develop - -environment: -name: visual-regression/$CI_COMMIT_REF_NAME -``` - -```yaml title=".gitlab-ci.yml - Java Example" -stages: -- test - -variables: -MAVEN_OPTS: "-Dmaven.repo.local=.m2/repository" -LT_USERNAME: $LT_USERNAME -LT_ACCESS_KEY: $LT_ACCESS_KEY -SMARTUI_PROJECT_NAME: $SMARTUI_PROJECT_NAME - -visual_regression_tests: -stage: test -image: maven:3.8-openjdk-11 - -cache: -paths: -- .m2/repository/ - -before_script: -- mvn clean install -DskipTests - -script: -# Get GitLab project ID and commit SHA -- | -PROJECT_ID=${CI_PROJECT_ID} -COMMIT_SHA=${CI_COMMIT_SHA} - -# For merge requests, use the merge request commit SHA -if [ -n "$CI_MERGE_REQUEST_IID" ]; then -COMMIT_SHA=${CI_MERGE_REQUEST_SHA:-${CI_COMMIT_SHA}} -fi - -# Construct GitLab API URL for status updates -GITHUB_URL="https://gitlab.com/api/v4/projects/${PROJECT_ID}/statuses/${COMMIT_SHA}" - -echo "GitLab Project ID: ${PROJECT_ID}" -echo "Commit SHA: ${COMMIT_SHA}" -echo "GitLab Status URL: ${GITHUB_URL}" - -# Export GITHUB_URL as environment variable for use in test capabilities -export GITHUB_URL="${GITHUB_URL}" - -# Run your tests normally - SmartUI Hooks work automatically through capabilities -mvn test -# Or: ./gradlew test (for Gradle) - -only: -- merge_requests -- main -- develop - -environment: -name: visual-regression/$CI_COMMIT_REF_NAME -``` - -```yaml title=".gitlab-ci.yml - Python Example" -stages: -- test - -variables: -PYTHON_VERSION: "3.9" -LT_USERNAME: $LT_USERNAME -LT_ACCESS_KEY: $LT_ACCESS_KEY -SMARTUI_PROJECT_NAME: $SMARTUI_PROJECT_NAME - -visual_regression_tests: -stage: test -image: python:${PYTHON_VERSION} - -before_script: -- pip install -r requirements.txt - -script: -# Get GitLab project ID and commit SHA -- | -PROJECT_ID=${CI_PROJECT_ID} -COMMIT_SHA=${CI_COMMIT_SHA} - -# For merge requests, use the merge request commit SHA -if [ -n "$CI_MERGE_REQUEST_IID" ]; then -COMMIT_SHA=${CI_MERGE_REQUEST_SHA:-${CI_COMMIT_SHA}} -fi - -# Construct GitLab API URL for status updates -GITHUB_URL="https://gitlab.com/api/v4/projects/${PROJECT_ID}/statuses/${COMMIT_SHA}" - -echo "GitLab Project ID: ${PROJECT_ID}" -echo "Commit SHA: ${COMMIT_SHA}" -echo "GitLab Status URL: ${GITHUB_URL}" - -# Export GITHUB_URL as environment variable for use in test capabilities -export GITHUB_URL="${GITHUB_URL}" - -# Run your tests normally - SmartUI Hooks work automatically through capabilities -pytest -# Or: python -m unittest discover -# Or: behave - -only: -- merge_requests -- main -- develop - -environment: -name: visual-regression/$CI_COMMIT_REF_NAME -``` - -```yaml title=".gitlab-ci.yml - Ruby Example" -stages: -- test - -variables: -RUBY_VERSION: "3.1" -LT_USERNAME: $LT_USERNAME -LT_ACCESS_KEY: $LT_ACCESS_KEY -SMARTUI_PROJECT_NAME: $SMARTUI_PROJECT_NAME - -visual_regression_tests: -stage: test -image: ruby:${RUBY_VERSION} - -before_script: -- bundle install - -script: -# Get GitLab project ID and commit SHA -- | -PROJECT_ID=${CI_PROJECT_ID} -COMMIT_SHA=${CI_COMMIT_SHA} - -# For merge requests, use the merge request commit SHA -if [ -n "$CI_MERGE_REQUEST_IID" ]; then -COMMIT_SHA=${CI_MERGE_REQUEST_SHA:-${CI_COMMIT_SHA}} -fi - -# Construct GitLab API URL for status updates -GITHUB_URL="https://gitlab.com/api/v4/projects/${PROJECT_ID}/statuses/${COMMIT_SHA}" - -echo "GitLab Project ID: ${PROJECT_ID}" -echo "Commit SHA: ${COMMIT_SHA}" -echo "GitLab Status URL: ${GITHUB_URL}" - -# Export GITHUB_URL as environment variable for use in test capabilities -export GITHUB_URL="${GITHUB_URL}" - -# Run your tests normally - SmartUI Hooks work automatically through capabilities -bundle exec rspec -# Or: bundle exec cucumber - -only: -- merge_requests -- main -- develop - -environment: -name: visual-regression/$CI_COMMIT_REF_NAME -``` - -### Key Configuration Points - -1. **No SmartUI CLI exec needed**: With Hooks, you run your tests normally (e.g., `npm test`, `mvn test`, `pytest`) -2. **GitLab Project ID**: Automatically available as `CI_PROJECT_ID` in GitLab CI/CD -3. **Commit SHA**: Use `CI_COMMIT_SHA` for regular commits, or `CI_MERGE_REQUEST_SHA` for merge requests -4. **GitLab API URL**: Export as `GIT_URL` environment variable, which your test capabilities will use -5. **GitLab API URL Format**: `https://gitlab.com/api/v4/projects/{projectId}/statuses/{commitId}` - -**How Hooks Work** - -With SmartUI Hooks: -- You pass SmartUI capabilities (including `github.url` with `GIT_URL` for GitLab) in your test configuration -- Run your tests normally (no `npx smartui exec` command) -- SmartUI integration happens automatically through the capabilities -- GitLab PR checks are updated automatically when tests complete - -## Step 4: Set Up GitLab CI/CD Variables - -Configure the following variables in your GitLab project: - -1. Go to your GitLab project → **Settings** → **CI/CD** → **Variables** -2. Add the following variables: - -| Variable Name | Description | -|--------------|-------------| -| `LT_USERNAME` | Your TestMu AI username | -| `LT_ACCESS_KEY` | Your TestMu AI access key | -| `SMARTUI_PROJECT_NAME` | Your SmartUI project name | - -## Step 5: View Pipeline Results in GitLab - -After your pipeline runs, you can view the results in the GitLab Pipelines page: - -The pipeline will show: -- **Pipeline status** (Success/Failed) -- **Job status** for SmartUI tests -- **Screenshot statistics** (Total, Approved, Changes Found) in the job tooltip - -## Step 6: View PR Check Results in GitLab Merge Request - -After your pipeline runs, you'll see SmartUI status checks in your GitLab merge request: - -### Successful Status - -When all visual tests pass: -- ✅ **Status**: Success -- **Details**: Click "Details" to view the SmartUI build in the dashboard -- **Screenshot**: All screenshots match baseline or are approved - -### Failed Status - -When visual differences are detected: -- ❌ **Status**: Failed -- **Details**: Click "Details" to review differences in SmartUI dashboard -- **Action Required**: Review and approve/reject changes in SmartUI dashboard - -## Complete Working Examples - -```typescript title="example.spec.ts - Complete Web Test with SmartUI Hooks and GitLab" -import { Builder, Capabilities } from 'selenium-webdriver'; - -describe('Web Visual Regression Tests', () => { -let driver; - -before(async () => { -// Construct GitLab URL (in CI/CD, this would come from environment variable) -const gitUrl = process.env.GIT_URL || -`https://gitlab.com/api/v4/projects/${process.env.CI_PROJECT_ID}/statuses/${process.env.CI_COMMIT_SHA}`; - -const capabilities = { -browserName: 'Chrome', -browserVersion: 'latest', -platformName: 'Windows 10', -'LT:Options': { -username: process.env.LT_USERNAME, -accessKey: process.env.LT_ACCESS_KEY, -project: 'Your Project Name', -w3c: true, -name: 'Web Visual Tests', -build: process.env.CI -? `${process.env.CI_PROJECT_NAME}-${process.env.CI_PIPELINE_ID}` -: `local-build-${Date.now()}`, -"smartUI.project": `${process.env.SMARTUI_PROJECT_NAME}-visual`, -"smartUI.build": process.env.CI -? `${process.env.CI_PROJECT_NAME}-${process.env.CI_PIPELINE_ID}` -: `local-build-${Date.now()}`, -"smartUI.baseline": false, -// GitLab integration capability -github: { -url: gitUrl -} -} -}; - -driver = await new Builder() -.usingServer(`https://${process.env.LT_USERNAME}:${process.env.LT_ACCESS_KEY}@hub.lambdatest.com/wd/hub`) -.withCapabilities(capabilities) -.build(); -}); - -after(async () => { -if (driver) { -await driver.quit(); -} -}); - -it('should capture homepage screenshot', async () => { -await driver.get('https://example.com'); -await driver.executeScript("smartui.takeScreenshot=Homepage"); -}); - -it('should capture login page screenshot', async () => { -await driver.get('https://example.com/login'); -await driver.executeScript("smartui.takeScreenshot=LoginPage"); -}); -}); -``` - -```java title="BaseClassWebhook.java - Complete Java Web Test with SmartUI Hooks and GitLab" -package webhook; - -import org.openqa.selenium.JavaScriptExecutor; -import org.openqa.selenium.chrome.ChromeOptions; -import org.openqa.selenium.remote.RemoteWebDriver; -import org.testng.annotations.AfterClass; -import org.testng.annotations.BeforeClass; -import org.testng.annotations.Test; - -import java.net.MalformedURLException; -import java.net.URL; -import java.util.HashMap; -import java.util.Map; - -public class BaseClassWebhook { - -public RemoteWebDriver driver; -public String githubURL = System.getenv("GITHUB_URL"); // GitLab URL from CI/CD - -@BeforeClass -public void setup() throws MalformedURLException { -String username = System.getenv("LT_USERNAME") == null -? "Your LT Username" -: System.getenv("LT_USERNAME"); -String authkey = System.getenv("LT_ACCESS_KEY") == null -? "Your LT AccessKey" -: System.getenv("LT_ACCESS_KEY"); - -ChromeOptions browserOptions = new ChromeOptions(); -HashMap ltOptions = new HashMap(); - -// LambdaTest Options -ltOptions.put("username", username); -ltOptions.put("accessKey", authkey); -ltOptions.put("project", "Your Project Name"); -ltOptions.put("w3c", true); -ltOptions.put("browserName", "Chrome"); -ltOptions.put("browserVersion", "latest"); -ltOptions.put("platformName", "Windows 10"); - -// SmartUI Hooks Configuration -String projectName = System.getenv("SMARTUI_PROJECT_NAME") != null -? System.getenv("SMARTUI_PROJECT_NAME") + "-visual" -: "Your SmartUI Project Name"; -ltOptions.put("smartUI.project", projectName); - -String buildName = System.getenv("CI") != null -? System.getenv("CI_PROJECT_NAME") + "-" + System.getenv("CI_PIPELINE_ID") -: "smartui-local-build"; -ltOptions.put("smartUI.build", buildName); -ltOptions.put("smartUI.baseline", false); - -browserOptions.setCapability("LT:Options", ltOptions); - -// GitLab Integration Capability -if (githubURL != null) { -Map github = new HashMap(); -github.put("url", githubURL); -browserOptions.setCapability("github", github); -System.out.println("GitLab URL received successfully: " + githubURL); -} - -String remoteUrl = "https://" + username + ":" + authkey + "@hub.lambdatest.com/wd/hub"; -driver = new RemoteWebDriver(new URL(remoteUrl), browserOptions); -} - -@Test -public void testHomepageScreenshot() { -driver.get("https://example.com"); -((JavaScriptExecutor) driver).executeScript("smartui.takeScreenshot=Homepage"); -} - -@Test -public void testLoginPageScreenshot() { -driver.get("https://example.com/login"); -((JavaScriptExecutor) driver).executeScript("smartui.takeScreenshot=LoginPage"); -} - -@AfterClass -public void tearDown() { -if (driver != null) { -driver.quit(); -} -} -} -``` - -```typescript title="example.spec.ts - Complete Mobile App Test with SmartUI Hooks and GitLab" -import { remote, RemoteOptions } from 'webdriverio'; - -describe('Mobile App Visual Regression Tests', () => { -let driver: WebdriverIO.Browser; - -before(async () => { -// Construct GitLab URL (in CI/CD, this would come from environment variable) -const gitUrl = process.env.GIT_URL || -`https://gitlab.com/api/v4/projects/${process.env.CI_PROJECT_ID}/statuses/${process.env.CI_COMMIT_SHA}`; - -const capabilities: RemoteOptions['capabilities'] = { -deviceName: "iPhone 12", -platformName: "ios", -platformVersion: "14", -isRealMobile: true, -app: process.env.APP_URL || "YOUR_APP_URL", -visual: true, // Mandatory for SmartUI -name: "Mobile App Visual Tests", -build: process.env.CI -? `${process.env.CI_PROJECT_NAME}-${process.env.CI_PIPELINE_ID}` -: `local-build-${Date.now()}`, -"smartUI.project": `${process.env.SMARTUI_PROJECT_NAME}-visual`, -"smartUI.build": process.env.CI -? `${process.env.CI_PROJECT_NAME}-${process.env.CI_PIPELINE_ID}` -: `local-build-${Date.now()}`, -"smartUI.baseline": false, -"smartUI.cropStatusBar": true, -// GitLab integration capability -github: { -url: gitlabUrl -} -}; - -driver = await remote({ -hostname: 'mobile-hub.lambdatest.com', -port: 443, -path: '/wd/hub', -protocol: 'https', -user: process.env.LT_USERNAME, -key: process.env.LT_ACCESS_KEY, -capabilities: capabilities as any, -}); -}); - -after(async () => { -if (driver) { -await driver.deleteSession(); -} -}); - -it('should capture homepage screenshot', async () => { -// Navigate or perform actions -await driver.execute("smartui.takeScreenshot=Homepage"); -}); - -it('should capture login screen screenshot', async () => { -// Navigate to login screen -await driver.execute("smartui.takeScreenshot=LoginScreen"); -}); -}); -``` - -```java title="BaseClassWebhook.java - Complete Java Mobile Test with SmartUI Hooks and GitLab" -package webhook; - -import org.openqa.selenium.JavaScriptExecutor; -import org.openqa.selenium.chrome.ChromeOptions; -import org.openqa.selenium.remote.RemoteWebDriver; -import org.testng.annotations.AfterClass; -import org.testng.annotations.BeforeClass; -import org.testng.annotations.Test; - -import java.net.MalformedURLException; -import java.net.URL; -import java.util.HashMap; -import java.util.Map; - -public class BaseClassWebhook { - -public RemoteWebDriver driver; -public String githubURL = System.getenv("GITHUB_URL"); // GitLab URL from CI/CD - -@BeforeClass -public void setup() throws MalformedURLException { -String username = System.getenv("LT_USERNAME") == null -? "Your LT Username" -: System.getenv("LT_USERNAME"); -String authkey = System.getenv("LT_ACCESS_KEY") == null -? "Your LT AccessKey" -: System.getenv("LT_ACCESS_KEY"); - -ChromeOptions browserOptions = new ChromeOptions(); -HashMap ltOptions = new HashMap(); - -// LambdaTest Options -ltOptions.put("username", username); -ltOptions.put("accessKey", authkey); -ltOptions.put("project", "Your Project Name"); -ltOptions.put("w3c", true); -ltOptions.put("deviceName", "iPhone 12"); -ltOptions.put("platformName", "ios"); -ltOptions.put("platformVersion", "14"); -ltOptions.put("isRealMobile", true); -ltOptions.put("app", "APP_URL"); -ltOptions.put("visual", true); // Mandatory for SmartUI - -// SmartUI Hooks Configuration -String projectName = System.getenv("SMARTUI_PROJECT_NAME") != null -? System.getenv("SMARTUI_PROJECT_NAME") + "-visual" -: "Your SmartUI Project Name"; -ltOptions.put("smartUI.project", projectName); - -String buildName = System.getenv("CI") != null -? System.getenv("CI_PROJECT_NAME") + "-" + System.getenv("CI_PIPELINE_ID") -: "smartui-local-build"; -ltOptions.put("smartUI.build", buildName); -ltOptions.put("smartUI.baseline", false); -ltOptions.put("smartUI.cropStatusBar", true); - -browserOptions.setCapability("LT:Options", ltOptions); - -// GitLab Integration Capability -if (githubURL != null) { -Map github = new HashMap(); -github.put("url", githubURL); -browserOptions.setCapability("github", github); -System.out.println("GitLab URL received successfully: " + githubURL); -} - -String remoteUrl = "https://" + username + ":" + authkey + "@mobile-hub.lambdatest.com/wd/hub"; -driver = new RemoteWebDriver(new URL(remoteUrl), browserOptions); -} - -@Test -public void testHomepageScreenshot() { -// Navigate or perform actions -((JavaScriptExecutor) driver).executeScript("smartui.takeScreenshot=Homepage"); -} - -@Test -public void testLoginScreenScreenshot() { -// Navigate to login screen -((JavaScriptExecutor) driver).executeScript("smartui.takeScreenshot=LoginScreen"); -} - -@AfterClass -public void tearDown() { -if (driver != null) { -driver.quit(); -} -} -} -``` - -## Troubleshooting - -**Issue: PR Check Not Appearing in GitLab** - -**Symptoms**: Pipeline runs but no SmartUI status check appears in merge request. - -**Solutions**: -1. Verify GitLab integration is active in [TestMu AI Integrations](https://integrations.lambdatest.com/) -2. Check that `github.url` capability is correctly set in your test configuration -3. Verify `GIT_URL` environment variable is exported in CI/CD pipeline -4. Ensure `CI_PROJECT_ID` and `CI_COMMIT_SHA` are correctly set -5. For merge requests, use `CI_MERGE_REQUEST_SHA` instead of `CI_COMMIT_SHA` -6. Check test logs to ensure tests completed successfully -7. Verify GitLab API URL format: `https://gitlab.com/api/v4/projects/{projectId}/statuses/{commitId}` - -**Issue: Tests Run But No Screenshots in SmartUI** - -**Symptoms**: Pipeline completes successfully but no screenshots appear in SmartUI dashboard. - -**Solutions**: -1. Verify `visual: true` is set in capabilities -2. Check `smartUI.project` capability matches your SmartUI project name -3. Ensure `LT_USERNAME` and `LT_ACCESS_KEY` are correctly set -4. Verify you're using the correct TestMu AI grid URL: - - Web testing: `@hub.lambdatest.com/wd/hub` - - Mobile testing: `@mobile-hub.lambdatest.com/wd/hub` -5. Check test logs for SmartUI execute command errors -6. Verify SmartUI project exists and is accessible -7. Ensure screenshot commands are being called in your tests - -**Issue: GitLab URL Not Available in Tests** - -**Symptoms**: Tests run but GitLab PR check doesn't update. - -**Solutions**: -1. Verify `GIT_URL` is exported in CI/CD pipeline before test execution -2. Check that `github.url` capability is reading from `GIT_URL` environment variable -3. Add debug logging to verify URL is set -4. Ensure URL format is correct: `https://gitlab.com/api/v4/projects/{projectId}/statuses/{commitId}` -5. Verify the URL is set in the same script block that runs tests - -## Key Differences: Hooks vs SDK - -| Aspect | SmartUI Hooks (This Guide) | SmartUI SDK | -|--------|---------------------------|-------------| -| **Command** | Run tests normally (`npm test`, `mvn test`, `pytest`) | Use `npx smartui exec -- ` | -| **Integration** | Automatic via capabilities | Requires CLI wrapper | -| **Setup** | Add capabilities to test config | Configure CLI and run with exec | -| **GitLab Integration** | Add `github.url` capability with `GIT_URL` | Use `--gitURL` parameter with exec | -| **Languages** | TypeScript/JS/Java/Python/Ruby/C#/WebdriverIO/Appium | Java SDK, CLI projects | -| **Java Support** | ✅ Yes - Use capabilities with `github` capability | ✅ Yes - Use `npx smartui exec -- mvn test` | - -## Next Steps - -- Learn about [SmartUI Appium Hooks](/support/docs/smartui-appium-hooks) for detailed mobile testing guide -- Check the [SmartUI Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide/) for common issues -- Review [GitLab CI/CD Documentation](https://docs.gitlab.com/ee/ci/) for advanced pipeline configuration - -## Related Documentation - -- [SmartUI with GitLab](/support/docs/smartui-with-gitlab) - General GitLab integration guide -- [SmartUI Appium Hooks](/support/docs/smartui-appium-hooks) - Mobile testing with Appium Hooks -- [SmartUI Mobile Testing Overview](/support/docs/smartui-appium-sdk) - Mobile testing frameworks overview -- [SmartUI Project Settings](/support/docs/smartui-project-settings) - Configure SmartUI projects -- [GitHub App Integration](/support/docs/smartui-github-app-integration) - Similar integration pattern for GitHub diff --git a/static/docs/smartui-group-by-test-cases.md b/static/docs/smartui-group-by-test-cases.md deleted file mode 100644 index a39acd921..000000000 --- a/static/docs/smartui-group-by-test-cases.md +++ /dev/null @@ -1,220 +0,0 @@ -# Grouping SmartUI Screenshots by Test Cases - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI’s SmartUI allows you to organize your visual regression screenshots logically by linking them to the automation test cases they were captured in. Grouping screenshots by test cases reduces triage fatigue and allows you to review visual differences with full automation execution context. - -This guide’s **recommended flow** is **Selenium on the grid + SmartUI Hooks** (`smartUI.project` in Java `LT:Options` and **`smartui.takeScreenshot`** in the browser). That path works with **Hooks-oriented** SmartUI projects (including many **Omni** projects) and matches the **Capability Generator** output in the TestMu AI dashboard. - -It also covers how to get **one Automation / SmartUI build** with **multiple captures** (for example, five logical test cases), how to use **Group by → Test Case** in SmartUI, the **Automation → SmartUI Screenshots** tab, and **bulk** approve or reject. An **optional** Node SDK + `visual: true` flow is included for **Web-only** SmartUI projects. - -## Audience & Prerequisites - -- **Audience:** QA Engineers and Developers running automated visual tests on TestMu AI. -- **Prerequisites (recommended: Hooks):** - - **LambdaTest** username and access key: `LT_USERNAME`, `LT_ACCESS_KEY`. - - SmartUI project wired through **`smartUI.project`** in Java `LT:Options` (same string as **Smart UI Project Name** in the dashboard, e.g. `sample` or `new`). - - **`smartui.takeScreenshot`** with a **`screenshotName`** per capture when you want multiple named screenshots in one run. - - For **SmartUI CLI** static or exec workflows, a **`PROJECT_TOKEN`** from that project (often ends with `#` matching the project name). -- **Prerequisites (optional: Node SDK + `visual: true`):** The SmartUI project must be **Web** type; **Omni** names with **`smartUIProjectName`** + **`visual: true`** cause hub `failed to validate project`, see [Omni vs Web](#omni-vs-web). Prefer the **Hooks** flow above for Omni / Hooks projects. -- On the **grid**, grouping metadata usually comes from session **`name`**, suite/project, or runner test titles. For **local** `smartui exec` + Cypress or Playwright, see the SmartUI SDK docs. - -**`smartui capture` (static URLs)** uploads screenshots to a build but does **not** attach automation test-case metadata. To validate **Group by → Test Case** with real folders, run **automation on the grid** (for example the [Java Hooks flow](#hooks-java-webhook), or Cypress/Playwright with **`smartui exec`**) so captures are tied to test/session identity. - -## Recommended flow: Java Selenium + SmartUI Hooks {#hooks-java-webhook} - -Use this when your SmartUI project is **Hooks**-oriented (dashboard **Capability Generator** shows **`smartUI.project`** and you capture with **`smartui.takeScreenshot`**). This is the right default for many **Omni** projects; you do **not** need **`visual: true`** or **`smartUIProjectName`** on the hub for this path. - -### Clone and credentials - -Clone the official sample: - -```bash -git clone https://github.com/LambdaTest/java-selenium-sample -cd java-selenium-sample -``` - -Set credentials (macOS/Linux): - -```bash -export LT_USERNAME='' -export LT_ACCESS_KEY='' -``` - -For **SmartUI CLI** jobs (capture/exec/upload), also set **`PROJECT_TOKEN`** from the SmartUI project (suffix after the last `#` usually matches the project slug, e.g. **`#sample`**): - -```bash -export PROJECT_TOKEN='#sample' -``` - -### Capabilities and webhook - -Paste **Java** capabilities from the dashboard **Capability Generator** into your test class under `src/test/com/lambdatest/` (this repo uses **`src/test/com/...`**, not `src/test/java/...`). The important SmartUI line is: - -```java -ltOptions.put("smartUI.project", ""); -``` - -Use the same string as **Smart UI Project Name** in the UI (e.g. `new`, `sample`, or `Java-Selenium` as in the sample). - -After navigation, call the **SmartUI hook** (use a **different `screenshotName`** per step or per logical test so screenshots do not overwrite each other): - -```java -Map config = new HashMap<>(); -config.put("screenshotName", ""); -((JavascriptExecutor) driver).executeScript("smartui.takeScreenshot", config); -``` - -The runnable reference for **Hooks + SmartUI** (no DevTools) is: - -`src/test/com/lambdatest/SmartUI.java` - -Run it: - -```bash -mvn clean install exec:java -Dexec.mainClass="com.lambdatest.SmartUI" -Dexec.classpathScope=test -e -``` - -To align **`SmartUI.java`** with your dashboard project name, change the **`smartUI.project`** value in that file to match your **Smart UI Project Name**. - -`src/test/com/lambdatest/BasicAuthentication.java` is useful for **generated caps** pasted from the UI, but that sample uses **Chrome DevTools** on the grid; you may need a matching **`selenium-devtools-vNN`** dependency on your classpath if the remote Chrome major version does not match bundled CDP. Prefer **`SmartUI.java`** for a minimal Hooks-only run. - -Use **`mvn versions:display-dependency-updates`** only if you want to inspect dependency updates; it is not required to execute tests. - -For more Java options, see the SmartUI Selenium Java SDK. - -## Multiple test cases, one build (Hooks pattern) - -Use this when you want **one Automation / SmartUI build** with **several** captures that can be reviewed under **Group by → Test Case**. - -### Rules - -1. **Same `smartUI.project`** (and same **`PROJECT_TOKEN`** when using the CLI) for the whole run. -2. **Same `build`** in `LT:Options` for every session that should belong to the same Automation build. -3. **Distinct `screenshotName`** values for each **`smartui.takeScreenshot`** call so each capture is a separate asset in the build. -4. **Test case folders:** vary the session **`name`** in `LT:Options` per test method or per parallel session when you need multiple **Test Case** groups (same pattern as multi-session Selenium suites). For **one session** with multiple hooks, grouping still uses automation metadata from that session; align **`name`** / suite with your runner (JUnit/TestNG method names, etc.) per your organization’s conventions. -5. You can run sessions **sequentially or in parallel** as long as **`build`** and **`smartUI.project`** stay consistent. - -## Optional: Node SDK + `visual: true` (Web SmartUI projects only) - -Use **`@lambdatest/selenium-driver`** **`smartuiSnapshot`** + **`visual: true`** + **`smartUIProjectName`** only when the SmartUI project is **Web** type. **Omni** project names cause **`failed to validate project`** on the hub for this path; use the [Hooks flow](#hooks-java-webhook) instead. - -Workspace sample (five sessions, five names, one build): - -`smartui-node-sample/sdk/sdkCloud.js` - -Pattern: constant **`build`**, unique **`name`** per session, **`visual: true`**, **`smartUIProjectName`** set to the **Web** project name, then **`smartuiSnapshot(driver, screenshotName)`** per session. - -### Cypress / Playwright / other - -- Use **`smartui exec`** (or your CI job) with a **single `--buildName`** for the whole suite. -- Map each **`it` / test** (or file) to a unique snapshot name; consult the SmartUI SDK docs for `testName`, `suiteName`, or `testCaseId` on your stack. - -## Viewing Grouped Screenshots in SmartUI - -By default, screenshots in a SmartUI build are listed flatly. - -To view them hierarchically: - -1. Open your SmartUI Project and select the build you wish to review. -2. At the top right of the build view, click the **Group By** dropdown menu. -3. Select **Test Case**. - -4. Your screenshots will now be grouped into folders representing each automation test case. - -### Expanding and Collapsing Groups - -- Click the **arrow icon** or the **folder name** next to a test case to expand or collapse its screenshots. -- The group header provides a summary, including the total number of screenshots, the number of unreviewed/approved/rejected screenshots, and the average mismatch percentage for that specific test case. - -## The Automation SmartUI Tab - -You can review visual differences without leaving your test automation execution context. - -1. Navigate to your **Automation** dashboard and select a test run. -2. In the test detail view, locate the **SmartUI Screenshots** tab alongside the traditional Overview, Logs, and Network tabs. -3. The tab badge will display the total number of screenshots captured during this specific test execution. -4. Click the tab to view a thumbnail grid of all screenshots. Each thumbnail displays its status, mismatch percentage, and environment details. -5. Click any thumbnail to open the **Comparison Viewer Modal** to perform side-by-side, slider, or diff-overlay visual inspections right from the Automation page. - -## Bi-directional Navigation - -You can seamlessly jump between SmartUI and Automation contexts: - -### From SmartUI to Automation -If you are reviewing a test group in SmartUI and want to see the automation logs, click the **View in Automation →** link located under the group header. This will open the specific automation test detail page with the "SmartUI Screenshots" tab pre-selected. - -### From Automation to SmartUI -If you are in the Automation dashboard and prefer the full-screen SmartUI review experience, click the **Open in SmartUI →** link at the top of the SmartUI tab. This will navigate you to the SmartUI Build Review page with the relevant test group already expanded and focused. - -## Executing Bulk Actions - -When reviewing a large number of screenshots, you can triage them efficiently using bulk actions at the test case group level. - -**Group by Test Case** uses the **same bulk-action workflow** as the standard (flat) build view: select screenshots via the **group checkbox** on a test-case folder (or pick individual thumbnails), then use **Approve All** / **Reject All** on the floating action bar. You are only changing how screenshots are **organized**, not which bulk controls are available. - -1. Ensure your view is grouped by **Test Case**. -2. To select all screenshots within a test case, check the **Group Checkbox** located on the test group header. -3. A floating **Bulk Action Bar** will appear at the bottom of the screen. -4. Click **✓ Approve All** to accept the actual screenshots as the new baselines for all selected items. -5. Click **✗ Reject All** to reject the selected visual differences. -6. To cancel your selection, click the **✕** button on the action bar. - -You can expand a group and check or uncheck individual screenshots before applying a bulk action. The group checkbox will show an indeterminate state `[-]` if only a partial selection is made. - -## Status Filters - -To focus your review efforts, use the status filter pills at the top of the SmartUI build view. These filters work seamlessly with the grouped view: - -- **All (Gray):** Displays all screenshots. -- **To Review (Amber):** Displays only screenshots with unreviewed mismatches. -- **Approved (Green):** Displays only approved screenshots. -- **Rejected (Red):** Displays only rejected screenshots. - -Empty test groups will automatically hide if none of their screenshots match the selected filter. - -## Troubleshooting - -### SmartUI or Automation build fails: `failed to validate project` (Omni vs Web) {#omni-vs-web} - -**Symptoms:** The first session may start and even capture a snapshot, then the run stops with a hub error such as: *The project name … already exists with platform type **omni**. Please use different project name or create a new project of **web** platform type.* The SmartUI CLI may still show `✔ Build finalized with errors` or a non-zero exit code. - -**Likely cause:** You are using **`visual: true`** with **`smartUIProjectName`** (Node / SDK snapshot style), but the SmartUI project is **Omni**. That combination requires a **Web** SmartUI project. The [recommended Hooks flow](#hooks-java-webhook) (`smartUI.project` + **`smartui.takeScreenshot`**) does not use this hub validation path. - -**Fix:** - -1. In the SmartUI dashboard, create a **Web** visual project (or open an existing **Web** project). -2. Use that project’s **token** as `PROJECT_TOKEN` for `smartui exec` / CLI. -3. Set Selenium **`smartUIProjectName`** to that project’s **name** exactly (if you use an env override such as `SMARTUI_PROJECT_NAME`, it must match the Web project, not an Omni project). - -**Hooks vs SDK:** Omni-only or mixed-type projects are often used with **Hooks** (`smartui.takeScreenshot` and `smartUI.project` in Java). The **Omni vs Web** hub error applies when you combine an Omni **project name** with **`visual: true`** and **`smartUIProjectName`** (Node-style SDK snapshots on the grid). For Hooks-first flows, follow the [Hooks projects and Java Selenium](#hooks-java-webhook) section and the Capability Generator for your tool. - -### Screenshots appear in a "Flat" view despite grouping enabled -**Likely Cause:** The automation execution did not inject the `testName` or `suiteName` metadata into the SmartUI capture event. -**Fix:** Ensure you are using the latest version of the SmartUI SDK. If running tests locally, check the SDK documentation for your specific framework on how to explicitly pass `testName` as an environment variable or argument during capture. - -### All screenshots appear under one test case, or grouping shows a single folder -**Likely Cause:** Every session used the **same** automation test `name` (or the runner did not vary the test identifier). -**Fix:** Give each of the five tests a **unique** session `name` / test title while keeping **`build`** identical so you still get one SmartUI build. - -### Screenshots split across multiple SmartUI builds -**Likely Cause:** **`build`** or **project / `PROJECT_TOKEN`** differed between runs. -**Fix:** Use one shared `build` string and one SmartUI project for all five sessions in that validation run. - -### You used `smartui capture` and Group by Test Case is empty or useless -**Likely Cause:** Static URL capture does not send automation test metadata. -**Fix:** Run the five scenarios through **Automation + SDK** or **`smartui exec`**, then open the build and set **Group By → Test Case**. - -## Limits and Notes - -- Group actions are optimized for performance and will complete in under 2 seconds for up to 50 items. For larger test suites, processing might briefly pause. -- When navigating via deep links, your current state (including applied filters and zoom levels) is preserved. -- "View in Automation" links will only be available if the visual captures were tied to a valid automation session ID on the platform. - -## Related Resources - -- Visual Regression Testing with SmartUI -- SmartUI SDK Documentation -- SmartUI Selenium Java SDK -- SmartUI CLI (`capture` / `exec`) -- [LambdaTest Java Selenium sample (GitHub)](https://github.com/LambdaTest/java-selenium-sample) diff --git a/static/docs/smartui-guided-walkthrough.md b/static/docs/smartui-guided-walkthrough.md deleted file mode 100644 index de5a89451..000000000 --- a/static/docs/smartui-guided-walkthrough.md +++ /dev/null @@ -1,30 +0,0 @@ -# Guided Walkthrough of SmartUI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This page gives you a guided walkthrough of all the things on the HyperExecute Dashboard and how you can get the most out of the platform once you have successfully [run your first sample project](/support/docs/smartui-running-your-first-project/). - -## Projects Page -All the tests on SmartUI gets executed as a Project. All the Projects can be found on our Projects page. Each Project has a Approver, Tags, Platform, and a Sort By section followed by the details of the number of builds and who initiated the Project. - -## Projects Dashboard Details -The SmartUI dashboard presents a comparison between the baseline and the captured screenshots for visual regression testing. It provides essential insights to detect any UI anomalies. - -### Key Sections -- **Baseline View :** Displays the previously captured UI state for comparison. It serves as the reference point for regression testing. -- **Captured View :** Shows the latest UI version to compare against the baseline. Differences are highlighted for easy identification. - -### Diff Controls -Located at the center of the top menu, the Diff Controls allow testers to customize how visual differences are displayed, offering multiple options to refine the comparison process. The available modes are: - -#### Layers -- **Captured Diffs :** Highlights the differences detected in the current captured screenshots when compared to the baseline. Ideal for spotting changes introduced in the latest version. -- **Baseline Diffs :** Displays the differences by focusing on the baseline's unique elements. Helps identify elements missing or changed in the new capture. Annotations : Allows users to add notes or highlight specific regions of interest on the screenshots. Useful for documenting findings or collaborating with team members. -- **Annotations :** Controls the visibility of your annotations in both parallel and landscape views with a toggle, default set to off, ensuring a clean visual field when not needed. - -#### Diff Options -- **Strict Mode :** Highlights exact pixel mismatches between the baseline and captured screenshots. Best suited for detailed visual inspections where high precision is required. -- **Smart Ignore :** Iintelligently hides the displacement differences, allowing you to concentrate on actual content changes that impact the user experience. - -### Variants -The top menu includes browser variants, which allow cross-browser testing. Variants visible in the screenshot include: Chrome, Firefox, Edge, Playwright WebKit (Pw-Webkit). Each variant shows the same test across different browsers and resolutions, helping identify browser-specific visual issues. diff --git a/static/docs/smartui-handle-dynamic-data.md b/static/docs/smartui-handle-dynamic-data.md deleted file mode 100644 index a9fe94f30..000000000 --- a/static/docs/smartui-handle-dynamic-data.md +++ /dev/null @@ -1,497 +0,0 @@ -# Handle Dynamic Data in Visual Tests - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## What is Dynamic Data? - -Dynamic data refers to content on your web pages that changes between test runs, such as timestamps, user IDs, session tokens, randomly generated content, or data that updates in real-time. When conducting visual regression tests, these dynamic elements can cause false positives because the content differs between the baseline and current screenshots, even when the actual UI design remains unchanged. - -SmartUI provides two powerful options to handle dynamic data: - -- **Ignore DOM Elements**: Exclude specific elements from visual comparison -- **Select DOM Elements**: Include only specific elements in visual comparison - -**Related Documentation** -If you're experiencing high mismatch percentages or false positives, see our [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) for solutions. For project-level settings, check [Project Settings](/support/docs/smartui-project-settings). - -## When to Use Dynamic Data Handling - -You should use dynamic data handling in the following scenarios: - -1. **Timestamps and Dates**: Pages displaying current date/time that changes on each load -2. **User-Specific Content**: User IDs, usernames, or personalized content -3. **Session Tokens**: Authentication tokens or session identifiers -4. **Random Content**: Randomly generated IDs, UUIDs, or nonces -5. **Live Data**: Stock prices, weather data, or other real-time information -6. **Advertisements**: Rotating ads or promotional content -7. **Cookie Banners**: Consent banners that appear differently each time -8. **Notifications**: Unread message counts or notification badges - -## Dynamic Data Handling Methods - -**Ignore DOM Elements** - -Use `ignoreDOM` to exclude specific elements from visual comparison. This is useful when you want to compare the entire page but ignore certain dynamic elements. - -**Syntax** - -```javascript -let options = { -ignoreDOM: { -id: ["element-id-1", "element-id-2"], -class: ["class-name-1", "class-name-2"], -cssSelector: ["selector-1", "selector-2"], -xpath: ["xpath-1", "xpath-2"] -} -} -smartuiSnapshot(driver, 'Screenshot Name', options); -``` - -**Examples by Selector Type** - -**JavaScript (Selenium)** -```javascript -const { Builder } = require('selenium-webdriver'); -const { smartuiSnapshot } = require('@lambdatest/selenium-driver'); - -let driver = await new Builder().forBrowser("chrome").build(); -await driver.get('https://example.com'); - -let options = { -ignoreDOM: { -id: ["timestamp", "user-id", "session-token"] -} -}; -await smartuiSnapshot(driver, 'Home Page', options); -``` - -**Java (Selenium)** -```java - -HashMap> ignoreDOM = new HashMap<>(); -ArrayList ids = new ArrayList<>(); -ids.add("timestamp"); -ids.add("user-id"); -ids.add("session-token"); -ignoreDOM.put("id", ids); - -HashMap options = new HashMap<>(); -options.put("ignoreDOM", ignoreDOM); - -SmartUISnapshot.smartuiSnapshot(driver, "Home Page", options); -``` - -**Python (Selenium)** -```python -from lambdatest import smartui_snapshot - -options = { -"ignoreDOM": { -"id": ["timestamp", "user-id", "session-token"] -} -} -smartui_snapshot(driver, "Home Page", options) -``` - -**JavaScript (Selenium)** -```javascript -let options = { -ignoreDOM: { -class: ["dynamic-content", "timestamp", "user-info"] -} -}; -await smartuiSnapshot(driver, 'Home Page', options); -``` - -**Java (Selenium)** -```java -ArrayList classes = new ArrayList<>(); -classes.add("dynamic-content"); -classes.add("timestamp"); -classes.add("user-info"); -ignoreDOM.put("class", classes); -``` - -**Python (Selenium)** -```python -options = { -"ignoreDOM": { -"class": ["dynamic-content", "timestamp", "user-info"] -} -} -``` - -**JavaScript (Selenium)** -```javascript -let options = { -ignoreDOM: { -cssSelector: [ -"#timestamp", -".user-info", -"[data-testid='session-token']", -"div.dynamic-content > span" -] -} -}; -await smartuiSnapshot(driver, 'Home Page', options); -``` - -**Java (Selenium)** -```java -ArrayList selectors = new ArrayList<>(); -selectors.add("#timestamp"); -selectors.add(".user-info"); -selectors.add("[data-testid='session-token']"); -ignoreDOM.put("cssSelector", selectors); -``` - -**Python (Selenium)** -```python -options = { -"ignoreDOM": { -"cssSelector": [ -"#timestamp", -".user-info", -"[data-testid='session-token']", -"div.dynamic-content > span" -] -} -} -``` - -**JavaScript (Selenium)** -```javascript -let options = { -ignoreDOM: { -xpath: [ -"//div[@id='timestamp']", -"//span[@class='user-id']", -"//*[@data-testid='session-token']" -] -} -}; -await smartuiSnapshot(driver, 'Home Page', options); -``` - -**Java (Selenium)** -```java -ArrayList xpaths = new ArrayList<>(); -xpaths.add("//div[@id='timestamp']"); -xpaths.add("//span[@class='user-id']"); -xpaths.add("//*[@data-testid='session-token']"); -ignoreDOM.put("xpath", xpaths); -``` - -**Python (Selenium)** -```python -options = { -"ignoreDOM": { -"xpath": [ -"//div[@id='timestamp']", -"//span[@class='user-id']", -"//*[@data-testid='session-token']" -] -} -} -``` - -**Select DOM Elements** - -Use `selectDOM` to include only specific elements in visual comparison. This is useful when you want to compare only certain parts of the page, ignoring everything else. - -**Syntax** - -```javascript -let options = { -selectDOM: { -id: ["element-id-1", "element-id-2"], -class: ["class-name-1", "class-name-2"], -cssSelector: ["selector-1", "selector-2"], -xpath: ["xpath-1", "xpath-2"] -} -} -smartuiSnapshot(driver, 'Screenshot Name', options); -``` - -**Examples by Selector Type** - -**JavaScript (Selenium)** -```javascript -let options = { -selectDOM: { -id: ["main-content", "header", "footer"] -} -}; -await smartuiSnapshot(driver, 'Home Page', options); -``` - -**Java (Selenium)** -```java -HashMap> selectDOM = new HashMap<>(); -ArrayList ids = new ArrayList<>(); -ids.add("main-content"); -ids.add("header"); -ids.add("footer"); -selectDOM.put("id", ids); - -HashMap options = new HashMap<>(); -options.put("selectDOM", selectDOM); -``` - -**Python (Selenium)** -```python -options = { -"selectDOM": { -"id": ["main-content", "header", "footer"] -} -} -``` - -**JavaScript (Selenium)** -```javascript -let options = { -selectDOM: { -class: ["product-card", "price", "description"] -} -}; -await smartuiSnapshot(driver, 'Product Page', options); -``` - -**Java (Selenium)** -```java -ArrayList classes = new ArrayList<>(); -classes.add("product-card"); -classes.add("price"); -classes.add("description"); -selectDOM.put("class", classes); -``` - -**Python (Selenium)** -```python -options = { -"selectDOM": { -"class": ["product-card", "price", "description"] -} -} -``` - -**JavaScript (Selenium)** -```javascript -let options = { -selectDOM: { -cssSelector: [ -".main-content", -"#product-grid", -"[data-testid='product-list']" -] -} -}; -await smartuiSnapshot(driver, 'Product Page', options); -``` - -**Java (Selenium)** -```java -ArrayList selectors = new ArrayList<>(); -selectors.add(".main-content"); -selectors.add("#product-grid"); -selectors.add("[data-testid='product-list']"); -selectDOM.put("cssSelector", selectors); -``` - -**Python (Selenium)** -```python -options = { -"selectDOM": { -"cssSelector": [ -".main-content", -"#product-grid", -"[data-testid='product-list']" -] -} -} -``` - -**JavaScript (Selenium)** -```javascript -let options = { -selectDOM: { -xpath: [ -"//div[@class='main-content']", -"//section[@id='product-grid']", -"//ul[@data-testid='product-list']" -] -} -}; -await smartuiSnapshot(driver, 'Product Page', options); -``` - -**Java (Selenium)** -```java -ArrayList xpaths = new ArrayList<>(); -xpaths.add("//div[@class='main-content']"); -xpaths.add("//section[@id='product-grid']"); -xpaths.add("//ul[@data-testid='product-list']"); -selectDOM.put("xpath", xpaths); -``` - -**Python (Selenium)** -```python -options = { -"selectDOM": { -"xpath": [ -"//div[@class='main-content']", -"//section[@id='product-grid']", -"//ul[@data-testid='product-list']" -] -} -} -``` - -**Combining ignoreDOM and selectDOM** - -You can combine both `ignoreDOM` and `selectDOM` in the same options object. When both are specified, `selectDOM` is applied first to include elements, then `ignoreDOM` is applied to exclude specific elements from the selected set. - -```javascript -let options = { -selectDOM: { -cssSelector: [".main-content"] -}, -ignoreDOM: { -id: ["timestamp", "user-id"] -} -}; -await smartuiSnapshot(driver, 'Home Page', options); -``` - -**Use Cases** - -**Use Case 1: E-commerce Product Page** - -**Scenario**: Product pages display prices, stock counts, and user reviews that change frequently. - -**Solution**: Ignore dynamic pricing and review elements while comparing the overall layout. - -```javascript -let options = { -ignoreDOM: { -cssSelector: [ -".price", -".stock-count", -".review-count", -"[data-testid='timestamp']" -] -} -}; -await smartuiSnapshot(driver, 'Product Page', options); -``` - -**Use Case 2: Dashboard with Real-time Data** - -**Scenario**: Dashboard displays live metrics, timestamps, and user-specific data. - -**Solution**: Select only the static layout elements for comparison. - -```javascript -let options = { -selectDOM: { -cssSelector: [ -".dashboard-layout", -".navigation", -".sidebar" -] -}, -ignoreDOM: { -class: ["metric-value", "timestamp", "user-info"] -} -}; -await smartuiSnapshot(driver, 'Dashboard', options); -``` - -**Use Case 3: News Article Page** - -**Scenario**: Article pages have timestamps, author info, and related articles that change. - -**Solution**: Ignore dynamic metadata while comparing article content. - -```javascript -let options = { -ignoreDOM: { -id: ["article-timestamp", "author-info"], -class: ["related-articles", "social-share-count"] -} -}; -await smartuiSnapshot(driver, 'Article Page', options); -``` - -**Best Practices** - -1. **Use Specific Selectors**: Prefer IDs or data attributes over generic class names for more precise targeting. - -2. **Test Selectors First**: Verify your selectors work correctly before using them in visual tests. - -3. **Document Your Choices**: Document why certain elements are ignored or selected for future reference. - -4. **Combine Strategically**: Use `selectDOM` for broad filtering and `ignoreDOM` for fine-tuning. - -5. **Avoid Over-ignoring**: Only ignore elements that are truly dynamic. Over-ignoring can hide real UI issues. - -6. **Use Data Attributes**: Add `data-testid` attributes to elements you need to target for better test stability. - -## Troubleshooting - -**Issue: Elements Not Being Ignored** - -**Possible Causes:** -- Selector is incorrect or doesn't match any elements -- Element is loaded dynamically after the snapshot -- Selector syntax error - -**Solutions:** -1. Verify the selector using browser DevTools -2. Add a wait before taking the snapshot to ensure elements are loaded -3. Check selector syntax (CSS selectors vs XPath) - -```javascript -// Wait for element before snapshot -await driver.wait(until.elementLocated(By.id('timestamp')), 5000); -let options = { -ignoreDOM: { -id: ["timestamp"] -} -}; -await smartuiSnapshot(driver, 'Page', options); -``` - -**Issue: selectDOM Not Working** - -**Possible Causes:** -- No elements match the selectors -- Elements are not visible at snapshot time -- Selector targets elements outside the viewport - -**Solutions:** -1. Verify elements exist and are visible -2. Use more specific selectors -3. Ensure elements are in the viewport - -**Issue: False Positives Still Occurring** - -**Possible Causes:** -- Not all dynamic elements are ignored -- Elements change structure, not just content -- Timing issues with element loading - -**Solutions:** -1. Review the diff to identify missed dynamic elements -2. Add more selectors to ignoreDOM -3. Increase wait times or use explicit waits - -## Additional Resources - -- [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) - Detailed solutions for common issues -- [SmartUI SDK Configuration Options](/support/docs/smartui-sdk-config-options) -- [Handling Videos](/support/docs/smartui-handle-videos) -- [Handling Lazy Loading](/support/docs/smartui-handle-lazy-loading) -- [Smart Ignore Feature](/support/docs/smartui-smartignore) -- [Project Settings](/support/docs/smartui-project-settings) - Configure pixel thresholds and comparison settings -- [Baseline Management](/support/docs/smartui-baseline-management) - Learn how to manage baselines effectively diff --git a/static/docs/smartui-handle-lazy-loading.md b/static/docs/smartui-handle-lazy-loading.md deleted file mode 100644 index 9aa1c9117..000000000 --- a/static/docs/smartui-handle-lazy-loading.md +++ /dev/null @@ -1,316 +0,0 @@ -# Handle Pages with Lazy Loading - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## What is Lazy Loading? - -Lazy loading is a web performance optimization technique where content (images, videos, or other elements) is loaded only when it's about to enter the viewport or when the user scrolls near it. While this improves page load performance, it can cause issues in visual regression testing because elements may not be loaded when the snapshot is captured. - -## The Challenge - -When taking full-page screenshots of pages with lazy loading: - -- Elements below the fold may not be loaded -- Images may appear as placeholders -- Content may be missing from snapshots -- False positives can occur due to incomplete page rendering - -## Solutions - -SmartUI provides multiple approaches to handle lazy-loaded content: - -### Method 1: Using waitForTimeout (Recommended for Simple Cases) - -The `waitForTimeout` option in your SmartUI configuration file adds a delay before capturing screenshots, giving lazy-loaded content time to load. - -#### Configuration File Approach - -Add `waitForTimeout` to your `.smartui.json` file: - -```json -{ -"web": { -"browsers": ["chrome"], -"viewports": [[1920, 1080]] -}, -"waitForTimeout": 3000 -} -``` - -**How it works:** -- Waits for the specified time (in milliseconds) after page load -- Allows JavaScript to trigger lazy loading -- Simple and effective for pages with consistent loading times - -**When to use:** -- Pages with predictable loading times -- Simple lazy loading implementations -- When you know approximate load times - -**Limitations:** -- Fixed wait time may be too short or too long -- Doesn't wait for specific elements to load -- May not work for complex lazy loading scenarios - -### Method 2: Programmatic Scrolling (Recommended for Complex Cases) - -For pages with extensive lazy loading or infinite scroll, programmatically scroll through the page to trigger all lazy-loaded content before capturing the screenshot. - -#### JavaScript (Selenium) Example - -```javascript -const { Builder, By, until } = require('selenium-webdriver'); -const { smartuiSnapshot } = require('@lambdatest/selenium-driver'); - -(async function example() { -let driver = await new Builder().forBrowser("chrome").build(); - -try { -await driver.get("https://example.com"); - -// Function to scroll through the entire page -async function scrollToLoadLazyContent() { -// Get the total page height -let totalHeight = await driver.executeScript("return document.body.scrollHeight"); -let viewportHeight = await driver.executeScript("return window.innerHeight"); - -// Scroll in increments -let scrollStep = 500; -let currentPosition = 0; - -while (currentPosition < totalHeight) { -// Scroll down -await driver.executeScript(`window.scrollTo(0, ${currentPosition})`); - -// Wait for lazy content to load -await new Promise(resolve => setTimeout(resolve, 1000)); - -// Update position -currentPosition += scrollStep; - -// Recalculate total height (in case of infinite scroll) -let newHeight = await driver.executeScript("return document.body.scrollHeight"); -if (newHeight > totalHeight) { -totalHeight = newHeight; -} -} - -// Scroll back to top -await driver.executeScript("window.scrollTo(0, 0)"); - -// Final wait for any remaining content -await new Promise(resolve => setTimeout(resolve, 2000)); -} - -// Scroll to load all lazy content -await scrollToLoadLazyContent(); - -// Take snapshot -await smartuiSnapshot(driver, "Lazy Loaded Page"); - -} finally { -await driver.quit(); -} -})(); -``` - -#### Java (Selenium) Example - -```java - -public void handleLazyLoading(WebDriver driver) throws InterruptedException { -driver.get("https://example.com"); - -JavaScriptExecutor js = (JavaScriptExecutor) driver; - -// Get total page height -Long totalHeight = (Long) js.executeScript("return document.body.scrollHeight"); -Long viewportHeight = (Long) js.executeScript("return window.innerHeight"); - -int scrollStep = 500; -long currentPosition = 0; - -// Scroll through the page -while (currentPosition < totalHeight) { -js.executeScript("window.scrollTo(0, " + currentPosition + ")"); -Thread.sleep(1000); // Wait for lazy content - -currentPosition += scrollStep; - -// Recalculate height for infinite scroll -Long newHeight = (Long) js.executeScript("return document.body.scrollHeight"); -if (newHeight > totalHeight) { -totalHeight = newHeight; -} -} - -// Scroll back to top -js.executeScript("window.scrollTo(0, 0)"); -Thread.sleep(2000); - -// Take snapshot -SmartUISnapshot.smartuiSnapshot(driver, "Lazy Loaded Page"); -} -``` - -#### Python (Selenium) Example - -```python -from selenium import webdriver -from selenium.webdriver.common.by import By -from lambdatest import smartui_snapshot -import time - -driver = webdriver.Chrome() -driver.get("https://example.com") - -def scroll_to_load_lazy_content(driver): -# Get total page height -total_height = driver.execute_script("return document.body.scrollHeight") -viewport_height = driver.execute_script("return window.innerHeight") - -scroll_step = 500 -current_position = 0 - -# Scroll through the page -while current_position < total_height: -driver.execute_script(f"window.scrollTo(0, {current_position})") -time.sleep(1) # Wait for lazy content - -current_position += scroll_step - -# Recalculate height for infinite scroll -new_height = driver.execute_script("return document.body.scrollHeight") -if new_height > total_height: -total_height = new_height - -# Scroll back to top -driver.execute_script("window.scrollTo(0, 0)") -time.sleep(2) - -# Scroll to load all lazy content -scroll_to_load_lazy_content(driver) - -# Take snapshot -smartui_snapshot(driver, "Lazy Loaded Page") -``` - -### Method 3: Wait for Specific Elements - -For pages where you know which elements are lazy-loaded, wait for those specific elements before taking the snapshot. - -#### JavaScript Example - -```javascript -const { Builder, By, until } = require('selenium-webdriver'); -const { smartuiSnapshot } = require('@lambdatest/selenium-driver'); - -let driver = await new Builder().forBrowser("chrome").build(); -await driver.get("https://example.com"); - -// Wait for lazy-loaded images to load -await driver.wait(until.elementsLocated(By.css('img[data-src]')), 10000); - -// Trigger lazy loading by scrolling -let images = await driver.findElements(By.css('img[data-src]')); -for (let img of images) { -await driver.executeScript("arguments[0].scrollIntoView(true);", img); -await driver.sleep(500); -} - -// Wait for images to actually load -await driver.sleep(3000); - -await smartuiSnapshot(driver, "Lazy Loaded Images"); -``` - -## Best Practices - -**Choose the Right Method** - -- **Simple lazy loading**: Use `waitForTimeout` in configuration -- **Complex lazy loading**: Use programmatic scrolling -- **Known lazy elements**: Wait for specific elements - -**Optimize Wait Times** - -- Start with shorter wait times and increase if needed -- Balance between thorough loading and test execution time -- Monitor test execution times to optimize - -**Scroll Incrementally** - -- Use smaller scroll steps (200-500px) for better coverage -- Wait between scrolls to allow content to load -- Recalculate page height for infinite scroll scenarios - -**Return to Top** - -- Always scroll back to top after loading lazy content -- Ensures consistent baseline for comparison -- Prevents viewport-dependent differences - -**Combine Methods** - -- Use `waitForTimeout` for initial page load -- Add programmatic scrolling for lazy content -- Wait for specific critical elements - -**Combine Methods** - -- Use `waitForTimeout` for initial page load -- Add programmatic scrolling for lazy content -- Wait for specific critical elements - -## Use Cases - -### Use Case 1: Image Gallery with Lazy Loading - -**Scenario**: Image gallery that loads images as user scrolls. - -**Solution**: Scroll through the gallery to trigger all image loads before snapshot. - -### Use Case 2: Infinite Scroll Feed - -**Scenario**: Social media feed that loads more content on scroll. - -**Solution**: Scroll to bottom, wait for new content, then scroll back to top. - -### Use Case 3: Product Listing Page - -**Scenario**: E-commerce page with products that load as you scroll. - -**Solution**: Scroll through the page to load all product images and details. - -## Troubleshooting - -**Issue: Content Still Missing After Scrolling** - -**Solutions:** -1. Increase wait times between scrolls -2. Add explicit waits for lazy-loaded elements -3. Check if lazy loading uses intersection observer (may need different approach) - -**Issue: Test Takes Too Long** - -**Solutions:** -1. Reduce scroll step size but increase scroll speed -2. Only scroll to areas that matter for your test -3. Use `waitForTimeout` instead of scrolling when possible - -**Issue: Inconsistent Results** - -**Solutions:** -1. Ensure consistent scroll pattern -2. Always scroll back to top -3. Add sufficient wait times -4. Use fixed viewport sizes - -## Additional Resources - -- [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) - Solutions for lazy loading issues -- [SmartUI Configuration Options](/support/docs/smartui-sdk-config-options) - Configure `waitForTimeout` and other options -- [Handling Dynamic Data](/support/docs/smartui-handle-dynamic-data) - Handle other dynamic content types -- [Handling Videos](/support/docs/smartui-handle-videos) - Handle video content in tests -- [Handling Sticky Elements](/support/docs/smartui-handle-sticky-elements) - Handle sticky headers/footers -- [Project Settings](/support/docs/smartui-project-settings) - Configure comparison settings diff --git a/static/docs/smartui-handle-sticky-elements.md b/static/docs/smartui-handle-sticky-elements.md deleted file mode 100644 index e14fb83fe..000000000 --- a/static/docs/smartui-handle-sticky-elements.md +++ /dev/null @@ -1,221 +0,0 @@ -# Handle Pages with Sticky Elements - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## What are Sticky Elements? - -Sticky elements are CSS-positioned elements that remain fixed in the viewport while the user scrolls through a page. Common examples include: - -- **Sticky Headers**: Navigation bars that stay at the top while scrolling -- **Sticky Footers**: Footer elements that remain at the bottom -- **Sticky Sidebars**: Side navigation or widgets that stay visible -- **Floating Action Buttons**: Buttons that float over content -- **Cookie Banners**: Consent banners that stick to the viewport - -## The Challenge - -When taking full-page screenshots of pages with sticky elements: - -- Sticky elements can appear multiple times in a single screenshot -- Elements may overlap with content at different scroll positions -- False positives can occur due to element repetition -- Inconsistent positioning can cause comparison failures - -## How SmartUI Handles Sticky Elements - -SmartUI automatically handles sticky elements by: - -### 1. DOM Modification Before Capture - -SmartUI automatically adjusts the positions of sticky elements in the Document Object Model (DOM) before capturing a screenshot. This ensures that: - -- Sticky elements are displayed only once in visual snapshots -- Elements appear at their original position (typically top or bottom of page) -- No overlapping or repetition occurs in full-page screenshots - -### 2. Full-Page Rendering - -When capturing full-page screenshots, SmartUI renders the page as it would appear when initially loaded in the browser: - -- Sticky headers appear only at the top of the page -- Sticky footers appear only at the bottom of the page -- Elements don't repeat throughout the page length -- Consistent baseline for visual comparison - -## Benefits - -### Consistency Across Snapshots - -Sticky elements are handled consistently across all snapshots, ensuring: -- Predictable element positioning -- Reliable visual comparisons -- Reduced false positives - -### Reduced False Positives - -By preventing element repetition and overlap: -- Only actual UI changes are detected -- Sticky element movement doesn't trigger false positives -- More accurate test results - -### Stable Visual Baseline - -Sticky elements remain static in their original positions: -- Consistent baseline for comparison -- Easier to identify real changes -- Better test reliability - -## Use Cases - -### Use Case 1: E-commerce Site with Sticky Header - -**Scenario**: E-commerce site has a sticky navigation header with cart icon and search bar. - -**Solution**: SmartUI automatically positions the header only at the top, preventing it from appearing multiple times in full-page screenshots. - -### Use Case 2: Blog with Sticky Sidebar - -**Scenario**: Blog page has a sticky sidebar with social sharing buttons and related articles. - -**Solution**: The sidebar appears only in its original position, not repeated throughout the page. - -### Use Case 3: Dashboard with Floating Action Button - -**Scenario**: Dashboard has a floating "Add" button that stays visible while scrolling. - -**Solution**: SmartUI handles the floating button, ensuring it doesn't cause false positives in comparisons. - -### Use Case 4: Cookie Consent Banner - -**Scenario**: Site has a sticky cookie consent banner at the bottom of the page. - -**Solution**: The banner appears only at the bottom, not repeated in the middle of the page. - -## Configuration - -SmartUI handles sticky elements automatically with **no additional configuration required**. The feature works out of the box for: - -- Elements with `position: sticky` CSS property -- Elements with `position: fixed` CSS property -- Floating elements that remain in viewport - -### When Manual Handling is Needed - -In rare cases, you may need to manually handle sticky elements: - -#### Option 1: Ignore Sticky Elements - -If you want to exclude sticky elements from comparison entirely: - -```javascript -let options = { -ignoreDOM: { -cssSelector: [".sticky-header", ".sticky-footer", ".floating-button"] -} -}; -await smartuiSnapshot(driver, 'Page with Sticky Elements', options); -``` - -#### Option 2: Select Only Static Content - -If you only want to compare non-sticky content: - -```javascript -let options = { -selectDOM: { -cssSelector: [".main-content", ".article-body"] -} -}; -await smartuiSnapshot(driver, 'Content Only', options); -``` - -## Best Practices - -**Trust Automatic Handling** - -SmartUI's automatic handling works for most cases. Only use manual configuration if you encounter specific issues. - -**Test Sticky Element Pages** - -Verify that pages with sticky elements work correctly with SmartUI before running full test suites. - -**Review Snapshots** - -Occasionally review captured snapshots to ensure sticky elements are handled correctly. - -**Combine with Other Features** - -You can combine sticky element handling with other SmartUI features like dynamic data handling. - -**Combine with Other Features** - -You can combine sticky element handling with other SmartUI features like dynamic data handling. - -## Troubleshooting - -**Issue: Sticky Elements Still Causing False Positives** - -**Symptoms**: Sticky elements appear to cause differences in comparisons - -**Solutions**: -1. Verify the element is actually sticky (check CSS `position` property) -2. Use `ignoreDOM` to exclude problematic sticky elements -3. Check if element has dynamic content (may need dynamic data handling) - -```javascript -// Ignore sticky element with dynamic content -let options = { -ignoreDOM: { -cssSelector: [".sticky-header .notification-badge"] -} -}; -await smartuiSnapshot(driver, 'Page', options); -``` - -**Issue: Sticky Elements Not Visible in Snapshots** - -**Symptoms**: Sticky elements don't appear in captured screenshots - -**Solutions**: -1. Check if element is loaded via JavaScript (may need wait time) -2. Verify element is in the DOM before snapshot -3. Use `waitForTimeout` if element loads dynamically - -```javascript -// Wait for sticky element to load -await driver.wait(until.elementLocated(By.css('.sticky-header')), 10000); -await smartuiSnapshot(driver, 'Page'); -``` - -**Issue: Sticky Elements Overlapping Content** - -**Symptoms**: Sticky elements appear to overlap with page content - -**Solutions**: -1. This is normal behavior - SmartUI positions elements at their original location -2. If overlap causes issues, use `ignoreDOM` for the sticky element -3. Consider using viewport screenshots instead of full-page for specific tests - -## Limitations - -### JavaScript-Controlled Sticky Elements - -- Elements that become sticky via JavaScript (not CSS) may need special handling -- Ensure JavaScript has executed before taking snapshots - -### Dynamic Sticky Elements - -- Sticky elements with dynamic content may need additional handling -- Combine with dynamic data handling features if needed - -### Complex Sticky Behaviors - -- Very complex sticky behaviors (multiple sticky elements, conditional stickiness) may need manual configuration -- Test thoroughly before relying on automatic handling - -## Additional Resources - -- [Handling Dynamic Data](/support/docs/smartui-handle-dynamic-data) -- [Handling Videos](/support/docs/smartui-handle-videos) -- [Handling Lazy Loading](/support/docs/smartui-handle-lazy-loading) -- [SmartUI Configuration Options](/support/docs/smartui-sdk-config-options) diff --git a/static/docs/smartui-handle-videos.md b/static/docs/smartui-handle-videos.md deleted file mode 100644 index bf8628df6..000000000 --- a/static/docs/smartui-handle-videos.md +++ /dev/null @@ -1,203 +0,0 @@ -# Handle Pages with Videos - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Overview - -Web pages often contain video content that plays dynamically, changing frame by frame. In visual regression testing, this dynamic content can cause false positives because each frame of the video appears different, even when the actual page design and layout remain unchanged. - -SmartUI automatically handles video content by capturing the first frame of videos during visual comparison, ensuring that dynamic video playback doesn't interfere with your visual regression tests. - -## How SmartUI Handles Videos - -### Automatic First Frame Capture - -When SmartUI encounters a video element on a page, it automatically: - -1. **Captures the First Frame**: Extracts the initial frame of the video as a static image -2. **Treats as Static Content**: Uses this first frame as a stable reference point for comparison -3. **Ignores Playback Changes**: Subsequent frames and playback changes are not considered in the comparison - -This approach ensures that: -- Video content doesn't cause false positives -- The page layout and design are accurately compared -- Test results focus on actual UI changes, not video playback - -### Process Flow - -``` -Page Load → Video Detection → First Frame Extraction → Static Image Comparison → Results -``` - -## Benefits - -### Accurate Visual Testing - -By capturing only the first frame, SmartUI ensures that visual regression tests focus on static layout and design elements rather than dynamic video content. This provides more reliable and meaningful test results. - -### Efficient Workflow - -No manual configuration or custom scripts are required. SmartUI automatically handles video content, reducing the need for: -- Custom video handling code -- Manual video pausing scripts -- Complex workarounds for dynamic content - -### Improved Test Reliability - -False positives caused by video playback variations are eliminated, resulting in: -- More stable test results -- Reduced manual review time -- Higher confidence in test outcomes - -## Use Cases - -### Use Case 1: Product Demo Pages - -**Scenario**: Product pages feature embedded demo videos that auto-play or change frames. - -**Solution**: SmartUI automatically captures the first frame, allowing you to focus on comparing the page layout, product information, and static elements without video interference. - -### Use Case 2: Marketing Landing Pages - -**Scenario**: Landing pages include hero videos or background videos that play continuously. - -**Solution**: The first frame is captured, ensuring consistent comparison of the overall page design, call-to-action buttons, and static content. - -### Use Case 3: Video Gallery Pages - -**Scenario**: Pages with multiple video thumbnails or video galleries where thumbnails may change. - -**Solution**: SmartUI captures the initial state of all videos, providing stable baseline for comparing gallery layouts and page structure. - -### Use Case 4: Educational Content Pages - -**Scenario**: Educational platforms with embedded tutorial videos or course previews. - -**Solution**: First frame capture ensures that course layouts, navigation, and static content are accurately compared while ignoring video playback variations. - -## Configuration - -SmartUI handles videos automatically with no additional configuration required. However, you can enhance video handling by: - -### Using waitForTimeout - -If videos take time to load, you can add a wait timeout to ensure the first frame is captured correctly: - -```json -{ -"web": { -"browsers": ["chrome"], -"viewports": [[1920, 1080]] -}, -"waitForTimeout": 3000 -} -``` - -### Combining with Other Features - -You can combine video handling with other SmartUI features: - -**Example: Ignoring video controls while keeping the first frame** - -```javascript -let options = { -ignoreDOM: { -cssSelector: [".video-controls", ".play-button"] -} -}; -await smartuiSnapshot(driver, 'Video Page', options); -``` - -## Limitations and Considerations - -### Supported Video Formats - -SmartUI handles standard HTML5 video elements: -- `` tags with standard formats (MP4, WebM, OGG) -- Embedded videos via iframe (YouTube, Vimeo, etc.) -- Video elements with autoplay attributes - -### First Frame Consistency - -- The first frame captured may vary slightly if videos have different loading times -- Videos with preload="none" may not have a first frame immediately available -- Consider using `waitForTimeout` for videos that load slowly - -### Video Overlays - -- Video controls, overlays, and UI elements are included in the snapshot -- Use `ignoreDOM` if you need to exclude video controls from comparison -- Overlays that appear after the first frame are not captured - -### Autoplay Videos - -- Autoplay videos are handled the same way as regular videos -- The first frame is captured regardless of autoplay status -- Videos that start playing immediately still have their first frame captured - -## Best Practices - -**Allow Video Loading** - -Use appropriate `waitForTimeout` values to ensure videos are loaded before snapshots. - -**Test Video Pages** - -Verify that video pages work correctly with SmartUI before running full test suites. - -**Combine with Other Features** - -Use `ignoreDOM` to exclude video controls if needed. - -**Document Video Content** - -Note which pages contain videos for better test planning. - -**Review First Frames** - -Occasionally review captured first frames to ensure they represent the intended baseline. - -## Troubleshooting - -**Issue: Videos Not Captured Correctly** - -**Symptoms**: Video areas appear blank or incorrect in snapshots - -**Solutions**: -1. Increase `waitForTimeout` to allow videos to load -2. Check if videos are loaded via JavaScript (may need explicit waits) -3. Verify video elements are present in the DOM before snapshot - -```javascript -// Wait for video to load -await driver.wait(until.elementLocated(By.tagName('video')), 10000); -await driver.sleep(2000); // Additional wait for first frame -await smartuiSnapshot(driver, 'Video Page'); -``` - -**Issue: False Positives from Video Areas** - -**Symptoms**: Video areas show differences even when page design is unchanged - -**Solutions**: -1. Verify SmartUI is capturing first frames (check baseline images) -2. Use `ignoreDOM` to exclude video elements if first frame capture isn't sufficient -3. Consider using layout comparison mode for pages with many videos - -**Issue: Embedded Videos (YouTube, Vimeo)** - -**Symptoms**: Embedded videos via iframe may not be captured correctly - -**Solutions**: -1. Embedded videos in iframes are handled automatically -2. If issues persist, consider using `ignoreDOM` for iframe areas -3. Check if iframe content is accessible (CORS policies may affect this) - -## Additional Resources - -- [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) - Solutions for video-related issues -- [Handling Dynamic Data](/support/docs/smartui-handle-dynamic-data) - Handle other dynamic content types -- [Handling Lazy Loading](/support/docs/smartui-handle-lazy-loading) - Handle lazy-loaded content -- [SmartUI Configuration Options](/support/docs/smartui-sdk-config-options) - Configure `waitForTimeout` and other options -- [Smart Ignore Feature](/support/docs/smartui-smartignore) - Automatically ignore layout shifts -- [Project Settings](/support/docs/smartui-project-settings) - Configure comparison settings and thresholds diff --git a/static/docs/smartui-hooks-element-screenshot.md b/static/docs/smartui-hooks-element-screenshot.md deleted file mode 100644 index bbf095404..000000000 --- a/static/docs/smartui-hooks-element-screenshot.md +++ /dev/null @@ -1,219 +0,0 @@ -# Take a Screenshot of a Specific Element on a Webpage (SmartUI Hooks) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Use this guide when you run automated web tests on LambdaTest and want SmartUI to capture a visual baseline for a specific part of the page, such as a component, card, table, or section, instead of the full browser window. - -## Before You Start - -Make sure you have: - -- A LambdaTest account with Web Automation access, such as Selenium on the LambdaTest Grid. -- SmartUI enabled for the session. Your SmartUI project and build must be configured on the test session. -- A locator for the target element, such as a CSS selector, XPath, or HTML `id`. You can also pass an already-resolved element handle, see [Capture by Resolved Element Handle](#capture-by-resolved-element-handle-webelement). - -Do not store usernames or access keys in your source repository. Use environment variables or your CI secret manager instead. - -## Step 1: Open the Page in Your Test - -In your test script, navigate to the target URL and wait until the UI is fully loaded. - -Use explicit waits where possible so the element is present and stable before capture. - -## Step 2: Scroll the Element Into View - -Before taking the screenshot, scroll the target element into the visible area of the page. - -This helps ensure the correct region is ready for capture. Use your framework's normal scrolling method, such as JavaScript `scrollIntoView()` or your Selenium helper. - -## Step 3: Call the SmartUI Element Screenshot Hook - -Call the SmartUI hook through your driver's script executor, passing `"smartui.takeScreenshot"` as the command and a config object with your screenshot options: - -```js -const config = { -screenshotName: "Checkout_Summary_Block", -elementType: "css_selector", -element: "section.checkout-summary", -fullPage: false -}; - -await driver.executeScript("smartui.takeScreenshot", config); -``` - -The config object must include at least these fields: - -| Field | Purpose | -|---|---| -| `screenshotName` | Stable name for the screenshot in SmartUI. Used for baselines and comparisons. | -| `elementType` | Locator type. Supported values: `css_selector`, `xpath`, `id`, `class`, `webElement` | -| `element` | Locator value for the target element. When `elementType` is `webElement`, this is a resolved element handle instead of a locator string. See [Capture by Resolved Element Handle](#capture-by-resolved-element-handle-webelement). | -| `fullPage` | Set to `false` to capture only the target element | - -Update `elementType` and `element` to match the locator used in your test. - -## Step 4: Repeat for Other Components - -If you want to capture more than one component, call the hook again with a different `screenshotName` for each one. - -Keep screenshot names stable across runs so SmartUI compares against the correct baseline. - -## Capture by Resolved Element Handle (webElement) - -The locator-based flow above re-resolves your selector at the moment the screenshot runs. On pages where the DOM is rewritten after you locate the element, for example a workspace that rebuilds its layout every time a new tab is added, the selector can go stale before capture and the element screenshot fails. - -To avoid this, set `elementType` to `webElement` and pass an already-resolved element reference as `element`. SmartUI uses that live handle directly and skips locator re-resolution, so the capture stays reliable even when the surrounding DOM changes after the element was located. - -When `elementType` is `webElement`, you pass a real element object, not a string. Call the hook with the command name and a config object as two separate arguments so your automation framework serializes the element handle correctly. Do not use the `smartui.takeScreenshot,` string form for `webElement`, because a serialized string cannot carry a live element reference. - -First resolve the element in your test, then pass it to the hook: - -```javascript -const el = await driver.findElement(By.className('hero-heading')); - -const config = { -screenshotName: 'region-screenshot', -elementType: 'webElement', -element: el -}; - -await driver.executeScript('smartui.takeScreenshot', config); -``` - -| Field | Purpose | -|---|---| -| `screenshotName` | Stable name for the captured screenshot in SmartUI. | -| `elementType` | Set to `webElement` to pass a resolved element handle instead of a locator. | -| `element` | The resolved element reference to scope the capture to, for example the return value of `driver.findElement(...)`. | - -Use `webElement` when the element is present and stable when you locate it, but the page mutates the DOM before the screenshot runs. If your locator stays valid through capture, the locator-based flow in Step 3 is simpler. - -## Optional: Capture Many Elements Automatically - -If you want to capture many elements from the same page, SmartUI does not provide a single built-in hook that automatically splits the whole page into separate element screenshots. - -Instead, you can use a test-side workflow: - -1. Collect visible elements in the page with JavaScript. -2. Generate a locator for each element, such as XPath. -3. Loop through that list in your test. -4. Call `smartui.takeScreenshot` once per element. - -This is useful when you want to build a component inventory for a page without manually adding every selector one by one. - -### Recommended Guardrails - -- Limit the total number of elements you capture in one run. -- Skip hidden or very small elements. -- Prefer targeting a specific container like `main`, `.product-grid`, or `[data-testid='app-root']` instead of the entire DOM. -- Keep screenshot names deterministic so repeated runs stay comparable. - -### Example Pattern - -The following example collects visible elements, builds XPath locators, and returns metadata for the first `N` matches: - -```javascript -const elements = await driver.executeScript(` -function getPath(el) { -if (!el || el.nodeType !== 1) return ''; -if (el.id) return '//*[@id="' + el.id.replace(/"/g, '\\\\"') + '"]'; -if (el === document.body) return '/html/body'; -let ix = 0; -const siblings = el.parentNode ? el.parentNode.children : []; -for (let i = 0; i < siblings.length; i++) { -if (siblings[i] === el) { -return getPath(el.parentNode) + '/' + el.tagName.toLowerCase() + '[' + (ix + 1) + ']'; -} -if (siblings[i].tagName === el.tagName) ix++; -} -return ''; -} - -const out = []; -const nodes = Array.from(document.querySelectorAll('body *')); -const max = arguments[0]; - -for (let i = 0; i < nodes.length && out.length < max; i++) { -const n = nodes[i]; -const r = n.getBoundingClientRect(); -const st = window.getComputedStyle(n); - -if (r.width < 8 || r.height < 8) continue; -if (st.display === 'none' || st.visibility === 'hidden' || st.opacity === '0') continue; - -const xp = getPath(n); -if (!xp) continue; - -out.push({ -xpath: xp, -tag: n.tagName.toLowerCase(), -id: n.id || '', -cls: (n.className && String(n.className).split) ? String(n.className).split(/\\s+/)[0] : '', -idx: i -}); -} - -return out; -`, 25); -``` - -Then loop through the collected elements and upload one SmartUI element screenshot for each: - -```javascript -for (const item of elements) { -const label = item.id -? `${item.tag}_id_${item.id}` -: item.cls -? `${item.tag}_class_${item.cls}` -: `${item.tag}_idx_${item.idx}`; - -const screenshotName = `element_${label}`.replace(/[^a-zA-Z0-9_-]+/g, '_').slice(0, 80); - -await driver.executeScript("smartui.takeScreenshot", { -screenshotName, -elementType: 'xpath', -element: item.xpath, -fullPage: false -}); -} -``` - -### Best Use Cases - -- Capture all visible cards in a product grid. -- Capture all buttons or interactive controls on a page. -- Build a one-time baseline set for important components in a dashboard or design system page. - -### When to Avoid This - -- Very large pages with hundreds of nodes. -- Pages with highly dynamic content that changes on every load. -- Pages where stable component locators already exist and are easier to maintain manually. - -## Step 5: Find the Screenshot in SmartUI - -1. Open SmartUI from your LambdaTest account. -2. Select the project and build that match your test run. -3. Locate the screenshot using the `screenshotName`. - -## Tips - -- **Use unique locators:** Your selector should match only one main element. If it matches multiple nodes, the capture may be inconsistent. -- **Handle dynamic content carefully:** If the element contains content that changes every run, such as timers or ads, consider using ignore regions or layout comparison options if available in your SmartUI project. -- **Watch for large elements:** If the captured image appears cut off for a very tall or wide component, contact LambdaTest support with the session ID and `screenshotName`. - -## Quick Reference - -| Locator Type | `elementType` | Example `element` Value | -|---|---|---| -| CSS selector | `css_selector` | `main article:first-of-type` | -| XPath | `xpath` | `//div[@data-testid='invoice-panel']` | -| HTML `id` | `id` | `sidebar` | -| Resolved element handle | `webElement` | `await driver.findElement(By.className('hero-heading'))` | - -## Related Docs - -- [Selenium Hooks](/support/docs/selenium-visual-regression/) -- [Hooks Layout + Full Page](/support/docs/smartui-hooks-layout-fullpage-smartignore/) -- [Handle Dynamic Data](/support/docs/smartui-handle-dynamic-data/) -- [SmartUI Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide/) diff --git a/static/docs/smartui-hooks-fetch-status-timeout.md b/static/docs/smartui-hooks-fetch-status-timeout.md deleted file mode 100644 index 4c6c89e98..000000000 --- a/static/docs/smartui-hooks-fetch-status-timeout.md +++ /dev/null @@ -1,89 +0,0 @@ -# Auto-Wait and Retry for Screenshot Status - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The `smartui.fetchScreenshotStatus` hook now accepts an optional **`timeout`** option. With a timeout set, SmartUI **polls for the comparison result on the server side** and returns as soon as it is ready, instead of failing immediately when the screenshot is still processing. - -A screenshot goes through capture, upload, and comparison before its status exists. If your script calls `fetchScreenshotStatus` right after `smartui.takeScreenshot`, the result is often not ready yet and the call returns a `404`. Until now, the only way around this was a hand-written sleep-and-retry loop in every test, tuned by trial and error and different for every framework. The `timeout` option builds that wait into the hook itself: one argument replaces the custom loop, the call returns as soon as the result exists, and slow builds with large captures or many uploads no longer need their own polling code. - -## How it works - -| | | -|---|---| -| **Option** | `timeout` | -| **Type** | integer, seconds, optional | -| **Poll interval** | 5 seconds, fixed | -| **Maximum timeout** | 300 seconds | -| **Default (absent or `0`)** | Single lookup, same behavior as before | -| **Retry condition** | Retries only while the lookup returns `404` (result not ready yet). A `200` and real errors (`400`, `5xx`) return immediately | -| **On expiry** | Returns `404` with an enriched message that includes the number of attempts and the total seconds waited | - -The retry loop runs inside SmartUI, so your script makes a single hook call and blocks until the status is available or the timeout expires. - -## Usage - -Pass a config object with the `screenshotName` and the `timeout` in seconds. - -### Selenium - -```javascript -const result = await driver.executeScript("smartui.fetchScreenshotStatus", { -screenshotName: "checkout-page", -timeout: 120 -}); -console.log("status:", JSON.stringify(result, null, 2)); -``` - -```python -result = driver.execute_script("smartui.fetchScreenshotStatus", { -"screenshotName": "checkout-page", -"timeout": 120, -}) -print("status:", result) -``` - -```java -import java.util.HashMap; -import java.util.Map; -import org.openqa.selenium.JavascriptExecutor; - -Map config = new HashMap<>(); -config.put("screenshotName", "checkout-page"); -config.put("timeout", 120); - -Object result = ((JavascriptExecutor) driver) -.executeScript("smartui.fetchScreenshotStatus", config); -System.out.println(result); -``` - -### Playwright / Puppeteer - -```javascript -const result = await page.evaluate(_ => {}, `lambdatest_action: ${JSON.stringify({ -action: "smartui.fetchScreenshotStatus", -arguments: { -screenshotName: "checkout-page", -timeout: 120 -} -})}`); -console.log(result); -``` - -## Interaction with session `idleTimeout` - -- **Selenium:** `idleTimeout` applies between commands, so it has no impact on the hook's `timeout`. The polling happens inside a single command. -- **Playwright / Puppeteer (CDP):** the connection deadline is the session `idleTimeout` (default 300 seconds). This is why the maximum `timeout` is capped at 300 seconds. - -## Notes - -- Without `timeout` (or with `timeout: 0`) the hook behaves exactly as before: one lookup, and a `404` if the result is not ready yet. Existing scripts are unaffected. -- Pick a `timeout` that covers your slowest snapshot. Large full-page captures and builds with many parallel uploads take longer to compare. `120` seconds is a reasonable starting point. -- Only the "not ready yet" case is retried. Genuine failures surface immediately, so a misspelled `screenshotName` or a server error does not silently burn the whole timeout. -- On expiry the enriched `404` message tells you how many attempts were made and how long the hook waited, which makes it easy to distinguish "needs a larger timeout" from "screenshot never happened." - -## Related Docs - -- [Selenium Visual Regression](/support/docs/selenium-visual-regression/) -- [Playwright Visual Regression](/support/docs/playwright-visual-regression/) -- [SmartUI Hooks: Layout, Full-Page Screenshots, and Smart Ignore](/support/docs/smartui-hooks-layout-fullpage-smartignore/) -- [Take a Screenshot of a Specific Element (SmartUI Hooks)](/support/docs/smartui-hooks-element-screenshot/) diff --git a/static/docs/smartui-hooks-layout-fullpage-smartignore.md b/static/docs/smartui-hooks-layout-fullpage-smartignore.md deleted file mode 100644 index 150c7d781..000000000 --- a/static/docs/smartui-hooks-layout-fullpage-smartignore.md +++ /dev/null @@ -1,175 +0,0 @@ -# SmartUI Hooks: Layout, Full-Page Screenshots, and Smart Ignore - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Use this page when you run **SmartUI Hooks** on LambdaTest (for example, Selenium `executeScript` without the `smartui exec` CLI wrapper) and need **layout** comparison, **full-page** capture, or **Smart Ignore**. - -## How configuration is split - -SmartUI Hooks split configuration between the session capabilities (`LT:Options`) and the per-screenshot hook: - -| Goal | Where to configure | How | -|------|--------------------|-----| -| **Layout** comparison | `smartui.takeScreenshot` hook options | Pass `ignoreType: ["layout"]` in the options map for that screenshot. | -| **Full-page** capture | `smartui.takeScreenshot` hook options | Pass `fullPage: true` in the options map for that screenshot. | -| **Smart Ignore** | `LT:Options` | Set `smartUI.smartIgnore: true` on the session, for both baseline and comparison runs. | -| **Project** | `LT:Options` | Set `smartUI.project` (with `visual` and credentials). | - -Layout and full page are enabled per screenshot through the hook options, not through `LT:Options` capabilities. When you use Smart Ignore, choose either **Ignore DOM** or **Select DOM** in the dashboard for a given flow, not both. - -## 1. Session capabilities (`LT:Options`) - -Every Hooks session needs `username`, `accessKey`, `visual: true`, and `smartUI.project`. For Smart Ignore, also set `smartUI.smartIgnore: true` on both the baseline and comparison sessions. - -```java -import java.util.HashMap; -import org.openqa.selenium.chrome.ChromeOptions; - -ChromeOptions browserOptions = new ChromeOptions(); -HashMap ltOptions = new HashMap<>(); -ltOptions.put("username", System.getenv("LT_USERNAME")); -ltOptions.put("accessKey", System.getenv("LT_ACCESS_KEY")); -ltOptions.put("visual", true); -ltOptions.put("smartUI.project", "Your_Project_Name"); -ltOptions.put("smartUI.smartIgnore", true); - -browserOptions.setCapability("LT:Options", ltOptions); -``` - -```javascript -'LT:Options': { -user: process.env.LT_USERNAME, -accessKey: process.env.LT_ACCESS_KEY, -visual: true, -'smartUI.project': 'Your_Project_Name', -'smartUI.smartIgnore': true, -}, -``` - -```python -lt_options = { -"username": os.getenv("LT_USERNAME"), -"accessKey": os.getenv("LT_ACCESS_KEY"), -"visual": True, -"smartUI.project": "Your_Project_Name", -"smartUI.smartIgnore": True, -} -options.set_capability("LT:Options", lt_options) -``` - -```csharp -capabilities.SetCapability("visual", true); -capabilities.SetCapability("smartUI.project", "Your_Project_Name"); -capabilities.SetCapability("smartUI.smartIgnore", true); -``` - -## 2. Layout comparison - -Pass a single options map to `smartui.takeScreenshot` with a `screenshotName` and `ignoreType: ["layout"]`. Add `fullPage: true` to the same map for a full-page layout shot. - -```java -import java.util.Arrays; -import java.util.HashMap; -import java.util.Map; -import org.openqa.selenium.JavascriptExecutor; - -Map options = new HashMap<>(); -options.put("screenshotName", "my-layout-screenshot-01"); -options.put("ignoreType", Arrays.asList("layout")); - -((JavascriptExecutor) driver).executeScript("smartui.takeScreenshot", options); -``` - -```javascript -await driver.executeScript('smartui.takeScreenshot', { -screenshotName: 'my-layout-screenshot-01', -ignoreType: ['layout'], -}); -``` - -```python -driver.execute_script("smartui.takeScreenshot", { -"screenshotName": "my-layout-screenshot-01", -"ignoreType": ["layout"], -}) -``` - -```csharp -var options = new Dictionary -{ -{ "screenshotName", "my-layout-screenshot-01" }, -{ "ignoreType", new[] { "layout" } } -}; -((IJavaScriptExecutor)driver).ExecuteScript("smartui.takeScreenshot", options); -``` - -## 3. Full-page screenshot - -Pass `fullPage: true` to `smartui.takeScreenshot` along with a `screenshotName`. For a full-page layout shot in the same call, add `ignoreType: ["layout"]` to the map. - -```java -Map options = new HashMap<>(); -options.put("screenshotName", "Home_Page_Desktop"); -options.put("fullPage", true); - -((JavascriptExecutor) driver).executeScript("smartui.takeScreenshot", options); -``` - -```javascript -await driver.executeScript('smartui.takeScreenshot', { -screenshotName: 'Home_Page_Desktop', -fullPage: true, -}); -``` - -```python -driver.execute_script("smartui.takeScreenshot", { -"screenshotName": "Home_Page_Desktop", -"fullPage": True, -}) -``` - -```csharp -var options = new Dictionary -{ -{ "screenshotName", "Home_Page_Desktop" }, -{ "fullPage", true } -}; -((IJavaScriptExecutor)driver).ExecuteScript("smartui.takeScreenshot", options); -``` - -## 4. Baseline and comparison - -- Use the same `smartUI.project` and screenshot names on both runs. -- **Smart Ignore:** set the same `smartUI.smartIgnore` on the baseline and comparison sessions. -- **Layout:** pass the same `ignoreType: ["layout"]` in the hook for matching screenshot names. -- Changing the strategy or options usually requires a new baseline. - -## 5. Build attribution - -Runs that use a project token may show the project creator as the build author. Use the intended automation `username`, `accessKey`, and `project` where the product allows. - -## 6. Common failures and what not to do - -**Smart Ignore (Hooks + Java)** -These patterns do not turn on Smart Ignore: - -- `ltOptions.put("ignoreType", Arrays.asList("smartignore"));` without `smartUI.smartIgnore` -- `ltOptions.put("smartignore", true);` at the root of `LT:Options` - -Set `smartUI.smartIgnore: true` instead. - -| Problem | What to do | -|---------|------------| -| Layout never activates; only set in `LT:Options` | Move `ignoreType: ["layout"]` into the `smartui.takeScreenshot` options (see §2). | -| Smart Ignore never activates | Set `smartUI.smartIgnore: true` in `LT:Options`, then confirm it in the session metadata. | -| Tried `smartUI.layout` or nested `smartUI.options` layout blocks | These are not the Hooks switch for layout; use the hook options instead. | -| Strict (pixel) comparison still applies | Align the dashboard comparison mode with the session capabilities. | -| Relying on dashboard toggles only | Hooks still need the correct hook option and capability split shown above. | - -## Related Docs - -- [Layout Comparison in SmartUI SDK](/support/docs/smartui-layout-testing/) (SDK `smartuiSnapshot` path, different from Hooks). -- [Smart Ignore](/support/docs/smartui-smartignore/) -- [SmartUI SDK Config Options](/support/docs/smartui-sdk-config-options/) -- [Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide/) diff --git a/static/docs/smartui-hooks-region-ignore.md b/static/docs/smartui-hooks-region-ignore.md deleted file mode 100644 index e185d24dd..000000000 --- a/static/docs/smartui-hooks-region-ignore.md +++ /dev/null @@ -1,153 +0,0 @@ -# Ignore and Select Regions on SmartUI Web Hooks - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -On the **Web Hooks** path (Selenium with `driver.executeScript("smartui.takeScreenshot", config)`), you can define an ignore or select region in two ways: a **DOM selector** or raw **pixel coordinates**. Coordinate regions work the same way as selectors and need no extra setup. - -Both `ignoreDOM` and `selectDOM` support both input modes: - -- **`ignoreDOM`** excludes the given region(s) from comparison. -- **`selectDOM`** restricts the comparison to only the given region(s). - -This gives the Web Hooks path the same region controls already available on RD Hooks and the CLI SDK, so you can express dynamic areas the way that best fits your test: for example, an element matched by selector, or a fixed rectangle on the page. - -## 1. Ignore a region by coordinates - -Pass a `coordinates` entry (a list of `"x1,y1,x2,y2"` strings) under `ignoreDOM`. SmartUI excludes those rectangles from the comparison. The four values are the left, top, right, and bottom edges of the rectangle in pixels. - -For example, `"50,50,300,300"` defines the region whose top-left corner is at `(left=50, top=50)` and whose bottom-right corner is at `(right=300, bottom=300)`, a 250 × 250 rectangle. - -```java -import java.util.HashMap; -import java.util.Map; -import org.openqa.selenium.JavascriptExecutor; - -// x1,y1,x2,y2 -> left, top, right, bottom (pixels) -HashMap ignoreByCoord = new HashMap<>(); -ignoreByCoord.put("coordinates", new String[]{"847,185,1571,734"}); - -Map config = new HashMap<>(); -config.put("screenshotName", "home"); -config.put("ignoreDOM", ignoreByCoord); - -((JavascriptExecutor) driver).executeScript("smartui.takeScreenshot", config); -// the 847,185 -> 1571,734 rectangle is excluded from the comparison -``` - -```javascript -await driver.executeScript('smartui.takeScreenshot', { -screenshotName: 'home', -ignoreDOM: { -coordinates: ['847,185,1571,734'], // left, top, right, bottom -}, -}); -``` - -```python -driver.execute_script("smartui.takeScreenshot", { -"screenshotName": "home", -"ignoreDOM": { -"coordinates": ["847,185,1571,734"], # left, top, right, bottom -}, -}) -``` - -The origin the coordinates are measured from depends on `fullPage`: viewport-relative for a normal shot, full stitched-page-relative for a full-page shot. See [Coordinate space](#3-coordinate-space-viewport-vs-full-page). - -## 2. Select a region instead of ignoring it (`selectDOM`) - -Everything above works identically for `selectDOM`. Where `ignoreDOM` excludes a region, `selectDOM` restricts the comparison to only the given region(s). Both DOM selectors and `coordinates` are supported. - -```java -// compare ONLY this rectangle -HashMap selectByCoord = new HashMap<>(); -selectByCoord.put("coordinates", new String[]{"0,0,1280,200"}); -config.put("selectDOM", selectByCoord); - -// ...or compare only the region of a matched element -HashMap selectBySelector = new HashMap<>(); -selectBySelector.put("cssSelector", new String[]{"#price-table"}); -config.put("selectDOM", selectBySelector); -``` - -```javascript -// compare ONLY this rectangle -config.selectDOM = { coordinates: ['0,0,1280,200'] }; - -// ...or compare only the region of a matched element -config.selectDOM = { cssSelector: ['#price-table'] }; -``` - -```python -# compare ONLY this rectangle -config["selectDOM"] = {"coordinates": ["0,0,1280,200"]} - -# ...or compare only the region of a matched element -config["selectDOM"] = {"cssSelector": ["#price-table"]} -``` - -## 3. Coordinate space: viewport vs full page - -Coordinates are interpreted as absolute pixel coordinates **in the space of the produced screenshot image**. The origin depends on the `fullPage` option of the screenshot: - -| `fullPage` | Captured image | Coordinate origin `(0,0)` | -|------------|----------------|----------------------------| -| `false` (default) | Viewport / element shot | Top-left of the captured **viewport**; `y` within the visible area. | -| `true` | Stitched full-page shot | Top-left of the **full stitched page**; `y` increases down the entire scrollable page. | - -So for a full-page screenshot, supply coordinates relative to the whole page (a region below the fold has a large `top` / `bottom`), not relative to the current viewport. Selector regions need no such consideration; they are resolved live against the DOM, so they land correctly in either mode. - -## 4. Combine selectors and coordinates - -Within a single `ignoreDOM` (or `selectDOM`) the two input modes are **additive**: all resolved regions are combined. - -```java -Map ignore = new HashMap<>(); -ignore.put("cssSelector", new String[]{".promo"}); // selector -ignore.put("coordinates", new String[]{"847,185,1571,734"}); // coordinates - -config.put("ignoreDOM", ignore); -((JavascriptExecutor) driver).executeScript("smartui.takeScreenshot", config); -// both regions are ignored -``` - -```javascript -config.ignoreDOM = { -cssSelector: ['.promo'], // selector -coordinates: ['847,185,1571,734'], // coordinates -}; -await driver.executeScript('smartui.takeScreenshot', config); -// both regions are ignored -``` - -```python -config["ignoreDOM"] = { -"cssSelector": [".promo"], # selector -"coordinates": ["847,185,1571,734"], # coordinates -} -driver.execute_script("smartui.takeScreenshot", config) -# both regions are ignored -``` - -## Validation and errors - -Coordinates are validated at two levels. Note that only the **format** check happens on the hooks ingestion path; **page-bounds** elimination is a downstream comparison-engine behaviour, not part of the hooks request validation: - -| Level | Where | Checks | On failure | -|-------|-------|--------|------------| -| Format | Hooks ingestion | A `coordinates` entry must parse to **exactly four numeric components**, all **non-negative**, with `left < right` and `top < bottom`. | The `smartui.takeScreenshot` call returns a clear `400` error and the screenshot is **not taken** at all (the whole call is rejected, not just the one region). | -| Page bounds | Downstream (comparison engine) | The rectangle lies inside the rendered page / viewport. | A rectangle that falls outside the page is handled downstream; it simply produces no ignore/select box. This elimination is performed by the comparison engine, not by the hooks request validation. | - -## Notes - -- **Scope:** this applies to the Web Hooks path (Selenium `executeScript("smartui.takeScreenshot", ...)`). RD Hooks and the CLI SDK already support these inputs. -- **Empty or mixed configs:** a config containing only `coordinates` (no selector) is honoured; it is not treated as "no DOM region." Selector and coordinate regions in one `ignoreDOM` are additive. -- **Identical to selector-based ignore:** coordinate regions resolve to the same kind of ignored/selected box as selectors, so the comparison result matches an equivalent selector-based ignore. - -## Related Docs - -- [SmartUI Hooks: Layout, Full-Page Screenshots, and Smart Ignore](/support/docs/smartui-hooks-layout-fullpage-smartignore/) -- [Set SmartUI Comparison Modes at the Session Level](/support/docs/smartui-comparison-capabilities/) -- [Take a Screenshot of a Specific Element (SmartUI Hooks)](/support/docs/smartui-hooks-element-screenshot/) -- [Handling Dynamic Data with DOM Configuration and Options](/support/docs/html-dom-smartui-options/) -- [Selenium Visual Regression](/support/docs/selenium-visual-regression/) diff --git a/static/docs/smartui-iframes-and-embedded-content.md b/static/docs/smartui-iframes-and-embedded-content.md deleted file mode 100644 index de2a0b1aa..000000000 --- a/static/docs/smartui-iframes-and-embedded-content.md +++ /dev/null @@ -1,61 +0,0 @@ -# Iframes and Embedded Content in SmartUI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Pages often include **iframes**: embedded apps, chat widgets, consent managers, or **video players** (YouTube, Vimeo). SmartUI captures what the **browser paints** in your TestMu AI session. How much you can **inspect or target in the DOM** depends on whether the iframe is **same-origin** or **cross-origin** with the top page. - -## What to expect (summary) - -| Question | Short answer | -|----------|----------------| -| Are iframes supported? | **Yes**: snapshots reflect the **real browser output**, including iframe regions when they render. | -| Will third-party iframes always match pixel-for-pixel? | **No**: cross-origin embeds are **timing- and environment-dependent**; that is expected. | -| Can SmartUI read DOM inside another site’s iframe? | **No**: browsers block that for cross-origin content. | - -Embeds load asynchronously; third-party players (YouTube, Vimeo) may show different pixels between runs because of consent, ads, or regional UI. Use waits, **`ignoreDOM`** on the container, or layout comparison when the embed is out of scope. - -## Same-origin vs cross-origin - -| Case | What you can expect | -|------|---------------------| -| **Same-origin iframe** | The child document is part of your site’s origin. **Viewport / full-page** captures usually include the iframe’s rendered area. **Element locators** for nodes inside the iframe work from automation **after** you switch the driver into that frame (see below). | -| **Cross-origin iframe** | Browser security **blocks** parent JavaScript from reading the child document (similar to CORS). **Pixels** of the iframe’s **on-screen box** can still appear in screenshots if the embed renders. **DOM-based** tooling in the parent page **cannot** see inside the third-party document. | - -SmartUI does not override the browser’s security model; plan comparisons accordingly. - -## Video and media embeds - -For **``** elements and **embedded players** (often in iframes), SmartUI’s **first-frame** behavior and troubleshooting are documented here: - -- [Handle Pages with Videos](/support/docs/smartui-handle-videos/): includes guidance when **embedded videos via iframe** misbehave, **`ignoreDOM`** on the iframe region, and **CORS / accessibility** of iframe content. - -## Element screenshots and frame context - -When you use **SmartUI Hooks** to capture a **specific element** (for example [`smartui-hooks-element-screenshot`](/support/docs/smartui-hooks-element-screenshot/)), locators are resolved in the **current WebDriver browsing context**. - -- To capture a node **inside** an iframe, **switch into that frame** first (for example Selenium `driver.switchTo().frame(...)`), then run the hook against the element in that document. -- If you stay on the **top** document, selectors that only exist inside the iframe will not resolve. - -## Full-page and viewport captures - -**Full-page** and **viewport** screenshots reflect the **composed** page the browser draws. Same-origin iframes generally composite like any other content. Cross-origin embeds still draw a **rectangle**; what appears inside it depends on the embed loading, cookies, and network, so baselines can be **noisier** than static HTML. - -**Mitigations:** explicit waits, stable viewport size, and **`ignoreDOM`** (or annotations) on the iframe **container** when the embed is intentionally out of scope for the test. - -## Nested iframes - -Treat **nested** iframes like a stack of contexts: switch **in** level by level for inner elements, allow extra time for each document to load, and expect **more flakiness** when outer and inner origins differ. - -## Shadow DOM (not an iframe) - -**Shadow DOM** isolates markup inside a component but **same origin** as the host page. SmartUI’s Shadow DOM support is separate from iframe behavior: - -- [Shadow DOM](/support/docs/smartui-shadow-dom/) - -## Related docs - -- [Handle Pages with Videos](/support/docs/smartui-handle-videos/) -- [Take a Screenshot of a Specific Element (Hooks)](/support/docs/smartui-hooks-element-screenshot/) -- [Shadow DOM](/support/docs/smartui-shadow-dom/) -- [Handling Dynamic Data](/support/docs/smartui-handle-dynamic-data/) -- [Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide/) diff --git a/static/docs/smartui-ignore-colors.md b/static/docs/smartui-ignore-colors.md deleted file mode 100644 index c9945e28b..000000000 --- a/static/docs/smartui-ignore-colors.md +++ /dev/null @@ -1,94 +0,0 @@ -# Ignore Color-Only Changes in a Region (ignoreColors) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -`ignoreColors` marks regions where **color-only changes are ignored** during comparison, while **structural changes still flag**. It is the programmatic counterpart of the dashboard's **Ignore colors** box, available through both the SDK and the Lambda Hooks runtime. - -Teams that ship frequent theme swaps, brand-color rotations, or seasonal palettes usually face a bad trade-off: either every restyle floods the build with visual diffs that reviewers approve blindly, or the whole region gets ignored and real regressions slip through. `ignoreColors` removes that trade-off. The region stays under test for layout, text, and element changes, and only the recolor noise disappears. Reviews stay small, baselines stay meaningful, and a broken layout inside a restyled banner still fails the build. - -**Important** -Works only with the **Smart Ignore** comparison strategy. `ignoreColors` regions are honored by the smart-ignore engine. On the pixel-by-pixel comparison path they are dropped. Set your project or build comparison to Smart Ignore for these regions to take effect. See [Smart Ignore](/support/docs/smartui-smartignore/). - -## What it does - -Inside an `ignoreColors` region: - -- A **color-only change** (theme swap, brand-color rotation, seasonal palette) is **ignored**. -- A **structural change** (text, layout, element added or removed) is **still flagged**. - -Outside the region, everything is compared as usual. It composes with `ignoreDOM` and `selectDOM`. For example, you can select a region for comparison and ignore colors inside another. - -## SDK - -Pass `ignoreColors` in the snapshot options. All selector types are supported, along with coordinate rectangles and a whole-image switch. - -```javascript -await smartuiSnapshot(driver, "Home Page", { -ignoreColors: { -id: ["price-banner"], -class: ["seasonal-theme"], -cssSelector: ["#hero .cta"], -xpath: ["//div[@data-testid='promo']"], -coordinates: ["200,350,100,400"], // top,bottom,left,right -entireScreenshot: true // ignore colors across the whole image -} -}); -``` - -## Lambda Hooks - -### Selenium - -The hooks path additionally accepts a located `webElement` (or an array of them). - -```javascript -const el = await driver.findElement(By.id("price-banner")); - -await driver.executeScript("smartui.takeScreenshot", { -screenshotName: "checkout-page", -ignoreColors: { -id: ["price-banner"], -class: ["seasonal-theme"], -cssSelector: ["#nav .logo"], -xpath: ["//div[@data-testid='promo']"], -coordinates: ["100,200,400,350"], // left,top,right,bottom -webElement: el, // a located element, or an array of them -entireScreenshot: true // ignore colors across the whole image -} -}); -``` - -### Playwright / Puppeteer - -```javascript -await page.evaluate(`lambdatest_action: ${JSON.stringify({ -action: "smartui.takeScreenshot", -arguments: { -screenshotName: "checkout-page", -ignoreColors: { -id: ["price-banner"], -class: ["seasonal-theme"], -cssSelector: ["#nav .logo"], -xpath: ["//div[@data-testid='promo']"], -entireScreenshot: true // ignore colors across the whole image -} -} -})}`); -``` - -`coordinates` and `webElement` are not available on the Playwright / Puppeteer path. - -## Notes - -- **Smart Ignore only.** Regions have no effect on the pixel-comparison path. -- **`entireScreenshot: true`** covers the entire captured image (stitched full page, viewport, or element crop). It is additive. Combine it with selectors, or use it alone. -- **Coordinate order differs by path.** SDK: `top,bottom,left,right`. Hooks: `left,top,right,bottom`. Each entry is a comma-joined string of four numbers. -- **Composes with `selectDOM` / `ignoreDOM`.** Both apply. If a region is in both `ignoreDOM` and `ignoreColors`, full ignore wins: the whole region is masked, not just its colors. -- **Missing selectors do not fail the snapshot.** They are reported back via `domElementsMissing` and skipped, and the rest of the snapshot proceeds normally. - -## Related Docs - -- [Smart Ignore](/support/docs/smartui-smartignore/) -- [Region Ignore + Select on Web Hooks](/support/docs/smartui-hooks-region-ignore/) -- [Handling Dynamic Data with DOM Configuration and Options](/support/docs/html-dom-smartui-options/) -- [Selenium Visual Regression](/support/docs/selenium-visual-regression/) diff --git a/static/docs/smartui-k6-setup.md b/static/docs/smartui-k6-setup.md deleted file mode 100644 index 0c24e4206..000000000 --- a/static/docs/smartui-k6-setup.md +++ /dev/null @@ -1,347 +0,0 @@ -# Getting started with SmartUI using K6 - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This documentation will act as your step-by-step guide in to perform K6 test with SmartUI. - -### Prerequisites for running SmartUI with K6 - -- Basic understanding of K6 is required. -- Go to [SmartUI](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) and login along with your credentials. -- Copy `LT_USERNAME` and `LT_ACCESS_KEY` credentials from `Access Key` button on the top right of the dashboard. - -```bash -export LT_USERNAME="YOUR_USERNAME" -``` - -```bash -set LT_USERNAME=YOUR_USERNAME" -``` - -```powershell -$env:LT_USERNAME=YOUR_USERNAME" -``` - -The following steps will guide you in running your first Visual Regression test on TestMu AI platform - - -### **Step 1:** Create a SmartUI Project - -The first step is to create a project with the application in which we will combine all your **builds** run on the project. -To create a SmartUI Project, follow these steps: - -1. Go to [Projects page](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) -2. Click on the `new project` button -3. Select the platform as Web for executing your `K6` tests. -4. Add name of the project, approvers for the changes found, tags for any filter or easy navigation. -5. Click on the **Submit**. - -### **Step 2:** Configure your test with K6 Desired Capabilities - -Once you have created a SmartUI Project, you can generate screenshots by running automation scripts. Follow the below steps to successfully generate screenshots - - -1. Clone the sample GitHub repository: - -```bash -git clone https://github.com/LambdaTest/smartui-k6-sample -cd smartui-k6-sample -``` - -2. Install the required dependencies: - -```bash -npm install @lambdatest/smartui-cli @lambdatest/k6-driver -``` - -3. Install k6 by referring to the installation guide `https://k6.io/docs/get-started/installation/`: - -```bash -brew install k6 -``` - -```bash -winget install k6 -``` - -4. Set up the TestMu AI credentials by using the commands below in the terminal. The account details are available on your [TestMu AI Profile](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile) page. - -```bash -export LT_USERNAME="YOUR_LAMBDATEST_USERNAME" -export LT_ACCESS_KEY="YOUR_LAMBDATEST_ACCESS_KEY" -export PROJECT_TOKEN="YOUR_PROJECT_TOKEN" -``` - -```bash -set LT_USERNAME="YOUR_LAMBDATEST_USERNAME" -set LT_ACCESS_KEY="YOUR_LAMBDATEST_ACCESS_KEY" -set PROJECT_TOKEN="YOUR_PROJECT_TOKEN" -``` - -```powershell -$env:LT_USERNAME="YOUR_LAMBDATEST_USERNAME" -$env:LT_ACCESS_KEY="YOUR_LAMBDATEST_ACCESS_KEY" -$env:PROJECT_TOKEN="YOUR_PROJECT_TOKEN" -``` - -5. Create and configure SmartUI config file: - -```bash -npx smartui config:create smartui-web.json -``` - -6. Add the SmartUI snapshot function to your test file. Create or edit `test.js`: - -```javascript title="test.js" -import http from 'k6/http'; -import { smartuiSnapshot } from '@lambdatest/k6-driver'; - -export default function () { -const response = http.get('https://www.lambdatest.com'); // Enter your desired URL here -smartuiSnapshot(response, 'Homepage-Screenshot'); -// Please specify your response and the screenshot name in this function -// response - k6 http response instance (required) -// Screenshot_Name - Name of the screenshot; unique to each screenshot (required) -} -``` - -### **Step 3:** Executing the SmartUI Test Suite - -Execute the test using the following command: - -```bash -npx smartui exec k6 run test.js -``` - - - You can check the executed builds over at [TestMu AI SmartUI](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/). - -## Best Practices - -**Screenshot Naming** - -- Use descriptive, consistent names for screenshots -- Include page/component name in screenshot names -- Avoid special characters that might cause issues -- Use consistent naming conventions across your test suite - -**Example:** -```javascript -const response1 = http.get('https://example.com'); -smartuiSnapshot(response1, "HomePage-Header"); - -const response2 = http.get('https://example.com/products'); -smartuiSnapshot(response2, "ProductPage-MainContent"); -``` - -**Wait for Page Load** - -- Always wait for pages to fully load before taking screenshots -- Use K6's built-in check and sleep methods for timing control -- Consider adding delays for lazy-loaded content - -**Example:** -```javascript -import { sleep, check } from 'k6'; -import http from 'k6/http'; -import { smartuiSnapshot } from '@lambdatest/k6-driver'; - -const response = http.get('https://example.com'); -check(response, { -'status is 200': (r) => r.status === 200, -}); -sleep(2); // Wait for dynamic content -smartuiSnapshot(response, "Page-Loaded"); -``` - -**Handle Dynamic Content** - -- Use `ignoreDOM` for elements that change between runs -- Use `selectDOM` when you only need to compare specific areas -- Document why elements are ignored for future reference - -**Configuration Management** - -- Keep SmartUI configuration in version control -- Use environment variables for sensitive data -- Document custom configuration choices - -**Test Organization** - -- Group related screenshots in the same build -- Use meaningful build names -- Run tests in consistent environments - -## Troubleshooting - -**Issue: Screenshots Not Appearing in Dashboard** - -**Symptoms**: Tests run successfully but no screenshots appear in SmartUI dashboard - -**Possible Causes**: -- Project token not set or incorrect -- Project name mismatch -- Network connectivity issues -- K6 browser not enabled - -**Solutions**: -1. Verify `PROJECT_TOKEN` is set correctly: -```bash -echo $PROJECT_TOKEN -``` - -2. Check project name matches exactly (case-sensitive) - -3. Ensure you're using the SmartUI CLI wrapper: -```bash -npx smartui exec k6 run test.js -``` - -4. Check network connectivity to TestMu AI servers - -5. Review test execution logs for error messages - -**Issue: Project Not Found" Error** - -**Symptoms**: Error message indicating project cannot be found - -**Possible Causes**: -- Incorrect project token -- Project deleted or renamed -- Token from wrong project - -**Solutions**: -1. Verify project exists in SmartUI dashboard -2. Copy project token directly from Project Settings -3. Ensure token includes the project ID prefix (e.g., `123456#...`) -4. Check for extra spaces or quotes in token - -**Issue: Screenshots Show Blank or Incorrect Content** - -**Symptoms**: Screenshots captured but show blank pages or incorrect content - -**Possible Causes**: -- Page not fully loaded -- JavaScript not executed -- Viewport size issues -- Timing issues - -**Solutions**: -1. Add explicit waits before screenshots: -```javascript -import { sleep } from 'k6'; -import http from 'k6/http'; -import { smartuiSnapshot } from '@lambdatest/k6-driver'; - -const response = http.get('https://example.com'); -sleep(2); // Wait for content to load -smartuiSnapshot(response, 'Screenshot-Name'); -``` - -2. Check response status before taking screenshot: -```javascript -import { check } from 'k6'; - -const response = http.get('https://example.com'); -check(response, { -'status is 200': (r) => r.status === 200, -}); -smartuiSnapshot(response, 'Screenshot-Name'); -``` - -3. Verify viewport size in smartui-web.json configuration file - -**Issue: SmartUI CLI Not Working** - -**Symptoms**: Tests fail with SmartUI-related errors - -**Possible Causes**: -- `@lambdatest/smartui-cli` not installed -- `@lambdatest/k6-driver` not installed -- PROJECT_TOKEN not set - -**Solutions**: -1. Install required dependencies: -```bash -npm install @lambdatest/smartui-cli @lambdatest/k6-driver -``` - -2. Set PROJECT_TOKEN environment variable: -```bash -export PROJECT_TOKEN="YOUR_PROJECT_TOKEN" -``` - -3. Verify K6 is installed: -```bash -k6 version -``` - -4. Run tests using the SmartUI CLI wrapper: -```bash -npx smartui exec k6 run test.js -``` - -**Issue: Screenshot Status Not Approved** - -**Symptoms**: Screenshots appear but status is not Approved" - -**Possible Causes**: -- Screenshot comparison failed -- Baseline doesn't exist -- Mismatch threshold exceeded - -**Solutions**: -1. Check screenshot status in response: -```javascript -if (screenshot.screenshotStatus !== "Approved") { -// Handle non-approved status -} -``` - -2. Review comparison results in SmartUI dashboard -3. Verify baseline exists and is correct -4. Adjust pixel threshold if needed - -**Issue: Screenshot Names Not Matching Baseline** - -**Symptoms**: Screenshots appear as New" instead of comparing with baseline - -**Possible Causes**: -- Screenshot name changed -- Baseline doesn't exist -- Name contains special characters - -**Solutions**: -1. Ensure screenshot names are consistent across test runs -2. Verify baseline exists in project -3. Avoid special characters in screenshot names -4. Check for case sensitivity issues - -**Getting Help** - -If you encounter issues not covered here: - -- Review the [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) for detailed solutions -- Check [SmartUI Configuration Options](/support/docs/smartui-sdk-config-options) documentation -- See [Handling Dynamic Data](/support/docs/smartui-handle-dynamic-data) for dynamic content issues -- Visit [K6 Browser Testing Documentation](/support/docs/k6-browser-testing/) for K6-specific issues -- Visit [TestMu AI Support](https://www.testmuai.com/support) for additional resources -- Contact support at support@testmuai.com or use [24/7 Chat Support](https://www.testmuai.com/support) - -## Additional Resources - -- [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) -- [SmartUI Configuration Options](/support/docs/smartui-sdk-config-options) -- [Handling Dynamic Data](/support/docs/smartui-handle-dynamic-data) -- [Handling Lazy Loading](/support/docs/smartui-handle-lazy-loading) -- [Baseline Management](/support/docs/smartui-baseline-management) -- [Running Your First Project](/support/docs/smartui-running-your-first-project) -- [K6 Browser Testing Documentation](/support/docs/k6-browser-testing/) -- [SmartUI API Documentation](https://www.testmuai.com/support/api-doc/) - -For additional information about K6 framework please explore the documentation [here](/support/docs/k6-browser-testing/) - -## Advanced Options for Screenshot Comparison - -**Build Configuration** - If you have multiple screenshots running the same test suite and want to run the comparison for the same test suite, want to add a build as a baseline from your test suite or need to access more SmartUI Build Config Options, click [here](/support/docs/smart-ui-build-options/). - -## Advanced Options for Screenshot Comparison - -**Build Configuration** - If you have multiple screenshots running the same test suite and want to run the comparison for the same test suite, want to add a build as a baseline from your test suite or need to access more SmartUI Build Config Options, click [here](/support/docs/smart-ui-build-options/). diff --git a/static/docs/smartui-katalon-plugin.md b/static/docs/smartui-katalon-plugin.md deleted file mode 100644 index ee514da66..000000000 --- a/static/docs/smartui-katalon-plugin.md +++ /dev/null @@ -1,144 +0,0 @@ -# SmartUI Plugin for Katalon Studio - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The **Katalon Studio SmartUI Plugin** integrates [Katalon Studio](https://www.katalon.com/) with [TestMu AI SmartUI](/support/docs/smart-visual-regression-testing/), enabling **visual regression testing** directly inside your test automation workflows. - -With this plugin, you can: -- Capture screenshots at any point in your test flow -- Compare them with visual baselines -- Detect UI changes early in your CI/CD pipeline - -## Installation - -### Option 1: Install via Katalon Store (Recommended) - -1. Open [Katalon Store](https://store.katalon.com/). -2. Search for **“SmartUI”** or browse under *Plugins > Integrations*. -3. Click **Install** to add the plugin to your Katalon Studio instance. - -### Option 2: Manual Installation via JAR - -1. Download the latest `katalon-studio-smartui-plugin.jar` from the [official release page](https://github.com/LambdaTest/katalon-studio-smartui-plugin/releases). -2. In Katalon Studio, go to **Project > Settings > Plugins > Import Plugin**. -3. Select the downloaded `.jar` file and click **Import**. - -## Configuration & Usage - -### Step 1: Create a SmartUI Project - -1. Go to [SmartUI Projects Page](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/). -2. Click **New Project**. -3. Select platform as **CLI** (for Katalon integration). -4. Enter project name, approvers, and tags (optional). -5. Click **Submit** and copy your **Project Token** from the dashboard. - -> Your Project Token will be used to authenticate screenshot uploads to SmartUI. - -### Step 2: Start your Katalon Instance from terminal - -Run the following command in the terminal to start the Katalon instance. - -```bash -/Applications/Katalon\ Studio\ Enterprise.app/Contents/MacOS/katalon -``` - -### Step 3: Set Environment Variable - -In your Katalon project, set the `PROJECT_TOKEN` environment variable: - -#### Via Project Settings: -1. Go to **Project > Settings > Plugins > SmartUI Integration**. -2. Here you may set your project token - -```bash -PROJECT_TOKEN = "project_token..." -``` - -### Step 4: Configure SmartUI in Your Test Case - -Use the provided custom keywords to start, capture, and stop SmartUI sessions. - -#### Minimal Example: - -```groovy -// Start SmartUI Server -// Replace 'buildName' with your desired build name (optional) -CustomKeywords.'com.katalon.plugin.keyword.smartui.SmartKeywords.startServer'('My Katalon Build', '', '') - -// Open Browser -WebUI.openBrowser('') -WebUI.navigateToUrl('https://lambdatest.com') - -// Capture Snapshot with SmartUI -// The string parameter is the snapshot name (will appear in SmartUI Dashboard) -CustomKeywords.'com.katalon.plugin.keyword.smartui.SmartKeywords.takeSnapshot'('Homepage Snapshot') - -// Stop SmartUI Server -CustomKeywords.'com.katalon.plugin.keyword.smartui.SmartKeywords.stopServer'() - -// Close Browser -WebUI.closeBrowser() -``` - -### Step 5: Advanced Configuration (Optional) - -You can pass additional configuration via a JSON file or inline parameters. - -#### Using a Config File (`config.json`): - -```json -{ -"deviceName": "Chrome Desktop", -"platform": "Windows", -"fullPage": true, -"ignoreBoxes": { -"xpath": [ -"//*[@class='dynamic-ad']", -"//*[@id='timestamp']" -] -} -} -``` - -Then use it in your test: - -```groovy -CustomKeywords.'com.katalon.plugin.keyword.smartui.SmartKeywords.startServer'('Build v1.0', 'config.json', '') -``` - -> Use `ignoreBoxes` to exclude dynamic elements (ads, timestamps) from comparison. -> Use `selectBoxes` to compare only specific regions (e.g., main content area). - -## Running Tests - -1. Execute your test case as usual in Katalon Studio. -2. After test completion, visit your [SmartUI Dashboard](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) to: - - View captured screenshots - - Compare against baseline images - - Approve or reject visual changes - - Manage builds and baselines - -## Best Practices - -**Server Management** - -- Always include `startServer()` before taking screenshots and `stopServer()` after. - -**Snapshot Naming** - -- Use descriptive snapshot names for easy identification in SmartUI. - -**CI/CD Integration** - -- For CI/CD pipelines, set `PROJECT_TOKEN` as an environment variable in your CI tool (Jenkins, GitHub Actions, etc.). - -**Reporting** - -- Combine with Katalon's built-in reporting for comprehensive test insights. - -## Troubleshooting - -- **“Project Token not found”**: Ensure `PROJECT_TOKEN` is set in environment variables or passed explicitly. -- **No screenshots uploaded**: Verify network connectivity and that SmartUI server started successfully. -- **Dynamic content causing false positives**: Use `ignoreBoxes` or `selectBoxes` in config to stabilize comparisons. diff --git a/static/docs/smartui-layout-testing.md b/static/docs/smartui-layout-testing.md deleted file mode 100644 index 96f9a3203..000000000 --- a/static/docs/smartui-layout-testing.md +++ /dev/null @@ -1,172 +0,0 @@ -# Layout Comparison in SmartUI SDK - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Layout testing is a specialized approach to visual testing that focuses on verifying the structural integrity and arrangement of UI elements rather than their specific content or styling. It ensures that your application's visual hierarchy and element positioning remain consistent across different environments and updates. - -The example below demonstrates localisation testing, one of the major use cases of layout testing, where a webpage is tested across languages and locales. - -### Why Layout Testing Matters - -1. **Structural Consistency**: Ensures that UI elements maintain their intended positions and relationships, regardless of content changes or style updates. - -2. **Cross-Environment Reliability**: Validates that your application's layout remains intact across different: - - Operating systems - - Browsers - - Devices - - Screen sizes - - Viewport dimensions - -3. **Design System Compliance**: Helps maintain consistency with your design system by verifying that components follow established layout patterns. - -# What Layout Comparison Ignores - -When using layout comparison, the following aspects are ignored: - -1. **Text Content**: Changes in text content are not considered in the comparison -2. **Color Values**: Differences in color schemes or individual color values are ignored -3. **Style Properties**: Changes in CSS properties like font size, padding, margins, etc. are not compared -4. **Image Content**: Differences in image content are ignored, only their position and size are considered - -## Visual Diffs vs. Layout Diffs - -Understanding the difference between content and layout is crucial for effective visual testing: - -- **Visual Diffs** refer to the actual information that users interact with on a webpage, such as text, images, videos, and other elements that convey your message. -In the example below, you can see the visual differences between the baseline and comparison screenshot using Smart Ignore Diff Option. - -- **Layout** pertains to the arrangement and presentation of this content, including the positioning, styling, and structuring of elements. The goal of layout design is to ensure that content is visually appealing and well-organized. -In the example below, you can see the layout differences between the baseline and comparison screenshot using Layout Diff Option. - -SmartUI's layout comparison feature allows you to focus specifically on layout differences while ignoring content changes, giving you more precise control over your visual testing process. - -# Layout Comparison in SmartUI SDK - -## Prerequisites - -Before using the Layout Comparison feature, ensure you meet the following requirements: - -- **For SmartUI SDK (smartui exec):** SmartUI CLI version `4.1.8` or above is installed. - -> [!WARNING] -> The baseline images **must** be generated with the layout strategy enabled. Only then will the comparison work correctly. If you try to compare new images (with layout enabled) against old baseline images (without layout enabled), the layout comparison will fail or behave unexpectedly. - -## How to Use Layout Comparison with SmartUI - -Layout Comparison can be executed in two different ways depending on your integration. - -### 1. Using SmartUI Hooks (native automation) - -If you use **SmartUI Hooks** (visual tests through Selenium / Playwright / Cypress on the grid **without** the `smartui exec` wrapper), **layout comparison is applied per screenshot** by passing **`ignoreType: ["layout"]`** (and `screenshotName`, and optionally `fullPage`, and other layout-related keys) in the **object** you send to **`smartui.takeScreenshot`** via `executeScript`, **not** by relying on `ignoreType` or layout flags **only** inside `LT:Options`. - -For **`LT:Options`**, you still set **`smartUI.project`** (and `visual`, credentials). **Do not** assume `LT:Options.ignoreType: ['layout']` or similar capability-only snippets alone will enable layout for Hooks; that does not match current supported behavior. - -**See:** [SmartUI Hooks - Layout, Full Page, and Smart Ignore](/support/docs/smartui-hooks-layout-fullpage-smartignore/) for Java, JavaScript, and C# hook examples. - -### 2. Using SmartUI SDK (smartUISnapshot command) -If you are using the SmartUI SDK (`smartui exec`), you need to set the `ignoreType` option to `"layout"` when taking a specific screenshot within your code: - -```javascript -// Set options to focus only on layout structure -let options = { -ignoreType: ["layout"] -} - -// Take a screenshot with layout comparison enabled -await smartuiSnapshot(driver, "ScreenshotName", options); -``` - -```java -// Set options to focus only on layout structure -Map options = new HashMap<>(); -options.put("ignoreType", Arrays.asList("layout")); - -// Take a screenshot with layout comparison enabled -smartuiSnapshot(driver, "ScreenshotName", options); -``` - -```python -# Set options to focus only on layout structure -options = { -"ignoreType": ["layout"] -} - -# Take a screenshot with layout comparison enabled -smartui_snapshot(driver, "ScreenshotName", options) -``` - -```csharp -// Set options to focus only on layout structure -var options = new Dictionary -{ -{ "ignoreType", new List { "layout" } } -}; - -// Take a screenshot with layout comparison enabled -await SmartUI.Snapshot(driver, "ScreenshotName", options); -``` - -```ruby -# Set options to focus only on layout structure -options = { -ignoreType: ["layout"] -} - -# Take a screenshot with layout comparison enabled -smartui_snapshot(driver, "ScreenshotName", options) -``` - -## Known Limitations - -The Layout Comparison feature has the following limitations: - -- Not supported with Fetch APIs -- Not supported with Slack, Email, and GitHub integrations -- Performance may vary based on the complexity of the page structure - -## Use Cases for Layout Comparison - -The layout comparison feature is particularly valuable in the following scenarios: - -1. **Component Library Development**: When developing reusable components, you may want to verify that the layout structure remains consistent while allowing for content and style variations. - -2. **Responsive Design Testing**: When testing responsive layouts across different screen sizes, you may want to focus on ensuring the layout structure adapts correctly while ignoring specific content or style changes. - -3. **Design System Implementation**: When implementing a design system, you may need to verify that the layout structure follows the established patterns while allowing for content and style variations. - -4. **A/B Testing**: During A/B testing of different layouts, you may want to compare the structural integrity while ignoring the intentional content and style differences between variants. - -5. **Multi-language Testing**: When testing websites in different languages, you may want to verify that the layout structure remains consistent despite text length variations. - -6. **Cross-Environment Testing**: Ensure that your page structure remains intact across different operating systems, browsers, devices, viewport sizes, and orientations. - -## Example Implementation - -Here's a complete example showing how to implement layout comparison in a test: - -```javascript -describe('Layout Structure Test', () => { -it('should verify layout structure while ignoring content and style changes', async () => { -// Navigate to the page -await driver.get('https://example.com'); - -// Wait for layout to stabilize (important for dynamic content) -await driver.wait(until.elementLocated(By.cssSelector('.main-content')), 5000); - -// Configure options to focus only on layout structure -let options = { -ignoreType: ["layout"] -}; - -// Take screenshot with layout comparison -await smartuiSnapshot(driver, "HomePageLayout", options); -}); -}); -``` - -## Additional Resources - -- [SmartUI SDK Documentation](/docs/smartui-sdk-config-options) -- [Visual Regression Testing Guide](/support/docs/smart-visual-regression-testing/) -- [Comparison Settings Documentation](/docs/test-settings-options) -- [Hooks: Layout + Full Page + Smart Ignore](/support/docs/smartui-hooks-layout-fullpage-smartignore/) diff --git a/static/docs/smartui-mcp-server.md b/static/docs/smartui-mcp-server.md deleted file mode 100644 index f472adac6..000000000 --- a/static/docs/smartui-mcp-server.md +++ /dev/null @@ -1,71 +0,0 @@ -# Getting Started with SmartUI MCP Tool - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -SmartUI MCP Tool is part of the [TestMu AI MCP Server](/support/docs/testmu-mcp-server/). It lets you debug visual regressions from SmartUI comparison runs by returning natural-language summaries of pixel, layout, DOM, and perceptual differences. - -## How Does It Work? - -The SmartUI MCP Tool processes visual comparison data by calling specific analysis capabilities with a `comparisonId`. Each capability simulates a different debugging layer: pixel diff, layout, DOM structure, and human perception. - -The tool fetches the visual assets (screenshots, DOM snapshots, metadata) and returns natural-language outputs such as: - -- "The CTA button lost color contrast, impacting accessibility." -- "Header layout shifted left due to padding changes." - -## Capabilities - -All capabilities take a `comparisonId` as input. Start with `analyzeSmartUIRun` for a full picture, then use the individual capabilities to investigate a specific layer. - -### analyzeSmartUIRun - -Runs analysis across all layers - pixel, layout, DOM, and perception - and returns a single consolidated summary. Use this first when a visual diff fails and you are not sure where to look. - -> "The 'Subscribe' CTA lost its background color and dropped in emphasis. This is likely due to a CSS override on `.cta-btn`. Padding changes have also affected alignment. Suggest restoring background and resetting layout values." - -### summarizePixelDiff - -Use this when you suspect a color, image, or rendering change. It compares the baseline and current screenshots at the pixel level. - -> "Significant pixel differences were detected in the hero banner and footer. These likely stem from background color changes and text shifts." - -### summarizeLayoutDiff - -Use this when elements appear misaligned or have moved. It identifies spacing, alignment, and size-related changes. - -> "The login button has moved 16px downward compared to the baseline. This affects the visual balance of the form." - -### summarizeDomDiff - -Use this when you suspect an HTML or attribute change caused the visual difference. It compares the DOM structure between runs. - -> "The `aria-label` for the submit button has been removed. The button also now includes an extra `disabled` attribute." - -### analyzeHumanDiff - -Use this when the diff looks minor technically but feels visually wrong. It simulates how a human eye perceives the change in terms of color, weight, and emphasis. - -> "The pricing table appears visually heavier due to a font weight increase. This disrupts the balance between sections." - -### getSmartUIResources - -Fetches the screenshots, DOM snapshots, and metadata for a comparison run. Use this to confirm assets are available before running an analysis. - -Returns a description of what was found and flags anything missing. - -## How to Invoke - -Once your MCP client is connected, use natural language to interact with the SmartUI tool. Example prompts: - -- "Run a full SmartUI analysis on comparisonId ``." -- "What changed in the layout for comparisonId ``?" -- "Why did the visual diff fail for comparisonId ``?" -- "Summarize the DOM changes for comparisonId ``." - -Your AI client will route the request to the appropriate analysis layer. - -## Error Handling - -If the `comparisonId` is invalid or resources are missing, you will receive: - -> "Error analyzing SmartUI run: comparisonId not found or visual resources are incomplete." diff --git a/static/docs/smartui-mismatch-thresholds.md b/static/docs/smartui-mismatch-thresholds.md deleted file mode 100644 index f902f6a30..000000000 --- a/static/docs/smartui-mismatch-thresholds.md +++ /dev/null @@ -1,247 +0,0 @@ -# Mismatch Thresholds - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -When running visual regression tests, not every pixel-level difference is a real bug. Minor rendering variations such as font anti-aliasing, date/time stamps, or animated content can cause screenshots to fail even when the page looks correct to the human eye. - -**Mismatch Thresholds** let you define how much visual difference is acceptable before a screenshot is flagged. You can configure two values: - -| Parameter | Description | -|---|---| -| **Approval Threshold** | The maximum mismatch percentage at which a screenshot is **automatically approved**. If the detected difference is at or below this value, the screenshot passes without manual review. | -| **Rejection Threshold** | The minimum mismatch percentage at which a screenshot is **automatically rejected**. If the detected difference meets or exceeds this value, the screenshot is marked as failed. | - -Screenshots with a mismatch percentage **between** the approval and rejection thresholds will require **manual review** by an approver. - -**Example** -If you set `approvalThreshold: 2` and `rejectionThreshold: 5`: -- A screenshot with **1.5%** mismatch is **auto-approved**. -- A screenshot with **3%** mismatch needs **manual review**. -- A screenshot with **6%** mismatch is **auto-rejected**. - -## Supported Frameworks & Languages - -| Languages | Frameworks | -|---|---| -| Java, JavaScript, Python, C#, Ruby | Selenium, Appium | - -## Threshold Hierarchy - -You can set thresholds at three levels. When multiple levels are configured, the most specific setting takes priority: - -| Priority | Level | Scope | How to Set | -|---|---|---|---| -| 1 (Highest) | **Screenshot** | Applies to a single screenshot | Pass options in the `smartuiSnapshot` call | -| 2 | **Build** | Applies to all screenshots in a build | Set in your test capabilities (`LT:Options`) | -| 3 (Lowest) | **Project** | Applies to all builds in a project | Configure in the [LambdaTest SmartUI Dashboard](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com) | - -**Backward Compatible** -If you do not configure build or screenshot-level thresholds, your existing project-level settings continue to apply. No changes are needed to your existing tests. - -## Threshold Rules - -Keep the following constraints in mind when configuring thresholds: - -- Both values must be a number between **0** and **100** (representing a percentage). -- The `approvalThreshold` must be **less than** the `rejectionThreshold`. -- If only one threshold is provided, the other falls back to the next level in the hierarchy (build or project). - -## Build-Level Configuration - -Set thresholds for an entire build by adding them to your test capabilities. This overrides the project-level defaults for every screenshot captured in that build. - -### Capability Parameters - -| Parameter | Type | Description | -|---|---|---| -| `smartUI.approvalThreshold` | Number | Mismatch percentage at or below which screenshots are auto-approved. | -| `smartUI.rejectionThreshold` | Number | Mismatch percentage at or above which screenshots are auto-rejected. | - -### Examples - -```javascript title="Add thresholds to your LT:Options capabilities" -let capabilities = { -browserName: "chrome", -"LT:Options": { -user: process.env.LT_USERNAME, -accessKey: process.env.LT_ACCESS_KEY, -// highlight-next-line -"smartUI.approvalThreshold": 2, -// highlight-next-line -"smartUI.rejectionThreshold": 5 -} -}; -``` - -```java title="Add thresholds to your LT:Options capabilities" -HashMap ltOptions = new HashMap<>(); -ltOptions.put("user", System.getenv("LT_USERNAME")); -ltOptions.put("accessKey", System.getenv("LT_ACCESS_KEY")); -// highlight-next-line -ltOptions.put("smartUI.approvalThreshold", 2); -// highlight-next-line -ltOptions.put("smartUI.rejectionThreshold", 5); - -capabilities.setCapability("LT:Options", ltOptions); -``` - -```python title="Add thresholds to your LT:Options capabilities" -lt_options = { -"user": os.environ["LT_USERNAME"], -"accessKey": os.environ["LT_ACCESS_KEY"], -# highlight-next-line -"smartUI.approvalThreshold": 2, -# highlight-next-line -"smartUI.rejectionThreshold": 5 -} -capabilities["LT:Options"] = lt_options -``` - -```csharp title="Add thresholds to your LT:Options capabilities" -var ltOptions = new Dictionary -{ -{ "user", Environment.GetEnvironmentVariable("LT_USERNAME") }, -{ "accessKey", Environment.GetEnvironmentVariable("LT_ACCESS_KEY") }, -// highlight-next-line -{ "smartUI.approvalThreshold", 2 }, -// highlight-next-line -{ "smartUI.rejectionThreshold", 5 } -}; -capabilities.AddAdditionalOption("LT:Options", ltOptions); -``` - -```ruby title="Add thresholds to your LT:Options capabilities" -lt_options = { -user: ENV['LT_USERNAME'], -accessKey: ENV['LT_ACCESS_KEY'], -# highlight-next-line -'smartUI.approvalThreshold': 2, -# highlight-next-line -'smartUI.rejectionThreshold': 5 -} -capabilities['LT:Options'] = lt_options -``` - -## Screenshot-Level Configuration - -Override the build or project thresholds for individual screenshots. This is useful when specific pages contain dynamic content (e.g., live feeds, timestamps, ads) that naturally causes higher mismatch. - -The `smartuiSnapshot` method accepts an optional options object with threshold parameters. - -### Options Parameters - -| Parameter | Type | Description | -|---|---|---| -| `approvalThreshold` | Number | Mismatch percentage at or below which this screenshot is auto-approved. | -| `rejectionThreshold` | Number | Mismatch percentage at or above which this screenshot is auto-rejected. | - -### Examples - -```javascript title="Pass thresholds as the third argument to smartuiSnapshot" -// Screenshot using build/project-level thresholds (no override) -await smartuiSnapshot(driver, "Homepage"); - -// Screenshot with a custom threshold for a dynamic page -// highlight-start -await smartuiSnapshot(driver, "Live Dashboard", { -approvalThreshold: 5, -rejectionThreshold: 10 -}); -// highlight-end -``` - -```java title="Pass thresholds as a Map to smartuiSnapshot" -// Screenshot using build/project-level thresholds (no override) -SmartUISnapshot.smartuiSnapshot(driver, "Homepage"); - -// Screenshot with a custom threshold for a dynamic page -// highlight-start -Map options = new HashMap<>(); -options.put("approvalThreshold", 5); -options.put("rejectionThreshold", 10); -SmartUISnapshot.smartuiSnapshot(driver, "Live Dashboard", options); -// highlight-end -``` - -```python title="Pass thresholds in the options dictionary" -# Screenshot using build/project-level thresholds (no override) -driver.execute_script("smartui.takeScreenshot", {"screenshotName": "Homepage"}) - -# Screenshot with a custom threshold for a dynamic page -# highlight-start -options = { -"screenshotName": "Live Dashboard", -"approvalThreshold": 5, -"rejectionThreshold": 10 -} -driver.execute_script("smartui.takeScreenshot", options) -# highlight-end -``` - -```csharp title="Pass thresholds in a Dictionary to smartuiSnapshot" -// Screenshot using build/project-level thresholds (no override) -SmartUISnapshot.smartuiSnapshot(driver, "Homepage"); - -// Screenshot with a custom threshold for a dynamic page -// highlight-start -var options = new Dictionary -{ -{ "approvalThreshold", 5 }, -{ "rejectionThreshold", 10 } -}; -SmartUISnapshot.smartuiSnapshot(driver, "Live Dashboard", options); -// highlight-end -``` - -```ruby title="Pass thresholds as a hash to smartui_snapshot" -# Screenshot using build/project-level thresholds (no override) -smartui_snapshot(driver, "Homepage") - -# Screenshot with a custom threshold for a dynamic page -# highlight-start -smartui_snapshot(driver, "Live Dashboard", { -approvalThreshold: 5, -rejectionThreshold: 10 -}) -# highlight-end -``` - -## Combining Build and Screenshot Thresholds - -You can set a strict default at the build level and relax it only for specific screenshots that need it. Here is a full JavaScript example: - -```javascript title="Build-level strict + screenshot-level relaxed" -// Build-level: strict 1% approval, 3% rejection for most pages -let capabilities = { -browserName: "chrome", -"LT:Options": { -"smartUI.approvalThreshold": 1, -"smartUI.rejectionThreshold": 3 -} -}; - -let driver = new webdriver.Builder() -.usingServer("https://hub.lambdatest.com/wd/hub") -.withCapabilities(capabilities) -.build(); - -// This screenshot uses the strict build-level thresholds (1% / 3%) -await driver.get("https://example.com/checkout"); -await smartuiSnapshot(driver, "Checkout Page"); - -// This screenshot overrides with relaxed thresholds for a page with ads -await driver.get("https://example.com/homepage"); -await smartuiSnapshot(driver, "Homepage with Ads", { -approvalThreshold: 5, -rejectionThreshold: 15 -}); -``` - -## Common Use Cases - -| Scenario | Recommended Configuration | -|---|---| -| **Pixel-perfect pages** (login, checkout) | Set tight thresholds: `approvalThreshold: 0`, `rejectionThreshold: 1` | -| **Pages with minor dynamic content** (timestamps, counters) | Moderate thresholds: `approvalThreshold: 2`, `rejectionThreshold: 5` | -| **Highly dynamic pages** (live feeds, ad banners, dashboards) | Relaxed screenshot-level thresholds: `approvalThreshold: 10`, `rejectionThreshold: 20` | -| **Global default for all builds** | Set via the [SmartUI Dashboard](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com) at the project level | diff --git a/static/docs/smartui-multiple-assets-hosts.md b/static/docs/smartui-multiple-assets-hosts.md deleted file mode 100644 index 9f9c136d6..000000000 --- a/static/docs/smartui-multiple-assets-hosts.md +++ /dev/null @@ -1,19 +0,0 @@ -# Capturing Multiple Assets Hosts - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -In the SmartUI SDK, the `allowedHostnames` option is used to configure the capture of assets from specific hostnames during UI testing. By default, SmartUI captures only assets that are hosted on the same domain as the snapshot location. For example, if snapshots are taken on `https://xyz.com`, assets hosted on `https://cdn.xyz.com` will not be captured unless explicitly specified. The `allowedHostnames` configuration allows you to define multiple hostnames, ensuring assets from those sources are included during test execution. - -## Purpose of `allowedHostnames` - -The primary purpose of the `allowedHostnames` is to allow users to specify which external domains or subdomains should be trusted for asset capture. This is particularly useful when assets are hosted across multiple servers or CDNs, and you want to ensure that the complete set of resources required for rendering a page is captured during your tests. - -## Configuration File Structure -The `allowedHostnames` configuration option can be defined in your config file for SmartUI. You need to explicitly list each hostname from which assets should be captured. The SmartUI configuration file follows a JSON structure. Below is a sample configuration file: - -```javascript title=".smartui.json" -{ -..// mobile or web configurations... -"allowedHostnames": ["cdn.xyz.com"] -} -``` diff --git a/static/docs/smartui-multiple-projects-per-repo.md b/static/docs/smartui-multiple-projects-per-repo.md deleted file mode 100644 index 02a8ae27e..000000000 --- a/static/docs/smartui-multiple-projects-per-repo.md +++ /dev/null @@ -1,37 +0,0 @@ -# Configure Multiple Projects per repo - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -SmartUI allows you to manage and test UI components effectively within a single repository. By organizing multiple projects in a single repository, you can centralize your workflows and streamline collaboration. This guide will walk you through setting up multiple projects in a repository for SmartUI and how you can trigger the same via GitHub Actions pipeline. - -## Setting Up Multiple Projects in a Repository - -### Prerequisites -- Your TestMu AI [Username and Access key](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/security). -- Your SmartUI Project Token - -### Step 1: Organize Your Repository -Create a repository where you can organize your projects into various directories. You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** -Download or Clone the code sample from the TestMu AI GitHub repository to run the tests on the SmartUI. - View on GitHub - -### Step 2: Create your Secrets -- Click on the **Settings** of your repository. -- Go to the **Security** option > **Secrets and Variables** > **Actions**. -- Create your secrets with variable name **LT_USERNAME** and **LT_ACCESS_KEY**. You can fetch your credentials from the [Accounts and Settings](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/security) dashboard. - -### Step 3: Create your GitHub Actions Workflow -- Navigate to the main page of the repository. -- Under your repository name, click **Actions**. -- In the left sidebar, click the **New workflow** button. - -```yaml reference -https://github.com/LambdaTest/smartui-monorepo-sample/blob/main/.github/workflows/main.yml -``` - -### Step 4: Execute your projects -- Click on the **Actions** button of your repository to trigger the pipeline. -- Provide the **PROJECT TOKEN** to trigger your code projects. -- Go to the [SmartUI Dashboard](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) to check the results. diff --git a/static/docs/smartui-multiselect-bulkops.md b/static/docs/smartui-multiselect-bulkops.md deleted file mode 100644 index 571b0e2cd..000000000 --- a/static/docs/smartui-multiselect-bulkops.md +++ /dev/null @@ -1,126 +0,0 @@ -# Multiselect & Bulk Operations - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Managing large visual regression builds can be time-consuming. **Multiselect & Bulk Operations** allow you to perform actions on multiple screenshots at once, significantly speeding up the review process. - -## Key Capabilities -- **Batch Approval**: Select multiple "mismatch" screenshots and approve them as new baselines in one click. -- **Batch Rejection**: Quickly reject multiple incorrect screenshots. -- **Bulk Status Update**: Change the status of multiple tests simultaneously. - -This guide explains the end-to-end workflow, supported actions, and best practices to avoid accidental approvals or baseline edits. - -For a complete understanding of approval, reject, move, and merge functions across different workflows, see our [Approval & Baseline Management Guide](/support/docs/smartui-approval-workflow-guide/). - -## Where multiselect appears - -1. Sign in to [TestMu AI SmartUI](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/). -2. Open any project → choose a build. -3. Navigate across tabs (e.g., **New**, **Changes Found**). -4. Look for the **selection toolbar** above the screenshot grid. - -### Selection states - -| State | Description | Screenshot | -| --- | --- | --- | -| None selected | Bulk action buttons stay disabled until at least one screenshot is selected. | | -| Single selection | Shows contextual details (name, viewport, status). Bulk buttons activate. | | -| Multi selection | Displays count (e.g., “3 selected”) and unlocks all bulk actions. | | - -Use the checkbox beside each screenshot card or the **Select All** checkbox in the toolbar to toggle selections quickly. - -## Bulk actions - -### Approve screenshots - -Ideal when only a subset of diffs are valid. - -1. Filter to **Failed** or **Pending** tab. -2. Select the screenshots that represent expected UI updates. -3. Click **Approve**. - -SmartUI will: -- Mark each screenshot as **Approved**. -- Move them to the **Approved** tab for auditing. -- Preserve audit logs with the approver name and timestamp. - -### Reject screenshots - -Use this when a diff is unexpected, unstable, or blocked by other issues. - -1. Select the screenshots within **Failed**/**Pending**. -2. Click **Reject**. - -Rejected screenshots move to the **Rejected** tab and remain available for future comparison runs. - -### Move approved screenshots to Baseline - -Once approved diffs represent the new expected UI, promote them to Baseline: - -1. Go to the **Approved** tab. -2. Select individual screenshots, curated subsets, or **Select All**. -3. Click **Move to Baseline**. - -> Baseline moves update the reference snapshot for the next comparison. Ensure you only promote validated UI changes to keep noise low in future runs. - -## Multiselect workflow example - -1. Filter by tab `Changes Found`. -2. Use **Select All** (only filtered screenshots are targeted). -3. Deselect any screenshot needing manual review. -4. Click **Approve** → confirm in the dialog. -5. Bulk approval completes with a toast confirmation and updated counters. - -1. Sort failures by **Severity**. -2. Select the unexpected diffs (e.g., CTA color regressions). -3. Click **Reject** to block them from merging. -4. Add a note (optional) so teammates understand the rejection reason. - -1. After QA approval, hop to the **Approved** tab. -2. Select only the screenshots that represent canonical UI. -3. Click **Move to Baseline**. -4. Confirm to overwrite the existing baseline for the same test name + viewport. - -## Best practices - -Apply filters (approver, status, tab) before using Select All. -Confirm the selected count matches expectations to avoid cross-build approvals. - -Bulk actions are powerful: always double-check each screenshot thumbnail. -Use single selection for high-risk diffs (checkout, payments, authentication). - -After approving or rejecting, verify the activity log for transparency. -If anything was mis-clicked, undo by moving screenshots back via their tab. - -Promote to baseline only after QA sign-off. -Keep a release branch baseline separate using Smart Git. - -## Troubleshooting - -Symptoms: Checkboxes or bulk buttons stay disabled. -Fixes: - -Confirm you are in the Builds view, not inside an individual screenshot diff. -Refresh the page or clear cache if the toolbar fails to render. -Ensure your role has approval permissions on the project. - -Symptoms: Toast error when approving/rejecting. -Fixes: - -Retry after verifying network connectivity. -Confirm the build is still active (not archived). -Check SmartUI Troubleshooting Guide for API rate limits or backend incidents. - -Symptoms: Move to Baseline button disabled or no visible changes after promotion. -Fixes: - -Only Approved screenshots can be promoted; re-approve if needed. -Baseline updates can take a minute to propagate: refresh or reopen the build. -Ensure the associated branch/build is not locked by automated workflows. - -## Next steps - -- Compare multiselect approval speeds with your previous manual workflow and document the improvement. -- Pair multiselect with [Smart Ignore](/support/docs/smartui-smartignore/) to reduce noise before bulk approvals. -- Leverage [baseline management](/support/docs/smartui-baseline-management/) to keep production and release baselines in sync after bulk moves. diff --git a/static/docs/smartui-pdf-api-upload.md b/static/docs/smartui-pdf-api-upload.md deleted file mode 100644 index 1849a50ef..000000000 --- a/static/docs/smartui-pdf-api-upload.md +++ /dev/null @@ -1,206 +0,0 @@ -# Upload PDFs via API - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This functionality is exclusive to our enterprise plan subscribers on SmartUI. For additional details or inquiries, please [contact us](https://www.testmuai.com/demo). - -## Prerequisites for Using SmartUI - -- Familiarity with HTTP APIs is essential. -- Visit the [`TestMu AI SmartUI`](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) page and log in with your credentials. -- Obtain your `LT_USERNAME` and `LT_ACCESS_KEY` by clicking on the `Access Key` button, located at the top right corner of your dashboard. - -## Step 1: Establishing a SmartUI Project - -To initiate a SmartUI PDF Comparison Project, adhere to the following instructions: - -1. Navigate to the [SmartUI Projects Page](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/). -2. Tap on the `new project` button. -3. Specify your platform type as `PDF`. -4. Provide your `project` name, designate `approvers`, and add `tags` (optional). -5. Confirm your entry by clicking on **Submit**. - -Once your project is active, retrieve your `Project Token` from the application. Here's an example of a project token: - -```bash -projectToken = "123456#1234abcd-****-****-****-************" -``` - -## Uploading PDFs via API - -After setting up your SmartUI Project, you can upload your local PDF files to your project. This will automatically generate a build by capturing snapshots of every page. - -Here's how you can upload your PDFs: - -1. Retrieve your API `URL Endpoint` post activation of your enterprise plan. To schedule a demonstration, click [here](https://www.testmuai.com/demo). - -2. Append the following parameters to your request payload via `form-body`: - -| Variable | Type | Description | Required? | -| ------------ | ------ | ------------------------------------------------------------------------------------------------------------------------------------------------ | --------- | -| projectToken | string | This token is required to upload PDF files and validate your project. Example: `projectToken:123456#1234abcd-****-****-****-************` | Yes | -| pathToFiles | array | Add the path to the PDFs that will be uploaded. Example: `pathToFiles : [ "path/to/pdf-1", "path/to/pdf-2"]` | Yes | -| buildName | string | Assign a name of your choice to the build comprising the uploaded PDFs. Example: `buildName : #` | No | - -Only files in `.pdf` format are compatible with this feature. - -## API Request Example - -Here's a sample API request using cURL: - -```bash -curl -X POST "YOUR_API_ENDPOINT" \ --H "Content-Type: multipart/form-data" \ --F "projectToken=123456#1234abcd-****-****-****-************" \ --F "pathToFiles[]=path/to/document1.pdf" \ --F "pathToFiles[]=path/to/document2.pdf" \ --F "buildName=Release-v2.1" -``` - -## Response Format - -The API will return a JSON response containing: - -- **Build ID**: Unique identifier for the uploaded build -- **Status**: Upload status (success/failure) -- **Pages Captured**: Number of pages processed -- **Test Results**: Visual comparison results (if available) - -## Use Cases - -- **Automated CI/CD Integration**: Seamlessly integrate PDF testing into your deployment pipeline -- **Batch Processing**: Upload multiple PDFs simultaneously for efficient testing -- **Custom Workflows**: Build custom applications that interact with SmartUI programmatically -- **Enterprise Integration**: Connect with existing enterprise systems and workflows - -## Best Practices - -**PDF File Management** - -- Use consistent naming conventions for PDF files -- Organize PDFs in logical directory structures -- Verify PDF files are valid and not corrupted before upload -- Keep PDF files in version control when appropriate - -**Project Token Management** - -- Store project token as environment variable -- Never commit tokens to version control -- Use different tokens for different environments -- Rotate tokens regularly - -**Build Naming** - -- Use meaningful build names that include version info -- Include date or version in build names -- Use consistent naming conventions - -**Example:** -```bash -buildName="PDF-Comparison-v1.0-$(date +%Y%m%d)" -``` - -**Error Handling** - -- Always check API response status -- Handle network failures gracefully -- Implement retry logic for transient failures -- Log errors for debugging - -**Batch Processing** - -- Process PDFs in batches for efficiency -- Monitor upload progress -- Handle partial failures in batch operations -- Use appropriate batch sizes - -## Troubleshooting - -**Issue: PDF Upload Fails** - -**Symptoms**: PDF upload returns error or fails silently - -**Possible Causes**: -- Invalid PDF file -- File path incorrect -- File size too large -- Network connectivity issues -- Project token incorrect -- API endpoint incorrect - -**Solutions**: -1. Verify PDF file is valid and not corrupted -2. Check file path is correct and accessible -3. Verify file size is within limits -4. Check network connectivity to TestMu AI servers -5. Verify PROJECT_TOKEN is set correctly -6. Confirm API endpoint URL is correct - -**Issue: "Project Not Found" Error** - -**Symptoms**: Error message indicating project cannot be found - -**Possible Causes**: -- Incorrect project token -- Project deleted or renamed -- Token from wrong project - -**Solutions**: -1. Verify project exists in SmartUI dashboard -2. Copy project token directly from Project Settings -3. Ensure token includes the project ID prefix (e.g., `123456#...`) -4. Check for extra spaces or quotes in token - -**Issue: Upload Returns Error Response** - -**Symptoms**: API returns error status or failure message - -**Possible Causes**: -- Invalid request format -- Missing required parameters -- Authentication issues -- Server-side processing error - -**Solutions**: -1. Verify request format matches API specification -2. Check all required parameters are included -3. Verify authentication credentials -4. Review error response for specific details -5. Retry upload if transient error - -**Issue: PDFs Not Appearing in Dashboard** - -**Symptoms**: Uploads complete but PDFs don't appear in SmartUI dashboard - -**Possible Causes**: -- Incorrect project token -- Project name mismatch -- Upload not completed -- Dashboard refresh needed - -**Solutions**: -1. Verify PROJECT_TOKEN is correct -2. Check project name matches exactly (case-sensitive) -3. Wait a few moments and refresh dashboard -4. Check upload response for errors -5. Review API response for upload status - -**Getting Help** - -If you encounter issues not covered here: - -- Review the [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) for detailed solutions -- Check [PDF Comparison Overview](/support/docs/smartui-pdf-comparison) for PDF-specific information -- See [PDF CLI Upload](/support/docs/smartui-pdf-cli-upload) for alternative upload methods -- Visit [TestMu AI Support](https://www.testmuai.com/support) for additional resources -- Contact support at support@testmuai.com or use [24/7 Chat Support](https://www.testmuai.com/support) - -## Additional Resources - -- [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) -- [PDF Comparison Overview](/support/docs/smartui-pdf-comparison) -- [PDF CLI Upload](/support/docs/smartui-pdf-cli-upload) -- [PDF Java SDK Upload](/support/docs/smartui-pdf-java-sdk) -- [Baseline Management](/support/docs/smartui-baseline-management) -- [Running Your First Project](/support/docs/smartui-running-your-first-project) -- [SmartUI API Documentation](https://www.testmuai.com/support/api-doc/) diff --git a/static/docs/smartui-pdf-cli-upload.md b/static/docs/smartui-pdf-cli-upload.md deleted file mode 100644 index a5a541b35..000000000 --- a/static/docs/smartui-pdf-cli-upload.md +++ /dev/null @@ -1,296 +0,0 @@ -# Upload PDFs via CLI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This functionality is exclusive to our enterprise plan subscribers on SmartUI. For additional details or inquiries, please [contact us](https://www.testmuai.com/demo). - -## Prerequisites for Using SmartUI - -- Familiarity with command-line tools is essential. -- Visit the [`TestMu AI SmartUI`](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) page and log in with your credentials. -- Obtain your `LT_USERNAME` and `LT_ACCESS_KEY` by clicking on the `Access Key` button, located at the top right corner of your dashboard. - -## Step 1: Establishing a SmartUI Project - -To initiate a SmartUI PDF Comparison Project, adhere to the following instructions: - -1. Navigate to the [SmartUI Projects Page](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/). -2. Tap on the `new project` button. -3. Specify your platform type as `PDF`. -4. Provide your `project` name, designate `approvers`, and add `tags` (optional). -5. Confirm your entry by clicking on **Submit**. - -Once your project is active, retrieve your `Project Token` from the application. Here's an example of a project token: - -```bash -projectToken = "123456#1234abcd-****-****-****-************" -``` - -## Step 1: Install the SmartUI CLI - -Install the CLI globally using npm: - -```bash -npm install -g @lambdatest/smartui-cli -``` - -## Step 2: Setup your credentials - -```bash -export LT_USERNAME="${YOUR_LAMBDATEST_USERNAME}" -export LT_ACCESS_KEY="${YOUR_LAMBDATEST_ACCESS_KEY}" -export PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -``` - -```bash -set LT_USERNAME="${YOUR_LAMBDATEST_USERNAME}" -set LT_ACCESS_KEY="${YOUR_LAMBDATEST_ACCESS_KEY}" -set PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -``` - -```powershell -$env:LT_USERNAME="${YOUR_LAMBDATEST_USERNAME}" -$env:LT_ACCESS_KEY="${YOUR_LAMBDATEST_ACCESS_KEY}" -$env:PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -``` - -## Step 3: Upload PDFs Using CLI - -Use the `upload-pdf` command to upload one or multiple PDF files from a directory: - -```bash -smartui upload-pdf [options] -``` - -### Arguments: -- `directory_or_filename`: Path to a single PDF file or a directory containing multiple PDFs. - -### Options: -- `--fetch-results [filename]`: Fetch test results after upload. Optionally specify an output file (e.g., `results.json`). -- `--buildName `: Assign a custom name to the build. -- `--projectToken `: Specify the project token (if not set as environment variable). - -### Example Usage: - -Upload all PDFs from a folder and name the build: - -```bash -smartui upload-pdf ./pdfs/ --buildName "Release-v2.1" -``` - -Upload a single PDF file: - -```bash -smartui upload-pdf ./document.pdf --buildName "Single-PDF-Test" -``` - -Fetch results and save to a file: - -```bash -smartui upload-pdf ./spec.pdf --fetch-results results.json -``` - -Upload with custom project token: - -```bash -smartui upload-pdf ./pdfs/ --projectToken "123456#1234abcd-****-****-****-************" --buildName "Custom-Build" -``` - -## Advanced CLI Options - -### Batch Processing - -Process multiple directories: - -```bash -smartui upload-pdf ./documents/ --buildName "Batch-1" -smartui upload-pdf ./reports/ --buildName "Batch-2" -``` - -### CI/CD Integration - -Example for GitHub Actions: - -```yaml -- name: Upload PDFs to SmartUI -run: | -smartui upload-pdf ./generated-pdfs/ --buildName "${{ github.sha }}" --fetch-results test-results.json -``` - -## Use Cases - -- **CI/CD Pipelines**: Integrate PDF testing into automated deployment workflows -- **Batch Processing**: Upload multiple PDFs efficiently from command line -- **Automated Testing**: Schedule PDF uploads as part of automated test suites -- **Developer Workflows**: Quick PDF testing during development and debugging - -## Best Practices - -**PDF File Management** - -- Use consistent naming conventions for PDF files -- Organize PDFs in logical directory structures -- Verify PDF files are valid and not corrupted before upload -- Keep PDF files in version control when appropriate - -**Project Token Management** - -- Store project token as environment variable -- Never commit tokens to version control -- Use different tokens for different environments -- Rotate tokens regularly - -**Build Naming** - -- Use meaningful build names that include version info -- Include date or version in build names -- Use consistent naming conventions - -**Example:** -```bash -smartui upload-pdf ./pdfs/ --buildName Release-v1.0-$(date +%Y%m%d)" -``` - -**Error Handling** - -- Always check CLI exit codes -- Handle network failures gracefully -- Implement retry logic for transient failures -- Log errors for debugging - -**Batch Processing** - -- Process PDFs in batches for efficiency -- Monitor upload progress -- Handle partial failures in batch operations -- Use appropriate batch sizes - -## Troubleshooting - -**Issue: PDF Upload Fails** - -**Symptoms**: CLI command fails or returns error - -**Possible Causes**: -- Invalid PDF file -- File path incorrect -- File size too large -- Network connectivity issues -- Project token incorrect -- CLI not installed - -**Solutions**: -1. Verify PDF file is valid and not corrupted: -```bash -file document.pdf -``` - -2. Check file path is correct: -```bash -ls -la ./pdfs/ -``` - -3. Verify file size is within limits - -4. Check network connectivity to TestMu AI servers - -5. Verify PROJECT_TOKEN is set correctly: -```bash -echo $PROJECT_TOKEN -``` - -6. Verify SmartUI CLI is installed: -```bash -smartui --version -``` - -**Issue: Project Not Found" Error** - -**Symptoms**: Error message indicating project cannot be found - -**Possible Causes**: -- Incorrect project token -- Project deleted or renamed -- Token from wrong project - -**Solutions**: -1. Verify project exists in SmartUI dashboard -2. Copy project token directly from Project Settings -3. Ensure token includes the project ID prefix (e.g., `123456#...`) -4. Check for extra spaces or quotes in token - -**Issue: CLI Command Not Found** - -**Symptoms**: `smartui` command not recognized - -**Possible Causes**: -- CLI not installed -- npm not available -- PATH issues - -**Solutions**: -1. Install SmartUI CLI: -```bash -npm install -g @lambdatest/smartui-cli -``` - -2. Verify npm is available: -```bash -npm --version -``` - -3. Check PATH includes npm global bin directory - -**Issue: Upload Returns Error** - -**Symptoms**: CLI returns error status or failure message - -**Possible Causes**: -- Invalid command syntax -- Missing required parameters -- Authentication issues -- Server-side processing error - -**Solutions**: -1. Verify command syntax matches documentation -2. Check all required parameters are included -3. Verify authentication credentials -4. Review error message for specific details -5. Retry upload if transient error - -**Issue: PDFs Not Appearing in Dashboard** - -**Symptoms**: Uploads complete but PDFs don't appear in SmartUI dashboard - -**Possible Causes**: -- Incorrect project token -- Project name mismatch -- Upload not completed -- Dashboard refresh needed - -**Solutions**: -1. Verify PROJECT_TOKEN is correct -2. Check project name matches exactly (case-sensitive) -3. Wait a few moments and refresh dashboard -4. Check CLI output for errors -5. Use `--fetch-results` to verify upload status - -**Getting Help** - -If you encounter issues not covered here: - -- Review the [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) for detailed solutions -- Check [PDF Comparison Overview](/support/docs/smartui-pdf-comparison) for PDF-specific information -- See [PDF API Upload](/support/docs/smartui-pdf-api-upload) for alternative upload methods -- Visit [TestMu AI Support](https://www.testmuai.com/support) for additional resources -- Contact support at support@testmuai.com or use [24/7 Chat Support](https://www.testmuai.com/support) - -## Additional Resources - -- [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) -- [PDF Comparison Overview](/support/docs/smartui-pdf-comparison) -- [PDF API Upload](/support/docs/smartui-pdf-api-upload) -- [PDF Java SDK Upload](/support/docs/smartui-pdf-java-sdk) -- [Baseline Management](/support/docs/smartui-baseline-management) -- [Running Your First Project](/support/docs/smartui-running-your-first-project) -- [SmartUI API Documentation](https://www.testmuai.com/support/api-doc/) diff --git a/static/docs/smartui-pdf-comparison.md b/static/docs/smartui-pdf-comparison.md deleted file mode 100644 index b24db443d..000000000 --- a/static/docs/smartui-pdf-comparison.md +++ /dev/null @@ -1,55 +0,0 @@ -# SmartUI Visual Regression Testing for PDF Files - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This functionality is exclusive to our enterprise plan subscribers on SmartUI. For additional details or inquiries, please [contact us](https://www.testmuai.com/demo). - -## Prerequisites for Using SmartUI - -- Familiarity with HTTP APIs is essential. -- Visit the [`TestMu AI SmartUI`](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) page and log in with your credentials. -- Obtain your `LT_USERNAME` and `LT_ACCESS_KEY` by clicking on the `Access Key` button, located at the top right corner of your dashboard. - -In the following section, we will walk you through the process of conducting your first Visual Regression test using TestMu AI's SmartUI platform. - -## Getting Started with PDF Comparison - -## Step 1: Establishing a SmartUI Project - -To initiate a SmartUI PDF Comparison Project, adhere to the following instructions: - -1. Navigate to the [SmartUI Projects Page](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/). -2. Tap on the `new project` button. -3. Specify your platform type as `PDF`. -4. Provide your `project` name, designate `approvers`, and add `tags` (optional). -5. Confirm your entry by clicking on **Submit**. - -Once your project is active, retrieve your `Project Token` from the application. Here's an example of a project token: - -```bash -projectToken = "123456#1234abcd-****-****-****-************" -``` - -## Upload Methods - -SmartUI provides three different methods to upload PDFs for visual regression testing. Choose the method that best fits your workflow: - - API Upload - Upload PDFs using REST API for programmatic integration and automation. - - CLI Upload - Upload PDFs using command-line interface for quick testing and CI/CD integration. - - Java SDK Upload - Upload PDFs using Java SDK for enterprise applications and test automation frameworks. - -Only files in `.pdf` format are compatible with this feature. - -## Use Cases of Smart PDF Comparison - -- **Software Documentation**: Track changes and ensure consistency across document versions. -- **Legal & Compliance**: Spot differences in contracts or regulatory documents. -- **Design Validation**: Verify design updates and maintain visual consistency. -- **Proofreading**: Detect edits between document versions for quick review. -- **Quality Assurance**: Compare specs or blueprints to uphold standards. -- **Archiving**: Confirm integrity of records over time by highlighting modifications. diff --git a/static/docs/smartui-pdf-java-sdk.md b/static/docs/smartui-pdf-java-sdk.md deleted file mode 100644 index 978a57dc3..000000000 --- a/static/docs/smartui-pdf-java-sdk.md +++ /dev/null @@ -1,431 +0,0 @@ -# Upload PDFs via Java SDK - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This functionality is exclusive to our enterprise plan subscribers on SmartUI. For additional details or inquiries, please [contact us](https://www.testmuai.com/demo). - -## Prerequisites for Using SmartUI - -- Java 8 or higher installed on your system -- Maven or Gradle build tool -- Familiarity with Java development -- Visit the [`TestMu AI SmartUI`](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) page and log in with your credentials. -- Obtain your `LT_USERNAME` and `LT_ACCESS_KEY` by clicking on the `Access Key` button, located at the top right corner of your dashboard. - -## Step 1: Establishing a SmartUI Project - -To initiate a SmartUI PDF Comparison Project, adhere to the following instructions: - -1. Navigate to the [SmartUI Projects Page](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/). -2. Tap on the `new project` button. -3. Specify your platform type as `PDF`. -4. Provide your `project` name, designate `approvers`, and add `tags` (optional). -5. Confirm your entry by clicking on **Submit**. - -Once your project is active, retrieve your `Project Token` from the application. Here's an example of a project token: - -```bash -projectToken = "123456#1234abcd-****-****-****-************" -``` - -## Step 1: Clone the Sample Project - -First, clone the sample project to get started: - -```bash -git clone https://github.com/LambdaTest/junit-selenium-sample.git -cd junit-selenium-sample -``` - -## Step 2: Install the SmartUI Java SDK - -Add the SmartUI Java SDK to your `pom.xml`: - -```xml - -io.github.lambdatest -lambdatest-java-sdk -1.0.18 - -``` - -Then compile your project: - -```bash -mvn clean compile -``` - -## Step 3: Set up your credentials - -```bash -export LT_USERNAME="${YOUR_LAMBDATEST_USERNAME}" -export LT_ACCESS_KEY="${YOUR_LAMBDATEST_ACCESS_KEY}" -export PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -``` - -```bash -set LT_USERNAME="${YOUR_LAMBDATEST_USERNAME}" -set LT_ACCESS_KEY="${YOUR_LAMBDATEST_ACCESS_KEY}" -set PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -``` - -```powershell -$env:LT_USERNAME="${YOUR_LAMBDATEST_USERNAME}" -$env:LT_ACCESS_KEY="${YOUR_LAMBDATEST_ACCESS_KEY}" -$env:PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -``` - -## Step 4: Upload PDFs using Java SDK - -You can upload PDFs in two modes: - -Upload pre-existing PDFs from your local machine: - -"> 📁 **Sample File**: [`SmartuiPdfLocalTest.java`](https://github.com/LambdaTest/junit-selenium-sample/blob/master/src/test/java/com/smartuiPdf/SmartuiPdfLocalTest.java) - -```java - -public class SmartuiPdfLocalTest { -public void uploadLocalPdf() throws Exception { -String projectToken = System.getenv(PROJECT_TOKEN"); - -SmartUIConfig config = new SmartUIConfig() -.withProjectToken(projectToken) -.withFetchResult(true); - -SmartUIPdf pdfUploader = new SmartUIPdf(config); - -// Upload PDF file -String pdfPath = "path/to/your/document.pdf"; -FormattedResults result = pdfUploader.uploadPDF(pdfPath); - -System.out.println("Upload result: " + result); -} -} -``` - -Upload PDFs downloaded during TestMu AI cloud test execution: - -"> 📁 **Sample File**: [`SmartuiPdfCloudTest.java`](https://github.com/LambdaTest/junit-selenium-sample/blob/master/src/test/java/com/smartuiPdf/SmartuiPdfCloudTest.java) - -```java - -public class SmartuiPdfCloudTest { -public void uploadCloudPdf(WebDriver driver) throws Exception { -String projectToken = System.getenv(PROJECT_TOKEN"); - -// Download PDF from cloud session -String base64Content = (String) ((JavaScriptExecutor) driver) -.executeAsyncScript("lambda-file-content=LambdaTest.pdf"); - -// Convert base64 to PDF file -byte[] pdfBytes = Base64.getDecoder().decode(base64Content); -File pdfFile = new File("downloaded.pdf"); -try (FileOutputStream fos = new FileOutputStream(pdfFile)) { -fos.write(pdfBytes); -} - -// Upload to SmartUI -SmartUIConfig config = new SmartUIConfig() -.withProjectToken(projectToken) -.withFetchResult(true); - -SmartUIPdf pdfUploader = new SmartUIPdf(config); -FormattedResults result = pdfUploader.uploadPDF(pdfFile.getAbsolutePath()); - -System.out.println("Upload result: " + result); -} -} -``` - -## Step 5: Configuration Options - -| Method | Description | -|-------|-------------| -| `.withProjectToken(token)` | Required. Your SmartUI project token. | -| `.withFetchResult(true)` | Optional. Returns structured test results. | -| `.withBuildName("v2.1")` | Optional. Assign a custom build name. | - -## Step 6: Run your tests - -```bash -mvn test -``` - -## Advanced Java SDK Usage - -### Batch Upload Example - -```java -public class SmartuiPdfBatchTest { -public void uploadMultiplePdfs() throws Exception { -String projectToken = System.getenv("PROJECT_TOKEN"); - -SmartUIConfig config = new SmartUIConfig() -.withProjectToken(projectToken) -.withFetchResult(true) -.withBuildName("Batch-Upload-v1.0"); - -SmartUIPdf pdfUploader = new SmartUIPdf(config); - -String[] pdfPaths = { -"documents/report1.pdf", -"documents/report2.pdf", -"documents/specification.pdf" -}; - -for (String pdfPath : pdfPaths) { -FormattedResults result = pdfUploader.uploadPDF(pdfPath); -System.out.println("Uploaded " + pdfPath + ": " + result); -} -} -} -``` - -### Error Handling - -```java -public class SmartuiPdfErrorHandling { -public void uploadWithErrorHandling() { -try { -String projectToken = System.getenv("PROJECT_TOKEN"); - -SmartUIConfig config = new SmartUIConfig() -.withProjectToken(projectToken) -.withFetchResult(true); - -SmartUIPdf pdfUploader = new SmartUIPdf(config); -FormattedResults result = pdfUploader.uploadPDF("document.pdf"); - -System.out.println("Upload successful: " + result); - -} catch (Exception e) { -System.err.println("Upload failed: " + e.getMessage()); -e.printStackTrace(); -} -} -} -``` - -## Use Cases - -- **Enterprise Applications**: Integrate PDF testing into large-scale Java applications -- **Test Automation Frameworks**: Build comprehensive test suites with PDF validation -- **CI/CD Integration**: Automate PDF testing in Java-based deployment pipelines -- **Custom Tools**: Develop specialized tools for PDF comparison and validation - -## Best Practices - -**PDF File Management** - -- Use consistent naming conventions for PDF files -- Organize PDFs in logical directory structures -- Keep PDF files in version control when appropriate -- Document PDF sources and purposes - -**Example:** -```java -String[] pdfPaths = { -documents/reports/report-v1.0.pdf", -"documents/specs/spec-v2.1.pdf" -}; -``` - -**Project Token Management** - -- Store project token as environment variable -- Never commit tokens to version control -- Use different tokens for different environments -- Rotate tokens regularly - -**Build Naming** - -- Use meaningful build names that include version info -- Include date or version in build names -- Use consistent naming conventions - -**Example:** -```java -config.withBuildName(PDF-Comparison-v1.0-" + LocalDate.now()); -``` - -**Error Handling** - -- Always wrap upload calls in try-catch blocks -- Log errors for debugging -- Handle network failures gracefully -- Implement retry logic for transient failures - -**Batch Processing** - -- Process PDFs in batches for efficiency -- Monitor upload progress -- Handle partial failures in batch operations -- Use appropriate batch sizes - -**Batch Processing** - -- Process PDFs in batches for efficiency -- Monitor upload progress -- Handle partial failures in batch operations -- Use appropriate batch sizes - -## Troubleshooting - -**Issue: PDF Upload Fails** - -**Symptoms**: PDF upload returns error or fails silently - -**Possible Causes**: -- Invalid PDF file -- File path incorrect -- File size too large -- Network connectivity issues -- Project token incorrect - -**Solutions**: -1. Verify PDF file is valid and not corrupted: -```bash -file document.pdf -``` - -2. Check file path is correct: -```java -File pdfFile = new File(path/to/document.pdf"); -if (!pdfFile.exists()) { -throw new FileNotFoundException("PDF file not found"); -} -``` - -3. Verify file size is within limits - -4. Check network connectivity to TestMu AI servers - -5. Verify PROJECT_TOKEN is set correctly: -```bash -echo $PROJECT_TOKEN -``` - -**Issue: Project Not Found" Error** - -**Symptoms**: Error message indicating project cannot be found - -**Possible Causes**: -- Incorrect project token -- Project deleted or renamed -- Token from wrong project - -**Solutions**: -1. Verify project exists in SmartUI dashboard -2. Copy project token directly from Project Settings -3. Ensure token includes the project ID prefix (e.g., `123456#...`) -4. Check for extra spaces or quotes in token - -**Issue: Upload Returns Null or Empty Result** - -**Symptoms**: Upload completes but result is null or empty - -**Possible Causes**: -- `withFetchResult(false)` or not set -- Network timeout -- Server-side processing error - -**Solutions**: -1. Enable result fetching: -```java -config.withFetchResult(true); -``` - -2. Check upload response: -```java -FormattedResults result = pdfUploader.uploadPDF(pdfPath); -if (result == null) { -// Handle null result -} -``` - -3. Review error logs for server-side issues - -4. Retry upload if transient error - -**Issue: Maven Dependencies Not Resolving** - -**Symptoms**: Maven cannot find `lambdatest-java-sdk` or dependencies fail - -**Possible Causes**: -- Incorrect dependency version -- Maven repository access issues -- Network connectivity problems - -**Solutions**: -1. Check latest version on [Maven Central](https://mvnrepository.com/artifact/io.github.lambdatest/lambdatest-java-sdk) -2. Clear Maven cache: -```bash -mvn clean -``` -3. Verify internet connectivity for Maven repository access -4. Check pom.xml for version conflicts - -**Issue: Batch Upload Partially Fails** - -**Symptoms**: Some PDFs upload successfully, others fail - -**Possible Causes**: -- Individual file issues -- Network interruptions -- Timeout issues -- File size limits - -**Solutions**: -1. Implement individual error handling: -```java -for (String pdfPath : pdfPaths) { -try { -FormattedResults result = pdfUploader.uploadPDF(pdfPath); -System.out.println(Uploaded: " + pdfPath); -} catch (Exception e) { -System.err.println("Failed: " + pdfPath + " - " + e.getMessage()); -} -} -``` - -2. Verify each file individually -3. Check file sizes and formats -4. Implement retry logic for failed uploads - -**Issue: PDFs Not Appearing in Dashboard** - -**Symptoms**: Uploads complete but PDFs don't appear in SmartUI dashboard - -**Possible Causes**: -- Incorrect project token -- Project name mismatch -- Upload not completed -- Dashboard refresh needed - -**Solutions**: -1. Verify PROJECT_TOKEN is correct -2. Check project name matches exactly (case-sensitive) -3. Wait a few moments and refresh dashboard -4. Check upload response for errors -5. Review test execution logs - -**Getting Help** - -If you encounter issues not covered here: - -- Review the [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) for detailed solutions -- Check [PDF Comparison Overview](/support/docs/smartui-pdf-comparison) for PDF-specific information -- See [PDF API Upload](/support/docs/smartui-pdf-api-upload) for alternative upload methods -- Visit [TestMu AI Support](https://www.testmuai.com/support) for additional resources -- Contact support at support@testmuai.com or use [24/7 Chat Support](https://www.testmuai.com/support) - -## Additional Resources - -- [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) -- [PDF Comparison Overview](/support/docs/smartui-pdf-comparison) -- [PDF API Upload](/support/docs/smartui-pdf-api-upload) -- [PDF CLI Upload](/support/docs/smartui-pdf-cli-upload) -- [Baseline Management](/support/docs/smartui-baseline-management) -- [Running Your First Project](/support/docs/smartui-running-your-first-project) -- [SmartUI API Documentation](https://www.testmuai.com/support/api-doc/) diff --git a/static/docs/smartui-playwright-java-sdk.md b/static/docs/smartui-playwright-java-sdk.md deleted file mode 100644 index 6d0346b1f..000000000 --- a/static/docs/smartui-playwright-java-sdk.md +++ /dev/null @@ -1,521 +0,0 @@ -# Integrate SmartUI SDK with Playwright - Java - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The SmartUI SDK is a powerful tool that simplifies the process of capturing, comparing, and analyzing screenshots across multiple browsers and resolutions. By combining the strengths of Playwright's automation capabilities with SmartUI's visual testing features, you can: - -- Ensure your application's UI is consistent across all supported environments. -- Detect and address visual discrepancies early in the development lifecycle. -- Streamline your testing process with minimal setup and maximum coverage. - -This documentation provides step-by-step instructions for developers and QA engineers to integrate the SmartUI SDK with Playwright-Java, enabling reliable and efficient visual regression testing. - -## Prerequisites - -- Basic understanding of Command Line Interface and Playwright is required. -- Login to [TestMu AI SmartUI](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) with your credentials. - -## Steps to run your first test -Follow these steps to successfully run your first visual regression test on the TestMu AI platform with the SmartUI Playwright SDK integration. - -### Step 1: Create a SmartUI Project - -The first step is to create a project with the application in which we will combine all your builds run on the project. To create a SmartUI Project, follow these steps: - -1. Go to [Projects page](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) -2. Click on the `new project` button -3. Select the platform as CLI for executing your `SDK` tests. -4. Add name of the project, approvers for the changes found, tags for any filter or easy navigation. -5. Click on the **Submit**. - -### Step 2: Configure Your Test Suite - -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** -Download or Clone the code sample for the Java from the TestMu AI GitHub repository to run the tests on the SmartUI. - - View on GitHub - -### Step 3: Install the Dependencies - -Update your dependencies in `pom.xml` file Install required modules for TestMu AI SmartUI SDK in your frontend project. - -```bash -npm install @lambdatest/smartui-cli @lambdatest/playwright-driver playwright -mvn clean compile -``` - -If you face any problems executing tests with SmartUI-CLI `versions >= v4.x.x`, upgrade your Node.js version to `v20.3` or above. - -### Step 4: Setup your credentials - -```bash -export LT_USERNAME="${YOUR_LAMBDATEST_USERNAME}" -``` - -```bash -set LT_USERNAME="${YOUR_LAMBDATEST_USERNAME}" -``` - -```powershell -$env:LT_USERNAME="${YOUR_LAMBDATEST_USERNAME}" -``` - -### Step 5: Create and Configure SmartUI Config - -You can now configure your project configurations on using various available options to run your tests with the SmartUI integration. To generate the configuration file, please execute the following command: - -```bash -npx smartui config:create smartui-web.json -``` - -Once, the configuration file will be created, you will be seeing the default configuration pre-filled in the configuration file: - -```json title="/smartui-sdk-project/.smartui.json" -{ -"web": { -"browsers": [ -"chrome", -"firefox", -"safari", -"edge" -], -"viewports": [ -[ -1920 -], -[ -1366 -], -[ -1028 -] -] // Full Page screenshots are captured by default for web viewports -}, -"mobile": { -"devices": [ -"iPhone 14", //iPhone 14 viewport -"Galaxy S24" //Galaxy S24 viewport -], -"fullPage": true, //Full Page is true by default for mobile viewports -"orientation": "portrait" //Change to "landscape" for landscape snapshot -}, -"waitForTimeout": 1000, //Optional (Should only be used in case lazy-loading/async components are present) -"waitForPageRender": 50000, //Optional (Should only be used in case of websites which take more than 30s to load) -"enableJavaScript": false, //Enable javascript for all the screenshots of the project -"allowedHostnames": [] //Additional hostnames to capture assets from -} -``` -**Advanced options in SmartUI configuration** -- For capturing fullpage or viewport screenshots, please refer to this [documentation](/docs/smartui-sdk-config-options/#12-viewports) -- For the list of available mobile viewports, please refer to this [documentation](/docs/smartui-sdk-config-options/#list-of-supported-device-viewports) -- For more information about SmartUI config global options, please refer to this [documentation](/docs/smartui-sdk-config-options/#3-global-options-optional). - -### Step 6: Adding SmartUI function to take screenshot -You can incorporate SmartUI into your custom `Playwright` automation test (any platform) script by adding the `smartuiSnapshot` function in the required segment of Playwright script of which we would like to take the screenshot, as shown below: - -```java reference -https://github.com/LambdaTest/smartui-java-playwright-sample/blob/main/src/test/java/com/lambdatest/SmartUISDKPlaywrightCloud.java -``` - -### Step 6: Execute the Tests on SmartUI Cloud - -Execute visual regression tests on SmartUI using the following commands - -```bash -npx smartui exec -- mvn test -D suite="sdk-playwright-local-java.xml -``` - -You may use the `npx smartui --help` command in case you are facing issues during the execution of SmartUI commands in the CLI. - -## View SmartUI Results - -You have successfully integrated SmartUI SDK with your Playwright tests. Visit your SmartUI project to view builds and compare snapshots between different test runs. - -You can see the SmartUI dashboard to view the results. This will help you identify the Mismatches from the existing `Baseline` build and do the required visual testing. - -## Arguments supported in the `smartUISnapshot` function - -The following are the different options which are currently supported: - -| Key | Description | -| ------------------------- | ------------------------------------------------------------------------------------------------------------------------- | -| `driver` (instance) | The instance of the web driver used in your tests. | -| `"Screenshot Name"` (string) | Specify a name for the screenshot in your tests to match the same screenshot with the name from your baseline. | -| `options` (object) | Specify one or a combination of selectors in the `ignoreDOM` or `selectDOM` objects. These selectors can be based on `HTML DOM IDs, CSS classes, CSS selectors, or XPaths` used by your webpage. They define elements that should be excluded from or included in the visual comparison.| - -## Handling Dynamic Data in SmartUI SDK **** - -When conducting visual tests, you may encounter scenarios where certain elements within your application change between test runs. These changes might introduce inconsistencies in your test results.You can ignore / select specific element(s) to be removed from the comparison by parsing the options in the `smartuiSnapshot` function in the following way - -```java title="This is a sample for your configuration for Java to ignore by" ID" -List cssID = Arrays.asList(""); -Map options = new HashMap<>(); -Map> ignore = new HashMap<>(); -ignore.put("id", cssID); -options.put("ignoreDOM", ignore); - -driver.get("Required URL"); -SmartUISnapshot.smartuiSnapshot(driver, "Screenshot Name", options); -``` - -```java title="This is a sample for your configuration for Java to ignore by" Class" -List cssclass = Arrays.asList(""); -Map options = new HashMap<>(); -Map> ignore = new HashMap<>(); -ignore.put("class", cssclass); -options.put("ignoreDOM", ignore); - -driver.get("Required URL"); -SmartUISnapshot.smartuiSnapshot(driver, "Screenshot Name", options); -``` - -```java title="This is a sample for your configuration for Java to ignore by" XPath" -List path = Arrays.asList(""); -Map options = new HashMap<>(); -Map> ignore = new HashMap<>(); -ignore.put("xpath", path); -options.put("ignoreDOM", ignore); - -driver.get("Required URL"); -SmartUISnapshot.smartuiSnapshot(driver, "Screenshot Name", options); -``` - -```java title="This is a sample for your configuration for Java to ignore by CSS" Selector" -List selector = Arrays.asList(""); -Map options = new HashMap<>(); -Map> ignore = new HashMap<>(); -ignore.put("cssSelector", selector); -options.put("ignoreDOM", ignore); - -driver.get("Required URL"); -SmartUISnapshot.smartuiSnapshot(driver, "Screenshot Name", options); -``` - -```java title="This is a sample for your configuration for Java to select by" ID." -List cssID = Arrays.asList(""); -Map options = new HashMap<>(); -Map> select = new HashMap<>(); -select.put("id", cssID); -options.put("selectDOM", select); - -driver.get("Required URL"); -SmartUISnapshot.smartuiSnapshot(driver, "Screenshot Name", options); -``` - -```java title="This is a sample for your configuration for Java to select by" Class" -List cssclass = Arrays.asList(""); -Map options = new HashMap<>(); -Map> select = new HashMap<>(); -select.put("class", cssclass); -options.put("selectDOM", select); - -driver.get("Required URL"); -SmartUISnapshot.smartuiSnapshot(driver, "Screenshot Name", options); -``` - -```java title="This is a sample for your configuration for Java to select by" XPath" -List path = Arrays.asList(""); -Map options = new HashMap<>(); -Map> select = new HashMap<>(); -select.put("xpath", path); -options.put("selectDOM", select); - -driver.get("Required URL"); -SmartUISnapshot.smartuiSnapshot(driver, "Screenshot Name", options); -``` - -```java title="This is a sample for your webhook configuration for Java to select by CSS" Selector" -List selector = Arrays.asList(""); -Map options = new HashMap<>(); -Map> select = new HashMap<>(); -select.put("cssSelector", selector); -options.put("selectDOM", select); - -driver.get("Required URL"); -SmartUISnapshot.smartuiSnapshot(driver, "Screenshot Name", options); -``` - -## For capturing the screenshot of a specific element - -You can capture screenshots of targeted elements by leveraging various locator mechanisms such as XPath, CSS ID, class, and selectors. This precision-driven approach ensures accurate and specific visual regression testing for your web application's components. - -```java title="This is a sample for your configuration for Javas to capture an element by" ID." -HashMap options = new HashMap<>(); -HashMap locator = new HashMap<>(); -options.put("element", locator); -locator.put("id", "Required ID"); -driver.get("Required URL"); -SmartUISnapshot.smartuiSnapshot(driver, "Screenshot Name", options); -``` - -```java title="This is a sample for your configuration for Java to capture an element by" Class" -HashMap options = new HashMap<>(); -HashMap locator = new HashMap<>(); -options.put("element", locator); -locator.put("class", "Required Class"); -driver.get("Required URL"); -SmartUISnapshot.smartuiSnapshot(driver, "Screenshot Name", options); -``` - -```java title="This is a sample for your configuration for Java to capture an element by" XPath" -HashMap options = new HashMap<>(); -HashMap locator = new HashMap<>(); -options.put("element", locator); -locator.put("xpath", "Required Xpath"); -driver.get("Required URL"); -SmartUISnapshot.smartuiSnapshot(driver, "Screenshot Name", options); -``` - -```java title="This is a sample for your configuration for Java to capture an element by CSS" Selector" -HashMap options = new HashMap<>(); -HashMap locator = new HashMap<>(); -options.put("element", locator); -locator.put("cssSelector", "Required Selector"); -driver.get("Required URL"); -SmartUISnapshot.smartuiSnapshot(driver, "Screenshot Name", options); -``` - -## For capturing interactive lazy loading elements - -If you encounter difficulties loading interactive elements that appear on scroll in full-page screenshots, consider functionally incorporating a full-page scroll into your script before capturing the screenshot. This approach ensures the elements load first, facilitating the screenshot processing. - -```java Example for scrolling to bottom for lazy elements -//Rest of your code here - -@Test -public void basicTest() throws Exception { -System.out.println("Loading Url"); -driver.get("Required URL"); -quickScrollToBottom(); - -SmartUISnapshot.smartuiSnapshot(driver, "Screenshot Name"); -Thread.sleep(5000); // wait for 5 seconds -System.out.println("Test Finished"); -} - -public void quickScrollToBottom() throws InterruptedException { -long lastHeight = ((Number) ((JavaScriptExecutor) driver).executeScript("return document.body.scrollHeight")).longValue(); -while (true) { -((JavaScriptExecutor) driver).executeScript("window.scrollTo(0, document.body.scrollHeight);"); -Thread.sleep(2000); - -long newHeight = ((Number) ((JavaScriptExecutor) driver).executeScript("return document.body.scrollHeight")).longValue(); -if (newHeight == lastHeight) { -break; -} -lastHeight = newHeight; -} -((JavaScriptExecutor) driver).executeScript("window.scrollTo(0, 0);"); -Thread.sleep(1000); // wait for 1 second -} - -@AfterMethod -public void tearDown() { -if (driver != null) { -driver.quit(); -} -} -} -``` - -## Best Practices - -**Screenshot Naming** - -- Use descriptive, consistent names for screenshots -- Include page/component name in screenshot names -- Avoid special characters that might cause issues -- Use consistent naming conventions across your test suite - -**Example:** -```java -SmartUISnapshot.smartuiSnapshot(driver, HomePage-Header"); -SmartUISnapshot.smartuiSnapshot(driver, "ProductPage-MainContent"); -``` - -**Wait for Page Load** - -- Always wait for pages to fully load before taking screenshots -- Use Playwright's built-in wait methods for dynamic content -- Consider using `waitForTimeout` in configuration for lazy-loaded content - -**Example:** -```java -page.navigate(https://example.com"); -page.waitForSelector("#main-content", new Page.WaitForSelectorOptions().setState(WaitForSelectorState.VISIBLE)); -SmartUISnapshot.smartuiSnapshot(driver, "Page Loaded"); -``` - -**Handle Dynamic Content** - -- Use `ignoreDOM` for elements that change between runs -- Use `selectDOM` when you only need to compare specific areas -- Document why elements are ignored for future reference - -**Configuration Management** - -- Keep `.smartui.json` in version control -- Use environment variables for sensitive data -- Document custom configuration choices - -**Test Organization** - -- Group related screenshots in the same build -- Use meaningful build names -- Run tests in consistent environments - -**Troubleshooting** - -**Issue: Screenshots Not Appearing in Dashboard** - -**Symptoms**: Tests run successfully but no screenshots appear in SmartUI dashboard - -**Possible Causes**: -- Project token not set or incorrect -- Project name mismatch -- Network connectivity issues -- CLI not installed or outdated - -**Solutions**: -1. Verify `PROJECT_TOKEN` is set correctly: -```bash -echo $PROJECT_TOKEN -``` - -2. Check project name matches exactly (case-sensitive) - -3. Verify SmartUI CLI is installed: -```bash -npx smartui --version -``` - -4. Check network connectivity to TestMu AI servers - -5. Review test execution logs for error messages - -**Issue: Project Not Found" Error** - -**Symptoms**: Error message indicating project cannot be found - -**Possible Causes**: -- Incorrect project token -- Project deleted or renamed -- Token from wrong project - -**Solutions**: -1. Verify project exists in SmartUI dashboard -2. Copy project token directly from Project Settings -3. Ensure token includes the project ID prefix (e.g., `123456#...`) -4. Check for extra spaces or quotes in token - -**Issue: Screenshots Show Blank or Incorrect Content** - -**Symptoms**: Screenshots captured but show blank pages or incorrect content - -**Possible Causes**: -- Page not fully loaded -- JavaScript not executed -- Viewport size issues -- Timing issues - -**Solutions**: -1. Add explicit waits before screenshots: -```java -page.waitForSelector(#content", new Page.WaitForSelectorOptions().setState(WaitForSelectorState.VISIBLE)); -page.waitForLoadState(LoadState.NETWORKIDLE); -``` - -2. Enable JavaScript in configuration: -```json -{ -"enableJavaScript": true -} -``` - -3. Increase `waitForTimeout` in configuration - -4. Verify viewport size matches expected dimensions - -**Issue: Build Execution Fails** - -**Symptoms**: `npx smartui exec` command fails - -**Possible Causes**: -- Missing or incorrect configuration file -- Invalid JSON in configuration -- Port conflicts -- Permission issues - -**Solutions**: -1. Verify `.smartui.json` exists and is valid JSON -2. Check configuration file syntax -3. Try different port if default is in use: -```bash -npx smartui exec -P 5000 -- -``` -4. Check file permissions for configuration and project files - -**Issue: Maven Dependencies Not Resolving** - -**Symptoms**: Maven cannot find `lambdatest-java-sdk` or dependencies fail - -**Possible Causes**: -- Incorrect dependency version -- Maven repository access issues -- Network connectivity problems - -**Solutions**: -1. Check latest version on [Maven Central](https://mvnrepository.com/artifact/io.github.lambdatest/lambdatest-java-sdk) -2. Clear Maven cache: -```bash -mvn clean -``` -3. Verify internet connectivity for Maven repository access -4. Check pom.xml for version conflicts - -**Issue: Screenshot Names Not Matching Baseline** - -**Symptoms**: Screenshots appear as New" instead of comparing with baseline - -**Possible Causes**: -- Screenshot name changed -- Baseline doesn't exist -- Name contains special characters - -**Solutions**: -1. Ensure screenshot names are consistent across test runs -2. Verify baseline exists in project -3. Avoid special characters in screenshot names -4. Check for case sensitivity issues - -**Getting Help** - -If you encounter issues not covered here: - -- Review the [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) for detailed solutions -- Check [SmartUI Configuration Options](/support/docs/smartui-sdk-config-options) documentation -- See [Handling Dynamic Data](/support/docs/smartui-handle-dynamic-data) for dynamic content issues -- Visit [TestMu AI Support](https://www.testmuai.com/support) for additional resources -- Contact support at support@testmuai.com or use [24/7 Chat Support](https://www.testmuai.com/support) - -## Additional Resources - -- [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) -- [SmartUI Configuration Options](/support/docs/smartui-sdk-config-options) -- [Handling Dynamic Data](/support/docs/smartui-handle-dynamic-data) -- [Handling Lazy Loading](/support/docs/smartui-handle-lazy-loading) -- [Baseline Management](/support/docs/smartui-baseline-management) -- [Running Your First Project](/support/docs/smartui-running-your-first-project) -- [SmartUI API Documentation](https://www.testmuai.com/support/api-doc/) - -## Additional Resources - -- [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) -- [SmartUI Configuration Options](/support/docs/smartui-sdk-config-options) -- [Handling Dynamic Data](/support/docs/smartui-handle-dynamic-data) -- [Handling Lazy Loading](/support/docs/smartui-handle-lazy-loading) -- [Baseline Management](/support/docs/smartui-baseline-management) -- [Running Your First Project](/support/docs/smartui-running-your-first-project) -- [SmartUI API Documentation](https://www.testmuai.com/support/api-doc/) diff --git a/static/docs/smartui-playwright-python-sdk.md b/static/docs/smartui-playwright-python-sdk.md deleted file mode 100644 index 7e07c5316..000000000 --- a/static/docs/smartui-playwright-python-sdk.md +++ /dev/null @@ -1,537 +0,0 @@ -# Integrate SmartUI SDK with Playwright-Python - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Welcome to the world of simplified visual testing with the SmartUI SDK. - -Integrating seamlessly into your existing Playwright testing suite, SmartUI SDK revolutionizes the way you approach visual regression testing. Our robust solution empowers you to effortlessly capture, compare, and analyze screenshots across a multitude of browsers and resolutions, ensuring comprehensive coverage and accuracy in your visual testing endeavors. - -## Prerequisites - -- Basic understanding of Command Line Interface and Playwright is required. -- Login to [TestMu AI SmartUI](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) with your credentials. -- Install **virtualenv** which is the recommended way to run your tests. It will isolate the build from other setups you may have running and ensure that the tests run with the specified versions of the modules. - -```bash -pip install virtualenv -``` - -The following steps will guide you in running your first Visual Regression test on TestMu AI platform using SmartUI Playwright SDK integration. - -## Create a SmartUI Project - -The first step is to create a project with the application in which we will combine all your builds run on the project. To create a SmartUI Project, follow these steps: - -1. Go to [Projects page](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) -2. Click on the `new project` button -3. Select the platform as CLI for executing your `SDK` tests. -4. Add name of the project, approvers for the changes found, tags for any filter or easy navigation. -5. Click on the **Submit**. - -## Steps to run your first test - -Once you have created a SmartUI Project, you can generate screenshots by running automation scripts. Follow the below steps to successfully generate screenshots - -### Step 1: Create/Update your test - -- You can clone the sample repository to run `LambdaTest` automation tests with `SmartUI` and use the `SmartUI_SDK_LT_hub.py` file. - -```bash -git clone https://github.com/LambdaTest/smartui-playwright-python-sdk-sample -cd smartui-playwright-python-sdk-sample -``` - -- Create a virtual environment in your project folder (the environment name is arbitrary). - -```bash -virtualenv venv -``` - -- Activate the environment. - -```bash -source venv/bin/activate -``` - -### Step 2: Install the Dependencies - -Install required NPM modules for `TestMu AI SmartUI Playwright Python SDK` in your **Frontend** project. - -```bash -npm install @lambdatest/smartui-cli -``` - -If you face any problems executing tests with SmartUI-CLI `versions >= v4.x.x`, upgrade your Node.js version to `v20.3` or above. - -```bash -pip install lambdatest-playwright-driver -``` - -### Step 3: Configure your Project Token - -Setup your project token shown in the **SmartUI** app after creating your project. - -```bash -export PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -``` - -```bash -set PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -``` - -```powershell -$env:PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -``` - -### Step 4: Create and Configure SmartUI Config - -You can now configure your project configurations on using various available options to run your tests with the SmartUI integration. To generate the configuration file, please execute the following command: - -```bash -npx smartui config:create .smartui.json -``` - -Once, the configuration file will be created, you will be seeing the default configuration pre-filled in the configuration file: - -```json title="/smartui-sdk-project/.smartui.json" -{ -"web": { -"browsers": [ -"chrome", -"firefox", -"safari", -"edge" -], -"viewports": [ -[ -1920 -], -[ -1366 -], -[ -1028 -] -] // Full Page screenshots are captured by default for web viewports -}, -"mobile": { -"devices": [ -"iPhone 14", //iPhone 14 viewport -"Galaxy S24" //Galaxy S24 viewport -], -"fullPage": true, //Full Page is true by default for mobile viewports -"orientation": "portrait" //Change to "landscape" for landscape snapshot -}, -"waitForTimeout": 1000, //Optional (Should only be used in case lazy-loading/async components are present) -"waitForPageRender": 50000, //Optional (Should only be used in case of websites which take more than 30s to load) -"enableJavaScript": false, //Enable javascript for all the screenshots of the project -"allowedHostnames": [] //Additional hostnames to capture assets from -} -``` -**Advanced options in SmartUI configuration** -- For capturing fullpage or viewport screenshots, please refer to this [documentation](/docs/smartui-sdk-config-options/#12-viewports) -- For the list of available mobile viewports, please refer to this [documentation](/docs/smartui-sdk-config-options/#list-of-supported-device-viewports) -- For more information about SmartUI config global options, please refer to this [documentation](/docs/smartui-sdk-config-options/#3-global-options-optional). - -### **Step 5: Adding SmartUI function to take screenshot - -- You can incorporate SmartUI into your custom `Playwright` automation test (any platform) script by adding the `smartuiSnapshot` function in the required segment of playwright script of which we would like to take the screenshot, as shown below: - -```python -ffrom playwright.sync_api import sync_playwright, Playwright -from lambdatest_playwright_driver import smartui_snapshot - -def run(playwright: Playwright): -webkit = playwright.webkit -browser = webkit.launch() -context = browser.new_context() -page = context.new_page() - -try: -page.goto("Required URL") -smartui_snapshot(page, "Screenshot Name") - -print("SmartUI snapshot taken successfully!") - -except Exception as e: -print(f"Error occurred during SmartUI snapshot: {e}") - -finally: -browser.close() - -with sync_playwright() as playwright: -run(playwright) -``` - -### Step 6: Execute the Tests on SmartUI Cloud - -Execute `visual regression tests` on SmartUI using the following commands - -```bash -npx smartui --config .smartui.json exec -- python SmartUI_SDK_LT_hub.py -``` - -You may use the `npx smartui --help` command in case you are facing issues during the execution of SmartUI commands in the CLI. - -## View SmartUI Results - -You have successfully integrated SmartUI SDK with your Playwright tests. Visit your SmartUI project to view builds and compare snapshots between different test runs. - -You can see the SmartUI dashboard to view the results. This will help you identify the Mismatches from the existing `Baseline` build and do the required visual testing. - -## Arguments supported in the `smartUISnapshot` function - -The following are the different options which are currently supported: - -| Key | Description | -| ------------------------- | ------------------------------------------------------------------------------------------------------------------------- | -| `page` (instance) | The instance of the page used in your tests. | -| `"Screenshot Name"` (string) | Specify a name for the screenshot in your tests to match the same screenshot with the name from your baseline. | -| `options` (object) | Specify one or a combination of selectors in the `ignoreDOM` or `selectDOM` objects. These selectors can be based on `HTML DOM IDs, CSS classes, CSS selectors, or XPaths` used by your webpage. They define elements that should be excluded from or included in the visual comparison.| - -## Handling Dynamic Data in SmartUI SDK **** - -When conducting visual tests, you may encounter scenarios where certain elements within your application change between test runs. These changes might introduce inconsistencies in your test results.You can ignore / select specific element(s) to be removed from the comparison by parsing the options in the `smartuiSnapshot` function in the following way - -```rb title="This is a sample for your configuration for Python to ignore by ID" -options = { -ignoreDOM: { -id: ["ID-1", "ID-2"], -} -} -page.goto("Required URL") -smartui_snapshot(page,"", options) -``` - -```py title="This is a sample for your configuration for Python to ignore by Class" -options = { -ignoreDOM: { -class: ["Class-1", "Class-2"], -} -} -page.goto("Required URL") -smartui_snapshot(page,"", options) -``` - -```py title="This is a sample for your configuration for Python to ignore by XPath" -options = { -ignoreDOM: { -xpath: ["Xpath-1", "Xpath-2"], -} -} -page.goto("Required URL") -smartui_snapshot(page,"", options) -``` - -```py title="This is a sample for your configuration for Python to ignore by CSS Selector" -options = { -ignoreDOM: { -cssSelector: ["CSS-Selector-1", "CSS-Selector-2"], -} -} -page.goto("Required URL") -smartui_snapshot(page,"", options) -``` - -```py title="This is a sample for your configuration for Python to select by ID." -options = { -selectDOM: { -id: ["ID-1", "ID-2"], -} -} -page.goto("Required URL") -smartui_snapshot(page,"", options) -``` - -```py title="This is a sample for your configuration for Python to select by Class" -options = { -selectDOM: { -class: ["Class-1", "Class-2"], -} -} -page.goto("Required URL") -smartui_snapshot(page,"", options) -``` - -```py title="This is a sample for your configuration for Python to select by XPath" -options = { -selectDOM: { -xpath: ["Xpath-1", "Xpath-2"], -} -} -page.goto("Required URL") -smartui_snapshot(page,"", options) -``` - -```py title="This is a sample for your webhook configuration for Python to select by CSS Selector" -options = { -selectDOM: { -cssSelector: ["CSS-Selector-1", "CSS-Selector-2"], -} -} -page.goto("Required URL") -smartui_snapshot(page,"", options) -``` - -## For capturing the screenshot of a specific element - -You can capture screenshots of targeted elements by leveraging various locator mechanisms such as XPath, CSS ID, class, and selectors. This precision-driven approach ensures accurate and specific visual regression testing for your web application's components. - -```py title="This is a sample for your configuration for Python to capture an element by ID." -options = { -element: { -id: 'Required ID', -} -} -page.goto("Required URL") -smartui_snapshot(page,"", options) -``` - -```py title="This is a sample for your configuration for Python to capture an element by Class" -options = { -element: { -class: 'Required Class', -} -} -page.goto("Required URL") -smartui_snapshot(page,"", options) -``` - -```py title="This is a sample for your configuration for Python to capture an element by XPath" -options = { -element: { -xpath: 'Required Xpath', -} -} -page.goto("Required URL") -smartui_snapshot(page,"", options) -``` - -```py title="This is a sample for your webhook configuration for Python to capture an element by CSS Selector" -options = { -element: { -cssSelector: 'Required CSS Selector', -} -} -page.goto("Required URL") -smartui_snapshot(page,"", options) -``` - -## Best Practices - -**Screenshot Naming** - -- Use descriptive, consistent names for screenshots -- Include page/component name in screenshot names -- Avoid special characters that might cause issues -- Use consistent naming conventions across your test suite - -**Example:** -```python -smartui_snapshot(page, "HomePage-Header") -smartui_snapshot(page, "ProductPage-MainContent") -``` - -**Wait for Page Load** - -- Always wait for pages to fully load before taking screenshots -- Use Playwright's built-in wait methods for dynamic content -- Consider using `waitForTimeout` in configuration for lazy-loaded content - -**Example:** -```python -page.goto("https://example.com") -page.wait_for_selector("#main-content", state="visible") -page.wait_for_load_state("networkidle") -smartui_snapshot(page, "Page Loaded") -``` - -**Handle Dynamic Content** - -- Use `ignoreDOM` for elements that change between runs -- Use `selectDOM` when you only need to compare specific areas -- Document why elements are ignored for future reference - -**Configuration Management** - -- Keep `.smartui.json` in version control -- Use environment variables for sensitive data -- Document custom configuration choices - -**Test Organization** - -- Group related screenshots in the same build -- Use meaningful build names -- Run tests in consistent environments - -**Troubleshooting** - -**Issue: Screenshots Not Appearing in Dashboard** - -**Symptoms**: Tests run successfully but no screenshots appear in SmartUI dashboard - -**Possible Causes**: -- Project token not set or incorrect -- Project name mismatch -- Network connectivity issues -- CLI not installed or outdated - -**Solutions**: -1. Verify `PROJECT_TOKEN` is set correctly: -```bash -echo $PROJECT_TOKEN -``` - -2. Check project name matches exactly (case-sensitive) - -3. Verify SmartUI CLI is installed: -```bash -npx smartui --version -``` - -4. Check network connectivity to TestMu AI servers - -5. Review test execution logs for error messages - -**Issue: "Project Not Found" Error** - -**Symptoms**: Error message indicating project cannot be found - -**Possible Causes**: -- Incorrect project token -- Project deleted or renamed -- Token from wrong project - -**Solutions**: -1. Verify project exists in SmartUI dashboard -2. Copy project token directly from Project Settings -3. Ensure token includes the project ID prefix (e.g., `123456#...`) -4. Check for extra spaces or quotes in token - -**Issue: Screenshots Show Blank or Incorrect Content** - -**Symptoms**: Screenshots captured but show blank pages or incorrect content - -**Possible Causes**: -- Page not fully loaded -- JavaScript not executed -- Viewport size issues -- Timing issues - -**Solutions**: -1. Add explicit waits before screenshots: -```python -page.wait_for_selector("#content", state="visible") -page.wait_for_selector(".main-content", state="visible") -page.wait_for_load_state("networkidle") -``` - -2. Enable JavaScript in configuration: -```json -{ -"enableJavaScript": true -} -``` - -3. Increase `waitForTimeout` in configuration - -4. Verify viewport size matches expected dimensions - -**Issue: Build Execution Fails** - -**Symptoms**: `npx smartui exec` command fails - -**Possible Causes**: -- Missing or incorrect configuration file -- Invalid JSON in configuration -- Port conflicts -- Permission issues - -**Solutions**: -1. Verify `.smartui.json` exists and is valid JSON -2. Check configuration file syntax -3. Try different port if default is in use: -```bash -npx smartui exec -P 5000 -- -``` -4. Check file permissions for configuration and project files - -**Issue: pip Dependencies Not Resolving** - -**Symptoms**: pip cannot find `lambdatest-playwright-driver` or package installation fails - -**Possible Causes**: -- Incorrect package version -- pip registry access issues -- Network connectivity problems -- Python version mismatch - -**Solutions**: -1. Check latest version: -```bash -pip index versions lambdatest-playwright-driver -``` - -2. Clear pip cache: -```bash -pip cache purge -``` - -3. Verify internet connectivity for PyPI access - -4. Check Python version compatibility: -```bash -python --version -``` - -5. Use virtual environment: -```bash -python -m venv venv -source venv/bin/activate # On Windows: venv\Scripts\activate -pip install lambdatest-playwright-driver -``` - -**Issue: Screenshot Names Not Matching Baseline** - -**Symptoms**: Screenshots appear as "New" instead of comparing with baseline - -**Possible Causes**: -- Screenshot name changed -- Baseline doesn't exist -- Name contains special characters - -**Solutions**: -1. Ensure screenshot names are consistent across test runs -2. Verify baseline exists in project -3. Avoid special characters in screenshot names -4. Check for case sensitivity issues - -**Getting Help** - -If you encounter issues not covered here: - -- Review the [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) for detailed solutions -- Check [SmartUI Configuration Options](/support/docs/smartui-sdk-config-options) documentation -- See [Handling Dynamic Data](/support/docs/smartui-handle-dynamic-data) for dynamic content issues -- Visit [TestMu AI Support](https://www.testmuai.com/support) for additional resources -- Contact support at support@testmuai.com or use [24/7 Chat Support](https://www.testmuai.com/support) - -## Additional Resources - -- [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) -- [SmartUI Configuration Options](/support/docs/smartui-sdk-config-options) -- [Handling Dynamic Data](/support/docs/smartui-handle-dynamic-data) -- [Handling Lazy Loading](/support/docs/smartui-handle-lazy-loading) -- [Baseline Management](/support/docs/smartui-baseline-management) -- [Running Your First Project](/support/docs/smartui-running-your-first-project) -- [SmartUI API Documentation](https://www.testmuai.com/support/api-doc/) - -## Additional Resources - -- [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) -- [SmartUI Configuration Options](/support/docs/smartui-sdk-config-options) -- [Handling Dynamic Data](/support/docs/smartui-handle-dynamic-data) -- [Handling Lazy Loading](/support/docs/smartui-handle-lazy-loading) -- [Baseline Management](/support/docs/smartui-baseline-management) -- [Running Your First Project](/support/docs/smartui-running-your-first-project) -- [SmartUI API Documentation](https://www.testmuai.com/support/api-doc/) diff --git a/static/docs/smartui-playwright-sdk.md b/static/docs/smartui-playwright-sdk.md deleted file mode 100644 index 3600f46d7..000000000 --- a/static/docs/smartui-playwright-sdk.md +++ /dev/null @@ -1,540 +0,0 @@ -# Integrate SmartUI SDK with Playwright-JavaScript - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Welcome to the world of simplified visual testing with the SmartUI SDK. - -Integrating seamlessly into your existing Playwright testing suite, SmartUI SDK revolutionizes the way you approach visual regression testing. Our robust solution empowers you to effortlessly capture, compare, and analyze screenshots across a multitude of browsers and resolutions, ensuring comprehensive coverage and accuracy in your visual testing endeavors. - -## Prerequisites - -- Basic understanding of Command Line Interface and Playwright is required. -- Login to [TestMu AI SmartUI](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) with your credentials. - -The following steps will guide you in running your first Visual Regression test on TestMu AI platform using SmartUI Playwright SDK integration. - -## Create a SmartUI Project - -The first step is to create a project with the application in which we will combine all your builds run on the project. To create a SmartUI Project, follow these steps: - -1. Go to [Projects page](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) -2. Click on the `new project` button -3. Select the platform as CLI for executing your `SDK` tests. -4. Add name of the project, approvers for the changes found, tags for any filter or easy navigation. -5. Click on the **Submit**. - -## Steps to run your first test - -Once you have created a SmartUI Project, you can generate screenshots by running automation scripts. Follow the below steps to successfully generate screenshots - -### Step 1: Create/Update your test - -You can clone the sample repository to run `LambdaTest` automation tests with `SmartUI` and use the `plawrightCloud.js` file present in the `sdk` folder. - -```bash -git clone https://github.com/LambdaTest/smartui-playwright-sample -cd smartui-playwright-sample/sdk -``` -### Step 2: Install the Dependencies - -Install required NPM modules for `TestMu AI SmartUI Playwright SDK` in your **Frontend** project. - -```bash -npm install @lambdatest/smartui-cli @lambdatest/playwright-driver playwright -``` - -If you face any problems executing tests with SmartUI-CLI `versions >= v4.x.x`, upgrade your Node.js version to `v20.3` or above. - -### Step 3: Configure your Project Token - -Setup your project token shown in the **SmartUI** app after creating your project. - -```bash -export PROJECT_TOKEN=123456#1234abcd-****-****-****-************" -``` - -```bash -set PROJECT_TOKEN=123456#1234abcd-****-****-****-************" -``` - -```powershell -$env:PROJECT_TOKEN=123456#1234abcd-****-****-****-************" -``` - -### Step 4: Create and Configure SmartUI Config - -You can now configure your project configurations on using various available options to run your tests with the SmartUI integration. To generate the configuration file, please execute the following command: - -```bash -npx smartui config:create .smartui.json -``` - -Once, the configuration file will be created, you will be seeing the default configuration pre-filled in the configuration file: - -```json title="/smartui-sdk-project/.smartui.json" -{ -"web": { -"browsers": [ -"chrome", -"firefox", -"safari", -"edge" -], -"viewports": [ -[ -1920 -], -[ -1366 -], -[ -1028 -] -] // Full Page screenshots are captured by default for web viewports -}, -"mobile": { -"devices": [ -"iPhone 14", //iPhone 14 viewport -"Galaxy S24" //Galaxy S24 viewport -], -"fullPage": true, //Full Page is true by default for mobile viewports -"orientation": "portrait" //Change to "landscape" for landscape snapshot -}, -"waitForTimeout": 1000, //Optional (Should only be used in case lazy-loading/async components are present) -"waitForPageRender": 50000, //Optional (Should only be used in case of websites which take more than 30s to load) -"enableJavaScript": false, //Enable javascript for all the screenshots of the project -"allowedHostnames": [] //Additional hostnames to capture assets from -} -``` -**Advanced options in SmartUI configuration** -- For capturing fullpage or viewport screenshots, please refer to this [documentation](/docs/smartui-sdk-config-options/#12-viewports) -- For the list of available mobile viewports, please refer to this [documentation](/docs/smartui-sdk-config-options/#list-of-supported-device-viewports) -- For more information about SmartUI config global options, please refer to this [documentation](/docs/smartui-sdk-config-options/#3-global-options-optional). - -### Step 5: Adding SmartUI function to take screenshot - -- You can incorporate SmartUI into your custom `Playwright` automation test (any platform) script by adding the `smartuiSnapshot` function in the required segment of Playwright script of which we would like to take the screenshot, as shown below: - -```js -const { chromium } = require("playwright"); -const smartuiSnapshot = require("@lambdatest/playwright-driver"); - -(async () => { -// Launch a local browser instance -const browser = await chromium.launch({ -headless: false, // Set to false to see the browser UI -}); - -const page = await browser.newPage(); - -// Navigate to the desired URL -await page.goto("https://www.lambdatest.com"); - -// Use smartuiSnapshot to take a visual snapshot locally -await smartuiSnapshot.smartuiSnapshot(page, "LambdaTest"); - -// Close the browser -await browser.close(); -})(); - -``` - -### Step 6: Execute the Tests on SmartUI Cloud - -Execute `visual regression tests` on SmartUI using the following commands - -```bash -npx smartui exec node playwrightCloud.js --config .smartui.json -``` - -You may use the `npx smartui --help` command in case you are facing issues during the execution of SmartUI commands in the CLI. - -## View SmartUI Results - -You have successfully integrated SmartUI SDK with your Playwright tests. Visit your SmartUI project to view builds and compare snapshots between different test runs. - -You can see the SmartUI dashboard to view the results. This will help you identify the Mismatches from the existing `Baseline` build and do the required visual testing. - -## Arguments supported in the `smartUISnapshot` function - -The following are the different options which are currently supported: - -| Key | Description | -| ------------------------- | ------------------------------------------------------------------------------------------------------------------------- | -| `page` (instance) | The instance of page used in your tests. | -| `"Screenshot Name"` (string) | Specify a name for the screenshot in your tests to match the same screenshot with the name from your baseline. | -| `options` (object) | Specify one or a combination of selectors in the `ignoreDOM` or `selectDOM` objects. These selectors can be based on `HTML DOM IDs, CSS classes, CSS selectors, or XPaths` used by your webpage. They define elements that should be excluded from or included in the visual comparison.| - -## Handling Dynamic Data in SmartUI SDK **** - -When conducting visual tests, you may encounter scenarios where certain elements within your application change between test runs. These changes might introduce inconsistencies in your test results.You can ignore / select specific element(s) to be removed from the comparison by parsing the options in the `smartuiSnapshot` function in the following way - -```js title="This is a sample for your configuration for Playwright to ignore by" ID" -let options = { -ignoreDOM: { -id: ["ID-1", "ID-2"], -} -} -await page.goto('Required URL'); -await smartuiSnapshot.smartuiSnapshot(page, "Screenshot Name", options); -``` - -```js title="This is a sample for your configuration for Playwright to ignore by" Class" -let options = { -ignoreDOM: { -class: ["Class-1", "Class-2"], -} -} -await page.goto('Required URL'); -await smartuiSnapshot.smartuiSnapshot(page, "Screenshot Name", options); -``` - -```js title="This is a sample for your configuration for Playwright to ignore by" XPath" -let options = { -ignoreDOM: { -xpath: ["Xpath-1", "Xpath-2"], -} -} -await page.goto('Required URL'); -await smartuiSnapshot.smartuiSnapshot(page, "Screenshot Name", options); -``` - -```js title="This is a sample for your configuration for Playwright to ignore by CSS" Selector" -let options = { -ignoreDOM: { -cssSelector: ["CSS-Selector-1", "CSS-Selector-2"], -} -} -await page.goto('Required URL'); -await smartuiSnapshot.smartuiSnapshot(page, "Screenshot Name", options); -``` - -```js title="This is a sample for your configuration for Playwright to select by" ID." -let options = { -selectDOM: { -id: ["ID-1", "ID-2"], -} -} -await page.goto('Required URL'); -await smartuiSnapshot.smartuiSnapshot(page, "Screenshot Name", options); -``` - -```js title="This is a sample for your configuration for Playwright to select by" Class" -let options = { -selectDOM: { -class: ["Class-1", "Class-2"], -} -} -await page.goto('Required URL'); -await smartuiSnapshot.smartuiSnapshot(page, "Screenshot Name", options); -``` - -```js title="This is a sample for your configuration for Playwright to select by" XPath" -let options = { -selectDOM: { -xpath: ["Xpath-1", "Xpath-2"], -} -} -await page.goto('Required URL'); -await smartuiSnapshot.smartuiSnapshot(page, "Screenshot Name", options); -``` - -```js title="This is a sample for your webhook configuration for Playwright to select by CSS" Selector" -let options = { -selectDOM: { -cssSelector: ["CSS-Selector-1", "CSS-Selector-2"], -} -} -await page.goto('Required URL'); -await smartuiSnapshot.smartuiSnapshot(page, "Screenshot Name", options); -``` - -## For capturing the screenshot of a specific element - -You can capture screenshots of targeted elements by leveraging various locator mechanisms such as XPath, CSS ID, class, and selectors. This precision-driven approach ensures accurate and specific visual regression testing for your web application's components. - -```js title="This is a sample for your configuration for Playwright to capture an element by" ID." -let options = { -element: { -id: 'Required ID', -} -}; -await page.goto('Required URL'); -await smartuiSnapshot.smartuiSnapshot(page, "Screenshot Name", options); -``` - -```js title="This is a sample for your configuration for Playwright to capture an element by" Class" -let options = { -element: { -class: 'Required Class', -} -}; -await page.goto('Required URL'); -await smartuiSnapshot.smartuiSnapshot(page, "Screenshot Name", options); -``` - -```js title="This is a sample for your configuration for Playwright to capture an element by" XPath" -let options = { -element: { -xpath: 'Required Xpath', -} -}; -await page.goto('Required URL'); -await smartuiSnapshot.smartuiSnapshot(page, "Screenshot Name", options); -``` - -```js title="This is a sample for your webhook configuration for Playwright to capture an element by CSS" Selector" -let options = { -element: { -cssSelector: 'Required CSS Selector', -} -}; -await page.goto('Required URL'); -await smartuiSnapshot.smartuiSnapshot(page, "Screenshot Name", options); -``` - -## For capturing interactive lazy loading elements - -If you encounter difficulties loading interactive elements that appear on scroll in full-page screenshots, consider functionally incorporating a full-page scroll into your script before capturing the screenshot. This approach ensures the elements load first, facilitating the screenshot processing. - -```js Example for scrolling to bottom for lazy elements -const { chromium } = require('playwright'); -const smartuiSnapshot = require('@lambdatest/playwright-driver'); - -(async () => { -const browser = await chromium.launch({ headless: false }); // Set headless: false to see the browser UI -const page = await browser.newPage(); - -try { -await page.goto('Required URL'); - -// Function to scroll to the bottom of the page -async function quickScrollToBottom(lastPageWait) { -await page.evaluate(async (lastPageWait) => { -const scrollToBottom = async (lastPageWait) => { -const getScrollHeight = () => document.body.scrollHeight; -let lastHeight = await getScrollHeight(); -let currentHeight = 0; - -while (currentHeight < lastHeight) { -window.scrollTo(0, lastHeight); -await new Promise(resolve => setTimeout(resolve, 1000)); // Wait for page to load -currentHeight = lastHeight; -lastHeight = await getScrollHeight(); -} - -if (lastPageWait) { -await new Promise(resolve => setTimeout(resolve, lastPageWait)); // Additional wait at the bottom -} - -// Scroll back to the top after reaching the bottom -window.scrollTo(0, 0); -await new Promise(resolve => setTimeout(resolve, 1000)); // Wait for scroll to top -}; - -await scrollToBottom(lastPageWait); -}, lastPageWait); -} - -await quickScrollToBottom(100); // Adjust wait time as needed -await smartuiSnapshot.smartuiSnapshot(page, "Screenshot Name"); - -} finally { -await browser.close(); -} -})(); -``` - -## Best Practices - -**Screenshot Naming** - -- Use descriptive, consistent names for screenshots -- Include page/component name in screenshot names -- Avoid special characters that might cause issues -- Use consistent naming conventions across your test suite - -**Example:** -```javascript -await smartuiSnapshot.smartuiSnapshot(page, HomePage-Header"); -await smartuiSnapshot.smartuiSnapshot(page, "ProductPage-MainContent"); -``` - -**Wait for Page Load** - -- Always wait for pages to fully load before taking screenshots -- Use Playwright's built-in wait methods for dynamic content -- Consider using `waitForTimeout` in configuration for lazy-loaded content - -**Example:** -```javascript -await page.goto('https://example.com'); -await page.waitForSelector('#main-content', { state: 'visible' }); -await smartuiSnapshot.smartuiSnapshot(page, Page Loaded"); -``` - -**Handle Dynamic Content** - -- Use `ignoreDOM` for elements that change between runs -- Use `selectDOM` when you only need to compare specific areas -- Document why elements are ignored for future reference - -**Configuration Management** - -- Keep `.smartui.json` in version control -- Use environment variables for sensitive data -- Document custom configuration choices - -**Test Organization** - -- Group related screenshots in the same build -- Use meaningful build names -- Run tests in consistent environments - -## Troubleshooting - -**Issue: Screenshots Not Appearing in Dashboard** - -**Symptoms**: Tests run successfully but no screenshots appear in SmartUI dashboard - -**Possible Causes**: -- Project token not set or incorrect -- Project name mismatch -- Network connectivity issues -- CLI not installed or outdated - -**Solutions**: -1. Verify `PROJECT_TOKEN` is set correctly: -```bash -echo $PROJECT_TOKEN -``` - -2. Check project name matches exactly (case-sensitive) - -3. Verify SmartUI CLI is installed: -```bash -npx smartui --version -``` - -4. Check network connectivity to TestMu AI servers - -5. Review test execution logs for error messages - -**Issue: Project Not Found" Error** - -**Symptoms**: Error message indicating project cannot be found - -**Possible Causes**: -- Incorrect project token -- Project deleted or renamed -- Token from wrong project - -**Solutions**: -1. Verify project exists in SmartUI dashboard -2. Copy project token directly from Project Settings -3. Ensure token includes the project ID prefix (e.g., `123456#...`) -4. Check for extra spaces or quotes in token - -**Issue: Screenshots Show Blank or Incorrect Content** - -**Symptoms**: Screenshots captured but show blank pages or incorrect content - -**Possible Causes**: -- Page not fully loaded -- JavaScript not executed -- Viewport size issues -- Timing issues - -**Solutions**: -1. Add explicit waits before screenshots: -```javascript -await page.waitForSelector('#content', { state: 'visible' }); -await page.waitForLoadState('networkidle'); -``` - -2. Enable JavaScript in configuration: -```json -{ -enableJavaScript": true -} -``` - -3. Increase `waitForTimeout` in configuration - -4. Verify viewport size matches expected dimensions - -**Issue: Build Execution Fails** - -**Symptoms**: `npx smartui exec` command fails - -**Possible Causes**: -- Missing or incorrect configuration file -- Invalid JSON in configuration -- Port conflicts -- Permission issues - -**Solutions**: -1. Verify `.smartui.json` exists and is valid JSON -2. Check configuration file syntax -3. Try different port if default is in use: -```bash -npx smartui exec -P 5000 -- -``` -4. Check file permissions for configuration and project files - -**Issue: npm Dependencies Not Resolving** - -**Symptoms**: npm cannot find `@lambdatest/playwright-driver` or `@lambdatest/smartui-cli` - -**Possible Causes**: -- Incorrect package version -- npm registry access issues -- Network connectivity problems - -**Solutions**: -1. Check latest versions on npm: -```bash -npm view @lambdatest/playwright-driver version -npm view @lambdatest/smartui-cli version -``` - -2. Clear npm cache: -```bash -npm cache clean --force -``` - -3. Verify internet connectivity for npm registry access - -4. Check package.json for version conflicts - -**Issue: Screenshot Names Not Matching Baseline** - -**Symptoms**: Screenshots appear as New" instead of comparing with baseline - -**Possible Causes**: -- Screenshot name changed -- Baseline doesn't exist -- Name contains special characters - -**Solutions**: -1. Ensure screenshot names are consistent across test runs -2. Verify baseline exists in project -3. Avoid special characters in screenshot names -4. Check for case sensitivity issues - -**Getting Help** - -If you encounter issues not covered here: - -- Review the [SmartUI Configuration Options](/support/docs/smartui-sdk-config-options) documentation -- Check [Handling Dynamic Data](/support/docs/smartui-handle-dynamic-data) for dynamic content issues -- Visit [TestMu AI Support](https://www.testmuai.com/support) for additional resources -- Contact support at support@testmuai.com or use [24/7 Chat Support](https://www.testmuai.com/support) - -## Additional Resources - -- [SmartUI Configuration Options](/support/docs/smartui-sdk-config-options) -- [Handling Dynamic Data](/support/docs/smartui-handle-dynamic-data) -- [Handling Lazy Loading](/support/docs/smartui-handle-lazy-loading) -- [Running Your First Project](/support/docs/smartui-running-your-first-project) -- [SmartUI API Documentation](https://www.testmuai.com/support/api-doc/) diff --git a/static/docs/smartui-project-settings.md b/static/docs/smartui-project-settings.md deleted file mode 100644 index f03a76c43..000000000 --- a/static/docs/smartui-project-settings.md +++ /dev/null @@ -1,273 +0,0 @@ -# Project Settings - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This guide provides detailed information on how to navigate and configure the various options available in SmartUI's Project Settings. SmartUI offers a wide array of settings to customize your project, manage approvals, configure comparison parameters, and more. Before delving into the Project Settings, ensure you meet the following prerequisites: - -## Prerequisites for Accessing Project Settings in SmartUI - -To access the Project Settings in SmartUI, you must meet the following requirements: - -1. You need an active account with [TestMu AI](https://www.testmuai.com/login/). -2. Ensure you have an active [subscription plan](https://www.testmuai.com/pricing) with a valid screenshots limit. -3. You must have created a project within the SmartUI web app. - -## Navigating to Project Settings - -To access the Project Settings, follow these simple steps: - -1. Log in to your SmartUI account and navigate to your desired SmartUI project and click on it. - -2. Click on the "Project Settings" section within the SmartUI dashboard to access the settings related to your project. - -## Project Settings - -**Project Token** - -The Project Token is a unique identifier that authenticates your SmartUI project for executing tests locally with CLI and APIs. You'll find your Project Token in the Project Settings after creating a project. - -**Setting Up Project Token** - -Use this key to authenticate your SmartUI project for executing locally with CLI and APIs. - -```bash -export PROJECT_TOKEN='project#token' -``` - -```bash -set PROJECT_TOKEN='project#token' -``` - -```powershell -$env:PROJECT_TOKEN='project#token' -``` - -### Using Project Token - -The Project Token is required for: -- CLI executions (`npx smartui exec`, `npx smartui capture`) -- API uploads -- Local test execution -- CI/CD pipeline integrations - -Keep your Project Token secure and never commit it to version control. Use environment variables or secret management tools in CI/CD pipelines. - -**Basic Settings** - -- **Project Name:** Customize the name of your project to enhance its identification. - -Changing the Project Name will automatically update the `smartUI.project` capability's options for Web/App projects. - -- **Approvers:** Manage the list of authorized individuals responsible for approving changes within the project. - -Altering Approvers will impact email notifications, which will be sent to the latest approver list for recent builds. - -- **Tags:** Add tags to your project for better organization and categorization. - -**Comparison Settings** - -Comparison settings allow you to fine-tune how SmartUI compares screenshots, balancing between highlighting important changes and ignoring noise. - -**Pixel Threshold** - -Strike a balance between highlighting important changes and ignoring noise. The pixel threshold determines the sensitivity of the comparison algorithm. - -**Options:** -- **Relaxed**: Higher threshold, fewer differences detected (good for pages with minor rendering variations) -- **Recommended**: Balanced threshold (default, suitable for most use cases) -- **Strict**: Lower threshold, more differences detected (good for precise comparisons) -- **Custom**: Set a custom threshold value (0-10000) - -**Example:** -- Custom value: `5000` - Higher value means more tolerance for pixel differences - -Adjusting the Pixel Threshold in Comparison Settings will only affect new builds/screenshots captured after the update. - -**Error Highlight Color** - -Set the color to show/highlight the changes in the pixel differences on your test output. - -**Available Colors:** -- Red (default) -- Blue -- Orange -- Green -- Pink -- Gray -- Custom: Enter a hex color code (e.g., `#f687b3`) - -**Use Case**: Choose a color that provides good contrast against your page background for easier visual identification of differences. - -**Custom Mismatch Acceptance** - -Configure pixel-to-pixel acceptance percentage for auto-approval. - -**Settings:** -- **Accept**: Percentage threshold for auto-approval (0-100) -- **Reject**: Percentage threshold for auto-rejection (0-100) - -**Example:** -- Accept: `0%` - No auto-approval -- Reject: `100%` - Auto-reject all differences - -**Use Case**: Automatically approve screenshots with differences below the acceptance threshold, reducing manual review time. - -**Smart Ignore** - -Highlight content changes while smartly ignoring layout shifts and displacement differences. - -**Toggle**: Enable/disable Smart Ignore for the project - -**Benefits:** -- Reduces false positives from layout shifts -- Focuses on actual content changes -- Improves test reliability - -For detailed information, refer to [Smart Ignore Documentation](/support/docs/smartui-smartignore). - -**Bounding Boxes** - -Configure your areas to which needs to be ignored or select a specific area for comparison. - -**Use Cases:** -- Ignore specific regions (ads, dynamic content) -- Focus comparison on specific areas (main content, critical UI) -- Define regions of interest for comparison - -**Advanced Comparison Settings** - -Manage your Pixel to Pixel false positives and comparison view types. - -**Ignore Pixel Scaling Options** - -Choose options to remove the Pixel to Pixel false-positive rate in identifying the screenshot. - -**Options:** -- **Ignore Antialiasing**: Ignore differences caused by antialiasing rendering -- **Ignore Less**: Minimal pixel difference tolerance -- **Ignore Nothing**: No pixel difference tolerance (strictest) -- **Ignore Alpha**: Ignore alpha channel differences -- **Ignore Colors**: Ignore color differences, focus on structure - -**Use Case**: Reduce false positives from rendering differences, browser-specific rendering, or minor pixel variations. - -**Error Type Identifier** - -Show the differences in the output screen by identifying the pixel changes type and capture the intended view. - -**Options:** -- **Flat**: Simple difference highlighting -- **Movement**: Highlight elements that moved -- **Movement with Diff Intensity**: Show movement with intensity levels -- **Diff portion from the Input**: Show only the different portions -- **Flat with Diff Intensity**: Flat view with intensity levels - -**Use Case**: Choose the view that best helps you identify and understand the differences in your screenshots. - -**Transparency** - -Strike a balance between highlighting differences and maintaining visibility of the underlying content. - -**Options:** -- **Opaque**: Full opacity, clear difference highlighting -- **Transparent**: Lower opacity, see underlying content - -**Use Case**: Adjust transparency to see both the differences and the original content for better context. - -**Resize Image** - -Ability to scale the test screenshot according to baseline. - -**Options:** -- **Use Original Image**: Compare at original size -- **Scale to same size**: Scale screenshots to match baseline size - -**Use Case**: Handle cases where screenshots are captured at different resolutions or viewport sizes. - -Changes to Project Settings will impact builds/comparisons executed after the changes and the previous builds/comparisons remain unaffected. - -**Additional Information:** -For detailed information on these options, refer to our [Comparison Settings Documentation](/support/docs/test-settings-options/). - -**Build Settings** - -**Smart Baseline** - -Smart Baseline is a feature that simplifies and automates the process of updating the baseline for approved screenshots in your baseline build. This feature provides convenience and efficiency by ensuring your baseline remains in sync with approved changes without manual intervention. - -- **Baseline Branch:** A baseline branch build consists of screenshots captured using SmartUI. All screenshots in this build are compared against non-baseline branch builds. - -- **Non-Baseline Branches:** Non-baseline branch(s) builds are sets of screenshots captured using SmartUI. All screenshots in these builds are compared against the baseline build, matching them by screenshot names. - -When **`Smart Baseline`** is turned on, the system automatically updates the baseline for approved screenshots in the baseline build, streamlining the testing process and saving time. - -**Warning** -Please note that this feature is not supported for projects integrated with `Git` as `Git`-based projects follow a [Git branching strategy](/support/docs/smartui-github-app-integration/). - -**Git Settings** - -Configure Git-related settings for your project, including the baseline branch name, auto-approval branches, GitHub repository linkage, and run status checks. - -**Additional Information:** -For detailed information on these options, refer to our [Git Baseline Branching Documentation](/support/docs/smartui-github-app-integration/). - -**Overwrite Screenshot** - -- This feature allows to update and overwrite screenshots with the same name, ensuring that the latest version is updated for that screenshot. - -- This capability provides users with more control over the management of screenshots with identical names, enabling them to maintain accurate visual records and efficiently track changes in their application's user interface over time. - -**Notifications** - -Configure how you receive alerts and updates about your SmartUI project. - -**Email Notifications** - -Receive email notifications whenever there are changes found in your build or the project baseline changes. - -**Settings:** -- **Build Changes**: Get notified when visual differences are found in builds -- **Baseline Changes**: Get notified when project baseline is updated -- **Approver Changes**: Get notified when project approvers list is modified - -**Use Case**: Stay informed about visual changes without constantly checking the dashboard. - -**Slack Alerts** - -Receive Slack alerts whenever there are changes found in your build. - -**Setup:** -1. Connect your Slack workspace in Project Settings -2. Configure which events trigger Slack notifications -3. Choose the Slack channel for notifications - -**Use Case**: Integrate visual testing alerts into your team's communication workflow. - -We are continually adding more notification channels for SmartUI. - -**TestMu AI Badge** - -Show the world that you are using TestMu AI for your visual regression testing. - -**Badge Code:** -```markdown -[![This project is using Lambda test for visual regression testing.](https://smartui.lambdatest.com/static/media/LTBadge.64a05e73.svg)](https://lambdatest.com) -``` - -**Use Case**: Add to your README or documentation to showcase your testing setup. - -**Delete Project** - -The "Delete Project" option allows administrators or project creators to permanently delete a project. Once a project is deleted, it cannot be retrieved. - -Only the **Admin** or **Creator** of the project can delete a project. Once deleted, a project cannot be retrieved. - -## Additional Resources - -- [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) - Solutions for project configuration issues -- [Baseline Management](/support/docs/smartui-baseline-management) - Learn how to manage baselines effectively -- [Handling Dynamic Data](/support/docs/smartui-handle-dynamic-data) - Configure ignoreDOM/selectDOM for dynamic content -- [Smart Ignore Feature](/support/docs/smartui-smartignore) - Automatically ignore layout shifts -- [Configuration Options](/support/docs/smartui-sdk-config-options) - SDK configuration file options -- [Running Your First Project](/support/docs/smartui-running-your-first-project) - Get started with SmartUI -- [Test Settings Options](/support/docs/test-settings-options/) - Advanced comparison settings diff --git a/static/docs/smartui-puppeteer-sdk.md b/static/docs/smartui-puppeteer-sdk.md deleted file mode 100644 index 878abf16b..000000000 --- a/static/docs/smartui-puppeteer-sdk.md +++ /dev/null @@ -1,546 +0,0 @@ -# Integrate SmartUI SDK with Puppeteer Tests - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Welcome to the world of simplified visual testing with the SmartUI SDK. - -Integrating seamlessly into your existing Puppeteer testing suite, SmartUI SDK revolutionizes the way you approach visual regression testing. Our robust solution empowers you to effortlessly capture, compare, and analyze screenshots across a multitude of browsers and resolutions, ensuring comprehensive coverage and accuracy in your visual testing endeavors. - -## Prerequisites - -- Basic understanding of Command Line Interface and Puppeteer is required. -- Login to [TestMu AI SmartUI](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) with your credentials. - -The following steps will guide you in running your first Visual Regression test on TestMu AI platform using SmartUI Puppeteer SDK integration. - -## Create a SmartUI Project - -The first step is to create a project with the application in which we will combine all your builds run on the project. To create a SmartUI Project, follow these steps: - -1. Go to [Projects page](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) -2. Click on the `new project` button -3. Select the platform as CLI for executing your `SDK` tests. -4. Add name of the project, approvers for the changes found, tags for any filter or easy navigation. -5. Click on the **Submit**. - -## Steps to run your first test - -Once you have created a SmartUI Project, you can generate screenshots by running automation scripts. Follow the below steps to successfully generate screenshots - -### **Step 1:** Create/Update your test - -You can clone the sample repository to run `LambdaTest` automation tests with `SmartUI` and use the `puppeteerCloud.js` file present in the `sdk` folder. - -```bash -git clone https://github.com/LambdaTest/smartui-puppeteer-sample -cd smartui-puppeteer-sample/sdk -``` -### **Step 2**: Install the Dependencies - -Install required NPM modules for `TestMu AI SmartUI Puppeteer SDK` in your **Frontend** project. - -```bash -npm install @lambdatest/smartui-cli @lambdatest/puppeteer-driver puppeteer -``` - -If you face any problems executing tests with SmartUI-CLI `versions >= v4.x.x`, upgrade your Node.js version to `v20.3` or above. - -### **Step 3:** Configure your Project Token - -Setup your project token shown in the **SmartUI** app after creating your project. - -```bash -export PROJECT_TOKEN=123456#1234abcd-****-****-****-************" -``` - -```bash -set PROJECT_TOKEN=123456#1234abcd-****-****-****-************" -``` - -```powershell -$env:PROJECT_TOKEN=123456#1234abcd-****-****-****-************" -``` - -### **Step 4:** Create and Configure SmartUI Config - -You can now configure your project configurations on using various available options to run your tests with the SmartUI integration. To generate the configuration file, please execute the following command: - -```bash -npx smartui config:create .smartui.json -``` - -Once, the configuration file will be created, you will be seeing the default configuration pre-filled in the configuration file: - -```json title="/smartui-sdk-project/.smartui.json" -{ -"web": { -"browsers": [ -"chrome", -"firefox", -"safari", -"edge" -], -"viewports": [ -[ -1920 -], -[ -1366 -], -[ -1028 -] -] // Full Page screenshots are captured by default for web viewports -}, -"mobile": { -"devices": [ -"iPhone 14", //iPhone 14 viewport -"Galaxy S24" //Galaxy S24 viewport -], -"fullPage": true, //Full Page is true by default for mobile viewports -"orientation": "portrait" //Change to "landscape" for landscape snapshot -}, -"waitForTimeout": 1000, //Optional (Should only be used in case lazy-loading/async components are present) -"waitForPageRender": 50000, //Optional (Should only be used in case of websites which take more than 30s to load) -"enableJavaScript": false, //Enable javascript for all the screenshots of the project -"allowedHostnames": [] //Additional hostnames to capture assets from -} -``` -**Advanced options in SmartUI configuration** -- For capturing fullpage or viewport screenshots, please refer to this [documentation](/docs/smartui-sdk-config-options/#12-viewports) -- For the list of available mobile viewports, please refer to this [documentation](/docs/smartui-sdk-config-options/#list-of-supported-device-viewports) -- For more information about SmartUI config global options, please refer to this [documentation](/docs/smartui-sdk-config-options/#3-global-options-optional). - -### **Step 5:** Adding SmartUI function to take screenshot - -- You can incorporate SmartUI into your custom `Puppeteer` automation test (any platform) script by adding the `smartuiSnapshot` function in the required segment of Puppeteer script of which we would like to take the screenshot, as shown below: - -```js -const puppeteer = require("puppeteer"); -const { smartuiSnapshot } = require('@lambdatest/puppeteer-driver'); // Assuming you still want to use smartuiSnapshot locally - -(async () => { -// Launch a browser instance locally -const browser = await puppeteer.launch({ -headless: false, // Set to false to see the UI -args: ['--start-maximized'], // Start browser maximized, remove if not needed -}); -const page = await browser.newPage(); -await page.setViewport({ width: 1280, height: 720 }); // Set viewport size as needed - -// Navigate to the desired URL -await page.goto('https://www.lambdatest.com'); - -// Take a screenshot with SmartUI. Replace "LT-Home" with a relevant name for your use case -await smartuiSnapshot(page, "LT-Home"); - -// Close the browser -await browser.close(); -})(); -``` - -### **Step 6:** Execute the Tests on SmartUI Cloud - -Execute `visual regression tests` on SmartUI using the following commands - -```bash -npx smartui exec node puppeteerCloud.js --config .smartui.json -``` - -You may use the `npx smartui --help` command in case you are facing issues during the execution of SmartUI commands in the CLI. - -## View SmartUI Results - -You have successfully integrated SmartUI SDK with your Puppeteer tests. Visit your SmartUI project to view builds and compare snapshots between different test runs. - -You can see the SmartUI dashboard to view the results. This will help you identify the Mismatches from the existing `Baseline` build and do the required visual testing. - -## Arguments supported in the `smartUISnapshot` function - -The following are the different options which are currently supported: - -| Key | Description | -| ------------------------- | ------------------------------------------------------------------------------------------------------------------------- | -| `driver` (instance) | The instance of the web driver used in your tests. | -| `"Screenshot Name"` (string) | Specify a name for the screenshot in your tests to match the same screenshot with the name from your baseline. | -| `options` (object) | Specify one or a combination of selectors in the `ignoreDOM` or `selectDOM` objects. These selectors can be based on `HTML DOM IDs, CSS classes, CSS selectors, or XPaths` used by your webpage. They define elements that should be excluded from or included in the visual comparison.| - -## Handling Dynamic Data in SmartUI SDK **** - -When conducting visual tests, you may encounter scenarios where certain elements within your application change between test runs. These changes might introduce inconsistencies in your test results.You can ignore / select specific element(s) to be removed from the comparison by parsing the options in the `smartuiSnapshot` function in the following way - -```js title="This is a sample for your configuration for Puppeteer to ignore by" ID" -let options = { -ignoreDOM: { -id: ["ID-1", "ID-2"], -} -} -await page.goto('Required URL'); -await smartuiSnapshot.smartuiSnapshot(page, "Screenshot Name", options); -``` - -```js title="This is a sample for your configuration for Puppeteer to ignore by" Class" -let options = { -ignoreDOM: { -class: ["Class-1", "Class-2"], -} -} -await page.goto('Required URL'); -await smartuiSnapshot.smartuiSnapshot(page, "Screenshot Name", options); -``` - -```js title="This is a sample for your configuration for Puppeteer to ignore by" XPath" -let options = { -ignoreDOM: { -xpath: ["Xpath-1", "Xpath-2"], -} -} -await page.goto('Required URL'); -await smartuiSnapshot.smartuiSnapshot(page, "Screenshot Name", options); -``` - -```js title="This is a sample for your configuration for Puppeteer to ignore by CSS" Selector" -let options = { -ignoreDOM: { -cssSelector: ["CSS-Selector-1", "CSS-Selector-2"], -} -} -await page.goto('Required URL'); -await smartuiSnapshot.smartuiSnapshot(page, "Screenshot Name", options); -``` - -```js title="This is a sample for your configuration for Puppeteer to select by" ID." -let options = { -selectDOM: { -id: ["ID-1", "ID-2"], -} -} -await page.goto('Required URL'); -await smartuiSnapshot.smartuiSnapshot(page, "Screenshot Name", options); -``` - -```js title="This is a sample for your configuration for Puppeteer to select by" Class" -let options = { -selectDOM: { -class: ["Class-1", "Class-2"], -} -} -await page.goto('Required URL'); -await smartuiSnapshot.smartuiSnapshot(page, "Screenshot Name", options); -``` - -```js title="This is a sample for your configuration for Puppeteer to select by" XPath" -let options = { -selectDOM: { -xpath: ["Xpath-1", "Xpath-2"], -} -} -await page.goto('Required URL'); -await smartuiSnapshot.smartuiSnapshot(page, "Screenshot Name", options); -``` - -```js title="This is a sample for your webhook configuration for Puppeteer to select by CSS" Selector" -let options = { -selectDOM: { -cssSelector: ["CSS-Selector-1", "CSS-Selector-2"], -} -} -await page.goto('Required URL'); -await smartuiSnapshot.smartuiSnapshot(page, "Screenshot Name", options); -``` - -## For capturing the screenshot of a specific element - -You can capture screenshots of targeted elements by leveraging various locator mechanisms such as XPath, CSS ID, class, and selectors. This precision-driven approach ensures accurate and specific visual regression testing for your web application's components. - -```js title="This is a sample for your configuration for Puppeteer to capture an element by" ID." -let options = { -element: { -id: 'Required ID', -} -}; -await page.goto('Required URL'); -await smartuiSnapshot.smartuiSnapshot(page, "Screenshot Name", options); -``` - -```js title="This is a sample for your configuration for Puppeteer to capture an element by" Class" -let options = { -element: { -class: 'Required Class', -} -}; -await page.goto('Required URL'); -await smartuiSnapshot.smartuiSnapshot(page, "Screenshot Name", options); -``` - -```js title="This is a sample for your configuration for Puppeteer to capture an element by" XPath" -let options = { -element: { -xpath: 'Required Xpath', -} -}; -await page.goto('Required URL'); -await smartuiSnapshot.smartuiSnapshot(page, "Screenshot Name", options); -``` - -```js title="This is a sample for your webhook configuration for Puppeteer to capture an element by CSS" Selector" -let options = { -element: { -cssSelector: 'Required CSS Selector', -} -}; -await page.goto('Required URL'); -await smartuiSnapshot.smartuiSnapshot(page, "Screenshot Name", options); -``` - -## For capturing interactive lazy loading elements - -If you encounter difficulties loading interactive elements that appear on scroll in full-page screenshots, consider functionally incorporating a full-page scroll into your script before capturing the screenshot. This approach ensures the elements load first, facilitating the screenshot processing. - -```js Example for scrolling to bottom for lazy elements -const puppeteer = require('puppeteer'); -const { smartuiSnapshot } = require('@lambdatest/puppeteer-driver'); - -(async () => { -const browser = await puppeteer.launch({ headless: false }); // Launches a browser -const page = await browser.newPage(); // Opens a new page - -try { -await page.goto('Required URL'); // Navigate to the required URL - -// Function to scroll to the bottom of the page -async function quickScrollToBottom(lastPageWait) { -await page.evaluate(async (lastPageWait) => { -const scrollToBottom = async (lastPageWait) => { -const getScrollHeight = () => document.body.scrollHeight; -let lastHeight = await getScrollHeight(); -let currentHeight = 0; - -while (currentHeight < lastHeight) { -window.scrollTo(0, lastHeight); -await new Promise(resolve => setTimeout(resolve, 1000)); // Wait for the page to load more content -currentHeight = lastHeight; -lastHeight = await getScrollHeight(); -} - -if (lastPageWait) { -await new Promise(resolve => setTimeout(resolve, lastPageWait)); // Additional wait at the bottom -} - -// Scroll back to the top after reaching the bottom -window.scrollTo(0, 0); -await new Promise(resolve => setTimeout(resolve, 1000)); // Wait for scroll to top -}; -await scrollToBottom(lastPageWait); -}, lastPageWait); -} - -await quickScrollToBottom(100); // Adjust the wait time as needed - -// Taking a screenshot with SmartUI -await smartuiSnapshot(page, "Screenshot Name"); - -} finally { -await browser.close(); // Close the browser -} -})(); -``` - -## Best Practices - -**Screenshot Naming** - -- Use descriptive, consistent names for screenshots -- Include page/component name in screenshot names -- Avoid special characters that might cause issues -- Use consistent naming conventions across your test suite - -**Example:** -```javascript -await smartuiSnapshot(page, HomePage-Header"); -await smartuiSnapshot(page, "ProductPage-MainContent"); -``` - -**Wait for Page Load** - -- Always wait for pages to fully load before taking screenshots -- Use Puppeteer's built-in wait methods for dynamic content -- Consider using `waitForTimeout` in configuration for lazy-loaded content - -**Example:** -```javascript -await page.goto('https://example.com'); -await page.waitForSelector('#main-content', { visible: true }); -await page.waitForLoadState('networkidle'); -await smartuiSnapshot(page, Page Loaded"); -``` - -**Handle Dynamic Content** - -- Use `ignoreDOM` for elements that change between runs -- Use `selectDOM` when you only need to compare specific areas -- Document why elements are ignored for future reference - -**Configuration Management** - -- Keep `.smartui.json` in version control -- Use environment variables for sensitive data -- Document custom configuration choices - -**Test Organization** - -- Group related screenshots in the same build -- Use meaningful build names -- Run tests in consistent environments - -## Troubleshooting - -**Issue: Screenshots Not Appearing in Dashboard** - -**Symptoms**: Tests run successfully but no screenshots appear in SmartUI dashboard - -**Possible Causes**: -- Project token not set or incorrect -- Project name mismatch -- Network connectivity issues -- CLI not installed or outdated - -**Solutions**: -1. Verify `PROJECT_TOKEN` is set correctly: -```bash -echo $PROJECT_TOKEN -``` - -2. Check project name matches exactly (case-sensitive) - -3. Verify SmartUI CLI is installed: -```bash -npx smartui --version -``` - -4. Check network connectivity to TestMu AI servers - -5. Review test execution logs for error messages - -**Issue: Project Not Found" Error** - -**Symptoms**: Error message indicating project cannot be found - -**Possible Causes**: -- Incorrect project token -- Project deleted or renamed -- Token from wrong project - -**Solutions**: -1. Verify project exists in SmartUI dashboard -2. Copy project token directly from Project Settings -3. Ensure token includes the project ID prefix (e.g., `123456#...`) -4. Check for extra spaces or quotes in token - -**Issue: Screenshots Show Blank or Incorrect Content** - -**Symptoms**: Screenshots captured but show blank pages or incorrect content - -**Possible Causes**: -- Page not fully loaded -- JavaScript not executed -- Viewport size issues -- Timing issues - -**Solutions**: -1. Add explicit waits before screenshots: -```javascript -await page.goto('https://example.com'); -await page.waitForSelector('#content', { visible: true }); -await page.waitForLoadState('networkidle'); -``` - -2. Enable JavaScript in configuration: -```json -{ -enableJavaScript": true -} -``` - -3. Increase `waitForTimeout` in configuration - -4. Verify viewport size matches expected dimensions - -**Issue: Build Execution Fails** - -**Symptoms**: `npx smartui exec` command fails - -**Possible Causes**: -- Missing or incorrect configuration file -- Invalid JSON in configuration -- Port conflicts -- Permission issues - -**Solutions**: -1. Verify `.smartui.json` exists and is valid JSON -2. Check configuration file syntax -3. Try different port if default is in use: -```bash -npx smartui exec -P 5000 -- -``` -4. Check file permissions for configuration and project files - -**Issue: npm Dependencies Not Resolving** - -**Symptoms**: npm cannot find `@lambdatest/puppeteer-driver` or `@lambdatest/smartui-cli` - -**Possible Causes**: -- Incorrect package version -- npm registry access issues -- Network connectivity problems - -**Solutions**: -1. Check latest versions on npm: -```bash -npm view @lambdatest/puppeteer-driver version -npm view @lambdatest/smartui-cli version -``` - -2. Clear npm cache: -```bash -npm cache clean --force -``` - -3. Verify internet connectivity for npm registry access - -4. Check package.json for version conflicts - -**Issue: Screenshot Names Not Matching Baseline** - -**Symptoms**: Screenshots appear as New" instead of comparing with baseline - -**Possible Causes**: -- Screenshot name changed -- Baseline doesn't exist -- Name contains special characters - -**Solutions**: -1. Ensure screenshot names are consistent across test runs -2. Verify baseline exists in project -3. Avoid special characters in screenshot names -4. Check for case sensitivity issues - -**Getting Help** - -If you encounter issues not covered here: - -- Review the [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) for detailed solutions -- Check [SmartUI Configuration Options](/support/docs/smartui-sdk-config-options) documentation -- See [Handling Dynamic Data](/support/docs/smartui-handle-dynamic-data) for dynamic content issues -- Visit [TestMu AI Support](https://www.testmuai.com/support) for additional resources -- Contact support at support@testmuai.com or use [24/7 Chat Support](https://www.testmuai.com/support) - -## Additional Resources - -- [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) -- [SmartUI Configuration Options](/support/docs/smartui-sdk-config-options) -- [Handling Dynamic Data](/support/docs/smartui-handle-dynamic-data) -- [Handling Lazy Loading](/support/docs/smartui-handle-lazy-loading) -- [Baseline Management](/support/docs/smartui-baseline-management) -- [Running Your First Project](/support/docs/smartui-running-your-first-project) -- [SmartUI API Documentation](https://www.testmuai.com/support/api-doc/) diff --git a/static/docs/smartui-root-cause-analysis.md b/static/docs/smartui-root-cause-analysis.md deleted file mode 100644 index 134084f93..000000000 --- a/static/docs/smartui-root-cause-analysis.md +++ /dev/null @@ -1,68 +0,0 @@ -# Smart Root Cause Analysis (RCA) - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Smart Root Cause Analysis (RCA) in SmartUI helps you instantly identify the underlying reasons for visual mismatches. By analyzing the DOM differences between the baseline and the comparison screenshot, RCA pinpoints exactly what changed: whether it's a style update, content change, or layout shift. - -## Why use Smart RCA? -- **Speed**: Instantly find why a pixel-level mismatch occurred. -- **Precision**: Distinguish between actual bugs and intended changes. -- **Actionable Insights**: Get direct info on CSS property changes or DOM structure modifications. visually failing tests with speed and confidence. - -## What Problems Does Smart RCA Solve? - -Visual testing often tells you that something is wrong but not what caused it. Smart RCA eliminates the guesswork and manual debugging by providing immediate, actionable insights. - -| Problem | Smart RCA Solution | -|---|---| -| The UI looks broken, but you don’t know why. | Click to inspect the exact HTML, CSS, and Text changes. | -| The visual diff isn't providing enough detail. | RCA shows an in-depth DOM comparison. | -| The team wastes time on manual debugging. | RCA provides an instant root cause, saving time. | -| The layout shifted because of a small style change. | RCA pinpoints the exact CSS properties that changed. | -| Invisible elements are affecting the layout. | RCA highlights changes to all elements, even invisible ones. | - -## How to Use Root Cause Analysis - -### Prerequisites - -Before you begin, please ensure the build was generated using a **TestMu AI CLI (exec)** command. - -### Step 1: Activate RCA Mode - -1. Open a visual build from your [SmartUI dashboard](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/). -2. In the comparison toolbar, click the **RCA** icon. - -When RCA is activated, DOM diff boxes are automatically enabled. There's nothing manual to configure, you're ready to investigate immediately. - -### Step 2: Investigate DOM Differences Visually - -As soon as RCA mode is on, you can interact with the highlighted differences to see the underlying code changes. The selection is visually precise, with overlays that follow the true element boundaries, not just approximate pixel areas. - -| User Interaction | Visual Feedback & Result | -|---|---| -| **Default View** | A dark blue highlight box appears by default, pinpointing the exact DOM element where the primary issue is detected. | -| **Hover** on a diff region | A yellow bounding box appears, indicating the element is ready for selection. | -| **Click** a diff region | A red box locks onto the selected DOM region, and the RCA panel opens on the right. | -| **Hover** while selected | A blue highlight box appears over the element, helping to transition focus between the screenshot and the RCA panel. | - -### Step 3: Understand Detailed Changes in the RCA Panel - -Once an element is selected, the RCA panel opens on the right sidebar. It displays structured diff breakdowns across several categories: - -| Category | What You'll See | -|---|---| -| **DOM Path** | The full XPath for the element in both the baseline and captured screenshot. | -| **Computed Styles** | A list of all style changes. *Removed properties are highlighted in red*, and *added properties are highlighted in green*. | -| **Bounding Box** | The exact `left`/`top` position and `width`/`height` of the element, before and after the change. | -| **Attribute Changes** | A clear view of any added, removed, or modified HTML attributes. | -| **Tag Transforms** | Structural changes, such as a `` becoming an ``. | -| **Text Modifications** | A word-by-word diff of any edited content. | -| **Layout Shifts** | Details on how the layout was affected, using CSS diffs for properties like `display`, `transform`, `margin`, etc. | - -## Key Advantages of Smart RCA - -* **Accelerated Debugging**: By pinpointing the exact line of code or style property that caused a failure, RCA drastically cuts down the time from bug detection to resolution. -* **Reduced Friction**: Eliminate the need to constantly switch between your testing tool, browser dev tools, and application code. All the information you need is in one unified view. -* **Improved Collaboration**: Share precise, actionable bug reports. Instead of describing a visual bug, you can link your team directly to the RCA view, showing them the exact DOM and CSS changes responsible. -* **Focus on Meaningful Changes**: The system intelligently filters out noise, ensuring you and your team are alerted only to significant changes that impact user experience. -* **No False Triggers**: RCA works only where DOM diff data is available, ensuring a smooth and guided experience without misleading information. diff --git a/static/docs/smartui-running-your-first-project.md b/static/docs/smartui-running-your-first-project.md deleted file mode 100644 index 671236ae4..000000000 --- a/static/docs/smartui-running-your-first-project.md +++ /dev/null @@ -1,223 +0,0 @@ -# Running Your First Project on SmartUI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This guide will walk you through the process of running your first visual regression test using SmartUI, a powerful feature of TestMu AI that ensures your web applications look flawless across all browsers and devices. -## Prerequisites - -- Basic understanding of Command Line Interface and Selenium is required. -- Login to [TestMu AI SmartUI](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) with your credentials. - -The following steps will guide you in running your first Visual Regression test on TestMu AI platform using SmartUI Selenium SDK integration. - -## Step 1: Create a SmartUI Project -The first step is to create a project with the application in which we will combine all your builds run on the project. To create a SmartUI Project, follow these steps: - -- Go to [Projects page](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) and click on the **New Project** button. -- Select the platform as CLI for executing your **SDK** tests. -- Add name of the project, approver's name, and tags for any filter or easy navigation. -- Click on the **Continue** button. - -Now select your desired framework and click on the **Configure** button. For the demo purpose we will select the Java framework. - -## Step 2: Configure your Test Suite dependencies - -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** -Download or Clone the code sample code repository from the TestMu AI GitHub repository to run the project on the SmartUI. - - View on GitHub - -- Add the following dependency in your `pom.xml` file - -```xml title="pom.xml" - -io.github.lambdatest -lambdatest-java-sdk -1.0.2 - -``` - -You can check the latest version of [lambdatest-java-sdk]( https://mvnrepository.com/artifact/io.github.lambdatest/lambdatest-java-sdk) and update the latest version accordingly. - -- Install your CLI and required modules for running SmartUI SDK and compile your defined dependencies in the `pom.xml` file: - -```zsh -npm install -g @lambdatest/smartui-cli -mvn clean compile -``` - -If you face any problems executing tests with SmartUI-CLI `versions >= v4.x.x`, upgrade your Node.js version to `v20.3` or above. - -## Step 3: Setup your Project Token and Environment Variables - -Setup your project token and credentials shown in the **SmartUI** app after creating your project. - -### Setting Project Token - -```bash -export PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -``` - -```bash -set PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -``` - -```powershell -$env:PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -``` - -### Setting TestMu AI Credentials (for Hooks) - -```bash -export LT_USERNAME="${YOUR_LAMBDATEST_USERNAME}" -export LT_ACCESS_KEY="${YOUR_LAMBDATEST_ACCESS_KEY}" -``` - -```bash -set LT_USERNAME="${YOUR_LAMBDATEST_USERNAME}" -set LT_ACCESS_KEY="${YOUR_LAMBDATEST_ACCESS_KEY}" -``` - -```powershell -$env:LT_USERNAME="${YOUR_LAMBDATEST_USERNAME}" -$env:LT_ACCESS_KEY="${YOUR_LAMBDATEST_ACCESS_KEY}" -``` - -- For CLI projects, use `PROJECT_TOKEN` (no username/access key needed) -- For Hooks (Selenium, Playwright, etc.), use `LT_USERNAME` and `LT_ACCESS_KEY` - -## Step 4: Create and Configure SmartUI Config -You can now configure your project configurations on using various available options to run your project with the SmartUI integration. To generate the configuration file, please execute the following command: - -```bash -npx smartui config:create .smartui.json -``` -Once, the configuration file will be created, you will be seeing the default configuration pre-filled in the configuration file: - -```javascript title="/smartui-sdk-project/.smartui.json" -{ -"web": { -"browsers": [ -"chrome", -"firefox", -"safari", -"edge" -], -"viewports": [ -[ -1920 -], -[ -1366 -], -[ -1028 -] -] // Full Page screenshots are captured by default for web viewports -}, -"mobile": { -"devices": [ -"iPhone 14", //iPhone 14 viewport -"Galaxy S24" //Galaxy S24 viewport -], -"fullPage": true, //Full Page is true by default for mobile viewports -"orientation": "portrait" //Change to "landscape" for landscape snapshot -}, -"waitForTimeout": 1000, //Optional (Should only be used in case lazy-loading/async components are present) -"waitForPageRender": 50000, //Optional (Should only be used in case of websites which take more than 30s to load) -"enableJavaScript": false, //Enable javascript for all the screenshots of the project -"allowedHostnames": [] //Additional hostnames to capture assets from -} -``` -**Advanced options in SmartUI configuration** -- For capturing fullpage or viewport screenshots, please refer to this [documentation](/docs/smartui-sdk-config-options/#12-viewports) -- For the list of available mobile viewports, please refer to this [documentation](/docs/smartui-sdk-config-options/#list-of-supported-device-viewports) -- For more information about SmartUI config global options, please refer to this [documentation](/docs/smartui-sdk-config-options/#3-global-options-optional). - -## Step 5: Adding SmartUI function to take screenshot -You can incorporate SmartUI into your custom `Selenium` automation test (any platform) script by adding the `smartuiSnapshot` function in the required segment of selenium script of which we would like to take the screenshot, as shown below: - -```java -//Importing the lambdatest-java SDK -//Rest of your code here - -@Test -public void basicTest() throws Exception { -String spanText; -System.out.println("Loading URL"); - -driver.get(""); -// ... test code snippet -// highlight-next-line -SmartUISnapshot.smartuiSnapshot(driver, ""); -} -``` - -## Step 6: Execute the Tests on SmartUI Cloud -Execute `visual regression tests` on SmartUI using the following commands - -```bash -npx smartui --config .smartui.json exec -- mvn test -D suite="sdk-cloud.xml" -``` - -You may use the `npx smartui --help` command in case you are facing issues during the execution of SmartUI commands in the CLI. - -You can see the SmartUI dashboard to view the results. This will help you identify the Mismatches from the existing `Baseline` build and do the required visual testing. - -## Next Steps - -Now that you've run your first project, explore these resources to enhance your visual testing: - -- **Configure Your Project**: Learn about [Project Settings](/support/docs/smartui-project-settings) to customize comparison parameters -- **Handle Dynamic Content**: See [Handling Dynamic Data](/support/docs/smartui-handle-dynamic-data) for pages with changing content -- **Manage Baselines**: Understand [Baseline Management](/support/docs/smartui-baseline-management) for effective test comparisons -- **Troubleshooting**: If you encounter issues, check the [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) - -## Using the SmartUI Agent Skill with TestMu AI - -The [smartui-skill](https://github.com/LambdaTest/agent-skills/tree/main/smartui-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The smartui-skill package includes: - -``` -smartui-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing SmartUI Agent Skill - -Install a SmartUI Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/smartui-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/smartui-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only smartui-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). - -## Additional Resources - -- [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) - Solutions for common issues -- [Project Settings](/support/docs/smartui-project-settings) - Configure your project settings -- [Configuration Options](/support/docs/smartui-sdk-config-options) - Learn about all available configuration options -- [Handling Dynamic Data](/support/docs/smartui-handle-dynamic-data) - Handle pages with dynamic content -- [Baseline Management](/support/docs/smartui-baseline-management) - Manage your test baselines -- [SDK Documentation](/support/docs/smartui-selenium-java-sdk) - Framework-specific integration guides diff --git a/static/docs/smartui-sdk-capabilities.md b/static/docs/smartui-sdk-capabilities.md deleted file mode 100644 index 7d0e9584d..000000000 --- a/static/docs/smartui-sdk-capabilities.md +++ /dev/null @@ -1,80 +0,0 @@ -# SmartUI SDK Capabilities - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -SmartUI SDK provides several capabilities that allow you to configure your visual regression tests. These capabilities can be added to your test configuration to control various aspects of the testing process. - -**Important** -These capabilities only work when running tests on the TestMu AI Automation Grid. For local test executions, please refer to the standard authentication methods using project token or project name in the [Running your first project](/support/docs/smartui-running-your-first-project/) documentation. - -## Prerequisites - -Before using SmartUI SDK capabilities, ensure you have the following minimum versions installed: - -- SmartUI CLI version >= 4.1.0 -- For Java SDK: SmartUI SDK version >= 1.0.7 -- For JavaScript: Selenium WebDriver version >= 1.0.7 - -## Available Capabilities - -### For JavaScript - -```javascript -let capabilities = { -platform: "catalina", -browserName: "chrome", -version: "latest", -"LT:Options": { -username: USERNAME, -accessKey: KEY, -project: "ElementSS-MobileConfig", -w3c: true, -name: "Test Name", // name of the test -build: "Automation Build Name", // name of the build -// highlight-next-line -"smartUI.project": "YOUR_PROJECT_NAME", -// highlight-next-line -"smartUI.build": "YOUR_BUILD_NAME", -// highlight-next-line -"smartUI.baseline": true, -}, -}; -``` - -### For Java - -```java -DesiredCapabilities capabilities = new DesiredCapabilities(); -capabilities.setCapability("platform", "Windows 10"); -capabilities.setCapability("browserName", "chrome"); -capabilities.setCapability("version", "latest"); - -HashMap ltOptions = new HashMap(); -ltOptions.put("username", "YOUR_USERNAME"); -ltOptions.put("accessKey", "YOUR_ACCESS_KEY"); -ltOptions.put("w3c", true); -ltOptions.put("name", "Test Name"); -ltOptions.put("build", "Automation Build Name"); -// highlight-next-line -ltOptions.put("smartUI.project", "YOUR_PROJECT_NAME"); -// highlight-next-line -ltOptions.put("smartUI.build", "SmartUI_BUILD_NAME"); -// highlight-next-line -ltOptions.put("smartUI.baseline", true); - -capabilities.setCapability("LT:Options", ltOptions); -``` - -## Capability Descriptions - -### smartUI.project -This capability specifies the project Name where your screenshots will be stored and compared. You can add an existing project name or create a new project name. - -### smartUI.build -This capability allows you to group your screenshots into specific builds. This is particularly useful when you want to organize your screenshots and compare them against specific baselines. The build name should be unique and descriptive to help identify the purpose of that particular test run. - -### smartUI.baseline -This boolean capability determines whether the current build should be set as the baseline for future comparisons. When set to `true`, the build will be automatically approved and set as the baseline build. This capability works in conjunction with the [Git branching strategy](/support/docs/smartui-cli-git-branching-strategy/), meaning it will also update the baseline branch accordingly. - -**Note** -If you need capabilities' support for SDKs other than Selenium Java and JavaScript, please contact support@testmuai.com. Our team will be happy to assist you with integrating SmartUI capabilities into your preferred testing framework. diff --git a/static/docs/smartui-sdk-config-options.md b/static/docs/smartui-sdk-config-options.md deleted file mode 100644 index c22780b45..000000000 --- a/static/docs/smartui-sdk-config-options.md +++ /dev/null @@ -1,355 +0,0 @@ -# SmartUI SDK Advanced Configuration Options - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Welcome to the world of simplified visual testing with the SmartUI SDK. - -This guide is designed to provide you with comprehensive information about the various configuration options available within the SmartUI SDK. Whether you're a new user seeking to customize your SmartUI integration or an experienced developer looking to optimize your testing workflows, this documentation will serve as your go-to resource for understanding and utilizing the configuration options effectively. - -To generate the SmartUI SDK configuration file, please execute the following command: - -```bash -npx smartui config:create .smartui.json -``` - -For the latest features including advanced configurations, ensure you are using the latest version of SmartUI CLI. Install or update using: - -```bash -npm install -g @lambdatest/smartui-cli@latest -``` - -## Configuration File Structure - -The SmartUI configuration file follows a JSON structure. Below is a sample configuration file with detailed explanations of each option: - -```json -{ -"web": { -"browsers": [ -"chrome", -"firefox", -"safari", -"edge" -], -"viewports": [ -[1920], -[1366], -[1028] -] -}, -"mobile": { -"devices": [ -"iPhone 14", -"Galaxy S24" -], -"fullPage": true, -"orientation": "portrait" -}, -"waitForTimeout": 1000, -"waitForPageRender": 50000, -"enableJavaScript": false, -"allowedHostnames": ["cdn.xyz.com"] -} -``` - -## Configuration Options - -**browsers** - -An array of browsers to capture screenshots from. Supported browsers include `chrome`, `firefox`, `safari`, and `edge`. - -**viewports** - -An array of arrays representing different screen resolutions for web browsers. Each inner array contains viewport sizes. Each web viewport is automatically rendered for each of the browser mentioned in the config. - -**For capturing fullpage screenshots** - -To capture a screenshot of the entire page, you only need to define the viewport width in your configuration settings. Specify the desired width parameters as demonstrated in the following example to ensure a fullpage capture. - -```json title="Full Page Capture" -"viewports": [ -[ -1920 -], -[ -1366 -], -[ -360 -] -], -``` - -**For capturing viewport screenshots** - -To capture a screenshot of the content currently visible in your viewport, rather than the entire page, it's important to define the viewport's width and height in your configuration settings. Specify the desired width and height parameters as demonstrated in the following example to ensure that the screenshot encompasses only the viewport area. - -```json title="Viewport Capture" -"viewports": [ -[ -1920, -1080 -], -[ -1366, -768 -], -[ -360, -640 -] -], -``` - -**customViewports** - -The `customViewports` property provides a way to define precisely which browser + viewport combinations should be rendered. This bypasses the default **cartesian product behavior** (where every viewport is automatically rendered on every specified browser). - -This is particularly useful if you need to enforce a specific screen size only on Safari, but another screen size across Chrome and Firefox. - -**Mutually Exclusive Configurations** -The `customViewports` option and the standard `browsers` / `viewports` arrays are **mutually exclusive**. -- To use the standard cartesian matrix, configure `browsers` + `viewports`. -- To specify distinct pairs, use `customViewports` **only**, and ensure `browsers` and `viewports` are removed from your `web` object. - -**How to configure `customViewports`:** - -Below is an example of providing targeted sizes depending on the browser context. When using this configuration, SmartUI will generate screenshots only for the explicitly declared matches, allowing for highly optimized test metrics. - -```json title="Sample web object with customViewports" -"customViewports": [ -{ "browser": "chrome", "viewport": { "width": 1200, "height": 800 } }, -{ "browser": "chrome", "viewport": { "width": 375, "height": 667 } }, -{ "browser": "firefox", "viewport": { "width": 1200, "height": 800 } }, -{ "browser": "safari", "viewport": { "width": 1200 } } -] -``` - -*Note: As demonstrated with `safari` above, the `height` property within a viewport object is optional. If omitted, SmartUI performs a full-page capture automatically.* - -**devices** - -An array of mobile devices to capture screenshots from. List of supported device names can be found [here](#list-of-supported-device-viewports). - - Mobile viewports are emulated in desktop environments.Android devices will have the screenshots rendered in Chrome, while iOS devices in Safari. - SmartUI SDK will soon be supported simulation in case of iOS devices. - -**fullPage** - -Specifies whether to capture full-page screenshots for mobile devices. By default, `fullPage` is taken as true; set it to `false` in order to take a viewport screenshot on a mobile viewport. - -**orientation** - -Specifies the orientation of the mobile device. You can choose from `portrait` or `landscape` according to your usecase. By default, the orientation is taken as portrait. - -**waitForPageRender** - -If one or more URLs in your script require a relatively higher amount of time to load, you may use the `waitForPageRender` key in the config file to make sure the screenshots are rendered correctly. Avoid using the same in case your websites render in less than 30 seconds as it might increase the execution time of your tests. - -**waitForTimeout** - -If you are using any async components, you can add wait time for the page to load the DOM of your components. This can help avoid false-positive results for your tests. You can add the wait time in milliseconds, which might increase the execution time of your tests. - -**enableJavaScript** - -The `enableJavaScript` option is a boolean parameter that determines whether JavaScript is enabled for all snapshots within the project. Enabling JavaScript may lead to side-effects such as animations or redirects, potentially affecting the reliability of your snapshots. By default, this option is set to false. - -**allowedHostnames** - -The `allowedHostnames` option controls the capture of assets from specific hostnames. By default, the SmartUI SDK only captures assets that match the hostname of the snapshot location. For instance, if snapshots are taken on `https://xyz.com`, assets hosted on `https://cdn.xyz.com` will not be captured. To include assets from other hostnames, each additional hostname needs to be added to the allowedHostnames configuration. - -**lazyLoadConfiguration** - -The `lazyLoadConfiguration` option is used to handle lazy-loaded content on web pages. This configuration helps ensure that all content is loaded before capturing screenshots, preventing incomplete captures due to lazy loading. - -To use this feature, ensure you have the latest version of SmartUI CLI installed: - -```bash -npm install -g @lambdatest/smartui-cli@latest -``` - -Add the following configuration to your `.smartui.json` file: - -```json -{ -"lazyLoadConfiguration": { -"enabled": true, -"jumpBackToTop": true, -"scrollDelay": 250, -"scrollStep": 250 -} -} -``` - -- `enabled`: Boolean to enable/disable lazy load handling (default: false) -- `jumpBackToTop`: Boolean to scroll back to top after scrolling to bottom (default: true) -- `scrollDelay`: Delay in milliseconds between scroll actions (default: 250) -- `scrollStep`: Pixels to scroll in each step (default: 250) - -**List of supported Device viewports** - -**Note:** Use the exact string names provided below in the `devices` array of your `.smartui.json` mobile configuration. - -| iOS Devices | Android Devices | -|---|---| -| `iPad 10.2 (2019)` | `Aquos Sense 5G` | -| `iPad 10.2 (2020)` | `Blackberry KEY2 LE` | -| `iPad 10.2 (2021)` | `Galaxy A10s` | -| `iPad 10.9 (2022)` | `Galaxy A11` | -| `iPad 9.7 (2017)` | `Galaxy A12` | -| `iPad Air (2019)` | `Galaxy A13` | -| `iPad Air (2020)` | `Galaxy A21s` | -| `iPad Air (2022)` | `Galaxy A22` | -| `iPad Air 11 (2024)` | `Galaxy A31` | -| `iPad Air 13 (2024)` | `Galaxy A32` | -| `iPad mini (2019)` | `Galaxy A51` | -| `iPad mini (2021)` | `Galaxy A52s 5G` | -| `iPad Pro 11 (2021)` | `Galaxy A53 5G` | -| `iPad Pro 11 (2022)` | `Galaxy A7` | -| `iPad Pro 11 (2024)` | `Galaxy A70` | -| `iPad Pro 12.9 (2018)` | `Galaxy A8` | -| `iPad Pro 12.9 (2020)` | `Galaxy A8 Plus` | -| `iPad Pro 12.9 (2021)` | `Galaxy J7 Prime` | -| `iPad Pro 12.9 (2022)` | `Galaxy M12` | -| `iPad Pro 13 (2024)` | `Galaxy M31` | -| `iPhone 11` | `Galaxy Note10` | -| `iPhone 11 Pro` | `Galaxy Note10 Plus` | -| `iPhone 11 Pro Max` | `Galaxy Note20` | -| `iPhone 12` | `Galaxy Note20 Ultra` | -| `iPhone 12 Mini` | `Galaxy S10` | -| `iPhone 12 Pro` | `Galaxy S10 Plus` | -| `iPhone 12 Pro Max` | `Galaxy S10e` | -| `iPhone 13` | `Galaxy S20` | -| `iPhone 13 Mini` | `Galaxy S20 FE` | -| `iPhone 13 Pro` | `Galaxy S20 Plus` | -| `iPhone 13 Pro Max` | `Galaxy S20 Ultra` | -| `iPhone 14` | `Galaxy S21` | -| `iPhone 14 Plus` | `Galaxy S21 FE` | -| `iPhone 14 Pro` | `Galaxy S21 Plus` | -| `iPhone 14 Pro Max` | `Galaxy S21 Ultra` | -| `iPhone 15` | `Galaxy S22` | -| `iPhone 15 Plus` | `Galaxy S22 Ultra` | -| `iPhone 15 Pro` | `Galaxy S23` | -| `iPhone 15 Pro Max` | `Galaxy S23 Plus` | -| `iPhone 16` | `Galaxy S23 Ultra` | -| `iPhone 16 Plus` | `Galaxy S24` | -| `iPhone 16 Pro` | `Galaxy S24 Plus` | -| `iPhone 16 Pro Max` | `Galaxy S24 Ultra` | -| `iPhone 17` | `Galaxy S25` | -| `iPhone 17 Pro` | `Galaxy S25 Plus` | -| `iPhone 17 Pro Max` | `Galaxy S25 Ultra` | -| `iPhone 6` | `Galaxy S7` | -| `iPhone 6s` | `Galaxy S7 Edge` | -| `iPhone 6s Plus` | `Galaxy S8` | -| `iPhone 7` | `Galaxy S8 Plus` | -| `iPhone 7 Plus` | `Galaxy S9` | -| `iPhone 8` | `Galaxy S9 Plus` | -| `iPhone 8 Plus` | `Galaxy Tab A 10.1 (2019)` | -| `iPhone SE (2016)` | `Galaxy Tab A7 Lite` | -| `iPhone SE (2020)` | `Galaxy Tab A8` | -| `iPhone SE (2022)` | `Galaxy Tab S3` | -| `iPhone X` | `Galaxy Tab S4` | -| `iPhone XR` | `Galaxy Tab S7` | -| `iPhone XS` | `Galaxy Tab S8` | -| `iPhone XS Max` | `Galaxy Tab S8 Plus` | -| | `Galaxy Tab S9` | -| | `Galaxy Z Flip6` | -| | `Galaxy Z Flip7` | -| | `Galaxy Z Fold6` | -| | `Galaxy Z Fold7` | -| | `Honeywell CT40` | -| | `Honor X9a 5G` | -| | `Huawei Mate 20 Pro` | -| | `Huawei P20 Pro` | -| | `Huawei P30` | -| | `Huawei P30 Lite` | -| | `Huawei P30 Pro` | -| | `Huawei P50 Pro` | -| | `Microsoft Surface Duo` | -| | `Moto G Stylus (2022)` | -| | `Moto G22` | -| | `Moto G54 5G` | -| | `Moto G7 Play` | -| | `Moto G71 5G` | -| | `Moto G9 Play` | -| | `Motorola Edge 30` | -| | `Motorola Edge 40` | -| | `Motorola Edge 50 Pro` | -| | `Nexus 5` | -| | `Nexus 5X` | -| | `Nokia 5` | -| | `Nothing Phone (1)` | -| | `Nothing Phone 1` | -| | `Nothing Phone 2` | -| | `OnePlus 10 Pro` | -| | `OnePlus 11` | -| | `OnePlus 12` | -| | `OnePlus 6` | -| | `OnePlus 6T` | -| | `OnePlus 7` | -| | `OnePlus 7T` | -| | `OnePlus 8` | -| | `OnePlus 9` | -| | `OnePlus 9 Pro` | -| | `OnePlus Nord` | -| | `OnePlus Nord 2` | -| | `OnePlus Nord CE` | -| | `Oppo A12` | -| | `Oppo A15` | -| | `Oppo A54` | -| | `Oppo A5s` | -| | `Oppo F17` | -| | `Oppo K10` | -| | `Pixel 10 Pro` | -| | `Pixel 10 Pro XL` | -| | `Pixel 3` | -| | `Pixel 3 XL` | -| | `Pixel 3a` | -| | `Pixel 4` | -| | `Pixel 4 XL` | -| | `Pixel 4a` | -| | `Pixel 5` | -| | `Pixel 6` | -| | `Pixel 6 Pro` | -| | `Pixel 6a` | -| | `Pixel 7` | -| | `Pixel 7 Pro` | -| | `Pixel 7a` | -| | `Pixel 8` | -| | `Pixel 8 Pro` | -| | `Pixel 9` | -| | `Pixel 9 Pro` | -| | `Pixel 9 Pro XL` | -| | `Pixel Tablet` | -| | `Poco M2 Pro` | -| | `POCO X3 Pro` | -| | `Realme 5i` | -| | `Realme 7i` | -| | `Realme 8i` | -| | `Realme C21` | -| | `Realme C21Y` | -| | `Realme GT2 Pro` | -| | `Redmi 8` | -| | `Redmi 9` | -| | `Redmi 9A` | -| | `Redmi 9C` | -| | `Redmi Note 10 Pro` | -| | `Redmi Note 13 Pro` | -| | `Redmi Note 8` | -| | `Redmi Note 8 Pro` | -| | `Redmi Note 9` | -| | `Redmi Note 9 Pro Max` | -| | `Redmi Y2` | -| | `Tecno Spark 7` | -| | `Vivo T1` | -| | `Vivo V7` | -| | `Vivo Y11` | -| | `Vivo Y12` | -| | `Vivo Y20g` | -| | `Vivo Y22` | -| | `Vivo Y50` | -| | `Xiaomi 12 Pro` | -| | `Xperia 10 IV` | -| | `Xperia Z5` | -| | `Xperia Z5 Dual` | -| | `Zenfone 6` | diff --git a/static/docs/smartui-sdk-fetch-results.md b/static/docs/smartui-sdk-fetch-results.md deleted file mode 100644 index ba6499c26..000000000 --- a/static/docs/smartui-sdk-fetch-results.md +++ /dev/null @@ -1,317 +0,0 @@ -# Fetching Results through SmartUI SDK - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -SmartUI CLI allows you to fetch detailed build results after executing your visual tests. This feature enables you to access comprehensive information about your build and screenshots in a JSON file, making it easier to integrate with your CI/CD pipelines and automation workflows. - -## Prerequisites - -- Basic understanding of Command Line Interface -- Login to [TestMu AI SmartUI](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) with your credentials. -- Ensure you are using `@lambdatest/smartui-cli` version 4.1.43 or higher -- A properly configured SmartUI CLI project - -If you face any problems executing tests with SmartUI-CLI `versions >= v4.x.x`, upgrade your Node.js version to `v20.3` or above. - -## Steps to Use - -### **Step 1:** Install SmartUI CLI - -If you haven't already installed SmartUI CLI, install it using npm: - -**Global Installation (Recommended):** -```bash -npm install -g @lambdatest/smartui-cli -``` - -**Local Installation:** -```bash -npm install @lambdatest/smartui-cli -``` - -### **Step 2:** Configure your Project Token - -Setup your project token shown in the **SmartUI** app after creating your project. - -```bash -export PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -``` - -```bash -set PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -``` - -```powershell -$env:PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -``` - -### **Step 3:** Execute Tests with Results Fetching - -You can fetch build results by adding the `--fetch-results` flag to your test execution command. Here are different ways to use this feature: - -#### Default Usage -If no filename is specified, results will be stored in `results.json`: - -```bash -npx smartui --config .smartui.json exec --fetch-results -- -``` ->**Example:** - For a `Node.js` test script : ->```bash ->npx smartui --config .smartui.json exec --fetch-results -- node test.js ->``` - -#### Custom Filename -Specify a custom filename for your results: - -```bash -npx smartui --config .smartui.json exec --fetch-results custom-results.json -- node test.js -``` - -### **Step 4:** Understanding the Results - -The fetched results JSON file contains detailed information about your build and screenshots. Here's what you'll find in the results file: - -```json -{ -"screenshots": { -"Screenshot-1": [ -{ -"screenshot_name": "Screenshot-1", -"captured_image": "image_url", -"baseline_image": "image_url", -"compared_image": "image_url", -"browser_name": "edge", -"viewport": "1920", -"mismatch_percentage": 3.3, -"status": "Changes found" -}, -{ -"screenshot_name": "Screenshot-1", -"captured_image": "image_url", -"baseline_image": "image_url", -"compared_image": "image_url", -"browser_name": "firefox", -"viewport": "1366", -"mismatch_percentage": 4.74, -"status": "Changes found" -}, -{ -"screenshot_name": "Screenshot-1", -"captured_image": "image_url", -"baseline_image": "image_url", -"compared_image": "image_url", -"browser_name": "chrome", -"viewport": "1366", -"mismatch_percentage": 4.64, -"status": "Changes found" -}, -{ -"screenshot_name": "Screenshot-1", -"captured_image": "image_url", -"baseline_image": "image_url", -"compared_image": "image_url", -"browser_name": "chrome", -"viewport": "1920", -"mismatch_percentage": 3.3, -"status": "Changes found" -}, -], -"Screenshot-2": [ -{ -"screenshot_name": "Screenshot-2", -"captured_image": "image_url", -"baseline_image": "image_url", -"compared_image": "image_url", -"browser_name": "edge", -"viewport": "1920", -"mismatch_percentage": 0.0, -"status": "Approved" -}, -{ -"screenshot_name": "Screenshot-2", -"captured_image": "image_url", -"baseline_image": "image_url", -"compared_image": "image_url", -"browser_name": "firefox", -"viewport": "1366", -"mismatch_percentage": 4.74, -"status": "Changes found" -}, -{ -"screenshot_name": "Screenshot-2", -"captured_image": "image_url", -"baseline_image": "image_url", -"compared_image": "image_url", -"browser_name": "chrome", -"viewport": "1366", -"mismatch_percentage": 4.64, -"status": "Changes found" -}, -{ -"screenshot_name": "Screenshot-2", -"captured_image": "image_url", -"baseline_image": "image_url", -"compared_image": "image_url", -"browser_name": "chrome", -"viewport": "1920", -"mismatch_percentage": 3.3, -"status": "Changes found" -}, -] -}, -"build": { -"build_id": "b420b7a9-77c6-****-****", -"baseline": false, -"build_type": "smartui-cli", -"build_status_ind": "completed", -"build_status": "pending-approval", -"commitId": "2b93***", -"branch": "main", -"commitAuthor": "John Doe", -"commitMessage": "Merge pull request from xyz/main" -}, -"project": { -"project_id": "1dfb7712-7f20-446f-***-***", -"name": "Project-Name", -"username": "johndoe", -"project_type": "smartui-cli", -"projectCategory": "web", -"platform": "cli" -} -} -``` - -## Using SmartUI Reporter Tool - -The SmartUI Reporter is a web-based tool that allows you to visualize and analyze your SmartUI test results in a tabular format with comprehensive statistics and export capabilities. - -### Step 1: Obtain Results JSON File - -You can get the `results.json` file using one of the following methods: - -#### Method 1: Export from CLI (Recommended) - -Use the `--fetch-results` flag when executing your SmartUI tests: - -**For Capture Command:** -```bash -npx smartui capture urls.json --config config.json --fetch-results results.json -``` - -**For Exec Command:** -```bash -npx smartui --config .smartui.json exec --fetch-results results.json -- -``` - -**Example with Capture:** -```bash -npx smartui capture urlTest.json --config config.json --fetch-results results.json -``` - -**Example with Exec:** -```bash -npx smartui --config .smartui.json exec --fetch-results results.json -- npm test -``` - -If no filename is specified, results will be saved as `results.json` by default: -```bash -npx smartui capture urls.json --config config.json --fetch-results -``` - -#### Method 2: Fetch from SmartUI API - -You can also fetch the results JSON file directly from the SmartUI API using the [Fetch Build Screenshots endpoint](https://swagger-api-support.lambdatest.com/smartui/index.html#/Fetch%20Build%20Screenshots). - -**API Endpoint:** -``` -GET /build/screenshots -``` - -**Parameters:** -- `project_id` (required): Your SmartUI project ID -- `build_id` (optional): Specific build ID -- `build_name` (optional): Build name (if both build_id and build_name are provided, build_id takes priority) - -**Example API Request:** -```bash -curl -X GET "https://api.lambdatest.com/smartui/build/screenshots?project_id=YOUR_PROJECT_ID&build_id=YOUR_BUILD_ID" \ --H "Authorization: Bearer YOUR_ACCESS_TOKEN" -``` - -For detailed API documentation, visit the [SmartUI API Swagger Documentation](https://swagger-api-support.lambdatest.com/smartui/index.html#/Fetch%20Build%20Screenshots). - -### Step 2: Upload to SmartUI Reporter - -1. Navigate to [SmartUI Reporter](https://smartui-reporter.netlify.app/) -2. Click on the upload area or drag and drop your `results.json` file -3. Wait for the file to be processed and parsed - -### Step 3: View Results - -After uploading, you'll see: - -- **Tabular View**: All screenshots organized in a comprehensive table -- **Statistics**: Key metrics including: - - Total screenshots - - Mismatch percentage - - Status breakdown (Approved, Changes Found, etc.) - - Browser and viewport distribution -- **Screenshot Links**: Direct links to: - - Baseline images - - Captured images - - Comparison/diff images -- **Build Information**: Project details, build ID, branch, commit information - -### Step 4: Export Options - -The SmartUI Reporter provides two export options: - -#### Export to PDF -- Click the **Export to PDF** button -- Generates a comprehensive PDF report with all screenshots and statistics -- Perfect for sharing with stakeholders or archiving results - -#### Export to CSV -- Click the **Export to CSV** button -- Exports all data in CSV format including: - - Page Name - - Browser - - Resolution/Viewport - - Mismatch Percentage - - Status - - Screenshot URLs (Baseline, Captured, Diff) -- Ideal for data analysis in spreadsheet applications - -### Use Cases - -- **CI/CD Integration**: Automatically generate reports after test execution -- **Stakeholder Reporting**: Share visual test results with non-technical team members -- **Historical Analysis**: Track visual changes over time -- **Debugging**: Quickly identify which screenshots have mismatches -- **Documentation**: Create PDF reports for project documentation - -### Example Workflow - -```bash -# 1. Run SmartUI tests and fetch results -npx smartui capture urlTest.json --config config.json --fetch-results results.json - -# 2. Upload results.json to SmartUI Reporter -# Visit https://smartui-reporter.netlify.app/ and upload the file - -# 3. View results in tabular format with statistics - -# 4. Export to PDF or CSV for sharing -``` - -## Additional Resources - -- [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) -- [CLI Documentation](/support/docs/smartui-cli) -- [CLI Exec Commands](/support/docs/smartui-cli-exec) -- [Baseline Management](/support/docs/smartui-baseline-management) -- [Running Your First Project](/support/docs/smartui-running-your-first-project) -- [SmartUI API Documentation](https://www.testmuai.com/support/api-doc/) -- [SmartUI Reporter Tool](https://smartui-reporter.netlify.app/) -- [SmartUI API Swagger Documentation](https://swagger-api-support.lambdatest.com/smartui/index.html#/Fetch%20Build%20Screenshots) diff --git a/static/docs/smartui-sdk-tunnel.md b/static/docs/smartui-sdk-tunnel.md deleted file mode 100644 index 0095fff7e..000000000 --- a/static/docs/smartui-sdk-tunnel.md +++ /dev/null @@ -1,170 +0,0 @@ -# Using TestMu AI Tunnel with SmartUI SDK - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI Tunnel enables secure testing of your local, development, or privately hosted environments using SmartUI SDK. This guide explains how to configure and use TestMu AI Tunnel with SmartUI SDK. - -**Important** -Tunnel configuration is only supported with the `exec` mode of SmartUI SDK. It is not compatible with -`capture`, `upload`, or `figma` commands. For more information about exec mode, refer to our [SmartUI CLI Exec -documentation](/support/docs/smartui-cli-exec/). - -## Prerequisites - -- Working knowledge of npm CLI commands -- Understanding of tunneling concepts for local testing -- Valid TestMu AI credentials (username and access key) -- Familiarity with SmartUI CLI configuration files -- SmartUI CLI installed and configured - -## Configuration Options - -TestMu AI Tunnel supports two configuration modes: - -### 1. Automatic Tunnel Configuration - -For automatic tunnel setup, add the following configuration to your `.smartui.json` file: - -```json -{ -"tunnel": { -"type": "auto", -"user": "", // Optional: Overrides LT_USERNAME if present in env -"key": "", // Optional: Overrides LT_ACCESS_KEY if present in env -"port": "", // Optional: Default port if not specified -"proxyHost": "127.0.0.1", // Optional -"proxyPort": "8000", // Optional -"proxyUser": "user", // Optional: Proxy authentication -"proxyPass": "password", // Optional: Proxy authentication -"dir": "", // Optional: Directory to test -"v": true, // Optional: Enable verbose logging -"logFile": "/path/to/logs.txt" // Optional: Custom log file location -} -} -``` - -### 2. Manual Tunnel Configuration - -For manual tunnel setup, add the following configuration: - -```json -{ -"tunnel": { -"type": "manual", -"tunnelName": "my-tunnel-name" // Required for manual mode -//Add user and key in case LT_USERNAME and LT_ACCESSKEY not provided in env variables -} -} -``` - -Configuration values take precedence over environment variables when both are present. - -## Environment Variables Support - -The tunnel configuration automatically uses the following environment variables if not specified in the config: - -- `LT_USERNAME`: TestMu AI username -- `LT_ACCESS_KEY`: TestMu AI access key - -## Example Configurations - -### Basic Automatic Setup -```json -{ -"web": { -"browsers": ["chrome", "firefox"], -"viewports": [[1920, 1080], [1366, 768]] -}, -"tunnel": { -"type": "auto" -//Add user and key in case LT_USERNAME and LT_ACCESSKEY not provided in env variables - -} -} -``` - -### Advanced Automatic Setup with Proxy -```json -{ -"web": { -"browsers": ["chrome", "firefox"], -"viewports": [[1920, 1080], [1366, 768]] -}, -"tunnel": { -"type": "auto", -"user": "", // Optional: Overrides LT_USERNAME if present in env -"key": "", // Optional: Overrides LT_ACCESS_KEY if present in env -"proxyHost": "127.0.0.1", -"proxyPort": "8000", -"dir": "./src", -"v": true -} -} -``` - -### Manual Tunnel Setup -```json -{ -"web": { -"browsers": ["chrome", "firefox"], -"viewports": [[1920, 1080], [1366, 768]] -}, -"tunnel": { -"type": "manual", -"tunnelName": "my-tunnel", -"user": "", // Optional: Overrides LT_USERNAME if present in env -"key": "", // Optional: Overrides LT_ACCESS_KEY if present in env -} -} -``` - -## Best Practices - -**Automatic vs Manual Mode** - -- Use automatic mode for simple setups and quick testing - - Use manual mode when you need more control over tunnel lifecycle - -**Credentials Management** - -- Prefer environment variables for sensitive information - - Use config file for non-sensitive settings - -**Proxy Configuration** - -- Specify proxy settings in config when using corporate networks - - Test proxy settings before running full test suite - -**Logging** - -- Enable verbose logging (`"v": true`) for troubleshooting - - Specify custom log file location for persistent logs - -**Directory Testing** - -- Always specify the directory to test when working with local files - - Ensure the directory path is accessible and contains the necessary files - -For more detailed information about TestMu AI Tunnel features and configurations, refer to our [Advanced Tunnel Features documentation](/support/docs/advanced-tunnel-features). - -1. **Automatic vs Manual Mode** - - Use automatic mode for simple setups and quick testing - - Use manual mode when you need more control over tunnel lifecycle - -2. **Credentials Management** - - Prefer environment variables for sensitive information - - Use config file for non-sensitive settings - -3. **Proxy Configuration** - - Specify proxy settings in config when using corporate networks - - Test proxy settings before running full test suite - -4. **Logging** - - Enable verbose logging (`"v": true`) for troubleshooting - - Specify custom log file location for persistent logs - -5. **Directory Testing** - - Always specify the directory to test when working with local files - - Ensure the directory path is accessible and contains the necessary files - -For more detailed information about TestMu AI Tunnel features and configurations, refer to our [Advanced Tunnel Features documentation](/support/docs/advanced-tunnel-features). diff --git a/static/docs/smartui-selenium-csharp-sdk.md b/static/docs/smartui-selenium-csharp-sdk.md deleted file mode 100644 index 0768032a4..000000000 --- a/static/docs/smartui-selenium-csharp-sdk.md +++ /dev/null @@ -1,613 +0,0 @@ -# Integrate SmartUI SDK with Selenium-CSharp - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Welcome to the world of simplified visual testing with the SmartUI SDK. - -Integrating seamlessly into your existing Selenium testing suite, SmartUI SDK revolutionizes the way you approach visual regression testing. Our robust solution empowers you to effortlessly capture, compare, and analyze screenshots across a multitude of browsers and resolutions, ensuring comprehensive coverage and accuracy in your visual testing endeavors. - -## Prerequisites - -- Basic understanding of Command Line Interface and Selenium is required. -- Login to [TestMu AI SmartUI](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) with your credentials. - -The following steps will guide you in running your first Visual Regression test on TestMu AI platform using SmartUI Selenium SDK integration. - -## Create a SmartUI Project - -The first step is to create a project with the application in which we will combine all your builds run on the project. To create a SmartUI Project, follow these steps: - -1. Go to [Projects page](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) -2. Click on the `new project` button -3. Select the platform as CLI for executing your `SDK` tests. -4. Add name of the project, approvers for the changes found, tags for any filter or easy navigation. -5. Click on the **Submit**. - -## Steps to run your first test - -Once you have created a SmartUI Project, you can generate screenshots by running automation scripts. Follow the below steps to successfully generate screenshots - -### **Step 1:** Create/Update your test - -You can clone the sample repository to run `LambdaTest` automation tests with `SmartUI` and use `LTCloudTest.cs` file located in the `LambdaTest.Selenium.Driver.Test` folder. - -```bash -git clone https://github.com/LambdaTest/smartui-csharp-sample -cd smartui-csharp-sample/LambdaTest.Selenium.Driver.Test -``` - -### **Step 2**: Update the Dependencies - -- Add the following dependencies in your `.csproj` file - -```cs - - - -``` - -You can check the latest version of [TestMu AI.Selenium.Driver]( https://www.nuget.org/packages/LambdaTest.Selenium.Driver) and update the latest version accordingly. - -### **Step 3**: Install the Dependencies - -Install required NPM modules for `TestMu AI SmartUI Selenium SDK` in your **Frontend** project. - -```bash -npm install @lambdatest/smartui-cli -``` - -If you face any problems executing tests with SmartUI-CLI `versions >= v4.x.x`, upgrade your Node.js version to `v20.3` or above. - -```bash -dotnet restore -``` - -### **Step 4:** Configure your Project Token - -Setup your project token shown in the **SmartUI** app after creating your project. - -```bash -export PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -``` - -```bash -set PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -``` - -```powershell -$env:PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -``` - -### **Step 5:** Create and Configure SmartUI Config - -You can now configure your project configurations on using various available options to run your tests with the SmartUI integration. To generate the configuration file, please execute the following command: - -```bash -npx smartui config:create .smartui.json -``` - -Once, the configuration file will be created, you will be seeing the default configuration pre-filled in the configuration file: - -```json title="/smartui-sdk-project/.smartui.json" -{ -"web": { -"browsers": [ -"chrome", -"firefox", -"safari", -"edge" -], -"viewports": [ -[ -1920 -], -[ -1366 -], -[ -1028 -] -] // Full Page screenshots are captured by default for web viewports -}, -"mobile": { -"devices": [ -"iPhone 14", //iPhone 14 viewport -"Galaxy S24" //Galaxy S24 viewport -], -"fullPage": true, //Full Page is true by default for mobile viewports -"orientation": "portrait" //Change to "landscape" for landscape snapshot -}, -"waitForTimeout": 1000, //Optional (Should only be used in case lazy-loading/async components are present) -"waitForPageRender": 50000, //Optional (Should only be used in case of websites which take more than 30s to load) -"enableJavaScript": false, //Enable javascript for all the screenshots of the project -"allowedHostnames": [] //Additional hostnames to capture assets from -} -``` -**Advanced options in SmartUI configuration** -- For capturing fullpage or viewport screenshots, please refer to this [documentation](/docs/smartui-sdk-config-options/#12-viewports) -- For the list of available mobile viewports, please refer to this [documentation](/docs/smartui-sdk-config-options/#list-of-supported-device-viewports) -- For more information about SmartUI config global options, please refer to this [documentation](/docs/smartui-sdk-config-options/#3-global-options-optional). - -### **Step 6:** Adding SmartUI function to take screenshot - -- You can incorporate SmartUI into your custom `Selenium` automation test (any platform) script by adding the `smartuiSnapshot` function in the required segment of selenium script of which we would like to take the screenshot, as shown below: - -```csharp -using System; -using System.Threading.Tasks; -using OpenQA.Selenium; -using OpenQA.Selenium.Chrome; -using LambdaTest.Selenium.Driver; - -namespace LambdaTest.Selenium.TestProject -{ -public static class LocalTest -{ -using IWebDriver driver = new ChromeDriver(); -try -{ -Console.WriteLine("Driver started"); -driver.Navigate().GoToUrl("Required URL"); -await SmartUISnapshot.CaptureSnapshot(driver, "Screenshot Name"); //utilize this function to take the dom snapshot of your test -} -catch (Exception ex) -{ -Console.WriteLine(ex); -} -finally -{ -driver.Quit(); -} -} -} -``` - -#### **Additional Functionality: Using `sync` Option in SmartUI C# SDK** - -You can enable synchronous snapshot status response by setting the `sync` option to `true` in the options dictionary. This allows you to wait for the snapshot status and receive the result directly in your test script. - -- The `sync` functionality is supported only in TestMu AI.Selenium.Driver version **1.0.4 and above**. - -**Set the sync value in options:** - -```csharp -var optionsForSync = new Dictionary -{ -{ "sync", true }, -{ "timeout", 100 } // timeout in seconds (30-900, default 600) -}; - -Console.WriteLine("Driver started"); -driver.Navigate().GoToUrl("https://www.testmuai.com"); -var result = await SmartUISnapshot.CaptureSnapshot(driver, "NYC", optionsForSync); -Console.WriteLine(result); -``` - -**Notes:** -- The default value of `sync` is `false` if not specified. -- The `timeout` option defines how long to wait for the snapshot status response (in seconds). Allowed range: 30-900, default is 600. -- When `sync` is `true`, the `result` variable will hold the value of the snapshot status response. - -**Sample response:** -```json -{ -"snapshotName": "Sync-True", -"snapshotUUID": "95226130-72b6-4d45-ad6d-4ad8ddsa1", -"buildId": "8e0c078d-e85a-41ae-a8d5-4a0dsdf8bbd5", -"snapshotStatus": "failed", -"startedProcessingAt": "2025-08-26 09:58:21", -"finishedProcessingAt": "2025-08-26 10:03:12", -"screenshots": [ -{ -"captured_image": "", -"baseline_image": "", -"browser_name": "firefox", -"viewport": "1028", -"mismatch_percentage": 89.58, -"status": "Changes found", -"captured_image_timestamp": "2025-08-26 10:00:40", -"compared_image_timestamp": "2025-08-26 10:00:58", -"captured_diff": "", -"baseline_diff": "", -}, -// ... more screenshots in the same format -] -} -``` - -### **Step 7:** Execute the Tests on SmartUI Cloud - -Execute `visual regression tests` on SmartUI using the following commands - -```bash -npx smartui --config .smartui.json exec -- dotnet run cloud -``` - -You may use the `npx smartui --help` command in case you are facing issues during the execution of SmartUI commands in the CLI. - -## View SmartUI Results - -You have successfully integrated SmartUI SDK with your Selenium tests. Visit your SmartUI project to view builds and compare snapshots between different test runs. - -You can see the SmartUI dashboard to view the results. This will help you identify the Mismatches from the existing `Baseline` build and do the required visual testing. - -## Arguments supported in the `smartUISnapshot` function - -The following are the different options which are currently supported: - -| Key | Description | -| ------------------------- | ------------------------------------------------------------------------------------------------------------------------- | -| `driver` (instance) | The instance of the web driver used in your tests. | -| `"Screenshot Name"` (string) | Specify a name for the screenshot in your tests to match the same screenshot with the name from your baseline. | -| `options` (object) | Specify one or a combination of selectors in the `ignoreDOM` or `selectDOM` objects. These selectors can be based on `HTML DOM IDs, CSS classes, CSS selectors, or XPaths` used by your webpage. They define elements that should be excluded from or included in the visual comparison.| - -## Handling Dynamic Data in SmartUI SDK **** - -When conducting visual tests, you may encounter scenarios where certain elements within your application change between test runs. These changes might introduce inconsistencies in your test results.You can ignore / select specific element(s) to be removed from the comparison by parsing the options in the `smartuiSnapshot` function in the following way - -```cs title="This is a sample for your configuration for C# to ignore by ID" -driver.Navigate().GoToUrl("Required URL"); - -var options = new Dictionary -{ -{ "ignoreDOM", new Dictionary -{ -{ "id", new[] { "ID-1", "ID-2" } } -} -} -}; -await SmartUISnapshot.CaptureSnapshot(driver, "Screenshot Name",options); -``` - -```cs title="This is a sample for your configuration for C# to ignore by Class" -driver.Navigate().GoToUrl("Required URL"); - -var options = new Dictionary -{ -{ "ignoreDOM", new Dictionary -{ -{ "class", new[] { "class-1", "class-2" } } -} -} -}; -await SmartUISnapshot.CaptureSnapshot(driver, "Screenshot Name",options); -``` - -```cs title="This is a sample for your configuration for C# to ignore by XPath" -driver.Navigate().GoToUrl("Required URL"); - -var options = new Dictionary -{ -{ "ignoreDOM", new Dictionary -{ -{ "xpath", new[] { "xpath-1", "xpath-2" } } -} -} -}; -await SmartUISnapshot.CaptureSnapshot(driver, "Screenshot Name",options); -``` - -```cs title="This is a sample for your configuration for C# to ignore by CSS Selector" -driver.Navigate().GoToUrl("Required URL"); - -var options = new Dictionary -{ -{ "ignoreDOM", new Dictionary -{ -{ "cssSelector", new[] { "cssSelector-1", "cssSelector-2" } } -} -} -}; -await SmartUISnapshot.CaptureSnapshot(driver, "Screenshot Name",options); -``` - -```cs title="This is a sample for your configuration for C# to select by ID" -driver.Navigate().GoToUrl("Required URL"); - -var options = new Dictionary -{ -{ "selectDOM", new Dictionary -{ -{ "id", new[] { "ID-1", "ID-2" } } -} -} -}; -await SmartUISnapshot.CaptureSnapshot(driver, "Screenshot Name",options); -``` - -```cs title="This is a sample for your configuration for C# to select by Class" -driver.Navigate().GoToUrl("Required URL"); - -var options = new Dictionary -{ -{ "selectDOM", new Dictionary -{ -{ "class", new[] { "class-1", "class-2" } } -} -} -}; -await SmartUISnapshot.CaptureSnapshot(driver, "Screenshot Name",options); -``` - -```cs title="This is a sample for your configuration for C# to select by XPath" -driver.Navigate().GoToUrl("Required URL"); - -var options = new Dictionary -{ -{ "selectDOM", new Dictionary -{ -{ "xpath", new[] { "xpath-1", "xpath-2" } } -} -} -}; -await SmartUISnapshot.CaptureSnapshot(driver, "Screenshot Name",options); -``` - -```cs title="This is a sample for your configuration for C# to select by CSS Selector" -driver.Navigate().GoToUrl("Required URL"); - -var options = new Dictionary -{ -{ "selectDOM", new Dictionary -{ -{ "cssSelector", new[] { "cssSelector-1", "cssSelector-2" } } -} -} -}; -await SmartUISnapshot.CaptureSnapshot(driver, "Screenshot Name",options); -``` - -## For capturing the screenshot of a specific element - -You can capture screenshots of targeted elements by leveraging various locator mechanisms such as XPath, CSS ID, class, and selectors. This precision-driven approach ensures accurate and specific visual regression testing for your web application's components. - -```cs title="This is a sample for your configuration for C# to capture an element by ID" -driver.Navigate().GoToUrl("Required URL"); - -var options = new Dictionary -{ -{ "element", new Dictionary -{ -{ "id", new[] { "Required ID"} } -} -} -}; -await SmartUISnapshot.CaptureSnapshot(driver, "Screenshot Name",options); -``` - -```cs title="This is a sample for your configuration for C# to capture an element by Class" -driver.Navigate().GoToUrl("Required URL"); - -var options = new Dictionary -{ -{ "element", new Dictionary -{ -{ "class", new[] { "Required class" } } -} -} -}; -await SmartUISnapshot.CaptureSnapshot(driver, "Screenshot Name",options); -``` - -```cs title="This is a sample for your configuration for C# to capture an element by XPath" -driver.Navigate().GoToUrl("Required URL"); - -var options = new Dictionary -{ -{ "element", new Dictionary -{ -{ "xpath", new[] { "Required xpath"} } -} -} -}; -await SmartUISnapshot.CaptureSnapshot(driver, "Screenshot Name",options); -``` - -```cs title="This is a sample for your configuration for C# to capture an element by CSS Selector" -driver.Navigate().GoToUrl("Required URL"); - -var options = new Dictionary -{ -{ "element", new Dictionary -{ -{ "cssSelector", new[] { "Required cssSelector"} } -} -} -}; -await SmartUISnapshot.CaptureSnapshot(driver, "Screenshot Name",options); -``` - -## Best Practices - -**Screenshot Naming** - -- Use descriptive, consistent names for screenshots -- Include page/component name in screenshot names -- Avoid special characters that might cause issues -- Use consistent naming conventions across your test suite - -**Example:** -```csharp -await SmartUISnapshot.CaptureSnapshot(driver, "HomePage-Header"); -await SmartUISnapshot.CaptureSnapshot(driver, "ProductPage-MainContent"); -``` - -**Wait for Page Load** - -- Always wait for pages to fully load before taking screenshots -- Use Selenium's WebDriverWait for dynamic content -- Consider using `waitForTimeout` in configuration for lazy-loaded content - -**Example:** -```csharp -using OpenQA.Selenium.Support.UI; -using OpenQA.Selenium; - -driver.Navigate().GoToUrl("https://example.com"); -WebDriverWait wait = new WebDriverWait(driver, TimeSpan.FromSeconds(10)); -wait.Until(ExpectedConditions.ElementExists(By.Id("main-content"))); -await SmartUISnapshot.CaptureSnapshot(driver, "Page Loaded"); -``` - -**Handle Dynamic Content** - -- Use `ignoreDOM` for elements that change between runs -- Use `selectDOM` when you only need to compare specific areas -- Document why elements are ignored for future reference - -**Configuration Management** - -- Keep `.smartui.json` in version control -- Use environment variables for sensitive data -- Document custom configuration choices - -**Test Organization** - -- Group related screenshots in the same build -- Use meaningful build names -- Run tests in consistent environments - -**Troubleshooting** - -**Issue: Screenshots Not Appearing in Dashboard** - -**Symptoms**: Tests run successfully but no screenshots appear in SmartUI dashboard - -**Possible Causes**: -- Project token not set or incorrect -- Project name mismatch -- Network connectivity issues -- CLI not installed or outdated - -**Solutions**: -1. Verify `PROJECT_TOKEN` is set correctly: -```powershell -echo $env:PROJECT_TOKEN -``` - -2. Check project name matches exactly (case-sensitive) - -3. Verify SmartUI CLI is installed: -```bash -npx smartui --version -``` - -4. Check network connectivity to TestMu AI servers - -5. Review test execution logs for error messages - -**Issue: "Project Not Found" Error** - -**Symptoms**: Error message indicating project cannot be found - -**Possible Causes**: -- Incorrect project token -- Project deleted or renamed -- Token from wrong project - -**Solutions**: -1. Verify project exists in SmartUI dashboard -2. Copy project token directly from Project Settings -3. Ensure token includes the project ID prefix (e.g., `123456#...`) -4. Check for extra spaces or quotes in token - -**Issue: Screenshots Show Blank or Incorrect Content** - -**Symptoms**: Screenshots captured but show blank pages or incorrect content - -**Possible Causes**: -- Page not fully loaded -- JavaScript not executed -- Viewport size issues -- Timing issues - -**Solutions**: -1. Add explicit waits before screenshots: -```csharp -WebDriverWait wait = new WebDriverWait(driver, TimeSpan.FromSeconds(10)); -wait.Until(ExpectedConditions.ElementExists(By.Id("content"))); -wait.Until(ExpectedConditions.ElementIsVisible(By.CssSelector(".main-content"))); -``` - -2. Enable JavaScript in configuration: -```json -{ -"enableJavaScript": true -} -``` - -3. Increase `waitForTimeout` in configuration - -4. Verify viewport size matches expected dimensions - -**Issue: Build Execution Fails** - -**Symptoms**: `npx smartui exec` command fails - -**Possible Causes**: -- Missing or incorrect configuration file -- Invalid JSON in configuration -- Port conflicts -- Permission issues - -**Solutions**: -1. Verify `.smartui.json` exists and is valid JSON -2. Check configuration file syntax -3. Try different port if default is in use: -```bash -npx smartui exec -P 5000 -- -``` -4. Check file permissions for configuration and project files - -**Issue: NuGet Dependencies Not Resolving** - -**Symptoms**: NuGet cannot find `LambdaTest.Selenium.Driver` package - -**Possible Causes**: -- Incorrect package version -- NuGet registry access issues -- Network connectivity problems - -**Solutions**: -1. Check latest version on [NuGet.org](https://www.nuget.org/packages/LambdaTest.Selenium.Driver) -2. Clear NuGet cache: -```bash -dotnet nuget locals all --clear -``` -3. Verify internet connectivity for NuGet registry access -4. Check project file for version conflicts - -**Issue: Screenshot Names Not Matching Baseline** - -**Symptoms**: Screenshots appear as "New" instead of comparing with baseline - -**Possible Causes**: -- Screenshot name changed -- Baseline doesn't exist -- Name contains special characters - -**Solutions**: -1. Ensure screenshot names are consistent across test runs -2. Verify baseline exists in project -3. Avoid special characters in screenshot names -4. Check for case sensitivity issues - -**Getting Help** - -If you encounter issues not covered here: - -- Review the [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) for detailed solutions -- Check [SmartUI Configuration Options](/support/docs/smartui-sdk-config-options) documentation -- See [Handling Dynamic Data](/support/docs/smartui-handle-dynamic-data) for dynamic content issues -- Visit [TestMu AI Support](https://www.testmuai.com/support) for additional resources -- Contact support at support@testmuai.com or use [24/7 Chat Support](https://www.testmuai.com/support) - -## Additional Resources - -- [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) -- [SmartUI Configuration Options](/support/docs/smartui-sdk-config-options) -- [Handling Dynamic Data](/support/docs/smartui-handle-dynamic-data) -- [Handling Lazy Loading](/support/docs/smartui-handle-lazy-loading) -- [Baseline Management](/support/docs/smartui-baseline-management) -- [Running Your First Project](/support/docs/smartui-running-your-first-project) -- [SmartUI API Documentation](https://www.testmuai.com/support/api-doc/) diff --git a/static/docs/smartui-selenium-java-sdk.md b/static/docs/smartui-selenium-java-sdk.md deleted file mode 100644 index afdc95359..000000000 --- a/static/docs/smartui-selenium-java-sdk.md +++ /dev/null @@ -1,558 +0,0 @@ -# Integrate SmartUI SDK with Selenium-Java - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Welcome to the world of simplified visual testing with the SmartUI SDK. - -Integrating seamlessly into your existing Selenium testing suite, SmartUI SDK revolutionizes the way you approach visual regression testing. Our robust solution empowers you to effortlessly capture, compare, and analyze screenshots across a multitude of browsers and resolutions, ensuring comprehensive coverage and accuracy in your visual testing endeavors. - -## Prerequisites - -- Basic understanding of Command Line Interface and Selenium is required. -- Login to [TestMu AI SmartUI](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) with your credentials. - -The following steps will guide you in running your first Visual Regression test on TestMu AI platform using SmartUI Selenium SDK integration. - -## Create a SmartUI Project - -The first step is to create a project with the application in which we will combine all your builds run on the project. To create a SmartUI Project, follow these steps: - -1. Go to [Projects page](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) -2. Click on the `new project` button -3. Select the platform as CLI for executing your `SDK` tests. -4. Add name of the project, approvers for the changes found, tags for any filter or easy navigation. -5. Click on the **Submit**. - -## Steps to run your first test - -Once you have created a SmartUI Project, you can generate screenshots by running automation scripts. Follow the below steps to successfully generate screenshots - -### **Step 1:** Create/Update your test - -You can clone the sample repository to run `LambdaTest` automation tests with `SmartUI` and use `SmartUISDKCloud.java` file located in the `src/test/java/com/lambdatest/sdk` directory. - -```bash -git clone https://github.com/LambdaTest/smartui-java-testng-sample -``` - -### **Step 2**: Update the Dependencies - -- Add the following dependencies in your `pom.xml` file - -```xml - -io.github.lambdatest -lambdatest-java-sdk -1.0.7 - -``` - -You can check the latest version of [lambdatest-java-sdk]( https://mvnrepository.com/artifact/io.github.lambdatest/lambdatest-java-sdk) and update the latest version accordingly. - -### **Step 3**: Install the Dependencies - -Install required NPM modules for `TestMu AI SmartUI Selenium SDK` in your **Frontend** project. - -```bash -npm install @lambdatest/smartui-cli -``` - -If you face any problems executing tests with SmartUI-CLI `versions >= v4.x.x`, upgrade your Node.js version to `v20.3` or above. - -```bash -mvn clean compile -``` - -### **Step 4:** Configure your Project Token - -Setup your project token shown in the **SmartUI** app after creating your project. - -```bash -export PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -``` - -```bash -set PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -``` - -```powershell -$env:PROJECT_TOKEN=123456#1234abcd-****-****-****-************" -``` - -### **Step 5:** Create and Configure SmartUI Config - -You can now configure your project configurations on using various available options to run your tests with the SmartUI integration. To generate the configuration file, please execute the following command: - -```bash -npx smartui config:create .smartui.json -``` - -Once, the configuration file will be created, you will be seeing the default configuration pre-filled in the configuration file: - -```json title="/smartui-sdk-project/.smartui.json" -{ -"web": { -"browsers": [ -"chrome", -"firefox", -"safari", -"edge" -], -"viewports": [ -[ -1920 -], -[ -1366 -], -[ -1028 -] -] // Full Page screenshots are captured by default for web viewports -}, -"mobile": { -"devices": [ -"iPhone 14", //iPhone 14 viewport -"Galaxy S24" //Galaxy S24 viewport -], -"fullPage": true, //Full Page is true by default for mobile viewports -"orientation": "portrait" //Change to "landscape" for landscape snapshot -}, -"waitForTimeout": 1000, //Optional (Should only be used in case lazy-loading/async components are present) -"waitForPageRender": 50000, //Optional (Should only be used in case of websites which take more than 30s to load) -"enableJavaScript": false, //Enable javascript for all the screenshots of the project -"allowedHostnames": [] //Additional hostnames to capture assets from -} -``` -**Advanced options in SmartUI configuration** -- For capturing fullpage or viewport screenshots, please refer to this [documentation](/docs/smartui-sdk-config-options/#12-viewports) -- For the list of available mobile viewports, please refer to this [documentation](/docs/smartui-sdk-config-options/#list-of-supported-device-viewports) -- For more information about SmartUI config global options, please refer to this [documentation](/docs/smartui-sdk-config-options/#3-global-options-optional). - -### **Step 6:** Adding SmartUI function to take screenshot - -- You can incorporate SmartUI into your custom `Selenium` automation test (any platform) script by adding the `smartuiSnapshot` function in the required segment of selenium script of which we would like to take the screenshot, as shown below: - -```java -//Importing the lambdatest-java SDK - -//Rest of your code here - -@Test -public void basicTest() throws Exception { -String spanText; -System.out.println("Loading URL"); - -driver.get(""); - -SmartUISnapshot.smartuiSnapshot(driver, ""); - -Thread.sleep(5000); -Thread.sleep(1000); -System.out.println("TestFinished"); - -} - -``` - -### **Step 7:** Execute the Tests on SmartUI Cloud - -Execute `visual regression tests` on SmartUI using the following commands - -```bash -npx smartui --config .smartui.json exec -- mvn test -D suite="sdk-cloud.xml" -``` - -You may use the `npx smartui --help` command in case you are facing issues during the execution of SmartUI commands in the CLI. - -## View SmartUI Results - -You have successfully integrated SmartUI SDK with your Selenium tests. Visit your SmartUI project to view builds and compare snapshots between different test runs. - -You can see the SmartUI dashboard to view the results. This will help you identify the Mismatches from the existing `Baseline` build and do the required visual testing. - -## Arguments supported in the `smartUISnapshot` function - -The following are the different options which are currently supported: - -| Key | Description | -| ------------------------- | ------------------------------------------------------------------------------------------------------------------------- | -| `driver` (instance) | The instance of the web driver used in your tests. | -| `"Screenshot Name"` (string) | Specify a name for the screenshot in your tests to match the same screenshot with the name from your baseline. | -| `options` (object) | Specify one or a combination of selectors in the `ignoreDOM` or `selectDOM` objects. These selectors can be based on `HTML DOM IDs, CSS classes, CSS selectors, or XPaths` used by your webpage. They define elements that should be excluded from or included in the visual comparison.| - -## Handling Dynamic Data in SmartUI SDK **** - -When conducting visual tests, you may encounter scenarios where certain elements within your application change between test runs. These changes might introduce inconsistencies in your test results.You can ignore / select specific element(s) to be removed from the comparison by parsing the options in the `smartuiSnapshot` function in the following way - -```java title="This is a sample for your configuration for Java to ignore by" ID" -List cssID = Arrays.asList(""); -Map options = new HashMap<>(); -Map> ignore = new HashMap<>(); -ignore.put("id", cssID); -options.put("ignoreDOM", ignore); - -driver.get("Required URL"); -SmartUISnapshot.smartuiSnapshot(driver, "Screenshot Name", options); -``` - -```java title="This is a sample for your configuration for Java to ignore by" Class" -List cssclass = Arrays.asList(""); -Map options = new HashMap<>(); -Map> ignore = new HashMap<>(); -ignore.put("class", cssclass); -options.put("ignoreDOM", ignore); - -driver.get("Required URL"); -SmartUISnapshot.smartuiSnapshot(driver, "Screenshot Name", options); -``` - -```java title="This is a sample for your configuration for Java to ignore by" XPath" -List path = Arrays.asList(""); -Map options = new HashMap<>(); -Map> ignore = new HashMap<>(); -ignore.put("xpath", path); -options.put("ignoreDOM", ignore); - -driver.get("Required URL"); -SmartUISnapshot.smartuiSnapshot(driver, "Screenshot Name", options); -``` - -```java title="This is a sample for your configuration for Java to ignore by CSS" Selector" -List selector = Arrays.asList(""); -Map options = new HashMap<>(); -Map> ignore = new HashMap<>(); -ignore.put("cssSelector", selector); -options.put("ignoreDOM", ignore); - -driver.get("Required URL"); -SmartUISnapshot.smartuiSnapshot(driver, "Screenshot Name", options); -``` - -```java title="This is a sample for your configuration for Java to select by" ID." -List cssID = Arrays.asList(""); -Map options = new HashMap<>(); -Map> select = new HashMap<>(); -select.put("id", cssID); -options.put("selectDOM", select); - -driver.get("Required URL"); -SmartUISnapshot.smartuiSnapshot(driver, "Screenshot Name", options); -``` - -```java title="This is a sample for your configuration for Java to select by" Class" -List cssclass = Arrays.asList(""); -Map options = new HashMap<>(); -Map> select = new HashMap<>(); -select.put("class", cssclass); -options.put("selectDOM", select); - -driver.get("Required URL"); -SmartUISnapshot.smartuiSnapshot(driver, "Screenshot Name", options); -``` - -```java title="This is a sample for your configuration for Java to select by" XPath" -List path = Arrays.asList(""); -Map options = new HashMap<>(); -Map> select = new HashMap<>(); -select.put("xpath", path); -options.put("selectDOM", select); - -driver.get("Required URL"); -SmartUISnapshot.smartuiSnapshot(driver, "Screenshot Name", options); -``` - -```java title="This is a sample for your webhook configuration for Java to select by CSS" Selector" -List selector = Arrays.asList(""); -Map options = new HashMap<>(); -Map> select = new HashMap<>(); -select.put("cssSelector", selector); -options.put("selectDOM", select); - -driver.get("Required URL"); -SmartUISnapshot.smartuiSnapshot(driver, "Screenshot Name", options); -``` - -## For capturing the screenshot of a specific element - -You can capture screenshots of targeted elements by leveraging various locator mechanisms such as XPath, CSS ID, class, and selectors. This precision-driven approach ensures accurate and specific visual regression testing for your web application's components. - -You can also pass a resolved `WebElement` object directly to the `smartuiSnapshot` function. This is especially useful when stable locators (such as IDs or XPaths) are not available, as you can locate the element using any strategy in your test code and pass the reference directly. - -```java title="This is a sample for your configuration for Javas to capture an element by" ID." -HashMap options = new HashMap<>(); -HashMap locator = new HashMap<>(); -options.put("element", locator); -locator.put("id", "Required ID"); -driver.get("Required URL"); -SmartUISnapshot.smartuiSnapshot(driver, "Screenshot Name", options); -``` - -```java title="This is a sample for your configuration for Java to capture an element by" Class" -HashMap options = new HashMap<>(); -HashMap locator = new HashMap<>(); -options.put("element", locator); -locator.put("class", "Required Class"); -driver.get("Required URL"); -SmartUISnapshot.smartuiSnapshot(driver, "Screenshot Name", options); -``` - -```java title="This is a sample for your configuration for Java to capture an element by" XPath" -HashMap options = new HashMap<>(); -HashMap locator = new HashMap<>(); -options.put("element", locator); -locator.put("xpath", "Required Xpath"); -driver.get("Required URL"); -SmartUISnapshot.smartuiSnapshot(driver, "Screenshot Name", options); -``` - -```java title="This is a sample for your configuration for Java to capture an element by CSS" Selector" -HashMap options = new HashMap<>(); -HashMap locator = new HashMap<>(); -options.put("element", locator); -locator.put("cssSelector", "Required Selector"); -driver.get("Required URL"); -SmartUISnapshot.smartuiSnapshot(driver, "Screenshot Name", options); -``` - -```java title="This is a sample for your configuration for Java to capture an element by passing a WebElement object." -WebElement element = driver.findElement(By.id("Required ID")); // locate using any strategy -HashMap options = new HashMap<>(); -options.put("element", element); -driver.get("Required URL"); -SmartUISnapshot.smartuiSnapshot(driver, "Screenshot Name", options); -``` - -## For capturing interactive lazy loading elements - -If you encounter difficulties loading interactive elements that appear on scroll in full-page screenshots, consider functionally incorporating a full-page scroll into your script before capturing the screenshot. This approach ensures the elements load first, facilitating the screenshot processing. - -```java Example for scrolling to bottom for lazy elements -//Rest of your code here - -@Test -public void basicTest() throws Exception { -System.out.println("Loading Url"); -driver.get("Required URL"); -quickScrollToBottom(); - -SmartUISnapshot.smartuiSnapshot(driver, "Screenshot Name"); -Thread.sleep(5000); // wait for 5 seconds -System.out.println("Test Finished"); -} - -public void quickScrollToBottom() throws InterruptedException { -long lastHeight = ((Number) ((JavaScriptExecutor) driver).executeScript("return document.body.scrollHeight")).longValue(); -while (true) { -((JavaScriptExecutor) driver).executeScript("window.scrollTo(0, document.body.scrollHeight);"); -Thread.sleep(2000); - -long newHeight = ((Number) ((JavaScriptExecutor) driver).executeScript("return document.body.scrollHeight")).longValue(); -if (newHeight == lastHeight) { -break; -} -lastHeight = newHeight; -} -((JavaScriptExecutor) driver).executeScript("window.scrollTo(0, 0);"); -Thread.sleep(1000); // wait for 1 second -} - -@AfterMethod -public void tearDown() { -if (driver != null) { -driver.quit(); -} -} -} -``` - -## Best Practices - -**Screenshot Naming** - -- Use descriptive, consistent names for screenshots -- Include page/component name in screenshot names -- Avoid special characters that might cause issues -- Use consistent naming conventions across your test suite - -**Example:** -```java -SmartUISnapshot.smartuiSnapshot(driver, HomePage-Header"); -SmartUISnapshot.smartuiSnapshot(driver, "ProductPage-MainContent"); -``` - -**Wait for Page Load** - -- Always wait for pages to fully load before taking screenshots -- Use explicit waits for dynamic content -- Consider using `waitForTimeout` in configuration for lazy-loaded content - -**Example:** -```java -driver.get(https://example.com"); -WebDriverWait wait = new WebDriverWait(driver, Duration.ofSeconds(10)); -wait.until(ExpectedConditions.presenceOfElementLocated(By.id("main-content"))); -SmartUISnapshot.smartuiSnapshot(driver, "Page Loaded"); -``` - -**Handle Dynamic Content** - -- Use `ignoreDOM` for elements that change between runs -- Use `selectDOM` when you only need to compare specific areas -- Document why elements are ignored for future reference - -**Configuration Management** - -- Keep `.smartui.json` in version control -- Use environment variables for sensitive data -- Document custom configuration choices - -**Test Organization** - -- Group related screenshots in the same build -- Use meaningful build names -- Run tests in consistent environments - -## Troubleshooting - -**Issue: Screenshots Not Appearing in Dashboard** - -**Symptoms**: Tests run successfully but no screenshots appear in SmartUI dashboard - -**Possible Causes**: -- Project token not set or incorrect -- Project name mismatch -- Network connectivity issues -- CLI not installed or outdated - -**Solutions**: -1. Verify `PROJECT_TOKEN` is set correctly: -```bash -echo $PROJECT_TOKEN -``` - -2. Check project name matches exactly (case-sensitive) - -3. Verify SmartUI CLI is installed: -```bash -npx smartui --version -``` - -4. Check network connectivity to TestMu AI servers - -5. Review test execution logs for error messages - -**Issue: Project Not Found" Error** - -**Symptoms**: Error message indicating project cannot be found - -**Possible Causes**: -- Incorrect project token -- Project deleted or renamed -- Token from wrong project - -**Solutions**: -1. Verify project exists in SmartUI dashboard -2. Copy project token directly from Project Settings -3. Ensure token includes the project ID prefix (e.g., `123456#...`) -4. Check for extra spaces or quotes in token - -**Issue: Screenshots Show Blank or Incorrect Content** - -**Symptoms**: Screenshots captured but show blank pages or incorrect content - -**Possible Causes**: -- Page not fully loaded -- JavaScript not executed -- Viewport size issues -- Timing issues - -**Solutions**: -1. Add explicit waits before screenshots: -```java -WebDriverWait wait = new WebDriverWait(driver, Duration.ofSeconds(10)); -wait.until(ExpectedConditions.presenceOfElementLocated(By.id(content"))); -``` - -2. Enable JavaScript in configuration: -```json -{ -"enableJavaScript": true -} -``` - -3. Increase `waitForTimeout` in configuration - -4. Verify viewport size matches expected dimensions - -**Issue: Build Execution Fails** - -**Symptoms**: `npx smartui exec` command fails - -**Possible Causes**: -- Missing or incorrect configuration file -- Invalid JSON in configuration -- Port conflicts -- Permission issues - -**Solutions**: -1. Verify `.smartui.json` exists and is valid JSON -2. Check configuration file syntax -3. Try different port if default is in use: -```bash -npx smartui exec -P 5000 -- -``` -4. Check file permissions for configuration and project files - -**Issue: Maven Dependencies Not Resolving** - -**Symptoms**: Maven cannot find `lambdatest-java-sdk` dependency - -**Possible Causes**: -- Incorrect dependency version -- Maven repository access issues -- Network connectivity problems - -**Solutions**: -1. Check latest version on [Maven Central](https://mvnrepository.com/artifact/io.github.lambdatest/lambdatest-java-sdk) -2. Update `pom.xml` with correct version -3. Clear Maven cache: -```bash -mvn clean -``` -4. Verify internet connectivity for Maven repository access - -**Issue: Screenshot Names Not Matching Baseline** - -**Symptoms**: Screenshots appear as New" instead of comparing with baseline - -**Possible Causes**: -- Screenshot name changed -- Baseline doesn't exist -- Name contains special characters - -**Solutions**: -1. Ensure screenshot names are consistent across test runs -2. Verify baseline exists in project -3. Avoid special characters in screenshot names -4. Check for case sensitivity issues - -**Getting Help** - -If you encounter issues not covered here: - -- Review the [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) for detailed solutions -- Check [SmartUI Configuration Options](/support/docs/smartui-sdk-config-options) documentation -- See [Handling Dynamic Data](/support/docs/smartui-handle-dynamic-data) for dynamic content issues -- Visit [TestMu AI Support](https://www.testmuai.com/support) for additional resources -- Contact support at support@testmuai.com or use [24/7 Chat Support](https://www.testmuai.com/support) - -## Additional Resources - -- [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) -- [SmartUI Configuration Options](/support/docs/smartui-sdk-config-options) -- [Handling Dynamic Data](/support/docs/smartui-handle-dynamic-data) -- [Handling Lazy Loading](/support/docs/smartui-handle-lazy-loading) -- [Baseline Management](/support/docs/smartui-baseline-management) -- [Running Your First Project](/support/docs/smartui-running-your-first-project) -- [SmartUI API Documentation](https://www.testmuai.com/support/api-doc/) diff --git a/static/docs/smartui-selenium-js-sdk.md b/static/docs/smartui-selenium-js-sdk.md deleted file mode 100644 index d5d3b5752..000000000 --- a/static/docs/smartui-selenium-js-sdk.md +++ /dev/null @@ -1,534 +0,0 @@ -# Integrate SmartUI SDK with Selenium-JavaScript - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Welcome to the world of simplified visual testing with the SmartUI SDK. - -Integrating seamlessly into your existing Selenium testing suite, SmartUI SDK revolutionizes the way you approach visual regression testing. Our robust solution empowers you to effortlessly capture, compare, and analyze screenshots across a multitude of browsers and resolutions, ensuring comprehensive coverage and accuracy in your visual testing endeavors. - -## Prerequisites - -- Basic understanding of Command Line Interface and Selenium is required. -- Login to [TestMu AI SmartUI](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) with your credentials. - -The following steps will guide you in running your first Visual Regression test on TestMu AI platform using SmartUI Selenium SDK integration. - -## Create a SmartUI Project - -The first step is to create a project with the application in which we will combine all your builds run on the project. To create a SmartUI Project, follow these steps: - -1. Go to [Projects page](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) -2. Click on the `new project` button -3. Select the platform as CLI for executing your `SDK` tests. -4. Add name of the project, approvers for the changes found, tags for any filter or easy navigation. -5. Click on the **Submit**. - -## Steps to run your first test - -Once you have created a SmartUI Project, you can generate screenshots by running automation scripts. Follow the below steps to successfully generate screenshots - -### **Step 1:** Create/Update your test - -You can clone the sample repository to run `LambdaTest` automation tests with `SmartUI` and use the `sdk.js` file present in the `sdk` folder. - -```bash -git clone https://github.com/LambdaTest/smartui-node-sample -cd smartui-node-sample/sdk -``` -### **Step 2**: Install the Dependencies - -Install required NPM modules for `TestMu AI SmartUI Selenium SDK` in your **Frontend** project. - -```bash -npm install @lambdatest/smartui-cli @lambdatest/selenium-driver selenium-webdriver -``` - -If you face any problems executing tests with SmartUI-CLI `versions >= v4.x.x`, upgrade your Node.js version to `v20.3` or above. - -### **Step 3:** Configure your Project Token - -Setup your project token shown in the **SmartUI** app after creating your project. - -```bash -export PROJECT_TOKEN=123456#1234abcd-****-****-****-************" -``` - -```bash -set PROJECT_TOKEN=123456#1234abcd-****-****-****-************" -``` - -```powershell -$env:PROJECT_TOKEN=123456#1234abcd-****-****-****-************" -``` - -### **Step 4:** Create and Configure SmartUI Config - -You can now configure your project configurations on using various available options to run your tests with the SmartUI integration. To generate the configuration file, please execute the following command: - -```bash -npx smartui config:create .smartui.json -``` - -Once, the configuration file will be created, you will be seeing the default configuration pre-filled in the configuration file: - -```json title="/smartui-sdk-project/.smartui.json" -{ -"web": { -"browsers": [ -"chrome", -"firefox", -"safari", -"edge" -], -"viewports": [ -[ -1920 -], -[ -1366 -], -[ -1028 -] -] // Full Page screenshots are captured by default for web viewports -}, -"mobile": { -"devices": [ -"iPhone 14", //iPhone 14 viewport -"Galaxy S24" //Galaxy S24 viewport -], -"fullPage": true, //Full Page is true by default for mobile viewports -"orientation": "portrait" //Change to "landscape" for landscape snapshot -}, -"waitForTimeout": 1000, //Optional (Should only be used in case lazy-loading/async components are present) -"waitForPageRender": 50000, //Optional (Should only be used in case of websites which take more than 30s to load) -"enableJavaScript": false, //Enable javascript for all the screenshots of the project -"allowedHostnames": [] //Additional hostnames to capture assets from -} -``` -**Advanced options in SmartUI configuration** -- For capturing fullpage or viewport screenshots, please refer to this [documentation](/docs/smartui-sdk-config-options/#12-viewports) -- For the list of available mobile viewports, please refer to this [documentation](/docs/smartui-sdk-config-options/#list-of-supported-device-viewports) -- For more information about SmartUI config global options, please refer to this [documentation](/docs/smartui-sdk-config-options/#3-global-options-optional). - -### **Step 5:** Adding SmartUI function to take screenshot - -- You can incorporate SmartUI into your custom `Selenium` automation test (any platform) script by adding the `smartuiSnapshot` function in the required segment of selenium script of which we would like to take the screenshot, as shown below: - -```js -const { Builder, By, Key, until } = require('selenium-webdriver'); -const { smartuiSnapshot } = require('@lambdatest/selenium-driver'); - -(async function example() { -let driver = await new Builder().forBrowser("chrome").build(); - -try { -await driver.get("https://www.lambdatest.com"); -await smartuiSnapshot(driver, "LambdaTest"); -await driver.get("https://www.pinterest.com/pin/112801165652823604/"); -await smartuiSnapshot(driver, "NYC"); -} finally { -await driver.quit(); -} -})(); -``` - -### **Step 6:** Execute the Tests on SmartUI Cloud - -Execute `visual regression tests` on SmartUI using the following commands - -```bash -npx smartui exec node sdkCloud.js --config .smartui.json -``` - -You may use the `npx smartui --help` command in case you are facing issues during the execution of SmartUI commands in the CLI. - -## View SmartUI Results - -You have successfully integrated SmartUI SDK with your Selenium tests. Visit your SmartUI project to view builds and compare snapshots between different test runs. - -You can see the SmartUI dashboard to view the results. This will help you identify the Mismatches from the existing `Baseline` build and do the required visual testing. - -## Arguments supported in the `smartUISnapshot` function - -The following are the different options which are currently supported: - -| Key| Description | -| -- | ----------- | -| `driver` (instance) | The instance of the web driver used in your tests. | -| `"Screenshot Name"` (string) | Specify a name for the screenshot in your tests to match the same screenshot with the name from your baseline. | -| `options` (object) | Specify one or a combination of selectors in the `ignoreDOM` or `selectDOM` objects. These selectors can be based on `HTML DOM IDs, CSS classes, CSS selectors, or XPaths` used by your webpage. They define elements that should be excluded from or included in the visual comparison.| - -## Handling Dynamic Data in SmartUI SDK **** - -When conducting visual tests, you may encounter scenarios where certain elements within your application change between test runs. These changes might introduce inconsistencies in your test results.You can ignore / select specific element(s) to be removed from the comparison by parsing the options in the `smartuiSnapshot` function in the following way - -```js title="This is a sample for your configuration for JavaScript to ignore by" ID" -let options = { -ignoreDOM: { -id: ["ID-1", "ID-2"], -} -} -await driver.get('Required URL'); -await smartuiSnapshot(driver, 'Screenshot Name', options); -``` - -```js title="This is a sample for your configuration for JavaScript to ignore by" Class" -let options = { -ignoreDOM: { -class: ["Class-1", "Class-2"], -} -} -await driver.get('Required URL'); -await smartuiSnapshot(driver, 'Screenshot Name', options); -``` - -```js title="This is a sample for your configuration for JavaScript to ignore by" XPath" -let options = { -ignoreDOM: { -xpath: ["Xpath-1", "Xpath-2"], -} -} -await driver.get('Required URL'); -await smartuiSnapshot(driver, 'Screenshot Name', options); -``` - -```js title="This is a sample for your configuration for JavaScript to ignore by CSS" Selector" -let options = { -ignoreDOM: { -cssSelector: ["CSS-Selector-1", "CSS-Selector-2"], -} -} -await driver.get('Required URL'); -await smartuiSnapshot(driver, 'Screenshot Name', options); -``` - -```js title="This is a sample for your configuration for JavaScript to select by" ID" -let options = { -selectDOM: { -id: ["ID-1", "ID-2"], -} -} -await driver.get('Required URL'); -await smartuiSnapshot(driver, 'Screenshot Name', options); -``` - -```js title="This is a sample for your configuration for JavaScript to select by" Class" -let options = { -selectDOM: { -class: ["Class-1", "Class-2"], -} -} -await driver.get('Required URL'); -await smartuiSnapshot(driver, 'Screenshot Name', options); -``` - -```js title="This is a sample for your configuration for JavaScript to select by" XPath" -let options = { -selectDOM: { -xpath: ["Xpath-1", "Xpath-2"], -} -} -await driver.get('Required URL'); -await smartuiSnapshot(driver, 'Screenshot Name', options); -``` - -```js title="This is a sample for your webhook configuration for JavaScript to select by CSS" Selector" -let options = { -selectDOM: { -cssSelector: ["CSS-Selector-1", "CSS-Selector-2"], -} -} -await driver.get('Required URL'); -await smartuiSnapshot(driver, 'Screenshot Name', options); -``` - -## For capturing the screenshot of a specific element - -You can capture screenshots of targeted elements by leveraging various locator mechanisms such as XPath, CSS ID, class, and selectors. This precision-driven approach ensures accurate and specific visual regression testing for your web application's components. - -```js title="This is a sample for your configuration for JavaScript to capture an element by" ID." -let options = { -element: { -id: 'Required ID', -} -}; -await driver.get('Required URL'); -await smartuiSnapshot(driver, 'Screenshot Name', options); -``` - -```js title="This is a sample for your configuration for JavaScript to capture an element by" Class" -let options = { -element: { -class: 'Required Class', -} -}; -await driver.get('Required URL'); -await smartuiSnapshot(driver, 'Screenshot Name', options); -``` - -```js title="This is a sample for your configuration for JavaScript to capture an element by" XPath" -let options = { -element: { -xpath: 'Required Xpath', -} -}; -await driver.get('Required URL'); -await smartuiSnapshot(driver, 'Screenshot Name', options); -``` - -```js title="This is a sample for your webhook configuration for JavaScript to capture an element by CSS" Selector" -let options = { -element: { -cssSelector: 'Required CSS Selector', -} -}; -await driver.get('Required URL'); -await smartuiSnapshot(driver, 'Screenshot Name', options); -``` - -## For capturing interactive lazy loading elements - -If you encounter difficulties loading interactive elements that appear on scroll in full-page screenshots, consider functionally incorporating a full-page scroll into your script before capturing the screenshot. This approach ensures the elements load first, facilitating the screenshot processing. - -```js Example for scrolling to bottom for lazy elements -const { Builder, By, Key, until } = require('selenium-webdriver'); -const { smartuiSnapshot } = require('@lambdatest/selenium-driver'); - -(async function example() { -let driver = await new Builder().forBrowser("chrome").build(); - -try { -await driver.get("Required URL"); -async function quickScrollToBottom(lastPageWait) { -try { -let height = await driver.executeScript("return document.body.scrollHeight"); -let heightOfPage = parseInt(height, 10); -let size = 200; -let noOfLoop = Math.floor(heightOfPage / size); - -for (let i = 1; i <= noOfLoop; i++) { -await driver.executeScript(`window.scrollTo(${(i - 1) * size}, ${i * size})`); -await new Promise(resolve => setTimeout(resolve, 1000)); -if (i === noOfLoop) { - -await driver.executeScript(`window.scrollTo(${i * size}, ${heightOfPage})`); -await new Promise(resolve => setTimeout(resolve, lastPageWait)); -} -} - -// Now scroll to the top -await driver.executeScript("window.scrollTo(0,0)"); -await new Promise(resolve => setTimeout(resolve, 10000)); -console.log("Scroll Completed"); -} catch (e) { -console.log("Got some errors" + e.toString()); -} -} -await quickScrollToBottom(100); //use wait time accordingly -await smartuiSnapshot(driver, "Screenshot Name"); -} finally { -await driver.quit(); -} -})(); - -``` - -## Best Practices - -**Screenshot Naming** - -- Use descriptive, consistent names for screenshots -- Include page/component name in screenshot names -- Avoid special characters that might cause issues -- Use consistent naming conventions across your test suite - -**Example:** -```javascript -await smartuiSnapshot(driver, HomePage-Header"); -await smartuiSnapshot(driver, "ProductPage-MainContent"); -``` - -**Wait for Page Load** - -- Always wait for pages to fully load before taking screenshots -- Use Selenium's WebDriverWait for dynamic content -- Consider using `waitForTimeout` in configuration for lazy-loaded content - -**Example:** -```javascript -const { until, By } = require('selenium-webdriver'); -await driver.get('https://example.com'); -await driver.wait(until.elementLocated(By.id('main-content')), 10000); -await smartuiSnapshot(driver, Page Loaded"); -``` - -**Handle Dynamic Content** - -- Use `ignoreDOM` for elements that change between runs -- Use `selectDOM` when you only need to compare specific areas -- Document why elements are ignored for future reference - -**Configuration Management** - -- Keep `.smartui.json` in version control -- Use environment variables for sensitive data -- Document custom configuration choices - -**Test Organization** - -- Group related screenshots in the same build -- Use meaningful build names -- Run tests in consistent environments - -## Troubleshooting - -**Issue: Screenshots Not Appearing in Dashboard** - -**Symptoms**: Tests run successfully but no screenshots appear in SmartUI dashboard - -**Possible Causes**: -- Project token not set or incorrect -- Project name mismatch -- Network connectivity issues -- CLI not installed or outdated - -**Solutions**: -1. Verify `PROJECT_TOKEN` is set correctly: -```bash -echo $PROJECT_TOKEN -``` - -2. Check project name matches exactly (case-sensitive) - -3. Verify SmartUI CLI is installed: -```bash -npx smartui --version -``` - -4. Check network connectivity to TestMu AI servers - -5. Review test execution logs for error messages - -**Issue: Project Not Found" Error** - -**Symptoms**: Error message indicating project cannot be found - -**Possible Causes**: -- Incorrect project token -- Project deleted or renamed -- Token from wrong project - -**Solutions**: -1. Verify project exists in SmartUI dashboard -2. Copy project token directly from Project Settings -3. Ensure token includes the project ID prefix (e.g., `123456#...`) -4. Check for extra spaces or quotes in token - -**Issue: Screenshots Show Blank or Incorrect Content** - -**Symptoms**: Screenshots captured but show blank pages or incorrect content - -**Possible Causes**: -- Page not fully loaded -- JavaScript not executed -- Viewport size issues -- Timing issues - -**Solutions**: -1. Add explicit waits before screenshots: -```javascript -const { until, By } = require('selenium-webdriver'); -await driver.wait(until.elementLocated(By.id('content')), 10000); -await driver.wait(until.elementIsVisible(By.css('.main-content')), 10000); -``` - -2. Enable JavaScript in configuration: -```json -{ -enableJavaScript": true -} -``` - -3. Increase `waitForTimeout` in configuration - -4. Verify viewport size matches expected dimensions - -**Issue: Build Execution Fails** - -**Symptoms**: `npx smartui exec` command fails - -**Possible Causes**: -- Missing or incorrect configuration file -- Invalid JSON in configuration -- Port conflicts -- Permission issues - -**Solutions**: -1. Verify `.smartui.json` exists and is valid JSON -2. Check configuration file syntax -3. Try different port if default is in use: -```bash -npx smartui exec -P 5000 -- -``` -4. Check file permissions for configuration and project files - -**Issue: npm Dependencies Not Resolving** - -**Symptoms**: npm cannot find `@lambdatest/selenium-driver` or `@lambdatest/smartui-cli` - -**Possible Causes**: -- Incorrect package version -- npm registry access issues -- Network connectivity problems - -**Solutions**: -1. Check latest versions on npm: -```bash -npm view @lambdatest/selenium-driver version -npm view @lambdatest/smartui-cli version -``` - -2. Clear npm cache: -```bash -npm cache clean --force -``` - -3. Verify internet connectivity for npm registry access - -4. Check package.json for version conflicts - -**Issue: Screenshot Names Not Matching Baseline** - -**Symptoms**: Screenshots appear as New" instead of comparing with baseline - -**Possible Causes**: -- Screenshot name changed -- Baseline doesn't exist -- Name contains special characters - -**Solutions**: -1. Ensure screenshot names are consistent across test runs -2. Verify baseline exists in project -3. Avoid special characters in screenshot names -4. Check for case sensitivity issues - -**Getting Help** - -If you encounter issues not covered here: - -- Review the [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) for detailed solutions -- Check [SmartUI Configuration Options](/support/docs/smartui-sdk-config-options) documentation -- See [Handling Dynamic Data](/support/docs/smartui-handle-dynamic-data) for dynamic content issues -- Visit [TestMu AI Support](https://www.testmuai.com/support) for additional resources -- Contact support at support@testmuai.com or use [24/7 Chat Support](https://www.testmuai.com/support) - -## Additional Resources - -- [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) -- [SmartUI Configuration Options](/support/docs/smartui-sdk-config-options) -- [Handling Dynamic Data](/support/docs/smartui-handle-dynamic-data) -- [Handling Lazy Loading](/support/docs/smartui-handle-lazy-loading) -- [Baseline Management](/support/docs/smartui-baseline-management) -- [Running Your First Project](/support/docs/smartui-running-your-first-project) -- [SmartUI API Documentation](https://www.testmuai.com/support/api-doc/) diff --git a/static/docs/smartui-selenium-python-sdk.md b/static/docs/smartui-selenium-python-sdk.md deleted file mode 100644 index 025c299d4..000000000 --- a/static/docs/smartui-selenium-python-sdk.md +++ /dev/null @@ -1,563 +0,0 @@ -# Integrate SmartUI SDK with Selenium-Python - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Welcome to the world of simplified visual testing with the SmartUI SDK. - -Integrating seamlessly into your existing Selenium testing suite, SmartUI SDK revolutionizes the way you approach visual regression testing. Our robust solution empowers you to effortlessly capture, compare, and analyze screenshots across a multitude of browsers and resolutions, ensuring comprehensive coverage and accuracy in your visual testing endeavors. - -## Prerequisites - -- Basic understanding of Command Line Interface and Selenium is required. -- Login to [TestMu AI SmartUI](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) with your credentials. -- Install **virtualenv** which is the recommended way to run your tests. It will isolate the build from other setups you may have running and ensure that the tests run with the specified versions of the modules. - -```bash -pip install virtualenv -``` - -The following steps will guide you in running your first Visual Regression test on TestMu AI platform using SmartUI Selenium SDK integration. - -## Create a SmartUI Project - -The first step is to create a project with the application in which we will combine all your builds run on the project. To create a SmartUI Project, follow these steps: - -1. Go to [Projects page](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) -2. Click on the `new project` button -3. Select the platform as CLI for executing your `SDK` tests. -4. Add name of the project, approvers for the changes found, tags for any filter or easy navigation. -5. Click on the **Submit**. - -## Steps to run your first test - -Once you have created a SmartUI Project, you can generate screenshots by running automation scripts. Follow the below steps to successfully generate screenshots - -### **Step 1:** Create/Update your test - -- You can clone the sample repository to run `LambdaTest` automation tests with `SmartUI` and use the `SmartUI_SDK_LT_hub.py` file. - -```bash -git clone https://github.com/LambdaTest/smartui-python-sample -cd smartui-python-sample -``` - -- Create a virtual environment in your project folder (the environment name is arbitrary). - -```bash -virtualenv venv -``` - -- Activate the environment. - -```bash -source venv/bin/activate -``` - -### **Step 2**: Install the Dependencies - -Install required NPM modules for `TestMu AI SmartUI Selenium SDK` in your **Frontend** project. - -```bash -npm install @lambdatest/smartui-cli -``` - -If you face any problems executing tests with SmartUI-CLI `versions >= v4.x.x`, upgrade your Node.js version to `v20.3` or above. - -```bash -pip install lambdatest-selenium-driver -``` - -### **Step 3:** Configure your Project Token - -Setup your project token shown in the **SmartUI** app after creating your project. - -```bash -export PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -``` - -```bash -set PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -``` - -```powershell -$env:PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -``` - -### **Step 4:** Create and Configure SmartUI Config - -You can now configure your project configurations on using various available options to run your tests with the SmartUI integration. To generate the configuration file, please execute the following command: - -```bash -npx smartui config:create .smartui.json -``` - -Once, the configuration file will be created, you will be seeing the default configuration pre-filled in the configuration file: - -```json title="/smartui-sdk-project/.smartui.json" -{ -"web": { -"browsers": [ -"chrome", -"firefox", -"safari", -"edge" -], -"viewports": [ -[ -1920 -], -[ -1366 -], -[ -1028 -] -] // Full Page screenshots are captured by default for web viewports -}, -"mobile": { -"devices": [ -"iPhone 14", //iPhone 14 viewport -"Galaxy S24" //Galaxy S24 viewport -], -"fullPage": true, //Full Page is true by default for mobile viewports -"orientation": "portrait" //Change to "landscape" for landscape snapshot -}, -"waitForTimeout": 1000, //Optional (Should only be used in case lazy-loading/async components are present) -"waitForPageRender": 50000, //Optional (Should only be used in case of websites which take more than 30s to load) -"enableJavaScript": false, //Enable javascript for all the screenshots of the project -"allowedHostnames": [] //Additional hostnames to capture assets from -} -``` -**Advanced options in SmartUI configuration** -- For capturing fullpage or viewport screenshots, please refer to this [documentation](/docs/smartui-sdk-config-options/#12-viewports) -- For the list of available mobile viewports, please refer to this [documentation](/docs/smartui-sdk-config-options/#list-of-supported-device-viewports) -- For more information about SmartUI config global options, please refer to this [documentation](/docs/smartui-sdk-config-options/#3-global-options-optional). - -### **Step 5:** Adding SmartUI function to take screenshot - -- You can incorporate SmartUI into your custom `Selenium` automation test (any platform) script by adding the `smartuiSnapshot` function in the required segment of selenium script of which we would like to take the screenshot, as shown below: - -```python -from lambdatest_selenium_driver import smartui_snapshot -from selenium import webdriver - -driver = webdriver.Chrome() -try: -driver.get('') -smartui_snapshot(driver,"") -except Exception as err: -print(err) -finally: -driver.close() -``` - -### **Step 6:** Execute the Tests on SmartUI Cloud - -Execute `visual regression tests` on SmartUI using the following commands - -```bash -npx smartui exec python SmartUI_SDK_LT_hub.py --config .smartui.json -``` - -You may use the `npx smartui --help` command in case you are facing issues during the execution of SmartUI commands in the CLI. - -## View SmartUI Results - -You have successfully integrated SmartUI SDK with your Selenium tests. Visit your SmartUI project to view builds and compare snapshots between different test runs. - -You can see the SmartUI dashboard to view the results. This will help you identify the Mismatches from the existing `Baseline` build and do the required visual testing. - -## Arguments supported in the `smartUISnapshot` function - -The following are the different options which are currently supported: - -| Key | Description | -| ------------------------- | ------------------------------------------------------------------------------------------------------------------------- | -| `driver` (instance) | The instance of the web driver used in your tests. | -| `"Screenshot Name"` (string) | Specify a name for the screenshot in your tests to match the same screenshot with the name from your baseline. | -| `options` (object) | Specify one or a combination of selectors in the `ignoreDOM` or `selectDOM` objects. These selectors can be based on `HTML DOM IDs, CSS classes, CSS selectors, or XPaths` used by your webpage. They define elements that should be excluded from or included in the visual comparison.| - -## Handling Dynamic Data in SmartUI SDK **** - -When conducting visual tests, you may encounter scenarios where certain elements within your application change between test runs. These changes might introduce inconsistencies in your test results.You can ignore / select specific element(s) to be removed from the comparison by parsing the options in the `smartuiSnapshot` function in the following way - -```rb title="This is a sample for your configuration for Python to ignore by ID" -options = { -ignoreDOM: { -id: ["ID-1", "ID-2"], -} -} -driver.get('') -smartui_snapshot(driver,"", options) -``` - -```py title="This is a sample for your configuration for Python to ignore by Class" -options = { -ignoreDOM: { -class: ["Class-1", "Class-2"], -} -} -driver.get('') -smartui_snapshot(driver,"", options) -``` - -```py title="This is a sample for your configuration for Python to ignore by XPath" -options = { -ignoreDOM: { -xpath: ["Xpath-1", "Xpath-2"], -} -} -driver.get('') -smartui_snapshot(driver,"", options) -``` - -```py title="This is a sample for your configuration for Python to ignore by CSS Selector" -options = { -ignoreDOM: { -cssSelector: ["CSS-Selector-1", "CSS-Selector-2"], -} -} -driver.get('') -smartui_snapshot(driver,"", options) -``` - -```py title="This is a sample for your configuration for Python to select by ID." -options = { -selectDOM: { -id: ["ID-1", "ID-2"], -} -} -driver.get('') -smartui_snapshot(driver,"", options) -``` - -```py title="This is a sample for your configuration for Python to select by Class" -options = { -selectDOM: { -class: ["Class-1", "Class-2"], -} -} -driver.get('') -smartui_snapshot(driver,"", options) -``` - -```py title="This is a sample for your configuration for Python to select by XPath" -options = { -selectDOM: { -xpath: ["Xpath-1", "Xpath-2"], -} -} -driver.get('') -smartui_snapshot(driver,"", options) -``` - -```py title="This is a sample for your webhook configuration for Python to select by CSS Selector" -options = { -selectDOM: { -cssSelector: ["CSS-Selector-1", "CSS-Selector-2"], -} -} -driver.get('') -smartui_snapshot(driver,"", options) -``` - -## For capturing the screenshot of a specific element - -You can capture screenshots of targeted elements by leveraging various locator mechanisms such as XPath, CSS ID, class, and selectors. This precision-driven approach ensures accurate and specific visual regression testing for your web application's components. - -```py title="This is a sample for your configuration for Python to capture an element by ID." -options = { -element: { -id: 'Required ID', -} -} -driver.get('') -smartui_snapshot(driver,"", options) -``` - -```py title="This is a sample for your configuration for Python to capture an element by Class" -options = { -element: { -class: 'Required Class', -} -} -driver.get('') -smartui_snapshot(driver,"", options) -``` - -```py title="This is a sample for your configuration for Python to capture an element by XPath" -options = { -element: { -xpath: 'Required Xpath', -} -} -driver.get('') -smartui_snapshot(driver,"", options) -``` - -```py title="This is a sample for your webhook configuration for Python to capture an element by CSS Selector" -options = { -element: { -cssSelector: 'Required CSS Selector', -} -} -driver.get('') -smartui_snapshot(driver,"", options) -``` - -## For capturing interactive lazy loading elements - -If you encounter difficulties loading interactive elements that appear on scroll in full-page screenshots, consider functionally incorporating a full-page scroll into your script before capturing the screenshot. This approach ensures the elements load first, facilitating the screenshot processing. - -```py Example for scrolling to bottom for lazy elements -from selenium import webdriver -from lambdatest_selenium_driver import smartui_snapshot - -def quick_scroll_to_bottom(driver, last_page_wait): -try: -height = driver.execute_script("return document.body.scrollHeight") -height_of_page = int(height) -size = 200 -no_of_loop = height_of_page // size - -for i in range(1, no_of_loop + 1): -driver.execute_script(f"window.scrollTo({(i - 1) * size}, {i * size})") -driver.implicitly_wait(1) -if i == no_of_loop: -driver.execute_script(f"window.scrollTo({i * size}, {height_of_page})") -driver.implicitly_wait(last_page_wait / 1000.0) - -# Now scroll to the top -driver.execute_script("window.scrollTo(0,0)") -driver.implicitly_wait(10) # Wait for 10 seconds -print("Scroll Completed") -except Exception as e: -print(f"Got some errors: {e}") - -# Example usage -if __name__ == "__main__": -driver = webdriver.Chrome() # Initialize the WebDriver instance -try: -driver.get("Required URL") # Change "Required URL" to the actual URL you want to test -quick_scroll_to_bottom(driver, 100) # Use wait time accordingly -smartui_snapshot(driver, "Screenshot Name") -finally: -driver.quit() -``` - -## Best Practices - -**Screenshot Naming** - -- Use descriptive, consistent names for screenshots -- Include page/component name in screenshot names -- Avoid special characters that might cause issues -- Use consistent naming conventions across your test suite - -**Example:** -```python -smartui_snapshot(driver, "HomePage-Header") -smartui_snapshot(driver, "ProductPage-MainContent") -``` - -**Wait for Page Load** - -- Always wait for pages to fully load before taking screenshots -- Use Selenium's WebDriverWait for dynamic content -- Consider using `waitForTimeout` in configuration for lazy-loaded content - -**Example:** -```python -from selenium.webdriver.support.ui import WebDriverWait -from selenium.webdriver.support import expected_conditions as EC -from selenium.webdriver.common.by import By - -driver.get('https://example.com') -wait = WebDriverWait(driver, 10) -wait.until(EC.presence_of_element_located((By.ID, 'main-content'))) -smartui_snapshot(driver, "Page Loaded") -``` - -**Handle Dynamic Content** - -- Use `ignoreDOM` for elements that change between runs -- Use `selectDOM` when you only need to compare specific areas -- Document why elements are ignored for future reference - -**Configuration Management** - -- Keep `.smartui.json` in version control -- Use environment variables for sensitive data -- Document custom configuration choices - -**Test Organization** - -- Group related screenshots in the same build -- Use meaningful build names -- Run tests in consistent environments - -## Troubleshooting - -**Issue: Screenshots Not Appearing in Dashboard** - -**Symptoms**: Tests run successfully but no screenshots appear in SmartUI dashboard - -**Possible Causes**: -- Project token not set or incorrect -- Project name mismatch -- Network connectivity issues -- CLI not installed or outdated - -**Solutions**: -1. Verify `PROJECT_TOKEN` is set correctly: -```bash -echo $PROJECT_TOKEN -``` - -2. Check project name matches exactly (case-sensitive) - -3. Verify SmartUI CLI is installed: -```bash -npx smartui --version -``` - -4. Check network connectivity to TestMu AI servers - -5. Review test execution logs for error messages - -**Issue: "Project Not Found" Error** - -**Symptoms**: Error message indicating project cannot be found - -**Possible Causes**: -- Incorrect project token -- Project deleted or renamed -- Token from wrong project - -**Solutions**: -1. Verify project exists in SmartUI dashboard -2. Copy project token directly from Project Settings -3. Ensure token includes the project ID prefix (e.g., `123456#...`) -4. Check for extra spaces or quotes in token - -**Issue: Screenshots Show Blank or Incorrect Content** - -**Symptoms**: Screenshots captured but show blank pages or incorrect content - -**Possible Causes**: -- Page not fully loaded -- JavaScript not executed -- Viewport size issues -- Timing issues - -**Solutions**: -1. Add explicit waits before screenshots: -```python -from selenium.webdriver.support.ui import WebDriverWait -from selenium.webdriver.support import expected_conditions as EC -from selenium.webdriver.common.by import By - -wait = WebDriverWait(driver, 10) -wait.until(EC.presence_of_element_located((By.ID, 'content'))) -wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, '.main-content'))) -``` - -2. Enable JavaScript in configuration: -```json -{ -"enableJavaScript": true -} -``` - -3. Increase `waitForTimeout` in configuration - -4. Verify viewport size matches expected dimensions - -**Issue: Build Execution Fails** - -**Symptoms**: `npx smartui exec` command fails - -**Possible Causes**: -- Missing or incorrect configuration file -- Invalid JSON in configuration -- Port conflicts -- Permission issues - -**Solutions**: -1. Verify `.smartui.json` exists and is valid JSON -2. Check configuration file syntax -3. Try different port if default is in use: -```bash -npx smartui exec -P 5000 -- -``` -4. Check file permissions for configuration and project files - -**Issue: pip Dependencies Not Resolving** - -**Symptoms**: pip cannot find `lambdatest-selenium-driver` or package installation fails - -**Possible Causes**: -- Incorrect package version -- pip registry access issues -- Network connectivity problems -- Python version mismatch - -**Solutions**: -1. Check latest version: -```bash -pip index versions lambdatest-selenium-driver -``` - -2. Clear pip cache: -```bash -pip cache purge -``` - -3. Verify internet connectivity for PyPI access - -4. Check Python version compatibility: -```bash -python --version -``` - -5. Use virtual environment: -```bash -python -m venv venv -source venv/bin/activate # On Windows: venv\Scripts\activate -pip install lambdatest-selenium-driver -``` - -**Issue: Screenshot Names Not Matching Baseline** - -**Symptoms**: Screenshots appear as "New" instead of comparing with baseline - -**Possible Causes**: -- Screenshot name changed -- Baseline doesn't exist -- Name contains special characters - -**Solutions**: -1. Ensure screenshot names are consistent across test runs -2. Verify baseline exists in project -3. Avoid special characters in screenshot names -4. Check for case sensitivity issues - -**Getting Help** - -If you encounter issues not covered here: - -- Review the [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) for detailed solutions -- Check [SmartUI Configuration Options](/support/docs/smartui-sdk-config-options) documentation -- See [Handling Dynamic Data](/support/docs/smartui-handle-dynamic-data) for dynamic content issues -- Visit [TestMu AI Support](https://www.testmuai.com/support) for additional resources -- Contact support at support@testmuai.com or use [24/7 Chat Support](https://www.testmuai.com/support) - -## Additional Resources - -- [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) -- [SmartUI Configuration Options](/support/docs/smartui-sdk-config-options) -- [Handling Dynamic Data](/support/docs/smartui-handle-dynamic-data) -- [Handling Lazy Loading](/support/docs/smartui-handle-lazy-loading) -- [Baseline Management](/support/docs/smartui-baseline-management) -- [Running Your First Project](/support/docs/smartui-running-your-first-project) -- [SmartUI API Documentation](https://www.testmuai.com/support/api-doc/) diff --git a/static/docs/smartui-selenium-ruby-sdk.md b/static/docs/smartui-selenium-ruby-sdk.md deleted file mode 100644 index 181f52300..000000000 --- a/static/docs/smartui-selenium-ruby-sdk.md +++ /dev/null @@ -1,531 +0,0 @@ -# Integrate SmartUI SDK with Selenium-Ruby - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Welcome to the world of simplified visual testing with the SmartUI SDK. - -Integrating seamlessly into your existing Selenium testing suite, SmartUI SDK revolutionizes the way you approach visual regression testing. Our robust solution empowers you to effortlessly capture, compare, and analyze screenshots across a multitude of browsers and resolutions, ensuring comprehensive coverage and accuracy in your visual testing endeavors. - -## Prerequisites - -- Basic understanding of Command Line Interface and Selenium is required. -- Login to [TestMu AI SmartUI](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) with your credentials. - -The following steps will guide you in running your first Visual Regression test on TestMu AI platform using SmartUI Selenium SDK integration. - -## Create a SmartUI Project - -The first step is to create a project with the application in which we will combine all your builds run on the project. To create a SmartUI Project, follow these steps: - -1. Go to [Projects page](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) -2. Click on the `new project` button -3. Select the platform as CLI for executing your `SDK` tests. -4. Add name of the project, approvers for the changes found, tags for any filter or easy navigation. -5. Click on the **Submit**. - -## Steps to run your first test - -Once you have created a SmartUI Project, you can generate screenshots by running automation scripts. Follow the below steps to successfully generate screenshots - -### **Step 1:** Create/Update your test - -You can clone the sample repository to run `LambdaTest` automation tests with `SmartUI` and use the `sdkCloud.rb` file present in the `sdk` folder. - -```bash -git clone https://github.com/LambdaTest/smartui-ruby-selenium-sample -cd smartui-ruby-selenium-sample/sdk -``` -### **Step 2**: Install the Dependencies - -Install required NPM modules for `TestMu AI SmartUI Selenium SDK` in your **Frontend** project. - -```bash -npm install @lambdatest/smartui-cli -``` - -If you face any problems executing tests with SmartUI-CLI `versions >= v4.x.x`, upgrade your Node.js version to `v20.3` or above. - -``` -gem install lambdatest-selenium-driver selenium-webdriver -``` - -### **Step 3:** Configure your Project Token - -Setup your project token shown in the **SmartUI** app after creating your project. - -```bash -export PROJECT_TOKEN=123456#1234abcd-****-****-****-************" -``` - -```bash -set PROJECT_TOKEN=123456#1234abcd-****-****-****-************" -``` - -```powershell -$env:PROJECT_TOKEN=123456#1234abcd-****-****-****-************" -``` - -### **Step 4:** Create and Configure SmartUI Config - -You can now configure your project configurations on using various available options to run your tests with the SmartUI integration. To generate the configuration file, please execute the following command: - -```bash -npx smartui config:create .smartui.json -``` - -Once, the configuration file will be created, you will be seeing the default configuration pre-filled in the configuration file: - -```json title="/smartui-sdk-project/.smartui.json" -{ -"web": { -"browsers": [ -"chrome", -"firefox", -"safari", -"edge" -], -"viewports": [ -[ -1920 -], -[ -1366 -], -[ -1028 -] -] // Full Page screenshots are captured by default for web viewports -}, -"mobile": { -"devices": [ -"iPhone 14", //iPhone 14 viewport -"Galaxy S24" //Galaxy S24 viewport -], -"fullPage": true, //Full Page is true by default for mobile viewports -"orientation": "portrait" //Change to "landscape" for landscape snapshot -}, -"waitForTimeout": 1000, //Optional (Should only be used in case lazy-loading/async components are present) -"waitForPageRender": 50000, //Optional (Should only be used in case of websites which take more than 30s to load) -"enableJavaScript": false, //Enable javascript for all the screenshots of the project -"allowedHostnames": [] //Additional hostnames to capture assets from -} -``` -**Advanced options in SmartUI configuration** -- For capturing fullpage or viewport screenshots, please refer to this [documentation](/docs/smartui-sdk-config-options/#12-viewports) -- For the list of available mobile viewports, please refer to this [documentation](/docs/smartui-sdk-config-options/#list-of-supported-device-viewports) -- For more information about SmartUI config global options, please refer to this [documentation](/docs/smartui-sdk-config-options/#3-global-options-optional). - -### **Step 5:** Adding SmartUI function to take screenshot - -- You can incorporate SmartUI into your custom `Selenium` automation test (any platform) script by adding the `smartuiSnapshot` function in the required segment of selenium script of which we would like to take the screenshot, as shown below: - -```ruby -require "selenium-webdriver" -require "lambdatest/selenium/driver" - -driver = Selenium::WebDriver.for :chrome -driver.navigate.to "https://www.pinterest.com/pin/16958936087791895/" - -# Take a snapshot -LambdaTest::Selenium::Driver.smartui_snapshot(driver, "Screenshot Name") - -driver.quit -``` - -### **Step 6:** Execute the Tests on SmartUI Cloud - -Execute `visual regression tests` on SmartUI using the following commands - -```bash -npx smartui exec ruby sdkCloud.rb --config .smartui.json -``` - -You may use the `npx smartui --help` command in case you are facing issues during the execution of SmartUI commands in the CLI. - -## View SmartUI Results - -You have successfully integrated SmartUI SDK with your Selenium tests. Visit your SmartUI project to view builds and compare snapshots between different test runs. - -You can see the SmartUI dashboard to view the results. This will help you identify the Mismatches from the existing `Baseline` build and do the required visual testing. - -## Arguments supported in the `smartUISnapshot` function - -The following are the different options which are currently supported: - -| Key | Description | -| ------------------------- | ------------------------------------------------------------------------------------------------------------------------- | -| `driver` (instance) | The instance of the web driver used in your tests. | -| `"Screenshot Name"` (string) | Specify a name for the screenshot in your tests to match the same screenshot with the name from your baseline. | -| `options` (object) | Specify one or a combination of selectors in the `ignoreDOM` or `selectDOM` objects. These selectors can be based on `HTML DOM IDs, CSS classes, CSS selectors, or XPaths` used by your webpage. They define elements that should be excluded from or included in the visual comparison.| - -## Handling Dynamic Data in SmartUI SDK **** - -When conducting visual tests, you may encounter scenarios where certain elements within your application change between test runs. These changes might introduce inconsistencies in your test results.You can ignore / select specific element(s) to be removed from the comparison by parsing the options in the `smartuiSnapshot` function in the following way - -```rb title="This is a sample for your configuration for Ruby to ignore by" ID" -options = { -ignoreDOM: { -id: ["ID-1", "ID-2"], -} -} -driver.navigate.to 'Required URL' -LambdaTest::Selenium::Driver.smartui_snapshot(driver, "Screenshot Name", options) -``` - -```rb title="This is a sample for your configuration for Ruby to ignore by" Class" -options = { -ignoreDOM: { -class: ["Class-1", "Class-2"], -} -} -driver.navigate.to 'Required URL' -LambdaTest::Selenium::Driver.smartui_snapshot(driver, "Screenshot Name", options) -``` - -```rb title="This is a sample for your configuration for Ruby to ignore by" XPath" -options = { -ignoreDOM: { -xpath: ["Xpath-1", "Xpath-2"], -} -} -driver.navigate.to 'Required URL' -LambdaTest::Selenium::Driver.smartui_snapshot(driver, "Screenshot Name", options) -``` - -```rb title="This is a sample for your configuration for Ruby to ignore by CSS" Selector" -options = { -ignoreDOM: { -cssSelector: ["CSS-Selector-1", "CSS-Selector-2"], -} -} -driver.navigate.to 'Required URL' -LambdaTest::Selenium::Driver.smartui_snapshot(driver, "Screenshot Name", options) -``` - -```rb title="This is a sample for your configuration for Ruby to select by" ID." -options = { -selectDOM: { -id: ["ID-1", "ID-2"], -} -} -driver.navigate.to 'Required URL' -LambdaTest::Selenium::Driver.smartui_snapshot(driver, "Screenshot Name", options) -``` - -```rb title="This is a sample for your configuration for Ruby to select by" Class" -options = { -selectDOM: { -class: ["Class-1", "Class-2"], -} -} -driver.navigate.to 'Required URL' -LambdaTest::Selenium::Driver.smartui_snapshot(driver, "Screenshot Name", options) -``` - -```rb title="This is a sample for your configuration for Ruby to select by" XPath" -options = { -selectDOM: { -xpath: ["Xpath-1", "Xpath-2"], -} -} -driver.navigate.to 'Required URL' -LambdaTest::Selenium::Driver.smartui_snapshot(driver, "Screenshot Name", options) -``` - -```rb title="This is a sample for your webhook configuration for Ruby to select by CSS" Selector" -options = { -selectDOM: { -cssSelector: ["CSS-Selector-1", "CSS-Selector-2"], -} -} -driver.navigate.to 'Required URL' -LambdaTest::Selenium::Driver.smartui_snapshot(driver, "Screenshot Name", options) -``` - -## For capturing the screenshot of a specific element - -You can capture screenshots of targeted elements by leveraging various locator mechanisms such as XPath, CSS ID, class, and selectors. This precision-driven approach ensures accurate and specific visual regression testing for your web application's components. - -```rb title="This is a sample for your configuration for Ruby to capture an element by" ID." -options = { -element: { -id: 'Required ID', -} -} -driver.navigate.to 'Required URL' -LambdaTest::Selenium::Driver.smartui_snapshot(driver, "Screenshot Name", options) -``` - -```rb title="This is a sample for your configuration for Ruby to capture an element by" Class" -options = { -element: { -class: 'Required Class', -} -} -driver.navigate.to 'Required URL' -LambdaTest::Selenium::Driver.smartui_snapshot(driver, "Screenshot Name", options) -``` - -```rb title="This is a sample for your configuration for Ruby to capture an element by" XPath" -options = { -element: { -xpath: 'Required Xpath', -} -} -driver.navigate.to 'Required URL' -LambdaTest::Selenium::Driver.smartui_snapshot(driver, "Screenshot Name", options) -``` - -```rb title="This is a sample for your webhook configuration for Ruby to capture an element by CSS" Selector" -options = { -element: { -cssSelector: 'Required CSS Selector', -} -} -driver.navigate.to 'Required URL' -LambdaTest::Selenium::Driver.smartui_snapshot(driver, "Screenshot Name", options) -``` - -## For capturing interactive lazy loading elements - -If you encounter difficulties loading interactive elements that appear on scroll in full-page screenshots, consider functionally incorporating a full-page scroll into your script before capturing the screenshot. This approach ensures the elements load first, facilitating the screenshot processing. - -```rb Example for scrolling to bottom for lazy elements -require 'selenium-webdriver' -require 'lambdatest/selenium/driver' - -def quick_scroll_to_bottom(driver, last_page_wait) -height = driver.execute_script("return document.body.scrollHeight") -height_of_page = height.to_i -size = 200 -no_of_loop = height_of_page / size - -(1..no_of_loop).each do |i| -driver.execute_script("window.scrollTo(#{(i - 1) * size}, #{i * size})") -sleep 1 -if i == no_of_loop -driver.execute_script("window.scrollTo(#{i * size}, #{height_of_page})") -sleep last_page_wait / 1000.0 -end -end - -# Now scroll to the top -driver.execute_script("window.scrollTo(0,0)") -sleep 10 # Wait for 10 seconds -puts "Scroll Completed" -rescue => e -puts "Got some errors: #{e}" -end - -driver = Selenium::WebDriver.for :chrome -begin -driver.get("Required URL") -quick_scroll_to_bottom(driver, 100) # Use wait time accordingly -LambdaTest::Selenium::Driver.smartui_snapshot(driver, "Screenshot Name") -ensure -driver.quit -end - -``` - -## Best Practices - -**Screenshot Naming** - -- Use descriptive, consistent names for screenshots -- Include page/component name in screenshot names -- Avoid special characters that might cause issues -- Use consistent naming conventions across your test suite - -**Example:** -```ruby -LambdaTest::Selenium::Driver.smartui_snapshot(driver, HomePage-Header") -LambdaTest::Selenium::Driver.smartui_snapshot(driver, "ProductPage-MainContent") -``` - -**Wait for Page Load** - -- Always wait for pages to fully load before taking screenshots -- Use Selenium's wait methods for dynamic content -- Consider using `waitForTimeout` in configuration for lazy-loaded content - -**Example:** -```ruby -require 'selenium-webdriver' - -driver.navigate.to 'https://example.com' -wait = Selenium::WebDriver::Wait.new(timeout: 10) -wait.until { driver.find_element(id: 'main-content') } -LambdaTest::Selenium::Driver.smartui_snapshot(driver, Page Loaded") -``` - -**Handle Dynamic Content** - -- Use `ignoreDOM` for elements that change between runs -- Use `selectDOM` when you only need to compare specific areas -- Document why elements are ignored for future reference - -**Configuration Management** - -- Keep `.smartui.json` in version control -- Use environment variables for sensitive data -- Document custom configuration choices - -**Test Organization** - -- Group related screenshots in the same build -- Use meaningful build names -- Run tests in consistent environments - -**Troubleshooting** - -**Issue: Screenshots Not Appearing in Dashboard** - -**Symptoms**: Tests run successfully but no screenshots appear in SmartUI dashboard - -**Possible Causes**: -- Project token not set or incorrect -- Project name mismatch -- Network connectivity issues -- CLI not installed or outdated - -**Solutions**: -1. Verify `PROJECT_TOKEN` is set correctly: -```bash -echo $PROJECT_TOKEN -``` - -2. Check project name matches exactly (case-sensitive) - -3. Verify SmartUI CLI is installed: -```bash -npx smartui --version -``` - -4. Check network connectivity to TestMu AI servers - -5. Review test execution logs for error messages - -**Issue: Project Not Found" Error** - -**Symptoms**: Error message indicating project cannot be found - -**Possible Causes**: -- Incorrect project token -- Project deleted or renamed -- Token from wrong project - -**Solutions**: -1. Verify project exists in SmartUI dashboard -2. Copy project token directly from Project Settings -3. Ensure token includes the project ID prefix (e.g., `123456#...`) -4. Check for extra spaces or quotes in token - -**Issue: Screenshots Show Blank or Incorrect Content** - -**Symptoms**: Screenshots captured but show blank pages or incorrect content - -**Possible Causes**: -- Page not fully loaded -- JavaScript not executed -- Viewport size issues -- Timing issues - -**Solutions**: -1. Add explicit waits before screenshots: -```ruby -wait = Selenium::WebDriver::Wait.new(timeout: 10) -wait.until { driver.find_element(id: 'content') } -wait.until { driver.find_element(css: '.main-content').displayed? } -``` - -2. Enable JavaScript in configuration: -```json -{ -enableJavaScript": true -} -``` - -3. Increase `waitForTimeout` in configuration - -4. Verify viewport size matches expected dimensions - -**Issue: Build Execution Fails** - -**Symptoms**: `npx smartui exec` command fails - -**Possible Causes**: -- Missing or incorrect configuration file -- Invalid JSON in configuration -- Port conflicts -- Permission issues - -**Solutions**: -1. Verify `.smartui.json` exists and is valid JSON -2. Check configuration file syntax -3. Try different port if default is in use: -```bash -npx smartui exec -P 5000 -- -``` -4. Check file permissions for configuration and project files - -**Issue: Gem Dependencies Not Resolving** - -**Symptoms**: Bundler cannot find `lambdatest-selenium-driver` gem - -**Possible Causes**: -- Incorrect gem version -- RubyGems registry access issues -- Network connectivity problems -- Ruby version mismatch - -**Solutions**: -1. Check latest version on [RubyGems.org](https://rubygems.org/gems/lambdatest-selenium-driver) -2. Clear gem cache: -```bash -gem cleanup -``` -3. Verify internet connectivity for RubyGems access -4. Check Ruby version compatibility: -```bash -ruby --version -``` -5. Update Gemfile with specific version: -```ruby -gem 'lambdatest-selenium-driver', '~"> 1.0' -``` - -**Issue: Screenshot Names Not Matching Baseline** - -**Symptoms**: Screenshots appear as New" instead of comparing with baseline - -**Possible Causes**: -- Screenshot name changed -- Baseline doesn't exist -- Name contains special characters - -**Solutions**: -1. Ensure screenshot names are consistent across test runs -2. Verify baseline exists in project -3. Avoid special characters in screenshot names -4. Check for case sensitivity issues - -**Getting Help** - -If you encounter issues not covered here: - -- Review the [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) for detailed solutions -- Check [SmartUI Configuration Options](/support/docs/smartui-sdk-config-options) documentation -- See [Handling Dynamic Data](/support/docs/smartui-handle-dynamic-data) for dynamic content issues -- Visit [TestMu AI Support](https://www.testmuai.com/support) for additional resources -- Contact support at support@testmuai.com or use [24/7 Chat Support](https://www.testmuai.com/support) - -## Additional Resources - -- [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) -- [SmartUI Configuration Options](/support/docs/smartui-sdk-config-options) -- [Handling Dynamic Data](/support/docs/smartui-handle-dynamic-data) -- [Handling Lazy Loading](/support/docs/smartui-handle-lazy-loading) -- [Baseline Management](/support/docs/smartui-baseline-management) -- [Running Your First Project](/support/docs/smartui-running-your-first-project) -- [SmartUI API Documentation](https://www.testmuai.com/support/api-doc/) diff --git a/static/docs/smartui-shadow-dom.md b/static/docs/smartui-shadow-dom.md deleted file mode 100644 index de6a180f6..000000000 --- a/static/docs/smartui-shadow-dom.md +++ /dev/null @@ -1,18 +0,0 @@ -# Shadow DOM - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The Shadow DOM feature in SmartUI allows for testing and capturing assets from content encapsulated within web components. Web components often use Shadow DOM to encapsulate their internal structure and styling, isolating it from the main document. As a result, traditional DOM queries and asset capturing methods might not access or interact with content inside the shadow tree. - -SmartUI's Shadow DOM support ensures that assets and UI components rendered within shadow roots are correctly captured and tested, making it easier to handle and test web applications built using web components that leverage the Shadow DOM. - -## Purpose of the Shadow DOM Feature -In typical UI tests, SmartUI captures assets and evaluates elements directly from the DOM of the page under test. However, when content is encapsulated inside a Shadow DOM (within a web component), it becomes isolated, and the test framework might not be able to access or interact with it. - -SmartUI's Shadow DOM feature addresses this limitation by: - -- Enabling SmartUI to traverse and interact with shadow trees. -- Allowing tests to capture assets from the shadow DOM. -- Ensuring that resources (e.g., styles, images, scripts) inside shadow roots are included in the testing process. - -This feature is critical for comprehensive testing of modern web applications that make use of custom elements and web components, which are often built with Shadow DOM for encapsulation. diff --git a/static/docs/smartui-slack-integration.md b/static/docs/smartui-slack-integration.md deleted file mode 100644 index 90300dcc5..000000000 --- a/static/docs/smartui-slack-integration.md +++ /dev/null @@ -1,45 +0,0 @@ -# Slack Integration with SmartUI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This document will guide you to integrate your SmartUI jobs and Slack, allowing you to receive real-time notifications and crucial job details directly within your Slack workspace. - -## Prerequisite - -- An active TestMu AI account with Admin or User-level access. -- Set up a dedicated Slack channel where you want to receive notifications from SmartUI. -- Should have setup the SmartUI suite. - -## Step 1: Navigate to the Integration Page - -- Login to your TestMu AI Account. -- Navigate to the **Settings** > **Integration** page. -- Select the **Communication** tab and search for Slack. - -## Step 2: Integrate the Slack with your TestMu AI Account - -- Click on the **Connect** button and then **Install** for the Slack integration. - -> **NOTE :** If you are already logged into Slack, you'll be redirected to a page where you have to post to a channel to confirm your identity or else you will be asked to provide Slack URL of your workspace. - -- Select the channel you want to post on. Click on **Allow** button. - -A notification would be shared on to all the members belonging to that channel, informing about your integration. - -## Step 3: Update the Notification Settings and Trigger the Job - -Once you integrate Slack, you need to configure notification settings to get test automation notifications on your integrated Slack channel. - -- Go to your desired Project. -- Click on the **Project Settings** button. -- In the **Notifications** tab, toggle the slack alerts to true -- Click on the **Update Settings** button. -- Now again trigger the project with the changes and visit the Slack channel to view the notification containing meta information about the build. - -- Once you save that configuration, you will start receiving the custom notifications in Slack! - -## Remove the Slack Integration - -- Go to the **Settings** > **Integration** page. -- Select the **Communication** tab and search for Slack. -- Click on the **Remove** button. diff --git a/static/docs/smartui-smart-comments.md b/static/docs/smartui-smart-comments.md deleted file mode 100644 index bde7c86f0..000000000 --- a/static/docs/smartui-smart-comments.md +++ /dev/null @@ -1,407 +0,0 @@ -# SmartUI Smart Comments - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Smart Comments enables real-time collaboration on SmartUI visual test results. Add comments directly on screenshots and builds, mention teammates, and manage threaded discussions, all without leaving the SmartUI dashboard. - -**Key Benefits:** -- **Contextual Discussions**: Comment directly on specific screenshots and builds -- **Team Collaboration**: Mention teammates using @mentions -- **Threaded Conversations**: Reply to comments to create discussion threads -- **Comment Management**: Mark comments as resolved or keep them active -- **Draft Comments**: Auto-saved drafts persist across the session -- **Comment History**: All comments are retained and searchable - -> Need help with other SmartUI features? Jump to the [SmartUI Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide/). - -## Where Comments Appear - -Comments can be added and viewed in multiple locations within SmartUI: - -### 1. Screenshot Level - -Comments appear on individual screenshots with a comment icon indicator showing the total number of comments. - -**Comment indicators:** -- Comment icon with count in the screenshot header -- Comment count badge showing total comments for that screenshot - -### 2. Build Level - -Comments can be viewed at the build level, showing the total count of comments across all screenshots in the build. - -### 3. Comment Drawer - -Access all comments via the **Comment** button in the Actions tab. The comment drawer displays all comments for the current build or screenshot. - -## How to Add Comments - -You can add comments from multiple locations in SmartUI: - -### 1. Screenshot Comment Box - -Add comments directly on a screenshot using the comment textbox in the screenshot drawer. - -**Steps:** -1. Open a screenshot from the build view -2. Locate the comment textbox in the screenshot drawer -3. Type your comment (max 500 characters) -4. Click **Post** to publish your comment - -### 2. Accept/Reject Modals - -Add optional comments when approving or rejecting screenshots. - -#### Accept Modal - -When accepting a screenshot, you can add an optional comment: - -**Modal options:** -- **Accept**: Approve only that specific viewport and browser -- **Accept All**: Approve in all viewports and browsers of the screenshot - -#### Accept All Modal - -Add comments when accepting all variants: - -#### Reject Modal - -Add comments when rejecting screenshots: - -**Modal options:** -- **Reject**: Reject only that specific viewport and browser -- **Reject All**: Reject in all viewports and browsers of the screenshot - -### 3. Move to Baseline Modal - -Add comments when moving screenshots to baseline: - -**Modal options:** -- **Move**: Move only that specific viewport and browser -- **Move All**: Move in all viewports and browsers of the screenshot - -### 4. Bulk Operations Comment Box - -Add comments when performing bulk operations (multiselect): - -**Steps:** -1. Select multiple screenshots using [multiselect](/support/docs/smartui-multiselect-bulkops/) -2. Choose your bulk action (Approve, Reject, Move to Baseline) -3. Add an optional comment in the comment box -4. Confirm the action - -## Comment Features - -### Draft Comments (Auto-Save) - -Smart Comments automatically saves your comment drafts for the active session. If you start typing a comment but close the modal or navigate away, your draft will be preserved. - -**How it works:** -- Drafts are saved automatically as you type -- Drafts persist across modals and pages during the session -- You can retrieve and post your draft from any comment box - -**Example scenarios:** - -**1. Draft in Accept Modal:** - -If you write a comment in the Accept modal but close it, the draft is saved. - -**2. Draft in Reject Modal:** - -**3. Draft in Multiselect:** - -You can then post the saved draft from: -- The comment textbox in the screenshot drawer -- Any modal (Accept, Reject, Move) -- The bulk operations comment box - -> **Note:** Drafts are session-based and will be cleared when you refresh the page or start a new session. - -### Mentions (@mentions) - -Mention teammates in your comments to notify them and bring them into the discussion. - -**How to mention:** -1. Type `@` in the comment box -2. Start typing the teammate's name -3. Select from the dropdown list -4. The mentioned user will be highlighted in the comment - -### Threaded Comments - -Reply to existing comments to create threaded discussions. - -**How to reply:** -1. Click **Reply** on any existing comment -2. Type your reply in the textbox -3. Click **Post** to add your reply to the thread - -Threaded comments help maintain context and organize discussions around specific topics. - -### Comment States - -Comments can be in two states: - -#### Unresolved (Active) - -Unresolved comments are active and visible in the main comments list. - -**Characteristics:** -- Visible in the main comments view -- Require attention or follow-up -- Can be marked as resolved when addressed - -#### Resolved (Archived) - -Resolved comments are archived and moved to a separate column. - -**Characteristics:** -- Moved to the "Resolved" column -- Still accessible for reference -- Can be reopened if needed - -**How to resolve:** -1. Open the comment drawer -2. Find the comment you want to resolve -3. Click **Mark as Resolved** or **Mark as Done** -4. The comment moves to the Resolved column - -**How to reopen:** -1. Navigate to the Resolved column -2. Find the comment you want to reopen -3. Click **Reopen** or **Mark as Unresolved** -4. The comment moves back to the Active column - -## Comment Management - -### Viewing Comments - -#### Screenshot Drawer - -View comments directly in the screenshot drawer: - -#### Comment Drawer (Actions Tab) - -Access all comments via the **Comment** button in the Actions tab: - -1. Open a build -2. Click the **Actions** tab -3. Click the **Comment** button -4. View all comments in the comment drawer - -The comment drawer shows: -- All comments for the current build -- Screenshot-level comment counts -- Resolved and unresolved comments in separate columns - -### Comment Count Indicators - -Comment counts are displayed at multiple levels: - -| Location | Indicator | Description | -|----------|-----------|-------------| -| **Screenshot Header** | Comment icon with number | Total comments for that specific screenshot | -| **Build Level** | Comment count badge | Total comments across all screenshots in the build | -| **Screenshot Drawer** | Comment count in drawer | Comments visible in the drawer view | - -### Comment History - -All comments are retained and searchable: -- Comments persist even after screenshots are approved or rejected -- Comment history is maintained per screenshot ID -- You can search and filter comments in the comment drawer - -## Special Cases - -### Delete Screenshot - -When a screenshot is deleted, **all associated comments are also deleted**. - -> **Important:** Deletion is permanent. Make sure to export or save important comment discussions before deleting screenshots. - -### Move to Baseline / Overwrite - -When screenshots are moved to baseline or overwritten: -- **Comments are not transferred** to the new baseline screenshot -- Comments remain with the original screenshot ID -- If the original screenshot is replaced, comments are deleted - -> **Note:** Comments are mapped to Screenshot IDs. When a screenshot is overwritten, the new screenshot has a different ID, so comments don't carry over. - -### Screenshot Variants - -Comments are specific to each screenshot variant (viewport + browser combination): -- Comments on one variant don't appear on other variants -- Each variant maintains its own comment thread -- Use "Accept All" or "Reject All" with comments to apply the same comment across all variants - -## Best Practices - -**Use Mentions Strategically** - -- Mention relevant team members when their input is needed -- Use @mentions to bring attention to critical visual issues -- Avoid over-mentioning: only tag people who need to be involved -- Combine mentions with clear action items in your comments - -**Organize with Threads** - -- Reply to existing comments to maintain context -- Create separate comment threads for different topics -- Use threads to track discussions on specific visual changes -- Keep threads focused on one issue or question - -**Resolve When Complete** - -- Mark comments as resolved when issues are addressed -- Keep the active comments list clean and focused -- Reopen resolved comments if follow-up is needed -- Use resolved comments as a reference for similar issues - -**Add Context in Comments** - -- Include specific details about what you're commenting on -- Reference the viewport, browser, or specific UI element -- Explain why a change is acceptable or needs revision -- Link to related tickets or documentation when relevant - -**Use Bulk Operation Comments** - -- Add comments when performing bulk approve/reject operations -- Explain the reasoning for bulk actions -- Use bulk comments to document batch approvals or rejections -- Combine with [multiselect](/support/docs/smartui-multiselect-bulkops/) for efficient workflows - -**Manage Drafts** - -- Review your draft before posting from a different location -- Use drafts to compose longer comments across multiple steps -- Clear drafts if you change your mind (refresh the page) -- Remember that drafts are session-based only - -## Troubleshooting - -**Issue: Comment Not Appearing** - -**Symptoms**: Comment was posted but doesn't show up in the comments list. - -**Solutions**: -1. Refresh the page to reload comments -2. Check if you're viewing the correct screenshot or build -3. Verify the comment wasn't filtered out (check Resolved column) -4. Ensure you have proper permissions to view comments -5. Check the comment drawer in the Actions tab - -**Issue: Draft Not Saving** - -**Symptoms**: Draft comment disappears after closing modal or navigating. - -**Solutions**: -1. Drafts are session-based: ensure you haven't refreshed the page -2. Drafts persist only during the active browser session -3. Try posting the draft immediately if you need to preserve it -4. Check browser console for any JavaScript errors - -**Issue: @Mention Not Working** - -**Symptoms**: Can't find teammates when typing @mention. - -**Solutions**: -1. Ensure teammates are added to the project as approvers or members -2. Type the full name or email address -3. Wait for the dropdown to populate -4. Check your network connection -5. Verify you have permission to mention users - -**Issue: Comment Too Long** - -**Symptoms**: Can't post comment or character count exceeds limit. - -**Solutions**: -1. Maximum comment length is **500 characters** -2. Break long comments into multiple shorter comments -3. Use threaded replies to continue the discussion -4. Check the character counter in the comment box - -**Issue: Comments Deleted Unexpectedly** - -**Symptoms**: Comments disappeared after an action. - -**Solutions**: -1. **Screenshot deleted**: Comments are deleted when screenshots are deleted -2. **Move to baseline**: Comments don't transfer to new baseline screenshots -3. **Overwrite**: Comments are deleted when screenshots are overwritten -4. Check if comments were moved to the Resolved column -5. Verify you're viewing the correct build or screenshot - -**Issue: Can't Resolve or Unresolve Comments** - -**Symptoms**: Resolve button doesn't work or comments don't move. - -**Solutions**: -1. Refresh the page and try again -2. Check if you have permission to resolve comments -3. Verify the comment drawer is fully loaded -4. Try clicking the comment directly to open the resolve option -5. Check browser console for errors - -## Comment Workflow Examples - -**Scenario:** Reviewing visual changes and adding comments for the team. - -1. Open a build with visual changes -2. Navigate to **Changes Found** tab -3. Open a screenshot with differences -4. Add a comment explaining the change: "This color update matches the new design system" -5. Mention the designer: "@designer-name, can you confirm this is expected?" -6. Wait for reply in the comment thread -7. Mark comment as resolved once confirmed - -**Scenario:** Approving multiple screenshots with a bulk comment. - -1. Use [multiselect](/support/docs/smartui-multiselect-bulkops/) to select multiple screenshots -2. Click **Approve** in the toolbar -3. Add a comment: "All screenshots approved after design system update" -4. Confirm the bulk approval -5. Comment appears in all selected screenshots' comment lists - -**Scenario:** Rejecting a screenshot and explaining why. - -1. Open a screenshot with unexpected changes -2. Click **Reject** button -3. Add a comment in the reject modal: "This breaks the mobile layout. Needs fix before approval." -4. Mention the developer: "@dev-name, please review the responsive breakpoints" -5. Confirm rejection -6. Comment is saved and visible in the comment drawer - -**Scenario:** Having a discussion about a visual change. - -1. QA adds initial comment: "Is this color change intentional?" -2. Designer replies: "Yes, this is part of the rebrand. Approved." -3. Developer adds: "I'll update the CSS to match the new palette." -4. QA marks the thread as resolved once the change is implemented -5. All participants can see the full discussion thread - -**Scenario:** Starting a comment in one place and finishing it elsewhere. - -1. Open Accept modal and start typing: "This looks good, but..." -2. Close the modal without posting -3. Navigate to screenshot drawer -4. Open comment textbox, your draft appears -5. Complete the comment and post it -6. Draft is cleared after posting - -## Future Enhancements - -The following features are planned for future releases: - -- **Email/Slack Notifications**: @mentions will trigger email and Slack notifications -- **RBAC (Role-Based Access Control)**: Users can edit/delete their own comments; Admins can moderate all comments -- **Cross-Build Comments**: Comments across builds for the same screenshot name (optional feature) -- **Area-Specific Comments**: Comments on specific areas of a screenshot using drawing boxes - -## Next Steps - -- Learn about [SmartUI Multiselect & Bulk Operations](/support/docs/smartui-multiselect-bulkops/) to combine comments with bulk actions -- Check the [SmartUI Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide/) for common issues diff --git a/static/docs/smartui-smart-git-strategy.md b/static/docs/smartui-smart-git-strategy.md deleted file mode 100644 index 06ea230b9..000000000 --- a/static/docs/smartui-smart-git-strategy.md +++ /dev/null @@ -1,174 +0,0 @@ -# Smart Git Strategy in SmartUI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Smart Git is an advanced feature in SmartUI that provides intelligent branch comparison and management capabilities. This guide explains how to use Smart Git effectively in your visual regression testing workflow. - -## What is Smart Git? - -Smart Git is a feature that enhances the default Git branching strategy in SmartUI by providing more flexible and intelligent branch comparison capabilities. When enabled, it automatically compares the latest build of a branch against the latest approved version of the same branch, regardless of the project's baseline settings. - -### Key Benefits - -1. **Intelligent Comparison**: Automatically compares against the latest approved version of the same branch -2. **Flexible Workflow**: Works independently of project baseline settings -3. **Easy Configuration**: Simple environment variable toggle -4. **Fallback Support**: Gracefully falls back to default behavior when disabled - -## Enabling Smart Git - -Smart Git can be enabled using an environment variable: - -```bash -export SMART_GIT=true -``` - -```bash -set SMART_GIT=true -``` - -```powershell -$env:SMART_GIT="true" -``` - -## How Smart Git Works - -### Default vs Smart Git Behavior - -#### Default Behavior (SMART_GIT=false) -- Compares builds against the project's configured baseline branch -- All branches are compared against the same baseline -- Changes must be approved in the baseline branch - -#### Smart Git Behavior (SMART_GIT=true) -- Compares builds against the latest approved version of the same branch -- Each branch maintains its own comparison history -- Changes can be approved independently in each branch -- **Automatic Approval**: New branches are automatically approved when first created, serving as their own baseline - -### Comparison Flow - -1. **Build Execution**: -```bash -# Enable Smart Git -export SMART_GIT=true - -# Run your tests -npx smartui --config .smartui.json exec -- -``` - -2. **Comparison Process**: - - SmartUI identifies the current branch - - Finds the latest approved build in the same branch - - Compares the new build against the latest approved version - - Generates a comparison report - -## Use Cases and Simulations - -### Use Case 1: Feature Branch Development - -**Scenario**: You're developing a new feature in a feature branch and want to track visual changes independently. - -```bash -# Create and switch to feature branch -git checkout -b feature/new-login - -# Enable Smart Git -export SMART_GIT=true - -# Make changes and run tests with different CLI commands -npx smartui --config .smartui.json exec -- --buildName "feature-login-tests" -npx smartui upload-figma-web designs.json --buildName "feature-login-designs" -npx smartui capture urls.json --buildName "feature-login-capture" -``` - -**Result**: -- Builds are compared against the latest approved version in `feature/new-login` -- Changes can be approved without affecting the main branch -- Independent visual regression tracking for the feature -- First build in the new branch is automatically approved as its baseline - -### Use Case 2: Parallel Development - -**Scenario**: Multiple teams working on different features in separate branches. - -```bash -# Team A: Feature branch -git checkout -b feature/team-a -export SMART_GIT=true -npx smartui --config .smartui.json exec -- - -# Team B: Different feature branch -git checkout -b feature/team-b -export SMART_GIT=true -npx smartui --config .smartui.json exec -- -``` - -**Result**: -- Each team's changes are tracked independently -- No interference between different feature branches -- Teams can approve their changes without waiting for others - -### Use Case 3: Hotfix Development - -**Scenario**: Need to make urgent fixes in a hotfix branch. - -```bash -# Create hotfix branch -git checkout -b hotfix/security-patch - -# Enable Smart Git -export SMART_GIT=true - -# Run tests -npx smartui --config .smartui.json exec -- -``` - -The Smart Git management works consistently across all SmartUI CLI commands (`exec`, `capture`, `upload`, `upload-figma-web`, `upload-figma-app`), ensuring a unified experience whether you're executing tests, capturing screenshots, uploading images, or comparing Figma designs with web pages or mobile apps. Each command supports the `--buildName` flag to provide custom names for your builds. - -**Result**: -- Quick visual regression testing for hotfix -- Independent approval process -- No impact on other branches' baselines - -## Best Practices - -**Branch Naming** - -Use clear, consistent branch naming conventions. - -**Regular Updates** - -Keep branches up to date with their parent branches. - -**Approval Process** - -Establish clear approval criteria for each branch. - -**Documentation** - -Document branch-specific visual changes. - -**Cleanup** - -Regularly clean up old branches and their builds. - -## Troubleshooting - -**Comparison Issues** - -1. **Comparison Issues**: - - Verify Smart Git is enabled: `echo $SMART_GIT` - - Check branch name: `git branch --show-current` - - Ensure previous builds exist in the branch - -**Approval Issues** - -1. **Approval Issues**: - - Verify you have necessary permissions - - Check if the build is in an approvable state - - Ensure the branch has previous approved builds - -### Getting Help - -If you encounter any issues with Smart Git, please contact our support team at support@testmuai.com. diff --git a/static/docs/smartui-smartignore.md b/static/docs/smartui-smartignore.md deleted file mode 100644 index 935d4378d..000000000 --- a/static/docs/smartui-smartignore.md +++ /dev/null @@ -1,189 +0,0 @@ -# Ignoring Displacement Differences using Smart Ignore - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -SmartUI's `Smart Ignore` engine is specifically designed to address the challenges that arise when testing dynamic and content-rich web applications. Often, when content is added or removed from a page, it causes other elements to be displaced, leading to numerous false positives in traditional pixel-by-pixel comparisons. Smart Ignore intelligently hides these displacement differences, allowing you to concentrate on actual content changes that impact the user experience. - -### Key Features of Smart Ignore - -#### 1. Displacement-Aware Comparison: -Smart Ignore distinguishes between content changes and mere content displacement. It hides differences arising from elements that have shifted position but have not otherwise changed, reducing visual noise and making it easier to spot true content differences. - -#### 2. Clutter Reduction: - -In scenarios where large areas of the page are impacted by content displacement, traditional pixel to pixel comparison would highlight these areas in a blurred and cluttered manner making it difficult to identify the actual content changes. Smart Ignore minimizes this clutter, highlighting only those areas where content has genuinely changed. In the image below you can see confusing comparison due to the `pixel to pixel` approach, which is unable to highlight the actual reasons behind the mismatch. - -#### 3. Targeted Visual Testing: - -By focusing only on the significant changes, **Smart Ignore** enhances the precision of your visual testing. This feature is particularly useful when testing pages where frequent updates or dynamic content are expected, ensuring that only the most relevant differences are brought to your attention. In the image below you can see the actual changes being highlighted as Smart Ignore understands the images and yields out the best results. - -## How to Use Smart Ignore? - -#### 1. Applying Smart Ignore via Project Settings - -In your SmartUI dashboard, navigate to your project settings. -Under "Comparison Options," switch on the `Smart Ignore` toggle across all visual tests in the project.This ensures Smart Ignore will be applied throughout the project with option to see other comparison modes as well on screenshot level according to the usecase. - -#### 2. Using Smart Ignore per Screenshot - -After running your visual test, go to the comparison page. -For each screenshot, choose the "Smart Ignore" mode from the "Diff Options" dropdown. -This allows you to selectively apply Smart Ignore to specific screenshots, making it easier to identify true differences in content. - -#### 3. Using Smart Ignore in Hooks Flow (Automation Capabilities) - -If you are using SmartUI Hooks (for example Selenium `executeScript("smartui.takeScreenshot=...")` style), enable Smart Ignore using `ignoreType` in capabilities. - -```javascript -const capabilities = { -browserName: 'Chrome', -'LT:Options': { -user: process.env.LT_USERNAME, -accessKey: process.env.LT_ACCESS_KEY, -visual: true, -'smartUI.project': 'My-Project', -ignoreType: ['smartignore'] -} -}; -``` - -> Smart Ignore is a strategy mode. Prefer `ignoreType` strategy configuration over a standalone `smartignore: true` flag. - -For full Hooks examples with Layout, Full Page, and Smart Ignore, see: -- [Hooks: Layout + Full Page + Smart Ignore](/support/docs/smartui-hooks-layout-fullpage-smartignore/) - -## Use Cases of Smart Ignore - -#### Content Management Systems - -**Scenario:** A page frequently updated with new articles or images causes other elements to shift position. - -**Solution:** Smart Ignore hides the displacement-related differences, allowing you to focus solely on the content that has been added or modified. - -#### E-commerce Platforms - -**Scenario:** Adding or removing products from a page often displaces other items, leading to a large number of false positives. - -**Solution:** Use Smart Ignore to ignore these shifts and concentrate on detecting actual changes in product listings, descriptions, or images. - -#### Large-Scale UI Changes - -**Scenario:** A major UI overhaul causes a significant portion of the page layout to change, making it difficult to identify specific content changes. - -**Solution:** Apply Smart Ignore to filter out layout shifts, making it easier to identify key differences in the content itself. - -## Limitations - -### Current Limitations - -Smart Ignore has the following current limitations: - -1. **Element Ignoring Not Supported**: Smart Ignore currently does not support ignoring specific DOM elements using `ignoreDOM` options. If you need to ignore specific elements, use the standard comparison mode with `ignoreDOM` in your test code. - -2. **Project Settings Integration**: Smart Ignore does not currently integrate with all comparison options available in Project Settings (such as pixel threshold, custom mismatch options). These settings may not apply when Smart Ignore is enabled. - -3. **Layout Comparison Mode**: Smart Ignore works differently from Layout Comparison mode. If you need to compare only layout structure, use Layout Comparison mode instead. - -### When Not to Use Smart Ignore - -Smart Ignore may not be suitable for: - -- **Precise Pixel Comparisons**: When you need exact pixel-by-pixel accuracy -- **Layout-Only Testing**: When you want to test only layout structure (use Layout Comparison instead) -- **Element-Specific Ignoring**: When you need to ignore specific elements (use `ignoreDOM` instead) -- **Very Small Changes**: When you need to detect very minor visual changes - -## Best Practices - -**Start with Project-Level Toggle** - -Enable Smart Ignore at the project level for consistent behavior across all tests. You can still switch to other comparison modes per screenshot if needed. - -**Combine with Other Features** - -Smart Ignore works well with: -- Dynamic data handling for content that changes -- Layout comparison for structure-focused testing -- Custom CSS for test-specific styling - -**Review Results Regularly** - -While Smart Ignore reduces false positives, regularly review results to ensure it's not hiding important changes. - -**Use Per-Screenshot Mode for Testing** - -When testing Smart Ignore, use per-screenshot mode to compare results with other comparison modes and find the best approach for each test case. - -**Use Per-Screenshot Mode for Testing** - -When testing Smart Ignore, use per-screenshot mode to compare results with other comparison modes and find the best approach for each test case. - -## Troubleshooting - -**Issue: Smart Ignore Not Available** - -**Symptoms**: Smart Ignore toggle or option is not visible - -**Solutions**: -1. Verify you're using a supported SmartUI plan -2. Check if your project type supports Smart Ignore -3. Contact support if the feature should be available - -**Issue: Too Many Differences Still Shown** - -**Symptoms**: Smart Ignore still shows many differences - -**Solutions**: -1. Verify Smart Ignore is actually enabled (check toggle state) -2. Some differences may be actual content changes, not displacement -3. Consider using Layout Comparison mode for layout-only testing -4. Combine with `ignoreDOM` for specific elements - -**Issue: Important Changes Hidden** - -**Symptoms**: Smart Ignore hides changes you want to see - -**Solutions**: -1. Switch to Pixel-to-Pixel mode for that specific screenshot -2. Use per-screenshot mode to see all comparison types -3. Review the baseline to ensure it's correct -4. Consider if the change is actually a displacement vs content change - -**Issue: Inconsistent Results** - -**Symptoms**: Smart Ignore shows different results for similar changes - -**Solutions**: -1. Ensure consistent baseline images -2. Check if page content is loading consistently -3. Verify Smart Ignore is enabled consistently -4. Review page structure for dynamic content issues - -## Comparison with Other Modes - -### Smart Ignore vs Pixel-to-Pixel - -| Feature | Smart Ignore | Pixel-to-Pixel | -|---------|--------------|----------------| -| Displacement Handling | ✅ Automatic | ❌ Shows all differences | -| False Positives | ✅ Reduced | ❌ Higher rate | -| Precision | Medium | High | -| Best For | Dynamic content pages | Precise comparisons | - -### Smart Ignore vs Layout Comparison - -| Feature | Smart Ignore | Layout Comparison | -|---------|--------------|-------------------| -| Content Changes | ✅ Detected | ❌ Ignored | -| Layout Changes | ✅ Detected | ✅ Detected | -| Displacement | ✅ Ignored | ✅ Ignored | -| Best For | Content + layout | Layout only | - -## Additional Resources - -- [Project Settings](/support/docs/smartui-project-settings) -- [Layout Comparison](/support/docs/smartui-layout-testing) -- [Handling Dynamic Data](/support/docs/smartui-handle-dynamic-data) -- [Baseline Management](/support/docs/smartui-baseline-management) - -> **Note**: Smart Ignore is continuously being improved. Features like element ignoring and enhanced Project Settings integration are planned for future releases. Check the [release notes](/support/docs/) for updates. diff --git a/static/docs/smartui-testcafe-sdk.md b/static/docs/smartui-testcafe-sdk.md deleted file mode 100644 index 231b2460d..000000000 --- a/static/docs/smartui-testcafe-sdk.md +++ /dev/null @@ -1,512 +0,0 @@ -# Integrate SmartUI SDK with Testcafe Tests - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Welcome to the world of simplified visual testing with the SmartUI SDK. - -Integrating seamlessly into your existing Testcafe testing suite, SmartUI SDK revolutionizes the way you approach visual regression testing. Our robust solution empowers you to effortlessly capture, compare, and analyze screenshots across a multitude of browsers and resolutions, ensuring comprehensive coverage and accuracy in your visual testing endeavors. - -## Prerequisites - -- Basic understanding of Command Line Interface and Testcafe is required. -- Login to [TestMu AI SmartUI](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) with your credentials. - -The following steps will guide you in running your first Visual Regression test on TestMu AI platform using SmartUI Testcafe SDK integration. - -## Create a SmartUI Project - -The first step is to create a project with the application in which we will combine all your builds run on the project. To create a SmartUI Project, follow these steps: - -1. Go to [Projects page](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) -2. Click on the `new project` button -3. Select the platform as CLI for executing your `SDK` tests. -4. Add name of the project, approvers for the changes found, tags for any filter or easy navigation. -5. Click on the **Submit**. - -## Steps to run your first test - -Once you have created a SmartUI Project, you can generate screenshots by running automation scripts. Follow the below steps to successfully generate screenshots - -### **Step 1:** Create/Update your test - -You can clone the sample repository to run `Testcafe` tests with `SmartUI` and use the `testcafeSDKLocal.js` file. - -```bash -git clone https://github.com/LambdaTest/smartui-testcafe-sample -cd smartui-testcafe-sample -``` -### **Step 2**: Install the Dependencies - -Install required NPM modules for `LambdaTest SmartUI Testcafe SDK` in your **Frontend** project. - -```bash -npm install @lambdatest/smartui-cli @lambdatest/testcafe-driver testcafe -``` - -If you face any problems executing tests with SmartUI-CLI `versions >= v4.x.x`, upgrade your Node.js version to `v20.3` or above. - -### **Step 3:** Configure your Project Token - -Setup your project token shown in the **SmartUI** app after creating your project. - -```bash -export PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -``` - -```bash -set PROJECT_TOKEN=123456#1234abcd-****-****-****-************" -``` - -```powershell -$env:PROJECT_TOKEN=123456#1234abcd-****-****-****-************" -``` - -### **Step 4:** Create and Configure SmartUI Config - -You can now configure your project configurations on using various available options to run your tests with the SmartUI integration. To generate the configuration file, please execute the following command: - -```bash -npx smartui config:create .smartui.json -``` - -Once, the configuration file will be created, you will be seeing the default configuration pre-filled in the configuration file: - -```json title="/smartui-sdk-project/.smartui.json" -{ -"web": { -"browsers": [ -"chrome", -"firefox", -"safari", -"edge" -], -"viewports": [ -[ -1920 -], -[ -1366 -], -[ -1028 -] -] // Full Page screenshots are captured by default for web viewports -}, -"mobile": { -"devices": [ -"iPhone 14", //iPhone 14 viewport -"Galaxy S24" //Galaxy S24 viewport -], -"fullPage": true, //Full Page is true by default for mobile viewports -"orientation": "portrait" //Change to "landscape" for landscape snapshot -}, -"waitForTimeout": 1000, //Optional (Should only be used in case lazy-loading/async components are present) -"waitForPageRender": 50000, //Optional (Should only be used in case of websites which take more than 30s to load) -"enableJavaScript": false, //Enable javascript for all the screenshots of the project -"allowedHostnames": [] //Additional hostnames to capture assets from -} -``` -**Advanced options in SmartUI configuration** -- For capturing fullpage or viewport screenshots, please refer to this [documentation](/docs/smartui-sdk-config-options/#12-viewports) -- For the list of available mobile viewports, please refer to this [documentation](/docs/smartui-sdk-config-options/#list-of-supported-device-viewports) -- For more information about SmartUI config global options, please refer to this [documentation](/docs/smartui-sdk-config-options/#3-global-options-optional). - -### **Step 5:** Adding SmartUI function to take screenshot - -- You can incorporate SmartUI into your custom `Testcafe` automation test (any platform) script by adding the `smartuiSnapshot` function in the required segment of testcafe script of which we would like to take the screenshot, as shown below: - -```js - -fixture('Amazon Test') -.page('https://www.testmuai.com'); - -test('Take Amazon Homepage Screenshot', async (t) => { -// Take a screenshot using LambdaTest's TestCafe driver -await smartuiSnapshot(t, 'LT-Homepage'); -}); - -``` - -### **Step 6:** Execute the Tests on SmartUI Cloud - -Execute `visual regression tests` on SmartUI using the following commands - -```bash -npx smartui exec testcafe chrome testcafeSDKLocal.js --config .smartui.json -``` - -You may use the `npx smartui --help` command in case you are facing issues during the execution of SmartUI commands in the CLI. - -## View SmartUI Results - -You have successfully integrated SmartUI SDK with your Testcafe tests. Visit your SmartUI project to view builds and compare snapshots between different test runs. - -You can see the SmartUI dashboard to view the results. This will help you identify the Mismatches from the existing `Baseline` build and do the required visual testing. - -## Arguments supported in the `smartUISnapshot` function - -The following are the different options which are currently supported: - -| Key | Description | -| ------------------------- | ------------------------------------------------------------------------------------------------------------------------- | -| `t` (test controller) | This object allows you to control the test execution and interact with the web page under test. | -| `"Screenshot Name"` (string) | Specify a name for the screenshot in your tests to match the same screenshot with the name from your baseline. | -| `options` (object) | Specify one or a combination of selectors in the `ignoreDOM` or `selectDOM` objects. These selectors can be based on `HTML DOM IDs, CSS classes, CSS selectors, or XPaths` used by your webpage. They define elements that should be excluded from or included in the visual comparison.| - -## Handling Dynamic Data in SmartUI SDK **** - -When conducting visual tests, you may encounter scenarios where certain elements within your application change between test runs. These changes might introduce inconsistencies in your test results.You can ignore / select specific element(s) to be removed from the comparison by parsing the options in the `smartuiSnapshot` function in the following way - -```js title="This is a sample for your configuration for Testcafe to ignore by" ID" -let options = { -ignoreDOM: { -id: ["ID-1", "ID-2"], -} -} -await smartuiSnapshot(t, 'Screenshot Name', options); -``` - -```js title="This is a sample for your configuration for Testcafe to ignore by" Class" -let options = { -ignoreDOM: { -class: ["Class-1", "Class-2"], -} -} -await smartuiSnapshot(t, 'Screenshot Name', options); -``` - -```js title="This is a sample for your configuration for Testcafe to ignore by" XPath" -let options = { -ignoreDOM: { -xpath: ["Xpath-1", "Xpath-2"], -} -} -await smartuiSnapshot(t, 'Screenshot Name', options); -``` - -```js title="This is a sample for your configuration for Testcafe to ignore by CSS" Selector" -let options = { -ignoreDOM: { -cssSelector: ["CSS-Selector-1", "CSS-Selector-2"], -} -} -await smartuiSnapshot(t, 'Screenshot Name', options); -``` - -```js title="This is a sample for your configuration for Testcafe to select by" ID." -let options = { -selectDOM: { -id: ["ID-1", "ID-2"], -} -} -await smartuiSnapshot(t, 'Screenshot Name', options); -``` - -```js title="This is a sample for your configuration for Testcafe to select by" Class" -let options = { -selectDOM: { -class: ["Class-1", "Class-2"], -} -} -await smartuiSnapshot(t, 'Screenshot Name', options); -``` - -```js title="This is a sample for your configuration for Testcafe to select by" XPath" -let options = { -selectDOM: { -xpath: ["Xpath-1", "Xpath-2"], -} -} -await smartuiSnapshot(t, 'Screenshot Name', options); -``` - -```js title="This is a sample for your webhook configuration for Testcafe to select by CSS" Selector" -let options = { -selectDOM: { -cssSelector: ["CSS-Selector-1", "CSS-Selector-2"], -} -} -await smartuiSnapshot(t, 'Screenshot Name', options); -``` - -## For capturing the screenshot of a specific element - -You can capture screenshots of targeted elements by leveraging various locator mechanisms such as XPath, CSS ID, class, and selectors. This precision-driven approach ensures accurate and specific visual regression testing for your web application's components. - -```js title="This is a sample for your configuration for Testcafe to capture an element by" ID." -let options = { -element: { -id: 'Required ID', -} -}; -await smartuiSnapshot(t, 'Screenshot Name', options); -``` - -```js title="This is a sample for your configuration for Testcafe to capture an element by" Class" -let options = { -element: { -class: 'Required Class', -} -}; -await smartuiSnapshot(t, 'Screenshot Name', options); -``` - -```js title="This is a sample for your configuration for Testcafe to capture an element by" XPath" -let options = { -element: { -xpath: 'Required Xpath', -} -}; -await smartuiSnapshot(t, 'Screenshot Name', options); -``` - -```js title="This is a sample for your webhook configuration for Testcafe to capture an element by CSS" Selector" -let options = { -element: { -cssSelector: 'Required CSS Selector', -} -}; -await smartuiSnapshot(t, 'Screenshot Name', options); -``` - -## For capturing interactive lazy loading elements - -If you encounter difficulties loading interactive elements that appear on scroll in full-page screenshots, consider functionally incorporating a full-page scroll into your script before capturing the screenshot. This approach ensures the elements load first, facilitating the screenshot processing. - -```js Example for scrolling to bottom for lazy elements - -// Function to scroll through the page and then back to the top -async function scrollToBottomAndTop(t, lastPageWait = 100) { -const getScrollHeight = Selector(() => document.body.scrollHeight); -const heightOfPage = await getScrollHeight(); -let size = 200; -let noOfLoop = Math.floor(heightOfPage / size); - -for (let i = 1; i <= noOfLoop; i++) { -await t.scrollBy({ x: 0, y: size }); -await t.wait(1000); -if (i === noOfLoop) { -await t.scrollTo({ x: 0, y: heightOfPage }); -await t.wait(lastPageWait); -} -} - -// Scroll to the top -await t.scrollTo({ x: 0, y: 0 }); -await t.wait(10000); -console.log("Scroll Completed"); -} - -fixture `Page Scroll Test` -.page `Required URL`; // Replace `Required URL` with the actual URL - -test('Scroll through the page, then take a smartUI Snapshot', async t => { -await scrollToBottomAndTop(t, 100); // Adjust wait time as needed -await smartuiSnapshot(t, 'Screenshot Name'); // Adjust the screenshot name as needed -}); - -``` - -## Best Practices - -**Screenshot Naming** - -- Use descriptive, consistent names for screenshots -- Include page/component name in screenshot names -- Avoid special characters that might cause issues -- Use consistent naming conventions across your test suite - -**Example:** -```javascript -await smartuiSnapshot(t, 'HomePage-Header'); -await smartuiSnapshot(t, 'ProductPage-MainContent'); -``` - -**Wait for Page Load** - -- Always wait for pages to fully load before taking screenshots -- Use TestCafe's built-in wait methods for dynamic content -- Consider using `waitForTimeout` in configuration for lazy-loaded content - -**Example:** -```javascript - -fixture `Page Test` -.page `https://example.com`; - -test('Take screenshot after page loads', async t ="> { -await t.expect(Selector('#main-content').exists).ok(); -await t.wait(1000); // Wait for animations -await smartuiSnapshot(t, 'Page Loaded'); -}); -``` - -**Handle Dynamic Content** - -- Use `ignoreDOM` for elements that change between runs -- Use `selectDOM` when you only need to compare specific areas -- Document why elements are ignored for future reference - -**Configuration Management** - -- Keep `.smartui.json` in version control -- Use environment variables for sensitive data -- Document custom configuration choices - -**Test Organization** - -- Group related screenshots in the same build -- Use meaningful build names -- Run tests in consistent environments - -**Troubleshooting** - -**Issue: Screenshots Not Appearing in Dashboard** - -**Symptoms**: Tests run successfully but no screenshots appear in SmartUI dashboard - -**Possible Causes**: -- Project token not set or incorrect -- Project name mismatch -- Network connectivity issues -- CLI not installed or outdated - -**Solutions**: -1. Verify `PROJECT_TOKEN` is set correctly: -```bash -echo $PROJECT_TOKEN -``` - -2. Check project name matches exactly (case-sensitive) - -3. Verify SmartUI CLI is installed: -```bash -npx smartui --version -``` - -4. Check network connectivity to TestMu AI servers - -5. Review test execution logs for error messages - -**Issue: Project Not Found" Error** - -**Symptoms**: Error message indicating project cannot be found - -**Possible Causes**: -- Incorrect project token -- Project deleted or renamed -- Token from wrong project - -**Solutions**: -1. Verify project exists in SmartUI dashboard -2. Copy project token directly from Project Settings -3. Ensure token includes the project ID prefix (e.g., `123456#...`) -4. Check for extra spaces or quotes in token - -**Issue: Screenshots Show Blank or Incorrect Content** - -**Symptoms**: Screenshots captured but show blank pages or incorrect content - -**Possible Causes**: -- Page not fully loaded -- JavaScript not executed -- Viewport size issues -- Timing issues - -**Solutions**: -1. Add explicit waits before screenshots: -```javascript -await t.expect(Selector('#content').exists).ok(); -await t.expect(Selector('.main-content').visible).ok(); -await t.wait(2000); // Wait for content to load -``` - -2. Enable JavaScript in configuration: -```json -{ -enableJavaScript": true -} -``` - -3. Increase `waitForTimeout` in configuration - -4. Verify viewport size matches expected dimensions - -**Issue: Build Execution Fails** - -**Symptoms**: `npx smartui exec` command fails - -**Possible Causes**: -- Missing or incorrect configuration file -- Invalid JSON in configuration -- Port conflicts -- Permission issues - -**Solutions**: -1. Verify `.smartui.json` exists and is valid JSON -2. Check configuration file syntax -3. Try different port if default is in use: -```bash -npx smartui exec -P 5000 -- -``` -4. Check file permissions for configuration and project files - -**Issue: npm Dependencies Not Resolving** - -**Symptoms**: npm cannot find `@lambdatest/testcafe-driver` or `@lambdatest/smartui-cli` - -**Possible Causes**: -- Incorrect package version -- npm registry access issues -- Network connectivity problems - -**Solutions**: -1. Check latest versions on npm: -```bash -npm view @lambdatest/testcafe-driver version -npm view @lambdatest/smartui-cli version -``` - -2. Clear npm cache: -```bash -npm cache clean --force -``` - -3. Verify internet connectivity for npm registry access - -4. Check package.json for version conflicts - -**Issue: Screenshot Names Not Matching Baseline** - -**Symptoms**: Screenshots appear as New" instead of comparing with baseline - -**Possible Causes**: -- Screenshot name changed -- Baseline doesn't exist -- Name contains special characters - -**Solutions**: -1. Ensure screenshot names are consistent across test runs -2. Verify baseline exists in project -3. Avoid special characters in screenshot names -4. Check for case sensitivity issues - -**Getting Help** - -If you encounter issues not covered here: - -- Review the [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) for detailed solutions -- Check [SmartUI Configuration Options](/support/docs/smartui-sdk-config-options) documentation -- See [Handling Dynamic Data](/support/docs/smartui-handle-dynamic-data) for dynamic content issues -- Visit [TestMu AI Support](https://www.testmuai.com/support) for additional resources -- Contact support at support@testmuai.com or use [24/7 Chat Support](https://www.testmuai.com/support) - -## Additional Resources - -- [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) -- [SmartUI Configuration Options](/support/docs/smartui-sdk-config-options) -- [Handling Dynamic Data](/support/docs/smartui-handle-dynamic-data) -- [Handling Lazy Loading](/support/docs/smartui-handle-lazy-loading) -- [Baseline Management](/support/docs/smartui-baseline-management) -- [Running Your First Project](/support/docs/smartui-running-your-first-project) -- [SmartUI API Documentation](https://www.testmuai.com/support/api-doc/) diff --git a/static/docs/smartui-troubleshooting-guide.md b/static/docs/smartui-troubleshooting-guide.md deleted file mode 100644 index 7afc1b18a..000000000 --- a/static/docs/smartui-troubleshooting-guide.md +++ /dev/null @@ -1,989 +0,0 @@ -# Comprehensive Troubleshooting Guide for SmartUI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This comprehensive troubleshooting guide covers common issues you may encounter when using SmartUI for visual regression testing. Each issue includes symptoms, possible causes, and step-by-step solutions. - -## Quick Diagnostic Checklist - -Before diving into specific issues, run through this quick checklist: - -- [ ] Project token is set correctly (`PROJECT_TOKEN` environment variable) -- [ ] SmartUI CLI is installed and up-to-date (`npx smartui --version`) -- [ ] Configuration file (`.smartui.json`) exists and is valid JSON -- [ ] Network connectivity to TestMu AI servers -- [ ] Project exists in SmartUI dashboard -- [ ] Credentials (for Hooks) are set correctly (`LT_USERNAME`, `LT_ACCESS_KEY`) -- [ ] Project name matches exactly (case-sensitive) - -## Troubleshooting by Category - -**Issue: Screenshots Not Appearing in Dashboard** - -**Symptoms**: -- Tests run successfully but no screenshots appear in SmartUI dashboard -- Build appears empty in dashboard -- No error messages in logs - -**Possible Causes**: -- Project token not set or incorrect -- Project name mismatch -- Network connectivity issues -- CLI not installed or outdated -- Configuration file issues - -**Solutions**: - -1. **Verify Project Token**: - -```bash -echo $PROJECT_TOKEN -``` - If empty, set it: -```bash -export PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -``` - -```bash -echo %PROJECT_TOKEN% -``` - If empty, set it: -```bash -set PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -``` - -```powershell -echo $env:PROJECT_TOKEN -``` - If empty, set it: -```powershell -$env:PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -``` - -2. **Check Project Name**: Verify project name in dashboard matches exactly (case-sensitive) - -3. **Verify SmartUI CLI Installation**: -```bash -npx smartui --version -``` - If outdated, update: -```bash -npm install -g @lambdatest/smartui-cli@latest -``` -4. **Check Network Connectivity**: -```bash -curl -I https://smartui.lambdatest.com -``` -5. **Review Test Execution Logs**: Look for error messages or warnings in test output - -**Related Documentation**: [Project Settings](/support/docs/smartui-project-settings), [Running Your First Project](/support/docs/smartui-running-your-first-project) - -### Issue: "Project Not Found" Error - -**Symptoms**: -- Error message: "Project not found" or "Invalid project token" -- Tests fail to start -- Authentication errors - -**Possible Causes**: -- Incorrect project token -- Project deleted or renamed -- Token from wrong project -- Extra spaces or quotes in token -- Token format incorrect - -**Solutions**: - -1. **Verify Project Exists**: - - Log into [SmartUI Dashboard](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) - - Check if project exists - - Verify project name matches - -2. **Copy Token Directly**: - - Go to Project Settings in dashboard - - Copy project token directly (don't type manually) - - Ensure token includes project ID prefix (e.g., `123456#...`) - -3. **Check Token Format**: - - Token should be in format: `PROJECT_ID#TOKEN_VALUE` - - No extra spaces before or after - - No quotes needed when setting environment variable - -4. **Verify Token Scope**: - - Ensure token is from the correct project - - Check if project was moved or renamed - -**Related Documentation**: [Project Settings](/support/docs/smartui-project-settings) - -### Issue: Credentials Not Working (Hooks) - -**Symptoms**: -- Tests fail with authentication errors -- "Invalid credentials" message -- Connection refused errors - -**Possible Causes**: -- `LT_USERNAME` or `LT_ACCESS_KEY` not set -- Incorrect credentials -- Credentials from wrong account -- Environment variables not exported - -**Solutions**: - -1. **Verify Credentials Are Set**: - -```bash -echo $LT_USERNAME -echo $LT_ACCESS_KEY -``` - -```bash -echo %LT_USERNAME% -echo %LT_ACCESS_KEY% -``` - -```powershell -echo $env:LT_USERNAME -echo $env:LT_ACCESS_KEY -``` - -2. **Get Correct Credentials**: - - Log into [TestMu AI Dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile) - - Click "Access Key" button - - Copy username and access key - -3. **Set Credentials Correctly**: - -```bash -export LT_USERNAME="your_username" -export LT_ACCESS_KEY="your_access_key" -``` - -```bash -set LT_USERNAME="your_username" -set LT_ACCESS_KEY="your_access_key" -``` - -```powershell -$env:LT_USERNAME="your_username" -$env:LT_ACCESS_KEY="your_access_key" -``` - -**Related Documentation**: [Selenium Visual Regression](/support/docs/selenium-visual-regression), [Playwright Visual Regression](/support/docs/playwright-visual-regression) - -### Issue: CLI Installation and Version Management - -**Symptoms**: -- "command not found: smartui" error -- CLI commands not working -- Using outdated CLI version -- Inconsistent behavior between terminals - -**Possible Causes**: -- CLI not installed globally -- CLI installed locally but not accessible -- Using outdated version -- Multiple versions installed -- PATH not configured correctly - -**Solutions**: - -1. **Understanding Global vs Local Installation**: - - **Global Installation (`-g` flag)**: - - Installs CLI system-wide, accessible from any terminal - - Command available as `smartui` directly - - Requires administrator/sudo privileges - - Best for: Single user, consistent environment -```bash -npm install -g @lambdatest/smartui-cli -``` - **Local Installation (without `-g`)**: - - Installs CLI in project's `node_modules` - - Access via `npx smartui` or `./node_modules/.bin/smartui` - - No admin privileges needed - - Best for: Project-specific versions, CI/CD pipelines -```bash -npm install @lambdatest/smartui-cli -npx smartui --version -``` -2. **Installing Latest Versions**: - - **SmartUI CLI**: -```bash -# Global installation (latest) -npm install -g @lambdatest/smartui-cli@latest - -# Local installation (latest) -npm install @lambdatest/smartui-cli@latest - -# Using npx (always uses latest) -npx @lambdatest/smartui-cli@latest --version -``` - **SmartUI Storybook CLI**: -```bash -# Global installation (latest) -npm install -g @lambdatest/smartui-storybook@latest - -# Local installation (latest) -npm install @lambdatest/smartui-storybook@latest - -# Using npx (always uses latest) -npx @lambdatest/smartui-storybook@latest --version -``` -3. **Checking Current Version**: -```bash -# If installed globally -smartui --version - -# If installed locally or using npx -npx smartui --version -``` -4. **Updating to Latest Version**: -```bash -# Global update -npm update -g @lambdatest/smartui-cli - -# Local update -npm update @lambdatest/smartui-cli - -# Force reinstall latest -npm install -g @lambdatest/smartui-cli@latest --force -``` -5. **Resolving "Command Not Found"**: - - **If installed globally**: - - Verify npm global bin is in PATH: -```bash -echo $PATH | grep -i node -npm config get prefix -``` - - Add to PATH if missing: -```bash -# macOS/Linux - Add to ~/.bashrc or ~/.zshrc -export PATH="$PATH:$(npm config get prefix)/bin" -``` - **If installed locally**: - - Always use `npx` prefix: -```bash -npx smartui --version -npx smartui exec -- -``` - - Or use direct path: -```bash -./node_modules/.bin/smartui --version -``` -6. **Best Practices**: - - **For Development**: Use local installation with `npx` for project-specific versions - - **For CI/CD**: Use `npx @lambdatest/smartui-cli@latest` to ensure latest version - - **For Global Use**: Install globally with `-g` flag for convenience - - **Version Pinning**: Use specific versions in `package.json` for reproducible builds - -**Related Documentation**: [CLI Complete Reference](/support/docs/smartui-cli-complete-reference), [Running Your First Project](/support/docs/smartui-running-your-first-project) - -**Issue: Screenshots Show Blank or Incorrect Content** - -**Symptoms**: -- Screenshots captured but show blank pages -- Partial content missing -- Incorrect viewport size -- Elements not rendered - -**Possible Causes**: -- Page not fully loaded -- JavaScript not executed -- Viewport size issues -- Timing issues -- Lazy loading not handled - -**Solutions**: - -1. **Add Explicit Waits** (Framework-specific): - - **Selenium (Java)**: -```java -WebDriverWait wait = new WebDriverWait(driver, Duration.ofSeconds(10)); -wait.until(ExpectedConditions.presenceOfElementLocated(By.id("content"))); -SmartUISnapshot.smartuiSnapshot(driver, "Screenshot Name"); -``` - **Playwright (JavaScript)**: -```javascript -await page.goto('https://example.com'); -await page.waitForSelector('#content', { state: 'visible' }); -await page.waitForLoadState('networkidle'); -await smartuiSnapshot.smartuiSnapshot(page, "Screenshot Name"); -``` - **Cypress**: -```javascript -cy.visit('https://example.com'); -cy.get('#content').should('be.visible'); -cy.wait(2000); // Wait for animations -cy.smartuiSnapshot('Screenshot Name'); -``` -2. **Enable JavaScript in Configuration**: -```json -{ -"enableJavaScript": true, -"waitForTimeout": 2000, -"waitForPageRender": 5000 -} -``` -3. **Increase Wait Timeouts**: -```json -{ -"waitForTimeout": 3000, -"waitForPageRender": 10000 -} -``` -4. **Verify Viewport Size**: - - Check viewport dimensions in configuration - - Ensure viewport matches expected page layout - - Test with different viewport sizes - -5. **Handle Lazy Loading**: - - Scroll to bottom before screenshot - - Use `waitForPageRender` option - - See [Handling Lazy Loading](/support/docs/smartui-handle-lazy-loading) for details - -**Related Documentation**: [Handling Lazy Loading](/support/docs/smartui-handle-lazy-loading), [Configuration Options](/support/docs/smartui-sdk-config-options) - -**Issue: Screenshot Names Not Matching Baseline** - -**Symptoms**: -- Screenshots appear as "New" instead of comparing with baseline -- Baseline comparison not working -- Screenshot names inconsistent - -**Possible Causes**: -- Screenshot name changed -- Baseline doesn't exist -- Name contains special characters -- Case sensitivity issues - -**Solutions**: - -1. **Ensure Consistent Naming**: - - Use same screenshot names across test runs - - Document naming convention - - Avoid dynamic names with timestamps - -2. **Verify Baseline Exists**: - - Check SmartUI dashboard for baseline build - - Ensure baseline was approved - - Verify baseline build name matches - -3. **Avoid Special Characters**: - - Use alphanumeric characters and hyphens - - Avoid spaces, special characters, or unicode - - Example: `HomePage-Header` not `HomePage Header!` - -4. **Check Case Sensitivity**: - - Screenshot names are case-sensitive - - `HomePage` ≠ `homepage` - - Use consistent casing - -5. **Use Meaningful Names**: -```javascript -// Good -cy.smartuiSnapshot('ProductPage-MainContent'); -cy.smartuiSnapshot('CheckoutPage-PaymentForm'); - -// Bad -cy.smartuiSnapshot('screenshot1'); -cy.smartuiSnapshot('test-' + Date.now()); -``` -**Related Documentation**: [Baseline Management](/support/docs/smartui-baseline-management) - -**Issue: High Mismatch Percentage or False Positives** - -**Symptoms**: -- Mismatch percentage higher than expected -- Many false positives in comparisons -- Dynamic content causing issues - -**Possible Causes**: -- Dynamic content not ignored -- Threshold settings too strict -- Rendering differences -- Baseline issues - -**Solutions**: - -1. **Use ignoreDOM for Dynamic Content**: -```javascript -let options = { -ignoreDOM: { -id: ["timestamp", "user-id", "session-token"], -class: ["dynamic-content", "ad-banner"], -cssSelector: ["[data-testid='timestamp']"] -} -}; -smartuiSnapshot(driver, "Screenshot Name", options); -``` -2. **Adjust Pixel Threshold**: - - Go to Project Settings - - Adjust Pixel Threshold (Relaxed/Recommended/Strict) - - Or set custom threshold value - -3. **Use Smart Ignore**: - - Enable Smart Ignore in project settings - - Automatically ignores layout shifts and displacements - -4. **Review Comparison Settings**: - - Check Error Highlight Color - - Adjust Custom Mismatch Acceptance - - Review Advanced Comparison Settings - -5. **Handle Dynamic Data Properly**: - - See [Handling Dynamic Data](/support/docs/smartui-handle-dynamic-data) for comprehensive guide - - Use `selectDOM` for focused comparisons - - Document ignored elements - -**Related Documentation**: [Handling Dynamic Data](/support/docs/smartui-handle-dynamic-data), [Project Settings](/support/docs/smartui-project-settings), [Smart Ignore](/support/docs/smartui-smartignore) - -**Issue: Build Execution Fails** - -**Symptoms**: -- `npx smartui exec` command fails -- Tests don't start -- Configuration errors - -**Possible Causes**: -- Missing or incorrect configuration file -- Invalid JSON in configuration -- Port conflicts -- Permission issues -- CLI version issues - -**Solutions**: - -1. **Verify Configuration File Exists**: -```bash -ls -la .smartui.json -``` - If missing, create it: -```bash -npx smartui config:create .smartui.json -``` -2. **Validate JSON Syntax**: -```bash -cat .smartui.json | python -m json.tool -``` - Or use online JSON validator - -3. **Check Port Availability**: -```bash -# Check if port 5000 is in use -lsof -i :5000 -``` - Use different port: -```bash -npx smartui exec -P 5001 -- -``` -4. **Check File Permissions**: -```bash -chmod 644 .smartui.json -``` -5. **Update CLI**: -```bash -npm install -g @lambdatest/smartui-cli@latest -``` -6. **Check CLI Help**: -```bash -npx smartui --help -npx smartui exec --help -``` -**Related Documentation**: [CLI Documentation](/support/docs/smartui-cli), [Configuration Options](/support/docs/smartui-sdk-config-options) - -**Issue: Dependencies Not Resolving** - -**Symptoms**: -- Package manager cannot find dependencies -- Installation fails -- Version conflicts - -**Possible Causes**: -- Incorrect dependency version -- Package registry access issues -- Network connectivity problems -- Node version mismatch - -**Solutions**: - -1. **Check Latest Versions**: - - **npm packages**: -```bash -npm view @lambdatest/smartui-cli version -npm view @lambdatest/playwright-driver version -``` - **Maven (Java)**: - - Check [Maven Central](https://mvnrepository.com/artifact/io.github.lambdatest/lambdatest-java-sdk) - - **pip (Python)**: -```bash -pip index versions lambdatest-selenium-driver -``` -2. **Clear Package Manager Cache**: - - **npm**: -```bash -npm cache clean --force -``` - **Maven**: -```bash -mvn clean -``` - **pip**: -```bash -pip cache purge -``` -3. **Verify Internet Connectivity**: -```bash -ping registry.npmjs.org -``` -4. **Check Node Version** (for npm): -```bash -node --version -``` - Ensure Node.js v20.3+ is installed (required for SmartUI CLI v4.x.x) - -5. **Use Package Lock Files**: - - Use `package-lock.json` for npm - - Use `requirements.txt` with pinned versions for pip - - Use `pom.xml` with specific versions for Maven - -**Related Documentation**: [SDK Documentation](/support/docs/smartui-selenium-java-sdk), [CLI Documentation](/support/docs/smartui-cli) - -**Issue: Pipeline/Workflow Fails with "Secret/Variable Not Found"** - -**Symptoms**: -- CI/CD pipeline fails -- Error about missing secrets/variables -- Authentication failures in pipeline - -**Possible Causes**: -- Secrets/variables not created -- Secret/variable names don't match -- Secrets/variables not accessible -- Scope issues - -**Solutions**: - -1. **Verify Secrets/Variables Exist**: - - **GitHub Actions**: - - Go to Settings → Secrets and variables → Actions - - Verify `LT_USERNAME`, `LT_ACCESS_KEY`, `PROJECT_TOKEN` exist - - **GitLab**: - - Go to Settings → CI/CD → Variables - - Check variables are set - - **Bitbucket**: - - Go to Repository Settings → Pipelines → Repository variables - - Verify variables exist - -2. **Check Variable Names**: - - Ensure exact match (case-sensitive) - - No extra spaces - - Correct prefix/suffix - -3. **Verify Scope**: - - Check if variables are repository, group, or instance level - - Ensure variables accessible to pipeline - - Check protected branch settings - -4. **Pass Variables to Steps**: - - **GitHub Actions**: -```yaml -env: -PROJECT_TOKEN: ${{ secrets.PROJECT_TOKEN }} -``` - **GitLab**: -```yaml -variables: -PROJECT_TOKEN: $PROJECT_TOKEN -``` -**Related Documentation**: [GitHub Actions Integration](/support/docs/smartui-with-github-actions), [GitLab Integration](/support/docs/smartui-with-gitlab), [Bitbucket Integration](/support/docs/smartui-with-bitbucket) - -**Issue: Tests Run But No Results in Dashboard (CI/CD)** - -**Symptoms**: -- Pipeline completes successfully -- No screenshots in SmartUI dashboard -- Build appears empty - -**Possible Causes**: -- Incorrect PROJECT_TOKEN -- Project name mismatch -- Network issues -- Workflow step failure - -**Solutions**: - -1. **Verify PROJECT_TOKEN**: - - Check token in SmartUI Project Settings - - Ensure token includes project ID prefix - - Verify token is set as secret/variable - -2. **Check Pipeline Logs**: - - Review CI/CD logs for errors - - Look for SmartUI CLI output - - Check for network errors - -3. **Verify Network Connectivity**: - - Ensure CI/CD runner can access TestMu AI servers - - Check firewall/proxy settings - - Verify DNS resolution - -4. **Check SmartUI CLI Step**: - - Ensure `npx smartui exec` step completed - - Verify command syntax is correct - - Check for timeout issues - -**Related Documentation**: [CI/CD Integration Guides](/support/docs/smartui-with-github-actions) - -**Issue: Configuration File Errors** - -**Symptoms**: -- "must NOT have additional properties" error -- Invalid JSON errors -- Configuration not applied - -**Possible Causes**: -- Invalid JSON syntax -- Properties in wrong location -- Unsupported options -- File encoding issues - -**Solutions**: - -1. **Validate JSON Syntax**: -```bash -cat .smartui.json | python -m json.tool -``` -2. **Check Property Placement**: - - `customCSS` must be at top level, not inside `web` - - Verify structure matches [Configuration Options](/support/docs/smartui-sdk-config-options) - -3. **Review Supported Options**: - - Check [Configuration Options](/support/docs/smartui-sdk-config-options) for valid properties - - Remove unsupported options - - Verify option names are correct - -4. **Check File Encoding**: - - Ensure file is UTF-8 encoded - - No BOM (Byte Order Mark) - - Use standard line endings - -**Related Documentation**: [Configuration Options](/support/docs/smartui-sdk-config-options), [Custom CSS](/support/docs/smartui-custom-css) - -**Issue: Environment Variables Not Persisting or Not Working** - -**Symptoms**: -- Environment variables work in one terminal but not another -- Variables reset after closing terminal -- Variables not available in VS Code terminal -- Variables not working in CI/CD pipelines -- "Variable not found" errors - -**Possible Causes**: -- Variables set only in current session -- Shell startup file not configured -- VS Code terminal not loading shell config -- Variables not exported correctly -- Different shell environments - -**Solutions**: - -1. **Making Environment Variables Persistent**: - - To keep environment variables consistent across all terminals, configure them in your shell's startup file: - -```bash -# Edit ~/.bashrc -nano ~/.bashrc - -# Add your environment variables -export PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -export LT_USERNAME="your_username" -export LT_ACCESS_KEY="your_access_key" - -# Reload the configuration -source ~/.bashrc -``` - -```bash -# Edit ~/.zshrc -nano ~/.zshrc - -# Add your environment variables -export PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -export LT_USERNAME="your_username" -export LT_ACCESS_KEY="your_access_key" - -# Reload the configuration -source ~/.zshrc -``` - -```powershell -# Check if profile exists -Test-Path $PROFILE - -# Create profile if it doesn't exist -if (!(Test-Path $PROFILE)) { -New-Item -Type File -Path $PROFILE -Force -} - -# Edit profile -notepad $PROFILE - -# Add your environment variables -$env:PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -$env:LT_USERNAME="your_username" -$env:LT_ACCESS_KEY="your_access_key" - -# Reload the profile -. $PROFILE -``` - -2. **Configuring Environment Variables in VS Code**: - - If environment variables don't work in VS Code terminals, configure them in VS Code settings: - - **Option 1: User Settings (settings.json)** - - Open VS Code settings (`Ctrl+,` or `Cmd+,`) and add: -```json -{ -"terminal.integrated.env.linux": { -"PROJECT_TOKEN": "123456#1234abcd-****-****-****-************", -"LT_USERNAME": "your_username", -"LT_ACCESS_KEY": "your_access_key" -}, -"terminal.integrated.env.osx": { -"PROJECT_TOKEN": "123456#1234abcd-****-****-****-************", -"LT_USERNAME": "your_username", -"LT_ACCESS_KEY": "your_access_key" -}, -"terminal.integrated.env.windows": { -"PROJECT_TOKEN": "123456#1234abcd-****-****-****-************", -"LT_USERNAME": "your_username", -"LT_ACCESS_KEY": "your_access_key" -} -} -``` - **Option 2: Workspace Settings (.vscode/settings.json)** - - Create or edit `.vscode/settings.json` in your project root: -```json -{ -"terminal.integrated.env.linux": { -"PROJECT_TOKEN": "123456#1234abcd-****-****-****-************" -}, -"terminal.integrated.env.osx": { -"PROJECT_TOKEN": "123456#1234abcd-****-****-****-************" -}, -"terminal.integrated.env.windows": { -"PROJECT_TOKEN": "123456#1234abcd-****-****-****-************" -} -} -``` - **Note**: Restart VS Code or reload the window after making changes. - -3. **Using .env Files**: - - For project-specific environment variables, use a `.env` file: - - **Create `.env` file in project root**: -```bash -PROJECT_TOKEN=123456#1234abcd-****-****-****-************ -LT_USERNAME=your_username -LT_ACCESS_KEY=your_access_key -``` - **Load .env file automatically**: - - **For Node.js projects**: - - Install `dotenv` package: -```bash -npm install dotenv -``` - - Load in your scripts: -```javascript -require('dotenv').config(); -``` - - Or use `dotenv-cli`: -```bash -npm install -g dotenv-cli -dotenv smartui exec -- -``` - **For Python projects**: - - Install `python-dotenv`: -```bash -pip install python-dotenv -``` - - Load in your scripts: -```python -from dotenv import load_dotenv -load_dotenv() -``` - **For VS Code**: - - Install "DotENV" extension - - Or use "Python" extension which auto-loads `.env` files - -4. **Verifying Environment Variables**: - -```bash -# Check if variable is set -echo $PROJECT_TOKEN -echo $LT_USERNAME -echo $LT_ACCESS_KEY - -# List all SmartUI-related variables -env | grep -E "PROJECT_TOKEN|LT_USERNAME|LT_ACCESS_KEY|SMARTUI" -``` - -```bash -# Check if variable is set -echo %PROJECT_TOKEN% -echo %LT_USERNAME% -echo %LT_ACCESS_KEY% - -# List all environment variables -set | findstr /i "PROJECT_TOKEN LT_USERNAME LT_ACCESS_KEY SMARTUI" -``` - -```powershell -# Check if variable is set -echo $env:PROJECT_TOKEN -echo $env:LT_USERNAME -echo $env:LT_ACCESS_KEY - -# List all SmartUI-related variables -Get-ChildItem Env: | Where-Object { $_.Name -like "*PROJECT_TOKEN*" -or $_.Name -like "*LT_*" -or $_.Name -like "*SMARTUI*" } -``` - -5. **Troubleshooting Common Issues**: - - **Issue: Variables not available after adding to startup file** - - Solution: Restart terminal or run `source ~/.bashrc` / `source ~/.zshrc` - - **Issue: VS Code terminal not picking up variables** - - Solution: Add to VS Code settings.json as shown above, then restart VS Code - - **Issue: Variables work in terminal but not in scripts** - - Solution: Ensure variables are exported (use `export` in bash/zsh) - - **Issue: Different values in different terminals** - - Solution: Use startup file configuration instead of session-only exports - - **Issue: .env file not loading** - - Solution: Verify file is in project root, check file permissions, ensure extension/package is installed - -6. **Best Practices**: - - - **Never commit sensitive data**: Add `.env` to `.gitignore` - - **Use different values per environment**: Development, staging, production - - **Document required variables**: List in README or documentation - - **Use CI/CD secrets**: Store sensitive values in CI/CD platform secrets/variables - - **Validate on startup**: Check if required variables are set before running tests - -**Related Documentation**: [CLI Environment Variables](/support/docs/smartui-cli-env-variables), [Running Your First Project](/support/docs/smartui-running-your-first-project) - -**Issue: Proxy Configuration Not Working** - -**Symptoms**: -- Tests fail behind proxy -- Connection timeouts -- Authentication issues - -**Possible Causes**: -- Proxy not configured -- Authentication not set -- Proxy URL incorrect -- Network restrictions - -**Solutions**: - -1. **Set Proxy Environment Variables**: -```bash -export HTTP_PROXY="http://proxy.example.com:8080" -export HTTPS_PROXY="http://proxy.example.com:8080" -export NO_PROXY="localhost,127.0.0.1" -``` -2. **For Authenticated Proxy**: - - Currently, SmartUI CLI may not support proxy authentication - - Use proxy without authentication if possible - - Contact support for proxy authentication support - -3. **Verify Proxy Settings**: -```bash -echo $HTTP_PROXY -echo $HTTPS_PROXY -``` -**Related Documentation**: [CLI Environment Variables](/support/docs/smartui-cli-env-variables) - -**Issue: Tunnel Connection Issues** - -**Symptoms**: -- Tunnel connection fails -- Localhost not accessible -- Connection timeouts - -**Possible Causes**: -- Tunnel not started -- Port conflicts -- Firewall blocking -- Network issues - -**Solutions**: - -1. **Start Tunnel**: -```bash -./LT --user --key -``` -2. **Verify Tunnel Status**: - - Check tunnel is running - - Verify tunnel shows as "Active" in dashboard - -3. **Check Port Availability**: - - Ensure no conflicts on tunnel ports - - Check firewall settings - -**Related Documentation**: [Tunnel Documentation](/support/docs/testing-locally-hosted-pages), [SDK Tunnel](/support/docs/smartui-sdk-tunnel) - -## Getting Additional Help - -If you encounter issues not covered in this guide: - -1. **Check Related Documentation**: - - Review framework-specific documentation - - Check [Project Settings](/support/docs/smartui-project-settings) - - Review [Configuration Options](/support/docs/smartui-sdk-config-options) - -2. **Search Existing Issues**: - - Check [TestMu AI Support](https://www.testmuai.com/support) - - Review community forums - - Search knowledge base - -3. **Contact Support**: - - Email: support@testmuai.com - - [24/7 Chat Support](https://www.testmuai.com/support) - - Include error messages, logs, and configuration details - -4. **Provide Diagnostic Information**: - - CLI version: `npx smartui --version` - - Configuration file (redact sensitive data) - - Error messages and logs - - Steps to reproduce - -## Additional Resources - -- [Project Settings](/support/docs/smartui-project-settings) -- [Configuration Options](/support/docs/smartui-sdk-config-options) -- [Handling Dynamic Data](/support/docs/smartui-handle-dynamic-data) -- [Baseline Management](/support/docs/smartui-baseline-management) -- [Running Your First Project](/support/docs/smartui-running-your-first-project) -- [CLI Documentation](/support/docs/smartui-cli) diff --git a/static/docs/smartui-upload-api-v2.md b/static/docs/smartui-upload-api-v2.md deleted file mode 100644 index af1f1aa85..000000000 --- a/static/docs/smartui-upload-api-v2.md +++ /dev/null @@ -1,58 +0,0 @@ -# Getting Started With Uploading Screenshots through API for Visual Regression Testing - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This is the guide to setup the **version 2** of the SmartUI upload API. - -### Prerequisites for running SmartUI - -- Basic understanding of HTTP APIs is required. -- Go to [`LambdaTest SmartUI`](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) and login along with your credentials. -- Copy `LT_USERNAME` and `LT_ACCESS_KEY` credentials from `Access Key` button on the top right of the dashboard. - -The following steps will guide you in running your first Visual Regression test on TestMu AI SmartUI platform - - -### Create a SmartUI Project - -To create a SmartUI Project, follow these steps: - -- Go to [Projects page](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) -- Click on the `new project` button -- Add name of the project, approvers for the changes found, tags for any filter or easy navigation. -- Click on **Submit**. - -Once, the project is created then you copy the `Project Token` from the application, here is a sample project token: - -```bash -projectToken = "123456#1234abcd-****-****-****-************" -``` - -### Upload screenshots via API - -Once you have created a SmartUI Project, you can upload your local screenshots to the projects which will create a build. - -Follow the below steps to upload screenshots - - -**Step 1:** Copy the below API endpoint to your project. - -``` -API_URL="https://api.lambdatest.com/automation/smart-ui/v2/upload" -``` - -**NOTE** -For more information, please visit our [API documentation](https://www.testmuai.com/support/api-doc/). - -**Step 2:** Add the following information to your API payload `form-body`: - -| Variable | Type | Description | Is Mandatory? | -| ------------------- | ------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------- | -| files | array | You can add the path to the files which will be uploaded. `files : [ "path/to/file-1", ""path/to/file-2"]` | True | -| projectToken | string | Project token needs to be specified to upload the files and validate the project . `projectToken:123456#1234abcd-****-****-****-************` | True | -| buildName | string | Name anything of your choice specified for the uploaded screenshots to a build. `buildName : #` | Optional | -| baseline | boolean | An option to make your build as a baseline build for comparison . `baseline : true/false` | Optional | -| screenshotName new | string | Pass the unique screenshot name in order to compare it. | Optional | - -**NOTE** -We only support the following formats: `.png` `.jpeg` `.jpg` - -Please see the below screenshot for adding your screenshots which needs to be uploaded for your project to generate a build. diff --git a/static/docs/smartui-visual-ai.md b/static/docs/smartui-visual-ai.md deleted file mode 100644 index e01dc1adc..000000000 --- a/static/docs/smartui-visual-ai.md +++ /dev/null @@ -1,61 +0,0 @@ -# Visual AI Agent - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The **Visual AI Agent** is an intelligent layer on top of SmartUI that reduces noise in visual testing. Unlike traditional pixel-by-pixel comparison, the AI Agent understands the context of visual elements, allowing it to distinguish between critical visual defects and insignificant rendering differences between your baseline and captured screenshots. This allows you to instantly focus on significant UI shifts and understand the context of the changes without getting lost in minor, irrelevant noise. - -## What Problems Does SmartUI Visual Agent Solve? - -Traditional pixel-to-pixel comparison can be noisy and time-consuming. SmartUI Visual Agent eliminates the guesswork by providing context-aware, intelligent analysis. - -| Problem | SmartUI Visual Agent Solution | -| :--- | :--- | -| Pixel diffs are cluttered with noise from anti-aliasing or rendering glitches. | The AI agent filters out irrelevant diffs and only highlights changes a human user would notice. | -| It's hard to understand the scope or meaning of a change from a simple highlighted area. | Get a plain-English summary that explains the entire change in context. | -| Reviewing hundreds of minor visual differences is slow and inefficient. | Focus your attention only on significant UI and UX regressions that impact the user experience. | - -## How to Use SmartUI Visual Agent - -### Step 1: Activate Visual AI Mode - -Navigate to any SmartUI comparison build. In the toolbar above the screenshots, locate and click the **Visual AI** toggle. This activates the "Human Intelligence AI Agent." - -### Step 2: Investigate AI-Detected Differences - -Once Visual AI is active, the system automatically draws boxes around what it has identified as significant visual changes on the "Captured" screenshot. You can then interact with these boxes to learn more. - -| User Interaction | Visual Feedback | -| :--- | :--- | -| **Default View** | Boxes are drawn around each meaningful visual difference the AI has detected. | -| **Hover** on a diff box | The box under your cursor highlights slightly, indicating it is interactive. | -| **Click** a diff box | The box turns red to confirm your selection, and a floating summary panel appears with a description of the change. | - -### Step 3: Understand the AI-Generated Summary - -When you click on a diff box, a panel appears with a concise, human-readable explanation of the change. This summary describes the full scope of the difference, allowing for quick recognition and understanding. For example, it might summarize a complex visual change as "Content Grid Replaced with Placeholders." - -## Benefits for Your Team - -- ✅ **Fewer False Positives:** Only see changes that a human would notice. -- ✅ **Faster Approvals:** Review fewer screenshots without missing real defects. -- ✅ **Actionable Insights:** Get change location, type, and a human-readable summary. -- ✅ **Works at Scale:** Designed for high-volume regression testing pipelines. - -## Perfect For - -- ✅ Teams tired of noisy traditional visual testing tools -- ✅ CI/CD pipelines that need fast, trustworthy visual QA -- ✅ Product teams that care about real user experience impact - -## Provide Feedback - -All members can give feedback on AI-generated differences and summaries. Following are the available feedback options: - -* **Thumbs up/down.** -* **Comment (optional):** Enter up to 120 characters. This option is only available when a thumbs-up or thumbs-down is given. - -## Things to Keep in Mind - -* Visual AI Agent is currently available for comparisons with vertical lengths under **5000px**. -* If the Visual AI agent is active and you open the **RCA panel**, the panel will still display all differences from the standard Smart RCA comparison. -> **Visual AI Agent can make mistakes. Please visually verify the results.** diff --git a/static/docs/smartui-wdio-sdk.md b/static/docs/smartui-wdio-sdk.md deleted file mode 100644 index dacd79024..000000000 --- a/static/docs/smartui-wdio-sdk.md +++ /dev/null @@ -1,484 +0,0 @@ -# Integrate SmartUI SDK with WebdriverIO Tests - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Welcome to the world of simplified visual testing with the SmartUI SDK. - -Integrating seamlessly into your existing WebdriverIO testing suite, SmartUI SDK revolutionizes the way you approach visual regression testing. Our robust solution empowers you to effortlessly capture, compare, and analyze screenshots across a multitude of browsers and resolutions, ensuring comprehensive coverage and accuracy in your visual testing endeavors. - -## Prerequisites - -- Basic understanding of Command Line Interface and WebdriverIO is required. -- Login to [TestMu AI SmartUI](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) with your credentials. - -The following steps will guide you in running your first Visual Regression test on TestMu AI platform using SmartUI WebdriverIO SDK integration. - -## Create a SmartUI Project - -The first step is to create a project with the application in which we will combine all your builds run on the project. To create a SmartUI Project, follow these steps: - -1. Go to [Projects page](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) -2. Click on the `new project` button -3. Select the platform as CLI for executing your `SDK` tests. -4. Add name of the project, approvers for the changes found, tags for any filter or easy navigation. -5. Click on the **Submit**. - -## Steps to run your first test - -Once you have created a SmartUI Project, you can generate screenshots by running automation scripts. Follow the below steps to successfully generate screenshots - -### **Step 1:** Create/Update your test - -You can clone the sample repository to run `LambdaTest` automation tests with `SmartUI` and use the `cloud.e2e.js` file present in the `test/specs` folder. - -```bash -git clone https://github.com/LambdaTest/smartui-wdio-sample -cd smartui-wdio-sample -``` -### **Step 2**: Install the Dependencies - -Install required NPM modules for `LambdaTest SmartUI WebdriverIO SDK` in your **Frontend** project. - -```bash -npm install @lambdatest/smartui-cli @lambdatest/wdio-driver webdriverio wdio-lambdatest-service -``` - -If you face any problems executing tests with SmartUI-CLI `versions >= v4.x.x`, upgrade your Node.js version to `v20.3` or above. - -If you are using TestMu AI automation grid to run webdriverio, please update the required configuration in the `capability configuration (wdio.conf.js) file`. - -### **Step 3:** Configure your Project Token - -Setup your project token shown in the **SmartUI** app after creating your project. - -```bash -export PROJECT_TOKEN="123456#1234abcd-****-****-****-************" -``` - -```bash -set PROJECT_TOKEN=123456#1234abcd-****-****-****-************" -``` - -```powershell -$env:PROJECT_TOKEN=123456#1234abcd-****-****-****-************" -``` - -### **Step 4:** Create and Configure SmartUI Config - -You can now configure your project configurations on using various available options to run your tests with the SmartUI integration. To generate the configuration file, please execute the following command: - -```bash -npx smartui config:create .smartui.json -``` - -Once, the configuration file will be created, you will be seeing the default configuration pre-filled in the configuration file: - -```json title="/smartui-sdk-project/.smartui.json" -{ -"web": { -"browsers": [ -"chrome", -"firefox", -"safari", -"edge" -], -"viewports": [ -[ -1920 -], -[ -1366 -], -[ -1028 -] -] // Full Page screenshots are captured by default for web viewports -}, -"mobile": { -"devices": [ -"iPhone 14", //iPhone 14 viewport -"Galaxy S24" //Galaxy S24 viewport -], -"fullPage": true, //Full Page is true by default for mobile viewports -"orientation": "portrait" //Change to "landscape" for landscape snapshot -}, -"waitForTimeout": 1000, //Optional (Should only be used in case lazy-loading/async components are present) -"waitForPageRender": 50000, //Optional (Should only be used in case of websites which take more than 30s to load) -"enableJavaScript": false, //Enable javascript for all the screenshots of the project -"allowedHostnames": [] //Additional hostnames to capture assets from -} -``` -**Advanced options in SmartUI configuration** -- For capturing fullpage or viewport screenshots, please refer to this [documentation](/docs/smartui-sdk-config-options/#12-viewports) -- For the list of available mobile viewports, please refer to this [documentation](/docs/smartui-sdk-config-options/#list-of-supported-device-viewports) -- For more information about SmartUI config global options, please refer to this [documentation](/docs/smartui-sdk-config-options/#3-global-options-optional). - -### **Step 5:** Adding SmartUI function to take screenshot - -- You can incorporate SmartUI into your custom `WebdriverIO` automation test (any platform) script by adding the `smartuiSnapshot` function in the required segment of WebdriverIO script of which we would like to take the screenshot, as shown below: - -```js -const { expect, browser, $ } = require('@wdio/globals') -const { smartuiSnapshot } = require('@lambdatest/wdio-driver'); - -describe('My first visual test', () => { -it('should use appropriate project token', async () => { -await browser.url(`https://webdriver.io`) -await smartuiSnapshot(browser, "SS-1"); - -}) -}) -``` - -### **Step 6:** Execute the Tests on SmartUI Cloud - -Execute `visual regression tests` on SmartUI using the following commands - -```bash -npx smartui --config .smartui.json exec -- wdio run ./wdio.conf.js -``` - -You can use your custom runner command in place of `wdio run ./wdio.conf.js` -You may use the `npx smartui --help` command in case you are facing issues during the execution of SmartUI commands in the CLI. - -## View SmartUI Results - -You have successfully integrated SmartUI SDK with your WebdriverIO tests. Visit your SmartUI project to view builds and compare snapshots between different test runs. - -You can see the SmartUI dashboard to view the results. This will help you identify the Mismatches from the existing `Baseline` build and do the required visual testing. - -## Arguments supported in the `smartUISnapshot` function - -The following are the different options which are currently supported: - -| Key | Description | -| ------------------------- | ------------------------------------------------------------------------------------------------------------------------- | -| `browser` (instance) | The instance of the browser used in your tests. | -| `"Screenshot Name"` (string) | Specify a name for the screenshot in your tests to match the same screenshot with the name from your baseline. | -| `options` (object) | Specify one or a combination of selectors in the `ignoreDOM` or `selectDOM` objects. These selectors can be based on `HTML DOM IDs, CSS classes, CSS selectors, or XPaths` used by your webpage. They define elements that should be excluded from or included in the visual comparison.| - -## Handling Dynamic Data in SmartUI SDK **** - -When conducting visual tests, you may encounter scenarios where certain elements within your application change between test runs. These changes might introduce inconsistencies in your test results.You can ignore / select specific element(s) to be removed from the comparison by parsing the options in the `smartuiSnapshot` function in the following way - -```js title="This is a sample for your configuration for JavaScript to ignore by" ID" -let options = { -ignoreDOM: { -id: ["ID-1", "ID-2"], -} -} -await browser.url('Required URL'); -await smartuiSnapshot(driver, 'Screenshot Name', options); -``` - -```js title="This is a sample for your configuration for JavaScript to ignore by" Class" -let options = { -ignoreDOM: { -class: ["Class-1", "Class-2"], -} -} -await browser.url('Required URL'); -await smartuiSnapshot(driver, 'Screenshot Name', options); -``` - -```js title="This is a sample for your configuration for JavaScript to ignore by" XPath" -let options = { -ignoreDOM: { -xpath: ["Xpath-1", "Xpath-2"], -} -} -await browser.url('Required URL'); -await smartuiSnapshot(driver, 'Screenshot Name', options); -``` - -```js title="This is a sample for your configuration for JavaScript to ignore by CSS" Selector" -let options = { -ignoreDOM: { -cssSelector: ["CSS-Selector-1", "CSS-Selector-2"], -} -} -await browser.url('Required URL'); -await smartuiSnapshot(driver, 'Screenshot Name', options); -``` - -```js title="This is a sample for your configuration for JavaScript to select by" ID" -let options = { -selectDOM: { -id: ["ID-1", "ID-2"], -} -} -await browser.url('Required URL'); -await smartuiSnapshot(driver, 'Screenshot Name', options); -``` - -```js title="This is a sample for your configuration for JavaScript to select by" Class" -let options = { -selectDOM: { -class: ["Class-1", "Class-2"], -} -} -await browser.url('Required URL'); -await smartuiSnapshot(driver, 'Screenshot Name', options); -``` - -```js title="This is a sample for your configuration for JavaScript to select by" XPath" -let options = { -selectDOM: { -xpath: ["Xpath-1", "Xpath-2"], -} -} -await browser.url('Required URL'); -await smartuiSnapshot(driver, 'Screenshot Name', options); -``` - -```js title="This is a sample for your webhook configuration for JavaScript to select by CSS" Selector" -let options = { -selectDOM: { -cssSelector: ["CSS-Selector-1", "CSS-Selector-2"], -} -} -await browser.url('Required URL'); -await smartuiSnapshot(driver, 'Screenshot Name', options); -``` - -## For capturing the screenshot of a specific element - -You can capture screenshots of targeted elements by leveraging various locator mechanisms such as XPath, CSS ID, class, and selectors. This precision-driven approach ensures accurate and specific visual regression testing for your web application's components. - -```js title="This is a sample for your configuration for JavaScript to capture an element by" ID." -let options = { -element: { -id: 'Required ID', -} -}; -await browser.url('Required URL'); -await smartuiSnapshot(driver, 'Screenshot Name', options); -``` - -```js title="This is a sample for your configuration for JavaScript to capture an element by" Class" -let options = { -element: { -class: 'Required Class', -} -}; -await browser.url('Required URL'); -await smartuiSnapshot(driver, 'Screenshot Name', options); -``` - -```js title="This is a sample for your configuration for JavaScript to capture an element by" XPath" -let options = { -element: { -xpath: 'Required Xpath', -} -}; -await browser.url('Required URL'); -await smartuiSnapshot(driver, 'Screenshot Name', options); -``` - -```js title="This is a sample for your webhook configuration for JavaScript to capture an element by CSS" Selector" -let options = { -element: { -cssSelector: 'Required CSS Selector', -} -}; -await browser.url('Required URL'); -await smartuiSnapshot(driver, 'Screenshot Name', options); -``` - -## Best Practices - -**Screenshot Naming** - -- Use descriptive, consistent names for screenshots -- Include page/component name in screenshot names -- Avoid special characters that might cause issues -- Use consistent naming conventions across your test suite - -**Example:** -```javascript -await smartuiSnapshot(driver, 'HomePage-Header'); -await smartuiSnapshot(driver, 'ProductPage-MainContent'); -``` - -**Wait for Page Load** - -- Always wait for pages to fully load before taking screenshots -- Use WebdriverIO's built-in wait methods for dynamic content -- Consider using `waitForTimeout` in configuration for lazy-loaded content - -**Example:** -```javascript -await browser.url('https://example.com'); -await $('#main-content').waitForDisplayed({ timeout: 10000 }); -await smartuiSnapshot(driver, 'Page Loaded'); -``` - -**Handle Dynamic Content** - -- Use `ignoreDOM` for elements that change between runs -- Use `selectDOM` when you only need to compare specific areas -- Document why elements are ignored for future reference - -**Configuration Management** - -- Keep `.smartui.json` in version control -- Use environment variables for sensitive data -- Document custom configuration choices - -**Test Organization** - -- Group related screenshots in the same build -- Use meaningful build names -- Run tests in consistent environments - -**Troubleshooting** - -**Issue: Screenshots Not Appearing in Dashboard** - -**Symptoms**: Tests run successfully but no screenshots appear in SmartUI dashboard - -**Possible Causes**: -- Project token not set or incorrect -- Project name mismatch -- Network connectivity issues -- CLI not installed or outdated - -**Solutions**: -1. Verify `PROJECT_TOKEN` is set correctly: -```bash -echo $PROJECT_TOKEN -``` - -2. Check project name matches exactly (case-sensitive) - -3. Verify SmartUI CLI is installed: -```bash -npx smartui --version -``` - -4. Check network connectivity to TestMu AI servers - -5. Review test execution logs for error messages - -**Issue: Project Not Found" Error** - -**Symptoms**: Error message indicating project cannot be found - -**Possible Causes**: -- Incorrect project token -- Project deleted or renamed -- Token from wrong project - -**Solutions**: -1. Verify project exists in SmartUI dashboard -2. Copy project token directly from Project Settings -3. Ensure token includes the project ID prefix (e.g., `123456#...`) -4. Check for extra spaces or quotes in token - -**Issue: Screenshots Show Blank or Incorrect Content** - -**Symptoms**: Screenshots captured but show blank pages or incorrect content - -**Possible Causes**: -- Page not fully loaded -- JavaScript not executed -- Viewport size issues -- Timing issues - -**Solutions**: -1. Add explicit waits before screenshots: -```javascript -await browser.url('https://example.com'); -await $('#content').waitForDisplayed({ timeout: 10000 }); -await $('.main-content').waitForDisplayed({ timeout: 10000 }); -``` - -2. Enable JavaScript in configuration: -```json -{ -enableJavaScript": true -} -``` - -3. Increase `waitForTimeout` in configuration - -4. Verify viewport size matches expected dimensions - -**Issue: Build Execution Fails** - -**Symptoms**: `npx smartui exec` command fails - -**Possible Causes**: -- Missing or incorrect configuration file -- Invalid JSON in configuration -- Port conflicts -- Permission issues - -**Solutions**: -1. Verify `.smartui.json` exists and is valid JSON -2. Check configuration file syntax -3. Try different port if default is in use: -```bash -npx smartui exec -P 5000 -- -``` -4. Check file permissions for configuration and project files - -**Issue: npm Dependencies Not Resolving** - -**Symptoms**: npm cannot find `@lambdatest/wdio-driver` or `@lambdatest/smartui-cli` - -**Possible Causes**: -- Incorrect package version -- npm registry access issues -- Network connectivity problems - -**Solutions**: -1. Check latest versions on npm: -```bash -npm view @lambdatest/wdio-driver version -npm view @lambdatest/smartui-cli version -``` - -2. Clear npm cache: -```bash -npm cache clean --force -``` - -3. Verify internet connectivity for npm registry access - -4. Check package.json for version conflicts - -**Issue: Screenshot Names Not Matching Baseline** - -**Symptoms**: Screenshots appear as New" instead of comparing with baseline - -**Possible Causes**: -- Screenshot name changed -- Baseline doesn't exist -- Name contains special characters - -**Solutions**: -1. Ensure screenshot names are consistent across test runs -2. Verify baseline exists in project -3. Avoid special characters in screenshot names -4. Check for case sensitivity issues - -**Getting Help** - -If you encounter issues not covered here: - -- Review the [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) for detailed solutions -- Check [SmartUI Configuration Options](/support/docs/smartui-sdk-config-options) documentation -- See [Handling Dynamic Data](/support/docs/smartui-handle-dynamic-data) for dynamic content issues -- Visit [TestMu AI Support](https://www.testmuai.com/support) for additional resources -- Contact support at support@testmuai.com or use [24/7 Chat Support](https://www.testmuai.com/support) - -## Additional Resources - -- [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) -- [SmartUI Configuration Options](/support/docs/smartui-sdk-config-options) -- [Handling Dynamic Data](/support/docs/smartui-handle-dynamic-data) -- [Handling Lazy Loading](/support/docs/smartui-handle-lazy-loading) -- [Baseline Management](/support/docs/smartui-baseline-management) -- [Running Your First Project](/support/docs/smartui-running-your-first-project) -- [SmartUI API Documentation](https://www.testmuai.com/support/api-doc/) diff --git a/static/docs/smartui-with-azure.md b/static/docs/smartui-with-azure.md deleted file mode 100644 index 00fe4607b..000000000 --- a/static/docs/smartui-with-azure.md +++ /dev/null @@ -1,301 +0,0 @@ -# Azure Pipeline Integration with SmartUI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Azure Pipelines is a cloud-based CI/CD service offered by Microsoft, part of the Azure DevOps suite. It helps automate the process of building, testing, and deploying applications to various platforms. - -This guide explains how to integrate your project with the Azure CI/CD pipeline to trigger visual regression testing with TestMu AI SmartUI whenever changes are made to your repository. - -## Steps to Integrate Azure Pipeline with SmartUI -To integrate Azure Pipeline with SmartUI, follow the below steps. You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** -Download or Clone the code sample from the TestMu AI GitHub repository to run the tests on the SmartUI. - - View on GitHub - -### Step 1: Set Up Your Repository -Ensure your project is hosted in Azure Repos, GitHub, or any supported repository. - -### Step 2: Create a New Pipeline: - -- Navigate to Pipelines in your Azure DevOps project. -- Select New Pipeline and connect your repository. - -### Step 3: Add Environment Variables - -Go to Pipeline Settings > Variables. Add the following variables: -- `LT_USERNAME`: Your TestMu AI username. -- `LT_ACCESS_KEY`: Your TestMu AI access key. - -### Step 4: Setup your Workflow -```yaml title="azure-pipelines.yml" -trigger: -- '*' - -variables: -LT_USERNAME: $(LT_USERNAME) -LT_ACCESS_KEY: $(LT_ACCESS_KEY) - -jobs: -- job: SmartUI_Tests -pool: -vmImage: 'ubuntu-latest' - -steps: -- task: UseNode@2 -inputs: -version: '16.x' - -- script: | -echo "Installing dependencies" -npm install @lambdatest/smartui-cli -displayName: 'Install Dependencies' - -- script: | -echo "Running SmartUI tests" -npx smartui --version -npx smartui config:create smartui-web.json -npx smartui --config smartui-web.json exec -- mvn --quiet test -D suite=sdk-cloud.xml -displayName: 'Execute SmartUI Tests' -``` - -### Step 5: Check the output - -- After triggering the workflow, check your results in the [SmartUI Dashboard](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/projects) - -## Best Practices - -**Secret Management** - -- Never commit credentials to repository -- Use Azure Pipeline Variables for all sensitive data -- Mark variables as secret to hide values in logs -- Rotate secrets regularly -- Use different secrets for different environments - -**Pipeline Optimization** - -- Use parallel jobs for faster execution -- Cache dependencies to speed up pipelines -- Only run visual tests on relevant branches -- Set up pipeline conditions to avoid unnecessary runs - -**Example:** -```yaml -trigger: -branches: -include: -- main -- develop -``` - -**Build Naming** - -- Use meaningful build names that include branch/commit info -- Include commit SHA for traceability -- Use consistent naming conventions - -**Example:** -```yaml -variables: -BUILD_NAME: $(Build.SourceBranchName)-$(Build.SourceVersion)" -``` - -**Error Handling** - -- Set up proper error handling in pipelines -- Use pipeline status checks -- Configure notifications for failures -- Add retry logic for flaky tests - -**Resource Management** - -- Limit concurrent pipeline runs -- Clean up old builds regularly -- Monitor pipeline execution time -- Optimize test execution order - -**Resource Management** - -- Limit concurrent pipeline runs -- Clean up old builds regularly -- Monitor pipeline execution time -- Optimize test execution order - -## Troubleshooting - -**Issue: Pipeline Fails with "Variable Not Found"** - -**Symptoms**: Pipeline fails with error about missing variables - -**Possible Causes**: -- Variables not created in Azure DevOps -- Variable names don't match -- Variables not accessible to pipeline -- Variable scope issues - -**Solutions**: -1. Verify variables exist in pipeline settings: - - Go to Pipelines → Edit → Variables - - Check `LT_USERNAME`, `LT_ACCESS_KEY`, and `PROJECT_TOKEN` exist - -2. Ensure variable names match exactly (case-sensitive) - -3. Check variable scope (pipeline, stage, or job level) - -4. Verify variables are marked as secret if needed - -**Issue: PROJECT_TOKEN Not Available** - -**Symptoms**: Pipeline prompts for PROJECT_TOKEN or token not found - -**Possible Causes**: -- PROJECT_TOKEN not set as pipeline variable -- Variable not passed to job -- Variable marked as secret incorrectly - -**Solutions**: -1. Add PROJECT_TOKEN as Azure Pipeline Variable - -2. Pass variable to job: -```yaml -variables: -PROJECT_TOKEN: $(PROJECT_TOKEN) -``` - -3. Check variable is accessible to the job - -4. Verify variable scope includes your pipeline - -**Issue: Tests Run But No Results in Dashboard** - -**Symptoms**: Pipeline completes but screenshots don't appear in SmartUI - -**Possible Causes**: -- Incorrect PROJECT_TOKEN -- Project name mismatch -- Network issues -- Pipeline job failure - -**Solutions**: -1. Verify PROJECT_TOKEN is correct: - - Check token in SmartUI Project Settings - - Ensure token includes project ID prefix - -2. Check pipeline logs for errors: -```yaml -- task: PowerShell@2 -displayName: 'Check Logs' -condition: failed() -inputs: -script: | -Get-Content $(Agent.TempDirectory)/*.log -``` - -3. Verify network connectivity in pipeline - -4. Check if SmartUI CLI step completed successfully - -**Issue: Pipeline Times Out** - -**Symptoms**: Pipeline execution exceeds time limit - -**Possible Causes**: -- Too many tests running -- Slow test execution -- Network latency -- Resource constraints - -**Solutions**: -1. Increase pipeline timeout: -```yaml -timeoutInMinutes: 60 -``` - -2. Run tests in parallel using matrix: -```yaml -strategy: -matrix: -TestGroup1: -TEST_GROUP: 1 -TestGroup2: -TEST_GROUP: 2 -``` - -3. Optimize test execution -4. Split tests across multiple pipeline stages - -**Issue: Dependencies Installation Fails** - -**Symptoms**: npm install or dependency installation fails - -**Possible Causes**: -- Network issues -- Package registry problems -- Version conflicts -- Node version mismatch - -**Solutions**: -1. Use specific Node version: -```yaml -- task: UseNode@2 -inputs: -version: '18.x' -``` - -2. Clear npm cache: -```yaml -- script: | -npm cache clean --force -npm install -``` - -3. Use package-lock.json for consistent installs - -4. Check for version conflicts in package.json - -**Issue: SmartUI CLI Not Found** - -**Symptoms**: `npx smartui` command fails with command not found" - -**Possible Causes**: -- Node.js not installed -- npm not available -- PATH issues - -**Solutions**: -1. Ensure Node.js setup task is included: -```yaml -- task: UseNode@2 -inputs: -version: '18.x' -``` - -2. Verify npm is available: -```yaml -- script: npm --version -``` - -3. Install SmartUI CLI explicitly: -```yaml -- script: | -npm install -g @lambdatest/smartui-cli -``` - -**Getting Help** - -If you encounter issues not covered here: - -- Review [Azure Pipelines Documentation](https://docs.microsoft.com/en-us/azure/devops/pipelines/) -- Check [SmartUI CLI Documentation](/support/docs/smartui-cli) for CLI-specific issues -- Visit [TestMu AI Support](https://www.testmuai.com/support) for additional resources -- Contact support at support@testmuai.com or use [24/7 Chat Support](https://www.testmuai.com/support) - -## Additional Resources - -- [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) -- [SmartUI CLI Documentation](/support/docs/smartui-cli) -- [Azure Pipelines Documentation](https://docs.microsoft.com/en-us/azure/devops/pipelines/) -- [Project Settings](/support/docs/smartui-project-settings) -- [Running Your First Project](/support/docs/smartui-running-your-first-project) diff --git a/static/docs/smartui-with-bitbucket.md b/static/docs/smartui-with-bitbucket.md deleted file mode 100644 index 178b51430..000000000 --- a/static/docs/smartui-with-bitbucket.md +++ /dev/null @@ -1,257 +0,0 @@ -# Bitbucket Pipeline Integration with SmartUI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Bitbucket is a web-based version control repository hosting service owned by Atlassian. It is primarily designed for development teams to manage their code, collaborate on projects, and streamline their workflows. - -This document will show you how to integrate Bitbucket Pipeline with SmartUI to shorten your test cycles. - -## Steps to Integrate Bitbucket Pipeline with SmartUI -To integrate Bitbucket Pipeline with SmartUI, follow the below steps. You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** -Download or Clone the code sample from the TestMu AI GitHub repository to run the tests on the SmartUI. - - View on GitHub - -### Step 1: Setup your Projects and Repository in Bitbucket -- Click on the **Create** >> **Project**. -- Enter your Project details and click on **Create Project**. - -- Now click on the **Create Repository** button. You can either create a new repository or import your existing repository. - -### Step 2: Create a New Workflow -- Navigate to the **Deployment** section. Select your required template for CI/CD workflow file. For the demo we are using the Test template. -- Now, write your workflow YAML file. Here is the sample file for your reference. -- Commit this yaml file in your repository and make the required changes in your code to automatically trigger the pipeline. - -```yaml reference title="bitbucket-pipelines.yml" -https://github.com/amanchopra1905/smartui-ci-cd-integrations/blob/bitbucket/bitbucket-pipelines.yml -``` - -You can also store your *LT_USERNAME*, *LT_ACCESS_KEY* and *PROJECT_TOKEN* as secrets in your Bitbucket project repository. - -### Step 3: Check the output - -- After triggering the workflow, check your results in the [SmartUI Dashboard](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/projects) - -## Best Practices - -**Secret Management** - -- Never commit credentials to repository -- Use Bitbucket Pipelines Variables for all sensitive data -- Rotate secrets regularly -- Use different secrets for different environments - -**Pipeline Optimization** - -- Use parallel steps for faster execution -- Cache dependencies to speed up pipelines -- Only run visual tests on relevant branches -- Set up pipeline conditions to avoid unnecessary runs - -**Example:** -```yaml -branches: -main: -- step: -name: Run Visual Tests -``` - -**Build Naming** - -- Use meaningful build names that include branch/commit info -- Include commit SHA for traceability -- Use consistent naming conventions - -**Example:** -```yaml -variables: -BUILD_NAME: $BITBUCKET_BRANCH-$BITBUCKET_COMMIT" -``` - -**Error Handling** - -- Set up proper error handling in pipelines -- Use pipeline status checks -- Configure notifications for failures -- Add retry logic for flaky tests - -**Resource Management** - -- Limit concurrent pipeline runs -- Clean up old builds regularly -- Monitor pipeline execution time -- Optimize test execution order - -**Resource Management** - -- Limit concurrent pipeline runs -- Clean up old builds regularly -- Monitor pipeline execution time -- Optimize test execution order - -## Troubleshooting - -**Issue: Pipeline Fails with "Variable Not Found"** - -**Symptoms**: Pipeline fails with error about missing variables - -**Possible Causes**: -- Variables not created in Bitbucket repository -- Variable names don't match -- Variables not accessible to pipeline - -**Solutions**: -1. Verify variables exist in repository settings: - - Go to Repository Settings → Pipelines → Repository variables - - Check `LT_USERNAME`, `LT_ACCESS_KEY`, and `PROJECT_TOKEN` exist - -2. Ensure variable names match exactly (case-sensitive) - -3. Check variable scope (repository, workspace, or deployment level) - -4. Verify variables are secured if needed - -**Issue: PROJECT_TOKEN Not Available** - -**Symptoms**: Pipeline prompts for PROJECT_TOKEN or token not found - -**Possible Causes**: -- PROJECT_TOKEN not set as repository variable -- Variable not passed to step -- Variable secured incorrectly - -**Solutions**: -1. Add PROJECT_TOKEN as Bitbucket Repository Variable - -2. Pass variable to step: -```yaml -variables: -PROJECT_TOKEN: $PROJECT_TOKEN -``` - -3. Check variable is secured if needed - -4. Verify variable scope includes your branch - -**Issue: Tests Run But No Results in Dashboard** - -**Symptoms**: Pipeline completes but screenshots don't appear in SmartUI - -**Possible Causes**: -- Incorrect PROJECT_TOKEN -- Project name mismatch -- Network issues -- Pipeline step failure - -**Solutions**: -1. Verify PROJECT_TOKEN is correct: - - Check token in SmartUI Project Settings - - Ensure token includes project ID prefix - -2. Check pipeline logs for errors: -```yaml -after-script: -- echo Checking logs..." -``` - -3. Verify network connectivity in pipeline - -4. Check if SmartUI CLI step completed successfully - -**Issue: Pipeline Times Out** - -**Symptoms**: Pipeline execution exceeds time limit - -**Possible Causes**: -- Too many tests running -- Slow test execution -- Network latency -- Resource constraints - -**Solutions**: -1. Increase pipeline timeout in Bitbucket settings - -2. Run tests in parallel using parallel steps: -```yaml -parallel: -- step: -name: Test Group 1 -- step: -name: Test Group 2 -``` - -3. Optimize test execution -4. Split tests across multiple pipeline steps - -**Issue: Dependencies Installation Fails** - -**Symptoms**: npm install or dependency installation fails - -**Possible Causes**: -- Network issues -- Package registry problems -- Version conflicts -- Node version mismatch - -**Solutions**: -1. Use specific Node version: -```yaml -image: node:18 -``` - -2. Clear npm cache: -```yaml -script: -- npm cache clean --force -- npm install -``` - -3. Use package-lock.json for consistent installs - -4. Check for version conflicts in package.json - -**Issue: SmartUI CLI Not Found** - -**Symptoms**: `npx smartui` command fails with command not found" - -**Possible Causes**: -- Node.js not available in image -- npm not available -- PATH issues - -**Solutions**: -1. Ensure Node.js is available: -```yaml -image: node:18 -``` - -2. Verify npm is available: -```yaml -script: -- npm --version -``` - -3. Install SmartUI CLI explicitly: -```yaml -script: -- npm install -g @lambdatest/smartui-cli -``` - -**Getting Help** - -If you encounter issues not covered here: - -- Review [Bitbucket Pipelines Documentation](https://support.atlassian.com/bitbucket-cloud/docs/get-started-with-bitbucket-pipelines/) -- Check [SmartUI CLI Documentation](/support/docs/smartui-cli) for CLI-specific issues -- Visit [TestMu AI Support](https://www.testmuai.com/support) for additional resources -- Contact support at support@testmuai.com or use [24/7 Chat Support](https://www.testmuai.com/support) - -## Additional Resources - -- [SmartUI CLI Documentation](/support/docs/smartui-cli) -- [Bitbucket Pipelines Documentation](https://support.atlassian.com/bitbucket-cloud/docs/get-started-with-bitbucket-pipelines/) -- [Project Settings](/support/docs/smartui-project-settings) -- [Running Your First Project](/support/docs/smartui-running-your-first-project) diff --git a/static/docs/smartui-with-buildkite.md b/static/docs/smartui-with-buildkite.md deleted file mode 100644 index 37206424c..000000000 --- a/static/docs/smartui-with-buildkite.md +++ /dev/null @@ -1,283 +0,0 @@ -# Buildkite Pipeline Integration with SmartUI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Buildkite is a hybrid CI/CD platform that allows you to run builds in your own infrastructure, alongside cloud-based execution. This means you can leverage the power of your existing machines or cloud resources to run builds. - -This guide explains how to integrate your project with the Buildkite CI/CD pipeline to trigger visual regression testing with TestMu AI SmartUI whenever changes are made to your repository. - -## Steps to Integrate Buildkite Pipeline with SmartUI -To integrate Buildkite Pipeline with SmartUI, follow the below steps. You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** -Download or Clone the code sample from the TestMu AI GitHub repository to run the tests on the SmartUI. - - View on GitHub - -### Step 1: Set Up Your Repository -Ensure your project is hosted in GitHub, or any supported repository. - -### Step 2: Create a New Pipeline: - -- Navigate to Pipelines in your Buildkite project. -- Select New Pipeline and connect your repository. - -### Step 3: Add Environment Variables - -Go to Pipeline Settings > Variables. Add the following variables: -- `LT_USERNAME`: Your TestMu AI username. -- `LT_ACCESS_KEY`: Your TestMu AI access key. - -### Step 4: Setup your Workflow - -```yaml title="pipeline.yml" -steps: -- label: "SmartUI Tests" -agents: -queue: "default" -env: -LT_USERNAME: "${LT_USERNAME}" -LT_ACCESS_KEY: "${LT_ACCESS_KEY}" -commands: -- echo "Checking out code" -- git clone -- cd -- echo "Installing SmartUI CLI" -- npm install @lambdatest/smartui-cli -- echo "Running SmartUI tests" -- npx smartui --version -- npx smartui config:create smartui-web.json -- npx smartui --config smartui-web.json exec -- mvn --quiet test -D suite=sdk-cloud.xml -``` - -### Step 5: Check the output - -- After triggering the workflow, check your results in the [SmartUI Dashboard](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/projects) - -## Best Practices - -**Secret Management** - -- Never commit credentials to repository -- Use Buildkite Environment Variables for all sensitive data -- Mark variables as secret to hide values in logs -- Rotate secrets regularly -- Use different secrets for different environments - -**Pipeline Optimization** - -- Use parallel steps for faster execution -- Cache dependencies to speed up pipelines -- Only run visual tests on relevant branches -- Set up pipeline conditions to avoid unnecessary runs - -**Example:** -```yaml -branches: main develop" -``` - -**Build Naming** - -- Use meaningful build names that include branch/commit info -- Include commit SHA for traceability -- Use consistent naming conventions - -**Example:** -```yaml -env: -BUILD_NAME: ${BUILDKITE_BRANCH}-${BUILDKITE_COMMIT:0:7}" -``` - -**Error Handling** - -- Set up proper error handling in pipelines -- Use pipeline status checks -- Configure notifications for failures -- Add retry logic for flaky tests - -**Resource Management** - -- Limit concurrent pipeline runs -- Clean up old builds regularly -- Monitor pipeline execution time -- Optimize test execution order - -**Resource Management** - -- Limit concurrent pipeline runs -- Clean up old builds regularly -- Monitor pipeline execution time -- Optimize test execution order - -## Troubleshooting - -**Issue: Pipeline Fails with "Variable Not Found"** - -**Symptoms**: Pipeline fails with error about missing environment variables - -**Possible Causes**: -- Variables not created in Buildkite project -- Variable names don't match -- Variables not accessible to pipeline -- Variable scope issues - -**Solutions**: -1. Verify variables exist in project settings: - - Go to Project Settings → Environment Variables - - Check `LT_USERNAME`, `LT_ACCESS_KEY`, and `PROJECT_TOKEN` exist - -2. Ensure variable names match exactly (case-sensitive) - -3. Check variable scope (pipeline or organization level) - -4. Verify variables are marked as secret if needed - -**Issue: PROJECT_TOKEN Not Available** - -**Symptoms**: Pipeline prompts for PROJECT_TOKEN or token not found - -**Possible Causes**: -- PROJECT_TOKEN not set as environment variable -- Variable not passed to step -- Variable masked incorrectly - -**Solutions**: -1. Add PROJECT_TOKEN as Buildkite Environment Variable - -2. Pass variable to step: -```yaml -env: -PROJECT_TOKEN: ${PROJECT_TOKEN}" -``` - -3. Check variable is accessible to the pipeline - -4. Verify variable scope includes your project - -**Issue: Tests Run But No Results in Dashboard** - -**Symptoms**: Pipeline completes but screenshots don't appear in SmartUI - -**Possible Causes**: -- Incorrect PROJECT_TOKEN -- Project name mismatch -- Network issues -- Pipeline step failure - -**Solutions**: -1. Verify PROJECT_TOKEN is correct: - - Check token in SmartUI Project Settings - - Ensure token includes project ID prefix - -2. Check pipeline logs for errors: -```yaml -- label: Check Logs" -if: "build.state == 'failed'" -commands: -- cat /tmp/*.log || true -``` - -3. Verify network connectivity in pipeline - -4. Check if SmartUI CLI step completed successfully - -**Issue: Pipeline Times Out** - -**Symptoms**: Pipeline execution exceeds time limit - -**Possible Causes**: -- Too many tests running -- Slow test execution -- Network latency -- Resource constraints - -**Solutions**: -1. Increase pipeline timeout in Buildkite settings - -2. Run tests in parallel using parallel steps: -```yaml -- wait -- parallel: -- label: Test Group 1" -commands: -- npx smartui exec -- -- label: "Test Group 2" -commands: -- npx smartui exec -- -``` - -3. Optimize test execution -4. Split tests across multiple pipeline steps - -**Issue: Dependencies Installation Fails** - -**Symptoms**: npm install or dependency installation fails - -**Possible Causes**: -- Network issues -- Package registry problems -- Version conflicts -- Node version mismatch - -**Solutions**: -1. Use specific Node version: -```yaml -commands: -- nvm use 18 -``` - -2. Clear npm cache: -```yaml -commands: -- npm cache clean --force -- npm install -``` - -3. Use package-lock.json for consistent installs - -4. Check for version conflicts in package.json - -**Issue: SmartUI CLI Not Found** - -**Symptoms**: `npx smartui` command fails with command not found" - -**Possible Causes**: -- Node.js not installed -- npm not available -- PATH issues - -**Solutions**: -1. Ensure Node.js is available: -```yaml -commands: -- nvm use 18 -``` - -2. Verify npm is available: -```yaml -commands: -- npm --version -``` - -3. Install SmartUI CLI explicitly: -```yaml -commands: -- npm install -g @lambdatest/smartui-cli -``` - -**Getting Help** - -If you encounter issues not covered here: - -- Review [Buildkite Documentation](https://buildkite.com/docs) -- Check [SmartUI CLI Documentation](/support/docs/smartui-cli) for CLI-specific issues -- Visit [TestMu AI Support](https://www.testmuai.com/support) for additional resources -- Contact support at support@testmuai.com or use [24/7 Chat Support](https://www.testmuai.com/support) - -## Additional Resources - -- [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) -- [SmartUI CLI Documentation](/support/docs/smartui-cli) -- [Buildkite Documentation](https://buildkite.com/docs) -- [Project Settings](/support/docs/smartui-project-settings) -- [Running Your First Project](/support/docs/smartui-running-your-first-project) diff --git a/static/docs/smartui-with-circle-ci.md b/static/docs/smartui-with-circle-ci.md deleted file mode 100644 index 88d23fac4..000000000 --- a/static/docs/smartui-with-circle-ci.md +++ /dev/null @@ -1,307 +0,0 @@ -# Circle CI Pipeline Integration with SmartUI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -CircleCI is a popular CI/CD tool known for its speed, flexibility, and seamless integration with version control systems like GitHub and Bitbucket. It is cloud-native and also offers an on-premises solution. - -This guide explains how to integrate your project with the CircleCI CI/CD pipeline to trigger visual regression testing with TestMu AI SmartUI whenever changes are made to your repository. - -## Steps to Integrate CircleCI Pipeline with SmartUI -To integrate CircleCI Pipeline with SmartUI, follow the below steps. You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** -Download or Clone the code sample from the TestMu AI GitHub repository to run the tests on the SmartUI. - - View on GitHub - -### Step 1: Set Up Your Repository -Ensure your project is hosted in GitHub, or any supported repository. - -### Step 2: Create a New Pipeline: - -- Navigate to Pipelines in your CircleCI project. -- Select New Pipeline and connect your repository. - -### Step 3: Add Environment Variables - -Go to Pipeline Settings > Variables. Add the following variables: -- `LT_USERNAME`: Your TestMu AI username. -- `LT_ACCESS_KEY`: Your TestMu AI access key. - -### Step 4: Setup your Workflow - -```yaml title=".circleci/config.yml" -version: 2.1 -jobs: -smartui_test: -docker: -- image: circleci/node:16 -environment: -LT_USERNAME: $LT_USERNAME -LT_ACCESS_KEY: $LT_ACCESS_KEY -steps: -- checkout -- run: -name: Install Dependencies -command: npm install @lambdatest/smartui-cli -- run: -name: Execute SmartUI Tests -command: | -npx smartui --version -npx smartui config:create smartui-web.json -npx smartui --config smartui-web.json exec -- mvn --quiet test -D suite=sdk-cloud.xml - -workflows: -version: 2 -smartui_pipeline: -jobs: -- smartui_test -``` - -### Step 5: Check the output - -- After triggering the workflow, check your results in the [SmartUI Dashboard](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/projects) - -## Best Practices - -**Secret Management** - -- Never commit credentials to repository -- Use CircleCI Environment Variables for all sensitive data -- Mark variables as secret to hide values in logs -- Rotate secrets regularly -- Use different secrets for different environments - -**Pipeline Optimization** - -- Use parallel jobs for faster execution -- Cache dependencies to speed up pipelines -- Only run visual tests on relevant branches -- Set up pipeline conditions to avoid unnecessary runs - -**Example:** -```yaml -workflows: -version: 2 -smartui_pipeline: -jobs: -- smartui_test: -filters: -branches: -only: -- main -- develop -``` - -**Build Naming** - -- Use meaningful build names that include branch/commit info -- Include commit SHA for traceability -- Use consistent naming conventions - -**Example:** -```yaml -environment: -BUILD_NAME: ${CIRCLE_BRANCH}-${CIRCLE_SHA1:0:7}" -``` - -**Error Handling** - -- Set up proper error handling in pipelines -- Use pipeline status checks -- Configure notifications for failures -- Add retry logic for flaky tests - -**Resource Management** - -- Limit concurrent pipeline runs -- Clean up old builds regularly -- Monitor pipeline execution time -- Optimize test execution order - -**Resource Management** - -- Limit concurrent pipeline runs -- Clean up old builds regularly -- Monitor pipeline execution time -- Optimize test execution order - -## Troubleshooting - -**Issue: Pipeline Fails with "Variable Not Found"** - -**Symptoms**: Pipeline fails with error about missing environment variables - -**Possible Causes**: -- Variables not created in CircleCI project -- Variable names don't match -- Variables not accessible to job -- Variable scope issues - -**Solutions**: -1. Verify variables exist in project settings: - - Go to Project Settings → Environment Variables - - Check `LT_USERNAME`, `LT_ACCESS_KEY`, and `PROJECT_TOKEN` exist - -2. Ensure variable names match exactly (case-sensitive) - -3. Check variable scope (project or context level) - -4. Verify variables are not masked if you need to see them in logs - -**Issue: PROJECT_TOKEN Not Available** - -**Symptoms**: Pipeline prompts for PROJECT_TOKEN or token not found - -**Possible Causes**: -- PROJECT_TOKEN not set as environment variable -- Variable not passed to job -- Variable masked incorrectly - -**Solutions**: -1. Add PROJECT_TOKEN as CircleCI Environment Variable - -2. Pass variable to job: -```yaml -environment: -PROJECT_TOKEN: $PROJECT_TOKEN -``` - -3. Check variable is accessible to the job - -4. Verify variable scope includes your project - -**Issue: Tests Run But No Results in Dashboard** - -**Symptoms**: Pipeline completes but screenshots don't appear in SmartUI - -**Possible Causes**: -- Incorrect PROJECT_TOKEN -- Project name mismatch -- Network issues -- Pipeline job failure - -**Solutions**: -1. Verify PROJECT_TOKEN is correct: - - Check token in SmartUI Project Settings - - Ensure token includes project ID prefix - -2. Check pipeline logs for errors: -```yaml -- run: -name: Check Logs -when: on_failure -command: | -cat /tmp/*.log || true -``` - -3. Verify network connectivity in pipeline - -4. Check if SmartUI CLI step completed successfully - -**Issue: Pipeline Times Out** - -**Symptoms**: Pipeline execution exceeds time limit - -**Possible Causes**: -- Too many tests running -- Slow test execution -- Network latency -- Resource constraints - -**Solutions**: -1. Increase pipeline timeout: -```yaml -- run: -name: Execute Tests -no_output_timeout: 60m -``` - -2. Run tests in parallel using matrix: -```yaml -jobs: -smartui_test: -matrix: -parameters: -test_group: [1, 2, 3] -``` - -3. Optimize test execution -4. Split tests across multiple pipeline jobs - -**Issue: Dependencies Installation Fails** - -**Symptoms**: npm install or dependency installation fails - -**Possible Causes**: -- Network issues -- Package registry problems -- Version conflicts -- Node version mismatch - -**Solutions**: -1. Use specific Node version: -```yaml -docker: -- image: circleci/node:18 -``` - -2. Clear npm cache: -```yaml -- run: -name: Install Dependencies -command: | -npm cache clean --force -npm install -``` - -3. Use package-lock.json for consistent installs - -4. Check for version conflicts in package.json - -**Issue: SmartUI CLI Not Found** - -**Symptoms**: `npx smartui` command fails with command not found" - -**Possible Causes**: -- Node.js not installed -- npm not available -- PATH issues - -**Solutions**: -1. Ensure Node.js is available in Docker image: -```yaml -docker: -- image: circleci/node:18 -``` - -2. Verify npm is available: -```yaml -- run: -name: Check npm -command: npm --version -``` - -3. Install SmartUI CLI explicitly: -```yaml -- run: -name: Install SmartUI CLI -command: npm install -g @lambdatest/smartui-cli -``` - -**Getting Help** - -If you encounter issues not covered here: - -- Review [CircleCI Documentation](https://circleci.com/docs/) -- Check [SmartUI CLI Documentation](/support/docs/smartui-cli) for CLI-specific issues -- Visit [TestMu AI Support](https://www.testmuai.com/support) for additional resources -- Contact support at support@testmuai.com or use [24/7 Chat Support](https://www.testmuai.com/support) - -## Additional Resources - -- [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) -- [SmartUI CLI Documentation](/support/docs/smartui-cli) -- [CircleCI Documentation](https://circleci.com/docs/) -- [Project Settings](/support/docs/smartui-project-settings) -- [Running Your First Project](/support/docs/smartui-running-your-first-project) diff --git a/static/docs/smartui-with-github-actions.md b/static/docs/smartui-with-github-actions.md deleted file mode 100644 index 27999b0e3..000000000 --- a/static/docs/smartui-with-github-actions.md +++ /dev/null @@ -1,270 +0,0 @@ -# GitHub Actions Pipeline Integration with SmartUI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -GitHub Actions is a powerful automation and continuous integration/continuous delivery (CI/CD) platform built into GitHub. It allows you to create custom automated YAML workflows directly within your GitHub repositories. This helps you to build and test every pull request to your repository, or deploy merged pull requests to production. - -This document will show you how to integrate GitHub Actions Pipeline with SmartUI to greatly shorten your test cycles. - -## Steps to Integrate GitHub Actions Pipeline with SmartUI -To integrate GitHub Actions Pipeline with SmartUI, follow the below steps. You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** -Download or Clone the code sample from the TestMu AI GitHub repository to run the tests on the SmartUI. - - View on GitHub - -### Step 1: Create your Secrets -- Click on the **Settings** of your repository. -- Go to the **Security** option > **Secrets and Variables** > **Actions**. -- Create your secrets with variable name **LT_USERNAME** and **LT_ACCESS_KEY**. You can fetch your credentials from the [Accounts and Settings dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/security). - -### Step 2: Create a New Workflow -- Navigate to the main page of the repository. -- Under your repository name, click **Actions**. -- In the left sidebar, click the **New workflow** button. - -### Step 3: Create the GitHub Actions workflow YAML file -To create the GitHub Actions pipeline YAML file, follow the sample command below: - -```yaml reference title="github-actions.yml" -https://github.com/amanchopra1905/smartui-ci-cd-integrations/blob/main/.github/workflows/main.yml -``` - -### Step 4: Run the Workflow -To run the new pipeline that you just created, click the **Run workflow** button on the workflow page. A prompt will ask you to enter your **PROJECT_TOKEN**. You can get your project token from the dashboard after creating your SmartUI project. - -> Check your output in the [SmartUI Dashboard](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/projects) - -## Best Practices - -**Secret Management** - -- Never commit credentials to repository -- Use GitHub Secrets for all sensitive data -- Rotate secrets regularly -- Use different secrets for different environments - -**Workflow Optimization** - -- Use matrix strategies for parallel execution -- Cache dependencies to speed up workflows -- Only run visual tests on relevant branches -- Set up workflow conditions to avoid unnecessary runs - -**Example:** -```yaml -on: -push: -branches: [ main, develop ] -pull_request: -branches: [ main ] -``` - -**Build Naming** - -- Use meaningful build names that include branch/PR info -- Include commit SHA for traceability -- Use consistent naming conventions - -**Example:** -```yaml -- name: Set build name -run: | -BUILD_NAME="PR-${{ github.event.pull_request.number }}-${{ github.sha }}" -echo "BUILD_NAME=$BUILD_NAME" >> $GITHUB_ENV -``` - -**Error Handling** - -- Set up proper error handling in workflows -- Use workflow status checks -- Configure notifications for failures -- Add retry logic for flaky tests - -**Resource Management** - -- Limit concurrent workflow runs -- Clean up old builds regularly -- Monitor workflow execution time -- Optimize test execution order - -**Resource Management** - -- Limit concurrent workflow runs -- Clean up old builds regularly -- Monitor workflow execution time -- Optimize test execution order - -## Troubleshooting - -**Issue: Workflow Fails with "Secret Not Found"** - -**Symptoms**: Workflow fails with error about missing secrets - -**Possible Causes**: -- Secrets not created in repository -- Secret names don't match -- Secrets not accessible to workflow - -**Solutions**: -1. Verify secrets exist in repository settings: - - Go to Settings → Secrets and variables → Actions - - Check `LT_USERNAME`, `LT_ACCESS_KEY`, and `PROJECT_TOKEN` exist - -2. Ensure secret names match exactly (case-sensitive) - -3. Check workflow has permission to access secrets - -4. Verify secrets are set for the correct repository/environment - -**Issue: PROJECT_TOKEN Prompt Appears** - -**Symptoms**: Workflow prompts for PROJECT_TOKEN during execution - -**Possible Causes**: -- PROJECT_TOKEN not set as secret -- Secret not passed to workflow step -- Workflow input not configured - -**Solutions**: -1. Add PROJECT_TOKEN as GitHub Secret - -2. Pass secret to workflow step: -```yaml -env: -PROJECT_TOKEN: ${{ secrets.PROJECT_TOKEN }} -``` - -3. For manual workflows, add workflow input: -```yaml -on: -workflow_dispatch: -inputs: -project_token: -required: true -type: string -``` - -**Issue: Tests Run But No Results in Dashboard** - -**Symptoms**: Workflow completes but screenshots don't appear in SmartUI - -**Possible Causes**: -- Incorrect PROJECT_TOKEN -- Project name mismatch -- Network issues -- Workflow step failure - -**Solutions**: -1. Verify PROJECT_TOKEN is correct: - - Check token in SmartUI Project Settings - - Ensure token includes project ID prefix - -2. Check workflow logs for errors: -```yaml -- name: View logs -if: failure() -run: | -# Check previous step logs -``` - -3. Verify network connectivity in workflow - -4. Check if SmartUI CLI step completed successfully - -**Issue: Workflow Times Out** - -**Symptoms**: Workflow execution exceeds time limit - -**Possible Causes**: -- Too many tests running -- Slow test execution -- Network latency -- Resource constraints - -**Solutions**: -1. Increase workflow timeout: -```yaml -timeout-minutes: 60 -``` - -2. Run tests in parallel using matrix: -```yaml -strategy: -matrix: -test-group: [1, 2, 3] -``` - -3. Optimize test execution -4. Split tests across multiple workflows - -**Issue: Dependencies Installation Fails** - -**Symptoms**: npm install or dependency installation fails - -**Possible Causes**: -- Network issues -- Package registry problems -- Version conflicts -- Node version mismatch - -**Solutions**: -1. Use specific Node version: -```yaml -- uses: actions/setup-node@v3 -with: -node-version: '18' -``` - -2. Clear npm cache: -```yaml -- run: npm cache clean --force -``` - -3. Use package-lock.json for consistent installs - -4. Check for version conflicts in package.json - -**Issue: SmartUI CLI Not Found** - -**Symptoms**: `npx smartui` command fails with command not found" - -**Possible Causes**: -- Node.js not installed -- npm not available -- PATH issues - -**Solutions**: -1. Ensure Node.js setup step is included: -```yaml -- uses: actions/setup-node@v3 -with: -node-version: '18' -``` - -2. Verify npm is available: -```yaml -- run: npm --version -``` - -3. Install SmartUI CLI explicitly: -```yaml -- run: npm install -g @lambdatest/smartui-cli -``` - -**Getting Help** - -If you encounter issues not covered here: - -- Review [GitHub Actions Documentation](https://docs.github.com/en/actions) -- Check [SmartUI CLI Documentation](/support/docs/smartui-cli) for CLI-specific issues -- Visit [TestMu AI Support](https://www.testmuai.com/support) for additional resources -- Contact support at support@testmuai.com or use [24/7 Chat Support](https://www.testmuai.com/support) - -## Additional Resources - -- [SmartUI CLI Documentation](/support/docs/smartui-cli) -- [GitHub Actions Documentation](https://docs.github.com/en/actions) -- [Project Settings](/support/docs/smartui-project-settings) -- [Running Your First Project](/support/docs/smartui-running-your-first-project) diff --git a/static/docs/smartui-with-gitlab.md b/static/docs/smartui-with-gitlab.md deleted file mode 100644 index 34e847b51..000000000 --- a/static/docs/smartui-with-gitlab.md +++ /dev/null @@ -1,259 +0,0 @@ -# GitLab Pipeline Integration with SmartUI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -A GitLab Pipeline automates building, testing, and deploying code changes. It's essential for rapid development, improved code quality, and faster delivery by catching errors early and reducing manual work. - -This document will show you how to integrate GitLab Pipeline with SmartUI to shorten your test cycles. - -## Steps to Integrate GitLab Pipeline with SmartUI -To integrate GitLab Pipeline with SmartUI, follow the below steps. You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** -Download or Clone the code sample from the TestMu AI GitHub repository to run the tests on the SmartUI. - - View on GitHub - -### Step 1: Setup your Projects in GitLab -- Click on the **New Project** > **Run CI/CD for external repository**. - -- Enter your repository URL and click on Create Project. - -### Step 2: Create a New Workflow -- Navigate to the **Build** section > **Pipelines**. - -- Select your required template for CI/CD workflow file. For the demo we are using the Test template. -- Now, write your workflow YAML file. Here is the sample file for your reference. -- Commit this yaml file in your repository and make the required changes in your code to automatically trigger the pipeline. - -```yaml reference title=".gitlab-ci.yml" -https://github.com/amanchopra1905/smartui-ci-cd-integrations/blob/gitlab/.gitlab-ci.yml -``` - -You can also store your *LT_USERNAME*, *LT_ACCESS_KEY* and *PROJECT_TOKEN* as secrets in your GitLab project repository. - -### Step 3: Check the output - -- After triggering the workflow, check your results in the [SmartUI Dashboard](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/projects) - -## Best Practices - -**Secret Management** - -- Never commit credentials to repository -- Use GitLab CI/CD Variables for all sensitive data -- Rotate secrets regularly -- Use different secrets for different environments - -**Pipeline Optimization** - -- Use parallel jobs for faster execution -- Cache dependencies to speed up pipelines -- Only run visual tests on relevant branches -- Set up pipeline conditions to avoid unnecessary runs - -**Example:** -```yaml -only: -- main -- develop -- merge_requests -``` - -**Build Naming** - -- Use meaningful build names that include branch/commit info -- Include commit SHA for traceability -- Use consistent naming conventions - -**Example:** -```yaml -variables: -BUILD_NAME: MR-$CI_MERGE_REQUEST_IID-$CI_COMMIT_SHORT_SHA" -``` - -**Error Handling** - -- Set up proper error handling in pipelines -- Use pipeline status checks -- Configure notifications for failures -- Add retry logic for flaky tests - -**Resource Management** - -- Limit concurrent pipeline runs -- Clean up old builds regularly -- Monitor pipeline execution time -- Optimize test execution order - -**Resource Management** - -- Limit concurrent pipeline runs -- Clean up old builds regularly -- Monitor pipeline execution time -- Optimize test execution order - -## Troubleshooting - -**Issue: Pipeline Fails with "Variable Not Found"** - -**Symptoms**: Pipeline fails with error about missing CI/CD variables - -**Possible Causes**: -- Variables not created in GitLab project -- Variable names don't match -- Variables not accessible to pipeline - -**Solutions**: -1. Verify variables exist in project settings: - - Go to Settings → CI/CD → Variables - - Check `LT_USERNAME`, `LT_ACCESS_KEY`, and `PROJECT_TOKEN` exist - -2. Ensure variable names match exactly (case-sensitive) - -3. Check variable scope (project, group, or instance level) - -4. Verify variables are not protected if needed for protected branches - -**Issue: PROJECT_TOKEN Not Available** - -**Symptoms**: Pipeline prompts for PROJECT_TOKEN or token not found - -**Possible Causes**: -- PROJECT_TOKEN not set as CI/CD variable -- Variable not passed to job -- Variable masked or protected incorrectly - -**Solutions**: -1. Add PROJECT_TOKEN as GitLab CI/CD Variable - -2. Pass variable to job: -```yaml -variables: -PROJECT_TOKEN: $PROJECT_TOKEN -``` - -3. Check variable is not masked if you need to see it in logs - -4. Verify variable scope includes your branch - -**Issue: Tests Run But No Results in Dashboard** - -**Symptoms**: Pipeline completes but screenshots don't appear in SmartUI - -**Possible Causes**: -- Incorrect PROJECT_TOKEN -- Project name mismatch -- Network issues -- Pipeline job failure - -**Solutions**: -1. Verify PROJECT_TOKEN is correct: - - Check token in SmartUI Project Settings - - Ensure token includes project ID prefix - -2. Check pipeline logs for errors: -```yaml -after_script: -- if [ $CI_JOB_STATUS == 'failed' ]; then cat job.log; fi -``` - -3. Verify network connectivity in pipeline - -4. Check if SmartUI CLI step completed successfully - -**Issue: Pipeline Times Out** - -**Symptoms**: Pipeline execution exceeds time limit - -**Possible Causes**: -- Too many tests running -- Slow test execution -- Network latency -- Resource constraints - -**Solutions**: -1. Increase pipeline timeout: -```yaml -default: -timeout: 1h -``` - -2. Run tests in parallel using parallel jobs: -```yaml -parallel: -matrix: -- TEST_GROUP: [1, 2, 3] -``` - -3. Optimize test execution -4. Split tests across multiple pipeline stages - -**Issue: Dependencies Installation Fails** - -**Symptoms**: npm install or dependency installation fails - -**Possible Causes**: -- Network issues -- Package registry problems -- Version conflicts -- Node version mismatch - -**Solutions**: -1. Use specific Node version: -```yaml -image: node:18 -``` - -2. Clear npm cache: -```yaml -before_script: -- npm cache clean --force -``` - -3. Use package-lock.json for consistent installs - -4. Check for version conflicts in package.json - -**Issue: SmartUI CLI Not Found** - -**Symptoms**: `npx smartui` command fails with command not found" - -**Possible Causes**: -- Node.js not available in image -- npm not available -- PATH issues - -**Solutions**: -1. Ensure Node.js is available: -```yaml -image: node:18 -``` - -2. Verify npm is available: -```yaml -before_script: -- npm --version -``` - -3. Install SmartUI CLI explicitly: -```yaml -before_script: -- npm install -g @lambdatest/smartui-cli -``` - -**Getting Help** - -If you encounter issues not covered here: - -- Review [GitLab CI/CD Documentation](https://docs.gitlab.com/ee/ci/) -- Check [SmartUI CLI Documentation](/support/docs/smartui-cli) for CLI-specific issues -- Visit [TestMu AI Support](https://www.testmuai.com/support) for additional resources -- Contact support at support@testmuai.com or use [24/7 Chat Support](https://www.testmuai.com/support) - -## Additional Resources - -- [SmartUI CLI Documentation](/support/docs/smartui-cli) -- [GitLab CI/CD Documentation](https://docs.gitlab.com/ee/ci/) -- [Project Settings](/support/docs/smartui-project-settings) -- [Running Your First Project](/support/docs/smartui-running-your-first-project) diff --git a/static/docs/smartui-with-netlify.md b/static/docs/smartui-with-netlify.md deleted file mode 100644 index 3b9e4853f..000000000 --- a/static/docs/smartui-with-netlify.md +++ /dev/null @@ -1,234 +0,0 @@ -# SmartUI Netlify Integration With TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Netlify is a cloud computing company that offers hosting and serverless backend services for web applications and static websites. It allows developers to deploy their websites and applications quickly and easily, with features such as continuous deployment, serverless functions, and global CDN. - -This document will show you how to integrate Netlify Pipeline with SmartUI. - -## Prerequisite - -1. A TestMu AI account. If you don't have an account, [sign up for free](https://www.testmuai.com/register/?redirectTo=https://accounts.lambdatest.com/dashboard). - -2. You need to have a site deployed in your Netlify account. - -## Steps to Setup the Netlify Plugin - -### Step 1: Configure your TestMu AI Integration on Netlify Dashboard - -- Go to your Netlify Dashboard. -- Click on the Integrations tab, search for **TestMu AI** and click on the **Enable** button. - -- You need to provide your configuration details - - Enter your **Username** and **Access Key** from the **TestMu AI Dashboard**. - - Enter your **Smart-UI Project Name** - -For the Smart-UI Project Name, you can enter any project name. Even if you have not set up any projects in the SmartUI Dashboard. - -You can also enter your already-existing smart-UI project name. - -- Click on the **Save** button - -### Step 2: Deploy the Site for which you want to compare the results - -- Go to the **Deploys** section, click on **Deploy site** button and deploy your project. - -- This deployment is before any changes you have made and will be your **Baseline** image. -- Now you need to make the required changes in your project and push the changes. -- Visit the Netlify Dashboard again, and re-deploy your project. - -Your changes will be triggered and deployed. - -### Step 4: Compare the Changes Smart-UI Dashboard - -- Go to your Smart-UI Dashbard. You will notice there is already a project with the same name you entered while configuring in **Step 1** - -- Go on that project. Click on the latest build. You can now compare the changes done to your project with-respect-to the provious build. - -## Best Practices - -**Project Naming** - -- Use descriptive, consistent names for SmartUI projects -- Match project names with your Netlify site names for easy identification -- Avoid special characters that might cause issues -- Use consistent naming conventions across deployments - -**Baseline Management** - -- Establish baseline from stable deployment -- Review baseline before using for comparisons -- Update baseline when intentional changes are made -- Document baseline deployment for reference - -**Deployment Strategy** - -- Deploy baseline before making changes -- Test changes in staging before production -- Use meaningful deployment messages -- Tag important deployments - -**Integration Configuration** - -- Keep TestMu AI credentials secure -- Use Netlify's environment variables for credentials -- Rotate credentials regularly -- Use different projects for different environments - -**Comparison Workflow** - -- Review visual differences carefully -- Approve intentional changes promptly -- Document reasons for baseline updates -- Use SmartUI dashboard for detailed analysis - -**Comparison Workflow** - -- Review visual differences carefully -- Approve intentional changes promptly -- Document reasons for baseline updates -- Use SmartUI dashboard for detailed analysis - -## Troubleshooting - -**Issue: Screenshots Not Appearing in Dashboard** - -**Symptoms**: Deployments complete but no screenshots appear in SmartUI dashboard - -**Possible Causes**: -- Incorrect project name in Netlify integration -- TestMu AI credentials incorrect -- Integration not enabled -- Network connectivity issues - -**Solutions**: -1. Verify project name matches exactly (case-sensitive) in Netlify integration settings - -2. Check TestMu AI credentials: - - Go to Netlify Dashboard → Integrations → TestMu AI - - Verify username and access key are correct - - Re-enter credentials if needed - -3. Ensure integration is enabled: - - Check integration status in Netlify dashboard - - Re-enable if disabled - -4. Check network connectivity to TestMu AI servers - -5. Review Netlify deployment logs for errors - -**Issue: Project Not Found" Error** - -**Symptoms**: Error indicating SmartUI project cannot be found - -**Possible Causes**: -- Project name typo in Netlify integration -- Project doesn't exist in SmartUI dashboard -- Wrong account credentials - -**Solutions**: -1. Verify project exists in SmartUI dashboard: - - Log into [SmartUI Dashboard](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) - - Check if project with specified name exists - - Create project if it doesn't exist - -2. Copy project name directly from SmartUI dashboard - -3. Update project name in Netlify integration settings - -4. Check credentials match the account with the project - -**Issue: Integration Not Triggering** - -**Symptoms**: Deployments don't trigger SmartUI comparisons - -**Possible Causes**: -- Integration not properly configured -- Integration disabled -- Deployment not triggering integration -- Configuration errors - -**Solutions**: -1. Verify integration is enabled: - - Go to Netlify Dashboard → Integrations → TestMu AI - - Ensure integration shows as Enabled" - -2. Check integration configuration: - - Verify all required fields are filled - - Check for any error messages - - Re-save configuration if needed - -3. Test with a new deployment: - - Make a small change to trigger deployment - - Check if integration triggers - -4. Review Netlify deployment logs for integration errors - -**Issue: Baseline Not Set Correctly** - -**Symptoms**: Comparisons show unexpected differences or baseline issues - -**Possible Causes**: -- Baseline deployment not completed -- Wrong deployment used as baseline -- Baseline not approved -- Deployment state issues - -**Solutions**: -1. Verify baseline deployment: - - Check SmartUI dashboard for baseline build - - Ensure baseline was from stable deployment - - Verify baseline shows as approved - -2. Set new baseline if needed: - - Use stable deployment as baseline - - Approve baseline in SmartUI dashboard - -3. Review deployment history: - - Check which deployment was used as baseline - - Verify deployment was successful - -**Issue: Comparison Results Unexpected** - -**Symptoms**: Visual differences don't match expected changes - -**Possible Causes**: -- Dynamic content causing false positives -- Deployment timing issues -- Content not fully loaded -- Configuration issues - -**Solutions**: -1. Review comparison in SmartUI dashboard: - - Check diff view for actual differences - - Verify differences are expected - -2. Handle dynamic content: - - See [Handling Dynamic Data](/support/docs/smartui-handle-dynamic-data) - - Configure ignoreDOM/selectDOM if needed - -3. Check deployment timing: - - Ensure deployment completed fully - - Wait for all assets to load - -4. Review project settings: - - Check pixel threshold settings - - Adjust comparison settings if needed - -**Getting Help** - -If you encounter issues not covered here: - -- Review the [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) for detailed solutions -- Check [SmartUI Configuration Options](/support/docs/smartui-sdk-config-options) documentation -- See [Handling Dynamic Data](/support/docs/smartui-handle-dynamic-data) for dynamic content issues -- Visit [TestMu AI Support](https://www.testmuai.com/support) for additional resources -- Contact support at support@testmuai.com or use [24/7 Chat Support](https://www.testmuai.com/support) - -## Additional Resources - -- [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) -- [Project Settings](/support/docs/smartui-project-settings) - Configure comparison settings -- [Baseline Management](/support/docs/smartui-baseline-management) - Learn how to manage baselines -- [Handling Dynamic Data](/support/docs/smartui-handle-dynamic-data) - Handle dynamic content -- [Running Your First Project](/support/docs/smartui-running-your-first-project) - Get started with SmartUI diff --git a/static/docs/smartui-with-semaphore.md b/static/docs/smartui-with-semaphore.md deleted file mode 100644 index 46f5a1070..000000000 --- a/static/docs/smartui-with-semaphore.md +++ /dev/null @@ -1,269 +0,0 @@ -# Semaphore Pipeline Integration with SmartUI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Semaphore is a modern CI/CD (Continuous Integration and Continuous Delivery) platform designed for developers to automate the process of building, testing, and deploying software. - -This guide explains how to integrate your project with the Semaphore CI/CD pipeline to trigger visual regression testing with TestMu AI SmartUI whenever changes are made to your repository. - -## Steps to Integrate Semaphore Pipeline with SmartUI -To integrate Semaphore Pipeline with SmartUI, follow the below steps. You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** -Download or Clone the code sample from the TestMu AI GitHub repository to run the tests on the SmartUI. - - View on GitHub - -### Step 1: Create your Project -- Click on the **Create New** >> **Choose Repository**. -- Select your desired repository from your VCS (GitHub or Bitbucket) - -### Step 2: Setup your Workflow -- Select your desired build tool and edit your workflow as per your requirement. A sample workflow is given for your reference: -- Click on **Run the Workflow** button to trigger your tests -```yaml reference title="semaphore.yml" -https://github.com/amanchopra1905/smartui-ci-cd-integrations/blob/semaphore/.semaphore/semaphore.yml -``` - -You can also store your *LT_USERNAME*, *LT_ACCESS_KEY* and *PROJECT_TOKEN* as secrets in your Semaphore project repository. - -### Step 3: Check the output - -- After triggering the workflow, check your results in the [SmartUI Dashboard](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/projects) - -## Best Practices - -**Secret Management** - -- Never commit credentials to repository -- Use Semaphore Secrets for all sensitive data -- Rotate secrets regularly -- Use different secrets for different environments - -**Pipeline Optimization** - -- Use parallel blocks for faster execution -- Cache dependencies to speed up pipelines -- Only run visual tests on relevant branches -- Set up pipeline conditions to avoid unnecessary runs - -**Example:** -```yaml -auto_cancel: -running: -when: branch != 'main'" -``` - -**Build Naming** - -- Use meaningful build names that include branch/commit info -- Include commit SHA for traceability -- Use consistent naming conventions - -**Example:** -```yaml -env_vars: -- name: BUILD_NAME -value: ${SEMAPHORE_GIT_BRANCH}-${SEMAPHORE_GIT_SHA}" -``` - -**Error Handling** - -- Set up proper error handling in pipelines -- Use pipeline status checks -- Configure notifications for failures -- Add retry logic for flaky tests - -**Resource Management** - -- Limit concurrent pipeline runs -- Clean up old builds regularly -- Monitor pipeline execution time -- Optimize test execution order - -**Resource Management** - -- Limit concurrent pipeline runs -- Clean up old builds regularly -- Monitor pipeline execution time -- Optimize test execution order - -## Troubleshooting - -**Issue: Pipeline Fails with "Secret Not Found"** - -**Symptoms**: Pipeline fails with error about missing secrets - -**Possible Causes**: -- Secrets not created in Semaphore project -- Secret names don't match -- Secrets not accessible to pipeline -- Secret scope issues - -**Solutions**: -1. Verify secrets exist in project settings: - - Go to Project Settings → Secrets - - Check `LT_USERNAME`, `LT_ACCESS_KEY`, and `PROJECT_TOKEN` exist - -2. Ensure secret names match exactly (case-sensitive) - -3. Check secret scope (project or organization level) - -4. Verify secrets are accessible to the pipeline - -**Issue: PROJECT_TOKEN Not Available** - -**Symptoms**: Pipeline prompts for PROJECT_TOKEN or token not found - -**Possible Causes**: -- PROJECT_TOKEN not set as secret -- Secret not passed to job -- Secret scope issues - -**Solutions**: -1. Add PROJECT_TOKEN as Semaphore Secret - -2. Pass secret to job: -```yaml -env_vars: -- name: PROJECT_TOKEN -value: ${PROJECT_TOKEN} -``` - -3. Check secret is accessible to the pipeline - -4. Verify secret scope includes your project - -**Issue: Tests Run But No Results in Dashboard** - -**Symptoms**: Pipeline completes but screenshots don't appear in SmartUI - -**Possible Causes**: -- Incorrect PROJECT_TOKEN -- Project name mismatch -- Network issues -- Pipeline job failure - -**Solutions**: -1. Verify PROJECT_TOKEN is correct: - - Check token in SmartUI Project Settings - - Ensure token includes project ID prefix - -2. Check pipeline logs for errors: -```yaml -- name: Check Logs -commands: -- cat /tmp/*.log || true -``` - -3. Verify network connectivity in pipeline - -4. Check if SmartUI CLI step completed successfully - -**Issue: Pipeline Times Out** - -**Symptoms**: Pipeline execution exceeds time limit - -**Possible Causes**: -- Too many tests running -- Slow test execution -- Network latency -- Resource constraints - -**Solutions**: -1. Increase pipeline timeout in Semaphore settings - -2. Run tests in parallel using parallel blocks: -```yaml -blocks: -- name: Test Group 1 -task: -jobs: -- name: Run Tests -commands: -- npx smartui exec -- -- name: Test Group 2 -task: -jobs: -- name: Run Tests -commands: -- npx smartui exec -- -``` - -3. Optimize test execution -4. Split tests across multiple pipeline blocks - -**Issue: Dependencies Installation Fails** - -**Symptoms**: npm install or dependency installation fails - -**Possible Causes**: -- Network issues -- Package registry problems -- Version conflicts -- Node version mismatch - -**Solutions**: -1. Use specific Node version: -```yaml -prologue: -commands: -- nvm use 18 -``` - -2. Clear npm cache: -```yaml -commands: -- npm cache clean --force -- npm install -``` - -3. Use package-lock.json for consistent installs - -4. Check for version conflicts in package.json - -**Issue: SmartUI CLI Not Found** - -**Symptoms**: `npx smartui` command fails with command not found" - -**Possible Causes**: -- Node.js not installed -- npm not available -- PATH issues - -**Solutions**: -1. Ensure Node.js is available: -```yaml -prologue: -commands: -- nvm use 18 -``` - -2. Verify npm is available: -```yaml -commands: -- npm --version -``` - -3. Install SmartUI CLI explicitly: -```yaml -commands: -- npm install -g @lambdatest/smartui-cli -``` - -**Getting Help** - -If you encounter issues not covered here: - -- Review [Semaphore Documentation](https://docs.semaphoreci.com/) -- Check [SmartUI CLI Documentation](/support/docs/smartui-cli) for CLI-specific issues -- Visit [TestMu AI Support](https://www.testmuai.com/support) for additional resources -- Contact support at support@testmuai.com or use [24/7 Chat Support](https://www.testmuai.com/support) - -## Additional Resources - -- [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) -- [SmartUI CLI Documentation](/support/docs/smartui-cli) -- [Semaphore Documentation](https://docs.semaphoreci.com/) -- [Project Settings](/support/docs/smartui-project-settings) -- [Running Your First Project](/support/docs/smartui-running-your-first-project) diff --git a/static/docs/smartui-with-travis-ci.md b/static/docs/smartui-with-travis-ci.md deleted file mode 100644 index e00d4e033..000000000 --- a/static/docs/smartui-with-travis-ci.md +++ /dev/null @@ -1,280 +0,0 @@ -# Travis CI Pipeline Integration with SmartUI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Travis CI is one of the most popular CI/CD platforms, known for its simplicity and support for open-source projects. It provides continuous integration for projects hosted on GitHub and Bitbucket. - -This guide explains how to integrate your project with the Travis CI CI/CD pipeline to trigger visual regression testing with TestMu AI SmartUI whenever changes are made to your repository. - -## Steps to Integrate Travis CI Pipeline with SmartUI -To integrate Travis CI Pipeline with SmartUI, follow the below steps. You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** -Download or Clone the code sample from the TestMu AI GitHub repository to run the tests on the SmartUI. - - View on GitHub - -### Step 1: Set Up Your Repository -Ensure your project is hosted in GitHub, or any supported repository. - -### Step 2: Create a New Pipeline: - -- Navigate to Pipelines in your TravisCI project. -- Select New Pipeline and connect your repository. - -### Step 3: Add Environment Variables - -Go to Pipeline Settings > Variables. Add the following variables: -- `LT_USERNAME`: Your TestMu AI username. -- `LT_ACCESS_KEY`: Your TestMu AI access key. - -### Step 4: Setup your Workflow - -```yaml title=".travis.yml" -language: node_js -node_js: -- "16" - -env: -global: -- LT_USERNAME=${LT_USERNAME} -- LT_ACCESS_KEY=${LT_ACCESS_KEY} - -script: -- echo "Installing SmartUI CLI" -- npm install @lambdatest/smartui-cli -- echo "Running SmartUI tests" -- npx smartui --version -- npx smartui config:create smartui-web.json -- npx smartui --config smartui-web.json exec -- mvn --quiet test -D suite=sdk-cloud.xml -``` - -### Step 5: Check the output - -- After triggering the workflow, check your results in the [SmartUI Dashboard](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/projects) - -## Best Practices - -**Secret Management** - -- Never commit credentials to repository -- Use Travis CI Environment Variables for all sensitive data -- Mark variables as secret to hide values in logs -- Rotate secrets regularly -- Use different secrets for different environments - -**Pipeline Optimization** - -- Use matrix builds for parallel execution -- Cache dependencies to speed up pipelines -- Only run visual tests on relevant branches -- Set up pipeline conditions to avoid unnecessary runs - -**Example:** -```yaml -branches: -only: -- main -- develop -``` - -**Build Naming** - -- Use meaningful build names that include branch/commit info -- Include commit SHA for traceability -- Use consistent naming conventions - -**Example:** -```yaml -env: -global: -- BUILD_NAME="${TRAVIS_BRANCH}-${TRAVIS_COMMIT:0:7}" -``` - -**Error Handling** - -- Set up proper error handling in pipelines -- Use pipeline status checks -- Configure notifications for failures -- Add retry logic for flaky tests - -**Resource Management** - -- Limit concurrent pipeline runs -- Clean up old builds regularly -- Monitor pipeline execution time -- Optimize test execution order - -**Resource Management** - -- Limit concurrent pipeline runs -- Clean up old builds regularly -- Monitor pipeline execution time -- Optimize test execution order - -## Troubleshooting - -**Issue: Pipeline Fails with "Variable Not Found"** - -**Symptoms**: Pipeline fails with error about missing environment variables - -**Possible Causes**: -- Variables not created in Travis CI project -- Variable names don't match -- Variables not accessible to build -- Variable scope issues - -**Solutions**: -1. Verify variables exist in repository settings: - - Go to Repository Settings → Environment Variables - - Check `LT_USERNAME`, `LT_ACCESS_KEY`, and `PROJECT_TOKEN` exist - -2. Ensure variable names match exactly (case-sensitive) - -3. Check variable scope (public or private) - -4. Verify variables are not displayed in logs if marked as secret - -**Issue: PROJECT_TOKEN Not Available** - -**Symptoms**: Pipeline prompts for PROJECT_TOKEN or token not found - -**Possible Causes**: -- PROJECT_TOKEN not set as environment variable -- Variable not passed to build -- Variable marked as secret incorrectly - -**Solutions**: -1. Add PROJECT_TOKEN as Travis CI Environment Variable - -2. Pass variable to build: -```yaml -env: -global: -- PROJECT_TOKEN=$PROJECT_TOKEN -``` - -3. Check variable is accessible to the build - -4. Verify variable scope includes your repository - -**Issue: Tests Run But No Results in Dashboard** - -**Symptoms**: Pipeline completes but screenshots don't appear in SmartUI - -**Possible Causes**: -- Incorrect PROJECT_TOKEN -- Project name mismatch -- Network issues -- Pipeline build failure - -**Solutions**: -1. Verify PROJECT_TOKEN is correct: - - Check token in SmartUI Project Settings - - Ensure token includes project ID prefix - -2. Check build logs for errors: -```yaml -after_failure: -- cat /tmp/*.log || true -``` - -3. Verify network connectivity in build - -4. Check if SmartUI CLI step completed successfully - -**Issue: Pipeline Times Out** - -**Symptoms**: Pipeline execution exceeds time limit - -**Possible Causes**: -- Too many tests running -- Slow test execution -- Network latency -- Resource constraints - -**Solutions**: -1. Increase build timeout in Travis CI settings - -2. Run tests in parallel using matrix: -```yaml -matrix: -include: -- env: TEST_GROUP=1 -- env: TEST_GROUP=2 -``` - -3. Optimize test execution -4. Split tests across multiple build jobs - -**Issue: Dependencies Installation Fails** - -**Symptoms**: npm install or dependency installation fails - -**Possible Causes**: -- Network issues -- Package registry problems -- Version conflicts -- Node version mismatch - -**Solutions**: -1. Use specific Node version: -```yaml -node_js: -- 18" -``` - -2. Clear npm cache: -```yaml -before_install: -- npm cache clean --force -``` - -3. Use package-lock.json for consistent installs - -4. Check for version conflicts in package.json - -**Issue: SmartUI CLI Not Found** - -**Symptoms**: `npx smartui` command fails with command not found" - -**Possible Causes**: -- Node.js not installed -- npm not available -- PATH issues - -**Solutions**: -1. Ensure Node.js is specified: -```yaml -node_js: -- "18" -``` - -2. Verify npm is available: -```yaml -before_script: -- npm --version -``` - -3. Install SmartUI CLI explicitly: -```yaml -before_script: -- npm install -g @lambdatest/smartui-cli -``` - -**Getting Help** - -If you encounter issues not covered here: - -- Review [Travis CI Documentation](https://docs.travis-ci.com/) -- Check [SmartUI CLI Documentation](/support/docs/smartui-cli) for CLI-specific issues -- Visit [TestMu AI Support](https://www.testmuai.com/support) for additional resources -- Contact support at support@testmuai.com or use [24/7 Chat Support](https://www.testmuai.com/support) - -## Additional Resources - -- [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) -- [SmartUI CLI Documentation](/support/docs/smartui-cli) -- [Travis CI Documentation](https://docs.travis-ci.com/) -- [Project Settings](/support/docs/smartui-project-settings) -- [Running Your First Project](/support/docs/smartui-running-your-first-project) diff --git a/static/docs/socks5-proxy-support.md b/static/docs/socks5-proxy-support.md deleted file mode 100644 index 3ef0bea28..000000000 --- a/static/docs/socks5-proxy-support.md +++ /dev/null @@ -1,31 +0,0 @@ -# SOCKS5 Proxy Support in TestMu AI Tunnel - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Overview - -TestMu AI Tunnel offers comprehensive support for SOCKS5 proxy, enabling secure and private connections between your local development environment and the TestMu AI cloud platform. This support is crucial for users operating within networks that require advanced proxy configurations for security and privacy. This document outlines the key features, benefits, and usage of the SOCKS5 proxy support in TestMu AI Tunnel. - -## Key Features - -- **Enhanced Security:** By using a SOCKS5 proxy, TestMu AI Tunnel facilitates more secure data transmission by routing traffic through a proxy server, providing an additional layer of security. -- **Easy Configuration:** The TestMu AI Tunnel can be easily configured to use a SOCKS5 proxy with minimal setup, allowing for straightforward integration into your testing workflow. -- **Bypass Network Restrictions:** The SOCKS5 proxy support helps navigate corporate firewalls and network restrictions, enabling testing of local and internal web applications seamlessly. - -## Usage - -To use TestMu AI Tunnel with a SOCKS5 proxy, follow the steps outlined below: - -1. **Start TestMu AI Tunnel:** Open your command-line interface (CLI) and navigate to the directory where the TestMu AI Tunnel is installed. -2. **Activate SOCKS5 Proxy:** Use the `--proxy-type` flag with the value `socks5` to specify the SOCKS5 proxy type for tunneling. Here is an example command: - -```sh -./LambdaTestTunnel --user YourLambdaTestUsername --key YourLambdaTestAccessKey --proxy-type socks5 -``` - Replace `YourLambdaTestUsername` and `YourLambdaTestAccessKey` with your actual TestMu AI credentials. - -3. **Run Your Tests:** With the tunnel running and configured to use a SOCKS5 proxy, you can now execute your tests against local or internal web applications as if they were accessible on the public internet. - -## Conclusion - -The SOCKS5 proxy support in TestMu AI Tunnel is an essential feature for users who require secure, encrypted traffic transmission and the ability to bypass network restrictions during the testing process. By leveraging this feature, developers and QA professionals can ensure that their web applications are thoroughly tested in environments that closely mimic real-world scenarios. diff --git a/static/docs/specflow-on-hyperexecute-grid.md b/static/docs/specflow-on-hyperexecute-grid.md deleted file mode 100644 index 10cb081a7..000000000 --- a/static/docs/specflow-on-hyperexecute-grid.md +++ /dev/null @@ -1,244 +0,0 @@ -# Running SpecFlow Framework Tests on HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -SpecFlow is a behavior-driven development (BDD) framework for .NET languages, often used with Selenium for automated testing. It enables collaboration between technical and non-technical team members by expressing tests in natural language. - -HyperExecute is an AI Native Test Orchestration Cloud Platform that empowers you to run **end-to-end** tests **quickly** and **efficiently**. It provides Just-in-Time (JIT) testing infrastructure with fast execution **speeds**, **smart orchestration**, and **detailed logs**. - -This guide details how to execute your **Specflow** framework tests on **HyperExecute** via two different methods: - -- [**Using Local System**](/support/docs/specflow-on-hyperexecute-grid/#1-testing-using-local-system) - You can use your own local machine to execute tests. -- [**Using Gitpod Platform**](/support/docs/specflow-on-hyperexecute-grid/#2-testing-using-gitpod) - Execute tests using GitPod. (Requires a [Gitpod](https://gitpod.io/login/) account) - -## 1. Testing Using Local System - -Follow the step-by-step guide to execute your test on HyperExecute. - -### Prerequisites - -To run the Tests on HyperExecute from your Local System, you are required: - -- Your TestMu AI [Username and Access key](/support/docs/hyperexecute-how-to-get-my-username-and-access-key/) -- [HyperExecute YAML](/support/docs/hyperexecute-yaml-version0.2/) file which contains all the necessary instructions. -- [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) in order to initiate a test execution Job . -- Setup the [Environmental Variable](/support/docs/hyperexecute-environment-variable-setup/) - -### Step 1: Configure Your Test Suite - -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** - -Download or Clone the code sample for the JUnit from the TestMu AI GitHub repository to run the tests on the HyperExecute. - - View on GitHub - -If you are using your own project, make sure you update the **Hub endpoint** in your tests file. - -By setting up the Hub endpoint, you establish the communication channel between your tests and the browser nodes, enabling effective test distribution and execution. - -Configure the desired capabilities based on your test requirements. For example: - -```bash -DesiredCapabilities capabilities = new DesiredCapabilities(); -capabilities.SetCapability(CapabilityType.BrowserName, "Chrome"); -capabilities.SetCapability(CapabilityType.Version, "96"); -capabilities.SetCapability(CapabilityType.Platform, "Windows 10"); -``` - -> You can also use **dev** and **beta** browser versions. To generate capabilities for your test requirements, you can use our inbuilt 🔗 Capabilities Generator. - -### Step 2: Setup the CLI in your Test Suite - -After cloning / downloading the sample repo, you need to setup the CLI and the environment variables. - -#### Download the HyperExecute CLI - -The CLI is used for triggering the tests on HyperExecute. It is recommend to download the CLI binary on the host system and keep it in the root directory of the suite to perform the tests on HyperExecute. - -You can download the CLI for your desired platform from the below mentioned links: - -| Platform | HyperExecute CLI | -| ---------| ---------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| MacOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -#### Setup Environment Variable - -Now, you need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in the [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). - -Run the below mentioned commands in your terminal to setup the CLI and the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -### Step 3: Configure YAML in your Test Suite - -Configure your YAML file as per your use cases using **key value** pairs. - -In this sample YAML file, we have mentioned: - -- **version** of the YAML file -- **Timeouts** for executing your project -- **Mode of execution** is [Autosplit](/support/docs/hyperexecute-auto-split-strategy/). You can also opt for [Matrix](/support/docs/hyperexecute-matrix-multiplexing-strategy/) or [Hybrid](/support/docs/hyperexecute-hybrid-strategy/) mode. -- **Pre and Post** commands -- **Reports and Artefacts** that will be generated after the completion of tests -- and other necessary YAML Parameters - -```yaml ---- -version: 0.1 -globalTimeout: 90 -testSuiteTimeout: 90 -testSuiteStep: 90 - -runson: linux - -autosplit: true -retryOnFailure: true - -maxRetries: 1 -concurrency: 3 - -env: -NUGET_PACKAGES: '/home/ltuser/.nuget/packages/' -NUGET_HTTP_CACHE_PATH: '/home/ltuser/.local/share/NuGet/v3-cache' -NUGET_PLUGINS_CACHE_PATH: '/home/ltuser/.local/share/NuGet/plugins-cache' - -pre: -- dotnet restore -- dotnet list OnlySpecTest.csproj package > packages.txt -# - chmod 777 /tmp/NuGetScratch -# - nuget locals all -clear -- dotnet build -c Release - -cacheKey: '{{ checksum "packages.txt" }}' - -post: -- cat yaml/linux/specflow_hyperexecute_autosplit_sample.yaml - -mergeArtifacts: true - -uploadArtefacts: -- name: Execution_Report -path: -- Report/** -- name: Execution_Screenshots -path: -- Screenshots/**/** - -report: true -partialReports: -location: Report/ -type: html -frameworkName: specflow - -# The below testDiscovery command will give the feature name in the respective feature files -testDiscovery: -type: raw -mode: remote -#Parallel execution at feature level -#command: grep -rni 'Features' -e 'Feature:' | sed 's/.*://' - -command: grep -rni 'Features' -e '@' --include=\*.feature | sed 's/.*@//' - -#Parallel execution at feature level -#testRunnerCommand: dotnet test --filter "(Name~$test)" -testRunnerCommand: dotnet test --filter "(Category=$test)" - -jobLabel: [selenium-specflow, linux, autosplit] -``` - -### Step 4: Execute your Test Suite - -> **NOTE :** In case of MacOS, if you get a permission denied warning while executing CLI, simply run **`chmod u+x ./hyperexecute`** to allow permission. In case you get a security popup, allow it from your **System Preferences** → **Security & Privacy** → **General tab**. - -Run the below command in your terminal at the root folder of the project: - -```bash -./hyperexecute --config RELATIVE_PATH_OF_YOUR_YAML_FILE -``` - -OR use this command if you have not exported your username and access key in the step 2. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE `} - -### Step 5: Monitor the Test Execution - -Visit the [HyperExecute Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) and check your Job status. - -### Step 6: Download Artifacts and Reports - -HyperExecute also facilitates the provision to download the [Artifacts](/support/docs/hyperexecute-artifacts/) and [Reports](/support/docs/hyperexecute-reports/) on your local machine. Click on the corresponding button to download your generated artifacts and reports. - -## 2. Testing Using Gitpod - -You can also use the Gitpod platform to execute our sample repository. It will fetch all the sample codebases and trigger the CLI to execute the tests. - -Follow the below steps to run Test using Gitpod: - -**Step 1:** Click '**Open in Gitpod**' button. You will be redirected to Login/Signup page. This button is configured to redirect you to the Gitpod platform where you will be able to execute our sample repository. - -**Step 2:** Login with TestMu AI credentials. Once logged in, a pop-up confirmation will appear, asking you to **'Proceed'** to the Gitpod editor in a new tab. The current tab will display the HyperExecute Dashboard. - -**Step 3:** Choose your preferred editor (we recommend VS Code Editor) - -**Step 4:** As you are running a sample project, Fetching of the Test Scripts, [HyperExecute YAML](/support/docs/deep-dive-into-hyperexecute-yaml/), [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) and Triggering your tests using the `Execution Command` will be automated. - -**Step 5:** Once you see the `Job Link` in the logs, you can visit the [HyperExecute dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) to see the tests getting executed. - -You can also implement [Secret Keys](/support/docs/hyperexecute-how-to-save-and-manage-secrets/) in your YAML file. - -## Navigation in Automation Dashboard - -Every test run on the HyperExecute has a unique *jobId* associated with it. Each *jobId* can in turn constitute single (or multiple) *groupId*(s). You can visit [HyperExecute Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) for checking the status of the test execution. - -You can seamlessly navigate between JobId's and taskId's. You need to click on the *testID* to navigate from the HyperExecute logs to the Automation Dashboard. - -The snapshot below shows the videos, logs and other meta data for that specific *test_ID* - -> For any query or doubt, please feel free to contact us via **24×7 chat support** or you can also drop a mail to **support@testmuai.com**. -Happy testing! - -## Using the SpecFlow Agent Skill with TestMu AI - -The [specflow-skill](https://github.com/LambdaTest/agent-skills/tree/main/specflow-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The specflow-skill package includes: - -``` -specflow-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing SpecFlow Agent Skill - -Install a SpecFlow Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/specflow-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/specflow-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only specflow-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). diff --git a/static/docs/specflow-report.md b/static/docs/specflow-report.md deleted file mode 100644 index 571e6aea0..000000000 --- a/static/docs/specflow-report.md +++ /dev/null @@ -1,24 +0,0 @@ -# SpecFlow Report - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -SpecFlow is a free tool for automating tests using BDD. It's often used to create automation scripts for .NET projects. -This technical document provides a guide on generating SpecFlow reports after executing tests on HyperExecute. - -> We also offer [Custom Specflow report](/support/docs/custom-speclow-report/). - -## Steps to Generate Specflow Reports on HyperExecute - -**Step 1:** Configure all the necessary dependencies and configurations to your project. - -**Step 2:** Configure the report parameters in the HyperExecute YAML file. - -```yaml -report: true -partialReports: -location: YOUR_REPORTS_LOCATION -type: html -frameworkName: specflow -``` - -**Step 3:** Now execute your job by triggering the HyperExecute CLI. You can visit the HyperExecute dashboard to download the report after job completion. diff --git a/static/docs/specflow-with-selenium-running-specflow-automation-scripts-on-testmu-selenium-grid.md b/static/docs/specflow-with-selenium-running-specflow-automation-scripts-on-testmu-selenium-grid.md deleted file mode 100644 index 0461c7a19..000000000 --- a/static/docs/specflow-with-selenium-running-specflow-automation-scripts-on-testmu-selenium-grid.md +++ /dev/null @@ -1,104 +0,0 @@ -# Selenium With SpecFlow - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Run SpecFlow tests on the TestMu AI cloud grid. This guide covers setup, running a sample test, configuring capabilities, and testing locally hosted pages. - -**Sample repo** -All the code used in this guide is available in the sample repository. - - View on GitHub - -## Prerequisites - -Complete the following steps before running SpecFlow Selenium tests. - -1. Download and install **Selenium WebDriver** from the [official website](https://www.selenium.dev/downloads/). -2. Install the latest version of **SpecFlow**. -3. Install the **.Net** framework for developing applications using C#. -4. Download [Selenium WebDriver Language Binding](https://www.selenium.dev/downloads/) for C# and extract them to the appropriate folder. -5. Install a [.NET Core SDK](https://dotnet.microsoft.com/en-us/download) of 2.1 or greater version. -6. Get your TestMu AI Username and Access Key from the [TestMu AI Dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard). - -## Step 1: Clone the Sample Project - -Clone the repository and navigate to the project directory. - -```bash -git clone https://github.com/LambdaTest/SpecFlow-Selenium-Sample -cd SpecFlow-Selenium-Sample -``` - -## Step 2: Set Your Credentials - -Configure your credentials to connect to the TestMu AI Selenium Grid. - -Set TestMu AI Username and Access Key in environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -## Step 3: Configure Your Test Capabilities - -Define browser, version, and OS settings for your test run. - -In the test script, update your test capabilities. This code passes browser, browser version, and operating system information, along with TestMu AI Selenium grid capabilities via the capabilities object. - -```csharp -var browserOptions = new ChromeOptions(); -browserOptions.PlatformName = "Windows 10"; -browserOptions.BrowserVersion = "latest"; - -var ltOptions = new Dictionary -{ -{ "build", "SpecFlow Build" }, -{ "name", "SpecFlow Test" }, -{ "w3c", true } -}; -browserOptions.AddAdditionalOption("LT:Options", ltOptions); -``` - -**Capabilities Generator** -Use the TestMu AI [Capabilities Generator](https://www.testmuai.com/capabilities-generator/) to auto-generate the capabilities class for your test requirements. - -## Step 4: Run the Test - -Execute the SpecFlow test from Visual Studio. - -1. Build the solution by clicking **Build > Build Solution**. -2. Open **Test Explorer** in Visual Studio. - -3. Click **Run** from the Test Explorer to run the sample test. - -To run parallel tests, click **Run All** in Test Explorer. - -## Step 5: View Your Results - -Check the test output on the console and the TestMu AI dashboard. - -Visit the [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/) to view your test results. The dashboard provides: - -- Text logs for each test step -- Screenshots captured during execution -- Video recordings of the full test session - -## Run SpecFlow Tests Using Agent Skills - -Use AI coding assistants to generate and run SpecFlow tests with the TestMu AI Agent Skill. - -The [specflow-skill](https://github.com/LambdaTest/agent-skills/tree/main/specflow-skill) is part of [TestMu AI Agent Skills](https://github.com/LambdaTest/agent-skills/) - structured packages that teach AI coding assistants how to write production-grade test automation. - -Install the skill: - -```bash -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/specflow-skill .claude/skills/ - -# For Cursor / Copilot -cp -r agent-skills/specflow-skill .cursor/skills/ -``` - -Install all available framework skills at once by cloning the repository directly into your tool's skills directory (e.g., `.claude/skills/`, `.cursor/skills/`). diff --git a/static/docs/speedup-espresso.md b/static/docs/speedup-espresso.md deleted file mode 100644 index 30b81028f..000000000 --- a/static/docs/speedup-espresso.md +++ /dev/null @@ -1,103 +0,0 @@ -# Filters for Espresso Tests - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Usually, all the test cases of your Espresso test suite are executed, but there is a way to filter these. You can use Espresso test runner (typically `AndroidJUnitRunner`), which provides you with multiple options to filter the test cases which you want to execute. - -To filter the test cases, you just need to pass the suitable parameters in TestMu AI’s [REST API](/support/docs/getting-started-with-espresso-testing/) request. Refer to the table below to understand how to use various filters provided by TestMu AI. - -Given below is the REST API endpoint: - -```bash -POST /framework/v1/espresso/build -``` - -| Parameters | Description | Values | Datatype | -| -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| `package` | Allows the user to only run specific packages defined | Multiple packages can be passed in comma-separated format. E.g. `["com.example.proverbial.packagename"]` | Array | -| `skipPackage` | Allows the user to run all packages except the ones defined | Multiple packages can be passed in comma-separated format. E.g. `["com.example.proverbial.packagename"]` | Array | -| `class` | Allows the user to only run specific classes defined | Multiple classes can be passed in comma-separated format. E.g. `["com.example.proverbial.packagename.classname"]` | Array | -| `skipClass` | Allows the user to run all classes except the ones defined | Multiple classes can be passed in comma-separated format. E.g. `["com.example.proverbial.packagename.classname"]` | Array | -| `annotation` | Allows the user to only run specific annotations defined | Only a single annotation can be passed. E.g. `"com.example.proverbial.annotation"` | String | -| `skipAnnotation` | Allows the user to run all annotations except the ones defined | Only a single annotation can be passed. E.g. `"com.example.proverbial.annotation"` | String | -| `size` | Allows the user to run only tests that are annotated with the matching size value `@SmallTest, @MediumTest or @LargeTest` | You may specify only one value for this property from the three valid values i.e. `"small", "medium", "large"` | String | - -**Note** -You can not use the following filters simultaneously. -- `class` and `package` -- `class` and `skipClass` -- `package` and `skipPackage` - -**Note** -You can use the following filters simultaneously. -- `annotation` and `skipAnnotation` - -## Some Examples - -Please refer to the example `cURL` requests given below for your reference. - -### 1. To run specifically defined packages - - {`curl --location --request POST 'https://mobile-api.lambdatest.com/framework/v1/espresso/build' \\ - --header 'Authorization: Basic ' \\ - --header 'Content-Type: application/json' \\ - --data-raw '{ - "app" : "lt://APP_ID", - "testSuite": "lt://TestSuite_ID", - "device" : ["Pixel 6-12"], - "queueTimeout": 360, - "IdleTimeout": 150, - "deviceLog": true, - "build" : "Proverbial-Espresso" - "package" : ["com.example.proverbial.packagename1","com.example.proverbial.packagename2"] - }'`} - -``` -{`curl --location --request POST "https://mobile-api.lambdatest.com/framework/v1/espresso/build" --header "Content-Type: application/json" --header "Authorization: Basic " --data-raw "{\"app\" : \"lt://APP_ID\",\"testSuite\": \"lt://APP_ID\",\"device\" : [\"Pixel 6-12\"],\"queueTimeout\": 360,\"IdleTimeout\": 150,\"deviceLog\": true,\"network\": false,\"build\" : \"Proverbial-Espresso\",\"geoLocation\" : \"FR\", \"package\" : [\"com.example.proverbial.packagename1\", \"com.example.proverbial.packagename2\"]}"`} -``` - -### 2. To run specifically defined classes - - {`curl --location --request POST 'https://mobile-api.lambdatest.com/framework/v1/espresso/build' \\ - --header 'Authorization: Basic ' \\ - --header 'Content-Type: application/json' \\ - --data-raw '{ - "app" : "lt://APP_ID", - "testSuite": "lt://TestSuite_ID", - "device" : ["Pixel 6-12"], - "queueTimeout": 360, - "IdleTimeout": 150, - "deviceLog": true, - "build" : "Proverbial-Espresso" - "class" : ["com.example.proverbial.packagename.classname1","com.example.proverbial.packagename.classname2"] - }'`} - -``` -{`curl --location --request POST "https://mobile-api.lambdatest.com/framework/v1/espresso/build" --header "Content-Type: application/json" --header "Authorization: Basic " --data-raw "{\"app\" : \"lt://APP_ID\",\"testSuite\": \"lt://APP_ID\",\"device\" : [\"Pixel 6-12\"],\"queueTimeout\": 360,\"IdleTimeout\": 150,\"deviceLog\": true,\"network\": false,\"build\" : \"Proverbial-Espresso\",\"geoLocation\" : \"FR\", \"class\" : [\"com.example.proverbial.packagename.classname1\",\"com.example.proverbial.packagename.classname2\"]}"`} -``` - -### 3. To run specifically defined annotations - - {`curl --location --request POST 'https://mobile-api.lambdatest.com/framework/v1/espresso/build' \\ - --header 'Authorization: Basic ' \\ - --header 'Content-Type: application/json' \\ - --data-raw '{ - "app": "lt://APP_ID", - "testSuite": "lt://TestSuite_ID", - "device": ["Pixel 6-12"], - "queueTimeout": 360, - "IdleTimeout": 150, - "deviceLog": true, - "build": "Proverbial-Espresso", - "filters": { - "annotation": [ - "com.lambdatest.proverbial.demo1","com.lambdatest.proverbial.demo2" - ] - } - }'`} - -``` -{`curl --location --request POST "https://mobile-api.lambdatest.com/framework/v1/espresso/build" --header "Authorization: Basic " --header "Content-Type: application/json" --data-raw "{\"app\": \"lt://APP_ID\", \"testSuite\": \"lt://APP_ID\", \"device\": [\"Pixel 6-12\"], \"queueTimeout\": 360, \"IdleTimeout\": 150, \"deviceLog\": true, \"network\": false, \"build\": \"Proverbial-Espresso\", \"geoLocation\": \"FR\", \"filters\": {\"annotation\": [\"com.lambdatest.proverbial.demo1\", \"com.lambdatest.proverbial.demo2\"]}}"`} -``` - -This configuration will run tests annotated with multiple annotations, i.e., only the intersection of all mentioned annotations will be considered. diff --git a/static/docs/speedup-xcui.md b/static/docs/speedup-xcui.md deleted file mode 100644 index 283614284..000000000 --- a/static/docs/speedup-xcui.md +++ /dev/null @@ -1,36 +0,0 @@ -# Filters for XCUI Tests - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Usually, all the test cases of your XCUI test suite are executed, but there is a way to filter these. You can specify some selected classes or tests, which provides you with options to filter the test cases which you want to execute. - -To filter the test cases, you just need to pass the suitable parameters in TestMu AI’s REST API request. Refer to the table below to understand how to use various filters provided by TestMu AI. - -Given below is the REST API endpoint: - -```bash -curl --location --request POST 'https://mobile-api.lambdatest.com/framework/v1/xcui/build' \ ---header 'Authorization: Basic BASIC_AUTH_TOKEN' \ ---header 'Content-Type: application/json' \ ---data-raw '{ -"app" : "APP_ID", -"testSuite": "TEST_SUITE_ID", -"device" : ["iPhone 11-14"], -"video" : true, -"queueTimeout": 10800, -"idleTimeout": 150, -"devicelog": true, -"network": false, -"build" : "Proverbial-XCUITest" -}' -``` - -| Parameters | Description | Values | Datatype | -|----------- | ----------- | ------ | -------- | -| `only-testing` | Allows the user to run only those tests/classes provided in the list | Values can be of the following format: className or className/testName. E.g. `["Class1/Test1", "Class2"]` | Array | -| `skip-testing` | Allows the user to run all the tests/classes except the ones provided in the list | Values can be of the following format: className or className/testName. E.g. `["Class1/Test1", "Class2"]` | Array | - -**Note** -You can not use the following filters simultaneously. -- `only-testing` and `skip-testing` -- `xctestplan` and `only-testing`/`skip-testing` diff --git a/static/docs/spirateam-integration.md b/static/docs/spirateam-integration.md deleted file mode 100644 index b8fd887d6..000000000 --- a/static/docs/spirateam-integration.md +++ /dev/null @@ -1,45 +0,0 @@ -# SpiraTeam Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -SpiraTeam is a software tool used for Application Lifecycle Management (ALM). It's designed to help teams manage various aspects of software development, from programs and portfolios to requirements, releases, test cases, issues, baselines, workflows, and tasks. - -It is very useful for managing teams programs, portfolios, requirements, releases, test cases, issues, baselines, workflows, and tasks in one unified environment. - -## Steps to Integrate TestMu AI with SpiraTeam - -### Step 1: Setup Your Accounts - -- Sign in to your TestMu AI account. If you don't have an account, **[Register for FREE !!](https://www.testmuai.com/register/?redirectTo=https://accounts.lambdatest.com/dashboard)**. - -- Create your [**SpiraTeam account**](https://www.inflectra.com/Trial/Process.aspx?catalogItemId=90) and login to your account. - -### Step 2: Configure the Integration - -- On the [TestMu AI Dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard), click on the **Settings** option and select the **Integrations**. - -- In the **Bug Tracker Section**, select the **SpiraTeam** and click on the **Connect** button. - -- Enter the configuration details and click on **Install** button. - - - **Username :** Go to your SpiraTeam account profile to find your Username. - - **SpiraTeam URL :** Enter your organization URL using for the integration. - - **RSS / API Key :** In your SpiraTeam profile, toggle the **Enable RSS Feeds** to **yes** and copy the **RSS / API Key** and click on the Save button. - -### Step 3: Create Your Incident Ticket - -- Go to the **Automation** section. - -- Click on the desired test for which you want to create the incident ticket. - -- Click on **Create an issue**. - -- Configure the ticket details as shown below and click on **Create Issue** button. - -### Step 4: (Optional) Complete Your Ticket Configuration if needed - -- Visit your SpiraTeam profile. - -- Go to your desired dashboard in which you have created the ticket in the above step. - -- Update your ticket information here and click on the **Save** button. diff --git a/static/docs/sso-azure-integration.md b/static/docs/sso-azure-integration.md deleted file mode 100644 index 4689dccce..000000000 --- a/static/docs/sso-azure-integration.md +++ /dev/null @@ -1,58 +0,0 @@ -# TestMu AI SSO & Azure AD Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI allows you to integrate an external user directory with your TestMu AI Organization through an IDP so your team could leverage the Single Sign-on setup. This document will help you integrate your TestMu AI organization with your Azure AD directory to configure SSO(Single sign-on). After you integrate your Azure AD instance with your TestMu AI for Single Sign-on, you can benefit from the following features: - -* Monitor who access to TestMu AI -* Manage your account in Azure Active Directory. -* Sign-in directly to the TestMu AI with Azure AD credentials. - -To know more, refer [How SSO is implemented in Azure](https://docs.microsoft.com/azure/active-directory/active-directory-appssoaccess-whatis). - -## Prerequisites - -To enable TestMu AI SSO through Azure AD integration, you will need: - -* Azure AD Subscription. If you haven't subscribed, you can subscribe by creating a [free Azure account](https://azure.microsoft.com/en-us/free/). -* An Enterprise plan with TestMu AI. -* The Admin of your TestMu AI organization needs to raise a request for enabling SSO. They could drop us an email over [support@testmuai.com](mailto:support@testmuai.com) or could reach out to us via our **24/7 in-app chat support**. -* The Admin level access to your TestMu AI organization. -* The Admin level access of your organization in Azure AD. - -## How To Configure TestMu AI & Azure AD Integration - -**Step 1:** To begin, you will need to login with your [Microsoft Azure](https://portal.azure.com/#home) account. - -**Step 2:** Navigate to **Manage Azure Active Directory**. - -**Step 3:** Select the **Enterprise Application** from the left sidebar. - -**Step 4:** Click on **New Application**. - -**Step 5:** Then click on **Create your own application**. - -**Step 6:** Now navigate back to the Enterprise applications category. Select the application that you just created and then click on **Set up single sign on**. - -**Step 7:** Go to TestMu AI Authentication & SSO settings and click on **Setup SSO**. - -**Step 8:** Enter Connection Name and Click on **Add New Connection**. - -**Step 9:** Select **Azure AD** from the list of Identity Providers. - -**Step 10:** Now copy the details required for setting up SSO connection in your **Identity Provider**. - -**Step 11:** Click on **Basic SAML Configuration**. Enter your SSO identifier ID and Reply URL information that you copied from TestMu AI SSO setup page and click on **Save**. - -**Step 12:** Copy the App Federation Metadata URL and paste it in the **Metadata URL** field in the TestMu AI SSO setup page. - -**Step 13:** Fill the required fields and click on **Create Connection**: - -* SSO Domains (Comma-separated list of the domains that can be authenticated in the Identity Provider.) -* Have Metadata URL (Select this option if you have metadata file URL of your Identity Provider.) -* Metadata URL (Enter the metadata file URL of your Identity Provider.) -Create Connection with Metadata URL - -**Step 14:** Once the connection is created, you will be redirected to the SSO settings page. From here you can Manage SSO Connection. - -> That’s all you need to know TestMu AI and Azure AD integration. In case you have any questions please feel free to reach out to us via the **24/7 chat support** or email us over [support@testmuai.com](mailto:support@testmuai.com). diff --git a/static/docs/sub-organizations.md b/static/docs/sub-organizations.md deleted file mode 100644 index a07ac6efc..000000000 --- a/static/docs/sub-organizations.md +++ /dev/null @@ -1,83 +0,0 @@ -# Sub Organizations - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Sub-organizations let you segregate test data across multiple business units within your TestMu AI account. You can also distribute the concurrency you have purchased among sub-organizations, so each operates independently with its own administrators, users, and settings. - -This feature is available exclusively on the **Enterprise plan**. To enable it for your organization, please reach out to your account manager. - -## Things to Note - -Before setting up sub-organizations, review the following constraints: - -- **Single membership:** A user can belong to only one organization or sub-organization at a time. Cross-membership is not supported. -- **Concurrency ceiling:** You can allocate the concurrency you have subscribed to across sub-organizations. The total concurrency allocated across all sub-organizations cannot exceed the concurrency available at the root organization level, on a per-product basis. - -## Create Your Sub-Organization - -**Step 1:** Sign in to your TestMu AI account. If you don't have an account, register for free. - -**Step 2:** Navigate to **Settings** and select **Organization Settings** from the dropdown. - -**Step 3:** Open the **Sub Organization** tab. - -**Step 4:** Click **Create**. - -**Step 5:** Enter a name for your sub-organization and click **Create**. - -Your **sub-organization** has been created and is ready to configure. - -## Manage Your Sub-Organization - -To manage a sub-organization, click the **three-dot (⋮) menu** next to its name. From this menu, you can **view, edit, or delete** the sub-organization. - -## Manage Users in Your Sub-Organization - -### Invite a User - -To **invite users** to your sub-organization, open the newly created sub-organization from the list. - -**Step 1:** Click **Invite**. - -**Step 2:** Enter the **email address** of the user you want to add, select a **role** from the dropdown, and click **Submit**. - -### Manage Existing Users - -After users have joined, you can manage them directly from the **Users** tab inside the sub-organization. - -Each user row has a **three-dot (⋮) menu** that provides the following actions: - -- **View Username & Access Key:** View the user's username and access key for use in automation or scripts. -- **Change Password:** Reset the password for the selected user. -- **Login As:** Sign in to TestMu AI as the selected user (see [Login As](#login-as) below). -- **Delete:** Remove the user from the sub-organization. - -### Login As - -The **Login As** action lets administrators sign in to TestMu AI as another user within the sub-organization. This is useful for troubleshooting access issues, validating role-based permissions, or reproducing problems reported by a team member. - -**Step 1:** Open the **three-dot (⋮) menu** next to the user and select **Login As**. - -**Step 2:** You are now signed in as the selected user and will see TestMu AI exactly as they do. An **Assumed Identity** indicator appears in the top-right corner showing whose account you are using. - -**Step 3:** To return to your own account, click the **Assumed Identity** indicator and select **Go Back to your session**. - -Use the **Login As** action responsibly. Any changes you make while impersonating another user are recorded against that user's account. - -## Manage Service Accounts - -A **service account** lets you generate an **access key** that can be used in automated systems (such as CI/CD pipelines) to trigger tests and make API calls. Unlike a regular user, a service account is not linked to any individual and can be managed by administrators. - -### Create a Service Account - -To create a new service account, open the **Service Account** tab within your sub-organization. - -**Step 1:** Click **Create**. - -**Step 2:** Enter a name for your **Service Account** and click **Create**. - -**Step 3:** Your **Service Account credentials** are displayed. Save them securely, as they will not be shown again. - -Your new **Service Account** is now ready to use. - -> That covers everything you need to know about the sub-organizations feature. If you have any questions, please reach out via our **24/7 chat support** or email us at [support@testmuai.com](mailto:support@testmuai.com). diff --git a/static/docs/sumo-logic-integration.md b/static/docs/sumo-logic-integration.md deleted file mode 100644 index f31e49f1b..000000000 --- a/static/docs/sumo-logic-integration.md +++ /dev/null @@ -1,73 +0,0 @@ -# Sumo Logic Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Sumo Logic is a cloud-based platform that modernizes your security operations. By analyzing security and operational issues with Sumo Logic, you can make smarter decisions and free up resources for other critical functions. - -TestMu AI and Sumo Logic enables you to view TestMu AI results in a custom Sumo Logic dashboard and gain deeper insight into your automation scripts to ship quality products at light speed. - -## Prerequisites - -1. A TestMu AI account. If you don't have an account, sign up for free. -2. A Sumo Logic account. - -## Configuring A Sumo Logic Collector - -In order to push test data from TestMu AI to Sumo Logic dashboard, you will need to create a collector in Sumo Logic. - -### Adding A Collector - -**note** -If you already have an existing collector, please refer to [Adding A Source](#adding-a-source) section. - -1. Login to your Sumo Logic account. From the left sidebar, navigate to **Manage Data > Collection**. - -2. To add a new collector, click **Add Collector** on the top-right corner. - -3. A *Select Collector Type* pop-up will appear. Select **Hosted Collector**. - -4. In the provided fields, enter the collector details like Name, Description, Category, Fields/Metadata, Budget, and, Time Zone. Then click **Save**. - -### Adding A Source - -1. From Sumo Logic Home page, navigate to **Manage Data > Collection**. - -2. Click **Add Source**. If you create a new collector, it will directly route you here. - -3. Select **HTTP Logs and Metrics**. - -4. Enter the Source details like **Name**, **Description**, **Source Host**, **Source Category** and click **Save**. - -> The **Source Category** field is required. Otherwise, the test data will not get reflected in your Sumo Logic dashboard. - -5. A *HTTP Source URL* will be generated that you need to provide while configuring the TestMu AI Sumo Logic webhook. Copy the generated HTTP Source URL. - -## Configuring The Sumo Logic Integration - -1. Login to your TestMu AI account and visit **Integrations** from the left sidebar. - -2. From the ANALYTICS category, select **SUMO LOGIC**. - -3. In the Collector endpoint URL field, paste the copied HTTP Source URL and press **Install**. - -The Sumo Logic will be integrated with your TestMu AI account. - -## Installing the TestMu AI App in Sumo Logic - -1. Navigate to Sumo Logic Home and select **App Catalog** from the left navigation panel. - -2. Find and select the TestMu AI App using the search bar. - -3. Click **Add Integration** to install the TestMu AI app into your Sumo Logic account. It will then add the TestMu AI dashboards (TestMu AI - Test Overview and TestMu AI - Test Error Overview) to your directory. - -4. Click on the TestMu AI **Test Overview** dashboard to get an overall insight about your testing trends. - -5. Click on the TestMu AI **Test Error Overview** dashboard to understand your test error trends. - -## Removing The Sumo Logic Integration - -1. Login to your TestMu AI account and visit **Integrations** from the left sidebar. - -2. From the MY INTEGRATIONS category, navigate to SUMO LOGIC and click **REMOVE**. - -The Sumo Logic integration will be uninstalled. diff --git a/static/docs/support-access.md b/static/docs/support-access.md deleted file mode 100644 index 5f4357860..000000000 --- a/static/docs/support-access.md +++ /dev/null @@ -1,22 +0,0 @@ -# TestMu AI Support Access - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The Support Access feature allows authorised TestMu AI support personnel to temporarily access your TestMu AI account for triaging and technical assistance purposes. This access is granted only with your consent and can be revoked at any time. - -**Quick Link** -You can directly navigate to the Support Access page by this [**link**](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/support). - -## Granting Support Access - -**Step 1:** Sign in to your TestMu AI account. Don't have an account, register for free. - -**Step 2:** Click on **Profile Icon** and go to **Organization Settings** as shown in the image below. - -**Step 3:** Head to the **Support Access** tab and click on **Grant Access**. - -You have now successfully granted support access to TestMu AI. - -## Revoking Support Access - -* Simply click on Revoke TestMu AI Access. diff --git a/static/docs/support.md b/static/docs/support.md deleted file mode 100644 index 64ab82bcd..000000000 --- a/static/docs/support.md +++ /dev/null @@ -1,164 +0,0 @@ -# TestMu AI Support and Knowledge Base - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI Documentation and Knowledge Hub -Your Ultimate Resource for Seamless Cross-Browser Testing Across Desktop and Mobile Platforms - - Web Automation - - Selenium Testing - Cypress Testing - Playwright Testing - Puppeteer Testing - K6 Testing - - App Automation - - Appium Testing - Espresso Testing - XCUI Testing - Flutter Testing - Virtual Devices - - HyperExecute - - Getting Started - HyperExecute YAML - HyperExecute CLI - HyperExecute GUI   - HyperExecute MCP Server - Features - Integrations - Private Cloud - - Insights - - Pre-built Dashboards - Custom Dashboards - Widgets - Dashboard CoPilot AI - Test Case Insights - Build Insights - Flaky Test Insights - Command Logs Insights - - Browser Cloud   - - What is Browser Cloud - Launch Session With SDK - Launch Session With Agent Skills - Connect to a Session - - SmartUI - - Getting Started - Explore SDKs - CLI - Upload Screenshots - Smart PDF Comparison - Build Config and Options - Advance Comparison Options - Handling Dynamic Data - - Web Scanner - - Overview - Getting Started - Visual UI Scans - Accessibility Scans - Adding URLs - Scheduling Options - - KaneAI   - - Getting Started - Author Desktop Browser Test - Author Mobile App Test - API Testing - Command Types - Test Automation with CI/CD - - Agent Testing Platform   - - Getting Started - Test Your First AI Agent - FAQs - - Real Time - - Web Browser Testing - Mobile Browser Testing - Mobile App Testing - ChromeOS Web Browser Testing - ChromeOS App Testing - Key Features - - Real Device - - Real Device App Testing - - Real Device Browser Testing - - Private Cloud - - Test Manager   - - Create Projects - Insights Dashboard - Manual Test Cases - Automated Test Cases - Test Run - Milestones - Issue Tracker Integration - - TestMu AI MCP Server   - - Setting up TestMu AI MCP Server - - Integrations - - Bug Tracking - Project Management - CI / CD Integration - Test Reporting - Communication Tools - Plugin and Extensions - - Accessibility Testing - - Getting Started - Choose Your Tool - Manual Testing - Automation - Test Scheduling - Web Scanner - Reports - Mobile App Testing - - Testing Locally - - Testing Locally Hosted Pages - TestMu AI Tunnel Modifiers - Docker Tunnel - Troubleshooting Tunnel - Load Balancing in Tunnel - IP Whitelisting - Charles Proxy - - Settings and Security - - Account Management - Network Whitelisting Guide - TestMu AI Public IP - Single Sign On - SCIM - - Other Docs - - LT Browser - Test Logs - Test Intelligence - Screenshot Testing - Responsive Testing - Concurrency Widget diff --git a/static/docs/supported-appium-plugins.md b/static/docs/supported-appium-plugins.md deleted file mode 100644 index 3d9fb9efa..000000000 --- a/static/docs/supported-appium-plugins.md +++ /dev/null @@ -1,29 +0,0 @@ -# Appium Plugins - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Enhance your testing experience on TestMu AI by leveraging a variety of Appium plugins. Plugins offer various ways to extend or modify Appium's behavior. They are completely optional and are not needed for standard automation functionality, but you may find them useful for more specialized automation workflows. -By using these plugins, you can tailor your testing environment to better suit your project's specific needs, leading to more efficient and effective test automation. - -## Supported Plugins - -Below is a list of the supported Appium plugins on TestMu AI: - -| Plugin Name | Description | Example | -|-------------------|-------------------------------------------------------------------------------------------------------|---------------------------------------------------| -| `images` | Enables image comparison features in tests. Allows for verification of visual elements through images. | "appiumPlugins": ["images"] | -| `element-wait` | Provides enhanced wait capabilities for elements, allowing tests to wait for elements to be in a certain state.For further details, please check [this documentation](https://github.com/AppiumTestDistribution/appium-wait-plugin). | "appiumPlugins": ["element-wait"] | -| `gestures` | Adds support for gesture-based interactions, enabling tests to perform complex gestures like swipe, pinch, and zoom. For further details, please check [this documentation](https://github.com/AppiumTestDistribution/appium-gestures-plugin). | "appiumPlugins": ["gestures"] - -**Python Example:** - -```python -capabilities = { -"appiumVersion": "2.2.1", -"platformName": "iOS", -"appiumPlugins": ["images", "element-wait", "gestures"], -# Add other capabilities as needed -} -``` - -- Appium plugins are only supported with version 2.0.0 and above appium versions. Please ensure that the `appiumVersion` capability is set correctly to utilize these plugins. diff --git a/static/docs/supported-appium-versions.md b/static/docs/supported-appium-versions.md deleted file mode 100644 index 869ddb156..000000000 --- a/static/docs/supported-appium-versions.md +++ /dev/null @@ -1,62 +0,0 @@ -# Supported Appium Versions - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Customize your testing experience on TestMu AI by selecting the appropriate Appium version for your tests on real devices. TestMu AI offers flexibility in choosing the Appium version to meet your testing requirements. By default, the suitable Appium version is selected based on the operating system (OS) when running tests. However, you can explicitly specify the version using the `appiumVersion` capability in your test scripts. - -- We also provide Support for Appium 3.0.2 on following versions - - **Android Devices:** 13, 15, 16 - - **iOS Devices:** 16, 17, 18, 26 - -## Appium Version Configuration for Real Devices -To configure the Appium version for your tests on TestMu AI, use the `appiumVersion` capability in your test scripts. Below is a table of supported Appium versions for different platforms: - -| Platform Versions | Supported Versions | Default Version | Latest Version | -|-------------------|--------------------|-----------------|----------------| -| Android (All) | 1.21.0, 1.22.0, 1.22.3, 2.0.0, 2.1.3, 2.2.1, 2.3.0, 2.4.1, 2.5.4, 2.6.0, 2.10.3, 2.11.2, 2.11.4, 2.12.1, 3.0.2 | 2.2.1 | 2.3.0 | -| iOS (12, 13, 14) | 1.22.3, 2.0.0, 2.1.3, 2.2.1 | 2.2.1 | 2.2.1 | -| iOS (15, 16, 17, 18) | 1.22.3, 2.0.0, 2.1.3, 2.2.1, 2.3.0, 2.4.1, 2.5.4, 2.6.0, 2.10.3, 2.11.2, 2.11.4, 2.12.1, 3.0.2 | 2.2.1 | 2.3.0 | -| fireos | 1.21.0, 1.22.0, 1.22.3, 2.0.0, 2.1.3, 2.2.1, 2.3.0 | 2.2.1 | 2.3.0 | -| tvos | 1.22.3, 2.2.1 | 1.22.3 | 1.22.3 | - -**Java Example:** -```java -// Java code for configuring Appium version in tests on LambdaTest -DesiredCapabilities capabilities = new DesiredCapabilities(); -capabilities.setCapability("appiumVersion", "2.2.1"); -capabilities.setCapability("platformName", "ios"); -// Add other capabilities as needed -``` - -**Latest Version**: We update the `latest` version for all platforms, including Android and iOS, to stay aligned with the latest stable releases of Appium. This ensures users always have access to the most current and reliable versions.To use the latest version, pass `latest` in **appiumVersion** capability. - -**Default Version**: A new `default` version is supported when the user does not explicitly specify a appium version or pass `default` in **appiumVersion** capability. - -## Supported Driver Versions - -| Appium Version | iOS Driver | Android Driver | -|----------------|--------------------------------|---------------------------------------| -| 1.22.0 | - | appium-uiautomator2-driver@1.69.0 | -| 1.21.0 | - | appium-uiautomator2-driver@1.65.0 | -| 1.22.3 | appium-xcuitest-driver@3.59.0 | appium-uiautomator2-driver@1.70.1 | -| 2.0.0 | appium-xcuitest-driver@4.32.9 | appium-uiautomator2-driver@2.27.0 | -| 2.1.3 | appium-xcuitest-driver@5.7.0 | appium-uiautomator2-driver@2.32.3 | -| 2.2.1 | appium-xcuitest-driver@5.7.0 | appium-uiautomator2-driver@2.32.3 | -| 2.3.0 | appium-xcuitest-driver@5.12.2 | appium-uiautomator2-driver@2.35.0 | -| 2.4.1 | appium-xcuitest-driver@5.12.2 | appium-uiautomator2-driver@2.35.0 | -| 2.5.4 | appium-xcuitest-driver@7.13.0 | appium-uiautomator2-driver@3.5.1 | -| 2.6.0 | appium-xcuitest-driver@7.13.0 | appium-uiautomator2-driver@3.5.1 | -| 2.10.3 | appium-xcuitest-driver@7.17.4 | appium-uiautomator2-driver@3.5.3 | -| 2.11.2 | appium-xcuitest-driver@7.23.1 | appium-uiautomator2-driver@3.7.2 | -| 2.11.4 | appium-xcuitest-driver@7.26.4 | appium-uiautomator2-driver@3.7.11 | -| 2.12.1 | appium-xcuitest-driver@7.28.3 | appium-uiautomator2-driver@3.8.0 | - -- If a user passes any Appium version other than the mentioned versions, appropriate error will be thrown. -- For devices running iOS 17 and newer, it is recommended to use Appium versions `2.2.1` and `above` to ensure the best compatibility and performance. - -## Appium Version Configuration for Virtual Devices - -| Platform Versions | Supported Versions | -|-------------------|--------------------| -| Android | 1.21.0, 1.22.3, 2.0.0, 2.1.3, 2.16.2 | -| iOS |1.22.3, 2.0.0, 2.1.3, 2.16.2 | diff --git a/static/docs/supported-browsers-and-operating-systems-for-the-web-interface.md b/static/docs/supported-browsers-and-operating-systems-for-the-web-interface.md deleted file mode 100644 index 2f5a1ead0..000000000 --- a/static/docs/supported-browsers-and-operating-systems-for-the-web-interface.md +++ /dev/null @@ -1,41 +0,0 @@ -# Supported Browsers and Operating Systems - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI supports 3000+ browser and OS combinations for Selenium and Appium testing. Use `"latest"` as the browser version to always test on the most recent stable release. - -## Desktop Browsers for Selenium - -Supported desktop browsers and the operating systems they run on. - -| Browser | Versions | Supported OS | -|---------|----------|--------------| -| **Chrome** | 26+ to latest | Windows 11, Windows 10, Windows 8.1, Windows 8, Windows 7, macOS Sequoia, macOS Sonoma, macOS Ventura, macOS Monterey, macOS Big Sur, macOS Catalina, macOS Mojave, macOS High Sierra, macOS Sierra, OS X El Capitan | -| **Firefox** | 4+ to latest | Windows 11, Windows 10, Windows 8.1, Windows 8, Windows 7, macOS Sequoia, macOS Sonoma, macOS Ventura, macOS Monterey, macOS Big Sur, macOS Catalina, macOS Mojave, macOS High Sierra, macOS Sierra, OS X El Capitan | -| **Safari** | 12+ to latest | macOS Sequoia, macOS Sonoma, macOS Ventura, macOS Monterey, macOS Big Sur, macOS Catalina, macOS Mojave | -| **Edge** | 15+ to latest | Windows 11, Windows 10 | -| **Edge Chromium** | 80+ to latest | Windows 11, Windows 10, macOS Sequoia, macOS Sonoma, macOS Ventura, macOS Monterey, macOS Big Sur, macOS Catalina | -| **Opera** | 43+ to latest | Windows 11, Windows 10, macOS Sequoia, macOS Sonoma, macOS Ventura, macOS Monterey, macOS Big Sur, macOS Catalina | -| **IE** | 8, 9, 10, 11 | Windows 10, Windows 8.1, Windows 8, Windows 7 | - -## Supported Operating Systems - -All desktop operating systems available on the TestMu AI grid. - -| Platform | Versions | -|----------|----------| -| **Windows** | Windows 11, Windows 10, Windows 8.1, Windows 8, Windows 7 | -| **macOS** | Sequoia, Sonoma, Ventura, Monterey, Big Sur, Catalina, Mojave, High Sierra, Sierra, El Capitan, Yosemite, Mavericks | -| **Linux** | Ubuntu 20.04 | - -## Browser Version Syntax - -Use these version formats in your capabilities. - -| Format | Example | Description | -|--------|---------|-------------| -| `latest` | `"browserVersion": "latest"` | Most recent stable release | -| `latest-N` | `"browserVersion": "latest-5"` | N versions before the latest | -| Specific version | `"browserVersion": "120.0"` | Exact version number | - -Use the [Capabilities Generator](https://www.testmuai.com/capabilities-generator/) to see the exact browser versions and OS combinations currently available. The list updates as new browser versions are released. diff --git a/static/docs/supported-browsers-and-os.md b/static/docs/supported-browsers-and-os.md deleted file mode 100644 index 8809f4d40..000000000 --- a/static/docs/supported-browsers-and-os.md +++ /dev/null @@ -1,103 +0,0 @@ -# Supported Browsers and Operating Systems for Cypress - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI supports the below listed browsers, browser versions and Operating Systems for Cypress testing: - -```json -{ -"lambdatest_auth": { -"username": "", -"access_key": "" -}, -"browsers": [ -{ -"browser": "Chrome", -"platform": "Windows 10", -"versions": [ -"latest","latest-1" -] -} -], -``` - -| OPERATING SYSTEM | CHROME | FIREFOX | EDGE | -| ---------------- | ------------------------ | ------------ | ------------------------ | -| macOS Ventura | 66 and above (Except 82) | 60 and above | 80 and above (Except 82) | -| macOS Monterey | 66 and above (Except 82) | 60 and above | 80 and above (Except 82) | -| macOS Big Sur | 66 and above (Except 82) | 60 and above | 80 and above (Except 82) | -| macOS Mojave | 66 and above (Except 82) | 60 and above | 80 and above (Except 82) | -| macOS Catalina | 66 and above (Except 82) | 60 and above | 80 and above (Except 82) | -| Windows 11 | 66 and above (Except 82) | 60 and above | 80 and above (Except 82) | -| Windows 10 | 66 and above (Except 82) | 60 and above | 80 and above (Except 82) | -| Windows 8.1 | 66 and above (Except 82) | 60 and above | 80 and above (Except 82) | -| Windows 8 | 66 and above (Except 82) | 60 and above | 80 and above (Except 82) | -| Windows 7 | 66 and above (Except 82) | 60 and above | 80 and above (Except 82) | - ->**Note**: TestMu AI Automation also supports Cypress testing on Electron browser and in WebKit. -* **Electron**: Supported on all OS. -* **WebKit**: Supported on Windows 10 and 11, and macOS Big Sur and Monterey. See [Test on WebKit](#test-on-webkit) below. - -You can run Cypress tests across multiple browsers and OS combinations using the following ways. - -1. Configuring the browser and platform keys in `lambdatest-config.json` -2. Using the **--browsers** flag - -## Configuring The Browser And Platform Keys In lambdatest-config.json - -To run Cypress tests on multiple browser and OS configurations, add the *browsers* object in the `lambdatest-config.json` file to define a list of browsers, browser versions and platforms. - -Below is the syntax for configuring the browser and platform keys in the `lambdatest-config.json` file. - -```js -"browsers": [ -{ -"browser": "Chrome", -"platform": "Windows 10", -"versions": [ -"latest-1" -] -}, -], -``` - -## Using The Cypress CLI Command - -You can also run Cypress tests on multiple configurations by specifying the browser and platform using our Cypress CLI command. - -| Flag | Purpose | Type | -|------|---------|------| -| **--brs, --browsers** | Test will be run on the specified browsers in the format: `platform:browser:version` |String | - -Below is the command for specifying the browser and platform using **--brs, --browsers** flag. - -```js -lambdatest-cypress run --browsers "platform:browser:version" -``` - -## Test on WebKit - -WebKit is Safari's browser engine, so running your Cypress tests on WebKit shows how your site behaves in Safari. To target it, set the browser to `Webkit` in the `browsers` array of `lambdatest-config.json`: - -```js -"browsers": [ -{ "browser": "Webkit", "platform": "Windows 11", "versions": ["latest"] }, -{ "browser": "Webkit", "platform": "Windows 10", "versions": ["latest"] }, -{ "browser": "Webkit", "platform": "MacOS Monterey", "versions": ["latest"] }, -{ "browser": "Webkit", "platform": "MacOS Big Sur", "versions": ["latest"] } -] -``` - -WebKit runs on **Cypress v10.8.0 only**, so pin these under `run_settings.npm_dependencies`: - -```js -"npm_dependencies": { -"cypress": "10.8.0", -"playwright-webkit": "^1.28.1" -} -``` - -**Limitations:** -- WebKit supports only the **latest** version. -- Works only with **Cypress v10.8.0**. -- Supported on **Windows 10 and 11**, and **macOS Big Sur and Monterey**. diff --git a/static/docs/supported-cypress-versions.md b/static/docs/supported-cypress-versions.md deleted file mode 100644 index 9ae7892b0..000000000 --- a/static/docs/supported-cypress-versions.md +++ /dev/null @@ -1,111 +0,0 @@ -# Supported Cypress Versions - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -With the newer versions of Cypress releasing, to gain the most from recent improvements and bug fixes, it is recommended that your test scripts use the latest version. - -TestMu AI supports every major, minor and patch versions for: - -* Cypress 10 & above -* Cypress 9 & below - -## Cypress Versions Supported By TestMu AI - -In *lambdatest-config.json* file, you can set the Cypress version as shown in the below code snippet. - -```javascript -// lambdatest-config.json - -"run_settings":{ -"cypress_config_file":"cypress.config.js", -"reporter_config_file":"base_reporter_config.json", -"build_name":"build-name", -"parallels":1, -"specs":"./*.cy.js", -"ignore_files":"", -"network":false, -"headless":false, -"npm_dependencies":{ -"cypress":"10.0.0" -} -}, - -``` - -```javascript -// lambdatest-config.json - -"run_settings":{ -"cypress_config_file": "cypress.json", -"reporter_config_file": "base_reporter_config.json", -"build_name": "build-name", -"parallels": 1, -"specs": "./*.spec.js", -"ignore_files": "", -"network": false, -"headless": false, -"npm_dependencies": { -"cypress": "9.0.0" -} - -``` - -## Setting Cypress Version - -Following are the different ways to set the Cypress version. - -1. Using `package.json`: TestMu AI will pick the Cypress version from `package.json`. - -Example: - -```json -"devDependencies": { -"@bahmutov/print-env": "1.2.0", -"@cypress/eslint-plugin-dev": "5.0.0", -"colon-names": "1.0.0", -"cypress": "9.2.1", -"eslint": "7.0.0", -``` - -2. Using `npm_dependencies`: If you are passing `npm_dependencies`, TestMu AI will pick the Cypress version from `npm_dependencies` in `lambdatest-config.json`. - -Example: - -```json -"run_settings": { -"cypress_config_file": "cypress.json", -"build_name": "Cypress v9 Demo", -"parallels": 2, -"specs": "./cypress/integration/examples/actions.spec.js", -"downloads": "./cypress/results/", -"ignore_files": "", -"network": false, -"headless": false, -"reporter_config_file": "", -"npm_dependencies": { -"cypress": "10.0.0" -}, -}, -``` - -3. Using `cypress_version`: You can pass `cypress_version` in `lambdatest-config.json` and it will override the `cypress_version` present in `npm_dependencies` or `package.json`. - -Example: - -```json -"run_settings": { -"cypress_config_file": "cypress.json", -"build_name": "Cypress v9 Demo", -"parallels": 2, -"specs": "./cypress/integration/examples/actions.spec.js", -"downloads": "./cypress/results/", -"ignore_files": "", -"network": false, -"headless": false, -"reporter_config_file": "", -"npm_dependencies": { -"typescript": "3.7.4" -}, -"cypress_version": "10.0.0" -}, -``` diff --git a/static/docs/supported-languages-and-frameworks.md b/static/docs/supported-languages-and-frameworks.md deleted file mode 100644 index 4efd3cfb1..000000000 --- a/static/docs/supported-languages-and-frameworks.md +++ /dev/null @@ -1,105 +0,0 @@ -# Languages and Frameworks Supported By TestMu AI Selenium Grid - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Although, our [Selenium Grid](https://www.testmuai.com/selenium-automation/) supports all the languages & frameworks that are compatible with [Selenium](https://www.testmuai.com/selenium). Below mentioned are the ones for which we either already have documentation ready or are a work in progress. - - Java - - Selenide - - Gauge - - TestNG - - Geb - - JUnit - - Cucumber - - Serenity BDD - - PHP - - Laravel - - Codeception - - PHPUnit - - Behat - - Ruby - - RSpec - - Cucumber - - Test::Unit - - Capybara - - C# - - SpecFlow - - MSTest - - NUnit - - Python - - Python - - Pytest - - Behave - - Robot - - Unittest - - JavaScript - - WebDriverIO 5.6.2 - - NightwatchJS - - WebDriverIO - - Mocha - - WD - - Jest Library - - AngularJS With Karma - - CucumberJS For BDD - - Jasmine For BDD - - Karma Test Runner - - TestCafe - - NemoJS - - Protractor - - We support all languages and frameworks that are compatible with Selenium, so in case your favorite isn't in the table.Don't worry, you can still run the test. Contact Us for any help. - -## Our Selenium Grid Also Support Hybrid Frameworks - -There are [test automation](https://www.testmuai.com/automation-testing) frameworks which provide out-of-the-box features & functionality to help users perform [Selenium test automation](https://www.testmuai.com/selenium-automation) in a convenient manner. We support those frameworks too. - -### Tesbo - -Tesbo is a hybrid framework for test automation that works on a keyword-driven approach. It allows you to write automation scripts in plain English like you’d do with a BDD framework. Built on top of Selenium, Tesbo helps you to perform cross browser testing over a Selenium Grid in parallel. - -Check our support document to run your first [Tesbo script over TestMu AI Selenium Grid](/docs/executing-tesbo-automation-script-over-online-selenium-grid/). - -You can run any framework which supports Selenium execution over TestMu AI Selenium Grid. - -We are preparing documentation for more frameworks. If you want us to prioritize documentation of your beloved framework then feel free to give us a **shout**. diff --git a/static/docs/supported-locales-espresso.md b/static/docs/supported-locales-espresso.md deleted file mode 100644 index 4d1a4c233..000000000 --- a/static/docs/supported-locales-espresso.md +++ /dev/null @@ -1,157 +0,0 @@ -# Supported Locales And Languages - Espresso - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Use the given below list of supported locale and language codes for app testing. - -## Android Locales and Language Codes - -To test localised strings in your android app, configure Appium's language capability with language code. - -Language to be passed in the capability - language:'fr' where fr is language code for french - -### Android Language and Language Codes - -| Language | Language Code | -| -----------------------| ---------------------------| -| Arabic | ar | -| Bulgarian | bg | -| Catalan | ca | -| Chinese | zh | -| Croatian | hr | -| Czech | cs | -| Danish | da | -| Dutch | nl | -| English | en | -| Finnish | fi | -| French | fr | -| German | de | -| Greek | el | -| Hebrew | he | -| Hindi | hi | -| Hungarian | hu | -| Indonesian | id | -| Italian | it | -| Japanese | ja | -| Korean | ko | -| Latvian | lv | -| Lithuanian | lt | -| Norwegian-Bokmol | nb | -| Polish | pl | -| Portuguese | pt | -| Romanian | ro | -| Russian | ru | -| Serbian | sr | -| Slovak | sk | -| Slovenian | sl | -| Spanish | es | -| Swedish | sv | -| Tagalog | tl | -| Thai | th | -| Turkish | tr | -| Ukrainian | uk | -| Vietnamese | vi | - -### Android Locale and Locale Codes - -Set Appium's locale capability with an appropriate country code to display or format data such as dates, times, decimal separators, and calendars in accordance with the specified country's regional conventions. - -Locale to be passed in the capability - locale:'FR' where FR is the locale code for France - -| Locale | Locale Code | -| -----------------------| ---------------------------| -| Australia | AU | -| Austria | AT | -| Belgium | BE | -| Brazil | BR | -| Britain | GB | -| Bulgaria | BG | -| Canada | CA | -| Croatia | HR | -| Czech Republic | CZ | -| Denmark | DK | -| Egypt | EG | -| Finland | FI | -| France | FR | -| Germany | DE | -| Greece | GR | -| Hong-Kong | HK | -| Hungary | HU | -| India | IN | -| Indonesia | ID | -| Ireland | IE | -| Israel | IL | -| Italy | IT | -| Japan | JP | -| Korea | KR | -| Latvia | LV | -| Liechtenstein | LI | -| Lithuania | LT | -| Mexico | ES | -| Netherlands | NL | -| New Zealand | NZ | -| Norway | NO | -| Philippines | PH | -| Poland | PL | -| Portugal | PT | -| PRC | CN | -| Romania | RO | -| Russia | RU | -| Serbia | RS | -| Singapore | SG | -| Slovakia | SK | -| Slovenia | SI | -| Spain | ES | -| Sweden | SE | -| Switzerland | CH | -| Taiwan | TW | -| Thailand | TH | -| Turkey | TR | -| Ukraine | UA | -| US | US | -| USA | US | -| Vietnam | VN | -| Zimbabwe | ZA | - -## How to Setup Locale and Language - -You can also configure both locale and language during Espresso test execution for a seamless user experience in diverse linguistic and regional contexts of your app. - -### Language - -To test a localized version of your app on TestMu AI, use the `language` parameter in the Espresso test execution API request. This allows you to change the language of the application under test. - -| Parameter | Description | Values | -|-----------|----------------------------------------|------------| -| language | Set the language of the app under test | Example: ‘hi’ | - -### Locale - -To test a localized version of your app on TestMu AI, use the `locale` parameter in the Espresso test execution API request. This allows you to set the locale for the application under test. - -| Parameter | Description | Values | -|-----------|-----------------------------------|------------| -| locale | Set locale for the app under test | Example: IN (Country name abbreviation) | - -**For Example:** - -```bash -curl --location --request POST 'https://mobile-api.lambdatest.com/espresso/v1/build' \ ---header 'Authorization: Basic ' \ ---header 'Content-Type: application/json' \ ---data-raw '{ -"app" : "app_id", -"testSuite": "testsuite_id", -"device" : ["Galaxy S20-10"], -"queueTimeout": 360, -"IdleTimeout": 150, -"deviceLog": true, -"build" : "Proverbial-Espresso", -"language": "fr", -"locale": "CA" -}' -``` - -- Ensure that both the **language** and **locale** parameters are passed simultaneously in the API request. -- App should support the language and locale mentioned in the API request to work. -- For espresso sharding tests, you have to mention this in the `.yaml` file. diff --git a/static/docs/supported-locales-xcui.md b/static/docs/supported-locales-xcui.md deleted file mode 100644 index e79ff4e46..000000000 --- a/static/docs/supported-locales-xcui.md +++ /dev/null @@ -1,151 +0,0 @@ -# Supported Locales And Languages - XCUI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Use the given below list of supported locale and language codes for app testing. - -## iOS Locales and Language Codes - -To test localised strings in your iOS app, configure Appium's language capability with language code. - -Language to be passed in the capability - language:'fr' where fr is language code for french - -### iOS Language and Language Codes - -| Language | Language Code | -| -----------------------| ---------------------------| -| Chinese | zh | -| Czech | cs | -| Dutch | nl | -| English | en | -| Finnish | fi | -| French | fr | -| German | de | -| Greek | el | -| Hebrew | he | -| Hindi | hi | -| Hungarian | hu | -| Indonesian | id | -| Italian | it | -| Japanese | ja | -| Korean | ko | -| Malay | ms | -| Norwegian (Bokmal) | nb | -| Polish | pl | -| Portuguese (Brazil) | pt | -| Romanian | ro | -| Russian | ru | -| Slovak | sk | -| Spanish | es | -| Swedish | sv | -| Tagalog | tl | -| Thai | th | -| Turkish | tr | -| Ukrainian | uk | -| Vietnamese | vi | - -Set Appium's locale capability with an appropriate country code to display or format data such as dates, times, decimal separators, and calendars in accordance with the specified country's regional conventions. - -Locale to be passed in the capability - locale: 'fr_FR' where fr is language code for french and FR is the locale code for France - -### iOS Locale and Locale Codes - -| Locale | Locale Code | -| -----------------------| ---------------------------| -| Australia  | en_AU | -| Belgium  | nl_BE | -| Belgium  | fr_BE | -| Brunei Darussalam | ms_BN | -| Canada  | en_CA | -| Canada | fr_CA | -| Czech Republic  | cs_CZ | -| Finland | fi_FI | -| Germany  | de_DE | -| Greece | el_GR | -| Hungary | hu_HU | -| India | hi_IN | -| Indonesia | id_ID | -| Israel | he_IL | -| Italy | it_IT | -| Japan | ja_JP | -| Malaysia | ms_MY | -| Netherlands  | nl_NL | -| New Zealand   | en_NZ | -| Norway | nb_NO | -| Philippines  | tl_PH | -| Poland | pl_PL | -| PRC  | zh_CN | -| Romania | ro_RO | -| Russia | ru_RU | -| Singapore | en_SG | -| Slovakia | sk_SK | -| Korea | ko_KR | -| Sweden  | sv_SE | -| Taiwan  | zh_TW | -| Thailand  | th_TH | -| Turkey  | tr_TR | -| UK  | en_GB | -| Ukraine  | uk_UA | -| US | es_US | -| USA | en_US | -| Vietnam  | vi_VN | -| Brazil | pt-BR | -| China (Simplified) | zh-Hans | -| China (Traditional) | zh-Hant | -| Hong Kong | zh-HK | -| India | en-IN | -| Ireland | en-IE | -| Latin America | es-419 | -| Mexico | es-MX | -| South Africa | en-ZA | - -## How to Setup Locale and Language - -You can also configure both locale and language during XCUI test execution for a seamless user experience in diverse linguistic and regional contexts of your app. - -### Language - -To test a localized version of your app on TestMu AI, use the `language` parameter in the XCUI test execution API request. This allows you to change the language of the application under test. - -| Parameter | Description | Values | -|-----------|----------------------------------------|------------| -| language | Set the language of the app under test | Example: ‘hi’ | - -### Locale - -To test a localized version of your app on TestMu AI, use the `locale` parameter in the XCUI test execution API request. This allows you to set the locale for the application under test. - -| Parameter | Description | Values | -|-----------|-----------------------------------|------------| -| locale | Set locale for the app under test | Example: IN (Country name abbreviation) | - -**For Example:** - -```bash -curl --location --request POST 'https://mobile-api.lambdatest.com/framework/v1/xcui/build' \ ---header 'Authorization: Basic ' \ ---header 'Content-Type: application/json' \ ---data-raw '{ -"app" : "lt://APP_ID", -"testSuite": "lt://TestSuite_ID", -"device" : ["iPhone 11-14"], -"video" : true, -"queueTimeout": 10800, -"idleTimeout": 150, -"devicelog": true, -"network": false, -"build" : "Proverbial-XCUITest", -"language": "fr", -"locale": "CA" -}' -``` - -- When setting language or locale parameters, make sure your tests use the `ProcessInfo` object to pass these arguments correctly. This ensures the app launches with the specified language and locale settings. -```swift -let app = XCUIApplication() -app.launchArguments += ProcessInfo().arguments -app.launch() -``` -- The language and locale settings are applied at the app level. -- To ensure your application under test displays the correct localized strings, dates, times, and calendar formats, include both language and locale in your desired capabilities. -- For XCUI sharding tests, you have to mention this in the `.yaml` file. diff --git a/static/docs/supported-timezone.md b/static/docs/supported-timezone.md deleted file mode 100644 index 30b855700..000000000 --- a/static/docs/supported-timezone.md +++ /dev/null @@ -1,64 +0,0 @@ -# Setting up timezone - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -In automated testing with Appium on TestMu AI, configuring the timezone accurately is crucial for precise testing scenarios. TestMu AI provides support for various timezones, allowing users to customize their testing environments to meet specific requirements. - -By default, the timezone for Appium testing on TestMu AI is selected based on the device's default settings. However, users have the flexibility to explicitly specify the timezone using the **timezone** capability in their desired capabilities. - -## Timezone Configuration - -To specify the timezone for automated tests in Appium on TestMu AI, the **timezone** capability can be utilized. This ensures that tests are executed in the desired timezone, facilitating accurate validation of application behavior across different regions. - -| Capability | Type | Description | -|------------|-----------|--------------------------------------------------| -| **timezone** | String | Sets a specific timezone on the mobile device. | - -Here's an example of how to set the timezone in your Appium test script: - -**Java Example:** -```java -// Java code example for configuring timezone in Appium tests on LambdaTest - -DesiredCapabilities capabilities = new DesiredCapabilities(); -capabilities.setCapability("timezone", "Tonga"); -capabilities.setCapability("platformName", "android"); -// Add other capabilities as needed | -``` - -## Supported Timezones - -Below is the list of supported **timezones** supported along with their corresponding cities and UTC times: - -| Timezone | City | UTC Time | -|-------------|---------------|----------| -| Tonga | Tonga | UTC+13:00 | -| Auckland | Auckland | UTC+12:00 | -| Pohnpei | Pohnpei | UTC+11:00 | -| Melbourne | Melbourne | UTC+11:00 | -| Adelaide | Adelaide | UTC+10:30 | -| Tokyo | Tokyo | UTC+9:00 | -| Kuching | Kuching | UTC+8:00 | -| Pontianak | Pontianak | UTC+7:00 | -| Rangoon | Rangoon | UTC+6:30 | -| Dhaka | Dhaka | UTC+6:00 | -| Kathmandu | Kathmandu | UTC+5:45 | -| Colombo | Colombo | UTC+5:30 | -| Oral | Oral | UTC+5:00 | -| Kabul | Kabul | UTC+4:30 | -| Dubai | Dubai | UTC+4:00 | -| Moscow | Moscow | UTC+3:00 | -| Cairo | Cairo | UTC+2:00 | -| London | London | UTC+0:00 | -| UTC | UTC | UTC+0:00 | -| Cape Verde | Cape Verde | UTC-1:00 | -| Nuuk | Nuuk | UTC-3:00 | -| Buenos Aires| Buenos Aires | UTC-3:00 | -| La Paz | La Paz | UTC-4:00 | -| Lima | Lima | UTC-5:00 | -| Tegucigalpa | Tegucigalpa | UTC-6:00 | -| Los Angeles | Los Angeles | UTC-8:00 | -| Pitcairn | Pitcairn | UTC-8:00 | -| Honolulu | Honolulu | UTC-10:00 | -| Niue | Niue | UTC-11:00 | -| Belgium | Brussels | UTC+1:00 | diff --git a/static/docs/sync-test-instance.md b/static/docs/sync-test-instance.md deleted file mode 100644 index 3195ac98e..000000000 --- a/static/docs/sync-test-instance.md +++ /dev/null @@ -1,54 +0,0 @@ -# Sync Test Instances in Manual Test Runs - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -When a master test case is updated (e.g., steps are added, descriptions are modified, or modules are restructured), the test instances in your existing manual test runs do not automatically reflect these changes. The **Sync Test Instance** feature lets you manually update a test instance to the latest version of its master test case, ensuring your test runs always reflect the most current requirements. - -Each test instance tracks the **version** of the master test case it was created from (e.g., v1). When the master test case is updated to a newer version (e.g., v2), TestMu AI Test Manager detects this version mismatch and makes the **sync action** available. - -**Applies to Manual Test Runs Only** -This feature is available only for **manual test runs**. KaneAI test runs always use the latest code export during execution, so syncing is not applicable. - -**Bulk Sync Coming Soon** -Currently, you can only sync one test instance at a time. Bulk sync support is planned for a future release. - -## Identifying Stale Test Instances - -### From the Test Run Listing - -Each test instance card displays its current version (e.g., **v1**). A **grey** version label indicates the instance is up to date. If a newer version is available, the version label is highlighted in colour, and a **sync icon** appears next to it. Hover over the icon to see a tooltip (e.g., *Update to v2*), and click it to initiate the sync. - -### From the Test Instance Detail - -When you open a stale test instance, a banner appears at the top indicating how many versions behind the instance is. From here, you can click **View comparison** to review the changes or **Update to vN** to sync. - -## Comparing Versions - -Before syncing, you can review exactly what changed. Click **View comparison** from the version banner to open a side-by-side diff view. This highlights all differences between the test case version currently linked to the test instance and the latest version of the same test case, including changes to descriptions, preconditions, and individual steps. - -## Syncing a Test Instance - -When you click the sync action, a confirmation modal appears with the following details: - -- The instance will be updated to the latest version. -- **All statuses will be reset to Not Started**: both the overall instance status and every individual step status. -- **Existing remarks and attachments will be preserved** for re-verification. - -Click **Update to Version** to confirm, or **Cancel** to keep the current version. - -### After Syncing - -Once synced, the test instance is updated with the latest master test case content, including any changes to the title, step descriptions, expected outcomes, and step structure (added, removed, or reordered steps). The instance and all step statuses are reset to **Not Started**, and a success notification confirms the update. - -## Audit Log - -Every sync action is recorded in the test instance's [Audit Log](/support/docs/test-instance-audit-logs/). The log captures the version update (e.g., v1 → v2), the instance and step-level status resets, and any related changes, giving your team full traceability. - -**Duplicating a Test Run** -When you duplicate a test run, all test instances in the new run are automatically linked to the **latest version** of their master test cases, regardless of which version was used in the original run. All statuses are set to **Not Started**. - -## Limitations - -- **No auto-sync**: Syncing is a manual action. Test instances do not update automatically when the master test case changes. -- **No partial status retention**: All statuses (instance and step level) are reset to **Not Started** upon sync. There is no option to selectively retain step statuses. -- **KaneAI-generated versions**: If the latest version of a master test case was generated via **Automate with KaneAI**, the sync action is disabled for that instance. A tooltip indicates the reason when you hover over the disabled action. diff --git a/static/docs/system-and-custom-fields.md b/static/docs/system-and-custom-fields.md deleted file mode 100644 index 453ceb213..000000000 --- a/static/docs/system-and-custom-fields.md +++ /dev/null @@ -1,30 +0,0 @@ -# System and Custom Fields - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Explore the structured organization of your test projects through the use of Fields and Values, accessible via the Project's dashboard. Dive into System Fields for essential categorizations or enhance your project's flexibility with Custom Fields to improve your testing workflows. - -To access the System & Custom Fields settings, click on **Settings** in the top right hand side in the Project's dashboard. - -You can manage your **System** and **Custom Fields** in this Fields page. - -### System Fields -Are default fields present in the Test Manager. These fields help organize and track your test cases, test runs or instances. They can be seamlessly integrated into your test management process and customized by adding values. Test Cases & Test Runs have their separate System Fields. - -- Test Cases have these pre defined fields: **Priority**, **Status**, and **Type**. -- Test Runs can have only one System Field that is the `Status`. - -To manage these fields, navigate to the System Fields tab, where you'll see dedicated sections for both Test Case and Test Runs fields. - -These System Fields can have Custom Values which can be added by simply selecting any field and clicking on **Add Value**. - - Every new Status value of Test Runs will have a unique random color defined to it on creation. - -### Custom Fields -Allow you to store additional information beyond what System Fields offer. To create a new field, enter the required details and choose the appropriate data type from the **Type** function. Available types include String, Textarea, Number, Dropdown (Single Select), Dropdown (Multi Select), Boolean (Checkbox), Date, User, and URL. - -For Dropdown types (Single Select and Multi Select), you also have the option to add values. - -Enter the name, placeholder, mark the field, apply it to all future projects if required and click create. - -You can also link a single or multiple projects of your choice to the custom fields and click on **Save changes**. diff --git a/static/docs/taiga-integration.md b/static/docs/taiga-integration.md deleted file mode 100644 index 8ea5e92f2..000000000 --- a/static/docs/taiga-integration.md +++ /dev/null @@ -1,69 +0,0 @@ -# Taiga Integration with TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Taiga is an open-source project management platform that supports agile methodologies like Scrum and Kanban. Integrating Taiga with TestMu AI enables you to log bugs directly from your testing sessions into Taiga, streamlining your workflow and enhancing team collaboration. - -> Taiga Integration with TestMu AI is available for both freemium and premium plans. - -## Integrate Taiga from your TestMu AI Account - -**Step 1:** Log in to your [TestMu AI](https://www.testmuai.com/login/) account. Ensure you have Admin or User level access to manage integrations. - -**Step 2:** Navigate to `Settings` -> `Integrations` -> `Bug Tracker`. - -**Step 3:** Click on the **Connect** button of the Taiga block. - -**Step 4:** In the integration setup form, select your Taiga type using the radio buttons: - -- **Cloud-based Taiga:** Select this option if you are using the cloud-hosted version of Taiga. The `Taiga Hosted URL` field will not be shown. -- **On-premise/Self-hosted Taiga:** Select this option if you are using a self-hosted Taiga instance. You will need to provide the URL of your Taiga instance (e.g., `https://your-taiga-instance.com`). - -After selecting the appropriate option, enter the following details: - -- **Taiga Username:** Your Taiga account username. -- **Taiga Password:** Your Taiga account password. - -We do not store your password. It is only used for initial authentication during the integration setup. - -- **Taiga Hosted URL:** This field will only appear if you select the **On-premise/Self-hosted Taiga** option. - -After filling in the details, click on the **Install** button to establish the integration. - -## Lodge your First Bug - -**Step 1:** Ensure you have an active project set up in your Taiga account. - -**Step 2:** While working on TestMu AI, if you encounter a bug, click on the **Mark as Bug** button available on difderent products. - -**Step 3:** In the bug logging form, provide the necessary details such as: - -- **Subject:** A concise summary of the bug. -- **Description:** Detailed information about the issue. -- **Project:** Select the Taiga project where the bug should be logged. -- **Assignee:** The user in Taiga to whom ticket should be assigned. -- **Type:** Select if its Issue or a Task - -**Step 4:** Click on the **Create Issue** button. The bug will be logged into your selected Taiga project, and you can view it directly from your Taiga dashboard. - -## Resync Taiga Integration - -To ensure that the latest projects and users from your Taiga system are available in TestMu AI, you can resync the integration. - -**Step 1:** Navigate to `Settings` -> `Integrations` in your TestMu AI account. - -**Step 2:** Locate the Taiga integration and click on the **Resync** button. - -- This will fetch the latest projects and users from your Taiga system. - -- If your authentication token has expired, you will be prompted to re-enter your Taiga password to authenticate again. - -Once resynced, the latest data will be available for use. - -## Uninstall Taiga Integration - -**Step 1:** Navigate to `Settings` -> `Integrations` in your TestMu AI account. - -**Step 2:** Locate the Taiga integration and click on the **Remove** button to uninstall the integration. - -> That was all you need to know for TestMu AI + Taiga Integration. Increase your productivity with our integrations. If you still have any questions for us, please feel free to let us know. Our experts are always available on **chat** to help you out with any roadblock. diff --git a/static/docs/taking-screenshots-of-pages-behind-login.md b/static/docs/taking-screenshots-of-pages-behind-login.md deleted file mode 100644 index 9f1fac8dd..000000000 --- a/static/docs/taking-screenshots-of-pages-behind-login.md +++ /dev/null @@ -1,89 +0,0 @@ -# Taking Screenshots of Pages Behind Login - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI offers Screenshot testing, a feature to automatically capture bulk, full-paged screenshots of any URL over different browsers. You can perform [cross browser compatibility testing](https://www.testmuai.com/feature) on both mobile and desktop browsers at the same time. Also, you can capture up to 25 screenshots in a single test session. - -This document will help you understand how you can capture bulk screenshots of your web-application **after the login page** for different browsers to measure the cross browser compatibility. Refer to our [Automated Screenshot](https://www.testmuai.com/automated-screenshot) testing document to know more. - -## How This Works? - -A Login URL is fed into the browser. This URL is going to be the one pointing towards the login page of the website your wish to test and would be different from the Screenshot Test URL. - -Then the locators(Username, Password, Submit button) specified by you for your login page are searched by the Screenshot driver. If all fields are correct, your test will pass onto the next stage; if fields are incorrect, then the test is marked as a fail. - -Now, the form gets submitted, and your browser will wait for the new page to render. - -Finally, the Screenshot test URL is fed into the Screenshot driver, allowing it to start generating automated screenshots in bulk on your desired test configurations. - -**Note: We recommend using your Login profile for a single test session before attempting multiple screenshots for cross browser testing.** - -## Getting Started With Automated Screenshot After Login - -To begin, we need to configure a login profile that would specify the website under test, locators, and login credentials. This is a one-time process. Without further ado, let’s configure the Login profile. - -Login into your TestMu AI account and go to **Screenshot** by clicking on the drop down for Visual UI Testing from the left-navigation menu. - -### Set Up Login Profile - -Configure the Login profile by clicking on the **Login** button and selecting a **New Login+**. - -### Provide The Login URL - -A window would pop-up where you need to fill in the details, starting from the **Login URL**. Kindly ensure that you provide the URL of the login page for the website under test. Now, hit **next**. - -### Provide The Locators - -Select the **locator** as per your respective fields in the Login form. - -For the demonstration, let us capture the details for TestMu AI Login. -> -**Note:** If you are already logged in to [LambdaTest](https://www.testmuai.com), make sure to perform this drill in **incognito mode**. - -**Step 1:** Go to https://www.testmuai.com/login/. Right-click and select the option to open the inspection tools. For demonstration, we are using Google Chrome’s Dev Tools. - -**Step 2:** Click on the highlighted arrow to help you select any element on your page to inspect. - -**Step 3:** Now, click on the **Email** field in the login form, and you will notice that the name locator is populated with the value ’**email**’. - -**Step 4:** Switch to your normal browser where you are logged into the TestMu AI application. Provide the Email field details in the **Username Locators**. - -**Step 5:** Switch back to incognito mode and inspect the **password** field in the TestMu AI login form. - -**Step 6:** Provide the Password field details in the **Password Locators**. - -**Step 7:** Back to incognito mode, and inspect the Login button. Perform a right click on the piece of code highlighted for the login button, go to **copy**, and click **copy XPath**. - -> -**Note:** In the above demonstration, you will find the class name placed for the login button in the TestMu AI login. However, the class name may be similar for multiple HTML elements located in a login form, so it is considered a best practice to copy the XPath for such scenarios. - -**Step 8:** Now, on your normal browser, Select the XPath locator and paste the copied XPath for the **Login button locator**, and hit **Next**. - -### Leveraging AutonomIQ ChroPath To Easily Locate Elements For Selenium Testing - -[ChroPath](https://www.autonomiq.io/deviq-chropath.html) is a productive Browser extension that instantly locates the elements over your web page inside the developer tool. Add the extension to your browser and visit your web-application. After that, open the Developer tools while inspecting the element for which you need the locators value. Select **ChroPath** from the sidebar. - -Once you select ChroPath, you can copy the locator value. For example, in the shared image, we are copying the relative XPath. - -### Provide The Login Credentials - -Under the credentials, provide your login details for the website you wish to test and hit **Next**. For a demonstration, you can provide your TestMu AI login details. - -### Save The Login Profile - -When you are done configuring the Login URL, Locators, and Credentials for your web-application under test, you only need to provide a name to save this login profile. - -Once you hit the save button, you will find the below prompt indicating that the profile has been successfully created. - -Kudos, you have successfully configured the Login profile. This profile will be saved in your TestMu AI account, so the next time you need to run a similar test, all you need is to enable the flag for this Login profile. - -## Running Automated Screenshot After The Login Page - -**Step 1:** Login to TestMu AI and go to [Screenshot Testing](/support/docs/automated-screenshot-testing/). Enable the flag for the profile you wish to test. - -**Step 2:** Now, enter the Screenshot Test URL and hit the capture button. To carry out the above demonstration, we are making use of the Test URL: [https://integrations.lambdatest.com/](https://integrations.lambdatest.com) - -**Step 3:** You will find all the screenshots generated for the web page that is accessible after login. - -> -In case you have any questions, drop us an email at support@testmuai.com or simply give us a shout. We are here to help you speed your test suites 24/7. Happy testing! 🙂 diff --git a/static/docs/target-process-integration.md b/static/docs/target-process-integration.md deleted file mode 100644 index 92aa2664b..000000000 --- a/static/docs/target-process-integration.md +++ /dev/null @@ -1,74 +0,0 @@ -# TargetProcess Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TargetProcess is an integrated project management tool that offers teams with a modern, customizable, and powerful Agile approach. TargetProcess provides an intuitive visual interface for teams to manage projects with ease. Whether you are new to Agile or want to scale it across, this tool has options to manage projects based on Scrum, Kanban, SAFe, NEXUS, or any other approach. You can create custom reports, dynamic visualizations, and more in seconds. The powerful boards make it unique enabling you to plan and track progress, and can be shared to broadcast information to others. Overall TargetProcess provides users a holistic solution to help them plan, track, and steer complex work at all levels. - -Using TestMu AI integration with TargetProcess, you’ll be able to log bugs over your project in TargetProcess from TestMu AI in just a single click. You can [capture a screenshot](https://www.testmuai.com/full-page-screen-capture) of the bug, highlight the anomaly, annotate extra information, choose an assignee, provide a suitable title, and description to help relay further actions. You can do all that directly from TestMu AI. - -## How To Integrate TargetProcess With Your TestMu AI Account? - -**Step 1**: Login to your TestMu AI account. You would need to have Admin or User level access to see and install integrations. - -**Step 2**: Select Integrations from the left navigation menu bar. This will guide you to a screen where you will find a list of 3rd party applications, available to integrate with your TestMu AI account. Search for TargetProcess and click on the block. - ->**Note**: If you have already installed a TestMu AI integration to any project management tool then you would come across a prompt asking you to disable your existing TestMu AI integration for integrating TargetProcess with your TestMu AI account. - -**Step 3**: You would now be routed to a screen where you’ll need to enter your TargetProcess **Dashboard URL**, Access Domain and **TargetProcess Access Token**. - -**What is Dashboard URL?** -TargetProcess uses boards to visualise data. The dashboard URL would be the link of your customised board that you would create to track any project, tasks, or bugs etc. We will be using the highlighted User Story for demonstration. - -**What are Access Tokens?** - -Access tokens are strings with the authorization key required to access an API. They are issued to the client-server and are usually opaque. They are used for requesting access to protected, user-specific resources. Access tokens are vital from a security point of view & can be generated in different formats, depending upon security requirements specified on the resource server. - -## Where can I create my TargetProcess Access Token? - -You can generate your TargetProcess access token by clicking on the user avatar displayed on the top right and going to **Admin User**. - -Go to Access Tokens tab and Add a new token. - -As you create a new token a key would be generated. Copy this key as you will need to provide it on TestMu AI application to integrate with TargetProcess. - -**Step 4**: Provide the access token, Dashboard URL and Domain URL to TestMu AI for integrating TargetProcess and hit the install button. - -Kudos! You’ve successfully integrated TestMu AI with TargetProcess. You will find TargetProcess Integration highlighted under the section **"My Integration"**. - -## How To Log Bug Using TestMu AI Integration To TargetProcess? - -**Step 1**: Go for any of the test from the left navigation menu. For demo, we will be taking Real Time Test option. - -**Step 2**: Present a URL of the web-app you need to test in the dialog box. After that, select any configuration for browser and operating system of your choice & hit ‘Start’. - -**Step 3**: After the VM is launched and operable. You can perform testing on your web-app for finding bugs. If a bug gets revealed, then you need to click on the Bug icon from the left panel to capture a screenshot of the bug. - -**Step 4**: After a screenshot is captured, you can annotate any issue or a task with a built-in image editor. Once you are done highlighting the bug, click on the button that says Mark as Bug. - -**Step 5**: After clicking on Mark as Bug button a form specific to TargetProcess would open up. Fill the fields as per your requirement, and create the issue. - -> -> **You can specify**: -* The project you want to assign an issue. -* The issue type whether it is a bug or a task. -* The user story around an issue. -* Summary and description to help relay the observation. -* Different tags as well(comma separated). - -> **Note**: After you click on Create Issue, you will be able to observe it being successfully marked through a single click effort. You will get prompt messages on top of your Virtual Machine indicating the progress of bug logging. After few seconds you will be notified with a prompt message Bug successfully marked indicating that the screenshot has been pushed to your TargetProcess project. - -**Step 6**: Visit your Task Board in TargetProcess and you’ll find the bug that you logged from TestMu AI in it on the appropriate User Story that you specified while logging the bug. - -**Step 7**: If you click on the bug, you will find all the details specified by you while logging it from TestMu AI. Along with the screenshot of the bug, test ID, environment details and a link to reproduce the issue which would open the same test environment over TestMu AI. - -## How To Remove TestMu AI Integration With TargetProcess? - -> You can work with one integration at a time. So if you would want to integrate into a similar 3rd party application, then you would have to remove your current integration. Here is how you can do that. - -**Step 1**: Login to your TestMu AI account. - -**Step 2**: Select **Integrations** from the left-navigation menu. This will guide you to a screen where you will find 3rd party applications, available to integrate with your TestMu AI account. - -**Step 3**: Under **"My Integrations"**, you’ll find the TargetProcess Integration. Click on Remove button to uninstall the integration with TestMu AI and TargetProcess. - -That was all you need to know for TestMu AI + TargetProcess Integration. Increase your productivity with our integrations. If you still have any questions for us, please feel free to let us know. Our experts are always **available on chat** to help you out with any roadblock regarding our product. Happy testing! diff --git a/static/docs/team-communication-tools.md b/static/docs/team-communication-tools.md deleted file mode 100644 index 85013bdd1..000000000 --- a/static/docs/team-communication-tools.md +++ /dev/null @@ -1,30 +0,0 @@ -# Team Communication Tools - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI allows you to integrate your TestMu AI account with your favourite communication tools. That way you can instantly share any UI bug that you come across while cross browser testing over 3000+ real browsers hosted by TestMu AI cloud servers. Here are the instant messaging tools that you can integrate with TestMu AI. - -| [Slack Integration](/docs/slack-integration/) | -|:---------------------------------------:| - -| [OpsGenie Integration](/docs/opsgenie-integration/) | -|:---------------------------------------:| - -| [Microsoft Teams Integration](/docs/microsoft-teams-integration/) | -|:-----------------------------------------------------------:| - -| [Rocket.Chat Integration](/docs/rocketchat-integration-with-testmu/) | -|:-----------------------------------------------------------:| - -| [Mattermost Integration](/docs/integrating-mattermost-with-testmu/) | -|:-----------------------------------------------------------:| - -| [Flock Integration](/docs/flock-integration/) | -|:-----------------------------------------------------------:| - -| [Google Chat Integration](/docs/google-chat-integration/) | -|:-----------------------------------------------------------:| - -**Got something to recommend?** - -We would love to hear it. Drop the suggestion in the **24/7 customer chat support** for any other instant-messaging or team communication tool that you may want us to integrate with. diff --git a/static/docs/team-management.md b/static/docs/team-management.md deleted file mode 100644 index a07dd8630..000000000 --- a/static/docs/team-management.md +++ /dev/null @@ -1,71 +0,0 @@ -# Team Management - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The TestMu AI Team Management empowers organizations to collaborate effectively on their Web and App Automation testing by creating isolated testing environments. - -This functionality addresses the need for: - -- **Organized Test Management:** Group users and corresponding tests based on specific projects, teams, or departments. -- **Improved Visibility and Control:** Admins can control user access to specific test data, preventing unauthorized viewing of confidential tests. -- **Enhanced Team Collaboration:** Facilitate communication and knowledge sharing within designated teams. - -## Create an Organization - -- Go to the **Settings** page > select **Organization Settings** section. -- Click on the **Teams** tab and edit your **Organization Name**. - -## Create your First Team - -- Click on the **Create Team** button in the Teams page. -- Enter the name of your team. - -You cannot leave the Team name as empty. It is mandatory to enter the name of your team. - -- Select all the **users** you want to add to your team and click on **Finish** button. - -## Edit your Team Settings - -- Go to your **Teams** section. -- Click on the **three dots (...)** > **Edit** button. -- Edit your team setting by updating your teams name or by adding/removing the users from the team. -- Click on the **Finish** button. - -## Delete your Team - -- Go to your **Teams** section. -- Click on the **three dots (...)** > **Delete** button. - -Your created team will be permanently deleted. - -## Access Rules for Builds, Tests, and Test Details - -This section outlines a series of cases designed to verify test visibility within the system. - -- All the user roles (admin, user, guest) are independent of the tests visibility in teams. - -The following scenarios explain who can see the test results and under what circumstances: - -> - To restrict any user from accessing all tests, please ensure to add that user to a team. -- A user can be a part of multiple teams. - -### Scenario 1: No Teams are Created -In this case, any of the user can see all test results. - -### Scenario 2: Teams are Created -If you create a team, only members of that team and users who are not part of any team can see the test results for that team. Additionally, team members can view their own team's tests as well as the tests run by users who are not associated with any team. - -- #### Users are in Different Teams - Suppose if you have created two teams, each having five users, then neither can see each other's team's test results. - -- #### Users are in the Same Team - If you have created a team, then all the users of the same team can see each other's test results. - -- #### Users are not in a Team, but Teams exist - If any of the users aren't assigned to any team, but there are teams in the organization, they can still see all test results. - -### Scenario 3: Users Switches Teams -When a user is switched from one team (Team A) to another (Team B), they will no longer see test results from Team A. They will only have access to test results associated with their new team, Team B. - -### Scenario 4: All Teams are Deleted -If all teams are removed, then all users can view all test results again. diff --git a/static/docs/teamcity-integration-with-testmu.md b/static/docs/teamcity-integration-with-testmu.md deleted file mode 100644 index 8a168e677..000000000 --- a/static/docs/teamcity-integration-with-testmu.md +++ /dev/null @@ -1,131 +0,0 @@ -# TeamCity Integration With TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Developed by JetBrains, TeamCity is Java-based build management to help you with continuous integration and continuous deployment. With TeamCity, you can deploy your automated pipeline quicker, pacing your go-to-market launch. - -TestMu AI now integrates with TeamCity to boost your go-to-market delivery. Perform automated cross browser testing with TestMu AI to ensure your development code renders seamlessly through an online Selenium grid providing 3000+ real browsers running through machines, on the cloud. Perform automation testing in parallel with TestMu AI’s Selenium grid to drastically trim down your test cycles. - -## Prerequisites - -- TeamCity server along with one TeamCity Agent installed. - -- Make sure team city Agent has Node and Protractor installed - -- A GIT repository. - -- **TestMu AI Authentication Credentials** - Be aware of your TestMu AI authentication credentials i.e. your TestMu AI username, access key and HubURL. You need to set them up as your environment variables. You can retrieve them from your [TestMu AI automation dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/) by clicking on the key icon near the help button. - - * **For Linux/Mac:** - - $ export LT_USERNAME= `{YOUR_LAMBDATEST_USERNAME}` - - $ export LT_ACCESS_KEY= `{YOUR_LAMBDATEST_ACCESS_KEY}` - - * **For Windows:** - - $ set LT_USERNAME= `{YOUR_LAMBDATEST_USERNAME}` - - $ set LT_ACCESS_KEY= `{YOUR_LAMBDATEST_ACCESS_KEY}` - -## Integrating TestMu AI with TeamCity - -**Step 1:** Go to your TeamCity server URL. - -**Step 2:** Go to Administration and create a new project manually. - -**Step 3:** After filing details and click on the Create button. - -**Step 4:** Create a new VCS root using the below form. Required inputs are: Git Repo URL into Fetch URL and Authentication method is password where you need to pass your git username/ password. - -**Step 5:** Fill details, test connection and click on the Create button. - -**Step 6:** Go to the General Settings screen for the added project. Now, create a build configuration. - -**Step 7:** Go to Build Step page and add a build step as below. - -**Step 8:** Save build step and go to Parameters and create two environment variables as below where LT_USERNAME and LT_ACCESS_KEY will be your authentication credentials for TestMu AI. - -**Step 9:** Go to Project Dashboard and Run your project. Now, notice the status of test running successfully on TestMu AI Grid with the below log. - -## Running Test in Parallel - -Parallel Testing is one of the most demanding features of TestMu AI Selenium Grid. By parallel testing, you can run more than one test case, simultaneously. This means that Parallel testing would allow you to execute numerous automation test cases altogether. So you execute a single test scenario across different browsers or could run different test scenarios across the same browser but with different browser versions. - -> Clone our [GitHub repository for TeamCity](https://github.com/LambdaTest/protractor-selenium-teamcity-sample.git) to execute your first parallel test of TeamCity + TestMu AI Integration. - -After you clone our GitHub repository, execute the below command in cmd/terminal. - -``` -npm run parallel -``` - -The above command will trigger the execution of below code for performing parallel testing at TestMu AI Selenium Grid. - -```js -//parallel.conf.js file -username= process.env.LT_USERNAME || "", -accessKey= process.env.LT_ACCESS_KEY || "", - -exports.config = { -'specs': [ '../specs/single.js' ], - -seleniumAddress: 'https://'+username+':'+accessKey+'@hub.lambdatest.com/wd/hub', - -'commonCapabilities': { -'build': 'protractor-selenium-sample', -'name': 'parallel-test', -'tunnel': true -}, - -'multiCapabilities': [{ -'browserName': 'Chrome', -'version':'67.0', -'platform': 'WIN10' -},{ -'browserName': 'Safari', -'version':'11.1', -'platform': 'macOS 10.12' -},{ -'browserName': 'Edge', -'version':'15.0', -'platform': 'WIN10' -},{ -'browserName': 'Firefox', -'version':'62.0', -'platform': 'WIN10' -},{ -'browserName': 'Internet explorer', -'version':'10.0', -'platform': 'WIN10' -}], - -onPrepare: () => { - -myReporter = { -specStarted: function(result) { -specStr= result.id -spec_id = parseInt(specStr[specStr.length -1]) -browser.getProcessedConfig().then(function (config) { -var fullName = config.specs[spec_id]; -//var fileName = fullName.substring(fullName.lastIndexOf('/')+1); -browser.executeScript("lambda-name="+fullName.split(/(\\|\/)/g).pop()) -}); -} -}; -jasmine.getEnv().addReporter(myReporter); -}, -onComplete: () => { -browser.quit(); -} - -}; - -// Code to support common capabilities -exports.config.multiCapabilities.forEach(function(caps){ -for(var i in exports.config.commonCapabilities) caps[i] = caps[i] || exports.config.commonCapabilities[i]; -}); -``` - -Kudos! Now you can Deploy your code in a reliable manner at scale using TeamCity integration with TestMu AI, and ensure it looks robust across every browser to provide a seamless user experience to all your visitors. Happy Testing! 🙂 diff --git a/static/docs/teamcity-plugin.md b/static/docs/teamcity-plugin.md deleted file mode 100644 index 33e453ec2..000000000 --- a/static/docs/teamcity-plugin.md +++ /dev/null @@ -1,67 +0,0 @@ -# TeamCity Plugin Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The TeamCity plugin will help you to manage your TestMu AI credentials, write and execute tests directly on the TestMu AI platform, and test your local, internal, dev, or stage environments securely with the help of Lambda Tunnel. - -## Downloading The TeamCity Plugin From Marketplace - -Visit the [JetBrains marketplace](https://plugins.jetbrains.com/) and enter "TestMu AI" in the search bar. - -You will need to clear the filter (set as default by JetBrains) to view the result, as shown in the image below. - -Once you do, you can see the TestMu AI Plugin appearing in the search results. - -Select the TestMu AI test result and click on Get button to download the TestMu AI’s TeamCity Plugin. - -## Installing the TeamCity plugin - -To install the TeamCity plugin, you need to download the compressed zip file from [here](https://plugins.jetbrains.com/plugin/download?rel=true&updateId=105571). Once downloaded, follow the below steps to upload the plugin on your TeamCity server. - -1. Go to your server **Administration** and then to **Plugins List** - -2. Now inorder to add the TestMu AI’s TeamCity plugin, choose the option Upload ZIP file and upload the earlier downloaded zip file. - -3. You will need to restart your TeamCity server. Once done, the TestMu AI TeamCity plugin will be shown, as an *External plugin*, as shown below. - -That's it. The TestMu AI TeamCity plugin has been added in your TeacmCity Server successfully. - -## Configuring Project To Use TestMu AI TeamCity Plugin - -For demo purpose, we have used this sample Github repository. Below are the steps on how to configure the project: - -1. In this sample TeamCity server project, we have used two basic steps as a part of the test, based on the Github repo. - -2. Now to configure your *Build Features*, visit the **Build Features** from the left bar and click on the button to **Add Build Feature**. - -3. In the drop down box that appears, choose the **TestMu AI** feature. - -4. Now a window will appear where you can configure the TestMu AI feature. Insert your TestMu AI username and access key into it and save the feature. You can fetch your username and access key from your TestMu AI [profile](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). - -5. Upon saving, you can see that the TestMu AI build feature has been added in your project. - -That’s it. Your project has been successfully configured to run tests using the TestMu AI TeamCity plugin. - -## Executing The Test On TestMu AI - -Upon executing the project, you can see the test status and the automation report directly on your [automation dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/timeline/). - -## Configuring Lambda Tunnel For Local Testing - -To test your locally or privately hosted website, or your dev or stage environments securely, you can make use of [Lambda Tunnel](/docs/testing-locally-hosted-pages/). To activate the Tunnel in your project, while adding the TestMu AI build feature, make sure to tick the **Enable LAMBDATEST TUNNEL** option. - -If you want to save the Tunnel binary file in a specific location, you provide its path in the **LAMBDATEST TUNNEL Path** option. If not provided, ‘checkout directory’ will be chosen to download and install the Lambda Tunnel Binary. - -Also, you can also customize this tunnel based on your requirements, using the Tunnel modifiers. For example, we have used the *–tunnelName* modifier to customize the Tunnel connection name. - -`--tunnelName TeamCity` - -To know more about the Tunnel modifiers, please refer [here](/docs/lambda-tunnel-modifiers/). - -## Testing Privately Hosted Site Using TestMu AI TeamCity Plugin - -After the above configuration of Lambda Tunnel in your project’s Build Features, you can now execute your local projects and test your localhost, dev, or even stage environments securely with just a click. - -Once you run your tests, you can see the Lambda Tunnel getting activated during your test execution, on the toolbar of your TestMu AI’s automation dashboard, as shown below. - -That's it folks! If you have any doubt or questions, feel free to contact our experts at **24/7 chat support** or mail us at [support@testmuai.com](mailto:support@testmuai.com). Happy testing! 🙂 diff --git a/static/docs/teamcity-with-hyperexecute.md b/static/docs/teamcity-with-hyperexecute.md deleted file mode 100644 index eb94c1aa1..000000000 --- a/static/docs/teamcity-with-hyperexecute.md +++ /dev/null @@ -1,65 +0,0 @@ -# TeamCity Integration With HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TeamCity is a build management and continuous integration server developed by JetBrains. It is used to automate the process of building, testing, and deploying software. It supports a wide range of programming languages and technologies, including Java, .NET, Ruby, and Python. - -This document will show you how to integrate TeamCity with HyperExecute to greatly shorten your test cycles. - -## How to Integrate TeamCity with HyperExecute - -To integrate TeamCity with HyperExecute, follow the below mentioned steps: - -### Pre-requisite: - -- Install [Java 17](https://www.oracle.com/java/technologies/javase/jdk17-archive-downloads.html) (recommend) in your device. -- Download the [TeamCity](https://www.jetbrains.com/teamcity/download/) in your device. - -### Step 1: Start the TeamCity GUI - -- Go to the **teamcity/bin** folder and run the command **./runAll.sh start** to start the TeamCity Server in your system. -- Now type ``http://localhost:8111/`` in your browser to open the TeamCity GUI. -- Click on Create Project button. - -### Step 2: Create the Project - -- For the demo purposes, we will be using the GitHub repository to create the project. Thus, select **From a repository URL** section. -- Enter the **Repository URL** from which you want to set up the project. -- Enter the **Username** and **Password** of your GitHub. (It is required in case the access to the repo needed authentication). -- Click on **Proceed** button. - -### Step 3: Connect your Version Control System. - -- Connect your GitHub / GitLab wherever you have stored your tests. -- Enter your **Project name**. -- Enter your **Build config name**. -- Mention your **Default branch** which needs to be monitored. -- If you want that besides your default branch, TeamCity should monitor other branches too, define it in the **Branch specification** section. -- Click on **Proceed** button. - -### Step 4: Start the build process - -- Configure your build setup either **automatically** (fetched by your VCS) or **manually**. For the demo purpose, we will configure the build process manually. - -- Select the **Command Line** for the build setup - -- Enter the **Step name** (optional). -- Select the **Custom Script** in the **Run** field. -- Enter the below mentioned script in the space provided. - -```bash -curl -O https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute -chmod +x hyperexecute -./hyperexecute --user --key --config -``` - -> Since my default agent is mac I am using darwin CLI in the **cURL** command. Kindly change it according to your system default agent. - - **Mac** -> darwin - - **Linux** -> linux - - **Windows**-> windows - -### Step 5: Build process is complete - -- Now your build process is completed. This will take a couple of minutes to completely execute all the stages. - -- After the build is successful, go to your TestMu AI Dashboard and check the output of the triggered Job. diff --git a/static/docs/teamwork-integration.md b/static/docs/teamwork-integration.md deleted file mode 100644 index dd240cc7c..000000000 --- a/static/docs/teamwork-integration.md +++ /dev/null @@ -1,53 +0,0 @@ -# Teamwork Integration With TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Teamwork Projects is a cloud-based project management tool that tracks project operations and business functionality. It is available on iOS, Android, Google Chrome, and the Web. TestMu AI allows users to push bugs directly to the project. - -TestMu AI integrates with Teamwork Projects, allowing users to file issues directly to their chosen project, assign them to teammates, and attach necessary screenshots during a test session. - -> Teamwork Integration with TestMu AI is available for freemium as well as premium plan. - -## Integrate Teamwork from your TestMu AI Account - -**Step 1:** Login to your TestMu AI account. You should have Admin or User level access to see and install integrations. - -**Step 2:** Click on Settings -> Integrations -> Project Management. - -**Step 3:** Click on **Connect** button of `Teamwork` block. - -**Step 4:** Now, provide your Teamwork API Token and Teamwork site address to establish integration with TestMu AI and click on **install** button. - -**Fetch your Teamwork API Token** - -- Visit your Teamwork account -> **Edit my details** button. -- You will find the "**API Token**" under the section **API & Mobile**. Copy your API token. - -## Lodge your First Bug - -> Note: If you are using Teamwork for the first time, then make sure to create a project for yourself. It is a pre-requisite in order to push screenshots from your TestMu AI account. - -**Step 1:** Start with any type of testing, for the demo purpose we are going with the **Real Time Testing**. - -**Step 2:** Enter your Project URL and configure for browser and operating system of your choice & hit **Start**. - -**Step 3:** After the VM is launched and operable. You can perform testing on your web-app for finding bugs. If a bug gets revealed, then you need to click on the Bug icon from the left panel for capturing a screenshot of the same. - -**Step 4:** After clicking on "Mark as Bug" button a Teamwork specific form would open up. Fill the fields as per your requirement. -- Select which project should the ticket go under. -- Set which Task List should the bug go under. -- Set the priority of the issue. -- Mention the Title as well to easily organize your bugs. -- Write a description to help relate the cause of the issue or the task. - -**Step 5:** Click on **Mark as Bug** button. Now go to your dashboard and check a ticket will be created for the same. - -## Uninstall Teamwork Integration - -**Step 1:** Login to your TestMu AI account. You should have Admin or User level access to see and install integrations. - -**Step 2:** Click on Settings -> Integrations -> Project Management. - -**Step 3:** Click on the **Remove** button. - -> That was all you need to know for TestMu AI + Teamwork Integration. Increase your productivity with our integrations. If you still have any questions for us, please feel free to let us know. Our experts are always **available on chat** to help you out with any roadblock regarding our product. Happy testing! diff --git a/static/docs/test-amazon-connect-bots.md b/static/docs/test-amazon-connect-bots.md deleted file mode 100644 index fb03eb939..000000000 --- a/static/docs/test-amazon-connect-bots.md +++ /dev/null @@ -1,55 +0,0 @@ -# How to Test Amazon Connect Bots With TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Amazon Connect is a contact centre rather than an agent builder. The conversational logic lives in a Lex bot invoked from a contact flow, while the flow controls routing, queueing, and escalation around it. TestMu AI tests both layers together, drives full multi-turn conversations across personas and edge cases, and scores every run. - -## Before You Begin - -Before you connect an Amazon Connect bot, make sure you have: - -- An Amazon Connect instance with a published contact flow -- The Connect phone number, for phone testing -- TestMu AI workspace with agent-testing permissions -- Source docs for scenario generation: bot definition, contact flow export, or knowledge base - -## Test an Amazon Connect Phone Bot - -Phone testing covers callers reaching your bot inbound or outbound on the Connect number. Upload the contact flow and bot definition to generate scenarios, then add the Connect number and pick from 100+ voices, background noise conditions, and personas. It covers what telephony introduces around the bot's conversational accuracy: DTMF entry, queue routing, hold behaviour, transfer to a representative, and carrier latency. - -Full setup: [Phone agent testing](/support/docs/phone-agent/) - -## Test an Amazon Connect Chat Bot - -Chat testing covers text conversations through the Connect chat widget or a custom client. Upload the contact flow and supporting docs to generate scenarios, then create the endpoint profile using the chat contact API, with messages received over the participant WebSocket. It catches reasoning, slot filling, and routing failures. - -Full setup: [Chat agent testing](/support/docs/chat-agent/) - -## What You Get With Agent Testing - -Every Amazon Connect run, on any surface it supports, is scored across: - -- 30+ metrics across 8 categories, with configurable thresholds -- Context-aware scenario generation -- Adversarial testing and automated issue detection -- 100+ voices and personas -- Multilingual conversations -- Quality scoring and real-time call monitoring - -## Amazon Connect-Specific Considerations - -A few Amazon Connect behaviours are worth building dedicated scenarios around: - -- A failure can sit in the bot or in the contact flow, so each metric needs to name which layer it reflects -- Contact attributes carry state between the flow and the bot, which makes a value set in one place and read in another a real failure mode -- Queue behaviour, hold, and representative handoff are part of the caller experience and belong in scenarios -- The bot itself is covered on the Amazon Lex page - -## Troubleshooting - -Common failure modes to watch for: - -- Bot not invoked from the flow -- Contact attribute missing -- Queue misrouted -- Chat session timed out diff --git a/static/docs/test-amazon-lex-bots.md b/static/docs/test-amazon-lex-bots.md deleted file mode 100644 index a783206a2..000000000 --- a/static/docs/test-amazon-lex-bots.md +++ /dev/null @@ -1,57 +0,0 @@ -# How to Test Amazon Lex Bots With TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Lex handles both text and speech natively, built from intents, slots, and fulfilment Lambdas. It reaches phone callers through Amazon Connect rather than on its own. TestMu AI connects to the bot alias directly, drives full multi-turn conversations across personas and edge cases, and scores every run. - -## Before You Begin - -Before you connect an Amazon Lex bot, make sure you have: - -- A published Lex bot with a live alias -- TestMu AI workspace with agent-testing permissions -- Source docs for scenario generation: bot definition, PRD, or knowledge base - -## Test a Lex Voice Bot - -Voice testing applies to bots reached over the speech endpoint rather than a phone number. Upload the bot definition and supporting docs to generate scenarios, then create the endpoint profile using the streaming conversation API. It isolates the speech pipeline, so recognition accuracy, slot capture from spoken input, and interruption handling get tested without telephony noise in the results. - -Full setup: [Voice agent testing](/support/docs/voice-agent/) - -## Test a Lex Chat Bot - -Chat testing covers text conversations against the bot alias. Upload the bot definition and supporting docs to generate scenarios, then create the endpoint profile using the text recognition API. It catches intent matching, slot filling, and fulfilment failures. - -Full setup: [Chat agent testing](/support/docs/chat-agent/) - -Phone testing for Lex bots runs through Amazon Connect. See how to [test Amazon Connect bots](/support/docs/test-amazon-connect-bots/). - -## What You Get With Agent Testing - -Every Amazon Lex run, on any surface it supports, is scored across: - -- 30+ metrics across 8 categories, with configurable thresholds -- Context-aware scenario generation -- Adversarial testing and automated issue detection -- 100+ voices and personas -- Multilingual conversations -- Quality scoring - -## Lex-Specific Considerations - -A few Amazon Lex behaviours are worth building dedicated scenarios around: - -- Slot filling is the main failure surface, so partial input, corrections, and out-of-order answers all need scenarios -- Intent confidence thresholds decide the fallback path, which makes borderline utterances worth testing deliberately -- Fulfilment Lambdas sit outside the bot, so a Lambda error surfaces as a bot failure -- Aliases and versions matter. Testing the draft version does not test what is live -- Lex has no telephony of its own, so there is no phone section on this page - -## Troubleshooting - -Common failure modes to watch for: - -- Wrong intent matched -- Slot not captured -- Lambda timeout -- Testing the draft instead of the live alias diff --git a/static/docs/test-bland-agents.md b/static/docs/test-bland-agents.md deleted file mode 100644 index 3b5c5979c..000000000 --- a/static/docs/test-bland-agents.md +++ /dev/null @@ -1,64 +0,0 @@ -# How to Test Bland AI Agents With TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Bland builds agents as node-based Conversational Pathways rather than a single prompt, and the same pathway powers phone, web, chat, and SMS. TestMu AI connects to the pathway on whichever channel it runs on, drives full multi-turn conversations across personas and edge cases, and scores every run. - -## Before You Begin - -Before you connect a Bland agent, make sure you have: - -- Bland account with a published Conversational Pathway -- A provisioned phone number, for phone agents only -- TestMu AI workspace with agent-testing permissions -- The pathway's source docs: PRD, knowledge base, or pathway export - -## Test a Bland Phone Agent - -Phone testing covers inbound and outbound callers running over the phone network. Import the pathway or paste the prompt to generate scenarios across every labeled branch, then add the number under Phone Numbers and pick from 100+ voices, background noise conditions, and personas. It covers DTMF entry, warm transfers, voicemail detection, and carrier latency. - -Full setup: [Phone agent testing](/support/docs/phone-agent/) - -## Test a Bland Voice Agent - -Voice testing applies to web agents reached in the browser rather than over a phone number. Import the pathway or paste the prompt to generate scenarios, then create the endpoint profile using the Web Agents API. It isolates the speech pipeline, so STT and TTS accuracy, turn-taking, and interruption handling get tested without telephony noise in the results. - -Full setup: [Voice agent testing](/support/docs/voice-agent/) - -## Test a Bland Chat Agent - -Chat testing covers text conversations through the Bland web chat widget, which runs the same pathway as the phone agent. Upload the pathway export and supporting docs to generate scenarios, then create the endpoint profile using the widget ID or the chat API. It catches branching and reasoning failures, and because one pathway serves every channel, it also exercises the voice path. - -Full setup: [Chat agent testing](/support/docs/chat-agent/) - -## What You Get With Agent Testing - -Every Bland run, on any surface it supports, is scored across: - -- 30+ metrics across 8 categories, with configurable thresholds -- Context-aware scenario generation -- Adversarial testing and automated issue detection -- 100+ voices and personas -- Multilingual conversations -- Quality scoring and real-time call monitoring - -## Bland-Specific Considerations - -A few Bland behaviours are worth building dedicated scenarios around: - -- Pathways are graphs, not prompts. Unexpected phrasing breaks condition matching and strands the caller on a node -- Dynamic variables use double-curly syntax, and webhook node results are referenced downstream -- Voicemail detection is imperfect, so the human-speaks-first path needs its own scenario -- Warm transfer requires the proxy on the same Twilio account -- Batch calling hits concurrency and daily caps and can end partially complete -- Shared node edits silently affect every other pathway using that node -- One pathway serves phone, web, chat, and SMS, so a text fix can break the voice branch - -## Troubleshooting - -Common failure modes to watch for: - -- No transcript returned -- Call stranded on a node -- Webhook variable missing downstream -- Batch run ends partial diff --git a/static/docs/test-case-repository.md b/static/docs/test-case-repository.md deleted file mode 100644 index 8368d1912..000000000 --- a/static/docs/test-case-repository.md +++ /dev/null @@ -1,17 +0,0 @@ -# Test Case Repository - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -A test case repository serves as a centralized location for all test cases within a project. Users can access this repository to view a comprehensive list of test cases associated with the project. - -**Navigating the Test Case Repository** - -1. Within your project, navigate to the test cases section to view the repository. Use the search bar and filter options for efficient location of specific test cases. - -2. By clicking on **Status**, filter the test cases by their current state - Draft, Open or Closed. - -3. The **Creator** option allows you to select test cases based on assignee. - -4. Select test cases based on **Tags**, which helps categorize and access tests more efficiently according to specific criteria. - -5. Lastly, there's flexibility to **Edit** or **Delete** the test cases enabling easy management and updates to your testing workflow. diff --git a/static/docs/test-case-versioning.md b/static/docs/test-case-versioning.md deleted file mode 100644 index 9e21d4c9c..000000000 --- a/static/docs/test-case-versioning.md +++ /dev/null @@ -1,25 +0,0 @@ -# Test Case Versioning - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Test Case Versioning allows teams to track every change made to a test case over time. It provides a complete audit trail, ensures traceability, supports rollbacks to earlier versions, and makes collaboration across teams more reliable. - -## Creating Versions -Every edit to a test case whether it’s updating steps, descriptions, or fields creates a new version once saved. Here’s how the save test case flow works: - -- After making changes to a test case, the `Save Changes` button in the top right corner will be enabled. - -- When saving, you can add a commit message to help identify the change in the audit history. - -- In the `Version History` tab, you can view all commits made to that test case. - -## Managing Versions - -### Version Comparison -To see the change log and compare different versions of a test case, click the `Compare` button in the **Version History** tab. - -### View Older Versions -To view the details of a test case at a specific version, open the desired commit and switch to **View Mode**. This shows the state of the test case at that point in time. - -### Revert to a Version -When you revert a test case to a specific version, a new version is created that copies the contents of the selected version. diff --git a/static/docs/test-copilot-studio-agents.md b/static/docs/test-copilot-studio-agents.md deleted file mode 100644 index bb1aabcf0..000000000 --- a/static/docs/test-copilot-studio-agents.md +++ /dev/null @@ -1,56 +0,0 @@ -# How to Test Copilot Studio Agents With TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Copilot Studio agents run over digital messaging channels and, once the telephony channel is enabled, over phone through Dynamics 365 Contact Center. Two voice flavours exist: basic IVR agents on classic orchestration, and real-time voice agents. TestMu AI connects to the agent on whichever channel it runs on, drives full multi-turn conversations across personas and edge cases, and scores every run. - -## Before You Begin - -Before you connect a Copilot Studio agent, make sure you have: - -- A published Copilot Studio agent -- The telephony channel enabled and connected to Dynamics 365 Contact Center, for phone agents only -- TestMu AI workspace with agent-testing permissions -- Source docs for scenario generation: topic export, PRD, or knowledge base - -## Test a Copilot Studio Phone Agent - -Phone testing covers callers who reach the agent over PSTN or SIP on the telephony channel. Upload the topic export and knowledge sources to generate scenarios, then add the telephony number and pick from 100+ voices, background noise conditions, and personas. It covers what telephony introduces across every topic the agent can trigger: DTMF entry, barge-in, silence detection, transfers, and carrier latency. - -Full setup: [Phone agent testing](/support/docs/phone-agent/) - -## Test a Copilot Studio Chat Agent - -Chat testing covers text conversations on the agent's published web or messaging channel. Upload the topic export and knowledge sources to generate scenarios, then create the endpoint profile against the published channel endpoint. It catches topic triggering, knowledge grounding, and tool failures before they reach the voice channel. - -Full setup: [Chat agent testing](/support/docs/chat-agent/) - -## What You Get With Agent Testing - -Every Copilot Studio run, on any surface it supports, is scored across: - -- 30+ metrics across 8 categories, with configurable thresholds -- Context-aware scenario generation -- Adversarial testing and automated issue detection -- 100+ voices and personas -- Multilingual conversations -- Quality scoring and real-time call monitoring - -## Copilot Studio-Specific Considerations - -A few Copilot Studio behaviours are worth building dedicated scenarios around: - -- The telephony channel must be enabled before connecting Dynamics, and the order matters for reconnection -- Basic voice agents and real-time voice agents behave differently enough to need separate scenarios -- Transfers carrying SIP UUI context require direct routing, since PSTN numbers do not support UUI header transfer -- Topics trigger on either phrases or descriptions, and accuracy settings only affect the phrase path -- There is no direct audio surface outside telephony, so there is no voice section on this page - -## Troubleshooting - -Common failure modes to watch for: - -- Topic not triggered -- Telephony channel not connected -- UUI transfer rejected -- Agent published to the wrong environment diff --git a/static/docs/test-data-generation.md b/static/docs/test-data-generation.md deleted file mode 100644 index 38f755f9e..000000000 --- a/static/docs/test-data-generation.md +++ /dev/null @@ -1,57 +0,0 @@ -# Test Data Generation - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI provides built-in capabilities to generate, manage, and protect the test data your teams use across automation. You can author synthetic datasets for data-driven tests, mask sensitive payloads in network logs and test recordings, and parameterize tests so the same flow runs across environments and inputs without rework. - -This page summarizes the three capabilities and points you to the deeper documentation for each. - -## Synthetic Data Generation - -Author and manage datasets directly inside KaneAI Test Manager to drive the same test case with multiple inputs. Datasets remove the need to hand-craft fixtures or maintain external data files. - -**Capabilities:** -- **Auto-generated default datasets**: every test case that uses parameters generates an immutable default dataset during authoring -- **Custom datasets**: create copies of default datasets, add rows, and edit values to cover additional scenarios -- **Autofill with AI**: populate dataset fields automatically based on parameter names (for example, generate realistic names, ages, and phone numbers) -- **CSV import**: bring existing test data into KaneAI by uploading a CSV -- **Version history**: track changes, revert, or restore previous dataset versions for audit and recovery - -**Documentation:** -- [KaneAI Datasets](/support/docs/kane-ai-using-datasets/) - -## Data Masking - -Protect credentials, tokens, and other confidential values from showing up in network logs and test recordings. Masking is enforced at the platform level so sensitive data never lands in shared artifacts. - -**Capabilities:** -- **Network payload masking (Selenium)**: set the `network.mask` capability to `true` to automatically mask values for sensitive keys (`key`, `password`, `token`, `auth`, `email`, `cipher`, `secret`, `nonce`, `salt`) across request headers, response headers, and cookies -- **Lambda Masking for HyperExecute recordings**: automatically hide sensitive interactions captured in Playwright test recordings, including credentials, geolocation, and storage state -- **Compliance-ready logs**: share test results, debug network traffic, and store CI/CD reports without exposing production secrets - -**Documentation:** -- [Network Data Masking for Selenium](/support/docs/network-data-masking/) -- [HyperExecute Lambda Masking (Release 2.3.1)](/support/docs/hyperexecute-release-notes-2024/#version-231) - -## Parameterization - -Pass dynamic values into test cases at runtime so a single test runs across environments, accounts, and configurations. KaneAI provides a layered model (variables, secrets, smart variables, parameters, and datasets) so each piece of dynamic data is stored in the most appropriate way. - -**Capabilities:** -- **Variables**: reusable placeholders for values that change between runs (URLs, user IDs, configuration flags) -- **Secrets**: encrypted storage for credentials and other sensitive inputs that must never appear in plain text -- **Smart variables**: context-aware values that resolve at runtime (for example, the current build's artifact URL) -- **Parameters**: values supplied to a test case at execution time, including URL and data input fields, with the ability to convert any literal step value into a parameter on the fly -- **Datasets**: combine parameters into rows for data-driven runs across multiple inputs - -**Documentation:** -- [KaneAI Variables & Parameters Overview](/support/docs/kaneai-variables-and-parameters/) -- [Using Variables](/support/docs/kane-ai-using-variables/) -- [Using Parameters](/support/docs/kane-ai-using-parameters/) -- [Secrets](/support/docs/kane-ai-secrets/) - -## Related Resources - -- [Enterprise Readiness Overview](/support/docs/enterprise-ready/) -- [Product Security](/support/docs/enterprise-ready/#product-security) -- [Data Protection & GDPR](/support/docs/enterprise-ready/#data-protection--gdpr) diff --git a/static/docs/test-elevenlabs-agents.md b/static/docs/test-elevenlabs-agents.md deleted file mode 100644 index b219993d2..000000000 --- a/static/docs/test-elevenlabs-agents.md +++ /dev/null @@ -1,60 +0,0 @@ -# How to Test ElevenLabs Agents With TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -ElevenLabs deploys one agent configuration to phone, web voice, and text, with voice conversations running over WebRTC and text-only conversations over WebSocket. TestMu AI connects to the agent on whichever surface it runs on, drives full multi-turn conversations across personas and edge cases, and scores every run. - -## Before You Begin - -Before you connect an ElevenLabs agent, make sure you have: - -- ElevenLabs account with a published agent -- A number imported under Phone Numbers, for phone agents only -- TestMu AI workspace with agent-testing permissions -- Source docs for scenario generation: prompt, PRD, or knowledge base - -## Test an ElevenLabs Phone Agent - -Phone testing covers inbound and outbound callers over native telephony, Twilio, or a SIP trunk. Paste the agent's system prompt to generate scenarios, then add the imported number under Phone Numbers and pick from 100+ voices, background noise conditions, and personas. It covers DTMF entry, transfers, voicemail handling, and carrier latency. - -Full setup: [Phone agent testing](/support/docs/phone-agent/) - -## Test an ElevenLabs Voice Agent - -Voice testing applies to agents reached over a direct audio stream rather than a phone number. Paste the system prompt to generate scenarios, then create the endpoint profile using WebRTC or WebSocket. It isolates the speech pipeline, so STT and TTS accuracy, turn-taking, and interruption handling get tested without telephony noise in the results. - -Full setup: [Voice agent testing](/support/docs/voice-agent/) - -## Test an ElevenLabs Chat Agent - -Chat testing applies to agents running in text-only mode, either configured that way or forced with a runtime override. Upload the supporting docs and knowledge base the agent answers from to generate scenarios, then create the endpoint profile using WebSocket. It skips every audio metric and scores logic, tool calls, grounding, and safety, which can run on every commit. - -Full setup: [Chat agent testing](/support/docs/chat-agent/) - -## What You Get With Agent Testing - -Every ElevenLabs run, on any surface it supports, is scored across: - -- 30+ metrics across 8 categories, with configurable thresholds -- Context-aware scenario generation -- Adversarial testing and automated issue detection -- 100+ voices and personas -- Multilingual conversations -- Quality scoring and real-time call monitoring - -## ElevenLabs-Specific Considerations - -A few ElevenLabs behaviours are worth building dedicated scenarios around: - -- SIP audio runs at G711 8kHz or G722 16kHz regardless of the audio format set on the agent for WebSocket connections. Mismatches here look like quality failures but are codec failures -- Private agents need a signed link for WebSocket or a conversation token for WebRTC. Public agents need only the agent ID -- Text-only mode depends on the agent response event firing, so a silent failure reads as a timeout - -## Troubleshooting - -Common failure modes to watch for: - -- No transcript returned -- Codec mismatch on SIP calls -- Signed link expired -- Text-only run hangs diff --git a/static/docs/test-instance-audit-logs.md b/static/docs/test-instance-audit-logs.md deleted file mode 100644 index 5894e040b..000000000 --- a/static/docs/test-instance-audit-logs.md +++ /dev/null @@ -1,21 +0,0 @@ -# Test Instance Audit Logs - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Test Manager now provides Audit Logs for test instance execution. Audit Logs bring visibility into the **who, what, and when** for every test execution. This is critical for teams working in regulated environments, or those needing high accountability in their QA processes. - -With Audit logs you can store the execution history of your tests and even run parallel execution sessions without losing the context on individual executions as every action will be logged with clear indication of time stamps & executor. - -## Details captured in Audit Logs - -- Test Instance & Steps `Status` changes. -- Test Instance & Steps `Remarks or attachment` changes. -- Test Instance Assignee changes. - -You can view the Audit Logs by clicking on the `View Execution Log`. - -For the Steps level execution logs, you can preview the step with respect to which the log was created. - - As step level Audit logs are with respect to that specific steps in case the step changes the old logs will still show the preview of the older step. - - When you delete a configuration or a test case, all associated audit logs are removed. This is because the action deletes the entire instance. If you then add the same test case and configuration again, it creates a brand new instance with no prior audit history. diff --git a/static/docs/test-intelligence-command-error-logs-analytics.md b/static/docs/test-intelligence-command-error-logs-analytics.md deleted file mode 100644 index 2aa70d922..000000000 --- a/static/docs/test-intelligence-command-error-logs-analytics.md +++ /dev/null @@ -1,23 +0,0 @@ -# Test Intelligence - Command Error Logs Analytics - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## About Command Error Logs Analytics - -Analyzing your failed tests is a crucial part of your test automation strategy. It helps you to identify the errors that are causing your test runs to fail. This helps you to fix the errors and improve the quality of your test suite. - -TestMu AI provides you with the Command Logs Analytics feature to analyze your failed commands or steps. It provides you with the list of errors that are causing your test runs to fail. You can also view the history of the errors and the number of times the error has occurred in the past. This helps you to identify the errors and fix them to improve the quality of your test suite. - -## How does the Command Error Logs Analytics work? - -#### Selenium Automation Tests -Command Logs Analytics uses the WebDriver command logs to analyze your failed commands or steps. The WebDriver commands logs are the steps which are executed by the test script during the test execution. To know more about the WebDriver command logs, you can refer to the [W3C Command Logs](https://www.w3.org/TR/webdriver2/#endpoints). - -**Command Logs Analytics** -To read more about the Command Logs Analytics, you can refer to the [documentation](/docs/analytics-modules-test-intelligence-command-logs-analytics/) here. - -#### Cypress Automation Tests -Command Logs Analytics uses the Cypress command logs to analyze your failed commands or steps. The Cypress command logs are the steps which are executed by the test script during the test execution. To know more about the Cypress command logs, you can refer to the [Cypress Command Logs](https://docs.cypress.io/api/table-of-contents#Commands). - -#### CDP Framework Tests (Puppeteer, Playwright, Taiko) -Command Logs Analytics uses the CDP command logs to analyze your failed commands or steps. The CDP command logs are the steps which are executed by the test script during the test execution. To know more about the CDP command logs, you can refer to the [CDP Command Logs](https://chromedevtools.github.io/devtools-protocol/). diff --git a/static/docs/test-intelligence-flakiness-test-detection.md b/static/docs/test-intelligence-flakiness-test-detection.md deleted file mode 100644 index 1842cdd55..000000000 --- a/static/docs/test-intelligence-flakiness-test-detection.md +++ /dev/null @@ -1,155 +0,0 @@ -# Test Intelligence - Flaky Test Detection - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Flaky Test Detection helps you to identify the flaky tests in your test suite. It provides you with the list of tests that are flaky and the reason behind the flakiness of the test. You can also view the history of the flaky tests and the number of times the test has failed in the past. This helps you to identify the flaky tests and fix them to improve the quality of your test suite. - -Flaky Test Detection serves various use cases, including: - -1. **Identifying Flaky Tests:** Detecting tests that produce inconsistent results due to environmental factors, manual errors or other factors. -2. **Improving Test Reliability:** Providing insights into the reliability of your automated tests. -3. **Identifying Sources of Flakiness:** Pinpointing potential sources of flakiness, such as specific commands within a test. -4. ** [Customization:](/support/docs/test-intelligence-flakiness-test-detection/#customizing-flaky-test-detection-)** Allowing you to tailor Flaky Test Detection settings to your specific testing environment and requirements. -5. **Flaky Test Analytics:** Analysing the flaky tests and the number of times the test has changed in the past with pattern analysis using Flaky Test Analytics. - -**Flaky Test Analytics** -To read more about the Flaky Tests Analytics, you can refer the documentation at [Flaky Tests Analytics](/docs/analytics-modules-test-intelligence-flaky-test-analytics/) - -## How to use Flaky Test Detection? - -There are following Prerequisites to use Flaky Test Detection: - -- You should have a TestMu AI account. If you don't have one, you can sign up here. -- You should have a test suite with same test name run on the platform at least 10 times. - -## How does the Flaky Test Detection work? - -### What are Flaky Tests? - -Flaky tests refer to tests that exhibit inconsistent behavior when executed multiple times under the same conditions. These tests may produce varying results (pass or fail) when run repeatedly, even without any changes to the application or test script. - -### Flaky Test Detection Logics Supported - -**Summary:** - -Flaky Test Detection on the platform supports two complementary methods for identifying genuinely flaky tests. However, only one logic can be active at a time, based on user selection in the configuration. The detection of flaky tests will be performed exclusively using the selected logic: - -1. **Command Logs Mapping:** - - Detects flakiness by analyzing transitions in test status and mapping command logs across test runs. - - Useful for identifying inconsistencies at the command level, such as when specific steps within a test behave differently across executions. - -2. **Error Message Comparison:** - - Detects flakiness by comparing error messages (custom remark messages set by the user) between consecutive test runs. - - A test is classified as flaky only if it fails with *different* error messages across runs. If a test fails consistently with the *same* error message, it is not considered flaky. - - Reduces false positives by focusing on genuinely unstable tests. - -**Configuration:** -- Users can select either Command Logs Mapping or Error Message Comparison as the active logic for flaky test detection in their settings. -- Only the selected logic will be applied for the detection of flaky tests. The other logic will not be considered until selected. - -**How it works:** - -- The system captures and stores both command logs and error messages from each test run. -- Flakiness is detected using only the logic chosen by the user: - - If Command Logs Mapping is selected, only command log inconsistencies are analyzed. - - If Error Message Comparison is selected, only error message differences are analyzed. -- Tests are marked as flaky if the selected method identifies inconsistent behavior or error messages. - -**Examples:** -- If Command Logs Mapping is active: A test's command logs show different outcomes for the same command across runs → flaky. -- If Error Message Comparison is active: A test fails with "TimeoutError" in one run and "ElementNotFound" in another → flaky. -- If a test fails with the same error message and consistent command logs, it is not considered flaky under either logic. - -**Technical Details:** -- The test result processing pipeline extracts and stores both command logs and error messages for each test run. -- Flakiness detection logic applies only the selected method. -- The database schema supports storage for both command logs and error messages. -- Logging is added to track the decision-making process for the active detection method. -- The UI reflects the detection mechanism, showing command log history or error message history based on the selected logic. -- Backward compatibility is maintained with existing test history data. - -**Business Value:** -- Offers flexibility and precision in flaky test detection. -- Reduces false positives and helps teams focus on genuinely unstable tests. -- Improves the reliability of the test insights dashboard. -- Enables more targeted test stabilization efforts. - -**Supported Frameworks** -Currently, these features only support `Selenium` based tests for Web Automation and HyperExecute subscribers. Integration with Playwright reporter infrastructure is ensured. Support for additional frameworks is upcoming. - -## How to view the Flaky Test Detection for Web Automation tests? - -To view the Flaky Test Detection report, you can follow the below steps: - -- Go to the [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/). -- Select the Build for which you want to view the flaky tests list. -- For flaky marked tests, you will see a Flake icon in the test list. -- Now, select the test for which you want to view the flaky test details. -- You will be redirected to the Test Details page. Here, you can view the flaky test details in the right corner of the page. - -## Exploring Different Metrics in Flaky Test Detection - -In Flaky Test Detection, several metrics play a crucial role in assessing and categorizing test results. Understanding these metrics is essential for identifying and addressing inconsistencies in automated testing. Please note that some metrics are only relevant when the **Command Logs Mapping** logic is selected. - -1. **Flake Rate** - The Flake Rate is a metric that measures the current flake rate within a specific group of tests. In Flaky Test Detection, tests are grouped based on criteria such as test name, browser, operating system (OS), and resolution. This metric assesses the inconsistency across test runs of the same name by quantifying transitions in test status and provides valuable insights into the collective performance of similar tests under varying conditions. - - **Use Case:** Picture a group of tests. The Flake Rate serves as a measure of the degree of consistency or inconsistency observed in the test status when these tests are executed repeatedly. - -2. **Flaky Commands** *(Available only with Command Logs Mapping logic)* - The Number of Flaky Commands is a metric used to define the order of flakiness of an individual test. It signifies the inconsistency of different commands within the same test. - - **Use Case:** Consider a single test that performs a series of commands. The number of flaky commands will let the user know how many commands have inconsistencies in a single test. - -3. **Command Flakiness Percentage** *(Available only with Command Logs Mapping logic)* - The Command Flakiness Percentage is a metric employed to define the order of flakiness of a specific command within a test. It assesses the inconsistency of the same command executed across different test runs within the same group. This level of granularity helps pinpoint potential sources of flakiness at the command level. - - **Use Case:** Within a test, individual commands may produce inconsistent results. The Command Flakiness Percentage identifies the fluctuations in each command, aiding in precise root cause analysis. - -Understanding these metrics allows you to: - -- Assess the overall consistency of tests within a group. -- Identify specific tests that exhibit inconsistent behavior. -- Drill down to individual commands to pinpoint sources of flakiness (when Command Logs Mapping is active). -- Prioritize improvements and optimizations based on the level of inconsistency revealed by these metrics. - -## Customizing Flaky Test Detection -### How to use the Custom Flaky Test Detection settings? - -Go to the `Test Intelligence` section in `Org Product Preferences` under `Organization Settings` to locate the following settings: - -- **Grouping Mechanism:** - The grouping mechanism provides you with the ability to determine the criteria for categorizing tests for flakiness . By default, tests are grouped based on your testing environment specified in the [capabilities](https://www.testmuai.com/capabilities-generator/), which include parameters such as test name, browser, operating system (OS), and resolution. - -**Test Capabilities** - For instance, if you execute a test suite for the login functionality of your application, the test group name could be designated as `Login` in the capabilities of your test. -``` -const capability = { -"browserName": "Chrome", -"browserVersion": "118.0", -"LT:Options": { -"username": "", -"accessKey": "", -"platformName": "Windows 10", -"project": "Flaky Test Detection", -"name": "Login", -"w3c": true, -"plugin": "node_js-node_js" -} -} -``` - - The primary criterion for grouping is the `test name`, and you also have the flexibility to tailor this grouping according to your specific needs. This customization can be achieved by deselecting one or more of the environment variables, allowing you to precisely define how tests are categorized. - -- **Sliding Window of previous test runs:** -The sliding window represents the number of preceding tests taken into account when assessing the potential flakiness of the current test. By default, this threshold is configured to include the last 10 tests in the analysis. Altering this threshold allows you to customize the historical data considered for evaluation. Increasing the threshold may provide a more extensive historical context, potentially identifying subtle patterns or fluctuations in test results. Conversely, reducing the threshold can lead to a more focused assessment, ideal for quicker insights when dealing with a large number of tests. Finding the right balance in selecting the sliding window threshold is key to optimizing your test reliability analysis. - -- **Flake Rate Threshold:** -You can set the flake rate threshold to define when a test is marked as flaky. The default threshold is 20%. Tests crossing this threshold will be marked flaky . By adjusting the threshold, you can fine-tune the sensitivity of flakiness detection. Lowering it may result in more tests being flagged as flaky, helping you catch minor inconsistencies, while increasing it can provide a higher level of tolerance, focusing on more significant deviations in test behavior. Selecting an appropriate threshold empowers you to tailor your flakiness detection precisely to your testing needs. - -## Identifying flakiness in test status -Flakiness is characterized by the inconsistency observed in prior executions of the same test, which implies that a test can be deemed flaky even in the absence of flaky individual commands within the test. Since flakiness is predicated on the variability of the test's status across executions, a test will retain its flaky classification until its Flake Rate falls below the predefined threshold. The decline in the Flake Rate corresponds to achieving consistency in test executions. - -## Support for Cypress, Playwright, Puppeteer and Appium - -We are actively working on extending its Flaky Test Detection capabilities to include popular testing frameworks like `Cypress, Playwright, Puppeteer, and Appium`. This expansion will empower users to assess and manage test reliability across a broader spectrum of testing scenarios in both web and app based tests, enhancing the overall testing experience. Stay tuned for updates on the availability of Flaky Test Detection support for these frameworks. diff --git a/static/docs/test-intelligence-overview.md b/static/docs/test-intelligence-overview.md deleted file mode 100644 index 72bae9ff0..000000000 --- a/static/docs/test-intelligence-overview.md +++ /dev/null @@ -1,49 +0,0 @@ -# AI Native Test Intelligence For Advance Test Analysis - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Introduction to Test Intelligence - -With advanced AI algorithms and powerful analysis capabilities, Test Intelligence empowers you to enhance the quality and efficiency of your testing processes. - -Join the wait list today and be the first to experience the power of Test Intelligence. We will notify you as soon as the platform is ready for use. - -## Features of Test Intelligence - -### Intelligent Flakiness Detection of WebDriver Command logs - -Experience the power of our AI algorithms that can detect inconsistent test results in your WebDriver command logs. With this feature, identifying flaky tests becomes a breeze, allowing you to focus on resolving issues effectively. - -### Error Classification of Command Logs - -Our platform goes beyond simple error detection by classifying different types of errors in your command logs. This classification enables you to prioritize and address issues accordingly, ensuring a more efficient and streamlined QA process. - -### Command Logs Error Trends Forecast - -Harness the capabilities of our advanced algorithms to identify trends in errors. By leveraging historical data and predictive analysis, Test Intelligence helps you anticipate and prevent future issues, ultimately improving the overall quality of your applications. - -### Anomalies in Test Execution Across Environments - -Detecting anomalies in test execution is crucial for ensuring consistency and reliability. Our platform excels in identifying and highlighting anomalies that occur across various environments, providing you with valuable insights to optimize your application's performance. - -## Advantages of Test Intelligence - -By leveraging the features of Test Intelligence, you unlock numerous advantages for your QA testing processes: - -- Streamlined and efficient identification of flaky tests. -- Enhanced error management and prioritization. -- Proactive issue prevention through trend forecasting. -- Improved application consistency across different environments. -- Time and resource savings with automated analysis. -- Data-driven decision-making for test optimization. - -## Get Started with Test Intelligence - -Ready to experience the power of Test Intelligence? Getting started is simple: - -1. Sign up for our Test Intelligence platform and gain access to a world of transformation testing capabilities. -2. Integrate your Automation test suite with Web Automation plan and start leveraging the intelligent features of our platform. -3. Explore the intuitive interface and dive into the automated analysis of your test data. -4. Discover valuable insights, address issues proactively, and optimize your testing process for better results. - -Don't miss out on the opportunity to revolutionize your QA testing. Join us on this exciting journey with Test Intelligence and take your testing efforts to new heights. Get started today and unlock the full potential of your applications. diff --git a/static/docs/test-langgraph-agents.md b/static/docs/test-langgraph-agents.md deleted file mode 100644 index d0f20314c..000000000 --- a/static/docs/test-langgraph-agents.md +++ /dev/null @@ -1,49 +0,0 @@ -# How to Test LangGraph Agents With TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -LangGraph agents are text agents exposed over HTTP, with state held in a graph and threaded across turns. There is no voice or phone surface. TestMu AI connects to the deployed endpoint, drives full multi-turn conversations across personas and edge cases, and scores every run. - -## Before You Begin - -Before you connect a LangGraph agent, make sure you have: - -- A deployed graph with a reachable endpoint, self-hosted or on LangGraph Platform -- TestMu AI workspace with agent-testing permissions -- Source docs for scenario generation: prompt, PRD, or knowledge base - -## Test a LangGraph Chat Agent - -Chat testing covers text conversations against a deployed graph, whether streaming or non-streaming. Upload the graph definition and supporting docs to generate scenarios, then create the endpoint profile using the deployment's REST endpoint, with a fresh thread per scenario. It scores reasoning, tool call correctness, grounding, and safety, and runs in CI on every commit. - -Full setup: [Chat agent testing](/support/docs/chat-agent/) - -## What You Get With Agent Testing - -Every LangGraph run, on any surface it supports, is scored across: - -- 30+ metrics across 8 categories, with configurable thresholds -- Context-aware scenario generation -- Adversarial testing and automated issue detection -- Multilingual conversations -- Quality scoring -- Automated issue detection - -## LangGraph-Specific Considerations - -A few LangGraph behaviours are worth building dedicated scenarios around: - -- State lives on the thread, so reusing one across scenarios contaminates results -- Human-in-the-loop interrupts pause the graph mid-run, and a harness that does not resume reads the pause as a hang -- Tool calls and their failure branches are where these agents actually break, so weight scenarios there -- Deployment varies across self-hosted, LangGraph Platform, and custom servers, which changes the endpoint shape -- There is no voice or phone surface, so voice metrics and personas do not apply - -## Troubleshooting - -Common failure modes to watch for: - -- Thread state carried over between runs -- Graph paused on an interrupt -- Tool call returns malformed output -- Endpoint auth failure diff --git a/static/docs/test-livekit-agents.md b/static/docs/test-livekit-agents.md deleted file mode 100644 index 670df79f0..000000000 --- a/static/docs/test-livekit-agents.md +++ /dev/null @@ -1,56 +0,0 @@ -# How to Test LiveKit Agents With TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -LiveKit is infrastructure rather than a hosted builder. The agent joins a WebRTC room as a participant, and phone calls arrive in that same room through a SIP trunk. TestMu AI connects to the room or the number, drives full multi-turn conversations across personas and edge cases, and scores every run. - -## Before You Begin - -Before you connect a LiveKit agent, make sure you have: - -- A running LiveKit agent, self-hosted or on LiveKit Cloud -- A SIP trunk with a provider such as Twilio or Telnyx, for phone agents only -- TestMu AI workspace with agent-testing permissions -- Source docs for scenario generation: prompt, PRD, or knowledge base - -## Test a LiveKit Phone Agent - -Phone testing covers inbound and outbound callers arriving over a SIP trunk. Paste the agent's prompt to generate scenarios, then add the number provisioned on the trunk and pick from 100+ voices, background noise conditions, and personas. It covers DTMF entry, SIP REFER transfers, voicemail handling, and carrier latency. - -Full setup: [Phone agent testing](/support/docs/phone-agent/) - -## Test a LiveKit Voice Agent - -Voice testing applies to agents reached by joining a room directly rather than over a phone number. Paste the prompt to generate scenarios, then create the endpoint profile using WebRTC against the room. It isolates the speech pipeline, so STT and TTS accuracy, turn-taking, and interruption handling get tested without telephony noise in the results. - -Full setup: [Voice agent testing](/support/docs/voice-agent/) - -## What You Get With Agent Testing - -Every LiveKit run, on any surface it supports, is scored across: - -- 30+ metrics across 8 categories, with configurable thresholds -- Context-aware scenario generation -- Adversarial testing and automated issue detection -- 100+ voices and personas -- Multilingual conversations -- Quality scoring and real-time call monitoring - -## LiveKit-Specific Considerations - -A few LiveKit behaviours are worth building dedicated scenarios around: - -- The unit under test is a room, not an agent ID, so setup differs from every hosted platform -- SIP REFER must be enabled on the trunk for transfers, and Twilio additionally needs PSTN transfer switched on -- Connectors bridge Twilio calls over WebSocket without a SIP trunk, which is a separate connection path -- DTMF, cold transfer, and region pinning are trunk-level settings, so a failure may sit in the trunk rather than the agent -- There is no text channel to test, so the page covers phone and voice only - -## Troubleshooting - -Common failure modes to watch for: - -- Agent not present in the room -- SIP REFER rejected -- DTMF not received -- Codec negotiation failure diff --git a/static/docs/test-locally-hosted-websites-with-selenium.md b/static/docs/test-locally-hosted-websites-with-selenium.md deleted file mode 100644 index 42b743c81..000000000 --- a/static/docs/test-locally-hosted-websites-with-selenium.md +++ /dev/null @@ -1,154 +0,0 @@ -# Test Locally Hosted Websites With Selenium - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI Tunnel creates a secure, encrypted connection between your local machine and the TestMu AI cloud grid. Use it to test web apps running on `localhost`, staging servers, or behind a firewall without exposing them to the internet. The tunnel uses TCP with TLS 1.2 over port 443 or WebSocket. - -## How It Works - -The tunnel binary runs on your machine and establishes an encrypted connection to the TestMu AI cloud. When you set `tunnel: true` in your test capabilities, the cloud browser routes traffic through this connection to reach your local app. - -``` -Your Machine (localhost:3000) <-- Encrypted Tunnel --> TestMu AI Grid <--> Cloud Browser -``` - -## Step 1: Download the Tunnel Binary - -Download the binary for your operating system and architecture. - -| Platform | Download Link | -|----------|---------------| -| **Windows** (AMD64) | https://downloads.lambdatest.com/tunnel/v3/windows/amd64/LT.exe | -| **macOS** (AMD64) | https://downloads.lambdatest.com/tunnel/v3/mac/amd64/LT | -| **macOS** (ARM64 / Apple Silicon) | https://downloads.lambdatest.com/tunnel/v3/mac/arm64/LT | -| **Linux** (AMD64) | https://downloads.lambdatest.com/tunnel/v3/linux/amd64/LT | -| **Linux** (ARM64) | https://downloads.lambdatest.com/tunnel/v3/linux/arm64/LT | - -For the full list of platform binaries, see the [TestMu AI Tunnel documentation](/support/docs/testing-locally-hosted-pages/). - -After downloading, make the binary executable (macOS/Linux): - -```bash -chmod +x LT -``` - -## Step 2: Start the Tunnel - -Open a terminal, navigate to the folder with the binary, and start the tunnel with your credentials. - - {`LT --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()}`} - -You should see a message confirming the tunnel is active. Keep this terminal open while running your tests. - -**Named tunnels** -Run multiple tunnels by giving each a name: - -```bash -LT --user YOUR_USERNAME --key YOUR_ACCESS_KEY --tunnelName my-staging-tunnel -``` - -Then reference it in your capabilities with `"tunnelName": "my-staging-tunnel"`. - -## Step 3: Add the Tunnel Capability to Your Tests - -Set `tunnel: true` in your test capabilities. Here is how to do it in each language. - -```java -ChromeOptions browserOptions = new ChromeOptions(); -browserOptions.setPlatformName("Windows 10"); -browserOptions.setBrowserVersion("latest"); - -HashMap ltOptions = new HashMap(); -ltOptions.put("build", "Local Test Build"); -ltOptions.put("name", "Local Test"); -ltOptions.put("tunnel", true); // Enable tunnel -ltOptions.put("w3c", true); -browserOptions.setCapability("LT:Options", ltOptions); -``` - -```javascript -const capabilities = { -browserName: 'Chrome', -browserVersion: 'latest', -'LT:Options': { -platformName: 'Windows 10', -build: 'Local Test Build', -name: 'Local Test', -tunnel: true, // Enable tunnel -w3c: true, -} -}; -``` - -```python -lt_options = { -"build": "Local Test Build", -"name": "Local Test", -"tunnel": True, # Enable tunnel -"w3c": True, -} -options.set_capability("LT:Options", lt_options) -``` - -```csharp -var ltOptions = new Dictionary -{ -{ "build", "Local Test Build" }, -{ "name", "Local Test" }, -{ "tunnel", true }, // Enable tunnel -{ "w3c", true } -}; -browserOptions.AddAdditionalOption("LT:Options", ltOptions); -``` - -```php -$capabilities = [ -"browserName" => "Chrome", -"browserVersion" => "latest", -"LT:Options" => [ -"platformName" => "Windows 10", -"build" => "Local Test Build", -"name" => "Local Test", -"tunnel" => true, // Enable tunnel -"w3c" => true, -] -]; -``` - -```ruby -options = Selenium::WebDriver::Options.chrome( -platform_name: "Windows 10", -browser_version: "latest", -"LT:Options": { -build: "Local Test Build", -name: "Local Test", -tunnel: true, # Enable tunnel -w3c: true, -} -) -``` - -If you are using a named tunnel, add `"tunnelName": "my-staging-tunnel"` to the `LT:Options` as well. - -## Step 4: Point Your Test to localhost - -Use `localhost` or your internal URL in `driver.get()`. The tunnel routes the request from the cloud browser to your local machine. - -```java -// This works because the tunnel is active -driver.get("http://localhost:3000"); -``` - -## Common Issues - -Quick fixes for problems you might hit with local testing. - -| Problem | Solution | -|---------|----------| -| Tunnel won't start | Check that your username and access key are correct. Ensure no firewall is blocking outbound connections over port 443. | -| Page loads but shows "connection refused" | Verify your local app is running on the port you specified. Check that the app binds to `0.0.0.0`, not just `127.0.0.1`. | -| Tests timeout on local URLs | Increase the idle timeout capability: `"idleTimeout": 300`. Local apps may respond slower through the tunnel. | -| Multiple tunnels conflicting | Use named tunnels with `--tunnelName` and reference the name in capabilities with `"tunnelName"`. | -| SSL certificate errors on localhost | Start the tunnel with `--mitm` flag to test sites with self-signed certificates on your local machine or internal network. | - -For YAML-based configuration, verbose logging, Info APIs, MITM mode, and all advanced options, see the [TestMu AI Tunnel documentation](/support/docs/testing-locally-hosted-pages/) and [TestMu AI Tunnel Features](/support/docs/advanced-tunnel-features/). diff --git a/static/docs/test-logs.md b/static/docs/test-logs.md deleted file mode 100644 index 7eb0e730c..000000000 --- a/static/docs/test-logs.md +++ /dev/null @@ -1,33 +0,0 @@ -# Test Logs - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Test Logs is a library where you can find logs of all cross browser testing performed by you on TestMu AI. - -## Accessing Test Logs - -**Step 1:** Log in to your account. - -**Step 2:** Select **'Test Logs'** from the menu bar on the left-hand side. - -**Step 3:** After selecting **'Test Logs'**, you'll see a list of tests you performed. Select any of the tests to get the details. - -## From Where Can I Access The Details Of My Project 'Untitled'? - -**Step 1:** Open **'Test Logs'** from the menu bar. - -**Step 2:** From the list of projects, you can select any of the projects. Select the project named **Untitled**. - -**Step 3:** From the panel on the right-hand side, you can access the details of the project tests performed in the project **Untitled**. - -## How To Move A Test From One Project To Another? - -**Step 1:** From the **'Test Logs'** menu, open the project from where you want to move the test. - -**Step 2:** Open the test to be moved. - -**Step 3:** Click on the three dots present on the right-hand side under **Test Details**. - -**Step 4:** Select the project and project version where you want to move the test. - -**Step 5:** Once you select, the test is moved to your desired project. diff --git a/static/docs/test-manager-mcp-server.md b/static/docs/test-manager-mcp-server.md deleted file mode 100644 index 9e95dc7b0..000000000 --- a/static/docs/test-manager-mcp-server.md +++ /dev/null @@ -1,153 +0,0 @@ -# Getting Started with Test Manager MCP Tool - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Test Manager MCP Tool is part of the [TestMu AI MCP Server](/support/docs/testmu-mcp-server/). It connects your AI client to [Test Manager](/support/docs/test-manager/) so an assistant can run the complete test management lifecycle on your behalf: generate test cases from requirements with AI, organize them into folders, build test runs with the right configurations, record results in bulk, track milestones, link work to Jira or Azure DevOps, and report on coverage, without leaving your IDE or chat. - -Everything the assistant does happens in your own Test Manager projects, filtered to what your account is allowed to see. - -## How Does It Work? - -Once connected to the TestMu AI MCP Server, your AI client discovers a set of tools prefixed `tm_`. You describe what you want in natural language and the client picks the right tools, in the right order. A typical flow: - -1. **Discover:** list projects, then fetch the folder tree, existing Modules, and saved Configurations to reference by id. -2. **Author:** generate test cases from a requirement with AI, or create individual cases with steps. -3. **Execute:** create a milestone, build a run from chosen cases with Configurations applied, and record results in bulk as testing progresses. -4. **Report:** check run summaries, milestone progress, and automation coverage on demand. - -## Available Tools - -The Test Manager MCP Tool exposes 22 tools: 8 to read and discover, 14 to act. **Bold** inputs are required. Ids are referenced across tools: projects, folders, cases, runs, and milestones use ULID strings, configurations and run instances use numbers. - -### Projects - -Project and Organization Instructions (the Memory Layer) stay managed in the Test Manager UI. The AI generator applies them automatically. - -| Tool | Description | Inputs | Example prompt | -|---|---|---|---| -| `tm_list_projects` | Lists every project the user is allowed to see. Returns each project's id, name, description, and tags, paginated. | `search`, `page`, `limit` | "List my Test Manager projects that mention payments." | -| `tm_create_project` | Creates a project. Returns the new project id. | **`name`**, `description`, `tags` | "Create a Test Manager project called Mobile Banking with the tag q3-release." | -| `tm_update_project` | Updates a project's name, description, or tags. Only passed fields change, `tags` replace the current set. | **`project_id`**, `name`, `description`, `tags` | "Rename the Mobile Banking project to Mobile Banking App." | - -### Folders and Modules - -Modules are reusable step blocks shared across projects. The assistant references an existing Module by id when creating or editing a test case, which includes the Module's steps. The Module itself is never changed through the connection. Creating Modules stays in the Test Manager UI. - -| Tool | Description | Inputs | Example prompt | -|---|---|---|---| -| `tm_list_folders` | Returns a folder tree with ids, names, and nesting. Three trees exist: per-project `test-cases` (default) and `test-runs`, and the organization-level `modules`. `project_id` is required for the two per-project trees. | `tree`, `project_id` | "Show the test case folder tree for the Payments project." | -| `tm_create_folder` | Creates a folder in any of the three trees, optionally nested under a parent. Returns the new folder id. | **`name`**, `tree`, `project_id`, `parent_folder_id`, `description` | "Create a P0 Regression folder under Checkout in the Payments project." | -| `tm_update_folder` | Renames a folder, changes its description, or moves it under a different parent (`root` moves it to top level). | **`folder_id`**, `name`, `description`, `parent_folder_id`, `tree`, `project_id` | "Move the Smoke folder under Regression and rename it to Smoke Pack." | -| `tm_list_modules` | Lists the organization's reusable step [Modules](/support/docs/create-modules/) with their ids, paginated. | `search`, `folder_id`, `include_subfolders`, `page`, `limit` | "List our shared step Modules that mention login." | - -### Test Cases - -| Tool | Description | Inputs | Example prompt | -|---|---|---|---| -| `tm_get_test_cases` | Finds cases by filters and search. Filters combine with AND, search needs at least 3 characters and matches title, description, steps, or a case id like TC-123. Returns summaries. With `case_id`, returns that one case in full detail: steps, BDD scenarios, and field values. | `case_id`, `project_id` (required unless `case_id` is given), `folder_id`, `priority`, `automation_status`, `type`, `tags`, `search`, `page`, `limit` | "Find all High priority, not automated Regression cases in the Checkout folder." | -| `tm_create_test_case` | Creates one case with steps, preconditions, priority, type, tags, and referenced Modules. Each entry in `test_steps` has `action` (required), `outcome`, and `description`. Returns the new case id. | **`project_id`**, **`title`**, `folder_id`, `description`, `preconditions`, `test_steps`, `module_ids`, `priority`, `type`, `status`, `automation_status`, `estimated_time`, `tags` | "Create a High priority case titled Verify 3DS payment in the Checkout folder." | -| `tm_update_test_case` | Edits a case. Only passed fields change, `test_steps` replace the case's steps, `module_ids` append Module reference steps. | **`case_id`**, **`project_id`**, plus the same optional fields as `tm_create_test_case` | "Mark TC-101 as Automated and move it to the Automated folder." | -| `tm_generate_test_cases` | Generates cases from a requirement with the Test Manager AI generator. Returns a `request_id` and a browser progress URL immediately, poll with `request_id` for results. Saving returns every saved case with its new id, grouped by scenario. See [Generating Test Cases with AI](#generating-test-cases-with-ai). | **`prompt`**, **`project_id`**, **`folder_id`** (all three for a new generation), `test_scenario_limit`, `per_scenario_test_cases_limit`, `jira_ids`, `files`, `auto_save`, `request_id` | "Generate 15 checkout cases from this PRD into the P0 Regression folder, and let me review them before saving." | - -### Test Runs and Results - -Each test case included in a run becomes a **test run instance**: the executable copy of that case inside the run, carrying its own status, remarks, and exactly one Configuration. - -| Tool | Description | Inputs | Example prompt | -|---|---|---|---| -| `tm_list_configurations` | Lists saved run Configurations with their top-level id, name, and environments (OS, browser, resolution, or device). Pass the top-level id to the run tools, not the nested environment id. Filter by `run_type` to see what a manual or KaneAI run accepts. | `run_type`, `project_id`, `page`, `limit` | "Which saved configurations can I use for a KaneAI run?" | -| `tm_create_test_run` | Creates a manual (default) or KaneAI run from chosen cases. Returns the new run id and its `run_type`. | **`project_id`**, **`title`**, `test_case_ids`, `run_type`, `configuration_id`, `configuration_ids`, `milestone_ids`, `folder_id`, `assignee`, `objective`, `tags`, `is_sequential` | "Create a run called Sprint 47 Regression from the Checkout folder's High priority cases, apply our Chrome and Safari configurations, and assign the instances to priya@example.com." | -| `tm_update_test_run` | Updates an open run: overall status (manual runs only), title, objective, add cases, apply Configurations, attach milestones, move folder, assign, or archive. Adding appends, never replaces. | **`run_id`**, `status`, `title`, `objective`, `add_test_case_ids`, `configuration_id`, `configuration_ids`, `apply_to_test_case_ids`, `milestone_ids`, `folder_id`, `assignee`, `is_sequential`, `archive` | "Add TC-150 and TC-151 to the Sprint 47 Regression run and mark the run In Progress." | -| `tm_get_test_runs` | With `run_id`, one run's detail plus its result summary: instance counts by status (passed, failed, skipped, not started, in progress) and pass rate. Otherwise lists runs with filters. Every run carries `run_type`. | `run_id`, `project_id` (required without `run_id`), `run_type`, `status`, `search`, `folder_id`, `page`, `limit` | "Show the result summary of the Sprint 47 Regression run." | -| `tm_record_test_results` | Records up to 500 results on a manual run's instances, all or nothing. Each result has `status` (required), `instance_id` or `test_case_id` to pick the instance, and optional `remarks`, `assignee`, and per-step `steps`. Echoes every recorded result and per-step outcome. See [Recording Results in Bulk](#recording-results-in-bulk). | **`run_id`**, **`results`** | "In the Sprint 47 run, mark TC-101 to TC-140 as Passed and fail TC-141 with the remark payment timeout on 3DS." | - -### Milestones - -| Tool | Description | Inputs | Example prompt | -|---|---|---|---| -| `tm_create_milestone` | Creates a [milestone](/support/docs/milestone-creation-and-management/), starting as Open, with optional dates, owner, tags, and attached runs. Returns the new milestone id. | **`project_id`**, **`title`**, `description`, `start_at`, `end_at`, `owner`, `tags`, `test_run_ids` | "Create a Sprint 47 milestone from 2026-07-20 to 2026-08-02 owned by priya@example.com." | -| `tm_update_milestone` | Updates a milestone or flips its status between `Open` and `Complete` (`Active` and `Completed` are accepted as aliases and normalized). | **`milestone_id`**, **`project_id`**, `title`, `description`, `start_at`, `end_at`, `status`, `owner`, `tags` | "Mark the Sprint 47 milestone as Complete." | -| `tm_get_milestone` | With `milestone_id`, one milestone's detail with attached runs and live completion progress. Otherwise the project's milestone list. | `milestone_id`, `project_id` (required without `milestone_id`) | "How far along is the Sprint 47 milestone?" | - -### Issue Tracking and Coverage - -Linking a run result also links its test case and run. Both linking tools need the corresponding integration set up for your organization first. If it is not, the tool returns a clear "set up the integration first" message. - -| Tool | Description | Inputs | Example prompt | -|---|---|---|---| -| `tm_link_jira_issue` | Links a case, run, or run result to a [Jira issue](/support/docs/link-jira-issues-with-test-manager/) for coverage and traceability. `entity_type` is `test_case`, `test_run`, or `test_run_instance`. | **`entity_type`**, **`entity_id`**, **`issue_id`** | "Link test case TC-88 to Jira issue PROJ-456." | -| `tm_link_ado_issue` | Links a case, run, or run result to an [Azure DevOps work item](/support/docs/link-ado-issues-with-test-manager/) by its URL. | **`entity_type`**, **`entity_id`**, **`project_id`**, **`work_item_url`** | "Link the Sprint 47 run to this Azure DevOps work item URL." | -| `tm_get_coverage_summary` | Test coverage computed live: case totals and automation percentage, overall and per folder. With `jira_id`, that requirement's linked-case coverage (zero linked cases means uncovered). An empty scope returns zeros, not an error. | **`project_id`**, `folder_id`, `jira_id` | "Which folders in the Payments project have the lowest automation coverage?" | - -## Generating Test Cases with AI - -`tm_generate_test_cases` drives the same [AI test case generation](/support/docs/generate-test-cases-with-ai/) you get in the Test Manager UI, applying your Project and Organization Instructions automatically. A few behaviors worth knowing: - -- **Generation is asynchronous.** The call returns fast with a `request_id` and a browser progress URL. Generation takes roughly 30 to 90 seconds. The assistant polls with the `request_id` to check progress and fetch results. -- **Volume is capped at 50 cases per call.** Volume = scenarios x cases per scenario (both default to 3). -- **Review before saving.** By default all generated cases are saved into the chosen folder, one sub-folder per scenario. Pass `auto_save: false` to review first. That choice persists across polls, and nothing is saved until the assistant calls again with the `request_id` and `auto_save: true`. The save call returns every saved case with its new id. -- **Ground the generation with context.** Reference files (requirement docs, specs, spreadsheets, screenshots) can be attached, and Jira issue keys can be passed as requirement context. - -AI generation consumes AI generation credits, the same as generating from the UI. - -## Working with Configurations - -Configurations are reusable platform, browser, OS, and device setups, created in the Test Manager UI and referenced by id. Two parameters control how they apply to a run: - -- `configuration_ids` (plural) applies **multiple** Configurations: each targeted test case gets one instance per Configuration (case x configuration), like the Add Configuration checkboxes in the UI. On update it appends new instances and leaves existing ones untouched. -- `configuration_id` (singular) sets **one** Configuration on the run's instances. On update this follows the UI's latest-selection behavior and replaces an instance's current Configuration. - -Configurations are validated against the run's type. Applying one that does not support the run (manual or KaneAI) is rejected with a clear message and nothing is applied. Use the `run_type` filter on `tm_list_configurations` to see what is usable. - -To apply several Configurations, or to add one without touching what is already applied, use `configuration_ids`. - -## KaneAI Test Runs - -Test runs are manual by default. Passing `run_type: "kaneai"` to `tm_create_test_run` creates a KaneAI run instead: - -- Only KaneAI-authored test cases with completed code generation can be included. Mixing in a manual case is rejected with a clear message and nothing is written. -- Configurations applied to a KaneAI run must be KaneAI-supported and platform-compatible with each case. `tm_list_configurations` accepts a `run_type` filter to show only usable ones. -- Instances can be set to execute sequentially with `is_sequential`. -- A KaneAI run's status and results are managed by KaneAI executions. Manual status updates and manual result entry on a KaneAI run are rejected. - -Every run returned by `tm_get_test_runs` carries a `run_type` field of `kaneai` or `manual`, and the run list filters by `run_type` into three disjoint buckets: `manual`, `kaneai`, and `automation` (framework-uploaded runs). - -## Recording Results in Bulk - -`tm_record_test_results` records execution results on a manual run's instances, targeting each by `instance_id` or simply by `test_case_id`: - -- **Up to 500 results per request, all or nothing.** Larger requests are rejected cleanly and nothing is recorded. -- Instance statuses are `Passed`, `Failed`, `Skipped`, or `Not Started`, each with optional remarks and an assignee. -- Individual test steps and BDD scenario rows can be marked with per-step statuses and remarks, like the per-step Mark Status in the UI. Re-recording a step replaces its result, and an omitted step remark is cleared. -- A case that runs under several Configurations has one instance per Configuration. `test_case_id` resolves to a single instance, so target multi-configuration cases by `instance_id` (from `tm_get_test_runs` detail). -- Instance results are independent of the run's **overall** status. The overall status (`Passed`, `Failed`, `Skipped`, `In Progress`) is set separately through `tm_update_test_run`. - -## How to Invoke - -Once your MCP client is connected, use natural language. Multi-step prompts compose several tools in one request: - -- "List my Test Manager projects and show the folder tree for the Payments project." -- "Generate 15 checkout test cases from this PRD into the P0 Regression folder, must-haves as High priority." -- "Create a Sprint 47 milestone and a regression run of all High priority cases from the Checkout folder, apply our Chrome Windows and iOS Safari configurations, and attach the run to the milestone." -- "Mark TC-101 to TC-140 as Passed in run Sprint 47 Regression, and fail TC-141 with remark 'payment timeout on 3DS'." -- "Which folders in this project have the lowest automation coverage?" - -## Guardrails - -The tool set is designed so an assistant cannot cause data loss or overload shared systems: - -- **No delete tools.** Nothing through the connection deletes a project, folder, test case, run, or milestone. Removal stays a human action in the Test Manager UI. Runs can be archived through `tm_update_test_run`. -- **Reusable scaffolding stays human-owned.** Modules and Configurations are created in the UI. The assistant fetches and references them by id. -- **Only open runs can be updated.** Updating a closed or archived run returns a clear message instead of failing silently. -- **Volume caps are all or nothing.** 50 generated cases per call, 500 recorded results per request. Nothing partial is ever written. -- **Errors are plain and recoverable.** Missing integrations, permission limits, and oversized requests each return a message saying what happened and what to do next. - -## Error Handling - -If a referenced id is invalid or a precondition is not met, the tool responds with a plain-language message, for example: - -> "A KaneAI run only takes KaneAI-authored test cases. Incompatible: TC-97. Nothing was written." - -> "The Jira integration is not set up for this organization. Nothing was linked. Set it up in LambdaTest under Integrations, then retry." - -> "This test run is archived. Only open (active) runs can be updated. Re-open or duplicate it in the Test Manager UI first." diff --git a/static/docs/test-manager.md b/static/docs/test-manager.md deleted file mode 100644 index 9af5f6a4d..000000000 --- a/static/docs/test-manager.md +++ /dev/null @@ -1,13 +0,0 @@ -# Introduction to Test Manager - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Test Manager is a comprehensive tool that can be used for test management, test triggering, and test reporting. It serves as a centralized tool that streamlines each stage of the testing process and provides a seamless and efficient experience for users. Test manager offers various functionalities that streamline the process of organizing testing tasks, generating test scenarios, and tracking real-time test results to satisfy the needs of testing experts. - -**Features** - -- Test Manager provides a user-friendly interface for creating detailed and customized test cases, ensuring comprehensive coverage of testing scenarios. - -- Acting as a centralized repository, Test Manager allows users to manage and organize test cases efficiently. Test plans and builds are easily accessible and can be updated seamlessly. - -- Test Manager provides real-time visibility into test steps and progress. This feature enables testers to track the execution status, identify bottlenecks for better understanding. diff --git a/static/docs/test-pipecat-agents.md b/static/docs/test-pipecat-agents.md deleted file mode 100644 index a612e8890..000000000 --- a/static/docs/test-pipecat-agents.md +++ /dev/null @@ -1,56 +0,0 @@ -# How to Test Pipecat Agents With TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Pipecat is an open-source Python framework where the pipeline is assembled by hand: transport, STT, LLM, TTS. It is transport-agnostic, so the connection depends entirely on what the team built. TestMu AI connects to that transport, drives full multi-turn conversations across personas and edge cases, and scores every run. - -## Before You Begin - -Before you connect a Pipecat bot, make sure you have: - -- A deployed Pipecat bot with a reachable transport -- The provider and serializer in use, for telephony deployments -- TestMu AI workspace with agent-testing permissions -- Source docs for scenario generation: prompt, PRD, or knowledge base - -## Test a Pipecat Phone Agent - -Phone testing covers callers arriving over Twilio, Telnyx, or Plivo media streams. Paste the agent's prompt to generate scenarios, then add the number bound to the telephony transport and pick from 100+ voices, background noise conditions, and personas. It covers DTMF entry, transfers, voicemail handling, and carrier latency. - -Full setup: [Phone agent testing](/support/docs/phone-agent/) - -## Test a Pipecat Voice Agent - -Voice testing applies to bots reached over a direct audio stream rather than a phone number. Paste the prompt to generate scenarios, then create the endpoint profile using WebRTC or WebSocket, matching the transport the bot was built on. It isolates the speech pipeline, so STT and TTS accuracy, turn-taking, and interruption handling get tested without telephony noise in the results. - -Full setup: [Voice agent testing](/support/docs/voice-agent/) - -## What You Get With Agent Testing - -Every Pipecat run, on any surface it supports, is scored across: - -- 30+ metrics across 8 categories, with configurable thresholds -- Context-aware scenario generation -- Adversarial testing and automated issue detection -- 100+ voices and personas -- Multilingual conversations -- Quality scoring and real-time call monitoring - -## Pipecat-Specific Considerations - -A few Pipecat behaviours are worth building dedicated scenarios around: - -- No hosted dashboard and no agent ID. Connection details come from the team's own deployment -- Serializers are provider-specific, so a Twilio-serialized stream will not connect to a Telnyx endpoint -- Audio normalization, codec, and sample rate are the developer's responsibility, which makes format mismatch the most common first failure -- Interruption handling is built into the pipeline, so barge-in is a genuine test target rather than a vendor feature -- There is no text channel to test, so the page covers phone and voice only - -## Troubleshooting - -Common failure modes to watch for: - -- Transport unreachable -- Wrong serializer -- Sample rate mismatch -- No audio returned diff --git a/static/docs/test-retell-agents.md b/static/docs/test-retell-agents.md deleted file mode 100644 index 43d2ec9f8..000000000 --- a/static/docs/test-retell-agents.md +++ /dev/null @@ -1,61 +0,0 @@ -# How to Test Retell Agents With TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Retell builds agents as either a single prompt or a node-based Conversation Flow, then deploys that same logic to phone, web voice, chat, and SMS. TestMu AI connects to the agent on whichever surface it runs on, drives full multi-turn conversations across personas and edge cases, and scores every run. - -## Before You Begin - -Before you connect a Retell agent, make sure you have: - -- Retell account with a published agent -- A Retell number or a number connected over SIP trunking, for phone agents only -- TestMu AI workspace with agent-testing permissions -- Source docs for scenario generation: prompt, PRD, or knowledge base - -## Test a Retell Phone Agent - -Phone testing covers inbound, outbound, and batch callers running over the phone network. Paste the agent's prompt or import the Conversation Flow to generate scenarios, then add the number under Phone Numbers and pick from 100+ voices, background noise conditions, and personas. It covers DTMF entry, Press Digit navigation through external IVR menus, transfers, voicemail handling, and carrier latency. - -Full setup: [Phone agent testing](/support/docs/phone-agent/) - -## Test a Retell Voice Agent - -Voice testing applies to agents reached through a web call rather than a phone number. Paste the prompt or import the Conversation Flow to generate scenarios, then create the endpoint profile using the web call API. It isolates the speech pipeline, so STT and TTS accuracy, turn-taking, and interruption handling get tested without telephony noise in the results. - -Full setup: [Voice agent testing](/support/docs/voice-agent/) - -## Test a Retell Chat Agent - -Chat testing covers text conversations through a Retell chat agent or the embedded chat widget. Upload the supporting docs and knowledge base the agent answers from to generate scenarios, then create the endpoint profile using the Chat API. It catches reasoning, tool call, and grounding failures, and can run on every commit. - -Full setup: [Chat agent testing](/support/docs/chat-agent/) - -## What You Get With Agent Testing - -Every Retell run, on any surface it supports, is scored across: - -- 30+ metrics across 8 categories, with configurable thresholds -- Context-aware scenario generation -- Adversarial testing and automated issue detection -- 100+ voices and personas -- Multilingual conversations -- Quality scoring and real-time call monitoring - -## Retell-Specific Considerations - -A few Retell behaviours are worth building dedicated scenarios around: - -- Conversation Flow agents are graphs, so coverage means every node rather than every intent -- Chat agents are created by converting a voice agent, and the two drift apart after conversion. Test both -- The chat widget uses a public key while the API uses a private key. Confirm which one is in play -- Press Digit timing decides whether external IVR navigation succeeds, so DTMF needs dedicated scenarios - -## Troubleshooting - -Common failure modes to watch for: - -- No transcript returned -- Call ends on the wrong node -- DTMF fires early -- Chat session closes on inactivity timeout diff --git a/static/docs/test-run-creation-and-management.md b/static/docs/test-run-creation-and-management.md deleted file mode 100644 index d1db8e9e7..000000000 --- a/static/docs/test-run-creation-and-management.md +++ /dev/null @@ -1,104 +0,0 @@ -# Test Run - Creation and Management - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This guide outlines the steps required to create, configure, and manage test runs within TestMu AI's Test Manager. It provides developers and testers with a clear understanding of the process, enabling efficient test execution and organization. - -## 1. Creating a Test Run -### Initiate a New Test Run -- Navigate to the Test Manager and click on **Create Test Run**. -- Enter a Test Run Name and an optional Description to define the purpose of the test run. - -Use descriptive names to easily identify test runs later. - -### Add Tags -- Assign relevant Tags to categorize the test run. -- Click **Create Test Run** to proceed. - -## 2. Adding and Configuring Test Cases -### Access the Test Cases Tab -- Upon successful creation, you will be redirected to the Test Cases tab. -- Here, you can add multiple test cases to the test run. - -### Assign Test Cases -- Select the desired test cases from the list. -- Assign Assignees to each test case. -- Add test case configurations individually or in bulk. -> Note: Configurations allow you to define environment settings, such as browser and device combinations. - -### Add Configurations -- Choose multiple configurations from the dropdown or create new configurations as needed. -- Apply configurations to the selected test cases. - -## 3. Saving and Managing Test Runs -### Finalize the Test Run -- Click Save Test Run to finalize the setup. -- View the created test run and its associated test instances in the dashboard. - -Test instances are organized using a **Folders** sidebar on the left, reflecting the folder structure of the test cases added to the test run. Only folders whose test cases are included in the test run are displayed, and each folder shows the count of test instances it contains (e.g., `3/3`, `4/4`). - -- Select a folder to filter and view only its test instances. The status summary (Passed, Failed, Not Started, Skipped) is displayed in the top-right corner of the listing for the selected folder. -- Collapse or expand folders for easier navigation. -- Click the **three-dot menu** on the **Folders** header to toggle **Include Child Folders**. When enabled (default), selecting a parent folder also displays test instances from its child folders. - -### Bulk Update Options -- To update multiple test instances: -- Select multiple test instances. -- Use the Select Assignee and Select Status options to apply changes in bulk. - -### Update Test Step Status -- Within any test instance, individually update the status of each test step. -- Add remarks or actual outcomes for manual test steps. - -## 4. Filtering Test Instances - -A filter bar above the test instance list lets you narrow down a test run by: - -- **Test instance attributes**: Status, Assignee, Platform, OS, Browser, Resolution, Device. -- **Test case attributes**: Tags, Priority, Test Case Status. -- **Test Case Custom Fields**: available under **More Filters**. - -Filters work on both Manual and Automation test runs, apply at the **All Test Instances** view as well as at any folder, and persist as you switch between folders. The active filter state is also reflected in the page URL, so you can bookmark or share a filtered view. - -**Supported Custom Field types** -Custom Fields can be filtered only for types: **Single Select Dropdown**, **Multi Select Dropdown**, **User**, **Boolean**, and **Date**. - -## 5. Enhancing Test Runs with Test Evidences -### Adding Remarks and Attachments -- Add Remarks or actual outcomes to enhance test instance execution details. -- You can add remarks and attachments at both the individual test step level and the overall test instance level. - -The Remark field supports **rich text formatting**, including: -- Inline image embedding: paste images directly into the field. -- Text styling such as bold, italic, bullet lists, and other standard rich text options. - -Remarks have a maximum limit of **5000 characters**. - -- Attach supporting files or screenshots to the remarks for better context. - -## 6. Execute Test Runs on TestMu AI Cloud - -Execute your manual test instances directly on TestMu AI Cloud, no local setup or environment configuration required. This allows your team to run manual tests on real browsers and devices hosted on the cloud, making it one of the most efficient ways to validate your test cases. - -**To execute a test run:** - -1. Click the **Play** icon on any test instance to launch execution on TestMu AI Cloud. - -2. During execution, update the status of individual test cases and test steps in real time as you verify each one. - -**Why execute on the Cloud?** -Running test instances on TestMu AI Cloud gives you access to a wide range of real browsers, devices, and OS combinations, without maintaining local infrastructure. It ensures consistent, reliable test execution across environments. - -**Track bugs during execution** -Bugs raised or linked while executing an instance are surfaced under that Test Case instance. To link Jira or Azure DevOps tickets at the instance and step level, see [Track Bugs and Issues in Test Runs](/support/docs/track-issues-in-test-runs/). - -## 7. Test Run Options - -You can manage your test runs using the options available in the **three-dot menu** on each test run. The following actions are available: - -| Action | Description | -|---|---| -| **Edit** | Modify the test run by adding or removing test cases. KaneAI test runs can only be edited if they have not been executed. | -| **Duplicate** | Create a copy of the test run for re-execution or variation testing. | -| **Archive** | Move completed test runs to the archive to keep your workspace clean and organized. | -| **Delete** | Permanently remove a test run that is no longer needed. | diff --git a/static/docs/test-runs-configurations.md b/static/docs/test-runs-configurations.md deleted file mode 100644 index d1ae289c7..000000000 --- a/static/docs/test-runs-configurations.md +++ /dev/null @@ -1,216 +0,0 @@ -# Configurations in KaneAI & Test Manager - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Configurations in Test Manager allow you to define reusable environment setups for your test runs. You can specify platform, browser, OS, and device combinations once and apply them across multiple test runs. - -**Key Benefits:** -- **Centralized Management** - Create, edit, and delete configurations from a single dashboard -- **Reusability** - Apply the same configuration across multiple test runs -- **Multi-Platform Support** - Configure Desktop, Virtual Mobile, and Real Device environments - -## Accessing Configurations - -Navigate to configurations using either method: - -| Access Method | Steps | -|---------------|-------| -| **Sidebar** | Click **Configurations** under Test Manager in the left menu | -| **Test Runs** | Click **Manage Configurations** link within any test run | - -## Configurations Dashboard - -The central dashboard displays all your configurations with the following features: - -| Feature | Description | -|---------|-------------| -| **Search Bar** | Find configurations by name | -| **Filters** | Filter by OS, Browser, or Resolution | -| **Configuration Details** | View name, last updated date, OS, browser, version, and resolution | -| **Usage Count** | See how many test runs use each configuration | -| **Actions Menu** | Access Edit and Delete options via the ⋮ menu | - -## Creating a Configuration - -You can create configurations from the central Configurations page or directly within a test run. - -### Steps to Create - -1. Click **Create a Configuration** button -2. Toggle the **KaneAI** switch based on your test run type: set it to **ON** for KaneAI-generated test runs, or **OFF** for Non-KaneAI test runs. -3. Select your platform (Desktop, Real Device, or Virtual Mobile) -4. Fill in the required configuration fields -5. Click **Create Configuration** - -### KaneAI Toggle - -The KaneAI toggle determines which platforms and options are available: - -| Toggle State | Available Platforms | Use Case | -|--------------|---------------------|----------| -| **ON** | Desktop, Real Device | KaneAI test cases | -| **OFF** | Desktop, Virtual Mobile, Real Device | Non-KaneAI test cases | - -### Platform Options - -| Platform | Details | -|----------|---------| -| **Desktop** | OS, OS Version, Browser, Browser Version, Resolution, URL | -| **Virtual Mobile (non-KaneAI only)** | OS, Manufacturer, Device, OS Version, Browser/App | -| **Real Device** | Device Type, OS, Manufacturer, Device, OS Version, Browser/App | - -### Desktop - -Configure browser-based testing on desktop environments. - -**Environment Variables:** -| Field | Description | -|-------|-------------| -| Operating System | Windows, macOS, etc. | -| OS Version | Specific version of the OS | -| Browser | Chrome, Firefox, Safari, Edge, etc. | -| Browser Version | Specific browser version | -| Resolution | Screen resolution for testing | - -### Real Device - -Configure testing on physical mobile devices. Supports both **Native App** and **Mobile Browser** testing. - -#### Native Application - -**Note** -Native app testing on Real Devices is available only for KaneAI-generated test runs. - -**Environment Variables:** -| Field | Description | -|-------|-------------| -| Device Type | Public Cloud or Private Cloud | -| Operating System | Android or iOS | -| Manufacturer | Device manufacturer (Samsung, Apple, etc.) | -| Device | Specific device model | -| OS Version | Operating system version | -| App | Application to test | - -#### KaneAI-Exclusive Features - -The following options are available only for KaneAI-generated test runs: - -**Device Type: Private Cloud vs Public Cloud** - -For organizations with a private cloud real device plan, you can choose between: -- **Public Cloud** - Shared pool of devices available to all LambdaTest users -- **Private Cloud** - Dedicated device pool exclusive to your organization - -**Multi-Select Support** - -KaneAI allows selecting multiple values for Manufacturer, Device, and OS Version. This creates a larger device pool, ensuring test execution can proceed even if specific devices are unavailable. - -#### Application Options - -| Option | Description | -|--------|-------------| -| **Auto Update** | Automatically updates the configuration when a new app version (matching `appPackage` or `bundleId`) is uploaded | -| **Custom ID** | Links the configuration to an app uploaded via [API](/support/docs/application-setup-via-api/#upload-your-application) with a specific `custom_id` | -| **Pre-Installed App** | Available for Private Cloud only. Maps app capability to `Stock` for testing pre-installed applications | - -#### Mobile Browser - -**Note** -Mobile browser testing on Real Devices is available for non-KaneAI test runs only. - -**Environment Variables:** -| Field | Description | -|-------|-------------| -| Operating System | Android or iOS | -| Browser | Mobile browser to test | -| Manufacturer | Device manufacturer | -| Device | Specific device model | -| OS Version | Operating system version | - -### Virtual Mobile - -**Note** -Virtual Mobile configurations are available for non-KaneAI test runs only. - -Configure testing on emulators and simulators for both Native App and Mobile Browser testing. - -#### Native Application - -**Environment Variables:** -| Field | Description | -|-------|-------------| -| Operating System | Android or iOS | -| Manufacturer | Device manufacturer | -| Device | Specific device model | -| OS Version | Operating system version | -| App | Application to test | - -#### Mobile Browser - -**Environment Variables:** -| Field | Description | -|-------|-------------| -| Operating System | Android or iOS | -| Browser | Mobile browser to test | -| Manufacturer | Device manufacturer | -| Device | Specific device model | -| OS Version | Operating system version | -| URL *(Optional)* | Default URL to open | - -## Managing Configurations - -### Edit a Configuration - -1. Navigate to the **Configurations** page from the sidebar -2. Locate the configuration and click the **Actions menu** (⋮) -3. Select **Edit** -4. Modify the required fields -5. Save your changes - -**Important** -Any updates to the configuration will affect the new and planned test runs. Executed test runs will not be impacted. - -**Note** -Edit functionality is only available on the Configurations page, not within the test run modal. - -### Delete a Configuration - -1. Navigate to the **Configurations** page from the sidebar -2. Locate the configuration and click the **Actions menu** (⋮) -3. Select **Delete** -4. Confirm the deletion - -**Important** -Deleting a configuration is only allowed if it's not associated with any Test Instance. - -## Using Configurations in Test Runs - -Apply configurations to test cases when creating or editing test runs. - -### Steps to Add Configurations - -1. Select the test cases you want to configure -2. Click **Configure Test** -3. In the **Add Configuration** modal: - - Search for existing configurations by name - - Use filters to narrow down options (OS, Device Type, Browser) - - Select configurations using checkboxes - - Optionally, create a new configuration - - Click **Manage Configurations** to open the central dashboard in a new tab - -Adding configurations **appends** to existing ones. It does not replace previously assigned configurations. - -### Bulk Selection - -- Use the **Select All** toggle to select all configurations matching current filters -- This is useful for applying multiple configurations to test instances at once - -## Best Practices - -| Practice | Benefit | -|----------|---------| -| Use descriptive names | Easily identify configurations (e.g., "Chrome-Win11-1920x1080") | -| Manage from central dashboard | Edit and organize all configurations in one place | -| Enable Auto Update for apps | Always test with the latest app build | -| Use multi-select for devices | Increase device pool availability for KaneAI test runs | -| Review usage count before editing | Understand the impact of changes across test runs | diff --git a/static/docs/test-settings-options.md b/static/docs/test-settings-options.md deleted file mode 100644 index 625b542d0..000000000 --- a/static/docs/test-settings-options.md +++ /dev/null @@ -1,321 +0,0 @@ -# Comparison Settings for SmartUI - Pixel to Pixel Comparison - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Our Pixel to Pixel (P2P) Comparison Options - -We offer multiple options for comparing the **Baseline** and the **Test Output** screenshots captured during your automation testing suites. - -Here are common **pixel-to-pixel** comparison options. The first group is **actively used** through `smartUI.options` (or your framework’s equivalent) in typical integrations: - -- `largeImageThreshold`: Pixel granularity for how comparison blocks are formed. -- `errorType`: How differences are highlighted (`movement`, `flat`, etc.). -- `ignore`: Reduces P2P false positives (`antialiasing`, `alpha`, `colors`, `nothing`). -- `transparency`: Overlay transparency for the diff view. - -The sections **[Bounding Boxes](#bounding-boxes---compare-only-specific-area)**, **[Ignore Boxes](#ignore-boxes---ignore-only-specific-area)**, and **[Ignore Areas Colored](#ignore-areas-colored---removes-the-colored-content-from-the-comparison)** describe **region- and color-based** comparison. Examples nest keys under **`smartUI.options`** / **`smart_ui.options`** like the options above; **still validate** with your integration or support, see the warning before **Bounding Boxes**. For region-level control in the product UI, prefer **[Draw on UI / annotations](/support/docs/smartui-draw-on-ui/)**. - -## Examples with comparison settings - -### Image Threshold - -The image threshold settings helps you in analyzing the granularity of the pixel resolution for each block which shall be created by the algorithm for the comparison. - -You can specify the following `capability` in the following format: - -```js title="Make changes in your your test configuration file" -let capabilities = { - -/* ....Your Selenium capabilities go here */ - -"smartUI.project": "" // Your SmartUI project name -"smartUI.options": { -// highlight-next-line -"largeImageThreshold": 100 // The image threshold for comparison the min of 100 and max of 1200 -} -}; -``` - -```json title="Add the following in your /project/lambdatest-config.json" -"smart_ui": { -"project": "" // Add your SmartUI Project Name here -"options": { -// highlight-next-line -"largeImageThreshold": 100 // The image threshold for comparison the min of 100 and max of 1200 -} -} -``` - -### Error Identifier Types - -The pixel to pixel comparison at which we offer for the error highlighting in the test output is determined by the changes in the `pixel` distribution for the **Baseline** screenshot to the **Comparison** screenshot. This feature will help in determining for the approver that the pixels can be easily identified. - -The following are the supported options for the pixel comparison feature: - -- `movement` - specifies the pixel movements -- `flat` - specifies the pixel value changes - -You can specify the following `capability` in the following format: - -```js title="Make changes in your your test configuration file" -let capabilities = { - -/* ....Your Selenium capabilities go here */ - -"smartUI.project": "" // Your SmartUI project name -"smartUI.options": { -// highlight-next-line -"errorType": "movement" // To set the error identifier -} -}; -``` - -```json title="Make changes in your /project/lambdatest-config.json" -"smart_ui": { -"project": "" // Add your SmartUI Project Name here -"options": { -// highlight-next-line -"errorType": "movement" // To set the error identifier -} -} - -``` - -### Ignore - P2P False Positives - -When the screenshot is captured and it is then compressed to a file format such as `.png,.jpg,.jpeg,etc..` then there is high chance of pixelation enhancement will be made to the screenshot to enhance the quality of the image. This could be mis-leading to a visual appearance on the browser for the comparison with the current **Baseline** version. We provide you with options to mitigate this risk of flakiness in your comparison output. - -The following are the supported options for the pixel comparison feature: - -- `antialiasing` - ignores the smoothing of edges in digital images. -- `alpha` - ignores the alpha of the pixel color and makes into solid color without any opacity, click [here](https://sites.uci.edu/iap2015/2015/01/14/the-alpha-attribute-of-a-picture/#:~:text=In%20digital%20images%2C%20each%20pixel,of%200%20means%20totally%20transparent.) to know more. -- `colors` - added corrected lightness to the screenshot by ignoring the default pixel color value. -- `nothing` - ignores none of the false positive options from the comparison. - -You can specify the following `capability` in the following format: - -```js title="Make changes in your your test configuration file" -let capabilities = { - -/* ....Your Selenium capabilities go here */ - -"smartUI.project": "" // Your SmartUI project name -"smartUI.options": { -// highlight-next-line -"ignore": "antialiasing" // To set the ignore identifier -} -}; -``` - -```json title="Make changes in your /project/lambdatest-config.json" -"smart_ui": { -"project": "" // Add your SmartUI Project Name here -"options": { -// highlight-next-line -"ignore": "antialiasing" // To set the ignore identifier -} -} - -``` - -### Transparency - -An advanced configuration to set the comparison view image to be displayed as a `opquae` or `transparent` for easy identification for the approver. - -You can specify the following `capability` in the following format: - -```js title="Make changes in your your test configuration file" -let capabilities = { - -/* ....Your Selenium capabilities go here */ - -"smartUI.project": "" // Your SmartUI project name -"smartUI.options": { -// highlight-next-line -"transparency": 0.3 // minimum = 0 and maximum = 1 supports upto one decimal such as 0.1 to 0.9 -} -}; -``` - -```json title="Make changes in your /project/lambdatest-config.json" -"smart_ui": { -"project": "" // Add your SmartUI Project Name here -"options": { -// highlight-next-line -"transparency": 0.3 // minimum = 0 and maximum = 1 supports upto one decimal such as 0.1 to 0.9 -} -} - -``` - -**`boundingBoxes`, `ignoredBoxes`, `ignoreAreasColoredWith`** -These three comparison modes exist in SmartUI’s **pixel-to-pixel** model. The examples below nest them under **`smartUI.options`** (Selenium) and **`smart_ui.options`** (Cypress), the same shape as **Image Threshold** through **Transparency**, but **behavior can still vary by integration**; **do not copy into production** without validating against your session or with **[support](mailto:support@testmuai.com)**. For box/color-style ignores in the UI, use **[Draw on UI](/support/docs/smartui-draw-on-ui/)** or confirm the supported payload with support. - -### Bounding Boxes - Compare only specific area {#bounding-boxes---compare-only-specific-area} - -The bounding boxes are the areas created on the screenshot which needs to be compared with the baseline ignoring other areas from the screenshot. - -**Reference only:** verify with support before relying on capability wiring. - -This specific case is used to compare only a specific area of the screenshot from the **baseline**. - -You can specify the following `capability` in the following format: - -```js title="Make changes in your your test configuration file" -// Bounding box sample settings -const box1 = { -left: 100, -top: 500, -right: 800, -bottom: 300 -}; - -const box2 = { -left: 800, -top: 50, -right: 20, -bottom: 700 -}; - -let capabilities = { - -/* ....Your Selenium capabilities go here */ - -"smartUI.project": "", // Your SmartUI project name -"smartUI.options": { -// highlight-next-line -"boundingBoxes": [box1, box2] // Compare only these regions (P2P) -} -}; - -``` - -```json title="Make changes in your /project/lambdatest-config.json" -"smart_ui": { -"project": "", -"options": { -"boundingBoxes": [ -{ -"left": 100, -"top": 500, -"right": 800, -"bottom": 300 -}, -{ -"left": 800, -"top": 50, -"right": 20, -"bottom": 700 -} -] -} -} - -``` - -### Ignore Boxes - Ignore only specific area {#ignore-boxes---ignore-only-specific-area} - -**Reference only:** verify with support before relying on capability wiring. - -The ignored boxes are the areas created on the screenshot which needs to be ignored with the baseline comparing the other areas from the screenshot. - -This specific case is used to ignore only a specific area of the screenshot from the **baseline**. - -You can specify the following `capability` in the following format: - -```js title="Make changes in your your test configuration file" -// Bounding box sample settings -const box1 = { -left: 100, -top: 500, -right: 800, -bottom: 300 -}; - -const box2 = { -left: 800, -top: 50, -right: 20, -bottom: 700 -}; - -let capabilities = { - -/* ....Your Selenium capabilities go here */ - -"smartUI.project": "", // Your SmartUI project name -"smartUI.options": { -// highlight-next-line -"ignoredBoxes": [box1, box2] // Ignore these regions during P2P compare -} -}; - -``` - -```json title="Make changes in your /project/lambdatest-config.json" -"smart_ui": { -"project": "", -"options": { -"ignoredBoxes": [ -{ -"left": 100, -"top": 500, -"right": 800, -"bottom": 300 -}, -{ -"left": 800, -"top": 50, -"right": 20, -"bottom": 700 -} -] -} -} - -``` - -### Ignore Areas Colored - Removes the colored content from the comparison {#ignore-areas-colored---removes-the-colored-content-from-the-comparison} - -**Reference only:** verify with support before relying on capability wiring. - -You can exclude the pixels that match the specified color on a **baseline** image from the comparison view. This feature will ignore that specific regions with the color pixels and shows the comparison view. - -You can specify the following `capability` in the following format: - -```js title="Make changes in your your test configuration file" -// Bounding box sample settings -const color = { -r: 242, -g: 201, -b: 76, -a: 1 -}; - -let capabilities = { - -/* ....Your Selenium capabilities go here */ - -"smartUI.project": "", // Your SmartUI project name -"smartUI.options": { -// highlight-next-line -"ignoreAreasColoredWith": color // Ignore pixels matching this RGBA in P2P compare -} -}; - -``` - -```json title="Make changes in your /project/lambdatest-config.json" -"smart_ui": { -"project": "", -"options": { -"ignoreAreasColoredWith": { -"r": 242, -"g": 201, -"b": 76, -"a": 1 -} -} -} - -``` diff --git a/static/docs/test-setup-and-execution.md b/static/docs/test-setup-and-execution.md deleted file mode 100644 index bcccd46ed..000000000 --- a/static/docs/test-setup-and-execution.md +++ /dev/null @@ -1,77 +0,0 @@ -# Test Setup And Execution - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -After uploading the app to the TestMu AI cloud through REST API, you need to configure different capabilities for the uploaded app in your Appium test scripts. After that, you can run app automation tests using Appium on TestMu AI. - -The **TestMu AI Desired Capabilities Generator** will help in auto-generating the capabilities class required to run your app automation scripts on the TestMu AI Appium Grid. The Desired capabilities generator will generate the entire code of the capabilities class for the selected setups and programming languages. - -This documentation guides you how to configure the desired capability for app, device, group tests, debugging, and mark tests passed/failed etc. - -## Configure App - -In order to fetch and install apps on TestMu AI cloud, you will need to use the `app` capability to connect with your uploaded apps. - -| Key | Values | Description | Desired Capability | -| -------- | -----| ------------ | --------------| -| app | `lt://APP10051525539885437397` | Configure the app to test on | `"app":"lt://APP10051525539885437397"` | - -## Select Device And OS - -To perform Appium mobile testing on real Android and iOS devices, you need to define the `deviceName`, `platformName`, and `platformVersion` capabilities in your Appium automation scripts. - -| Key | Values | Description | Desired Capability | -| -------- | -----| ------------ | --------------| -| deviceName | Galaxy Tab S4 | Specify the device to test on | `capabilities.setCapability (deviceName","Galaxy Tab S4)` | -| platformName | Android | Specify the platform name | `capabilities.setCapability("platformName", "Android")` | -| platformVersion | 10 | Specify the platform version | `capabilities.setCapability("platformVersion", "10")` | - -## Group Tests - -For better analysis, you can name your test cases and organize your builds by build name and project. To organize Appium automated tests, you will need to use name, build and projectName capability. - -| Key | Values | Description | Desired Capability | -| -------- | -----| ------------ | ----------------| -| build | 5.1 | Represent the build number for your test | `capabilities.setCapability("build", "5.1")` | -| name | Demo | Represents the name of a test | `capabilities.setCapability("name", "Demo")` | -| projectName | Test1 | Represent the name of a project | `capabilities.setCapability("projectName", "Test1")` | - -## Debugging - -By specifying the capabilities for the debugging tools, you can debug and fix your failed Appium test sessions using Appium logs, device logs, network logs, crash logs, screenshots, video logs, or app profiling data. - ->**Note**: Appium logs are enabled by default. - -| Key | Values | Description | Desired Capability | -| -------- | -----| ------------ | --------------| -| network | true/false | Enable network logs | `capabilities.setCapability("network", "true")` | -| devicelog | true/false | Enable device logs | `capabilities.setCapability("devicelog", "true")` | -| visual | true/false | Enable Visual logs to capture screenshots. | `capabilities.setCapability("visual", "true")` | -| video | true/false | Enable Video recording of the entire screen | `capabilities.setCapability("video", "true")` | - -## Viewing Appium Test Results - -You can view the results of your app automation tests on the TestMu AI App Automation dashboard and using the REST API. Configure **build** and **project** capabilities to segregate your test sessions into distinct builds and projects. - -| Key | Values | Description | Desired Capability | -| -------- | -----| ------------ | --------------| -| build | My_Test | Name of the build to group tests | `capabilities.setCapability("build", "My_Test")` | -| project | Sprint | Name of the project to group tests | `capabilities.setCapability("project", "Sprint")` | - -## Mark Tests As Passed Or Failed - -While performing mobile app automation on TestMu AI Appium Grid, you may encounter a scenario in which a test that you labeled as failed in your local instance turns out to be performed successfully on TestMu AI. It's crucial to mark an automated test as **Passed** or **Failed** based on your testing requirements for validating expected behavior. - -By default, the Status of each test that runs successfully is marked as **Completed**, and if there are any issues, the Status is marked as **Error**. - -You can mark a test status as failed/passed using JavaScript executer on TestMu AI. - -1. To mark test status as **failed**. - -`driver.executeScript('lambda-status=failed')` - -2. To mark test status as **passed**. - -`driver.executeScript('lambda-status=passed')` - ->In case you have any questions or need any additional information, drop them at our **24X7 Chat Support** or mail us directly at support@testmuai.com. diff --git a/static/docs/test-synthflow-agents.md b/static/docs/test-synthflow-agents.md deleted file mode 100644 index d7f9cc2f3..000000000 --- a/static/docs/test-synthflow-agents.md +++ /dev/null @@ -1,61 +0,0 @@ -# How to Test Synthflow Agents With TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Synthflow is a no-code voice platform running on its own telephony infrastructure rather than a third-party carrier, with a visual flow designer and subflows for multi-agent setups. TestMu AI connects to the agent on whichever channel it runs on, drives full multi-turn conversations across personas and edge cases, and scores every run. - -## Before You Begin - -Before you connect a Synthflow agent, make sure you have: - -- Synthflow account with a published agent -- A number provisioned on Synthflow's telephony layer, for phone agents only -- TestMu AI workspace with agent-testing permissions -- Source docs for scenario generation: prompt, PRD, or knowledge base - -## Test a Synthflow Phone Agent - -Phone testing covers agents that handle inbound or outbound callers running over Synthflow's own telephony stack. Paste the agent's prompt or import the flow to generate scenarios, then add the provisioned number and pick from 100+ voices, background noise conditions, and personas. It covers DTMF entry, transfers, voicemail detection, and latency measured against in-house routing rather than a carrier's. - -Full setup: [Phone agent testing](/support/docs/phone-agent/) - -## Test a Synthflow Voice Agent - -Voice testing covers agents reached through the embedded web widget rather than a phone number. Paste the prompt or import the flow to generate scenarios, then create the endpoint profile against the widget's agent. It drops telephony out of the path and isolates the speech pipeline, so STT and TTS accuracy, turn-taking, and interruption handling get tested without telephony noise in the results. - -Full setup: [Voice agent testing](/support/docs/voice-agent/) - -## Test a Synthflow Chat Agent - -Chat testing covers text conversations through the chat widget, SMS, or WhatsApp Business. Upload the supporting docs the agent answers from to generate scenarios, then create the endpoint profile using the chat widget or the messaging channel. It catches reasoning and branching failures. The text channel runs shallower logic than voice on the same agent. - -Full setup: [Chat agent testing](/support/docs/chat-agent/) - -## What You Get With Agent Testing - -Every Synthflow run, on any surface it supports, is scored across: - -- 30+ metrics across 8 categories, with configurable thresholds -- Context-aware scenario generation -- Adversarial testing and automated issue detection -- 100+ voices and personas -- Multilingual conversations -- Quality scoring and real-time call monitoring - -## Synthflow-Specific Considerations - -A few Synthflow behaviours are worth building dedicated scenarios around: - -- In-house telephony means call quality issues belong to Synthflow's stack, not a carrier's, which changes how a failure gets triaged -- Flows are modular, so a subflow edit propagates to every parent flow using it -- Feature availability varies by plan, so some flow capabilities are absent on lower tiers -- Chat and SMS do not run at parity with voice on the same agent - -## Troubleshooting - -Common failure modes to watch for: - -- No transcript returned -- Subflow not triggered -- Plan-gated feature missing -- Messaging session expired diff --git a/static/docs/test-twilio-ivr.md b/static/docs/test-twilio-ivr.md deleted file mode 100644 index 99d5e2fe6..000000000 --- a/static/docs/test-twilio-ivr.md +++ /dev/null @@ -1,50 +0,0 @@ -# How to Test Twilio IVR Flows With TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -A Twilio IVR is a scripted flow built in Studio or TwiML, not a conversational AI agent. It matches on keypad digits and fixed speech results rather than intent, so coverage means traversing every menu branch rather than probing free-form conversation. TestMu AI dials the number, walks each path, and scores every run. - -## Before You Begin - -Before you connect a Twilio IVR flow, make sure you have: - -- Twilio account with a published Studio flow or TwiML application -- The provisioned number running the flow -- TestMu AI workspace with agent-testing permissions -- The flow export or menu map for scenario generation - -## Test a Twilio IVR - -The IVR test covers inbound callers who navigate a menu-driven flow. Upload the flow export or menu map to generate scenarios across every branch, then add the provisioned number and pick from 100+ voices, background noise conditions, and personas. It covers DTMF entry, speech input fallback, timeout and retry behaviour, and transfer to a queue or a representative. - -Full setup: [Phone agent testing](/support/docs/phone-agent/) - -## What You Get With Agent Testing - -Every Twilio run, on any surface it supports, is scored across: - -- 30+ metrics across 8 categories, with configurable thresholds -- Context-aware scenario generation -- Adversarial testing and automated issue detection -- 100+ voices and personas -- Multilingual conversations -- Quality scoring and real-time call monitoring - -## Twilio-Specific Considerations - -A few Twilio behaviours are worth building dedicated scenarios around: - -- Speech input and keypad input follow different paths in the flow, and both need coverage -- Timeout and retry behaviour on input collection is the main failure surface, not comprehension -- Barge-in is configured per input step rather than globally -- Hallucination, grounding, and safety metrics do not apply to a scripted flow. Leave them off rather than scoring against them -- Teams running Twilio's AI voice product connect over a WebSocket to their own application, which is a different setup - -## Troubleshooting - -Common failure modes to watch for: - -- Digit not registered -- Speech fallback never triggers -- Call drops on timeout -- Transfer target unreachable diff --git a/static/docs/test-vapi-agents.md b/static/docs/test-vapi-agents.md deleted file mode 100644 index fdd9116f2..000000000 --- a/static/docs/test-vapi-agents.md +++ /dev/null @@ -1,61 +0,0 @@ -# How to Test Vapi Agents With TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Vapi is a voice AI platform where a single assistant configuration carries the model, voice, transcriber, tools, and call behaviour, then ships across phone, web, and chat. TestMu AI connects to the assistant on whichever surface it runs on, drives full multi-turn conversations across personas and edge cases, and scores every run. - -## Before You Begin - -Before you connect a Vapi assistant, make sure you have: - -- Vapi account with a published assistant -- A provisioned phone number, for phone caller agents only -- TestMu AI workspace with agent-testing permissions -- Source docs for scenario generation: prompt, PRD, or knowledge base - -## Test a Vapi Phone Agent - -Phone testing covers inbound and outbound callers running over the phone network. Paste the assistant's system prompt to generate scenarios, then add the number under Phone Numbers and pick from 100+ voices, background noise conditions, and personas. It covers DTMF entry, transfers, voicemail handling, and carrier latency. - -Full setup: [Phone agent testing](/support/docs/phone-agent/) - -## Test a Vapi Voice Agent - -Voice testing applies to assistants reached over a direct audio stream rather than a phone number. Paste the system prompt to generate scenarios, then create the endpoint profile using REST API or WebSocket. It isolates the speech pipeline, so STT and TTS accuracy, turn-taking, and interruption handling get tested without telephony noise in the results. - -Full setup: [Voice agent testing](/support/docs/voice-agent/) - -## Test a Vapi Chat Agent - -Chat testing covers text conversations through Vapi's Chat API. Upload the supporting docs and Confluence pages the agent answers from to generate scenarios, then create the endpoint profile using REST API or WebSocket. It catches reasoning, tool call, and grounding failures, and can run on every commit. - -Full setup: [Chat agent testing](/support/docs/chat-agent/) - -## What You Get With Agent Testing - -Every Vapi run, on any surface it supports, is scored across: - -- 30+ metrics across 8 categories, with configurable thresholds -- Context-aware scenario generation -- Adversarial testing and automated issue detection -- 100+ voices and personas -- Multilingual conversations -- Quality scoring and real-time call monitoring - -## Vapi-Specific Considerations - -A few Vapi behaviours are worth building dedicated scenarios around: - -- Default tools each need their own scenario: `transferCall`, `endCall`, `dtmf`, `sms`, `apiRequest` -- Squads hand off mid-call, so the transfer boundary needs testing -- WebSocket transport rejects phone parameters, and audio pauses trigger silence timeouts -- Chat context carries through session IDs, so multi-turn state needs its own tests - -## Troubleshooting - -Common failure modes to watch for: - -- No transcript returned -- Call ends on the wrong turn -- Audio format mismatch -- Chat schema mismatch diff --git a/static/docs/test-vertex-agent-builder-agents.md b/static/docs/test-vertex-agent-builder-agents.md deleted file mode 100644 index e6627e7cf..000000000 --- a/static/docs/test-vertex-agent-builder-agents.md +++ /dev/null @@ -1,61 +0,0 @@ -# How to Test Vertex AI Agent Builder Agents With TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Conversational Agents model a conversation as a state machine of flows, pages, and routes rather than a single prompt, and the same agent serves web chat, streaming audio, and telephony. TestMu AI connects to the agent on whichever surface it runs on, drives full multi-turn conversations across personas and edge cases, and scores every run. - -## Before You Begin - -Before you connect a Vertex agent, make sure you have: - -- A Google Cloud project with a published agent -- A Phone Gateway number or a partner telephony integration, for phone agents only -- TestMu AI workspace with agent-testing permissions -- Source docs for scenario generation: flow export, PRD, or knowledge base - -## Test a Vertex Phone Agent - -Phone testing covers callers arriving on a Google-hosted Phone Gateway number, a CCAI telephony partner, or a SIP trunk. Import the flow export to generate scenarios across every route, then add the number under Phone Numbers and pick from 100+ voices, background noise conditions, and personas. It covers DTMF entry, transfers, and carrier latency, scored against the phone-call speech model the gateway enforces. - -Full setup: [Phone agent testing](/support/docs/phone-agent/) - -## Test a Vertex Voice Agent - -Voice testing covers agents reached through streaming audio rather than a phone number. Import the flow export to generate scenarios, then create the endpoint profile using the streaming detect intent API. It isolates the speech pipeline, so recognition accuracy, turn-taking, and interruption handling get tested without telephony noise in the results. - -Full setup: [Voice agent testing](/support/docs/voice-agent/) - -## Test a Vertex Chat Agent - -Chat testing covers text conversations through the Dialogflow Messenger widget or a custom client. Upload the flow export and data store contents to generate scenarios, then create the endpoint profile using the detect intent API. It runs logic regression on every change. - -Full setup: [Chat agent testing](/support/docs/chat-agent/) - -## What You Get With Agent Testing - -Every Vertex run, on any surface it supports, is scored across: - -- 30+ metrics across 8 categories, with configurable thresholds -- Context-aware scenario generation -- Adversarial testing and automated issue detection -- 100+ voices and personas -- Multilingual conversations -- Quality scoring and real-time call monitoring - -## Vertex-Specific Considerations - -A few Vertex behaviours are worth building dedicated scenarios around: - -- Phone Gateway forces the phone-call speech model regardless of what the agent has configured, so a voice run and a phone run can score differently on the same agent by design -- Phone Gateway supports US numbers on globally created agents only. Regional agents need a partner integration -- Routes are the branch unit, so coverage means every route rather than every intent -- Webhook fulfilment sits outside the agent, which makes a webhook timeout look like an agent failure - -## Troubleshooting - -Common failure modes to watch for: - -- Route not matched -- Webhook timeout -- Phone gateway quota exceeded -- Regional agent rejected by the gateway diff --git a/static/docs/test-voiceflow-agents.md b/static/docs/test-voiceflow-agents.md deleted file mode 100644 index 729611a2a..000000000 --- a/static/docs/test-voiceflow-agents.md +++ /dev/null @@ -1,56 +0,0 @@ -# How to Test Voiceflow Agents With TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Voiceflow is chat-first. Agents are built in a visual designer and deployed to a web widget or any channel through the Dialog API, with phone available as an integration rather than a native surface. TestMu AI connects to the agent on whichever channel it runs on, drives full multi-turn conversations across personas and edge cases, and scores every run. - -## Before You Begin - -Before you connect a Voiceflow agent, make sure you have: - -- Voiceflow account with a published agent -- A Twilio or Vonage number pointed at the agent, for phone agents only -- TestMu AI workspace with agent-testing permissions -- Source docs for scenario generation: prompt, PRD, or knowledge base - -## Test a Voiceflow Chat Agent - -Chat testing covers text conversations through the web chat widget or a custom channel, which is Voiceflow's native surface. Upload the supporting docs and knowledge base the agent answers from to generate scenarios, then create the endpoint profile using the Dialog API. It catches reasoning, tool call, and grounding failures. - -Full setup: [Chat agent testing](/support/docs/chat-agent/) - -## Test a Voiceflow Phone Agent - -Phone testing covers callers who arrive through a Twilio or Vonage number pointed at the agent. Paste the agent's prompt or import the flow to generate scenarios, then add the number under Phone Numbers and pick from 100+ voices, background noise conditions, and personas. It covers DTMF entry, transfers, and carrier latency. The speech layer belongs to the telephony provider rather than Voiceflow. - -Full setup: [Phone agent testing](/support/docs/phone-agent/) - -## What You Get With Agent Testing - -Every Voiceflow run, on any surface it supports, is scored across: - -- 30+ metrics across 8 categories, with configurable thresholds -- Context-aware scenario generation -- Adversarial testing and automated issue detection -- 100+ voices and personas -- Multilingual conversations -- Quality scoring and real-time call monitoring - -## Voiceflow-Specific Considerations - -A few Voiceflow behaviours are worth building dedicated scenarios around: - -- Voiceflow brings the logic while the telephony provider brings the number, STT, and TTS, so phone failures split across two vendors -- Latency and audio quality on phone runs reflect the provider's stack as much as the agent -- There is no native live-agent handoff, so escalation scenarios need a custom path to test against -- The Dialog API is the connection point for anything outside the widget, and it requires code -- There is no direct audio surface, so there is no voice section on this page - -## Troubleshooting - -Common failure modes to watch for: - -- Dialog API auth failure -- Telephony provider misrouted -- No live-agent path to escalate to -- Session state lost diff --git a/static/docs/test-watsonx-assistant-bots.md b/static/docs/test-watsonx-assistant-bots.md deleted file mode 100644 index 649e7d704..000000000 --- a/static/docs/test-watsonx-assistant-bots.md +++ /dev/null @@ -1,56 +0,0 @@ -# How to Test watsonx Assistant Bots With TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -watsonx Assistant is built from actions and steps, reached over a web chat widget or the message API, with phone available through a telephony integration. TestMu AI connects to the assistant on whichever channel it runs on, drives full multi-turn conversations across personas and edge cases, and scores every run. - -## Before You Begin - -Before you connect a watsonx Assistant, make sure you have: - -- A published assistant with an environment ID -- A telephony or SIP integration configured, for phone testing only -- TestMu AI workspace with agent-testing permissions -- Source docs for scenario generation: action export, PRD, or knowledge base - -## Test a watsonx Assistant Chat Bot - -Chat testing covers text conversations through the web chat widget or a custom client. Upload the action export and supporting docs to generate scenarios, then create the endpoint profile using the message API against the assistant and environment ID. It catches step conditions, search grounding, and tool failures. - -Full setup: [Chat agent testing](/support/docs/chat-agent/) - -## Test a watsonx Assistant Phone Bot - -Phone testing covers callers who arrive through the telephony or SIP integration. Upload the action export to generate scenarios, then add the number under Phone Numbers and pick from 100+ voices, background noise conditions, and personas. It covers what telephony introduces against the same actions the chat channel uses: DTMF entry, transfers, and carrier latency. - -Full setup: [Phone agent testing](/support/docs/phone-agent/) - -## What You Get With Agent Testing - -Every watsonx run, on any surface it supports, is scored across: - -- 30+ metrics across 8 categories, with configurable thresholds -- Context-aware scenario generation -- Adversarial testing and automated issue detection -- 100+ voices and personas -- Multilingual conversations -- Quality scoring and real-time call monitoring - -## watsonx-Specific Considerations - -A few watsonx behaviours are worth building dedicated scenarios around: - -- Draft and live environments are separate, so confirm which one the connection targets -- Actions and steps are the branch unit, and step conditions break on unexpected phrasing the same way pathway conditions do -- Search answers come from an external source, so a grounding failure may be a retrieval problem -- Sessions expire, which means a long scenario can fail on timeout rather than logic -- There is no direct audio surface outside telephony, so there is no voice section on this page - -## Troubleshooting - -Common failure modes to watch for: - -- Step condition not met -- Session expired mid-scenario -- Search returns nothing -- Connected to the draft environment diff --git a/static/docs/testcomplete-integration.md b/static/docs/testcomplete-integration.md deleted file mode 100644 index 61bdbe814..000000000 --- a/static/docs/testcomplete-integration.md +++ /dev/null @@ -1,64 +0,0 @@ -# TestComplete - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestComplete, from SmartBear, streamlines automated testing across desktop, web, and mobile applications. It goes beyond recording clicks build readable tests with keywords, identify UI elements for reliability, and leverage data-driven testing for comprehensive coverage. - -TestMu AI integration with TestComplete lets you perform web and app automation testing across 3000+ real browsers, devices, and OS combinations. - -> The TestComplete application will only work on the **Windows** operating system. - -## Prerequisites - -- TestMu AI account. You can sign up for free. -- TestMu AI [Username and Access Key](/support/docs/hyperexecute-how-to-get-my-username-and-access-key/). -- Download TestComplete from its official website and complete the signup process. -- Generate your App Id by uploading it to the TestMu AI Server. - -## Steps to Automate your Application - -**Step 1:** Upload your **_iOS_** application (.ipa file) or **_android_** application (.apk file) to the TestMu AI servers using our **REST API**. You need to provide your **Username** and **AccessKey** in the format `Username:AccessKey` in the **cURL** command for authentication. - -Make sure to add the path of the **appFile** in the cURL request. Here is an example cURL request to upload your app using our REST API: - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" --location --request POST 'https://manual-api.lambdatest.com/app/upload/realDevice' --form 'name="Android_App"' --form 'appFile=@"/Users/macuser/Downloads/proverbial_android.apk"' -`} - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/upload/realDevice" -F "appFile=@"/Users/macuser/Downloads/proverbial_android.apk""`} - -**Step 2 :** Click on the Record Test button. - -**Step 3 :** Enter your project name, select project location and select your desired test type. In this demo we are selecting the Javascript. Click on the Next button. - -**Step 4 :** Now select Mobile as your application type and click on Record button. - -**Step 5 :** Now your recording will start and a modal window will pop-up to enter the required capabilities in the parameters section as shown below. Click on OK button. - -Update your **APP_ID**, **YOUR_LAMBDATEST_USERNAME**, and **YOUR_LAMBDATEST_ACCESS_KEY** while passing the capabilities. - -```bash title="Sample code for capabilities" -{ -"platformName": "android", -"deviceName": "Galaxy s21 5G", -"lt:options": { -"w3c": true, -"platformName": "android", -"deviceName": "Galaxy s21 5G", -"platformVersion": "12", -"isRealMobile": true, -"app":"", -"user":"", -"accessKey":"" -} -} -``` - -**Step 6 :** Now whatever steps you will perform in the application will get recorded. Click on the stop button when you are completed with your testing. - -**Step 7 :** This will start processing the complete recording and within no time-stamp you will be able to check your recordings. - -Click on the run button to check your script. - -Your test results would be displayed on on the [**TestMu AI App Automation Dashboard**](https://www.testmuai.com/login/?redirectTo=https://appautomation.lambdatest.com/build). - -> In case you have any questions, feel free to share them with us through our **24*7 Chat Portal** or by mailing to us on [**support@testmuai.com**](mailto:support@testmuai.com). diff --git a/static/docs/testim-integration.md b/static/docs/testim-integration.md deleted file mode 100644 index cf569d64d..000000000 --- a/static/docs/testim-integration.md +++ /dev/null @@ -1,66 +0,0 @@ -# Testim Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Testim provides an automated testing platform to create stable tests using AI Native capabilities, along with tools that facilitate scaling quality across your software development process. - -Integrating TestMu AI with Testim lets you run Testim automated tests on TestMu AI cloud-based grid across 3000+ real browsers and OS. - -## Prerequisites -- Create an account on Testim. -- Create a TestMu AI account. You can [sign up for free](https://www.testmuai.com/register/?redirectTo=https://accounts.lambdatest.com/dashboard). -- Get your TestMu AI Username and Access Key. - -## Setting Up the Grid -Shown below are the steps to set up your Grid on Testim. - -1. Click on your profile icon > **Grids** button. - -2. Click on **Add New Grid**. Then, select **TestMu AI** as your Grid Type and click on **Next** button. - -3. To configure the Grid: - - Enter the Grid name. - - Update your Host and Port number. - - Enter your TestMu AI Username and Access Key and click on **Add** button. - -## Recording the Tests -Shown below are the steps to record your tests on Testim. - -1. Click on **New Test** > **Start Recording**. - -2. Enter your app URL and click on **Create Test**. - -3. It will start recording the tests. Once your testing is completed, stop the recording and save your tests. - -## Configuring the Capabilities -Include the capabilities you wish to use in a `.json` file and ensure this file is located in the same directory as your Testim runner. - -```json -{ -"resolution": "1280x800", -"browserName": "chrome", -"version": "latest" -} -``` - -## Running the Testim Tests on TestMu AI -To execute the tests, check: - -* If the Testim CLI is not already installed on your system, use the entire command provided by the Testim CLI or dashboard. - -**For example:** -```json -npm i -g @testim/testim-cli && testim --token "2jWxWxXcmJRON0PA73YZWt92fFGU76cd" --project "GTJL28fMHad" --grid "LAMBDA-TEST" --test-id "2u8QnIm454" --selenium-caps-file "local.json" -``` - -* If the Testim CLI is already installed, you can run a simpler command that includes your Testim token, project ID, grid, test ID, and the name of your Selenium capabilities file. - -**For example:** - -```json -testim --token "2jWxWxXcmJRON0PA73YZWt92fFGU76cd" --project "GTJL28fMHad" --grid "LAMBDA-TEST" --test-id "2u8QnIm454" --selenium-caps-file "local.json" -``` - -To view your test results, navigate to the TestMu AI Web Automation Dashboard. - -> **Note**: To explore additional configuration options, you can execute `testim --help` in PowerShell or bash, which will display further customizations available for writing and executing your test cases. diff --git a/static/docs/testing-flutter-apps.md b/static/docs/testing-flutter-apps.md deleted file mode 100644 index c6bd0453a..000000000 --- a/static/docs/testing-flutter-apps.md +++ /dev/null @@ -1,204 +0,0 @@ -# Testing Flutter Apps With Appium - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Google's Flutter is an open-source tool for developing native mobile apps. It enables developers to create cross-platform apps for both Android and iOS using a single codebase and programming language. - -You can now test Flutter apps on the TestMu AI Appium testing platform across 3000+ real Android and iOS devices. TestMu AI supports Appium's Flutter driver that lets you test Flutter apps using the Appium framework. To test Flutter apps, you will need to upload apps on TestMu AI cloud servers and then run your automated tests. - -## Testing Apps Using Appium Flutter Driver - -Before automating Flutter apps using Appium, make sure you have the following things configured. - -1. Your Flutter App must be compiled in `debug` or `profile` mode, as Appium Flutter Driver does not support Flutter App running in release mode. - -2. Add the below dependencies in Flutter app's `pubspec.yaml` - -``` -dev_dependencies: -test: Demo -flutter_test: -sdk: flutter -flutter_driver: -sdk: flutter -``` - -3. In `main.dart` file, ensure the app has `enableFlutterDriverExtension()` enabled before `runApp`. - -``` -void main() { -enableFlutterDriverExtension(); -init(); -runApp(MyApp()); -} -``` - - 4. Set the `automationName` capability to `flutter` in Appium desired capabilities. - -``` -desiredCapabilities.setCapability("automationName", "flutter"); -``` - -Shown below is the test script for automating Flutter apps on TestMu AI platform. - -```py -import os - -from appium.webdriver import Remote -from appium_flutter_finder.flutter_finder import FlutterElement, FlutterFinder - -# Example - -driver = Remote('http://:@mobile-hub.lambdatest.com/wd/hub/', dict( -platformName='Android', -automationName='flutter', -platformVersion='11', -deviceName='Google Pixel 4', -app='', -isRealMobile=True -)) - -finder = FlutterFinder() - -text_finder = finder.by_text('You have pushed the button this many times:') -text_element = FlutterElement(driver, text_finder) -print(text_element.text) - -key_finder = finder.by_value_key("next_route_key") -goto_next_route_element = FlutterElement(driver, key_finder) -print(goto_next_route_element.text) -goto_next_route_element.click() - -back_finder = finder.page_back() -back_element = FlutterElement(driver, back_finder) -back_element.click() - -tooltip_finder = finder.by_tooltip("Increment") -driver.execute_script('flutter:waitFor', tooltip_finder, 100) - -floating_button_element = FlutterElement(driver, tooltip_finder) -floating_button_element.click() - -counter_finder = finder.by_value_key("counter") -counter_element = FlutterElement(driver, counter_finder) -print(counter_element.text) -``` - -## Testing Apps Using Appium Flutter Integration Driver - -The [Appium Flutter Integration Driver](https://www.testmuai.com/support/docs/appium-flutter-integration/) lets you write tests in languages like Java, Python, JavaScript, and others that Appium supports. It works by embedding a small server inside your Flutter app which talks back to Appium and drives UI interactions. - -Check out this [Flutter Appium Java GitHub](https://github.com/LambdaTest/flutter-appium-java/) repository for a hands-on reference. - -To run Appium tests using the Flutter Integration Driver on TestMu AI, set the *automationName* capability to *FlutterIntegration*. - -```java -desired_caps = { -"deviceName": "Galaxy S20", -"platformName": "Android", -"platformVersion": "15", -"isRealMobile": True, # Set False for virtual devices -"app": "YOUR_APP_URL", -"build": "Sample Build", -"name": "Sample Test", -"automationName": "FlutterIntegration" # Use Flutter Integration Driver -} -``` - -The example below shows a basic login flow for an Android Flutter app using the Appium Flutter Integration Driver. It covers driver setup, Flutter-specific capabilities, and widget interaction using *ValueKey*. - -```java -import org.openqa.selenium.remote.DesiredCapabilities; -import org.testng.annotations.Test; -import org.openqa.selenium.WebElement; -import org.testng.annotations.Parameters; -import java.net.URL; -import java.util.HashMap; - -import io.appium.java_client.AppiumBy; -import io.appium.java_client.flutter.FlutterIntegrationTestDriver; -import io.appium.java_client.flutter.android.FlutterAndroidDriver; - -public class AndroidApp_Flutter_Integration { - -FlutterIntegrationTestDriver driver; - -@Test -@Parameters(value = { "device", "version", "platform" }) -public void AndroidApp1(String device, String version, String platform) { -try { -DesiredCapabilities capabilities = new DesiredCapabilities(); -HashMap ltOptions = new HashMap<>(); - -ltOptions.put("automationName", "FlutterIntegration"); -ltOptions.put("nativeFlutterLaunch", true); -ltOptions.put("platformName", platform); -ltOptions.put("deviceName", device); -ltOptions.put("platformVersion", version); -ltOptions.put("isRealMobile", true); - -capabilities.setCapability("LT:Options", ltOptions); - -driver = new FlutterAndroidDriver( -new URL("https://username:accessKey@mobile-hub.lambdatest.com/wd/hub"), -capabilities -); - -WebElement username = driver.findElement(AppiumBy.flutterKey("username")); -username.sendKeys("user@example.com"); - -WebElement password = driver.findElement(AppiumBy.flutterKey("password")); -password.sendKeys("password123"); - -WebElement loginBtn = driver.findElement(AppiumBy.flutterKey("login_btn")); -loginBtn.click(); - -driver.quit(); -} catch (Exception e) { -driver.quit(); -} -} -} -``` - -Below is a screenshot showing Appium test automation running with the Flutter Integration Driver on the TestMu AI platform. - ->In case you have any questions or need any additional information, you could reach out at our **24X7 Chat Support** or mail us directly at support@testmuai.com. - -## Using the Flutter Testing Agent Skill with TestMu AI - -The [flutter-testing-skill](https://github.com/LambdaTest/agent-skills/tree/main/flutter-testing-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The flutter-testing-skill package includes: - -``` -flutter-testing-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing Flutter Testing Agent Skill - -Install a Flutter Testing Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/flutter-testing-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/flutter-testing-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only flutter-testing-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). diff --git a/static/docs/testing-locally-hosted-apps.md b/static/docs/testing-locally-hosted-apps.md deleted file mode 100644 index bd17d95b9..000000000 --- a/static/docs/testing-locally-hosted-apps.md +++ /dev/null @@ -1,45 +0,0 @@ -# Testing Locally Hosted Apps For App Automation - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Using the TestMu AI tunnel, you can test locally and privately hosted apps across various real Android and iOS devices on the TestMu AI Appium test automation platform. TestMu AI tunnel uses protocols like **Web Socket, HTTPS/TCP** and more to let you build a secure and unique tunnel connection between your local system and TestMu AI cloud servers. - -In this documentation, learn how to configure TestMu AI tunnel to test locally or privately hosted apps while performing mobile app automation. - -To test apps locally, you will need to configure: - -1. Connection with TestMu AI tunnel. - -2. Test scripts to run via TestMu AI tunnel. - -## Setting Up Connection With TestMu AI Tunnel - -Shown below are the steps to configure the connection with TestMu AI tunnel. - -1. Download the binary file based on your operating system. - -- Windows **[64 Bit](https://downloads.lambdatest.com/tunnel/v3/windows/64bit/LT_Windows.zip) | [32 Bit](https://downloads.lambdatest.com/tunnel/v3/windows/32bit/LT_Windows.zip)** -- macOS **[64 Bit](https://downloads.lambdatest.com/tunnel/v3/mac/64bit/LT_Mac.zip) | [32 Bit](https://downloads.lambdatest.com/tunnel/v3/mac/32bit/LT_Mac.zip)** -- Linux **[64 Bit](https://downloads.lambdatest.com/tunnel/v3/linux/64bit/LT_Linux.zip) | [32 Bit](https://downloads.lambdatest.com/tunnel/v3/linux/32bit/LT_Linux.zip)** - -2. Extract the downloaded binary file. - -3. Navigate to the Command Prompt and point to the directory/folder where you extracted the binary file. - -4. Run the below command in the terminal. - -```js -./LT --user {user's login email} --key {user's access key} --tunnelName {user's tunnel name} -``` - -## Configuring Test Scripts - -After configuring the connection with TestMu AI tunnel, you will need to set the capability `tunnel` to `True`. - -| Key | Values | Description | Desired Capability | -| ------ | ---------- | -------------------- | ------------------ | -| tunnel | true/false | Configure the tunnel | `"tunnel" : True,` | - -You can also add the `tunnel` capability using TestMu AI Capability Generator. - -> In case you have any questions or need any additional information, drop them at our **24X7 Chat Support** or mail us directly at support@testmuai.com. diff --git a/static/docs/testing-locally-hosted-pages.md b/static/docs/testing-locally-hosted-pages.md deleted file mode 100644 index 056b441d9..000000000 --- a/static/docs/testing-locally-hosted-pages.md +++ /dev/null @@ -1,47 +0,0 @@ -# Test Locally Hosted Web Pages - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -**TestMu AI Tunnel** is a secure and encrypted tunneling feature that allows devs and QAs to test their locally hosted web applications or websites on the cloud-based real machines. It establishes a secure connection between the user's local machine and the real machine in the cloud. There are multiple connectivity options like **TCP with TLS 1.2 over port 443**, **Websocket (port 443)**,. Tunnel Binary performs network scanning and selects the best mode if not explicitly specified. - -> All Tunnel credentials (access keys, API tokens, etc.) are private and confidential. A system compromise may result in their unauthorized disclosure; therefore, you must secure the machine running the TestMu AI tunnel. Do not share or expose credentials in public code, repositories, or environments. In the event of a compromise, you are solely responsible for immediately revoking and regenerating the credentials. TestMu AI assumes no liability for any losses arising from failure to comply. - -## Why to Use TestMu AI Tunnel? - -TestMu AI Tunnel offers numerous benefits for web developers, testers, and QA professionals. These include secure and encrypted connection, cross-browser compatibility testing, localhost testing, etc. - -- **Secured and Encrypted :** It provides a secure and encrypted connection between your local machine and the virtual machines in the cloud, thereby ensuring the privacy of your test data and online communications. -- **Cost Effective :** With TestMu AI Tunnel, you can test your web applications or websites, local folder, and files across a wide range of browsers and operating systems without setting up complex and expensive local testing environments. -- **Test on Real Environment :** It lets you test your locally hosted web applications or websites on cloud-based real OS machines. You can even run accessibility tests on desktop browsers while testing locally hosted web applications and pages. - -> Please note that the TestMu AI tunnel client does not include any AI or ML capabilities, and none of the data it proxies is used for AI or ML training. - -## Download the Tunnel Binary -You can download the Tunnel Binary as per your respective machine architecture from the links given below : - -| Platform | Download Link | -|----------|---------------| -| Windows/AMD64 | https://downloads.lambdatest.com/tunnel/v3/windows/amd64/LT.exe | -| Windows/386 | https://downloads.lambdatest.com/tunnel/v3/windows/386/LT.exe | -| Linux/AMD64 | https://downloads.lambdatest.com/tunnel/v3/linux/amd64/LT | -| Linux/ARM64 | https://downloads.lambdatest.com/tunnel/v3/linux/arm64/LT | -| Linux/386 | https://downloads.lambdatest.com/tunnel/v3/linux/386/LT | -| Darwin/AMD64 | https://downloads.lambdatest.com/tunnel/v3/mac/amd64/LT | -| Darwin/ARM64 | https://downloads.lambdatest.com/tunnel/v3/mac/arm64/LT | -| Darwin/386 | https://downloads.lambdatest.com/tunnel/v3/mac/386/LT | -| FreeBSD/AMD64 | https://downloads.lambdatest.com/tunnel/v3/freebsd/amd64/LT | -| FreeBSD/ARM64 | https://downloads.lambdatest.com/tunnel/v3/freebsd/arm64/LT| -| FreeBSD/A386 | https://downloads.lambdatest.com/tunnel/v3/freebsd/386/LT | -| Solaris/AMD64 | https://downloads.lambdatest.com/tunnel/v3/solaris/amd64/LT | - -## Minimum System Requirements for TestMu AI Tunnel - -To ensure optimal performance and reliability when running TestMu AI Tunnel, your system should meet or exceed the following requirements: - -- **Operating System:** Windows 7 or later, macOS 10.12 or later, Linux (modern distributions), FreeBSD, Solaris -- **CPU Architecture:** x86_64 (AMD64), ARM64 -- **CPU Cores:** 1 physical cores or higher -- **Memory (RAM):** 2 GB minimum -- **Disk Space:** 200 MB available -- **Network:** Stable broadband connection (100 Mbps or higher recommended); must allow outbound connections to TestMu AI servers over port 443 (TCP/WebSocket) -- **Permissions:** User must have permission to execute the TestMu AI Tunnel binary diff --git a/static/docs/testing-your-first-ai-agent.md b/static/docs/testing-your-first-ai-agent.md deleted file mode 100644 index 883a4e9ec..000000000 --- a/static/docs/testing-your-first-ai-agent.md +++ /dev/null @@ -1,40 +0,0 @@ -# How to Test Your First AI Agent With TestMu AI Agent Testing Platform - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -You can test an AI agent on TestMu AI entirely from the dashboard, without writing code. You create an agent, upload documents that describe it, generate test scenarios, run an evaluation against your agent's API, and read the scored results. - -The whole flow takes a few minutes and works for chat and voice agents. - -## Prerequisites - -Before you start, make sure you have the following in place. - -- A TestMu AI account with access to the Agent Testing platform. -- Your agent's API endpoint URL. For a chat agent, this is the URL that receives messages. -- Documents that describe your agent's requirements, such as a PDF or DOCX. The platform uses them to generate relevant scenarios. - -## Run Your First Evaluation - -Follow these seven steps in the dashboard. Each step builds on the previous one. - -**Step 1: Create the agent.** In the left sidebar, click **Create Agent**, enter a name and description, then click **Create Agent** to confirm. The agent is created and ready for you to add its requirements. - -**Step 2: Upload requirement documents.** Upload the documents that describe your agent so the platform can analyze its requirements and generate relevant scenarios. - -**Step 3: Select test categories.** After the initial analysis, choose which categories of tests to generate. The platform offers a range of categories, such as **Personality & Tone**, which checks that your agent responds professionally. Select multiple categories to cover a wider range of behavior. - -**Step 4: Generate the scenarios.** Click **Generate Test Scenarios**. A team of specialized AI agents generates multiple scenarios in parallel in the background. - -**Step 5: Review the generated scenarios.** The scenarios appear in a list you can filter by the categories you selected. Each scenario carries a complexity and a risk level. For example, a "tone adaptation when delivering bad news" scenario describes how your agent should respond. - -**Step 6: Run the evaluation.** When you are satisfied with the scenarios, enter your agent's API URL and click **Run Evaluation**. For the API URL format and authentication, see how to [connect a chat agent over its API](/support/docs/chat-agent-api-integration/). - -**Step 7: Read the results.** Open the **Evaluation Results** tab to see a breakdown across the quality metrics, such as whether the conversation was relevant and whether the agent stayed on topic. - -If you are testing a voice bot, the platform generates audio for each test case automatically, then evaluates the transcribed audio responses from your agent to confirm they are relevant. - -## Related TestMu AI Guides - -- See how to [connect a chat agent over its API](/support/docs/chat-agent-api-integration/) for the API URL and authentication used in step 6. -- See how to [test agents from the terminal](/support/docs/testmu-a2a-cli/) with the A2A CLI. diff --git a/static/docs/testingwhiz-integration.md b/static/docs/testingwhiz-integration.md deleted file mode 100644 index 3c0349a7d..000000000 --- a/static/docs/testingwhiz-integration.md +++ /dev/null @@ -1,42 +0,0 @@ -# TestingWhiz Integration With TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -[TestingWhiz](https://www.testing-whiz.com/) is a codeless automation testing tool that can be used for Software testing, Web testing, Mobile testing, Database testing, Cloud testing, Web Services testing, and API testing. - -TestMu AI now integrates with TestingWhiz to help perform cross-browser testing of your website or web app, over 3000+ real desktop devices, mobile viewports, and operating system combinations. This document will help you learn how to integrate your TestingWhiz account with your TestMu AI account. - -## How To Integrate TestMu AI With TestingWhiz? - -Below is the step by step guide to help you integrate TestMu AI with your TestingWhiz app. - -1. Go to **Configurations** from the Setting menu in your TestingWhiz app. A window will open with all the configuration settings in it, as shown below. -2. Go to Cloud Execution menu from the left bar, and you will find **TestMu AI** there. -3. Now you need to provide access of your TestMu AI account to your TestingWhiz app. To do so, click on **Add** button. -4. In the open dialog box, you need to enter your Profile name, username, and access key. You can use any name for your profile on TestingWhiz. - - >To get your Username and Access Key, go to your TestMu AI automation dashboard and click on the "key" icon left to the help button, on the top right corner. - -5. Click on Validate and then on OK once the credentials are validated successfully. -6. Now apply your changes by clicking on Apply and then press OK. - -That's it. Your TestingWhiz account has been integrated successfully with your TestMu AI account. - -## Running First Test From Your TestingWhiz App On Your TestMu AI Account - -Consider a test scenario which includes: - -```javascript -- Opening [LambdaTest ToDo App](https://lambdatest.github.io/sample-todo-app/) -- Check first item in the list -- Check second item in the list -- Add a new item in the list -``` - -In TestingWhiz, record your test scenario. Once done, click on the Execute button. - -As soon as you do, an automation test will start instantly, and you can view its build running directly on your TestMu AI automation dashboard. Once the test is complete, you can view the result of the test on TestMu AI, with all its details, as shown in the image below: - -You can view logs, [mark bugs](/docs/mark-as-bug-in-automation-testing/), and do much more for each executed test from your TestMu AI platform. - -> That's it folks! Now you can run your codefree automation testing super easily with this TestingWhiz and TestMu AI integration. If you have any doubt or questions, feel free to contact our experts at **24/7 chat support** or mail us at [support@testmuai.com](mailto:support@testmuai.com). Happy testing! 🙂 diff --git a/static/docs/testmo-integration.md b/static/docs/testmo-integration.md deleted file mode 100644 index 3e3b3af80..000000000 --- a/static/docs/testmo-integration.md +++ /dev/null @@ -1,54 +0,0 @@ -# TestMo Integration With TestMu AI Grid - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMo is a unified test management software that helps teams manage and track all their testing activity and efforts in one place. It combines manual testing, exploratory & ad-hoc testing and test automation in one platform, so teams can work smarter and more productively. - -TestMo also offers integration with TestMu AI to help you perform automated cross browser testing through an on-cloud Selenium Grid offering 3000+ browsers. **TestMu AI integration to TestMo**  will help you run your automation scripts from TestMo instance to our cloud-based Grid. - -## Steps to Integrate your Project - -### Pre-requisite - -- Create your account in [LambdaTest](https://www.testmuai.com/register/?redirectTo=https://accounts.lambdatest.com/dashboard) and [TestMo](https://manage.testmo.com/trial) - -### Step 1: Build the Test Suite -Firstly, you need to run the build command as per your desired framework. You can choose any test automation framework and programming language you are familiar with. Selenium is one of the most common one used with JavaScript/Node.js, Java, C#, Python, Ruby or PHP, and other languages - -You can take the reference from [here](/support/docs/getting-started-with-testmu-automation/#supported-languages-and-frameworks) on all the languages and frameworks supported by TestMu AI online Selenium Grid. - -### Step 2: Generate your Testing Result XML Files -To submit and report your test results to Testmo, execute your automated tests and generate a JUnit-style XML report file. This file format has become a standard format to exchange test results between tools and is supported by practically any testing tool. Testmo also directly supports this format so it's very easy to report your test automation results. - -For this case, we are using TestNG Framework. Run the below command to generate the file - -``` -java org.testng.TestNG testng1.xml -``` - -The `testng1.xml` file here specifies one of your TestNG suites. After executing your tests, TestNG will automatically generate JUnit-style XML report files in the test-output directory (if this is not the case, please make sure that the org.testng.reporters.JUnitXMLReporter listener is active). - -### Step 3: Configure TestMo in your Project - -Our test automation suite has now been successfully executed, and an XML report file containing the test results has been produced. Simply use our cross-platform testmo CLI tool to submit the test results to Testmo. The CLI tool is distributed as an NPM package and is simple to install on any system. - -- Install the official `@testmo/testmo-cli` **NPM package** - -``` -npm install -g @testmo/testmo-cli - -testmo -h -``` -- Generate the **API Token** in TestMo and run the below command: - -``` -export TESTMO_TOKEN= -``` - -``` -testmo automation:run:submit --instance .testmo.net ---project-id 1 --name "lambdatest" --source "frontend" --results -.xml -``` - -- Output in the Dashboard diff --git a/static/docs/testmu-a2a-cli.md b/static/docs/testmu-a2a-cli.md deleted file mode 100644 index 15ec1793d..000000000 --- a/static/docs/testmu-a2a-cli.md +++ /dev/null @@ -1,1024 +0,0 @@ -# How to Test Chat and Phone Agents With TestMu AI Agent Testing CLI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The TestMu A2A CLI lets you test chat agents and phone agents directly from your terminal. You can run quick one-off tests, build scenario-driven test suites, perform adversarial security assessments, and integrate everything into your CI/CD pipeline. - -## Install the CLI - -```bash -pip install testmu-a2a-cli -``` - -Commands on this page are verified against the current `testmu-a2a-cli`. Run `testmu-a2a --version` to check your installed build. - -## Quick Reference - -```text -testmu-a2a auth Authenticate with TestMu AI -testmu-a2a test Quick chat agent test (one command) -testmu-a2a init Initialize testmu-a2a.yaml config -testmu-a2a run Run tests from testmu-a2a.yaml -testmu-a2a call Test phone agents with real calls -testmu-a2a redteam Adversarial security testing -testmu-a2a prompts Set agent prompt and upload requirements -testmu-a2a projects Manage projects -testmu-a2a results View chat evaluation results -testmu-a2a workflows Manage chat testing workflows -testmu-a2a sources Upload documents and generate scenarios -testmu-a2a scenarios Manage chat test scenarios -testmu-a2a phone-scenarios Manage phone test scenarios -testmu-a2a suites Manage test suites -testmu-a2a schedules Manage scheduled runs -testmu-a2a call-results View phone call results -testmu-a2a profiles Manage test/agent/endpoint profiles -testmu-a2a recordings Upload and analyze call recordings -testmu-a2a voices Browse available voices -testmu-a2a personas Manage test personas -testmu-a2a phone-numbers Manage phone numbers -testmu-a2a thresholds Manage pass/fail thresholds -testmu-a2a assessments Go-live readiness assessments -testmu-a2a health System health check -testmu-a2a credits View credit balance -``` - -## Authenticate Your Account - -All CLI commands require authentication with your TestMu AI credentials. - -### Log in Interactively - -```bash -testmu-a2a auth -u -k -``` - -You can point to a specific environment by passing `--base-url`: - -```bash -# Local development -testmu-a2a auth -u -k --base-url http://localhost:8000 - -# Staging -testmu-a2a auth -u -k --base-url https://stage-agent-testing.lambdatestinternal.com - -# Production (default) -testmu-a2a auth -u -k -``` - -### Authenticate in CI/CD - -For automated environments, set environment variables instead of running `testmu-a2a auth`: - -```bash -export TESTMU_USERNAME= -export TESTMU_ACCESS_KEY= -export TESTMU_BASE_URL=https://agent-testing.lambdatest.com -``` - -TestMu AI aliases are also supported: - -```bash -export LT_USERNAME= -export LT_ACCESS_KEY= -``` - -### Check Status and Log Out - -```bash -testmu-a2a auth status -testmu-a2a auth logout -``` - -Credentials are stored in `~/.testmu-a2a/credentials.json` with owner-only permissions (600). - -## Test a Chat Agent - -The fastest way to test a chat agent is with a single `test` command. Point it at your agent endpoint, describe what the agent does, and the CLI generates scenarios and evaluates responses automatically. - -```bash -testmu-a2a test \ ---agent https://my-bot.com/api/chat \ ---spec "A travel booking assistant that helps users find flights" \ ---count 10 -``` - -If your agent expects a custom request format, specify the body template and response path: - -```bash -testmu-a2a test \ ---agent https://my-bot.com/chat \ ---body-template '{"input": "{{message}}"}' \ ---response-path "output.text" \ ---spec "Customer support bot for an e-commerce store" \ --H "Authorization: Bearer " -``` - -| Flag | Description | Default | -|------|-------------|---------| -| `--agent`, `-a` | Target agent endpoint URL | Required | -| `--spec`, `-s` | Agent description or path to spec file | None | -| `--count`, `-n` | Number of test scenarios | 10 | -| `--categories`, `-c` | Comma-separated categories | All | -| `--threshold`, `-t` | Pass/fail threshold (0.0-1.0) | 0.80 | -| `--max-turns` | Max conversation turns per scenario | 10 | -| `--format`, `-f` | Output format (table, json, junit) | table | -| `--output`, `-o` | Write results to file | None | -| `--verbose`, `-v` | Show conversation transcripts | false | -| `--parallel`, `-p` | Number of parallel evaluations | 5 | -| `--body-template` | JSON body with `{{message}}` placeholder | None | -| `--response-path` | JSONPath to extract agent reply | None | -| `--method`, `-m` | HTTP method | POST | -| `--header`, `-H` | Custom header (repeatable) | None | - -### Run Tests from a Config File - -For repeated testing, initialize a config file instead of passing flags every time: - -```bash -testmu-a2a init --endpoint https://my-bot.com/api/chat -``` - -This creates `testmu-a2a.yaml` and supporting directories: - -```text -testmu-a2a.yaml Project configuration -specs/ Spec documents (PDF, DOCX, MD) -scenarios/ Custom scenario YAML files -reports/ Test reports -``` - -A typical `testmu-a2a.yaml` looks like this: - -```yaml -agent: -endpoint: "https://my-bot.com/api/chat" -type: chat -method: POST -headers: -Content-Type: "application/json" -body_template: -message: "{{message}}" -response_path: "data.reply" - -scenarios: -generate: -from: ./specs/ -categories: -- conversational-flow -- intent-recognition -- context-memory -- error-handling -- security -count: 30 - -evaluation: -thresholds: -accuracy: 0.80 -relevance: 0.80 -coherence: 0.80 -context_retention: 0.75 -max_turns: 10 -output_format: table - -security: -enabled: true -intensity: intermediate -categories: -- prompt-injection -- jailbreak -- pii-leakage -- data-exfiltration -``` - -Then run all tests, a specific category, or output for CI/CD: - -```bash -testmu-a2a run -testmu-a2a run --category security -testmu-a2a run --format junit --output results.xml -``` - -## Manage Workflows - -Workflows are the execution context for chat test scenarios within a project. - -```bash -testmu-a2a workflows create --project -testmu-a2a workflows list --project -testmu-a2a workflows summary -testmu-a2a workflows rename --project --name "My Workflow" -testmu-a2a workflows files --project -testmu-a2a workflows delete --project -testmu-a2a workflows delete --project --yes # skip confirmation -``` - -### Chat Testing Flow with Explicit Workflow - -Use this when you already have a project and want full control over each step: - -```bash -# Step 1: Create workflow -testmu-a2a workflows create --project - -# Step 2: Upload documents -testmu-a2a sources upload \ ---workflow \ ---project \ ---files ./spec.pdf,./faq.md - -# Step 3: Generate scenarios from the uploaded docs -testmu-a2a run --workflow --project - -# Step 4: View results -testmu-a2a results --project -``` - -Or combine upload and generation in one step: - -```bash -testmu-a2a sources upload-and-generate \ ---workflow \ ---project \ ---files ./spec.pdf,./faq.md \ ---categories intent-recognition,context-memory,error-handling -``` - -## Test a Phone Agent - -The CLI can place real phone calls to test inbound and outbound voice agents. You can either run a quick one-shot call or build a full suite-based workflow. - -### Run a Quick One-Shot Call - -```bash -testmu-a2a call \ ---number \ ---persona frustrated \ ---scenario "Customer wants to cancel their premium subscription" \ ---voice Neha \ ---voice-provider vapi -``` - -| Flag | Description | Default | -|------|-------------|---------| -| `--number`, `-n` | Phone number (E.164 format) | Required | -| `--persona`, `-p` | Test persona | neutral | -| `--scenario`, `-s` | Scenario description | General inquiry | -| `--provider` | Voice provider (vapi, pipecat, bolna) | vapi | -| `--voice` | Voice ID (e.g., Neha, andrew) | Provider default | -| `--voice-provider` | Voice synthesis (vapi, azure, 11labs, google) | vapi | -| `--type`, `-t` | Call type (inbound, outbound) | inbound | -| `--max-duration` | Max call duration in seconds | 180 | -| `--verbose`, `-v` | Show call transcript | false | -| `--format`, `-f` | Output format (table, json) | table | - -### Test an Inbound Phone Agent - -For structured testing, walk through these steps to create a project, generate scenarios, and run them as a suite. - -**Step 1: Create a phone project.** - -```bash -testmu-a2a projects create \ ---name "Airline Support Agent" \ ---description "Testing our IVR booking agent" \ ---type phone_caller_inbound -``` - -Note the project ID from the output. - -**Step 2: Set the agent prompt.** This is the most important step - the prompt drives scenario generation, evaluation criteria, and go-live assessments. - -From a YAML file (recommended, define everything in one place): - -```bash -testmu-a2a prompts set --project --from-file prompt.yaml -``` - -`prompt.yaml`: - -```yaml -prompt: | -You are an airline booking assistant. You help customers find flights, -make reservations, handle cancellations, and process refunds. -Always verify the customer's identity before making changes. -Never share other customers' booking information. - -# Or reference a separate file (path relative to this YAML) -# prompt_file: ./agent_system_prompt.md - -context: "Agent must comply with DOT airline passenger rights regulations" - -files: -- ./compliance_rules.pdf -- ./fare_structure.docx -``` - -Inline: - -```bash -testmu-a2a prompts set --project \ ---prompt "You are an airline booking assistant. You help customers find -flights, make reservations, handle cancellations, and process refunds. -Always verify the customer's identity before making changes. -Never share other customers' booking information. -If the customer is upset, empathize before offering solutions." -``` - -From a prompt file: - -```bash -testmu-a2a prompts set --project \ ---prompt-file ./agent_system_prompt.md -``` - -With additional requirement documents (compliance rules, product specs, etc.): - -```bash -testmu-a2a prompts set --project \ ---prompt-file ./agent_prompt.md \ ---files ./compliance_rules.pdf,./fare_structure.docx \ ---context "Agent must comply with DOT airline passenger rights regulations" -``` - -Verify what was saved: - -```bash -testmu-a2a prompts get --project -``` - -**Step 3: Generate test scenarios.** - -```bash -testmu-a2a phone-scenarios generate \ ---project \ ---count 5 \ ---personas "frustrated,confused,elderly,rushed" \ ---instructions "Test the agent's ability to handle flight booking, cancellation, and rebooking" -``` - -**Step 4: Review generated scenarios.** - -```bash -testmu-a2a phone-scenarios list --project -``` - -You can also create manual scenarios: - -```bash -testmu-a2a phone-scenarios create \ ---project \ ---title "Customer cancels mid-booking" \ ---description "Customer starts booking a flight, then changes mind halfway" \ ---persona "indecisive" -``` - -**Step 5: Create a test suite.** You can pass scenario IDs directly, or use a YAML file for per-scenario call configuration (number, voice, background sound). - -```bash -# Simple - same call config for all scenarios, supply number/voice at run time -testmu-a2a suites create \ ---project \ ---name "Booking Flow Regression" \ ---scenarios ",," - -# Per-scenario config from a YAML file -testmu-a2a suites create \ ---project \ ---name "Booking Flow Regression" \ ---from-file suite.yaml -``` - -`suite.yaml`: - -```yaml -scenarios: -- id: -phone_number: "+15551234567" -voice: Neha -voice_provider: vapi -background_sound_url: https://example.com/office-noise.mp3 -background_sound_enabled: true - -- id: -phone_number: "+15559876543" -voice: andrew -voice_provider: azure - -- id: -phone_number: "+15551234567" -``` - -Available YAML fields per scenario: - -| Field | Description | -|-------|-------------| -| `id` | Scenario ID (required) | -| `phone_number` | Phone number to call (E.164 format) | -| `voice` | Voice ID (e.g., Neha, andrew) | -| `voice_provider` | Voice synthesis: vapi, azure, 11labs, google | -| `background_sound_enabled` | Enable background noise (true/false) | -| `background_sound_url` | URL of background audio file | -| `voice_name` | Voice display name | -| `first_speaker` | Who speaks first: simulator (default) or agent | -| `wait_seconds` | Response delay in seconds (0.5-5.0) | -| `max_duration_seconds` | Max call duration in seconds (60-1800) | - -**Step 6: Run the suite.** - -```bash -# If per-scenario config was stored at create time -testmu-a2a suites run \ ---project \ ---name "Booking Flow Regression" - -# Or override all scenarios at run time with the same number/voice -testmu-a2a suites run \ ---project \ ---name "Booking Flow Regression" \ ---number +15551234567 \ ---voice Neha \ ---voice-provider vapi \ ---background-sound https://example.com/office-noise.mp3 -``` - -**Step 7: Check results.** - -```bash -testmu-a2a call-results list --project -testmu-a2a call-results get -testmu-a2a call-results summary -``` - -**Step 8: Schedule recurring runs.** - -```bash -testmu-a2a schedules create \ ---project \ ---suite \ ---frequency daily \ ---time 09:00 -``` - -**Step 9: Go-live readiness check.** - -```bash -testmu-a2a assessments create --project --type phone -``` - -### Test an Outbound Phone Agent - -The outbound workflow follows the same steps. Create the project with `--type phone_caller_outbound` and use outbound-specific scenario generation: - -```bash -testmu-a2a projects create \ ---name "Sales Outreach Agent" \ ---description "Testing outbound sales calls" \ ---type phone_caller_outbound - -testmu-a2a prompts set --project \ ---prompt "You are a sales agent for Acme Corp. You call existing customers -to offer premium plan upgrades. Be polite, handle objections gracefully, -and never pressure the customer. If they say no, thank them and end the call." - -testmu-a2a phone-scenarios generate \ ---project \ ---count 5 \ ---type outbound \ ---personas "busy executive,interested buyer,skeptical prospect" \ ---instructions "Agent offers premium plan upgrade, handles objections" - -testmu-a2a suites create \ ---project \ ---name "Outbound Sales Test" \ ---scenarios ",," - -testmu-a2a suites run --project --name "Outbound Sales Test" -``` - -## Run Red Team Security Tests - -The `redteam` command runs adversarial attacks across 9 categories at 3 difficulty levels, then grades your agent's resilience from A+ to F. - -```bash -testmu-a2a redteam \ ---agent https://my-bot.com/api/chat \ ---intensity advanced \ ---spec "Banking customer support agent" -``` - -To test specific categories: - -```bash -testmu-a2a redteam \ ---agent https://my-bot.com/api/chat \ ---categories prompt-injection,jailbreak,pii-leakage -``` - -Attack categories: `prompt-injection`, `jailbreak`, `data-exfiltration`, `pii-leakage`, `harmful-content`, `overreliance`, `hijacking`, `policy-violation`, `technical-injection` - -Intensity levels: `basic`, `intermediate`, `advanced` - -Output includes a letter grade (A+ through F) and per-category breakdown. - -## Set Agent Prompts and Requirements - -The prompt is the single most important input - it tells TestMu AI what your agent does so it can generate relevant scenarios and evaluate correctly. - -### Set Prompt from YAML (recommended) - -Define prompt, files, and context in one file: - -```bash -testmu-a2a prompts set --project --from-file prompt.yaml -``` - -`prompt.yaml`: - -```yaml -prompt: | -You are a customer support agent for a SaaS product. -You help users with billing, account issues, and technical troubleshooting. -Always verify the user's email before making account changes. -Escalate to a human if the customer asks for a refund over $500. - -# Or reference a separate file: -# prompt_file: ./agent_system_prompt.md - -context: "Agent must comply with GDPR and never store PII in logs" - -files: -- ./compliance_rules.pdf -- ./product_catalog.docx -- ./faq.md -``` - -All file paths in the YAML are resolved relative to the YAML file's location. - -### Set Prompt Inline - -```bash -testmu-a2a prompts set --project \ ---prompt "You are a customer support agent for a SaaS product. -You help users with billing, account issues, and technical troubleshooting. -Always verify the user's email before making account changes. -Escalate to a human if the customer asks for a refund over $500." -``` - -### Set Prompt from File - -```bash -testmu-a2a prompts set --project \ ---prompt-file ./agent_system_prompt.md -``` - -### Set Prompt with Additional Requirements - -Upload compliance docs, product specs, or knowledge base files alongside the prompt: - -```bash -testmu-a2a prompts set --project \ ---prompt-file ./agent_prompt.md \ ---files ./compliance_rules.pdf,./product_catalog.docx,./faq.md \ ---context "Agent must comply with GDPR and never store PII in logs" -``` - -Supported file types: PDF, DOCX, TXT, MD, XLSX, MP3, WAV, M4A - -### Manage Existing Prompts - -```bash -testmu-a2a prompts get --project -testmu-a2a prompts get --project --format json - -testmu-a2a prompts update --project --id \ ---prompt "Updated prompt text..." -testmu-a2a prompts update --project --id \ ---prompt-file ./updated_prompt.md - -testmu-a2a prompts delete --project --id -``` - -## Manage Projects - -Projects organize your agents and tests. Each project has a type that determines the available testing features. - -```bash -testmu-a2a projects list -testmu-a2a projects list --format json - -testmu-a2a projects create \ ---name "My Agent" \ ---description "Agent description" \ ---type chat - -testmu-a2a projects update --name "New Name" -testmu-a2a projects update --description "Updated description" -testmu-a2a projects update \ ---name "New Name" \ ---description "Updated description" - -testmu-a2a projects delete -testmu-a2a projects delete --yes # skip confirmation -``` - -Project types: `chat`, `phone_caller_inbound`, `phone_caller_outbound`, `image_analyzer` - -## Manage Scenarios - -### Chat Scenarios - -```bash -testmu-a2a scenarios list --workflow --project - -testmu-a2a scenarios create \ ---workflow \ ---project \ ---title "Edge case: empty input" \ ---description "Test how agent handles empty messages" \ ---persona "confused user" - -testmu-a2a scenarios delete \ ---workflow \ ---project \ ---ids "," - -# Import, export, and templates -testmu-a2a scenarios export \ ---workflow \ ---project \ ---output scenarios.csv - -testmu-a2a scenarios import \ ---workflow \ ---project \ ---file scenarios.csv - -testmu-a2a scenarios template --workflow -``` - -### Phone Scenarios - -```bash -testmu-a2a phone-scenarios list --project - -# Generate inbound scenarios -testmu-a2a phone-scenarios generate \ ---project \ ---count 10 \ ---personas "frustrated,confused,elderly" \ ---instructions "Focus on billing and refund scenarios" - -# Generate outbound scenarios -testmu-a2a phone-scenarios generate \ ---project \ ---count 5 \ ---type outbound \ ---personas "busy,skeptical" - -# Create manually -testmu-a2a phone-scenarios create \ ---project \ ---title "Angry customer wants refund" \ ---description "Customer received wrong item, demands immediate refund" \ ---persona "angry" - -# Edit a scenario -testmu-a2a phone-scenarios edit \ ---project \ ---id \ ---title "Updated title" \ ---persona "frustrated" - -# Delete scenarios -testmu-a2a phone-scenarios delete \ ---project \ ---ids "," - -# Bulk import/export -testmu-a2a phone-scenarios import --project --file scenarios.csv -testmu-a2a phone-scenarios template --project -``` - -## Manage Test Suites - -Suites group scenarios for repeatable test runs. - -```bash -testmu-a2a suites list --project - -testmu-a2a suites create \ ---project \ ---name "Regression Suite" \ ---scenarios ",," - -# Or create with per-scenario call config from YAML -testmu-a2a suites create \ ---project \ ---name "Regression Suite" \ ---from-file suite.yaml - -testmu-a2a suites run --project --name "Regression Suite" -testmu-a2a suites overview --project - -testmu-a2a suites update \ ---id \ ---name "Updated Suite Name" \ ---scenarios "," - -testmu-a2a suites update \ ---id \ ---from-file suite.yaml -``` - -## Schedule Recurring Runs - -Automate recurring test runs by attaching a schedule to a suite. - -```bash -testmu-a2a schedules list --project - -# Daily -testmu-a2a schedules create \ ---project \ ---suite \ ---frequency daily \ ---time 09:00 - -# Weekly -testmu-a2a schedules create \ ---project \ ---suite \ ---frequency weekly \ ---days mon,wed,fri \ ---time 14:00 - -testmu-a2a schedules trigger -testmu-a2a schedules update --frequency daily --time 10:00 -testmu-a2a schedules delete -``` - -## View Test Results - -### View Call Results - -```bash -testmu-a2a call-results list --project -testmu-a2a call-results list --suite -testmu-a2a call-results get -testmu-a2a call-results get --audio -testmu-a2a call-results summary - -# Bookmarks -testmu-a2a call-results bookmark -testmu-a2a call-results bookmark --remove -testmu-a2a call-results bookmarked --suite -``` - -### View Chat Evaluation Results - -```bash -testmu-a2a results --project -testmu-a2a results --project --format json -testmu-a2a results --project --format junit --output results.xml -``` - -## Analyze Call Recordings - -Upload and analyze existing call recordings without placing new calls. - -```bash -testmu-a2a recordings upload --project --files call1.mp3,call2.wav -testmu-a2a recordings analyze -testmu-a2a recordings result -testmu-a2a recordings transcript -testmu-a2a recordings list --project -testmu-a2a recordings metrics - -# Bookmark/unbookmark -testmu-a2a recordings bookmark -testmu-a2a recordings bookmark --remove - -testmu-a2a recordings delete -``` - -## Manage Profiles - -Profiles store reusable test data, agent configurations, and endpoint details. - -### Test Profiles - -```bash -testmu-a2a profiles test list --project -testmu-a2a profiles test get --project --id -testmu-a2a profiles test create \ ---project \ ---name "Premium User" \ ---data '{"name": "John Doe", "plan": "premium", "account_id": "ACC123"}' -testmu-a2a profiles test delete --project --ids "," -``` - -### Agent Profiles - -```bash -testmu-a2a profiles agent list -testmu-a2a profiles agent create \ ---name "Support Agent v2" \ ---data '{"agent_type": "support", "version": "2.0"}' -``` - -### Endpoint Profiles - -```bash -testmu-a2a profiles endpoint list --project -testmu-a2a profiles endpoint create \ ---project \ ---name "Production Endpoint" \ ---data '{"url": "https://api.example.com/chat", "method": "POST"}' -``` - -## Configure Pass/Fail Thresholds - -Set pass/fail criteria for evaluations. - -```bash -testmu-a2a thresholds get --project --type chat -testmu-a2a thresholds get --project --type phone - -testmu-a2a thresholds set \ ---project \ ---type chat \ ---config '{"accuracy": 0.85, "relevance": 0.80, "coherence": 0.80}' - -testmu-a2a thresholds set \ ---project \ ---type phone \ ---config '{"resolution_rate": 0.90, "avg_response_time": 2.0}' -``` - -## Run Go-Live Assessments - -Get a production-readiness verdict for your agent before deploying. - -```bash -testmu-a2a assessments create --project --type phone -testmu-a2a assessments create --project --type chat -testmu-a2a assessments get --project --type phone -testmu-a2a assessments history --project --type phone -``` - -## Browse the Voice Library - -Browse available voices to use in phone tests. Use the Name column value as `providerId` when configuring per-scenario voice in a suite. - -```bash -testmu-a2a voices list -testmu-a2a voices list --provider azure -testmu-a2a voices list --provider 11labs -testmu-a2a voices list --provider google - -# Filter by language (azure only) -testmu-a2a voices list --provider azure --language es # Spanish -testmu-a2a voices list --provider azure --language hi # Hindi -testmu-a2a voices list --provider azure --language multi # Multilingual -testmu-a2a voices list --provider azure --language all # All languages - -# Filter by target platform -testmu-a2a voices list --target bolna -testmu-a2a voices list --provider 11labs --target pipecat - -testmu-a2a voices list --format json -``` - -## Manage Personas - -Built-in personas are always available: neutral, frustrated, confused, elderly, tech-savvy, rushed, and 25+ more. You can also create custom personas: - -```bash -testmu-a2a personas list --org - -testmu-a2a personas create \ ---org \ ---name "Impatient Executive" \ ---description "A busy executive who expects quick, direct answers with no filler" -``` - -## Manage Phone Numbers - -```bash -testmu-a2a phone-numbers list --org - -testmu-a2a phone-numbers create \ ---org \ ---data '{"phoneNumber": "+15551234567", "name": "Support Line"}' - -testmu-a2a phone-numbers delete --org -``` - -## Check System Health and Credits - -```bash -testmu-a2a health -testmu-a2a health info # detailed system info -testmu-a2a health agents # list available agent types - -testmu-a2a credits # balance summary -testmu-a2a credits totals # detailed breakdown -``` - -## Integrate with CI/CD - -Add the CLI to your pipeline to test agents on every push. Use `--format junit` to produce standard test reports. - -### GitHub Actions - -```yaml -name: Agent Tests -on: [push] - -jobs: -test: -runs-on: ubuntu-latest -steps: -- uses: actions/checkout@v4 -- uses: actions/setup-python@v5 -with: -python-version: '3.11' - -- name: Install TestMu A2A CLI -run: pip install testmu-a2a-cli - -- name: Run agent tests -env: -TESTMU_USERNAME: ${{ secrets.TESTMU_USERNAME }} -TESTMU_ACCESS_KEY: ${{ secrets.TESTMU_ACCESS_KEY }} -run: | -testmu-a2a test \ ---agent ${{ vars.AGENT_ENDPOINT }} \ ---spec "Customer support chatbot" \ ---count 10 \ ---format junit \ ---output results.xml - -- name: Publish results -uses: dorny/test-reporter@v1 -if: always() -with: -name: Agent Test Results -path: results.xml -reporter: java-junit -``` - -### Exit Codes - -| Code | Meaning | -|------|---------| -| 0 | All tests passed | -| 1 | One or more tests failed, or a command error occurred | - -## Choose an Output Format - -| Format | Flag | Use Case | -|--------|------|----------| -| table | `--format table` | Human-readable terminal output | -| json | `--format json` | Programmatic consumption, piping | -| junit | `--format junit` | CI/CD test reporters | - -Write output to a file with `--output `: - -```bash -testmu-a2a test --agent --format junit --output results.xml -testmu-a2a test --agent --format json --output results.json -``` - -## Global Options - -| Flag | Description | -|------|-------------| -| `--version`, `-V` | Show CLI version | -| `--help`, `-h` | Show help for any command | -| `--install-completion` | Install shell completion | - -Shell completion works with bash, zsh, and fish: - -```bash -testmu-a2a --install-completion -``` - -## Troubleshoot Common CLI Failures - -Most failed runs come from authentication, network reach, or account limits. Each has a direct fix. - -- **What you see:** a command exits immediately or reports an authentication error. **Why:** the CLI is not logged in, or `TESTMU_USERNAME` and `TESTMU_ACCESS_KEY` are unset in CI. **Fix:** run `testmu-a2a auth status` locally, or set both environment variables in the pipeline. - -- **What you see:** the CLI cannot reach your agent. **Why:** the endpoint is private, behind a firewall, or not served over HTTPS. **Fix:** expose the agent over HTTPS, or reach it through the secure tunnel so the endpoint needs no public URL. - -- **What you see:** phone scenarios queue instead of running together. **Why:** each organization runs up to 5 parallel phone calls by default. **Fix:** reduce concurrent scenarios, or purchase additional parallel-call capacity. - -- **What you see:** a `call` command is rejected before it dials. **Why:** `--number` is not in E.164 format. **Fix:** pass the number with country code and no spaces, such as `+15551234567`. - -- **What you see:** a run stops partway with a credit-related message. **Why:** the account's monthly credits are exhausted. **Fix:** check the balance with `testmu-a2a credits`, then top up or wait for the monthly reset. - -## Related TestMu AI Guides - -- See how the platform [runs an evaluation end to end](/support/docs/architecture-and-how-evaluation-works/). -- See how to [connect a chat agent over its API](/support/docs/chat-agent-api-integration/). diff --git a/static/docs/testmu-azure-devops-app.md b/static/docs/testmu-azure-devops-app.md deleted file mode 100644 index 1482873ae..000000000 --- a/static/docs/testmu-azure-devops-app.md +++ /dev/null @@ -1,41 +0,0 @@ -# TestMu AI Azure DevOps App - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI Test Manager integrates directly with Azure DevOps through the Azure DevOps Marketplace App, allowing you to manage tests seamlessly within the Azure DevOps interface. It provides Test Manager functionalities in your Azure DevOps environment, boosting workflow efficiency and facilitating improved collaboration among teams. - -**Let's have a look at how to use the TestMu AI Azure DevOps App:** - -## Installation and Setup - -1. Head over to the [TestMu AI Test Manager](https://marketplace.visualstudio.com/items?itemName=Lambdatest.lambdatest-ai-tms-app) Azure DevOps App on the Azure DevOps Marketplace. Select **Get it free** as shown in the below image. - -2. A screen will appear asking you to select the Azure DevOps organization. Choose your organization and click on **Install** to install the app. - -3. In the Organization Settings, navigate to **Extensions** and look for **TestMu AI Test Manager**, click on it to configure the app. Enter the required credentials, **Username** and **Access Key** of your TestMu AI account. Then, click on **Authenticate** to proceed. - -> **Note:** The TestMu AI account credentials used to configure the App must also install and integrate Azure DevOps in their TestMu AI account through integrations. - -## Linking Existing Test Cases - -4. Go to any Azure DevOps work item and click on the **TestMu AI** tab located in the work item details. Click on **Link Test Case** to link a test case to Azure DevOps work item. - -5. Enter the test case name or test case ID and select any of the test cases for which you want to link a work item. Test Cases created across all projects in your TestMu AI Organization will be accessible here. - -Once linked, the Test Case will be visible in the Test Cases section. - -6. Go to **Execution History** to view all the executions of the test cases that are linked to the Azure DevOps work item. - -## Creating New Test Cases - -7. You can also create test cases with AI or manually within the Azure DevOps interface. To achieve that, click on **Add Test Case** and choose an option to start creating Test Cases. Let's see how we can utilize the **Generate with AI** functionality. - -> **Note:** To create test cases, the user must have access to Test Manager in their TestMu AI account. - -8. Provide your requirements by attaching files or adding textual requirements. By default, the existing Azure DevOps work item will be attached in the input box. - -9. AI will start generating Test scenarios and Test Cases based on the input requirements. You can view test case title, description, pre-conditions, steps and other fields. Once you have reviewed the test cases, you can select the test cases and save them in the specified Project and Folder. - -10. You can view all the selected test cases saved in the Test Cases section. All the test cases that are getting saved will get automatically linked to the corresponding Work Item. - -This allows you to seamlessly connect Test Cases right from Azure DevOps work items, enhancing your workflows and maintaining traceability between your development tasks and testing activities. diff --git a/static/docs/testmu-capability-map.md b/static/docs/testmu-capability-map.md deleted file mode 100644 index 052c07602..000000000 --- a/static/docs/testmu-capability-map.md +++ /dev/null @@ -1,103 +0,0 @@ -# TestMu AI Capability Map to Migrate from BrowserStack & SauceLabs - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Switching automation platforms can seem challenging, but a clear capability comparison helps simplify the decision and migration process. Teams often explore alternative testing platforms to improve speed, scalability, test coverage, and overall efficiency in their delivery pipelines. - -TestMu AI provides a cloud-based test execution environment that supports various automation frameworks, real device testing, visual validations, debugging tools, and analytics, making it a flexible option for organizations aiming to optimize and modernize their testing workflows. - -## Authentication & TestMu AI Hub Configuration - -To migrate your automation tests from BrowserStack or Sauce Labs to TestMu AI, you must add TestMu AI authentication credentials and configure the execution hub endpoint. - -### Add Authentication Credentials -Firstly, you need to change the authentication in your configuration settings of your test suite. For running tests on TestMu AI Selenium Grid, you need to have a valid user_name and access_key to perform tests on our Grid. In case you do not have an account on TestMu AI, visit the TestMu AI signup page and create a new account. - -When migrating your test from BrowserStack or Sauce Labs to TestMu AI, the following updates are required in your existing code: - -1. Get TestMu AI Credentials: You can find these credentials under Account Settings > [Password & Security](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/security/username-accesskey) and copy your Username and Access Key, then add them to the .env file to keep them safe from public exposure. - -2. Create .env file: Securely store your TestMu AI credentials, create a .env file in the root of your project and add the following values: - -``` -LT_USERNAME="" -LT_ACCESS_KEY="" -``` - -Once the .env file is set up, ensure your test framework correctly reads these variables at runtime. This helps keep your authentication secure and avoids hard-coding credentials within your scripts. With the credentials in place, you’re now ready to update your Hub URL for TestMu AI execution. - -### Add TestMu AI Hub URL -Next, you need to add the TestMu AI Hub URL to your test suite configuration. This endpoint defines where the tests will be executed on the TestMu AI Grid. - -```js -@hub.lambdatest.com/wd/hub -``` - -## Migrating from BrowserStack to TestMu AI - -Switching between cloud-based automation platforms may require capability evaluation, test configuration updates, and CI/CD integration mapping. A side-by-side comparison helps simplify this transition by clarifying feature coverage, execution options, testing frameworks, and platform-level differences. - -TestMu AI can serve as an alternative to BrowserStack for teams seeking improved execution speed, parallel scalability, cost-efficiency, advanced debugging, and deeper analytics for test optimization. With support for a wide range of automation frameworks, device coverage, and performance-focused orchestration, TestMu AI enables smoother migration for teams modernizing their testing infrastructure. - -Below are the capabilities of BrowserStack and TestMu AI to help teams map BrowserStack features to their TestMu AI equivalents. - -```js -MutableCapabilities capabilities = new MutableCapabilities(); -HashMap bstackOptions = new HashMap(); -capabilities.setCapability("browserName", "Safari"); -bstackOptions.put("os", "OS X"); -bstackOptions.put("osVersion", "Tahoe"); -bstackOptions.put("browserVersion", "26.0"); -bstackOptions.put("userName", "YOUR_USERNAME"); -bstackOptions.put("accessKey", "YOUR_ACCESS_KEY"); -capabilities.setCapability("bstack:options", bstackOptions); -``` - -```js -SafariOptions browserOptions = new SafariOptions(); -browserOptions.setPlatformName("MacOS Tahoe"); -browserOptions.setBrowserVersion("26"); -HashMap ltOptions = new HashMap(); -ltOptions.put("username", ""); -ltOptions.put("accessKey", ""); -ltOptions.put("w3c", true); -browserOptions.setCapability("LT:Options", ltOptions); -``` - -To migrate your test suites from BrowserStack, checkout this guide on [Migrating from BrowserStack to TestMu AI](/support/docs/browserstack-to-testmu-migration-guide/). - -## Migrating from Sauce Labs to TestMu AI - -Switching between cloud-based test execution platforms can involve configuration updates, capability adjustments, and integration alignment. A capability comparison makes this transition more seamless by helping teams understand feature availability, execution models, and tooling compatibility. - -TestMu AI can be adopted as an alternative to Sauce Labs for organizations looking to enhance performance, scale automation workloads efficiently, and streamline debugging and reporting within their CI/CD pipelines. - -Below are the capabilities of Sauce Labs and TestMu AI to help teams map Sauce Labs features to their TestMu AI equivalents. - -```js -SafariOptions browserOptions = new SafariOptions(); -browserOptions.setPlatformName("macOS 15"); -browserOptions.setBrowserVersion("latest"); -Map sauceOptions = new HashMap<>(); -sauceOptions.put("username", "YOUR_USERNAME"); -sauceOptions.put("accessKey", "YOUR_ACCESS_KEY"); -sauceOptions.put("build", ""); -sauceOptions.put("name", ""); -sauceOptions.put("armRequired", true); -browserOptions.setCapability("sauce:options", sauceOptions); -``` - -```js -SafariOptions browserOptions = new SafariOptions(); -browserOptions.setPlatformName("MacOS Tahoe"); -browserOptions.setBrowserVersion("26"); -HashMap ltOptions = new HashMap(); -ltOptions.put("username", ""); -ltOptions.put("accessKey", ""); -ltOptions.put("w3c", true); -browserOptions.setCapability("LT:Options", ltOptions); -``` - -To migrate your test suites from Sauce Labs, checkout this guide on [Migrating from Sauce Labs to TestMu AI](/support/docs/saucelabs-to-testmu-migration-guide/). - -To generate capabilities use [TestMu AI Capabilities Generator](https://www.testmuai.com/capabilities-generator/) to define key automation testing parameters, such as browser, version, operating system, and additional test settings. diff --git a/static/docs/testmu-grid-configuration.md b/static/docs/testmu-grid-configuration.md deleted file mode 100644 index cba89ff4f..000000000 --- a/static/docs/testmu-grid-configuration.md +++ /dev/null @@ -1,23 +0,0 @@ -# TestMu AI Selenium Grid Configuration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -We constantly upgrade our grid with the stable version of Selenium. - -The grid auto-selects the latest stable versions of Selenium and browser drivers. - -## Selenium Automation Grid Configuration - -Review the current driver versions, HUB URI, port, and default settings for the grid. - -| KEY | VALUE | -|-----------------------------|---------------------------------------------------------| -| Selenium version | Latest (grid auto-selects) - Auto-updated[Want to change version?](https://www.testmuai.com/capabilities-generator/) | -| IEDriver version | Latest (grid auto-selects) - Auto-updated[Want to change version?](https://www.testmuai.com/capabilities-generator/) | -| ChromeDriver version | Latest (grid auto-selects) - Auto-updated[Want to change version?](https://www.testmuai.com/capabilities-generator/) | -| Firefox/GeckoDriver Version | Latest (grid auto-selects) - Auto-updated[Want to change version?](https://www.testmuai.com/capabilities-generator/) | -| HUB URI | [hub.lambdatest.com](https://hub.lambdatest.com/) | -| HUB Port | 443 (HTTPS/SSL) | -| Window Mode | Multi-window | -| Default Idle Timeout | 120 seconds[Want to change Idle Timeout?](https://www.testmuai.com/capabilities-generator/) | -| Default Screen Resolution | 1024x768[Want to change Resolution?](https://www.testmuai.com/capabilities-generator/) | diff --git a/static/docs/testmu-jira-app.md b/static/docs/testmu-jira-app.md deleted file mode 100644 index 6a6234187..000000000 --- a/static/docs/testmu-jira-app.md +++ /dev/null @@ -1,21 +0,0 @@ -# TestMu AI Jira App - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI Test Manager integrates directly with the Jira App, allowing you to manage tests seamlessly within the Jira interface. It provides Test Manager functionalities in your Jira environment, boosting workflow efficiency and facilitating improved collaboration among teams. - -**Let's have a look at how to use the TestMu AI Jira App:** - -1. Download the Jira app from the Atlassian Marketplace. Once installed, navigate to **Manage your Apps** under the **Apps** section in your Jira project. - -2. Navigate to the sidebar on the left, select **TestMu AI** from the **Apps** menu, and enter the required credentials, such as your **Username** and **Access Token**. Then, click on **Authenticate** to proceed. - -3. Go to any Jira issue and click on the TestMu AI icon located just below the issue name. This will display the Test Cases linked to that issue. - -4. Select **Execution History** to have a look at all the executions of the test cases that are linked to the Jira Issue. - -5. Click on **Link Issue**, after which an instance will appear enabling you to link an issue to the test case. - -6. Enter the test case name and select any of the test cases for which you want to link an issue. All test cases created in your Test Manager dashboard will be available in this list. - -This allows you to seamlessly connect Test Cases right from Jira issues, enhancing your workflows. diff --git a/static/docs/testmu-mcp-server.md b/static/docs/testmu-mcp-server.md deleted file mode 100644 index 6f6ce99cd..000000000 --- a/static/docs/testmu-mcp-server.md +++ /dev/null @@ -1,252 +0,0 @@ -# Getting Started with TestMu AI MCP Server - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI MCP Server is a remote MCP server that connects MCP-compatible AI clients to your TestMu AI testing environment. It gives you access to tools for different TestMu AI products - HyperExecute, Automation, SmartUI, Accessibility, and Test Manager - which you can invoke through natural language prompts in your IDE or AI client. - -## Available Tools - -TestMu AI MCP Server provides five tools, each covering a different area of testing: - -1. **[HyperExecute Tool:](/support/docs/hyperexecute-mcp-server/)** AI-native test orchestration. Auto-generates test runner commands and HyperExecute YAML, monitors jobs, and surfaces session details. -2. **[Automation Tool:](/support/docs/automation-mcp-server/)** Triage and debug automation test failures. Pulls execution data, command/network/console logs, and helps identify root cause directly inside your IDE. -3. **[SmartUI Tool:](/support/docs/smartui-mcp-server/)** Debug visual regressions. Returns natural-language summaries of pixel, layout, DOM, and perceptual differences from any SmartUI comparison run. -4. **[Accessibility Tool:](/support/docs/accessibility-mcp-server/)** Run accessibility checks on hosted URLs or local React apps and get detailed remediation guidance. -5. **[Test Manager Tool:](/support/docs/test-manager-mcp-server/)** Run the complete test management lifecycle. Generate test cases with AI, build configured test runs, record results in bulk, and track milestones and coverage. - -## Connecting to TestMu AI MCP Server - -TestMu AI MCP Server works with any MCP-compatible AI client. Pick your client below, paste the configuration, and complete the OAuth flow with your TestMu AI account. - -- **Server URL:** `https://mcp.lambdatest.com/mcp` -- **Authentication:** OAuth (you will be redirected to testmuai.com on first use) - -### Cursor - -Use the one-click install button to add the server to Cursor: - -**Or configure manually:** - -1. Open **Cursor Settings > Tools & MCP** and click **Add new MCP server**. -2. Paste the configuration: - -```json -{ -"mcpServers": { -"mcp-lambdatest": { -"url": "https://mcp.lambdatest.com/mcp" -} -} -} -``` - -3. Save and restart Cursor. Complete the OAuth flow when prompted. - -**Project-level configuration:** Create `.cursor/mcp.json` in your project root to share with your team. - -### Claude Code - -Run this in your terminal: - -```shell -claude mcp add --transport http mcp-lambdatest https://mcp.lambdatest.com/mcp -``` - -Then run `/mcp` inside Claude Code, select **mcp-lambdatest**, and complete the OAuth flow. - -**Scope flags:** `--scope local` (default, current project), `--scope project` (shared via `.mcp.json`), `--scope user` (all projects). - -### Claude Desktop - -1. Open **Settings > Connectors**. -2. Click **Add Custom Connector** and enter the URL: - -``` -https://mcp.lambdatest.com/mcp -``` - -3. Complete the OAuth flow to connect your TestMu AI account. - -Custom connectors in Claude Desktop are configured through **Settings > Connectors**, not the `claude_desktop_config.json` file. Custom connectors are available on Pro, Max, Team, and Enterprise plans. - -### Claude.ai (Web) - -1. Go to [claude.ai/settings/connectors](https://claude.ai/settings/connectors). -2. Click **Add custom connector** and enter: - - **Name:** TestMu AI - - **URL:** `https://mcp.lambdatest.com/mcp` -3. Click **Add**, then **Connect** and complete the OAuth flow. -4. To use the connector in a chat, click the **+** button in the chat input and enable **TestMu AI** under **Connectors**. - -### GitHub Copilot (VS Code) - -1. Open the Command Palette (`Cmd+Shift+P` / `Ctrl+Shift+P`) and run **MCP: Add Server**, choosing **Workspace** or **Global**. Or create `.vscode/mcp.json` manually: - -```json -{ -"servers": { -"mcp-lambdatest": { -"type": "http", -"url": "https://mcp.lambdatest.com/mcp" -} -} -} -``` - -2. When prompted, confirm that you trust the server. -3. Complete the OAuth flow when first invoked. Use **MCP: List Servers** to manage installed servers. - -### Antigravity - -1. In the Agent panel, click the **...** menu, then **MCP Servers**, then **Manage MCP Servers**. -2. Click **View raw config** and add the configuration to `mcp_config.json`: - -```json -{ -"mcpServers": { -"mcp-lambdatest": { -"serverUrl": "https://mcp.lambdatest.com/mcp" -} -} -} -``` - -3. Save and refresh the MCP servers panel. Complete the OAuth flow when prompted. - -Antigravity has a global limit of around 100 active MCP tools across all servers. If you are hitting that limit, disable tools you do not need from the MCP settings panel. - -### OpenAI Codex CLI - -Run this in your terminal: - -```shell -codex mcp add mcp-lambdatest --url https://mcp.lambdatest.com/mcp -``` - -Then start an OAuth login: - -```shell -codex mcp login mcp-lambdatest -``` - -Verify with `codex mcp list`. - -**Manual config:** Add the following to `~/.codex/config.toml`: - -```toml -[mcp_servers.mcp-lambdatest] -url = "https://mcp.lambdatest.com/mcp" -``` - -### Cline (VS Code) - -1. Open the **Cline panel** in VS Code, then **MCP Servers > Configure MCP Servers**. -2. Paste the configuration: - -```json -{ -"mcpServers": { -"mcp-lambdatest": { -"disabled": false, -"timeout": 60, -"command": "npx", -"args": ["mcp-remote@latest", "https://mcp.lambdatest.com/mcp"], -"alwaysAllow": [] -} -} -} -``` - -3. Click **Authenticate** and complete the OAuth flow. - -### Continue - -1. Create `.continue/mcpServers/testmu-ai.yaml` in your workspace and add: - -```yaml -name: TestMu AI mcpServer -version: 0.0.1 -schema: v1 -mcpServers: -- name: TestMu AI -type: streamable-http -url: https://mcp.lambdatest.com/mcp -``` - -2. Switch to **Agent Mode** in Continue and complete the OAuth flow when prompted. - -MCP servers in Continue work in Agent Mode only. - -### Other Tools - -If your MCP client is not listed above, use one of these connection methods: - -| Transport | URL | Notes | -|---|---|---| -| **Streamable HTTP** (recommended) | `https://mcp.lambdatest.com/mcp` | Modern transport, widely supported | -| **STDIO** (via `mcp-remote`) | See below | Fallback for clients that only support stdio | - -**Universal STDIO configuration** for any client that accepts the standard `mcpServers` format: - -```json -{ -"mcpServers": { -"mcp-lambdatest": { -"command": "npx", -"args": ["-y", "mcp-remote@latest", "https://mcp.lambdatest.com/mcp"] -} -} -} -``` - -## A Typical Workflow - -Once connected, open your project in your AI-enabled IDE and use natural language to invoke tools: - -1. Open your project in your AI-enabled IDE. -2. Open the chat panel of your MCP client. -3. Type: `Analyze the project and create a TestMu AI YAML file using the TestMu AI MCP Server.` - -Your client will: - -- Analyze your project structure -- Identify your testing framework and test files -- Generate appropriate test runner commands -- Create a complete TestMu AI YAML configuration tailored to your project -- Save the YAML file to your project directory - -You can also ask questions like: - -- "What does failFast in HyperExecute mean?" -- "Triage the last failed test on TestID `XXXXX`." -- "Run a SmartUI analysis on comparisonId `YYYYY`." -- "Run an accessibility report on `https://example.com`." -- "Generate test cases from this requirement into my Regression folder in Test Manager." - -## Troubleshooting - -**OAuth flow does not open or fails** - -- Make sure pop-ups are not blocked in your default browser. -- If you have previously connected and the connection has expired, look for a **Disconnect** or **Clear authentication** option in your client's MCP settings, then reconnect. -- Verify your TestMu AI account has the necessary permissions for the workspace you are trying to access. - -**Tools not appearing after configuration** - -- Fully restart your client (close and reopen, not just reload the window). -- For Cursor, Claude Desktop, and Antigravity, this means quitting from the Dock or Taskbar. -- For VS Code, Cline, Continue, and Zed, reload the window via the Command Palette. -- For Continue, confirm you are in **Agent Mode**. -- For Antigravity, check that you have not exceeded the active tool limit across all MCP servers. -- Verify the configuration JSON, TOML, or YAML is valid (no trailing commas, matching braces, correct indentation). - -**Command not found: npx errors** - -This usually means your MCP client cannot find Node.js on its PATH. Use the full path to `npx`: - -```shell -which npx # macOS/Linux -where npx # Windows -``` - -Then replace `"command": "npx"` in your config with the full path returned (for example, `/usr/local/bin/npx`). diff --git a/static/docs/testmu-public-ip.md b/static/docs/testmu-public-ip.md deleted file mode 100644 index 5f1357a18..000000000 --- a/static/docs/testmu-public-ip.md +++ /dev/null @@ -1,90 +0,0 @@ -# TestMu AI Public IP Ranges - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -**Note:** The IP addresses listed below are static Public IPs. If your application or network is protected by IP-based firewall rules, you can whitelist the relevant TestMu AI IP ranges to allow TestMu AI cloud devices to access your environment. - -These are the list of the IP ranges that you need to whitelist with respect to the specific TestMu AI products. - -| Subnet Range | Standard Grid and Real Time | Real Device | KaneAI | -|--------------|-----------------------------|-------------|--------| -|**103.231.42.40/29**| ✅ | ✅ | ❌ | -|**103.231.79.40/29**| ✅ | ✅ | ❌ | -|**23.105.12.32/27**| ✅ | ❌ | ❌ | -|**23.106.34.192/26**| ✅ | ❌ | ❌ | -|**209.58.137.40/29**| ❌ | ✅ | ❌ | -|**23.83.156.64/26**| ✅ | ✅ | ❌ | -|**23.105.162.225**| ✅ | ❌ | ❌ | -|**23.105.174.26**| ✅ | ❌ | ❌ | -|**199.7.167.8/29**| ✅ | ❌ | ❌ | -|**208.78.110.56/29**| ✅ | ❌ | ❌ | -|**207.254.54.32/29**| ✅ | ❌ | ❌ | -|**207.254.53.0/29**| ✅ | ❌ | ❌ | -|**207.254.30.104/29**| ✅ | ❌ | ❌ | -|**217.112.145.88/29**| ✅ | ✅ | ❌ | -|**149.6.5.8/29**| ✅ | ✅ | ❌ | -|**3.221.56.233**| ❌ | ✅ | ❌ | -|**3.72.174.80**| ❌ | ✅ | ❌ | -|**3.73.105.110/32**| ❌ | ✅ | ❌ | -|**3.72.144.221/32**| ❌ | ✅ | ❌ | -|**18.235.85.58/32**| ❌ | ✅ | ❌ | -|**44.207.198.148/32**| ❌ | ✅ | ❌ | -|**3.109.252.59**| ❌ | ✅ | ❌ | -|**43.205.182.101**| ❌ | ✅ | ❌ | -|**18.138.79.89**| ❌ | ✅ | ❌ | -|**54.254.173.86**| ❌ | ✅ | ❌ | -|**52.72.255.172**| ✅ | ✅ | ❌ | -|**13.126.232.213**| ✅ | ✅ | ❌ | -|**34.246.27.205**| ✅ | ✅ | ❌ | -|**3.222.169.4**| ✅ | ✅ | ❌ | -|**43.204.134.9**| ✅ | ✅ | ❌ | -|**54.228.155.35**| ✅ | ✅ | ❌ | -|**54.225.186.4**| ✅ | ✅ | ❌ | -|**52.71.149.142**| ✅ | ✅ | ❌ | -|**44.238.12.62**| ✅ | ✅ | ❌ | -|**3.111.139.20**| ✅ | ✅ | ❌ | -|**54.255.17.88**| ✅ | ✅ | ❌ | -|**3.64.247.89**| ✅ | ✅ | ❌ | -|**23.106.127.152/29**| ✅ | ❌ | ❌ | -|**4.247.183.49**| ✅ | ❌ | ❌ | -|**20.234.234.192**| ✅ | ❌ | ❌ | -|**137.174.189.0/26**| ✅ | ✅ | ❌ | -|**104.45.207.170**| ✅ | ❌ | ❌ | -|**4.224.6.32/28**| ❌ | ❌ | ✅ | -|**20.253.69.48/28**| ❌ | ❌ | ✅ | -|**20.242.202.160/28**| ❌ | ❌ | ✅ | -|**20.102.24.108/31**| ❌ | ❌ | ✅ | -|**4.224.143.208/28**| ❌ | ❌ | ✅ | -|**20.204.184.92/31**| ❌ | ❌ | ✅ | -|**52.224.184.10/31**| ❌ | ❌ | ✅ | -|**20.204.224.62/31**| ❌ | ❌ | ✅ | -|**172.199.71.240/28**| ❌ | ❌ | ✅ | - -## Tunnel IP ranges - -- **199.58.84.59** -- **23.82.88.184** -- **23.106.34.219** -- **23.106.54.77** -- **3.214.241.254** -- **52.36.84.247** -- **13.126.37.58** -- **3.66.78.89** -- **20.106.137.134** -- **20.28.8.59** -- **20.235.62.15** -- **50.85.251.115** -- **20.171.1.172** - -## TestMu AI Integration Tools -- **3.208.126.109** - -- If you prefer not to whitelist the product-specific public IP ranges listed below, consider using a **Dedicated Proxy**. - -- It provides a unique static IP exclusively for your organization, making firewall management, auditing, and access control simpler. This is especially useful for organizations with strict security or compliance requirements that want to whitelist a single dedicated IP instead of multiple product-specific IP ranges. - -- For more information, refer to the documentation links provided below. - -## Additional Links -- [Network Whitelisting Guide](/support/docs/network-whitelisting-and-tunnel-guide/#2-when-dedicated-proxy-whitelisting-has-to-be-done/) -- [Dedicated Proxy - IP Whitelisting](/support/docs/dedicated-proxy/) diff --git a/static/docs/testmu-running-your-first-selenium-test.md b/static/docs/testmu-running-your-first-selenium-test.md deleted file mode 100644 index 263bbe6dc..000000000 --- a/static/docs/testmu-running-your-first-selenium-test.md +++ /dev/null @@ -1,95 +0,0 @@ -# Run Your First Selenium Test - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This guide walks you through running a sample Selenium test on the TestMu AI cloud grid using Java and TestNG. Clone a sample project, set your credentials, and execute a test. - -**Sample repo** -All the code used in this guide is available in the sample repository. Clone it to get started. - - View on GitHub - -## Prerequisites - -Make sure you have the following set up before you start. - -1. [Create a TestMu AI account](https://www.testmuai.com/register/) if you don't have one. -2. Get your **Username** and **Access Key** from the [TestMu AI Dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard). -3. Install [Java Development Kit (JDK)](https://www.oracle.com/java/technologies/downloads/) 11 or later. -4. Install [Maven](https://maven.apache.org/). - -## Step 1: Clone the Sample Project - -Pull the sample repo to your local machine and navigate into the project directory. - -```bash -git clone https://github.com/LambdaTest/java-testng-selenium -cd java-testng-selenium -``` - -## Step 2: Set Your Credentials - -Add your TestMu AI credentials as environment variables so the test can authenticate with the grid. - -Visit the [TestMu AI Dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard), navigate to the left sidebar, and select **Credentials**. Copy your **Username** and **Access Key**, then set them as environment variables: - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -## Step 3: Configure Your Test Capabilities - -Define which browser, version, and OS your test runs on. - -The sample project includes a preconfigured capabilities object that tells the grid which browser and OS to use. Here is an example: - -```java -ChromeOptions browserOptions = new ChromeOptions(); -browserOptions.setPlatformName("Windows 10"); -browserOptions.setBrowserVersion("latest"); - -HashMap ltOptions = new HashMap(); -ltOptions.put("build", "My First Test"); -ltOptions.put("name", "Sample Test - Single"); -ltOptions.put("w3c", true); - -browserOptions.setCapability("LT:Options", ltOptions); -``` - -You can customize these values to test on any browser, version, or OS. Use the [Capabilities Generator](https://www.testmuai.com/capabilities-generator/) to auto-generate the capabilities code for your target configuration. - -For the full list of supported capabilities, see [Desired Capabilities in Selenium](/support/docs/selenium-automation-capabilities/). - -## Step 4: Run the Test - -Trigger the test from your terminal using Maven. - -Execute a single test or run multiple tests in parallel: - - {`mvn test -D suite=single.xml`} - - {`mvn test -D suite=parallel.xml`} - -## Step 5: View Your Results - -Check the Automation Dashboard to see exactly what happened during your test. - -Visit the [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) to see your test results. Each session includes: - -- **Video recording** of the full test execution -- **Screenshots** captured at each step -- **Console logs** from the browser -- **Network logs** for every request and response -- **Selenium command logs** showing each driver action - -## Next Steps - -Now that your first test is running, explore what else you can do. - -- [Explore the dashboard](/support/docs/inside-testmu-platform/) - understand timelines, analytics, and automation logs -- [Configure capabilities](/support/docs/selenium-automation-capabilities/) - set browsers, versions, OS, and advanced options -- [Pick your language](/support/docs/selenium-supported-languages-and-frameworks/) - find framework-specific guides for Java, Python, JavaScript, C#, and more -- [Test locally hosted apps](/support/docs/testing-locally-hosted-pages/) - use TestMu AI Tunnel to test staging or internal sites -- [Debug your tests](/support/docs/debugging-options/) - enable extended logging and diagnostics diff --git a/static/docs/testmu-selenium-advance-capabilities.md b/static/docs/testmu-selenium-advance-capabilities.md deleted file mode 100644 index 8648959e9..000000000 --- a/static/docs/testmu-selenium-advance-capabilities.md +++ /dev/null @@ -1,19 +0,0 @@ -# TestMu AI Advanced Capabilities - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Advanced Capabilities Reference - -These capabilities extend beyond Selenium defaults and give you more control over video, screenshots, network, console logs, tunnels, and other session settings. - -| Key | Values | Capability Description | Default Value | Alias | Value Type | Example | Possible Errors | -|-----|--------|-------------------------|---------------|-------|------------|---------|-----------------| -| `video` | true, false | Records video of the complete screen while the test runs. You can record up to 10 minutes. By default, video is set to true. To turn off video recording, use the Desired Capabilities Generator or set the flag in your capabilities class. `capabilities.setCapability("video",false);` | TRUE | |Boolean |`capabilities.setCapability("video",false);` |Case Sensitive Unable to generate video Video is not playing in video player | -| `visual` | true, false | Records command-by-command screenshots at each test step. By default the flag is set to off. **Note:** 1. Test execution time increases when set to `true`.             2. This capability takes a total of 150 screenshots. After that, use the screenshot command. `capabilities.setCapability("visual",true)`;|FALSE |debug |Boolean |`capabilities.setCapability("visual",true);` |Case sensitive Unable to generate screenshot Screenshot is broken | -| `network` |true, false | Captures a recording for network packets while the test runs. By default, the network flag is set to false. **Note**: Test execution time increases when set to 'true'. `capabilities.setCapability("network",true)`; | FALSE |networkLogs |Boolean |`capabilities.setCapability("network",true);` |Case sensitive Network logs not supported| -| `console` | true, false | Records JavaScript logs for the test. By default, it is set to false. **Note:** Test execution time increases when set to 'true'. `capabilities.setCapability("console",true)`; | FALSE |Capture browser console errors |String |JavaScript logs are recorded for the test. By default, it is set to false. `capabilities.setCapability("console","disable");` |Case Sensitive No error found Not supported on this browser | -| `tunnel` | true, false | Tests your locally hosted web application with Lambda Tunnel, which establishes a TCP with TLS 1.2 secure connection between your system and the cloud servers. By default, the tunnel flag is set to false. `capabilities.setCapability("tunnel",true);` Find more about Lambda Tunnel from our documentation to test on [locally hosted web applications.](/docs/testing-locally-hosted-pages/) | FALSE |local |Boolean |`capabilities.setCapability("tunnel",true);` Find more about TestMu AI Tunnel from our documentation to test on locally hosted web applications. |Case Sensitive Tunnel not running | -| `tunnelName` | Your desired tunnel name | Provides the tunnel identifier in your test suite capability when you want to execute the test through a particular tunnel. `capabilities.setCapability("tunnelName","your desired tunnel name")`;| |localName |String |`capabilities.setCapability("tunnelName","your desired tunnel name");` |Case Sensitive Tunnel not running Tunnel Name not exists | -| `timezone` | Example: UTC+07:00 | Sets a custom timezone for executing a test. `capabilities.setCapability("timezone","UTC+07:00")`; | UTC+00:00| |String |`capabilities.setCapability("timezone","UTC+07:00");` |Case sensitive Timezone not supported by OS Timezone is different | -| `verboseWebDriverLogging` | true, false | Enables detailed Selenium logs to debug issues using insights including command execution, communication with the browser, and responses. `capabilities.setCapability("verboseWebDriverLogging", true)` | FALSE | | Boolean | `capabilities.setCapability("verboseWebDriverLogging", true)` | -| `network.mask` | true, false | Masks network traffic during automated testing. Use this capability when you need secure or anonymized network interactions during test execution - for example, when dealing with sensitive data, internal environments, or compliance requirements.`capabilities.setCapability("network.mask", true)` | FALSE | | Boolean | `capabilities.setCapability("network.mask", true)` | Case Sensitive | diff --git a/static/docs/testmu-sso-google.md b/static/docs/testmu-sso-google.md deleted file mode 100644 index 06f40e495..000000000 --- a/static/docs/testmu-sso-google.md +++ /dev/null @@ -1,157 +0,0 @@ -# TestMu AI SSO & Google Workspace Using SAML - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI provides seamless Single Sign-On (SSO) integration with Google Workspace, enabling your team to access TestMu AI using their existing Google credentials. This integration streamlines user management and enhances security by centralizing authentication through your Google Workspace identity provider. - -This comprehensive guide will walk you through configuring SAML-based SSO between your TestMu AI organization and Google Workspace, ensuring a smooth and secure authentication experience for your team. - -## Prerequisites - -Before configuring SSO integration, ensure you have the following: - -* **TestMu AI Enterprise Plan**: SSO functionality is available with our Enterprise subscription. Learn more about [managing your subscription](/docs/manage-subscriptions/). -* **TestMu AI Organization Admin Access**: You must have administrative privileges within your TestMu AI organization to configure SSO settings. -* **Google Workspace Super Admin Access**: Administrative access to your organization's Google Workspace instance is required to create and configure SAML applications. - -## Configuration Guide - -The SSO integration process involves configuring both Google Workspace and TestMu AI. We'll guide you through each step to ensure a successful setup. - -### Step 1: Create SAML App in Google Workspace - -1. **Access Google Admin Console**: Sign in to the [Google Admin Console](https://admin.google.com/) using a user account. This level of access is required to create and configure SAML applications. - -2. **Navigate to Web and Mobile Apps**: In the Google Admin Console, navigate to **Menu** > **Apps** > **Web and mobile apps** to access the application management section. - -3. **Add Custom SAML App**: Click **Add App** and select **Add custom SAML app** to create a new SAML application for TestMu AI integration. - -4. **Configure Application Details**: - - **App name**: Enter a descriptive name such as "TestMu AI SSO" or "TestMu AI SSO App" to easily identify this application in your Google Workspace admin console. - - Click **Continue** to proceed to the next configuration step. - -5. **Retrieve Google Identity Provider Details**: On the **Google Identity Provider details** page, you'll need to collect the following information for TestMu AI configuration: - - - **SSO URL**: Copy this URL - TestMu AI will use this endpoint to initiate authentication requests - - **Entity ID**: Copy this unique identifier - this serves as Google's identity in the SAML exchange - - **Certificate**: Download the X.509 certificate file in PEM format - TestMu AI will use this to verify Google's SAML responses - - **Important**: Keep this information secure and readily accessible, as you'll need to enter these details in TestMu AI's SSO configuration. - - Click **Continue** to proceed. - -### Step 2: Create Connection in TestMu AI - -6. **Access TestMu AI SSO Configuration**: Navigate to the [TestMu AI Authentication & SSO settings](https://accounts.lambdatest.com/auth/sso) page. This dashboard provides comprehensive management of your organization's single sign-on configuration. Click **Setup SSO** to initiate the integration process. - -7. **Create New SSO Connection**: In the connection setup wizard, enter a descriptive connection name such as "Google Workspace SSO" or "TestMu AI Google SSO". This name will help you identify and manage this connection in your TestMu AI organization. Click **Add New Connection** to proceed. - -8. **Select Identity Provider**: From the list of supported identity providers, select **Other SAML Provider** to configure Google Workspace integration. - -9. **Retrieve TestMu AI Configuration Details**: TestMu AI will generate and display the configuration details required for your Google Workspace setup. These critical details include: - - **Assertion Consumer Service (ACS) URL**: The secure endpoint where Google will send SAML authentication responses - - **Entity ID**: TestMu AI's unique identifier in the SAML exchange protocol - - **X.509 Certificate**: TestMu AI's public certificate for verifying SAML response authenticity - - **Important**: Securely copy these details as you'll need to enter them in your Google Workspace admin console to complete the integration. - -Click **Continue** to proceed to the next step. - -10. **Review Attribute Mapping Requirements**: TestMu AI displays the user attributes that must be mapped from your Google Workspace directory. The essential attributes for proper user provisioning include: - - **Email Address**: User's primary email address for account identification - - **First Name**: User's given name for personalization - - **Last Name**: User's surname for complete user profiles - - **Display Name**: User's full display name (optional but recommended) - - **Note**: Document these attribute requirements as you'll need to configure the corresponding mappings in Google Workspace. Click **Continue** after reviewing the requirements. - -### Step 3: Complete Google Workspace Configuration - -11. **Return to Google Workspace Configuration**: Navigate back to the Google Admin Console tab where you were configuring the SAML application. - -12. **Configure Service Provider Details**: In the **Service Provider Details** window, enter the TestMu AI configuration details you copied in step 9: - - **ACS URL**: Paste the Assertion Consumer Service URL from TestMu AI. This secure endpoint will receive SAML authentication responses from Google. - - **Entity ID**: Enter TestMu AI's Entity ID. This unique identifier enables Google to recognize TestMu AI in the SAML exchange. - - **Start URL**: (Optional) Leave blank or enter your TestMu AI organization URL for direct user redirection after authentication. - - **Name ID Format**: Set to **"X509_SUBJECT"** - this specifies the format for the Name ID attribute - - **Name ID**: Set to **"Basic Information > Primary Email"** - this serves as the primary user identifier - -13. **Configure Attribute Mapping**: Click **Add mapping** to establish the user attribute mappings required by TestMu AI: - - **Google Directory attributes**: Select the appropriate field from the Google directory attributes dropdown - - **App attributes**: Enter the corresponding attribute name that TestMu AI expects - - Configure the following essential mappings: - - **Primary Email**: Map to `email` - - **First Name**: Map to `given_name` - - **Last Name**: Map to `family_name` - -14. **Complete Google Workspace Setup**: Click **Finish** to finalize the SAML application configuration in Google Workspace. - -15. **Enable SAML Application**: In the Google Admin Console, navigate to **Menu** > **Apps** > **Web and mobile apps**, locate the TestMu AI SAML application you created, click **User access**, then configure access for your organization: - - **On for everyone**: Enable access for all users in your organization - - **On for some organizations**: Enable access for specific organizational units - - **On for some groups**: Enable access for specific groups of users - - **Off for everyone**: Disable access (not recommended for production) - - Select your preferred access level and click **Save** to activate the application. - -### Step 4: Complete TestMu AI Integration - -16. **Return to TestMu AI Configuration**: Navigate back to the TestMu AI SSO configuration page to complete the integration by providing Google Workspace details. - -17. **Provide Google Workspace Configuration**: Enter the Google Workspace SAML configuration details you collected earlier: - - **Sign-in URL**: Enter the SSO URL copied from Google Workspace in [step 5](#step-1-create-saml-app-in-google-workspace) - - **Certificate**: Upload the X.509 certificate file (PEM format) downloaded from Google Workspace in [step 5](#step-1-create-saml-app-in-google-workspace) - - **Entity ID**: Enter the Entity ID copied from Google Workspace in [step 5](#step-1-create-saml-app-in-google-workspace) - -18. **Configure SSO Domain Restrictions**: Specify the email domains that are authorized to authenticate through this SSO connection: - - `yourcompany.com` - enables authentication for all users with @yourcompany.com email addresses - - `subsidiary.com,partner.com` - supports multiple domains (comma-separated) - -19. **Pre-Integration Verification**: Before finalizing the connection, ensure the following: - - All required configuration fields are properly completed - - The X.509 certificate file has been successfully uploaded and validated - - SSO URL and Entity ID values are accurately entered - - SSO domain restrictions align with your organization's email domains - - Configuration details are consistent between Google Workspace and TestMu AI - -20. **Finalize SSO Integration**: After verification, click **Create Connection** to complete the SSO integration and activate the authentication flow. - -## SSO Verification and Testing - -After completing the integration, verify that your SSO setup is functioning correctly by testing both authentication flows. - -### Identity Provider-Initiated (IdP) SSO Testing - -1. **Access Test Functionality**: In the Google Admin Console, navigate to **Menu** > **Apps** > **Web and mobile apps**. -2. **Select TestMu AI Application**: Locate and select the TestMu AI SAML application you configured. -3. **Execute Test Login**: Click **Test SAML login** in the top-left corner of the application details page. -4. **Verify Successful Redirect**: TestMu AI should open in a new browser tab. If the test fails, review the SAML error messages and update your configuration accordingly, then retest. - -### Service Provider-Initiated (SP) SSO Testing - -1. **Access TestMu AI SSO**: Navigate to [https://accounts.lambdatest.com/sso](https://accounts.lambdatest.com/sso) to access the TestMu AI SSO login page. -2. **Enter Email Address**: Enter your Google Workspace email address in the email field. -3. **Initiate SSO**: Click **CONTINUE WITH SSO** to begin the authentication process. -4. **Complete Authentication**: You'll be redirected to Google's sign-in page. Enter your Google Workspace credentials. -5. **Verify Successful Authentication**: After successful authentication, you should be automatically redirected to your TestMu AI organization. - -## Troubleshooting Common Issues - -If you encounter issues with your SSO integration, the following troubleshooting guide will help you resolve common problems: - -### Application Not Enabled Error - -If you receive an "App not enabled" error, the SAML application hasn't been properly activated for your users in Google Workspace. - -**Resolution**: Return to the Google Admin Console, navigate to your SAML application, and ensure it's enabled for the appropriate users or organizational units. - -### Common Configuration Issues - -1. **Certificate Mismatch**: Verify that the X.509 certificate uploaded to TestMu AI exactly matches the certificate from Google Workspace -2. **URL Configuration**: Ensure the ACS URL and Entity ID are accurately copied between both platforms -3. **Attribute Mapping**: Confirm that attribute names match precisely between Google Workspace and TestMu AI -4. **Domain Restrictions**: Verify that SSO domains in TestMu AI correspond to your organization's email domains -5. **User Permissions**: Ensure users have appropriate access to the SAML application in Google Workspace - -> **Integration Complete!** Your TestMu AI organization is now successfully integrated with Google Workspace SSO. Your team can now access TestMu AI using their Google Workspace credentials. If you encounter any issues or need assistance, our **Support Team** is here to help. You can also reach us at [support@testmuai.com](mailto:support@testmuai.com). Happy Testing! diff --git a/static/docs/testmu-sso-manage-connection.md b/static/docs/testmu-sso-manage-connection.md deleted file mode 100644 index 5026dc5e2..000000000 --- a/static/docs/testmu-sso-manage-connection.md +++ /dev/null @@ -1,20 +0,0 @@ -# Manage SSO Connection - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Before proceeding you need to navigate to SSO Section in the **Security** tab of **Organization Settings**. - -### Reconfigure SSO Connection -1. For Updating the SSO connection, click on **Reconfigure**. - -2. Update the fields you need to change and click on **Update Connection**. - -### Enforce SSO Login (Forcing Users to Log in with SSO Only) -1. For Enforcing SSO login, click on **Enforce SSO Login** Toggle. - -### Exempt Users From SSO Login -1. For Exempting users from SSO login (Allows Users to Log in with TestMu AI Credentials and Social Auth), click on **SSO User Settings**. - -2. Check the toggle for the users you want to exempt from SSO login - -That's all you need to know for Managing TestMu AI SSO Integration. In case you have any questions please feel free to reach out to us via the **24/7 chat support** or email us over [support@testmuai.com](mailto:support@testmuai.com). diff --git a/static/docs/testmu-sso-oidc.md b/static/docs/testmu-sso-oidc.md deleted file mode 100644 index 514a26c43..000000000 --- a/static/docs/testmu-sso-oidc.md +++ /dev/null @@ -1,188 +0,0 @@ -# TestMu AI SSO Integration Using OIDC - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI provides seamless Single Sign-On (SSO) integration with various identity providers using OpenID Connect (OIDC), enabling your team to access TestMu AI using their existing organizational credentials. This managed SSO solution streamlines user management and enhances security by centralizing authentication through your identity provider. - -This comprehensive guide will walk you through configuring OIDC-based SSO between your TestMu AI organization and your identity provider, using Azure AD as the example throughout the documentation. - -## What is OIDC (OpenID Connect)? - -OpenID Connect (OIDC) is a simple identity layer built on top of the OAuth 2.0 protocol. It allows clients to verify the identity of the end-user based on the authentication performed by an authorization server, as well as to obtain basic profile information about the end-user in an interoperable and REST-like manner. - -**Key Benefits of OIDC:** -- **Simplified Integration**: Easier to implement compared to SAML -- **Modern Standard**: Built on OAuth 2.0, widely adopted -- **RESTful**: Uses JSON and HTTP, making it developer-friendly -- **Flexible**: Supports various authentication flows -- **Secure**: Industry-standard security practices - -## Prerequisites - -To enable TestMu AI SSO through OIDC integration, you will need: - -- **Identity Provider Access**: Administrative access to your identity provider -- **TestMu AI Enterprise Plan**: An active TestMu AI organization subscription -- **Administrative Access**: Admin-level access to your TestMu AI organization -- **SSO Request**: Contact TestMu AI support to enable SSO for your organization - -> **Note**: If you don't have SSO enabled for your TestMu AI organization, please contact our support team at [support@testmuai.com](mailto:support@testmuai.com) -. - -## Step-by-Step Integration Guide - -### Step 1: Create Application in Your Identity Provider - -The exact steps vary by identity provider, but the general process involves creating a new application or client registration. We'll use Azure AD as our example: -1. **Access Azure Portal**: Sign in to the [Azure Portal](https://portal.azure.com/) using an administrator account. - -2. **Navigate to App Registrations**: In the Azure Portal, search for "App registrations" in the search bar and select it from the results. - -3. **Create New Registration**: Click **New registration** to create a new application registration. - -4. **Configure Application Details**: - - **Name**: Enter a descriptive name such as "TestMu AI SSO" or "TestMu AI OIDC Integration" - - **Supported account types**: Select "Accounts in this organizational directory only" - - **Redirect URI**: Leave this blank for now (we'll configure it later) - - Click **Register** to create the application - -### Step 2: Configure Application Settings - -1. **Access Application Overview**: After registration, you'll be redirected to the application overview page. - -2. **Copy Client ID**: - - Note down the **Client ID** (also called Application ID in Azure AD) - you'll need this for TestMu AI configuration - - This unique identifier represents your application in your identity provider - -3. **Copy Tenant/Domain Information**: - - Note down the **Directory (tenant) ID** - this identifies your Azure AD tenant - - You'll need this for the OpenID Connect metadata document URL - -### Step 3: Create Client Secret - -1. **Navigate to Certificates & Secrets**: In your application's left sidebar, click **Certificates & secrets**. - -2. **Create New Client Secret**: - - Click **New client secret** - - **Description**: Enter a descriptive name like "TestMu AI SSO Secret" - - **Expires**: Select your preferred expiration period (recommended: 24 months) - - Click **Add** - -3. **Copy Secret Value**: - - **Important**: Copy the secret value immediately - it won't be shown again - - Store this securely as you'll need it for TestMu AI configuration - -> **Important**: Store the client secret securely as you'll need it for TestMu AI configuration. Most providers only show the secret once. - -### Step 4: Configure Redirect URI - -1. **Navigate to Authentication**: In your application's left sidebar, click **Authentication**. - -2. **Add Redirect URI**: - - Click **Add a platform** - - Select **Web** - - **Redirect URI**: Enter the redirect URI provided by TestMu AI (typically: `https://lambdatest.auth0.com/login/callback`) - - Click **Save** - -### Step 5: Collect OpenID Connect Metadata - -1. **Get Well-Known OIDC URL**: The OpenID Connect well-known configuration URL follows this pattern: -``` -https://login.microsoftonline.com/{tenant-id}/v2.0/.well-known/openid-configuration -``` - Replace `{tenant-id}` with your Directory (tenant) ID from Step 2. - - **Examples of Well-Known OIDC URLs:** - - **Azure AD**: `https://login.microsoftonline.com/12345678-1234-1234-1234-123456789012/v2.0/.well-known/openid-configuration` - - **Google Workspace**: `https://accounts.google.com/.well-known/openid-configuration` - - **Okta**: `https://yourcompany.okta.com/.well-known/openid-configuration` - - **Auth0**: `https://yourdomain.auth0.com/.well-known/openid-configuration` - -2. **Verify Well-Known URL**: You can test the URL in your browser to ensure it returns valid JSON configuration. - -> **Verify Well-Known URL**: Test the URL in your browser to ensure it returns valid JSON configuration. - -### Step 6: Contact TestMu AI Support for OIDC Configuration - -Since OIDC integration requires server-side configuration, you cannot create the connection directly through the TestMu AI interface. Instead, you need to contact our support team to configure the OIDC integration for you. - -1. **Gather Required Information**: Collect the following details from your identity provider configuration: - - **Client ID**: The client ID from Step 2 - - **Client Secret**: The secret value from Step 3 - - **OpenID Connect Metadata URL**: The metadata document URL from Step 5 - - **Connection Name**: A descriptive name like "Company OIDC" or "TestMu AI SSO" - -2. **Contact TestMu AI Support**: Reach out to our support team with the configuration details: - - **Email**: [support@testmuai.com](mailto:support@testmuai.com) - -3. **Provide Configuration Details**: Include the following information in your support request: -``` -Identity Provider: Azure AD -Client ID: [Your Application (client) ID] -Client Secret: [Your Client Secret] -Well-Known OIDC URL: [Your OIDC Well-Known URL] -Connection Name: [Unique Connection Name] -``` - -4. **Wait for Configuration**: Our support team will configure the OIDC integration on your behalf. - -### Step 7: Configure User Access (Optional) - -1. **Set User Assignment**: In Azure AD, navigate to your application and click **Users and groups**. - -2. **Assign Users**: - - Click **Add user/group** - - Select the users or groups who should have access to TestMu AI - - Click **Assign** - -3. **Configure App Roles** (if needed): You can define custom app roles for different access levels within TestMu AI. - -## Testing Your OIDC Integration - -### Method 1: Direct SSO Login - -1. **Access TestMu AI SSO**: Navigate to [https://accounts.lambdatest.com/sso](https://accounts.lambdatest.com/sso) - -2. **Enter Email**: Enter your Azure AD email address - -3. **Initiate SSO**: Click **CONTINUE WITH SSO** - -4. **Complete Authentication**: You'll be redirected to Microsoft's login page. Enter your Azure AD credentials. - -5. **Verify Success**: After successful authentication, you should be redirected to your TestMu AI organization. - -### Method 2: Direct OIDC Login (For Testing) - -Since OIDC doesn't support classic IDP-based SSO login, you can use this URL to simulate the login process: - -``` -https://accounts.lambdatest.com/auth0/login?connection={connection-name} -``` - -Replace `{connection-name}` with the connection name you created in TestMu AI. - -## Troubleshooting Common Issues - -### Application Not Found -- **Issue**: "Application not found" error during login -- **Solution**: Verify that the Application (Client) ID is correct and the application is properly registered in Azure AD - -### Invalid Client Secret -- **Issue**: "Invalid client secret" error -- **Solution**: Ensure the client secret is correctly copied and hasn't expired. Create a new secret if needed. - -### Redirect URI Mismatch -- **Issue**: "Redirect URI mismatch" error -- **Solution**: Verify that the redirect URI in Azure AD exactly matches the one provided by TestMu AI - -### User Not Assigned -- **Issue**: "User not assigned to application" error -- **Solution**: Ensure the user is assigned to the application in Azure AD under Users and groups - -### Well-Known URL Issues -- **Issue**: "Invalid well-known URL" error -- **Solution**: Verify the OpenID Connect well-known URL is accessible and contains valid JSON configuration - -### Domain Configuration -- **Issue**: Users from different domains cannot access TestMu AI -- **Solution**: Ensure all required domains are configured in both Azure AD and TestMu AI settings diff --git a/static/docs/testmu-sso-okta-integration.md b/static/docs/testmu-sso-okta-integration.md deleted file mode 100644 index 566d47d9d..000000000 --- a/static/docs/testmu-sso-okta-integration.md +++ /dev/null @@ -1,60 +0,0 @@ -# TestMu AI SSO & Okta Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI allows you to integrate an external user directory with your TestMu AI Organization through an IDP so your team could leverage the Single Sign-on setup. With Okta, you can integrate your TestMu AI organization in two ways i.e. through the Dev Console or the SAML. - -After you integrate your Okta instance with your TestMu AI for Single Sign-on, you can benefit from the following features: -* SP-initiated SSO. -* IdP-initiated SSO. - -To know more, refer to Okta's glossary. - -## Prerequisites -To enable TestMu AI SSO through Okta integration: - -* You will need an Enterprise plan with TestMu AI. -* The Admin of your TestMu AI organization needs to raise a request for enabling SSO Self Serve UI. They could drop us an email over support@testmuai.com or could reach out to us via our **24/7 in app chat support**. -* You will also need the admin level access of Okta to configure the SSO. - -## Configuring TestMu AI & Okta Integration Using SAML - -1. Go to TestMu AI Authentication & SSO settings and click on **Setup SSO**. - -2. Enter Connection Name and Click on **Add New Connection**. - -3. Select **Okta** as your identity provider. - -4. Now copy the details required for setting up SSO connection in your **Identity Provider**. - Once configured, click on continue - -5. Now Configure the Attributes and Claims in your **Identity Provider**. - Once configured, click on continue - -6. Login to your okta account. - -7. Go to Application menu from the toolbar and click on **Create App Integration**. - -8. Select the **SAML 2.0** option and click on **Next**. - -9. Give the created app a name of your choice, in the **App Name** box, and click Next. - -10. Fill in the details as per in step 4 and step 5 and then click Next. - -12. Set the Attribute Statements as per the image below and click Next. - -13. In the next window of this Okta SAML integration, choose App Type as **This is an internal app that we have created**, as shown in the image below, and then click on **Finish** to complete the app creation. - -14. Once the app is created, you will be redirected to the app's general settings. Click on the **Sign On** tab. Copy the Metadata URL and paste it in the TestMu AI SSO setup page. - -15. Fill the required fields and click on **Create Connection**: - -* SSO Domains (Comma-separated list of the domains that can be authenticated in the Identity Provider.) -* Have Metadata URL (Select this option if you have metadata file URL of your Identity Provider.) -* Metadata URL (Enter the metadata file URL of your Identity Provider.) -* Sign In URL (Enter the sign-in URL of your Identity Provider.) -* X509 Certificate (Upload the base64 encoded X509 certificate of your Identity Provider.) - -Create Connection with Metadata URL - -> That's all you need to know for Configuring TestMu AI SSO Integration with Okta. In case you have any questions please feel free to reach out to us via the **24/7 chat support** or email us over [support@testmuai.com](mailto:support@testmuai.com). diff --git a/static/docs/testmu-sso-onelogin-using-saml.md b/static/docs/testmu-sso-onelogin-using-saml.md deleted file mode 100644 index 932e9d154..000000000 --- a/static/docs/testmu-sso-onelogin-using-saml.md +++ /dev/null @@ -1,53 +0,0 @@ -# TestMu AI SSO & OneLogin Using SAML - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI allows you to integrate an external user directory with your TestMu AI Organization through an IDP so that your team could leverage the Single Sign-on setup. This document will help you integrate your TestMu AI organization with your OneLogin identity provider to configure SSO(Single sign-on). - -## Prerequisites - -To set-up and use OneLogin & TestMu AI Single Sign-on (SSO) feature: - -* An Enterprise plan with TestMu AI. ( Learn how to [manage your subscriptions](/docs/manage-subscriptions/). ) -* The Admin level access to your TestMu AI organization. -* The Admin level access of your organization’s OneLogin instance - -## Configure TestMu AI SSO & OneLogin Integration - -1. Go to TestMu AI Authentication & SSO settings and click on **Setup SSO**. - -2. Enter Connection Name and Click on **Add New Connection**. - -3. Select **OneLogin** as your identity provider. - -4. Now copy the details required for setting up SSO connection in your **Identity Provider**. - Once configured, click on continue - -5. Now Configure the Attributes and Claims in your **Identity Provider**. - Once configured, click on continue - -6. Log in to the [OneLogin Dashboard](https://app.onelogin.com/login). - -7. Go to Applications - -8. Add a new Application, by clicking on Add App. - -9. Search for SAML Test Connector and select **Add SAML Test Connector (Advanced)** - -10. In the window that appears, choose a suitable name for your app, and **save**. - -11. Click on configurations tab in the sidebar and fill the details as per the details copied from TestMu AI in step 4. - -12. Click on the parameters tab and fill the details as per the details copied from TestMu AI in step 5. - -13. Now go to the SSO tab set the SAML Signature Algorithm to SHA-256 - -14. Copy the Issuer URL - -15. Go back to TestMu AI and paste the copied Issuer URL in the **Metadata URL** field. Fill the required fields and click on **Create Connection**: - -* SSO Domains (Comma-separated list of the domains that can be authenticated in the Identity Provider.) -* Have Metadata URL (Select this option if you have metadata file URL of your Identity Provider.) -* Metadata URL (Enter the metadata URL of your Identity Provider.) - -> That's it. Your enterprise's TestMu AI SSO and OneLogin has been integrated successfully. If you face any challenge, or come across any issues, please feel free to contact our **Support Team**. You can even mail us at [support@testmuai.com](mailto:support@testmuai.com). Happy Testing! diff --git a/static/docs/testmu-sso-pingidentity-integration.md b/static/docs/testmu-sso-pingidentity-integration.md deleted file mode 100644 index f9c678ad2..000000000 --- a/static/docs/testmu-sso-pingidentity-integration.md +++ /dev/null @@ -1,57 +0,0 @@ -# TestMu AI SSO & PingIdentity Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI provides seamless integration with Ping Identity through SAML (Security Assertion Markup Language) to enable Single Sign-On (SSO) for your organization. By integrating Ping Identity with your TestMu AI organization, you can streamline user authentication and enhance security. - -## Key Features of Ping Identity Integration: -- **SP-Initiated SSO :** Single Sign-On initiated from the TestMu AI platform. -- **IdP-Initiated SSO :** Single Sign-On initiated from Ping Identity. - -## Prerequisites -Before configuring the integration, ensure you meet the following requirements: - -- **TestMu AI Enterprise Plan :** Your organization must have an active Enterprise plan. -- **SSO Enablement Request :** The admin of your TestMu AI organization must request SSO enablement. -> Raise a request via email at support@testmuai.com or through TestMu AI’s 24/7 in-app chat support. -- **Ping Identity Admin Access :** You need administrative access to your Ping Identity account. - -## Configuring TestMu AI & Ping Identity Integration -Follow the steps below to configure the SAML-based integration between TestMu AI and Ping Identity. - -### Step 1: Log in to Ping Identity -Log in to your Ping Identity account using admin credentials. - -### Step 2: Setup SSO in TestMu AI - -- Go to TestMu AI Authentication & SSO settings and click on **Setup SSO**. - -- Enter Connection Name and Click on **Add New Connection**. - -- Select **PingFederate** as your identity provider (PingIdentity & PingFederate has identical setup from TestMu AI side). - -### Step 3: Add a New Application in PingIdentity -- Navigate to the Applications menu on the toolbar. -- Click on Add Application. - -- In the Add Application window, select Create New App to create a new application for TestMu AI SSO integration. - -### Step 4: Add SAML configuration -- In TestMu AI SSO setup copy the **ACS URLs** and **Entity ID**. - -- In Pingidentity, select the **Manually Enter** option in **SAML Configuration** and enter **ACS URLs** and **Entity ID** copied from above step. - -### Step 5: Configure Attribute Mappings -- Copy following attribute from Lambdatest SSO setup. - -- Go to the Attribute Mappings tab of the created application in PingIdentity. -- Update the saml_object attribute value from user_ID to emailaddress. -- Add and update the other attributes - -### Step 6 : Upload Metadata URL & SSO Domain -- SSO Domains (Comma-separated list of the domains that can be authenticated in the Identity Provider.) -- Copy the **IDP Metadata URL** from PingIdentity - -- Go back to TestMu AI and paste the copied URL in the **Issuer URL** field and click on **Create Connection**: - -After completing these configurations, your TestMu AI and Ping Identity integration for SSO will be ready to use. Both SP-initiated and IdP-initiated SSO flows will be enabled, and you can manage users and groups via Ping Identity. diff --git a/static/docs/testmu-sso-self-serve.md b/static/docs/testmu-sso-self-serve.md deleted file mode 100644 index 712021cfd..000000000 --- a/static/docs/testmu-sso-self-serve.md +++ /dev/null @@ -1,48 +0,0 @@ -# Create SSO Connection With TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI allows you to integrate an external user directory with your TestMu AI Organization through an IDP so your team could leverage the Single Sign-on setup using SAML. - -TestMu AI SSO supports the following: -* SP-initiated SSO. -* IdP-initiated SSO. -* JIT (Just In Time) Provisioning. - -## Prerequisites ## - -Create SSO Connection With TestMu AI: - -* You will need an Enterprise plan with TestMu AI. -* The Admin of your TestMu AI organization needs to raise a request for enabling SSO Self Serve UI. They could drop us an email over support@testmuai.com or could reach out to us via our **24/7 in app chat support**. -* You will also need the admin level access of OneLogin to configure the SSO. - -## Configuring TestMu AI SSO Integration Using SAML - -1. Go to TestMu AI SSO Section in the **Security** tab of **Organization Settings** and click on **Setup SSO**. - -2. Enter Connection Name and Click on **Add New Connection**. - -3. Select your **Identity Provider**. - -4. Now copy the details required for setting up SSO connection in your **Identity Provider**. -Once configured, click on continue - -5. Now Configure the Attributes and Claims in your **Identity Provider**. -Once configured, click on continue - -6. Fill the required fields and click on **Create Connection**: - -* SSO Domains (Comma-separated list of the domains that can be authenticated in the Identity Provider.) -* Have Metadata URL (Select this option if you have metadata file URL of your Identity Provider.) -* Metadata URL (Enter the metadata URL of your Identity Provider.) -* Sign In URL (Enter the sign-in URL of your Identity Provider.) -* X509 Certificate (Upload the base64 encoded X509 certificate of your Identity Provider.) - -Create Connection with Metadata URL - -Create Connection without Metadata URL - -7. Once the connection is created, you will be redirected to the SSO settings page. From here you can Manage SSO Connection. - -That's all you need to know for Configuring TestMu AI SSO Integration Using SAML. In case you have any questions please feel free to reach out to us via the **24/7 chat support** or email us over [support@testmuai.com](mailto:support@testmuai.com). diff --git a/static/docs/testmu-tunnel.md b/static/docs/testmu-tunnel.md deleted file mode 100644 index 7ab01d271..000000000 --- a/static/docs/testmu-tunnel.md +++ /dev/null @@ -1,55 +0,0 @@ -# Getting started with TestMu AI-Tunnel - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI Tunnel is a secure and encrypted tunneling feature that allows developers and QA professionals to test their locally hosted web applications or websites on TestMu AI's cloud-based real machines. It establishes a secure connection between the user’s local machine and the real machine in the cloud, enabling testing of internal applications on various browsers and devices. - -## Getting Started -| Title | Description| -|-------|------------| -| [Testing Locally Hosted Pages](/support/docs/deep-dive-into-hyperexecute-yaml/#version) | Learn how to securely test websites hosted on your local machine using TestMu AI Tunnel. | -| [Local Testing for Linux](/support/docs/local-testing-linux/) | Setup instructions for running the tunnel on Linux-based systems. | -| [Local Testing for Windows](/support/docs/local-testing-windows/) | Step-by-step guide for setting up TestMu AI Tunnel on Windows OS.| -| [Local Testing for macOS](/support/docs/local-testing-macos/) | Step-by-step instructions for running the tunnel on macOS. | -| [Real-Time Locally Hosted Pages](/support/docs/real-time-locally-hosted-pages/) | Perform interactive, real-time testing of locally hosted sites.| -| [Testing Locally Hosted Apps for App Automation](/support/docs/testing-locally-hosted-apps/) | Use Tunnel to test private apps during mobile automation. | - -## Configuration and Modifiers -| Title | Description| -|-------|------------| -| [Tunnel Modifiers](/support/docs/lambda-tunnel-modifiers/) | Detailed list of command-line arguments and modifiers to customize TestMu AI Tunnel behavior. | -| [Advanced Tunnel Features](/support/docs/advanced-tunnel-features/) | Explore advanced configurations and features available in TestMu AI Tunnel.| -| [Supported Environment Variables](/support/docs/environment-variables-supported-by-lambda-tunnel/) | Set up environment variables to manage TestMu AI Tunnel settings effectively.| - -## GUI Application: Underpass -| Title | Description| -|-------|------------| -| [Underpass Tunnel Application](/support/docs/lambda-tunnel-modifiers/) | Use TestMu AI’s graphical tool to establish secure tunnels without CLI. | - -## Integration and Automation -| Title | Description| -|-------|------------| -| [Tunnel with HyperExecute](/support/docs/hyperexecute-how-to-configure-tunnel/) | Steps to set up TestMu AI Tunnel in HyperExecute for advanced test executions.| -| [Tunnel with SmartUI SDK](/support/docs/smartui-sdk-tunnel/) | Best practices and configurations for integrating TestMu AI Tunnel with SmartUI SDK. | -| [Maven Tunnel Setup](/support/docs/maven-tunnel-for-locally-hosted-web-applications/) | Instructions to integrate TestMu AI Tunnel with Maven projects for Selenium testing. | -| [Tunnel with Azure Pipelines](/support/docs/integrate-testmu-extension-with-azure-pipelines/#localhost-testing-using-lambdatest-extension) | Learn how to configure TestMu AI Tunnel within Azure Pipelines for CI/CD workflows.| - -## Docker Integration -| Title | Description| -|-------|------------| -| [Localhost Testing with Docker Tunnel](/support/docs/docker-tunnel/) | Guide to using TestMu AI Tunnel within Docker containers for local testing. | -| [Docker Tunnel Extension](/support/docs/docker-tunnel-extension/) | Instructions to set up and manage TestMu AI Tunnel using the Docker Tunnel Extension. | - -## Advanced Features -| Title | Description| -|-------|------------| -| [Sharing TestMu AI Tunnel](/support/docs/sharing-lambda-tunnel/) | Share tunnel access securely with team members or systems.| -| [Load Balancing in Tunnel](/support/docs/load-balancing-in-lambda-tunnel/) | Distribute test load across multiple tunnels for reliability. | -| [Charles Proxy Setup](/support/docs/charles-proxy/) | Use Charles Proxy with Tunnel for debugging network traffic. | -| [Dedicated Proxy for IP Whitelisting](/support/docs/dedicated-proxy/) | Use a fixed IP proxy to simplify firewall rules and whitelisting.| - -## Troubleshooting -| Title | Description| -|-------|------------| -| [Troubleshooting Tunnel](/support/docs/troubleshooting-lambda-tunnel/) | Comprehensive guide to diagnosing and resolving common issues with TestMu AI Tunnel.| -| [Open File Limit Configuration](/support/docs/how-to-set-the-open-file-limit-while-using-tunnel/) | Instructions to configure system limits for optimal Tunnel performance. | diff --git a/static/docs/testng-on-hyperexecute-grid.md b/static/docs/testng-on-hyperexecute-grid.md deleted file mode 100644 index 7c69f45be..000000000 --- a/static/docs/testng-on-hyperexecute-grid.md +++ /dev/null @@ -1,202 +0,0 @@ -# Running TestNG Framework Tests on HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestNG is a widely-used testing framework for Java applications, designed to simplify and enhance the testing process for developers. It provides a flexible and powerful platform for running test suites, enabling effective unit testing, integration testing, and end-to-end testing of Java applications - -HyperExecute is an AI Native Test Orchestration Cloud Platform that empowers you to run **end-to-end** tests **quickly** and **efficiently**. It provides Just-in-Time (JIT) testing infrastructure with fast execution **speeds**, **smart orchestration**, and **detailed logs**. - -This guide details how to execute your **TestNG** framework tests on **HyperExecute** using [YAML 0.2](/support/docs/hyperexecute-yaml-version0.2/) via two different methods: - -- [**Using Local System**](/support/docs/testng-on-hyperexecute-grid/#1-testing-using-local-system) - You can use your own local machine to execute tests. -- [**Using Gitpod Platform**](/support/docs/testng-on-hyperexecute-grid/#2-testing-using-gitpod) - Execute tests using GitPod. (Requires a [Gitpod](https://gitpod.io/login/) account) - -## 1. Testing Using Local System - -Follow the step-by-step guide to execute your test on HyperExecute. - -### Prerequisites - -To run the Tests on HyperExecute from your Local System, you are required: - -- Your TestMu AI [Username and Access key](/support/docs/hyperexecute-how-to-get-my-username-and-access-key/) -- [HyperExecute YAML](/support/docs/hyperexecute-yaml-version0.2/) file which contains all the necessary instructions. -- [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) in order to initiate a test execution Job . -- Setup the [Environmental Variable](/support/docs/hyperexecute-environment-variable-setup/) - -### Step 1: Configure Your Test Suite - -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** -Download or Clone the code sample for the TestNG from the TestMu AI GitHub repository to run the tests on the HyperExecute. - - View on GitHub - -If you are using your own project, make sure you update the **Hub endpoint** in your tests file. - -By setting up the Hub endpoint, you establish the communication channel between your tests and the browser nodes, enabling effective test distribution and execution. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config hyperexecute.yaml`} - -> **NOTE :** In case of macOS, if you get a permission denied warning while executing CLI, simply run **`chmod u+x ./hyperexecute`** to allow permission. In case you get a security popup, allow it from your **System Preferences** → **Security & Privacy** → **General tab**. - -🎉 Congratulations! You've successfully triggered the TestNG tests on HyperExecute. Visit the [HyperExecute Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) and check your Job status. - -## Configure your own Project - -### Step 1: Download the CLI in your Project -Download the HyperExecute CLI and copy it into the root folder . - -| Platform | HyperExecute CLI | -| ---------| ---------------- | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| macOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | - -### Step 2: Setup Hub Endpoints and Capabilities -Update your TestNG test configuration to point to the TestMu AI hub and set capabilities as needed: - -```java title="test.java" -DesiredCapabilities capabilities = new DesiredCapabilities(); -capabilities.setCapability("browserName", "chrome"); -capabilities.setCapability("version", "70.0"); -capabilities.setCapability("platform", "win10"); # If this cap isn't specified, it will just get the any available one -capabilities.setCapability("build", "TestMu AI SampleApp"); -capabilities.setCapability("name", "TestMu AI JavaSample"); -``` - -> You can also use **dev** and **beta** browser versions. To generate capabilities for your test requirements, you can use our inbuilt 🔗 Capabilities Generator. - -### Step 2: Setup the CLI in your Test Suite - -After cloning / downloading the sample repo, you need to setup the CLI and the environment variables. - -#### Download the HyperExecute CLI - -The CLI is used for triggering the tests on HyperExecute. It is recommend to download the CLI binary on the host system and keep it in the root directory of the suite to perform the tests on HyperExecute. - -You can download the CLI for your desired platform from the below mentioned links: - -| Platform | HyperExecute CLI | -| ---------| ---------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| MacOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -#### Setup Environment Variable - -Now, you need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in the [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). - -Run the below mentioned commands in your terminal to setup the CLI and the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -### Step 3: Configure YAML in your Test Suite - -Configure your YAML file as per your use cases using **key value** pairs. - -In this sample YAML file, we have mentioned: - -- **version** of the YAML file -- **Timeouts** for executing your project -- **Mode of execution** is [Autosplit](/support/docs/hyperexecute-auto-split-strategy/). You can also opt for [Matrix](/support/docs/hyperexecute-matrix-multiplexing-strategy/) or [Hybrid](/support/docs/hyperexecute-hybrid-strategy/) mode. -- **Pre and Post** commands -- **Reports and Artefacts** that will be generated after the completion of tests -- and other necessary YAML Parameters - -```yaml reference title="HyperExecute AutoSplit YAML" -https://github.com/LambdaTest/testng-selenium-hyperexecute-sample/blob/main/yaml/linux/v2/testng_hyperexecute_autosplit_sample.yaml -``` - -### Step 4: Execute your Test Suite - -> **NOTE :** In case of MacOS, if you get a permission denied warning while executing CLI, simply run **`chmod u+x ./hyperexecute`** to allow permission. In case you get a security popup, allow it from your **System Preferences** → **Security & Privacy** → **General tab**. - -Run the below command in your terminal at the root folder of the project: - -```bash -./hyperexecute --config RELATIVE_PATH_OF_YOUR_YAML_FILE -``` - -OR use this command if you have not exported your username and access key in the step 2. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE `} - -### Step 5: Monitor the Test Execution - -Visit the [HyperExecute Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) and check your Job status. - -### Step 6: Download Artifacts and Reports - -HyperExecute also facilitates the provision to download the [Artifacts](/support/docs/hyperexecute-artifacts/) and [Reports](/support/docs/hyperexecute-reports/) on your local machine. Click on the corresponding button to download your generated artifacts and reports. - -## 2. Testing Using Gitpod - -You can also use the Gitpod platform to execute our sample repository. It will fetch all the sample codebases and trigger the CLI to execute the tests. - -Follow the below steps to run Test using Gitpod: - -**Step 1:** Click '**Open in Gitpod**' button. You will be redirected to Login/Signup page. This button is configured to redirect you to the Gitpod platform where you will be able to execute our sample repository. - -**Step 2:** Login with TestMu AI credentials. Once logged in, a pop-up confirmation will appear, asking you to **'Proceed'** to the Gitpod editor in a new tab. The current tab will display the HyperExecute Dashboard. - -**Step 3:** Choose your preferred editor (we recommend VS Code Editor) - -**Step 4:** As you are running a sample project, Fetching of the Test Scripts, [HyperExecute YAML](/support/docs/deep-dive-into-hyperexecute-yaml/), [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) and Triggering your tests using the `Execution Command` will be automated. - -**Step 5:** Once you see the `Job Link` in the logs, you can visit the [HyperExecute dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) to see the tests getting executed. - -You can also implement [Secret Keys](/support/docs/hyperexecute-how-to-save-and-manage-secrets/) in your YAML file. - -## Navigation in Automation Dashboard - -Every test run on the HyperExecute has a unique *jobId* associated with it. Each *jobId* can in turn constitute single (or multiple) *groupId*(s). You can visit [HyperExecute Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) for checking the status of the test execution. - -You can seamlessly navigate between JobId's and taskId's. You need to click on the *testID* to navigate from the HyperExecute logs to the Automation Dashboard. - -The snapshot below shows the videos, logs and other meta data for that specific *test_ID* - -> For any query or doubt, please feel free to contact us via **24×7 chat support** or you can also drop a mail to **support@testmuai.com**. - -## Using the TestNG Agent Skill with TestMu AI - -The [testng-skill](https://github.com/LambdaTest/agent-skills/tree/main/testng-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The testng-skill package includes: - -``` -testng-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing TestNG Agent Skill - -Install a TestNG Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/testng-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/testng-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only testng-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). diff --git a/static/docs/testng-with-selenium-running-java-automation-scripts-on-testmu-selenium-grid.md b/static/docs/testng-with-selenium-running-java-automation-scripts-on-testmu-selenium-grid.md deleted file mode 100644 index e5a079ac0..000000000 --- a/static/docs/testng-with-selenium-running-java-automation-scripts-on-testmu-selenium-grid.md +++ /dev/null @@ -1,211 +0,0 @@ -# Selenium With TestNG - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Run TestNG tests on the TestMu AI cloud grid. This guide covers setup, running a sample test, configuring capabilities, and testing locally hosted pages. - -**Sample repo** -All the code used in this guide is available in the sample repository. - - View on GitHub - -## Prerequisites - -Make sure you have the following set up before you start. - -1. [Create a TestMu AI account](https://www.testmuai.com/register/) if you don't have one. -2. Get your **Username** and **Access Key** from the [TestMu AI Dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard). -3. Install the **Java development environment** (Java 11 recommended). -4. Install **Maven**. Download it from [the official website](https://maven.apache.org/) or install it on **Linux/MacOS** using [**Homebrew**](https://brew.sh/). - -## Step 1: Clone the Sample Project - -Pull the sample repo to your local machine and navigate into the project directory. - -```bash -git clone https://github.com/LambdaTest/Java-TestNG-Selenium -cd Java-TestNG-Selenium -``` - -You may also want to run the command below to check for outdated dependencies. - -```bash -mvn versions:display-dependency-updates -``` - -## Step 2: Set Your Credentials - -Add your TestMu AI credentials as environment variables so the test can authenticate with the grid. - -Visit the [TestMu AI Dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard), navigate to the left sidebar, and select **Credentials**. Copy your **Username** and **Access Key**, then set them as environment variables: - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -## Step 3: Configure Your Test Capabilities - -Define the browser, version, and OS for your test run. - -```java -ChromeOptions browserOptions = new ChromeOptions(); -browserOptions.setPlatformName("Windows 10"); -browserOptions.setBrowserVersion("latest"); - -HashMap ltOptions = new HashMap(); -ltOptions.put("build", "TestMu AISampleApp"); -ltOptions.put("name", "TestMu AIJavaSample"); -ltOptions.put("w3c", true); -browserOptions.setCapability("LT:Options", ltOptions); -``` - -Use the [Capabilities Generator](https://www.testmuai.com/capabilities-generator/) to auto-generate capabilities for any browser, version, and OS combination. - -## Step 4: Run the Test - -Trigger the test from your terminal. - -```bash -mvn test -D suite=single.xml -``` - -```bash -mvn test -D suite=parallel.xml -``` - -Below is the sample TestNG test used in this project: - -```java title="TestNG To Do" -import org.openqa.selenium.By; -import org.openqa.selenium.Platform; -import org.openqa.selenium.WebDriver; -import org.openqa.selenium.chrome.ChromeOptions; -import org.openqa.selenium.remote.RemoteWebDriver; -import org.testng.Assert; -import org.testng.annotations.AfterClass; -import org.testng.annotations.BeforeClass; -import org.testng.annotations.Test; -import java.net.MalformedURLException; -import java.net.URL; -import java.util.HashMap; -public class TestNGTodo{ -public String username = "YOUR_LAMBDATEST_USERNAME"; -public String accesskey = "YOUR_LAMBDATEST_ACCESS_KEY"; -public static RemoteWebDriver driver = null; -public String gridURL = "@hub.lambdatest.com/wd/hub"; -boolean status = false; -@BeforeClass -public void setUp() throws Exception { -ChromeOptions browserOptions = new ChromeOptions(); -browserOptions.setPlatformName("Windows 10"); -browserOptions.setBrowserVersion("latest"); - -HashMap ltOptions = new HashMap(); -ltOptions.put("build", "TestMu AISampleApp"); -ltOptions.put("name", "TestMu AIJavaSample"); -ltOptions.put("w3c", true); -browserOptions.setCapability("LT:Options", ltOptions); -try { -driver = new RemoteWebDriver(new URL("https://" + username + ":" + accesskey + gridURL), browserOptions); -} catch (MalformedURLException e) { -System.out.println("Invalid grid URL"); -} catch (Exception e) { -System.out.println(e.getMessage()); -} -} -@Test -public void testSimple() throws Exception { -try {//Change it to production page -driver.get("https://lambdatest.github.io/sample-todo-app/"); -//Let's mark done first two items in the list. -driver.findElement(By.name("li1")).click(); -driver.findElement(By.name("li2")).click(); -// Let's add an item in the list. -driver.findElement(By.id("sampletodotext")).sendKeys("Yey, Let's add it to list"); -driver.findElement(By.id("addbutton")).click(); -// Let's check that the item we added is added in the list. -String enteredText = driver.findElementByXPath("/html/body/div/div/div/ul/li[6]/span").getText(); -if (enteredText.equals("Yey, Let's add it to list")) { -status = true; -} -} catch (Exception e) { -System.out.println(e.getMessage()); -} -} -@AfterClass -public void tearDown() throws Exception { -if (driver != null) { -((JavascriptExecutor) driver).executeScript("lambda-status=" + status); -driver.quit(); -} -} -} -``` - -For parallel execution, the sample project includes a `testng.xml` that runs the test across multiple browsers simultaneously: - -```xml title="testng.xml" - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -``` - -## Step 5: View Your Results - -Check the Automation Dashboard to see exactly what happened during your test. - -Visit the [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) to see your test results. Each session includes: - -- **Video recording** of the full test execution -- **Screenshots** captured at each step -- **Console logs** from the browser -- **Network logs** for every request and response -- **Selenium command logs** showing each driver action - -## Run TestNG Tests Using Agent Skills - -Use AI coding assistants to generate and run TestNG tests with the TestMu AI Agent Skill. - -The [testng-skill](https://github.com/LambdaTest/agent-skills/tree/main/testng-skill) is part of [TestMu AI Agent Skills](https://github.com/LambdaTest/agent-skills/) - structured packages that teach AI coding assistants how to write production-grade test automation. - -Install the skill: - -```bash -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/testng-skill .claude/skills/ - -# For Cursor / Copilot -cp -r agent-skills/testng-skill .cursor/skills/ -``` - -Install all available framework skills at once by cloning the repository directly into your tool's skills directory (e.g., `.claude/skills/`, `.cursor/skills/`). diff --git a/static/docs/testrail-integration-with-testmu-selenium-grid.md b/static/docs/testrail-integration-with-testmu-selenium-grid.md deleted file mode 100644 index 15b9f1320..000000000 --- a/static/docs/testrail-integration-with-testmu-selenium-grid.md +++ /dev/null @@ -1,86 +0,0 @@ -# TestRail Integration With TestMu AI Selenium Grid - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestRail is a modern test case management tool which offers a comprehensive web-platform for software testers and developers. Offering a JIRA add-on for cloud-based integration, TestRail helps you to track, manage, and hasten software release cycle. - -TestRail also offers integration with TestMu AI to help you perform automated cross browser testing through an on-cloud Selenium Grid offering 3000+ browsers. **TestMu AI Selenium Grid integration to TestRail**  will help you run your Selenium automation scripts from TestRail instance to our cloud-based Selenium Grid. - -## Prerequisites - -1. Global Dependencies - * Make sure you have [Maven installed](https://maven.apache.org/install.html). - * You could also install Maven with Homebrew with the below command: -``` -$ install maven -``` - -2. Project Dependencies - - * Clone our [GitHub repository for Selenium & TestRail](https://github.com/LambdaTest/Java-TestNG-Selenium-TestRail). - * Ensure that the required packages are made available using the below command: -``` -$ cd Java-TestNG-Selenium-TestRail -``` - - * We recommend, you execute the below command for evaluating any outdated dependencies. As a best practice, it is always suggested to go for a verification check & perform any update, if needed, prior editing your pom.xml in order to avoid any code incompatibility. -``` -mvn versions:display-dependency-updates -``` - - * For authentication purpose, make sure you have your Testrail credentials – URL, Username and Password. - -## Integrating TestMu AI With TestRail - -**Step 1:**  Clone our [GitHub repository for Selenium & TestRail](https://github.com/LambdaTest/Java-TestNG-Selenium-TestRail). - -**Step 2:**  Open Eclipse and import the cloned project as maven project. - -**Step 3:**  Add Java Library into the project. - -**Step 4:**  Follow the Readme.md file for how to run the test. - -**Step 5:**  Once test would get run then find your result logged in Testrail for given test case and run id. - -## Running Automation Scripts Using TestMu AI Selenium Grid & TestRail - -**Step 1:**  Required JAVA files for TestRail integration are inside `package com.library.utils.testrail`. - -**Step 2:**  In your test file you actually create an object of TestRailHandler like below: - -``` -TestRailHandler trh = new TestRailHandler(testrailusername,testrailpassword, testrailurl) -``` - -**Step 3:**  Then call it’s method as below: -``` -trh.updateResultToTestRail(executionstatus, run_id, testcase_id); -``` - -**Step 4:**  In above arguments execution status is numeric number that denotes pass, fail statuses defined in TestRail. Example: Pass -> 1, Fail -> 5 etc. - -**Step 5:**Use the below command to execute a single test. -``` -//To run single test -$ mvn test -P single -``` - -Once Test gets run, it logs result in TestRail as output mentioned here: - -``` -Jun 12, 2019 2:34:50 PM org.openqa.selenium.remote.ProtocolHandshake createSession -INFO: Detected dialect: OSS -here -add_result_for_case/215/3351 -Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 35.235 sec -Results : -Tests run: 1, Failures: 0, Errors: 0, Skipped: 0 -[INFO] ------------------------------------------------------------------------ -[INFO] BUILD SUCCESS -[INFO] ------------------------------------------------------------------------ -[INFO] Total time: 40.209 s -[INFO] Finished at: 2019-06-12T14:35:10+05:30 -[INFO] ------------------------------------------------------------------------ -``` - -Kudos! Now you can deploy your code in a reliable & faster manner at scale, using TestMu AI integration with TestRail, for automated cross browser testing with Selenium Grid. Happy Testing! diff --git a/static/docs/testrigor-integration.md b/static/docs/testrigor-integration.md deleted file mode 100644 index e034d23ff..000000000 --- a/static/docs/testrigor-integration.md +++ /dev/null @@ -1,47 +0,0 @@ -# testRigor Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -testRigor is a Generative AI-based codeless test automation tool that helps companies empower manual testers to create test automation through executable specifications in plain English. This innovative system enables your team to easily build and maintain tests, while its AI Native features significantly reduce the effort required for test maintenance. - -TestMu AI integration with testRigor enables you to perform desktop web, mobile web and native app testing across 3000+ real browsers, devices, and operating systems. - -## Prerequisites - -1. Create a TestMu AI account to begin testing your web application. If you do not have an account, register for free. - -2. Ensure you have a testRigor account. - -## Creating Test Suite In testRigor - -1. Sign in to your testRigor account. - -2. After you log in to testRigor, it will prompt you to create a new test suite. You need to furnish details like test suite name, type of testing (Desktop Web, Mobile Web or Native and Hybrid Mobile), test URL and so on. - -3. Once you provide test suite details, click **Create Test Suite** and you will be routed to the testRigor dashboard. - -## Integrating TestMu AI With testRigor - -1. In the testRigor dashboard, select **Settings** from the left sidebar. - -2. Navigate to the **Integrations** tab. - -3. Click the *Enable TestMu AI* checkbox and enter your TestMu AI Username and Access Key. You can find your credentials in the [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build). - -In case, you wish to test locally or privately hosted websites, you can select the *Enable TestMu AI Tunnel* checkbox and set your desired Tunnel Name. - -4. Click **Save**. - -5. Next, go to the **Multiple browsers** tab. - -6. Choose your cloud **Provider** as TestMu AI. Select **OS**, **OS Version**, **Browser** and **Browser Version**. Click **Add Browser** and then **Save**. - -## Running Tests With testRigor On The TestMu AI Platform - -1. From the testRigor's left sidebar, click **Test Cases**. - -2. Open the dropdown menu of the test case you want to re-run, and click **Re-test**. - -> The below test case demonstrates TestMu AI Selenium Automation's booking a demo functionality. First, it visits the TestMu AI Selenium Automation website, clicks on the "Book A Demo" button, enters the required fields and then presses the "Schedule Demo" button. - -3. Now visit the TestMu AI Automation Dashboard to view your test execution results. diff --git a/static/docs/testsigma-integration.md b/static/docs/testsigma-integration.md deleted file mode 100644 index 73e906916..000000000 --- a/static/docs/testsigma-integration.md +++ /dev/null @@ -1,36 +0,0 @@ -# Testsigma Integration With TestMu AI’s Cloud Based Selenium Grid - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Testsigma is an unified codeless automation testing platform, designed and built for Agile and DevOps teams. With Testsigma, you can automate tests on Web, Mobile, and APIs; along with your team, with minimal or no coding skills required. - -Testsigma now offers integration with TestMu AI to help you perform automated cross-browser testing taking advantage of the cloud-based [Selenium Grid](https://www.testmuai.com/blog/why-selenium-grid-is-ideal-for-automated-browser-testing/) on over 2000 real browsers. TestMu AI Selenium Grid integration to Testsigma will help you run your Selenium automation scripts from Testsigma instance to our cloud-based Selenium Grid very easily. Lets see how. - -## How To Integrate TestMu AI With Testsigma - -To integrate your TestMu AI account with the Testsigma instance, go to the Plugins options in Settings and enable the TestMu AI environment under Test Lab category. - -The TestMu AI option will open and you will be needed to enter your TestMu AI credentials, including username and access key. Once you have entered the required information, click on Create button. - -As soon as you click on the Create button, your TestMu AI credentials will be verified. Once the credentials verification is successful, the TestMu AI option will show a green coloured Enabled button, stating that the integration is successful, as shown in the image below. - -## How To Execute Tests On TestMu AI From Testsigma - -Now that you have integrated TestMu AI as the Test Environment Provide, you can execute your tests on the TestMu AI's cloud-based Selenium Grid. To do so: - -1. Select the project of your choice, and choose the test lab as TestMu AI, from the list of active test labs. - -2. A list of options will appear now, as shown in the image below, stating the type of environment upon which you want to run your test. You can select any as per your choice. - -3. Fill the details of the test environment configuration on which you want to run your test. For example, if you select the option as _Web_, then you need to select the browser, browser version, operating system, and resolution. - -4. Once done, click on Run Now, and the test execution will start immediately. - -## How To View Test Execution Details On TestMu AI Platform - -As soon as you execute the test from Testsigma, the test execution will start on TestMu AI platform as well. To view the test execution details, open the [Automation dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/). You will see your test details on the top, as shown below: - -Not only this, but you can also confirm the test environment configuration, view test execution video, console logs, network logs, exceptions, step by step screenshots, and much more. All you need to do is click on test build to open the Automation log view. - -> -Kudos! Now you can deploy your codeless automation tests in a reliable & faster manner at scale, using TestMu AI integration with Testsigma, and perform automated cross browser testing with our cloud based online Selenium Grid. If you still have any questions, feel free to share them with us through our **24*7 Chat Portal** or by mailing to us on [**support@testmuai.com**](mailto:support@testmuai.com). Happy testing! 🙂 diff --git a/static/docs/testunit-with-selenium-running-testunit-automation-scripts-on-testmu-selenium-grid.md b/static/docs/testunit-with-selenium-running-testunit-automation-scripts-on-testmu-selenium-grid.md deleted file mode 100644 index b858e7bb9..000000000 --- a/static/docs/testunit-with-selenium-running-testunit-automation-scripts-on-testmu-selenium-grid.md +++ /dev/null @@ -1,108 +0,0 @@ -# Selenium With Test::Unit - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Run Test::Unit tests on the TestMu AI cloud grid. This guide covers setup, running a sample test, configuring capabilities, and testing locally hosted pages. - -**Sample repo** -All the code used in this guide is available in the sample repository. - - View on GitHub - -## Prerequisites - -Complete the following steps before you start Ruby automation testing with Selenium. - -1. Install Ruby and gem on your local system. Follow these instructions to install on different operating systems. - * For **Windows**, download from the [Ruby Installer for Windows](https://rubyinstaller.org/downloads/). - * For **Linux** or **Ubuntu**, run a simple apt command like below: -```bash -sudo apt-get install ruby-full -``` - * For **macOS**, run a [Homebrew](https://brew.sh/) command like this: -```bash -brew install ruby -``` -2. Install the [parallel_tests](https://github.com/grosser/parallel_tests) gem to run tests in parallel. -3. Get the TestMu AI binary file for running tests on your locally hosted web pages. - -## Step 1: Clone the Sample Project - -Clone the repository and install dependencies. - -```bash -git clone https://github.com/LambdaTest-sample-test-frameworks/Ruby-UnitTest-Selenium.git -cd Ruby-UnitTest-Selenium -``` - -Install project dependencies: -```bash -bundle install -``` - -## Step 2: Set Your Credentials - -Configure your credentials to connect to the TestMu AI Selenium Grid. - -Set TestMu AI Username and Access Key in environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -## Step 3: Configure Your Test Capabilities - -Define browser, version, and OS settings for your test run. - -In the test script, update your test capabilities. This code passes browser, browser version, and operating system information, along with TestMu AI Selenium grid capabilities via the capabilities object. - -```ruby -caps = { -:browserName => lt_browser, -:browser_version => lt_browser_version, -:platform_name => lt_os, -:name => "UnitTest Sample Test", -:build => "UnitTest Selenium Build" -} -``` - -**Capabilities Generator** -Use the TestMu AI [Capabilities Generator](https://www.testmuai.com/capabilities-generator/) to auto-generate the capabilities class for your test requirements. - -## Step 4: Run the Test - -Execute the Test::Unit test from the command line. - -```bash -ruby LambdaTest.rb -``` - -## Step 5: View Your Results - -Check the test output on the console and the TestMu AI dashboard. - -Visit the [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) to view your test results. The dashboard provides: - -- Text logs for each test step -- Screenshots captured during execution -- Video recordings of the full test session - -## Run Test::Unit Tests Using Agent Skills - -Use AI coding assistants to generate and run Test::Unit tests with the TestMu AI Agent Skill. - -The [testunit-skill](https://github.com/LambdaTest/agent-skills/tree/main/testunit-skill) is part of [TestMu AI Agent Skills](https://github.com/LambdaTest/agent-skills/) - structured packages that teach AI coding assistants how to write production-grade test automation. - -Install the skill: - -```bash -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/testunit-skill .claude/skills/ - -# For Cursor / Copilot -cp -r agent-skills/testunit-skill .cursor/skills/ -``` - -Install all available framework skills at once by cloning the repository directly into your tool's skills directory (e.g., `.claude/skills/`, `.cursor/skills/`). diff --git a/static/docs/timeouts-issues-and-resolutions.md b/static/docs/timeouts-issues-and-resolutions.md deleted file mode 100644 index b1a325547..000000000 --- a/static/docs/timeouts-issues-and-resolutions.md +++ /dev/null @@ -1,61 +0,0 @@ -# Timeouts - Errors and Resolutions - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -To prevent machines from being held due to incorrect tests, there is a restriction on the number of seconds a machine stays reserved. If our servers fail to retrieve a request from your local machine for more than 120 seconds, your tests are aborted from the queue with a Timeout error message. - -## Reasons for a Timeout Error - -Several conditions can trigger a timeout error during test execution. - -There are several common reasons you may encounter a **Timeout error**. - -### 1. Idle Timeout - -If the TestMu AI Selenium grid detects inactivity for 120 seconds, you may encounter the below error: - -**Idle Timeout:** Test was terminated because grid hasn't received any requests for 120 second. To avoid this use idleTimeout using desired capabilities. e.g: set 'idleTimeout':'270′. -Once the default or user-defined idle timeout period is reached, our platform marks the test in the `idle_timeout` state. This will not necessarily happen exactly at the idle timeout duration. The system identifies and marks such tests on a best-effort basis, so it may take some additional time in certain cases. We recommend defining the `idleTimeout` capability according to your requirements. - -The timeout could also occur due to network latency, or due to improper declaration of driver.quit(). - -**Resolution:** You can extend the 120-second idle timeout limit up to 1800 seconds by declaring the idleTimeout capability in your desired capabilities. - -```bash -"idleTimeout": "125" //extends idle timeout from 120 seconds to 125. -``` - -You can extend the inactivity limit to 1800 seconds, but doing so may cause performance issues. We recommend extending the idle timeout between 270-360 seconds. - -### 2. Improper Use of Exception Handling - -If your exception handling is missing the driver.quit() method, your test may lead to a timeout. driver.quit() eradicates every instance of WebDriver. It shuts down all browser windows belonging to the WebDriver and terminates the session safely. - -### 3. Network Latency - -A request triggered from your local machine sometimes fails to reach our server within 120 seconds. This could happen due to latency in your network, especially if you are using a VPN (Virtual Private Network). - -If you are using a VPN, use this tool to audit your network latency. - -### 4. Queuing Timeout - -The maximum number of tests allowed to be queued for your TestMu AI account depends on the number of concurrent sessions you are eligible for. Use the below formula to calculate your maximum queuing capacity. - -**Maximum Queuing Capacity** -Here is an example: if your TestMu AI account is eligible for 10 concurrent sessions, then your queue can have a maximum of (10 + 150) queued test cases, i.e. 160 queued test cases. TestMu AI handles the scheduling and execution of test cases in your queue. - -```javascript -Maximum number of test cases that can be queued = n + 150 - -// Here, n = number of concurrent sessions. -``` - -Queuing timeout could happen because of the below: - -**Tests in Queue Taking Too Long to Complete:** Tests are queued for only 15 minutes. If the tests in your queue exceed the 15-minute timeline, they are removed from queue. This aborts the execution of your tests, leading to a timeout error. - -### 5. Firewall Protected Network - -If you work within an organization that has an active firewall setup to help employees securely connect with internal servers, accessing the TestMu AI Selenium grid under a firewall-protected network may cause your tests to time out. - -Browse `https://hub.lambdatest.com/wd/hub/session` to validate whether your network has restricted access to TestMu AI. It should return some response. For example: **```{"status":404,"value":{"message":"method not allowed"}}```** If **no response** is received from TestMu AI, **you may need to whitelist this host from your URL firewall.** diff --git a/static/docs/tms-reports.md b/static/docs/tms-reports.md deleted file mode 100644 index cb5e20523..000000000 --- a/static/docs/tms-reports.md +++ /dev/null @@ -1,52 +0,0 @@ -# Reports in Test Manager - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Test Manager provides powerful reporting capabilities to help teams gain actionable insights from their testing activities. With **Execution and Traceability Reports**, you can measure test coverage, track progress, and ensure alignment between requirements, test cases, and defects. - -To access the reports you need to go inside the `Reports` section present inside a Project. - -There are two type of Reports that you can create here: - -## Execution Reports - -Execution based reports allow you to analyze the results of your test executions and identify patterns in test execution. - -When you create an Execution Report, you have two ways to filter the data: - -### 1. Filter By Test Runs: -This option lets you include the results from specific test runs you select. - -### 2. Filter By Date: -This option allows you to see all test results that were generated within a specific date range. - -You can optionally add **Description** for the Report too but adding **Name** is mandatory to proceed here. - -**Recurring Reports:** -You can select the `Recurring` option, add **Recipients** & **Frequency** to automatically get the mail notification with the reports timely. - -This is how the reports will look like: - -### Execution History Report - -The detailed Execution History Report lists each Test Instance in the selected test run(s) with the following information: Test Case ID and name, execution status, test run name, step count, configuration (platform, OS, and device), mode, executed by, and executed at. Each row includes a link to navigate directly to that test instance in the product. - -**Step-level data in CSV export** - -When you download the report as a CSV, each step for every test instance is listed as a separate row, including: - -- **Step** — The step description -- **Expected Outcome** — The expected result for that step -- **Step Status** — The execution status of the step, supporting standard statuses (Passed, Failed, Skipped) as well as any custom statuses configured for your project - -Step-level execution tracking in reports is currently in slow rollout. To get early access, raise a request to our support team. - - All the Reports can be Downloaded as CSV files. - -## Traceability Reports - -Traceability reports provide end-to-end visibility into how your test cases align with business requirements and defects. - -The Traceability reports show the Issues based test-execution report allowing you to see the Issues coverage in tests: - -The report data can be filtered using Test, Issue, and Run metadata. diff --git a/static/docs/tosca-integration-with-hyperexecute-for-sap.md b/static/docs/tosca-integration-with-hyperexecute-for-sap.md deleted file mode 100644 index f8f4ca7d2..000000000 --- a/static/docs/tosca-integration-with-hyperexecute-for-sap.md +++ /dev/null @@ -1,43 +0,0 @@ -# Tosca Integration for SAP - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This documentation provides a detailed guide on seamlessly integrating Tosca with HyperExecute. - -## Prerequisite - -- To run Tosca, ensure that you have all the required [licences](https://documentation.tricentis.com/tosca/1600/en/content/licensing/lic_administering_licenses.htm). - -## Integrate Tosca with HyperExecute - -To Integrate Tosca with HyperExecute, follow the below mentioned steps: - -### Step 1: Initialize the Tosca Project - -- Go to the [HyperExecute Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/). Click on the **Projects**. - -- Click on the **Setup a New Project** button. - -- Enter your **Project Name**. Click on Continue. - -### Step 2: Manage your License and Credentials - -- Enter the **License Details**, i.e. Username and Password with which you want to execute your tests. - -- Now select your **Database Type**. For the demo purpose, we are using Oracle. Choose how you want to establish the connection, i.e, **via Connection Data** or **via Connection String**, to access the common repository associated with your workspace. - - - For demo purposes, we are selecting the **Connection String**. Enter the connection string for your database. - - - Enter the **Workspace Details**, i.e., Username (mandatory) and Password (optional), that you want to use for this project. Click on the **Save** button. - -### Step 3: Execute Your Project - -- Please wait for the discovered execution list to get populated. Now, select the desired tests that you want to execute, and then click on the **Run** button. - -- Now set the total number of **Concurrent Sessions** that you want to run on HyperExecute and click on the **Run** button. - -### Step 4: Check the Logs Status in Dashboard - -- Now go to the [**Jobs**](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute/jobs) section and check the results. - -> Results will get updated in the **Common Repository** once execution is completed on HyperExecute. The live video of running tests can be viewed from the [HyperExecute job-detail](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute/jobs)dashboard. diff --git a/static/docs/tosca-integration-with-hyperexecute-using-commander.md b/static/docs/tosca-integration-with-hyperexecute-using-commander.md deleted file mode 100644 index f5b8647aa..000000000 --- a/static/docs/tosca-integration-with-hyperexecute-using-commander.md +++ /dev/null @@ -1,127 +0,0 @@ -# Tosca Integration using Commander - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This documentation provides a detailed guide on seamlessly integrating Tosca with HyperExecute. - -## Pre-requisite: - -- To run Tosca, ensure that you have all the required [licences](https://documentation.tricentis.com/tosca/1600/en/content/licensing/lic_administering_licenses.htm). - -## Integrate Tosca with HyperExecute - -To Integrate Tosca with HyperExecute, follow the below mentioned steps: - -**Step 1:** Go to the [HyperExecute Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/). Click on the **Projects**. - -**Step 2:** Click on the **Setup a New Project** button. - -**Step 3:** Enter your **Project Name**. Click on Continue. - -**Step 4:** Enter the **License Details**, i.e. Username and Password with which you want to execute your tests. - -**Step 5:** Now select your **Database Type**. For the demo purpose, we are using Oracle. Choose how you want to establish the connection, i.e, **via Connection Data** or **via Connection String**, to access the common repository associated with your workspace. For demo purposes, we are selecting the **Connection String**. Enter the connection string for your database. - -Enter the **Workspace Details**, i.e., Username (mandatory) and Password (optional), that you want to use for this project. Click on the **Save** button. - -**Step 6:** Please wait for the discovered execution list to get populated. Now, select the desired tests that you want to execute, and then click on the **Run** button. - -**Step 7:** Now set the total number of **Concurrent Sessions** that you want to run on HyperExecute and click on the **Run** button. - -**Step 8:** Now go to the [**Jobs**](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute/jobs) section and check the results. - -> Results will get updated in the **Common Repository** once execution is completed on HyperExecute. The live video of running tests can be viewed from the [HyperExecute job-detail](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute/jobs)dashboard. - -## Integrate Tosca with HyperExecute using YAML - -To Integrate Tosca with HyperExecute using YAML, follow the below mentioned steps: - -#### Prerequisite - -- Ensure that you have integrated Tosca with [TestMu AI�s cloud-based Selenium Grid](/support/docs/tricentis-tosca-integration-testmu-selenium-grid/). - -**Step 1:** Create an Execution List: - -- Go to the execution tab in your Tosca commander to create an execution list. - -- You can drag and drop test cases from the **Test Cases** tab to the execution list. In the example given below, the execution list is called LT. - -**Step 2:** Create a **TC-Shell Script**: - -- A TC-Shell script file usually has the extension _.tcs_, and it contains a sequence of [TC-Shell commands](https://support-hub.tricentis.com/open?id=kb_article_view&table=kb_knowledge&number=KB0035628&searchTerm=shell%20commands) in plain text to configure the steps to execute a Tosca test via command line. This script file is kept in the same folder where you have the actual Tosca project saved. - -- **Sample TC-Shell Script:** - -```bash -//script - -jumptonode "/Execution/ExecutionLists/LT" - -// reset actual log -task "clear log" - -//start execution list -task run - -//save results -save -``` - -- Click the arrow on the right side of the **Execution** tab to retrieve the node path used in **jumptonode** command. - -- The node path is highlighted in the image below. - -- Use the following command to execute this test via command line. - -```bash -- TCShell.exe -workspace -login "" -``` - -**Step 3:** After configuring the TC-Shell Script, integrate it with HyperExecute. To do this: - -- Create a Hyperexecute YAML. - -- Download the Hyperexecute CLI Binary based on the OS you use: - -| Platform | Download Link | -| ---------| --------------------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| macOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -> The binary for Mac is unsigned. You might get a permission denied warning while executing on mac and linux machines. To allow permissions you can use the following command: `chmod u+x ./hyperexecute`. -Thus, you might see a security popup on whether to allow the execution of the tool or not. You can allow it from your **System Preferences** ? **Security & Privacy** ? **General** tab. - - **Sample Hyperexecute YAML file for TOSCA**: -```yaml ---- -version: "0.1" -matrix: -os: [win] -runson: ${matrix.os} - -retryOnFailure: true -maxRetries: 2 - -pre: -- PATH=%TRICENTIS_LICENSING_HOME%;%PATH% && ToscaLicenseConfiguration.exe connect-cloud -u -p - -testSuites: -- TCShell.exe -workspace D:\foreman\\Demo_Sample.tws -login "D:\foreman\" - -post: -- cat hyex_tosca.yml - -uploadArtefacts: -- name: Logs -path: -- Demo_Sample/** - -``` -> Each machine has a Tosca commander Client installed on HyperExecute. To successfully run your tests, your Tosca licenses must be pre-configured on the machine. To do so, we use the **ToscaLicenseConfiguration.exe** along with the **connect-cloud** parameter to validate the licenses as a **pre-step** before triggering the tests. The complete command is added under **pre** in the above sample. - -- After configuring the YAML, execute it via terminal using the following command: - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE `} - -You have successfully integrated your Tricentis Tosca project with HyperExecute. diff --git a/static/docs/tosca-integration-with-hyperexecute-using-dex.md b/static/docs/tosca-integration-with-hyperexecute-using-dex.md deleted file mode 100644 index 582a59e0d..000000000 --- a/static/docs/tosca-integration-with-hyperexecute-using-dex.md +++ /dev/null @@ -1,69 +0,0 @@ -# Tosca Integration using DEX - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This documentation provides a detailed guide on seamlessly integrating Tosca with HyperExecute using DEX. - -## Prerequisites - -1. Ensure that you have the required Tosca DEX License. - -2. A valid running Tosca DEX Server. - -## Steps to Integrate Tosca - -### Step 1: Initialize the Tosca Project - -- Go to the [HyperExecute Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/). Click on the **Projects**. - -- Click on the **Setup a New Project** button. - -- Select **Tosca with DEX** tool. Enter your **Project Name**. Click on Continue. - -### Step 2: Manage your License and Credentials - -- Select the **Type** as Cloud License. Enter the **License Details**, i.e. Username and Password with which you want to execute your tests. **Cloud Server id** field is optional. - -- Now select your **Database Type**. For the demo purpose, we are using Oracle. Choose how you want to establish the connection, i.e, **via Connection Data** or **via Connection String**, to access the common repository associated with your workspace. - -For demo purposes, we are selecting the **Connection String**. Enter the connection string for your database. - -Enter the **Workspace Details**, i.e., Username (mandatory) and Password (optional), that you want to use for this project. Click on the **Continue** button. - -### Step 3: Configure DEX Server Details - -- You need to provide the details of your DEX Server as per the **prerequisites**. - -- Select the **Type** of the protocol. For the demo purpose, we are using HTTP protocol. -- Provide the **IP Address** of your DEX Server. -- Enter your required **Port** (optional). - -Click on **Save** button. - -### Step 4: Execute Your Project - -- Please wait for the discovered test events to get populated. Now, select the desired test events that you want to execute, and then click on the **Run** button. - -- Now set the total number of **Concurrent Sessions** that you want to run on HyperExecute, enable the [Retry on Failure](#auto-retry-for-non-assertion-errors) (optional) and click on the **Run** button. - -### Step 5: Check the Logs Status in Dashboard - -- Now go to the [**Jobs**](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute/jobs) section and check the results. - -You can also download the video output. - -## Auto Retry for Non-Assertion Errors - -You can enhance the efficiency of Tosca test execution by automating the retry process in the event of **Non-Assertion Errors**. It usually occurs when a program encounters an unexpected conditions like runtime error, timeouts, connectivity issues etc. - -To enable the **Auto Retry** on Failure of tests: - -**Step 1:** After setting up your DEX Server, select the events that you want to execute. Click on the Run button. - -**Step 2:** Now enable the toggle button for **Retry on Failure** and click on the Run button. - -Now, once your execution is finished, the platform will automatically analyze your execution list results and logs. - -If your execution lists fail due to any non-assertion error, HyperExecute will automatically trigger those specific failed lists again. - -> **NOTE :** Your event name should be same with the execution list. diff --git a/static/docs/tosca-integration-with-hyperexecute.md b/static/docs/tosca-integration-with-hyperexecute.md deleted file mode 100644 index 3847a4427..000000000 --- a/static/docs/tosca-integration-with-hyperexecute.md +++ /dev/null @@ -1,18 +0,0 @@ -# Tricentis Tosca Integration with HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Tricentis Tosca is a tool that optimizes and accelerates end-to-end testing for your entire digital landscape. With its no-code approach and the assistance of AI, Tosca efficiently removes bottlenecks from your testing process. - -HyperExecute seamlessly integrates with Tosca, providing a Just-in-Time Tosca Environment for both Tosca **Commander** and **DEX**. This ensures a smooth and efficient testing experience for users. - - Integration using Commander - Use Tosca Commander to integrate with HyperExecute. - - Integration using DEX - Use Tosca DEX Server to integrate with HyperExecute. - - Integration for SAP - Use Tosca for SAP to integrate with HyperExecute. - -> This is currently in the **Beta** version of the **Project** section in HyperExecute. To enable it for your organization, please contact our **24×7 chat support** or our [Customer Support](mailto:support@testmuai.com) team. diff --git a/static/docs/track-issues-in-test-runs.md b/static/docs/track-issues-in-test-runs.md deleted file mode 100644 index 87567a251..000000000 --- a/static/docs/track-issues-in-test-runs.md +++ /dev/null @@ -1,93 +0,0 @@ -# Track Bugs and Issues in Test Runs - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Overview - -Test Manager connects the bugs found during testing to the test instances that exposed them. Every test instance in a Test Run carries its own issues, so the run becomes a traceable map of test results and the defects behind them. You can see which tests have known bugs and open any of them in one click. - -There are two ways to put a bug on a test instance: - -- **Link an existing issue**: connect a Jira or Azure DevOps ticket that already exists. -- **Raise a new bug with Mark as Bug**: create a new ticket in your bug tracker while you execute the test. - -This works in Manual, KaneAI, and Automation Test Runs. - -## Before you begin - -- To link or raise bugs that are visible inside Test Manager, integrate **Jira** or **Azure DevOps** with your TestMu AI account: - - [Link Jira Issues with Test Manager](/support/docs/link-jira-issues-with-test-manager/) - - [Link Azure DevOps Issues with Test Manager](/support/docs/link-ado-issues-with-test-manager/) -- Mark as Bug can also create tickets in other [bug tracking tools](/support/docs/bug-tracking-tools/). See [Supported trackers and visibility](#supported-trackers-and-visibility) for what that means inside Test Manager. - -## How bug tracking works in a Test Run - -A bug is always associated with a **test instance**, one test case running with one configuration inside a Test Run. Whether you link an existing issue or raise a new one, the association is the same, and the bug becomes visible in three places: - -- On the test instance level. -- In the Test Run's consolidated **Issues** tab, which lists every bug in the run. -- Inside the Issues tab of the corresponding Test Case. - -You can add a bug from any of these points: from the run's instance list, from inside an instance, or from a specific step of an instance. A bug added at a step is listed with its parent instance, so you always see the complete picture at the instance level. - -## Link an existing issue to a test instance - -Use this when the defect is already tracked in Jira or Azure DevOps and you want to connect it to the test that found it. - -1. Open the Test Run and stay on the **Test Instances** tab. -2. On the test instance you want, open the bug menu on its row and select **Link Issue**. - -3. In the **Link Issues** dialog, choose the tracker: **Jira** or **Azure DevOps**. Only trackers you have integrated are available. -4. Enter the issue key or paste its URL. -5. Click **Link Issue**. - -The issue is now linked, and the test instance row shows a bug count. That count is your at-a-glance signal for which tests in the run have known defects. - -## Raise a new bug with Mark as Bug - -Use this when you find a defect while executing a test and no ticket exists yet. - -1. On a test instance, or a specific step within it, choose **Mark as Bug**. -2. In the **Create an Issue** panel, confirm the bug tracker. Use **Switch App** to file the bug in a different connected tracker. -3. Fill in the issue details, such as the project, issue type, and summary. Required fields are marked. The panel already carries the test instance's context. -4. Click **Create Issue**. - -Test Manager files the ticket in your bug tracker without leaving the run. When the ticket is created in Jira or Azure DevOps, it is associated with the test instance and appears in the same views as a linked issue. - -Mark as Bug can file tickets in several bug trackers, but only Jira and Azure DevOps tickets are tracked back to the test instance. See [Supported trackers and visibility](#supported-trackers-and-visibility). - -## Review the issues on an instance - -You can review the bugs on a test instance in two ways: - -- **From the run list**: open the bug menu on the instance row and select **View Issues**. A side panel lists every issue associated with that instance. -- **From the instance**: open the test instance and go to its **Issues** tab. The tab header shows how many issues the instance carries. This is only possible in Manual Test Runs. - -Each issue entry shows its title, key, type, status, priority, the project it belongs to, and who created it, enough to triage without opening the tracker. Use the search box and the **Status** and **Issue Type** filters to narrow a long list. - -## Add a bug while reviewing an instance or a step - -You do not have to act from the run list. Inside a test instance you can link an existing issue or raise a new bug at two levels: - -- **At the instance**: for a defect that affects the test case as a whole. -- **At a step**: open the **Test Steps** tab and use the bug action on the step where the failure occurred, so the defect is recorded against the exact point of failure. - -A bug added at a step is listed on the instance's **Issues** tab along with every other issue for that instance, so the instance always reflects the full set of defects found. - -## See every issue in the run - -The Test Run's **Issues** tab is the consolidated view of every bug raised or linked across all of its test instances, useful for a defect triage or a run sign-off. - -1. Open the Test Run and go to the **Issues** tab. -2. To add a bug that is not yet tied to a specific instance, use **Link Issue** on this tab. -3. To focus the list, filter by **Test Case**, for example, to see only the bugs behind one failing test case. You can also filter by **Status** and **Issue Type**, or search by keyword. - -## Unlink an issue - -If a bug no longer belongs on a test instance, open the instance's **Issues** tab (or the **View Issues** panel), find the issue, and use its unlink action. Unlinking removes the association in Test Manager only. The ticket itself stays in Jira or Azure DevOps. - -## Supported trackers and visibility - -Linking an existing issue is available for **Jira** and **Azure DevOps**. - -Mark as Bug can create tickets in several bug tracking tools, but only **Jira** and **Azure DevOps** tickets are associated with, and visible in, Test Runs, Test Cases, and test instances. A bug raised in any other tracker is created successfully but does not appear in these views. diff --git a/static/docs/travis-ci-with-hyperexecute.md b/static/docs/travis-ci-with-hyperexecute.md deleted file mode 100644 index 97ab7a58b..000000000 --- a/static/docs/travis-ci-with-hyperexecute.md +++ /dev/null @@ -1,50 +0,0 @@ -# Travis CI Integration with Hyperexecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Travis CI is a continuous integration (CI) service that allows to automate the process of testing and deploying their code. It is widely used for projects hosted on GitHub and other version control platforms like Bitbucket, and GitLab. -Using Travis CI, you can automate your entire testing suite for new commits, reducing the possibility of human error, while also automating deploys with orbs. - -This document will show you how to integrate Travis CI Pipeline with HyperExecute to greatly shorten your test cycles. - -## Steps to Integrate Travis CI Pipeline with Hyperexecute - -To integrate Travis CI Pipeline with HyperExecute, follow the below steps: -### Step 1. Log into your Travis CI account - -- Signup in your Travis CI account using your desired version control system. Here, we are using GitHub. - -- Click on your profile, then go to settings and then click on the green Activate button, and select the repositories you want to use with Travis CI or you can click on the Activate all repositories using GitHub Apps button to activate all your repos. - -### Step 2: Configure the YAML File - -Once you have set up your project, you need to add a `.travis.yml` file to your desired repository to trigger the build from the Travis CI. - -Below is a sample of Travis CI YAML created for your reference: - -```yaml -language: node - -services: -hyperexecute: -api_key: YOUR_API_KEY ## add your hyperexecute access key - -before_script: -- npm install -g cypress -- npm install - -script: -- ./hyperexecute --user --key --config -``` -- Add the `.travis.yml` file to root directory, commit it and then click to **"Trigger a Travis CI build"**. - -- Check the build status page to see if your build passes or fails according to the return status of the build command by visiting Travis CI and selecting your repository. - -### Step 3: Run Your Job - -Now commit your job and run the build. - -**Below is an example of a HyperExecute job that was triggered through the above pipeline:** - -> -**Run your tests at speeds never seen before. Happy testing! :)** diff --git a/static/docs/travis-ci-with-testmu.md b/static/docs/travis-ci-with-testmu.md deleted file mode 100644 index 98bbec861..000000000 --- a/static/docs/travis-ci-with-testmu.md +++ /dev/null @@ -1,178 +0,0 @@ -# Travis CI Integration with TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Travis CI is one of the most popular tool which facilitates continuous integration for GitHub projects. Open source projects can use Travis CI without having to pay anything from their pocket. Now, deploy your code rapidly and test it thoroughly for a robust delivery in your go-to market launch. - -TestMu AI now integrates with Travis CI to boost your go-to market delivery. Perform automated cross browser testing with TestMu AI to ensure your development code renders seamlessly through an online Selenium grid providing 3000+ real browsers running through machines, on cloud. Perform automation testing in parallel with TestMu AI’s Selenium grid to drastically trim down your test cycles. - -## Prerequisites For Integrating Travis CI With Our Online Selenium Grid - -In order to perform your karma tests with TestMu AI, you would need the below things to be already set up: - -**1. Global Dependencies** - -* A [Git or GitHub repository](https://github.com/) -* Download and [install node.js](https://nodejs.org/en/) and node package manager or npm. -To install node.js with homebrew use the below command. -``` -$ brew install node -``` - -* If you have npm already installed, you may want to upgrade it to latest version. Here the code you can run in your terminal to upgrade npm. -``` -npm install npm@latest -g -``` - -**2.TestMu AI Authentication Credentials** - -Be aware of your TestMu AI authentication credentials i.e. your TestMu AI username, access key and HubURL. You need to set them up as your environment variables. You can retrieve them from your [TestMu AI automation dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/) by clicking on the key icon near the help button. - -* For Linux/Mac: -``` -$ export LT_USERNAME= {YOUR_LAMBDATEST_USERNAME} -$ export LT_ACCESS_KEY= {YOUR_LAMBDATEST_ACCESS_KEY} -``` - -* For Windows: - -``` -$ set LT_USERNAME= {YOUR_LAMBDATEST_USERNAME} -$ set LT_ACCESS_KEY= {YOUR_LAMBDATEST_ACCESS_KEY} -``` - -## Integrating TestMu AI With Travis CI - -To integrate Travis CI with TestMu AI you need to make minor tweaks in the ***.travisci/travis.yml***  which is the configuration file for your Travis CI instance. The changes you need to make will revolve around the environment variables such as access key, username, grid config and so on. - -Below is a sample config file for integrating [Travis CI](https://github.com/LambdaTest/Nightwatch-travis-ci-selenium) with TestMu AI. You can use the sample file to test your own automation test suite by replacing the environment variables according to your need. All the code used for the demonstration of this topic can be find in our GitHub repository. - -``` -# Choose language and version -language: node_js -node_js: -- "8" -branches: -only: -- master - -# Setup custom environment variables -env: -global: -- LT_USERNAME=${LAMBDATEST_USERNAME} -- LT_ACCESS_KEY=${LAMBDATEST_ACCESS_KEY} - -# All the step which you want to run in container. -script: -# Fetching latest code from git -- git pull origin master -# Installing dependencies -- npm install -# Running test! -- node_modules/.bin/nightwatch -e chrome -``` - -TestMu AI offers a [**Selenium Desired Capabilities Generator**](https://www.testmuai.com/capabilities-generator/) to fetch coded values for your desired test configurations. - -For Instance, if you select the below configurations: - -|CAPABILITY|SELECTED VALUE| -| :---: | :---: | -|Operating System|Windows 8.1| -|Resolution|1280×1024| -|Browser|Firefox| -|Browser Version|62.0| -|Selenium Version|3.11.0| - -Then Capabilities Generator at TestMu AI will provide you with the below program: -``` -var capabilities = { -"build" : "your build name", //You can edit this and assign a build name -"name" : "your test name", // Assign a name to your Test -"platform" : "Windows 8.1", // The operating system on which you want to test your website -"browserName" : "Firefox", // The browser on which you want to test -"version" : "62.0", // The browser version which you've selected to perform the test upon -"resolution" : "1280x1024", // The resolution in which you want to run the test as per your operating system -"selenium_version" : "3.11.0", //The version of Selenium on which the test will run -"visual" : true, -"firefox.driver" : v0.21.0 -} -``` - -For more information, refer to our documentation on [Selenium Automation Capabilities](/docs/selenium-automation-capabilities/) - -## Testing Locally Hosted or Privately Hosted Projects - -To help you perform cross browser testing of your locally stored web pages, TestMu AI provides an SSH(Secure Shell) tunnel connection with the name Lambda Tunnel. With Lambda Tunnel, you can execute a test server inside your Travis CI build container to perform automated cross browser testing on browsers offered by Selenium grid on TestMu AI. So you make sure how well your changes look, even before your customers. Curious to know more about Lambda Tunnel? - ->Follow our documentation on Lambda Tunnel to know it all. OS specific instructions to download and setup tunnel binary can be found at the following links. -> ->* **[Documentation For Windows User](/docs/local-testing-for-windows/)** ->* **[Documentation For Mac User](/docs/local-testing-for-macos/)** ->* **[Documentation For Linux User](/docs/local-testing-for-linux/)** - ->Download the binary file of: ->* [Lambda Tunnel for Windows](https://downloads.lambdatest.com/tunnel/v3/windows/64bit/LT_Windows.zip) ->* [Lambda Tunnel for Mac](https://downloads.lambdatest.com/tunnel/v3/mac/64bit/LT_Mac.zip) ->* [Lambda Tunnel for Linux](https://downloads.lambdatest.com/tunnel/v3/linux/64bit/LT_Linux.zip) - -The below example of ***travis.yml***  file would demonstrate you on how to leverage TestMu AI’s Selenium grid by performing a browser test through your testing server contained in your Travis CI build. - -``` -# Choose language and version -language: node_js -node_js: -- "8" -branches: -only: -- master - -# Setup custom environment variables -env: -global: -- LT_USERNAME=${LAMBDATEST_USERNAME} -- LT_ACCESS_KEY=${LAMBDATEST_ACCESS_KEY} - -# All the step which you want to run in container. -script: -# Fetching latest code from git -- git pull origin master -# Downloading tunnel binary -- wget https://downloads.lambdatest.com/tunnel/v3/linux/64bit/LT_Linux.zip -# Extracting tunnel binary -- sudo apt-get install unzip -- unzip LT_Linux.zip -# Executing tunnel binary -- ./LT -user ${LAMBDATEST_EMAIL} -key ${LAMBDATEST_KEY} & -- sleep 30 -# Installing dependencies -- npm install -# Running test! -- node_modules/.bin/nightwatch -e chrome -``` - -Once, the tunnel is successfully set up. You can add the below code to your capabilities for testing internal servers on your network. - -``` -"desiredCapabilities": { -"tunnel":true -} -``` - -## Parallel Testing - -Parallel Testing is one of the most demanding feature of TestMu AI Selenium Grid. By parallel testing, you can run more than one test case, simultaneously. This means that, Parallel testing would allow you to execute numerous automation test cases altogether. So you execute a single test scenario across different browsers or could run different test scenarios across the same browser but with different browser versions. - -To perform automation testing in parallel on Windows use the below command. -``` -$ node_modules\.bin\nightwatch -e chrome,edge,firefox tests -``` - -To perform automation testing in parallel on macOS/Linux use the below command. -``` -$ ./node_modules/.bin/nightwatch -e chrome,edge,firefox tests -``` - -Monitor and analyze your test result on the [**TestMu AI Automation Dashboard**](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/) - -Deploy your code in a reliable manner at scale using Travis CI integration with TestMu AI, and ensure it looks robust across every browser to provide a seamless user experience to all your visitors. Happy Testing! diff --git a/static/docs/trello-integration.md b/static/docs/trello-integration.md deleted file mode 100644 index ec2984754..000000000 --- a/static/docs/trello-integration.md +++ /dev/null @@ -1,75 +0,0 @@ -# Trello Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -> Trello – A project management tool that was acquired by Atlassian in 2017. It helps in project management under agile development. Providing a highly usable dashboard, where you can easily comprehend on who is working on what? Allowing a team to pinpoint the areas where they feel they are missing something out. Team can plan every sprint in an organized manner by creating and updating cards, lists & boards. Along with project management, Trello also aids in effective bug tracking by providing a space where all your bugs can be logged and prioritized according to your release bandwidth. - -The TestMu AI Trello Integration allows you to create a card directly in Trello dashboard from TestMu AI platform. Push an issue to your respective project anytime, even in the middle of your test session. The fields populated by you when marking as bug through TestMu AI are displayed as information on the card in Trello dashboard for a testing instance. - -**Trello Integration with TestMu AI, like all of our other integrations to 3rd party applications, is available for freemium as well as premium plan.** - -## How To Integrate Trello With Your TestMu AI Account? - -**Step 1:** Login to your TestMu AI account. You would need to have Admin or User level access to see and install integrations. - -**Step 2:** Select **'Integration'** from the left navigation menu bar. This will guide you to a screen where you will find a list of 3rd party applications, available to integrate with your TestMu AI account. - -**Step 3:** Click on **'Add'** under the block that says 'Trello'. - -**Step 4:** Once you click on add, you'll redirected to the below screen if you are already logged into Trello. Click on the green button that says **'Allow'**. - -* **Why do you require a login with Trello?** - -The purpose of logging in is to bypass the authentication that is applied when two different applications interact using their respective APIs. - -Trello API make use of token-based authentication for integrating with third-party applications. Once a user clicks on **'Allow'** to grant the third-party application to access user-owned resources from API, then that third-party application is provided a token for communicate with the same on behalf of the user. - -Another way through which a client access can authorize is by using basic [OAuth 1.0](https://tools.ietf.org/html/rfc5849). - -* **What is Token-based authentication?** - -Once the user allows a third-party application to fetch user owned resources from the server to which the third-party application was making requests, then a token gets generated. This token allows third-party application to exchange information with the server API on behalf of the user. - -If you are logging into Trello for the first time, then you can generate a token for yourself with the help of API key & the URL: -```javascript -https://trello.com/1/authorize?expiration=1day&name=MyPersonalToken&scope=read&response_type=token&key={YourAPIKey} -``` - -If you are not logged into Trello then you will be redirected to the below screen. Kindly login to your account via gmail or create your account by given option. - -**Step 5:** That's it! Go to Integrations again and you will be able to notice a **green tick** indicating that Trello is successfully **installed**. You can now create cards in a fly from any of your running test session in Lambdatest to your Trello dashboard directly by a single click. - -## How To Log Your First Bug Through Trello Integration? - -**Step 1:** Go for any of the test from the left navigation menu. For demo, we will be taking "**Real Time Testing**" option. - -**Step 2:** Present a URL of the web-app you need to test in the dialog box. After that, select any configuration for browser and operating system of your choice & hit '**Start**'. - -**Step 3:** After the VM is launched and operable. You can perform testing on your web-app for finding bugs. If a bug gets revealed, then you need to click on the **Bug icon** from the left panel for capturing a screenshot of the same. We have highlighted that option with yellow in the below image. - -**Step 4:** After a screenshot is captured, you can annotate any issue or a task with an **in-built image editor**. Once you are done highlighting the bug, click on the button that says **"Mark as Bug"**. - -**Step 5:** After clicking on **"Mark as Bug"** button a form would open up. Fill the fields as per your requirement. - -* You can select which **board** should the card go under. -* You can select the **list** for that particular card. -* You can assign the bug to a colleague by populating the field **'Assignee'**. -* You also get to post a **description** to help relate the cause of the issue or the card. - -**Step 6:** Click on **"Create Issue"** & observe it being successfully marked through a single click effort. Once you hit the button, you will get **prompt messages** on top of your Virtual Machine indicating the progress of bug logging. You will get a prompt message in few seconds mentioning "**Bug successfully marked**" indicating that the card has been generated in your dashboard. - -**Step 7:** Visit your dashboard. You will be able to notice the logged issue right away! - -Click on the recent card and you will find all the data that you provided through Lambdatest already presented in it. TestMu AI automatically includes test environment details and related screenshots in the card as attachments. - -## How To Remove Trello Integration? - -> You can work with one integration at a time. So if you would want to integrate to a similar 3rd party application, then you would have to **remove** your current integration. Here is how you can do that. - -**Step 1:** Login to your TestMu AI account. - -**Step 2:** Select 'Settings' from the left navigation menu bar & click on 'Integrations'. This will guide you to a screen where you will find 3rd party applications, available to integrate with your TestMu AI account. - -**Step 3:** Click on **'REMOVE'**. You can find the remove button right under the **'Trello'** block which would be highlighted with a green tick. - -That was all you need to know for TestMu AI + Trello Integration. Increase your productivity with our integrations. If you still have any questions for us, please feel free to let us know. Our experts are always **available on chat** to help you out with any roadblock regarding our product. Happy testing! diff --git a/static/docs/tricentis-tosca-integration-testmu-selenium-grid.md b/static/docs/tricentis-tosca-integration-testmu-selenium-grid.md deleted file mode 100644 index bb8419a36..000000000 --- a/static/docs/tricentis-tosca-integration-testmu-selenium-grid.md +++ /dev/null @@ -1,137 +0,0 @@ -# Tricentis Tosca Integration With TestMu AI’s Cloud Based Selenium Grid - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI provides support with Selenium WebDriver for pacing the execution of your automation test scripts. TestMu AI is a cloud-based, cross browser testing tool, providing a [Selenium Grid](https://www.testmuai.com/selenium-automation/) of 3000+ browsers and browser versions running through real operating systems to speed up automation testing of your web-app or website. This topic will help you to automate your website testing using Tricentis Tosca on TestMu AI. - -This topic will be focusing on: - -- Running a sample Tricentis Tosca testcases on TestMu AI Automation. - -## Prerequisites For Running Tricentis Tosca Test Cases On Selenium Grid - -1. Log in to TestMu AI. If you haven’t signed up yet, click here. -2. TestMu AI Authentication Credentials: - Be aware of your TestMu AI authentication credentials i.e. your TestMu AI username, access key and HubURL.You can retrieve them from your [TestMu AI automation dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/) by clicking on the key icon near the help button. -3. [Download Tosca](https://www.tricentis.com/software-testing-tool-trial-demo/) by logging in to the Tosca website. - -## Sample Test Scenario - -For demonstrating Tosca-TestMu AI integration, we would be considering the following test scenario: - -1. Open [TestMu AI Selenium PlayGround](https://www.testmuai.com/selenium-playground/). -2. Click the "Simple Form Demo" link. -3. Enter the text "Happy Testing" in the text box. -4. Click the "Get Checked value" button. -5. Verify if the entered text is present under the "Your Message" section. - -## Integrating Tricentis Tosca With TestMu AI - -**Step 1:** Launch the Tosca Commander. - -**Step 2:** Click **Create new** . In **Create new workspace**, select **Type of Repository** as **None** and set workspace as **Tosca_Demo**. - -**Step 3:** On the successful project creation, click the **Modules** tab, right-click on the panel, then select **Scan -> Application**. - -**Step 4:** Open Chrome browser and enter [test URL](https://www.testmuai.com/selenium-playground/). - -**Step 5:** On the **Select Application** screen, select the Chrome browser instance that was open in the earlier step and click **Scan** - -**Step 6:** Now you will be prompted to install Tosca Extension. - -**Step 7:** Click on **Advanced** in Tosca XScan Window. - -**Step 8:** Click **Select on screen**. - -**Step 9:** Select the **Simple Form Demo** link and Tosca XScan will capture the highlighted element. - -**Step 10:** Click **Finish Screen**. - -**Step 11:** In the browser click the **Simple Form Demo** link, navigate to the **Simple Form Page**. ReOpen Tosca XScan ,Click **Select on screen** and capture required elements for test case.(input text box, "Get Checked value" button and text area to validate) - -**Step 12:** On completion click **Finish Screen** - All the captured elements are listed under Modules are below: - -**Step 13:** Go to TESTCASES and create a folder named **TestMu AI_TestCase**. - -**Step 14:** Drag the **Modules** created in the earlier steps to the new test case **Details** tab. - -**Step 15:** To add commonly used modules in the test case, go back to the **Modules** tab. Click **HOME** in the menu bar, then click **Import Subset**. - -**Step 16:** Select **standard.tsu** file from *<installation_path>/Tosca_Projects/ToscaCommander* folder. - -**Step 17:** Navigate back to TestCases Details tab, update the test step names: - - **New TestCase** to LT_PlayGroudTest_1 - - **New TestCase_1** to LT_PlayGroudTest_2 - -**Step 18:** Select the first test case and press **CTRL + T**. This will display a list of all the common modules. Type **OpenUrl** in the search box and select the **OpenUrl** module to add to the test case. - -**Step 19:** Again press **CTRL + T** type **CloseBrowser** in the search box, select **CloseBrowser** module to add to the test case. - -Shown below is the final test case view. - -**Step 20:** Enter relevant details in **Value** and **ActionMode** columns for every test step - -**Step 21:** For adding test configuration, navigate to the **TestCase -> Test Configuration**, right-click and click **Create Test configuration parameter** button. - -**Step 22:** Add below name and values in Test Configuration: - -**For Web Browser Automation** - -**For Mobile Browser Automation (Emulators & Simulators)** - -**For Mobile Browser Automation (Real Device)** - -* Enter relevant details in **Value** and **ActionMode** columns for every test step. - -* Add below name and values in Test Configuration. - -*You can refer our [Desired Capability Generator](https://www.testmuai.com/capabilities-generator) - to choose different browser combination.* - -**Step 23:** Go to the **Details** tab then Right-click on the test case and select **Run in ScratchBook**. This will launch the test on TestMu AI Platform. - -Once execution is completed result window populate in Tosca - -Monitor and analyze your test result on the [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/). - -## How To Use TestMu AI Custom Capabilities in Tosca -There are few custom capabilities like network logs,console logs in TestMu AI platform to enable Logs on execution. - -To use a WebDriver capability, - -- Prefix the capability with "WebDriver_"*, for ex: WebDriver_console. -- If the name of the capability contains a dot (.), replace this character with double underscore (\__). If the name of the capability contains a colon (:), replace this character with triple underscore (___). - -With this enabled log when you execute the test case, you can see logs in TestMu AI Plateform once execution completes. - -## Geolocation Testing in Tosca with TestMu AI - -Test the performance of your web-app through different geographic locations by setting LamdaTest Capability "geoLocation". Set this Capability in Tosca as "WebDriver_geoLocation"with Value of "IN"as shows below - -In TestMu AI Plateform , You can see check the log to validate the Web Application is launched with given GeoLocation. - -## Testing Locally Hosted or Privately Hosted Projects - -To help you perform [cross browser testing](https://www.testmuai.com) of your locally stored web pages, TestMu AI provides an SSH(Secure Shell) tunnel connection with the name Lambda Tunnel. With Lambda Tunnel, you can execute a test server inside your GitLab CI build a container to perform automated cross browser testing on browsers offered by Selenium grid on TestMu AI. So you make sure how well your changes look, even before your customers. -Curious to know more about Lambda Tunnel? - -Follow our documentation on Lambda Tunnel to know it all. OS-specific instructions to download and set up tunnel binary are at the following links. - -- [Documentation For Windows User](/docs/local-testing-for-windows/) -- [Documentation For Mac User](/docs/local-testing-for-macos/) -- [Documentation For Linux User](/docs/local-testing-for-linux/) - -**Download the binary file of:** - -- [Lambda Tunnel for Windows](https://downloads.lambdatest.com/tunnel/v3/windows/64bit/LT_Windows.zip) -- [Lambda Tunnel for Mac](https://downloads.lambdatest.com/tunnel/v3/mac/64bit/LT_Mac.zip) -- [Lambda Tunnel for Linux](https://downloads.lambdatest.com/tunnel/v3/linux/64bit/LT_Linux.zip) - -Set the Tunnel Capability in Toasca as shown below - -In TestMu AI logs you can validate whether Tunnel is enaled or not from Tosca as shown below. - -In case you face any problems during the integration, or have any other questions, reach out to us on our **24/7 chat support**. - -Happy testing! diff --git a/static/docs/troubleshoot-espresso-tests.md b/static/docs/troubleshoot-espresso-tests.md deleted file mode 100644 index 3e9515d81..000000000 --- a/static/docs/troubleshoot-espresso-tests.md +++ /dev/null @@ -1,38 +0,0 @@ -# Troubleshoot Your Espresso Tests - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This document is a guide to troubleshooting common errors encountered when running Espresso tests on TestMu AI. Espresso is an open-source Android UI testing framework that helps you automate tests for your mobile applications. Understanding these errors and their explanations can help you diagnose and resolve issues during your test execution on TestMu AI. - -## Espresso Build Errors - -This table highlights the errors encountered when running tests on [TestMu AI Real Devices without shards](/support/docs/getting-started-with-espresso-testing/). - -| Build Execution Errors | Root Cause of Error | -|--------------------------------------------|--------------------------| -|Application under test provided by you did not get installed. Please check the application.| The target application failed to install on the selected device during test execution. This indicates an issue with the provided application file or incompatibility with the chosen device configuration.| -|Test Suite provided by you did not get installed. Please check the application.| This indicates that the test suite you provided could not be installed on the chosen TestMu AI environment. This might be due to invalid test suite files, incompatibility issues, or missing dependencies.| -|Failed to fetch runner class for test extraction. Please recheck your test suite.| Test discovery failed. The system couldn't extract test cases or classes from the provided test suite application. This might be due to issues with the test suite app itself or its configuration.| -|Failed to extract classes or tests from runner app - test discovery failed.| Test discovery failed. The system couldn't extract test cases or classes from the provided runner application. This might be due to issues with the test suite app itself or an incompatibility with the testing framework used. | -|No tests found in the test suite. Please check your test suite or applied filters.| The test execution framework couldn't locate any test cases to run due to: Empty Test Suite: Ensure your test suite contains at least one test class with a @Test annotated method. Incorrect Filtering: Verify that any applied test filters aren't accidentally excluding all tests. You can see more on espresso test filtering [here](/support/docs/speedup-espresso/).| -|Tests could not be run as localization setup failed. Please check locale and try again.| The test suite encountered a localization setup error. This means the system's locale or language settings could not be configured correctly. Please verify your locale settings and try re-running the tests. | -|Oops! An error occurred at our end. Please try again.| A temporary infrastructure issue arose. While a device was allocated for your test, it became unavailable before the test execution started. Please retry the test or reach out to support@testmuai.com if the issue persists. | -|Desired Capabilities Error. Please check the desired capabilities that you have passed.| The test encountered a [`Desired Capabilities`](/support/docs/getting-started-with-espresso-testing/#capabilities-supported) Error. This indicates an issue with the configuration provided for the test execution. Please verify the values you have set for desired capabilities like device, platform, or application path. | - -## Espresso via HyperExecute Shard Errors - -This table highlights the errors encountered when running espresso tests with [Shards via HyperExecute](/support/docs/sharding-espresso-rd-hyperexecute/) on TestMu AI Real Device Cloud. - -| Shard Execution Errors | Root Cause of Error | -|--------------------------------------------|--------------------------| -|Application under test provided by you did not get installed. Please check the application.| The target application failed to install on the selected device during test execution. This indicates an issue with the provided application file (APK/IPA) or incompatibility with the chosen device configuration.| -|Test Suite provided by you did not get installed. Please check the application.| This indicates that the test suite you provided could not be installed on the chosen TestMu AI environment. This might be due to invalid test suite files, incompatibility issues, or missing dependencies.| -|Failed to fetch runner class for test extraction. Please recheck your test suite.| Test discovery failed. The system couldn't extract test cases or classes from the provided runner application. This might be due to issues with the runner app itself or its configuration.| -|Failed to extract classes or tests from runner app - test discovery failed.| Test discovery failed. The system couldn't extract test cases or classes from the provided runner application. This might be due to issues with the runner app itself or an incompatibility with the testing framework used. | -|No tests found in the test suite. Please check your test suite or applied filters.| The test execution framework couldn't locate any test cases to run due to: Empty Test Suite: Ensure your test suite contains at least one test class with a @Test annotated method. Incorrect Filtering: Verify that any applied test filters aren't accidentally excluding all tests. You can see more on espresso test filtering [here](/support/docs/speedup-espresso/).| -|Tests could not be run as localization setup failed. Please check locale and try again.| The test suite encountered a localization setup error. This means the system's locale or language settings could not be configured correctly. Please verify your locale settings and try re-running the tests. | -|Oops! An error occurred at our end. Please try again.| A temporary infrastructure issue arose. While a device was allocated for your test, it became unavailable before the test execution started. Please retry the test. | -|Desired Capabilities Error. Please check the desired capabilities that you have passed.| The test encountered a [`Desired Capabilities`](/support/docs/getting-started-with-espresso-testing/#capabilities-supported) Error. This indicates an issue with the configuration provided for the test execution. Please verify the values you have set for desired capabilities like device, platform, or application path. | -| Oops! An error occurred at our end. Please try again. | An internal error occurred while retrieving configuration details for the test execution environment (HyperExecute API). A temporary glitch might be preventing communication with the API. Please retry the test execution. If the issue persists, contact support for further assistance. | -|Build has been stopped. | The build process was terminated prematurely. User intervention caused this stoppage, likely due to errors encountered during the build phase. | -|Build breached queue timeout| The test execution encountered a "Build breached queue timeout" error. This indicates the build exceeded the maximum allowed wait time while in a queue. This could be due to high system load or insufficient resources on the TestMu AI platform. | diff --git a/static/docs/troubleshooting-ios-app-testing.md b/static/docs/troubleshooting-ios-app-testing.md deleted file mode 100644 index 42f50f397..000000000 --- a/static/docs/troubleshooting-ios-app-testing.md +++ /dev/null @@ -1,61 +0,0 @@ -# Troubleshooting iOS App Testing - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -While uploading `.app` files on Simulators, you may encounter issues such as the app crashing immediately or the error *"No .app found inside zip"*. These problems typically occur when the app is not built correctly for the Simulator platform. - -For a complete step-by-step guide on building your iOS app for Simulator testing, see [Building Your iOS App for Simulator Testing](/docs/build-ios-app-for-simulator-testing/). - -## Common Issues - -### App crashes immediately with "App quit unexpectedly" - -**Cause:** Your `.app` is built for physical iOS devices (Mach-O Platform 2: `iphoneos`) instead of the iOS Simulator (Platform 7: `iphonesimulator`). - -**Solution:** Rebuild your app targeting the iOS Simulator SDK. In Xcode, select a **Simulator destination** (e.g., *iPhone 15 Pro*) instead of *Any iOS Device*, then build. For command line builds, use the `-sdk iphonesimulator` flag: - -```bash -xcodebuild -project YourProject.xcodeproj \ --scheme YourScheme \ --configuration Debug \ --sdk iphonesimulator \ --arch arm64 \ -build -``` - -You can verify the platform using: - -```bash -vtool -show-build YourApp.app/YourApp -``` - -The output should show `platform: IOSSIMULATOR` (Platform 7). - -Your Simulator build must include the **arm64** architecture. There is no need to exclude it from Simulator builds. - -### "No .app found inside zip" error - -**Cause:** The `.zip` archive structure is incorrect. The `.app` bundle should be at the root of the archive. - -**Solution:** -1. Build your app for the Simulator (as described above). -2. Locate the `.app` file in Xcode: right-click the `.app` under **Products** and select **Show in Finder**. -3. Right-click the `.app` file in Finder and select **Compress** to create the `.zip` file. - -Your `.zip` file is now ready for upload on TestMu AI servers. - -### Build fails with architecture errors - -**Cause:** The project may have incorrect architecture settings. - -**Solution:** In Xcode **Build Settings**, verify: - -| Setting | Value | -|---|---| -| **Architectures** | `$(ARCHS_STANDARD)` | -| **Base SDK** | iOS | -| **Supported Platforms** | iOS Simulator | - -Ensure you are **not** excluding arm64 from Simulator builds. - -That's all! In case you have any questions or need any additional information, you could reach out at our **24X7 Chat Support** or mail us directly at support@testmuai.com. diff --git a/static/docs/troubleshooting-lambda-tunnel.md b/static/docs/troubleshooting-lambda-tunnel.md deleted file mode 100644 index 123083c62..000000000 --- a/static/docs/troubleshooting-lambda-tunnel.md +++ /dev/null @@ -1,103 +0,0 @@ -# Troubleshooting TestMu AI Tunnel - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI Tunnel helps in establishing an SSH connection between your local machine and our cloud servers to help you perform tests on locally hosted websites and web-apps. With TestMu AI Tunnel, you can ensure how robust your website rendering is across 3000+ real browsers, even before you make it live on the internet. - -If due to some unfortunate reason, you are unable to perform cross browser testing using TestMu AI Tunnel then this document will help you troubleshoot the most common challenges. - -Oops !! Facing issue while testing your locally hosted application through TestMu AI Tunnel? Here are a few guidelines for you. - -Before proceeding, here are some of the common issues: - -- [Localhost refused to connect](/docs/troubleshooting-lambda-tunnel/#1-localhost-refused-to-connect) -- [Invalid Host Header](/docs/troubleshooting-lambda-tunnel/#2-invalid-host-header) -- [WordPress - CSS not loading](/docs/troubleshooting-lambda-tunnel/#3-wordpress---css-not-loading) -- [IP whitelisting](/docs/troubleshooting-lambda-tunnel/#4-ip-whitelisting) -- [No available PORT found](/docs/troubleshooting-lambda-tunnel/#5-no-available-port-found) -- [Custom Host name](/docs/troubleshooting-lambda-tunnel/#6-custom-host-name) -- [CUI – Console UI](/docs/troubleshooting-lambda-tunnel/#7-still-got-issues-let-us-help-through-cuiconsole-ui) -- [Unable to establish a secure shell tunnel connection through Port 443](/docs/troubleshooting-lambda-tunnel/#8-unable-to-establish-a-secure-shell-tunnel-connection-through-port-443) -- [LT can’t be opened because Apple cannot check it for malicious software](/docs/troubleshooting-lambda-tunnel/#9-lt-cant-be-opened-because-apple-cannot-check-it-for-malicious-software) - -## 1. Localhost Refused To Connect - -After configuring the tunnel, you might get into the error similar to the below screenshot: - -The error "localhost refused to connect" occurs because of using the URL as localhost which is unfortunately not compatible with various browsers and browser versions. We are deliberately eliminating the URL localhost and recovering it with localhost.lambdatest.com or your local system IP. - -For example you might find above error with URL: `https://localhost/demo.html` - -however, URL: `https://localhost.lambdatest.com/demo.html` or `10.0.0.15/demo.html` would definitely work fine for you. - -## 2. Invalid Host Header - -While testing an application hosted on your local machine, you may receive an "Invalid Host Header" error message when testing on TestMu AI using a local connection. This error is most commonly caused by a misconfiguration of the application server that causes it to reject non-local connections or reject requests directed at hostnames other than localhost(or any custom one set) - -Since, web applications are now built with various different kinds of frameworks like angular, react etc; In order to test it using tunnel the command used to run your web app needs to modified in your project manifest file i.e. "package.json" - -- **Angular Framework Project**: For Angular framework based web app, ideally you might be using "ng serve" or "npm start" to run your web application or your package.json file configuration would be set as "ng serve" in order to start your web app and as usual, it would be working fine on local browser. However, this might throw an "Invalid Host Header"error or error something related to the invalid host in a Real Time Test. - -To resolve this, here is the quick small solution for you. While running your application, you can either use command `ng serve --host 0.0.0.0 --disable-host-check` or set your start configuration in package.json file as `"ng serve --host 0.0.0.0 --disable-host-check"` instead of changing your running command. - -- **React Framework Project**: For React framework based web app, you need to create an env file in which you have to add the hostname,syntax: `HOST=< hostname >`, once you are done with it, now you can add your localhost IP along with your hostname in your system hosts file. - -Example: `127.0.0.1 < hostname >` - -This would help you avoiding "Invalid Host Header" error. - -## 3. WordPress - CSS Not Loading - -While testing your local web app built through WordPress, you may find issue with the rendering of CSS, similar to the below screenshot: - -The general solution for this is to update the WordPress Address and Site Address on the General Settings tab in the WordPress dashboard with your system IP address rather than the default of localhost, then everything will be emitted relative to that. Here is a screenshot for your further reference: - -By applying the above changes, you would find the CSS loading issue as resolved. Below is a screenshot after the mentioned changed: - -## 4. IP Whitelisting - -If you are testing a server that requires IP whitelisting, then you just have to set up a Local Testing connection and whitelist the below few IP’s for the respective domain: - -`https://ts.lambdatest.com/` - -- 199.58.84.59 -- 23.82.88.184 -- 23.106.34.219 -- 23.106.54.77 -- 3.214.241.254 -- 52.36.84.247 -- 13.126.37.58 -- 3.66.78.89 - -## 5. Custom Host Name - -Because of some firewall restrictions, your web application might not be accessible on some other servers even after whitelisting the IP’s and configuring the tunnel, in such case you need to make an entry in the hosts file which is used to map hostnames to IP addresses. With the hosts file, you can change the IP address that you resolve a given domain name. This change only affects your own computer without affecting how the domain is resolved worldwide. - -This is particularly useful when you wish to see how a website will look like when hosted on a different server without making any DNS changes to your domain. - -The location of the hosts file, depending on the operating system that you are using, is: - -- **Windows** – SystemRoot > system32 > drivers > etc > **hosts** -By default the system root is C:\Windows, so if you are using Windows, your **hosts** file is most probably:C:\Windows\System32\drivers\etc\ **hosts**) -- **Linux** – /etc/**hosts** -- **Mac OS X** – /private/etc/**hosts** - -Let’s say that you wish to resolve **mydomain.com** to the IP address **10.20.30.40**. In this case, you would need to open up the hosts file with a text editor and append the following line: -10.20.30.40 mydomain.com www.mydomain.com - -## 6. Enable Verbose Logging - -The `- v` flag enables Verbose logging on the console. You can use this to log the HTTP headers & requests or to debug the TestMu AI Tunnel connection. - -The requests are also logged into a log file `lt.log` which is created in the same working directory as the TestMu AI Tunnel binary. You can also specify the tunnel log file name using the `--logFile` flag. - -## 7. LT Can’t Be Opened Because Apple Cannot Check It For Malicious Software - -Catalina OS users may get the below error from the command line while trying to configure the TestMu AI Tunnel to test their locally hosted web pages: -"LT can't be opened because Apple cannot check it for malicious software" - -This error occurs due to the increased restrictions on third-party software, downloaded from the internet. Since Catalina OS is recently launched by Apple, we are working to make our TestMu AI Tunnel compatible with the new OS. - -While we are at it, here is a one-time setup workaround. All you need to do is run the below command before you start configuring your TestMu AI Tunnel: - -`xattr -d com.apple.quarantine ./LT` diff --git a/static/docs/tunnel-allowHost-wildcard-support.md b/static/docs/tunnel-allowHost-wildcard-support.md deleted file mode 100644 index 9f588b956..000000000 --- a/static/docs/tunnel-allowHost-wildcard-support.md +++ /dev/null @@ -1,33 +0,0 @@ -# Support for Wildcard in --allowHosts Flag - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI Tunnel now supports wildcard entries in the [`--allowHosts`](/support/docs/lambda-tunnel-modifiers/#:~:text=TYPE-,%2D%2DallowHosts,-Comma%20separated%20list) flag. This enhancement allows users to specify patterns for hostnames instead of listing each subdomain separately, making it more flexible and efficient. - -## Usage of `--allowHosts` with Wildcards -The `--allowHosts` flag enables users to define which domains should be routed through the TestMu AI Tunnel while allowing other traffic to bypass it. With the new wildcard support, you can match multiple subdomains easily. - -### Formatting Domains with Wildcards -Follow these guidelines when using wildcards in domain names: - -#### Use only the domain name -- Do not include `http://` or `https://`. -- ✅ Example: `example.com`.        - -#### Comma-separated list without spaces -- Ensure that the list of domains is comma-separated with no spaces. -- ✅ Example: `example.com`, `testsite.com`, `anotherdomain.com` - -#### Using Wildcards to Match Subdomains -- Prefix the domain with a dot `(.)` to match all its subdomains. -- Alternatively, you can use an asterisk `(*)` for the same effect. -- ✅ Example: - - `".example.com"` or `"*.example.com"` will match `sub.example.com`, `api.example.com`, `blog.example.com`, etc. - -**Note** -Enclose the argument in **quotes** to prevent shell expansion of the **asterisk (*)**. - -## Benefits of Wildcard Support in `--allowHosts` -- **Simplifies Configuration :** No need to manually list each subdomain. -- **Increased Flexibility :** Easily manage multiple subdomains with a single entry. -- **Better Performance :** Reduces manual domain management, improving efficiency. diff --git a/static/docs/tunnel-release-notes.md b/static/docs/tunnel-release-notes.md deleted file mode 100644 index dd4d5e48c..000000000 --- a/static/docs/tunnel-release-notes.md +++ /dev/null @@ -1,97 +0,0 @@ -# Tunnel Client Release Notes - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Version 3.2.31 (19th July 2026) -- **TCP Tunnel Reliability** - - The tunnel now waits for a healthy data-path heartbeat before marking itself ready, and automatically falls back to WebSocket/SSH when the TCP data path is unhealthy. This prevents intermittent HTTP 502 errors caused by egress proxies that accept the initial connection upgrade but drop the multiplexed traffic. -- **Auto-Update Stability** - - Bounded the client auto-update retries to prevent restart loops when an update cannot be applied (e.g. a read-only binary path or an unpublished version). The client now retries a limited number of times and keeps running the current binary instead of restarting indefinitely. - -## Version 3.2.29 (14th June 2026) -- **gRPC / HTTP/2 Tunnel Fix** - - Fixed gRPC-over-HTTP/2 traffic hanging through the tunnel for strict HTTP/1.1 CONNECT clients (e.g. Flutter / grpc-dart apps on real devices). The tunnel now replies `HTTP/1.1 200` on the passthrough CONNECT path, so these clients establish the tunnel instead of timing out. -- **Windows System Proxy Auto-Detection** - - Auto-detect the Windows system PAC (`Use setup script`) and manual proxy when no proxy flags or `--pacfile` are provided. -- **Security Updates and Stability Fixes** - - Updated `golang.org/x/net` and `golang.org/x/crypto` dependencies to address reported CVEs. - -## Version 3.2.28 (24th May 2026) -- **Security Updates and Stability Fixes** - - Security Fixes for CVE - - Fixed SOCKS5 proxy authentication when --proxy-user / --proxy-pass are supplied - -## Version 3.2.27 (25th April 2026) -- **Security Updates and Stability Fixes** - - Security and vulnerability fixes - -## Version 3.2.26 (8th March 2026) -- **Security Updates and Stability Fixes** - - Security fixes and code sanity - -## Version 3.2.25 (28th Dec 2025) -- **Security Updates and Stability Fixes** - - Security Fixes for CVE - -## Version 3.2.24 (7th Dec 2025) -- **Security Updates and Stability Fixes** - - Security Fixes for CVE - -## Version 3.2.23 (16th Nov 2025) -- **Security Updates and Stability Fixes** - - Implemented critical security updates - - TCP connection mode improvements - - TCP mode(with TLS 1.2+) is now the default connection mode. - -## Version 3.2.22 (2nd Nov 2025) -- **Security Updates and Stability Fixes** - - Implemented critical security updates - - Fixes in Tunnel Auto Upgrade process. - - Enhancement in tunnel connection process. - -## Version 3.2.20 (24th August 2025) -- **Security Updates and Stability Fixes** - - Implemented critical security updates - -## Version 3.2.19 (10th August 2025) -- **Security Updates and Stability Fixes** - - Support SHA-256 signed certificates within PFX files for mTLS authentication - -## Version 3.2.17 (13th July 2025) -- **Security Updates and Stability Fixes** - - Added fix for insecure websocket connection handling affecting locally hosted applications - - Implemented critical security updates - -## Version 3.2.16 (8th June 2025) -- **Support for forceLocal, contextual Logging** - - Added the ability to forcelocal, all traffic is resolved at local. - - Public websites will be resolved at Data center in selenium automation. - - Contextual Logging is enabled in selenium automation. - -## Version 3.2.14 (30th March 2025) -- **Support for WildCard in `ntlm-domains`** - - Added the ability to use wildcards in the `ntlm-domains`. - -## Version 3.2.13 (2nd March 2025) -- **Auto Downloading and Restarting the binary for new Updates** - - Added the ability to restarting the binary itself after downloading the update. - -## Version 3.2.11 (2nd February 2025) -- **Support for Wildcard in `allowHosts` and `bypassHosts`** - - Added the ability to use wildcards in the `allowHosts` and `bypassHosts` configurations, providing more flexibility in defining tunnel access rules. - -## Version 3.2.10 (18th December 2024) -- **Security Certificate Update** - - Updated security certificates to ensure enhanced security compliance and prevent potential vulnerabilities. - -## Version 3.2.9 (24th November 2024) -- **Logger Level Configurability** - - Introduced support for configuring logger levels in the Tunnel Client. This allows users to customize the verbosity of logs based on their requirements. - -## Version 3.2.8 (22nd December 2024) -- **Fix for Tunnel Stop Issue** - - Resolved a bug that caused issues when stopping the tunnel, ensuring smoother operation during tunnel termination. - -## Version 3.2.7 (25th October 2024) -- **Security Updates and Stability Fixes** - - Implemented critical security updates and resolved stability issues to enhance the overall performance and reliability of the Tunnel Client. diff --git a/static/docs/tunnel-security.md b/static/docs/tunnel-security.md deleted file mode 100644 index 62e9775f6..000000000 --- a/static/docs/tunnel-security.md +++ /dev/null @@ -1,39 +0,0 @@ -# Tunnel Security - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI allows you to configure various sharing and security options for TestMu AI Tunnel. With these options, you can restrict tunnel creation to specific users, turn tunnel sharing on or off and control which domains are accessible (or inaccessible) through the tunnel. - -> **Note:** Only administrators are able to view and manage these settings in the security tab of the organization settings page. - -## Restrict Tunnel Creation Permission - -With the Tunnel Creation option, administrators can choose who has the permission to start a tunnel in your organization. This can ensure that only authorized individuals are allowed to start a tunnel, for example, only admin accounts. By default, all users in the organization can create a tunnel. To allow only for specific users, follow these steps: - -**Step 1:** Go to the [TestMu AI Dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard). - -**Step 2:** Click on the **Settings** -> **Organization Settings**, navigate to **Security** tab and click on **Tunnel Security** button. - -**Step 3:** Click **Selected Users** in the **Tunnel creation** section. - -**Step 4:** Click **Yes, Continue** in the pop-up to continue permission change - -**Step 5:** Add users who will be able to create tunnel and confirm - -Tunnel creation is now restricted to only selected users. To remove users, click on the three dot menu in the table on the right side. To enable tunnel creation for all users, click **All Users** and confirm. - -## Enable/Disable Tunnel Sharing - -Tunnel sharing is a feature where multiple users can share a single tunnel instance. This is enabled by default. To disable tunnel sharing, click the toggle as shown. To enable it back, the same toggle can be used. - -## Setup Domain Controls - -Domain controls enable the allowing or disallowing access of specific domains through tunnel. The two options available are - Whitelisting and Blacklisting. With whitelisting, only the selected domains will be accessible through the tunnel. With blacklisting, access to selected domains will not be allowed through the tunnel. To set it up, the following steps can be followed: - -**Step 1:** In the **Security** tab of **Organisation Settings**, click on **Tunnel Security** - -**Step 2:** In the **Domain controls** type option, choose Whitelist or Blacklist based on the need. - -**Step 3:** Choose Yes, Continue in the pop-up and add all the domains that are to be whitelisted/blacklisted. - -> **NOTE :** Websites that are not publicly accessible and can only be accessed through the tunnel, such as locally hosted sites, can utilize either Blacklisting or Whitelisting. This domain control is not applicable to websites that are publicly accessible. diff --git a/static/docs/typescript-with-playwright.md b/static/docs/typescript-with-playwright.md deleted file mode 100644 index 02fa47a2f..000000000 --- a/static/docs/typescript-with-playwright.md +++ /dev/null @@ -1,173 +0,0 @@ -# TypeScript with Playwright: Running Your First Test - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Learn how to use Playwright with TypeScript to automate web application testing across 50+ real browsers and operating systems on TestMu AI cloud platform. - -## Prerequisites - -1. You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** -Download or clone the code sample for the Playwright TypeScript from the TestMu AI GitHub repository to run the tests. - - View on GitHub - -```js -git clone https://github.com/LambdaTest/playwright-sample.git -cd playwright-sample -cd playwright-test-ts -``` - -2. Install the npm dependencies. - -``` -npm install -``` - -3. A TestMu AI Username and Access key. You can get it from your TestMu AI Profile section. Don't have an account, sign up for free. - -4. To run Playwright tests, set your TestMu AI Username and Access key in the Environment Variables. - -## Run your Playwright tests with TypeScript - -Navigate to the `lambdatest-setup.ts` file in the `playwright-test-ts` directory. - -```js -/** -* Add the file in your test suite to run tests on LambdaTest. -* Import `test` object from this file in the tests. -*/ - -import * as base from "@playwright/test"; -import path from "path"; -import { chromium } from "playwright"; - -// LambdaTest capabilities -const capabilities = { -browserName: "Chrome", // Browsers allowed: `Chrome`, `MicrosoftEdge`, `pw-chromium`, `pw-firefox` and `pw-webkit` -browserVersion: "latest", -"LT:Options": { -platform: "Windows 10", -build: "Playwright TypeScript Build", -name: "Playwright TypeScript Test", -user: process.env.LT_USERNAME, -accessKey: process.env.LT_ACCESS_KEY, -network: true, -video: true, -console: true, -tunnel: false, // Add tunnel configuration if testing locally hosted webpage -tunnelName: "", // Optional -geoLocation: '', // country code can be fetched from https://www.lambdatest.com/capabilities-generator/ -}, -}; - -// Patching the capabilities dynamically according to the project name. -const modifyCapabilities = (configName, testName) => { -let config = configName.split("@lambdatest")[0]; -let [browserName, browserVersion, platform] = config.split(":"); -capabilities.browserName = browserName -? browserName -: capabilities.browserName; -capabilities.browserVersion = browserVersion -? browserVersion -: capabilities.browserVersion; -capabilities["LT:Options"]["platform"] = platform -? platform -: capabilities["LT:Options"]["platform"]; -capabilities["LT:Options"]["name"] = testName; -}; - -const getErrorMessage = (obj, keys) => -keys.reduce( -(obj, key) => (typeof obj == "object" ? obj[key] : undefined), -obj -); - -const test = base.test.extend({ -page: async ({ page, playwright }, use, testInfo) => { -// Configure LambdaTest platform for cross-browser testing -let fileName = testInfo.file.split(path.sep).pop(); -if (testInfo.project.name.match(/lambdatest/)) { -modifyCapabilities( -testInfo.project.name, -`${testInfo.title} - ${fileName}` -); - -const browser = await chromium.connect({ -wsEndpoint: `wss://cdp.lambdatest.com/playwright?capabilities=${encodeURIComponent( -JSON.stringify(capabilities) -)}`, -}); - -const ltPage = await browser.newPage(testInfo.project.use); -await use(ltPage); - -const testStatus = { -action: "setTestStatus", -arguments: { -status: testInfo.status, -remark: getErrorMessage(testInfo, ["error", "message"]), -}, -}; -await ltPage.evaluate(() => {}, -`lambdatest_action: ${JSON.stringify(testStatus)}`); -await ltPage.close(); -await browser.close(); -} else { -// Run tests in local in case of local config provided -await use(page); -} -}, -}); - -export default test; -``` - -Pass the below command in the terminal to run the test. - -```js -npm run test -``` - -## View your test results - -Go to the [TestMu AI Web Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) to see your Playwright TypeScript test results. - -## Using the Playwright Agent Skill with TestMu AI - -The [playwright-skill](https://github.com/LambdaTest/agent-skills/tree/main/playwright-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The playwright-skill package includes: - -``` -playwright-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing Playwright Agent Skill - -Install a Playwright Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/playwright-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/playwright-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only playwright-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). diff --git a/static/docs/ui-inspector.md b/static/docs/ui-inspector.md deleted file mode 100644 index 26be2847d..000000000 --- a/static/docs/ui-inspector.md +++ /dev/null @@ -1,49 +0,0 @@ -# UI Inspector - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI UI App Inspector provides a graphical interface to inspect and interact with the elements of a mobile application. When you launch your mobile app, it retrieves the DOM (Document Object Model) of the app. This DOM includes various UI elements such as buttons, text fields, labels, etc., organised in a hierarchical structure. - -## Benefits of UI Inspector - -- **DOM Representation:** It visually represents the DOM of the mobile application in a tree-like structure. This representation helps testers and developers understand the layout and structure of the app's UI. -- **Visual Representation:** It provides a graphical interface where you can expand and collapse nodes to view the elements nested within other elements. You can also select individual elements within the DOM by tapping on them. -- **Element Identification:** When you select an element, it provides information about that element, such as its type, attributes, and location within the DOM. This information helps in identifying elements uniquely for automation purposes. -- **Automation:** It aids in generating Xpath by providing information about the elements in the DOM. Test scripts can locate and interact with UI elements by referencing their properties in the DOM. For example, you can create Xpath to tap on a button by specifying its unique identifier, XPath expression, or accessibility label within the DOM. - -TestMu AI UI Inspector is divided into various components such as - -- Search Bar -- Hierarchy View -- Properties Table -- Application Control -- Screenshot View - -Let's have a detailed look at each of these components: - -### Search Bar - -- The Search Bar is a feature that allows users to quickly locate specific elements within the mobile application's DOM (Document Object Model). -- Users can enter locator strategies in the Search Bar to find the corresponding elements within the DOM. - -### Hierarchy View - -- The Hierarchy View provides a visual representation of the DOM structure of the mobile application. -- It presents the elements in a hierarchical tree-like format, where parent-child relationships are clearly depicted. -- You can also select and inspect an hybrid app which has a webview using the drop down. - -### Properties Table - -- The Properties Table displays detailed information about the selected UI element from the DOM. -- It includes various properties such as element type, attributes, accessibility labels, coordinates, and other relevant metadata. -- Users can refer to the Properties Table to obtain essential details about the selected element, which are instrumental in identifying and interacting with it programmatically. - -### Application Control - -- **Refresh Dump:** The "Refresh Dump" feature allows users to synchronize the inspection tool with the current state of the mobile application. -- **Relaunch App:** The "Relaunch App" feature allows users to restart the currently running instance of the mobile application. - -### ScreenshotView - -- The Screenshot View provides a comprehensive overview of the mobile application's user interface elements, presented in a visual representation typically in the form of a screenshot dump. -- Users can click on individual elements within the screenshot dump, triggering simultaneous highlighting of the corresponding element in both the Hierarchy View and the Properties Table, facilitating quick navigation and detailed inspection. diff --git a/static/docs/uipath-integration.md b/static/docs/uipath-integration.md deleted file mode 100644 index 6ad669294..000000000 --- a/static/docs/uipath-integration.md +++ /dev/null @@ -1,35 +0,0 @@ -# UiPath Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -UiPath is a leading robotic process automation (RPA) platform powered by artificial intelligence (AI), designed to help businesses streamline their operations by automating repetitive, time-consuming tasks. By leveraging AI, UiPath enables organizations to achieve greater efficiency, reduce errors, and focus on more strategic, value-driven activities. - -By integrating UiPath with TestMu AI's real device cloud, you can enhance your mobile test automation. - -## Integrating UiPath With TestMu AI - -**Step 1:** Log in to your UiPath account. From the right side, click on **Download Studio Desktop**. - -**Step 2:** Download and install the UIPath Studio and ensure that your Chrome and Edge browser are closed as it scans for them. After successful installation the dashboard will open up. - -**Step 3:** Now head to the **"New from template"** section and click on Mobile Testing. - -**Step 4:** Now enter the **project name**, **description** and other requirements and click **Create**. - -**Step 5:** Once done click on **Mobile Automation** in the header tab and select **mobile device manager** from the dropdown. - -**Step 6:** Under the **Devices** tab, click on the **plus icon** to add a new device. - -**Step 7:** Now enter the required information such as **Appium URL**, **Platform**, **Device Name**, and more. Click on the **Save and Close** option once done. - -**Step 8:** Head to the **Applications** tab and click the **plus icon** to create one. - -**Step 9:** Enter the required information such as the **app url**, **app package** and **app activity**. Click on the **Save and Close** option once done. - -**Step 10:** Now in order to run the application on a device, click on the **play icon** under the application name. - -**Step 11:** Select the device and the application from the dropdown based on the options saved. - -**Step 12:** Click on **Connect** to start. - -> That was all you need to know for TestMu AI + UiPath Integration. Increase your productivity with our integrations. If you still have any questions for us, please feel free to let us know. Our experts are always available on **chat** to help you out with any roadblock regarding our product. Happy testing! diff --git a/static/docs/underpass-tunnel-application.md b/static/docs/underpass-tunnel-application.md deleted file mode 100644 index 93ffbe9f0..000000000 --- a/static/docs/underpass-tunnel-application.md +++ /dev/null @@ -1,90 +0,0 @@ -# TestMu AI Underpass Application For Secure Tunnel - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -With TestMu AI Underpass, you can perform [cross browser testing](https://www.testmuai.com) of locally hosted web applications on 3000+ browsers for both mobile & desktop, without executing any command from your terminal. TestMu AI Underpass is a desktop application to help you establish an SSH(Secure Shell) tunnel between your operating system & our cloud servers, all through GUI. - -## Downloading & Installing The Underpass Tunnel Application - -| Important Links | -|-----------------| -| [Download Underpass For Windows](https://downloads.lambdatest.com/underpass/master/UnderPass.exe) | -| [Download Underpass For macOS](https://downloads.lambdatest.com/underpass/master/UnderPass.dmg) | -| [Download Underpass For Linux](https://downloads.lambdatest.com/underpass/master/UnderPass.AppImage) | - -You can also download the Underpass tunnel application from the TestMu AI web-application. Login to your TestMu AI account, and hit the button to configure tunnel. After that, switch to the tab "Download App" and click the button which says "Download Tunnel App". This will download the binary file using which you can perform browser compatibility testing over your locally hosted web-applications or websites. - -After downloading the Tunnel App, also known as the Underpass, you need to run it. You may get a prompt from your operating system asking you to allow Underpass through the firewall on your system. - -> Note: If you are using a Linux operating system, you need to give app executable permission using the command: - -`chmod +x UnderPass.AppImage` - -## Launching Underpass and the secure Tunnel - -#### Launching Underpass on Linux - -Make sure you’ve given the permissions to the Underpass executable file through the command highlighted in the above note. If you have, you can run the application by either double clicking it or by running the below command in terminal: - -`./UnderPass.AppImage` - -#### Launching Underpass on Windows - -Once Underpass is downloaded and installed, simply launch Underpass through the start menu or a shortcut. - -#### Launching Underpass on MacOS - -Once Underpass is downloaded and installed, simply launch Underpass through the Launchpad (Applications page). - -### Launching Tunnel - -To use TestMu AI Underpass, you need to begin by logging in with your preferred login method. Underpass supports login through Email and Password, Google, GitHub or your organisation SSO. - -After logging in, you will be presented with the home screen of Underpass. Here, you can change the tunnel name manually or using the arrow button to generate a random name. You can click launch to launch the tunnel. - -> On Windows, you may get another prompt from your firewall defender to allow LT, which is a binary file for Lambda Tunnel. Earlier, you used to trigger the LT file through your command line or terminal. Now, as you allow LT(Lambda Tunnel) the permissions, the Underpass will take care of the rest of the execution. - -Next, you just have to wait for a couple seconds while the tunnel loads. You can see the logs as well. - -You have successfully established an Secure Shell connection by running the Underpass application. You will find the status as Running. - -By clicking over the Logs button, you can find messages populated from through command line. - -> **Note:** If you don’t specify any advanced configurations while launching the tunnel, you may receive few warnings stating the environment & configuration files aren’t found. Don’t worry though, in such cases you will be automatically routed through default configurations. - -Now, if you go back to TestMu AI web-application, you will find an active tunnel connection being reflected with the same name as your Underpass tunnel application. - -## How to add Advanced Configurations? - -You can also specify **advanced configurations** such as proxy details, environment label, info API port, local server port and more before you launch the Underpass tunnel application. - -As you click on **Show Advanced Configurations**, you get the below screen. - -### Local Testing By MITM(Man-In-The-Middle) - -The MITM(Man-in-the-middle) toggle enables you to test websites using self-signed certificates on your local system or internal network. It happens very often that you may try to test a website on localhost which may not have valid SSL certificates before the website is made live. In such cases, you may receive the below error. - -You can test such websites by leveraging the MITM toggle under advanced configurations of the Underpass tunnel application. Here is a screenshot of the same website that was throwing an error earlier but can now be tested by enabling the MITM toggle. - -### Accessing A Local Directory - -With Underpass application, you can easily access a local folder or directory in your computer. All you need to do is paste the address of folder in the field "LOCAL DIRECTORY" under the advanced configurations. - -Now, when you launch the tunnel and run a Real time test over TestMu AI, you will find the files located under the local folder whose path was specified by you under the advanced configurations. - -Note: You need to specify `local-folder.lambdatest.com` as the testing URL in the real time testing. - -## Monitoring Tunnel Logs For Troubleshooting - -Through Underpass tunnel application, we’ve tried our best to provide you a bug-free experience. However, in any rare scenario, if you come across any issues around TestMu AI Underpass, you can share the tunnel logs with our 24/7 support, and we will help you out at once. - -To view the tunnel logs instantly, click on **Logs** after selecting a tunnel in the **View Tunnels** tab. You can now copy the logs using the copy button on the right hand side. - -You can find the tunnel logs stored in the following location: - -* On macOS: `~/Library/Application Support/Underpass/logs` -* On Windows: `C:\Users\%USERNAME%\AppData\Local\Underpass\logs` -* On Linux: `~/.config/Underpass/logs` - -> -That was all for TestMu AI Underpass tunnel application, in case you have any questions, feel free to share them with us through our **24/7 chat support** or drop us an email to [support@testmuai.com](mailto:support@testmuai.com). Happy testing! 🙂 diff --git a/static/docs/upload-apps-on-real-device-cloud.md b/static/docs/upload-apps-on-real-device-cloud.md deleted file mode 100644 index ce7a43901..000000000 --- a/static/docs/upload-apps-on-real-device-cloud.md +++ /dev/null @@ -1,42 +0,0 @@ -# Upload Apps on TestMu AI’s Real Device Cloud - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI allows you to upload applications either from your local system or directly from a public URL for testing on real Android and iOS devices. This ensures that your apps perform optimally across diverse user environments. - -Let's dive in to learn how to upload apps to the Real Device Cloud. - -## Supported Files and Sizes - -Upload the .apk, .aab files of your app from your system or from a public URL and ensure the size of the files is not more than 1 GB. - -Upload the .ipa files of your app from your system or from a public URL and ensure the size of the files is not more than 1 GB. - -## How to Upload Apps? - -**From Your System:** Select the **Upload** button to add an application from your system. - -**Via URL:** If your app is hosted online, you can also upload it by simply entering its public URL in the designated **Upload via URL** field. - -**Via API:** Upload apps via API onto the real device dashboard. Check out our detailed [support documentation](/support/docs/app-testing-apis/#uploading-your-application) to learn how to upload apps via API. - -## Sharing Apps - -If you want to share any app with your team members, simply click on the checkbox and then upload the app. The newly uploaded app will be visible for all the team members in their dashboards. - -Once your app is uploaded, you can see various app details such as App Name, Version, Bundle ID for iOS devices, App Package for android devices, the time and name of the person by whom it was uploaded. - -## App Settings - -You can customize the app settings based on your app requirements and testing scenarios. -Hover over the app, click on the settings icon for the app settings to open. - -- Set the **App Name** to easily identify it on the dashboard. - -- Use the **App ID** provided by Lambdatest to execute your automation scripts. - -- Control **Visibility** to manage who in your team can view or edit the app. - -- Use the **App Version Code** to differentiate between different builds. - -You can edit the information at any time and save the changes. If an app is no longer needed, you have the option to delete it permanently from your account. diff --git a/static/docs/upload-contacts.md b/static/docs/upload-contacts.md deleted file mode 100644 index c129b3fad..000000000 --- a/static/docs/upload-contacts.md +++ /dev/null @@ -1,60 +0,0 @@ -# Upload Contacts (.vcf) on Real Devices - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Preload contacts on real devices before your test session begins. Upload `.vcf` (vCard) contact files using the Upload Media API, and they are automatically injected into the device's native Contacts app when your session starts. - -This is useful for testing: -- Messaging and dialer apps -- CRM and contact management apps -- Social apps with contact picker flows -- Any app that reads from the device's contact list - -## Supported Devices - -| Platform | Minimum OS Version | -|----------|--------------------| -| Android | 10 and above | -| iOS | 14 and above | - -## Step 1: Upload a VCF File - -Upload your `.vcf` file using the Upload Media API. - -### Request - -{`curl --user "${YOUR_LAMBDATEST_USERNAME()}:${YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://api.lambdatest.com/mfs/v1.0/media/upload" -F "media_file=@"/path/to/contacts.vcf"" -F "type=contact" -F "custom_id=SampleContacts"` -} - -**Request Parameters** -- `media_file`: The `.vcf` contact file to upload from your local system. -- `type`: Set to `contact` for vCard files (Optional) -- `custom_id`: A custom identifier for the uploaded file. - -### Response - -```json -{ -"media_url": "lt://MEDIAvcf_abc123def456", -"name": "contacts.vcf", -"status": "success", -"custom_id": "my-contacts", -"contact_count": 12 -} -``` - -Use the `media_url` from the response in the next step. - -## Step 2: Set the Capability in Your Test Script - -Once the contacts are uploaded to TestMu AI's cloud, set the **uploadMedia** capability to the **media_url** parameter returned in the API response. - -```python -desired_capabilities = { -"uploadMedia": ["lt://MEDIAf446d4170cd946aa9ec307d10cb679b9", "lt://MEDIA8d13e569b3e140c18e82b066022518bd"] -} -``` - -- Each automation session permits a maximum of five file uploads. -- The contact file must be in vCard (`.vcf`) format. -- Maximum file size is 5 MB with up to 100 contacts per file. diff --git a/static/docs/upload-custom-chrome-profile.md b/static/docs/upload-custom-chrome-profile.md deleted file mode 100644 index 5733638cf..000000000 --- a/static/docs/upload-custom-chrome-profile.md +++ /dev/null @@ -1,66 +0,0 @@ -# How to Upload Custom Chrome Profile? - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Custom Chrome profiles allow you to simulate a real-user environment with specific browser settings. Learn how to upload a custom Chrome profile to run your automated tests on TestMu AI. - -## Uploading Your Custom Chrome Profile - -Zip your Chrome profile folder and upload it to the TestMu AI cloud using the API. - -### Before You Begin - -1. Create or locate the Chrome profile folder you want to use. -2. Get your API authorization credentials from your [TestMu AI dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard). - -### Steps - -1. Zip the custom Chrome profile folder and upload it to TestMu AI cloud servers using the API below: - -```bash -curl --location --request POST 'https://api.lambdatest.com/automation/api/v1/files/profile/chrome' \ - ---header 'Authorization: Basic xxxxxx' \ - ---form 'profile=@"/Users/abc/Desktop/zip.zip"' -``` - -After you upload the compressed file, a similar URL will be generated: - -```json -https://automation-prod-user-files.s3.amazonaws.com/profile/chrome/orgId-2939/zip.zip -``` - -2. Add the above generated URL in your test script using the `"browserProfile"` capability as shown below: - -```sql -"browserProfile":"https://automation-prod-user-files.s3.amazonaws.com/profile/chrome/orgId-242939/zip.zip" -``` - -## Managing Your Custom Chrome Profiles - -View or delete uploaded Chrome profiles using the API commands below. - -* **To View the List of Uploaded Profiles**: Use the below cURL command: - -```bash -curl --location --request GET 'https://api.lambdatest.com/automation/api/v1/files/profile/chrome' \ ---header 'Authorization: Basic your_auth_key' -``` - -* **To Delete the List of Uploaded Profiles**: Use the below cURL command: - -```bash -curl --location --request DELETE 'https://api.lambdatest.com/automation/api/v1/files/profile/chrome' \ ---header 'Authorization: Basic your_auth_key' \ ---header 'Content-Type: application/json' \ ---data-raw '{"key": "zip.zip"}' -``` - -## Important Notes - -Review these constraints before uploading your custom Chrome profile. - -* The compressed zip file must not exceed 100 MB. - -* Ensure that you compress the exact profile folder. Compressing the parent folder or a subfolder causes the functionality to fail. diff --git a/static/docs/upload-files-media.md b/static/docs/upload-files-media.md deleted file mode 100644 index d0be75e88..000000000 --- a/static/docs/upload-files-media.md +++ /dev/null @@ -1,71 +0,0 @@ -# Upload & Download Files and Media on Real Devices - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI’s **Files & Media** feature allows you to seamlessly upload files or media from your system to real devices and download files the app or website generated back for verification. This ensures comprehensive validation of workflows such as document generation, media export, and file handling across Android and iOS devices — during both **App** and **Browser** testing sessions. - -The same upload and download experience is available whether you are testing a native app or a website in a mobile browser, so there is nothing new to relearn when switching between the two. - -## Supported Devices - -| Platform | OS Version | -|----------|------------| -| Android | 9 and above | -| iOS | 13 and above | - -## Steps for Uploading & Downloading Files - -The Files & Media panel works the same way in both **App** and **Browser** sessions. Follow the common steps below — the only differences are the dashboard you start from and where uploaded files land on the device. - -1. **Open a Real Device Session** - Log in to your dashboard and launch a session on an Android or iOS real device: - - **App testing:** [App Live Dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard) — upload your app, then select a device. - - **Browser testing:** [Real Device Browser Dashboard](https://www.testmuai.com/login/?redirectTo=https://app.lambdatest.com/console/realtime/browser/mobile) — select a device and browser. - -2. **Use the Files & Media Panel** - From the **right-hand session toolbar**, click the **Files & Media** icon. - -### Upload - -Click **Upload** to select files or media from your local system. In a **browser** session, then tap the file input in the website under test and choose the uploaded file from the device. - -Once uploaded, files are stored in the device’s default locations. **This is the main difference between the two session types:** - -| Category | Platform | App session location | Browser session location | -|-----------------|----------|---------------------------------------------------|-------------------------------------------------------------| -| Media Files | Android | Gallery → `/sdcard/Pictures` or `/sdcard/Movies` | Downloads (reachable from the browser file picker) | -| | iOS | Camera Roll → `/private/var/mobile/Media/DCIM/` | Photo Library (reachable from the native *Photo Library* picker) | -| Non-Media Files | Android | Downloads folder | Downloads (reachable from the browser file picker) | -| | iOS | Files app → On My iPhone → Your app's directory | Files app → On My iPhone → **Chrome** (even when testing in Safari) | - -* **Target App Must Be Installed:** In manual testing, your iOS app must be installed on the device before you can upload non-media files. -* **Info.plist Configuration:** For non-media files, ensure your iOS app’s `Info.plist` file sets both `UIFileSharingEnabled` and `LSSupportsOpeningDocumentsInPlace` to `true`. This makes your app’s directory accessible in the native **Files** app under **On My iPhone → Your App’s Directory**. - -### Download - -Switch to the **Download** tab in the Files & Media panel and select the files you want to export. Click the **Download** button to save them to your system as a `.zip`, which you can then extract and verify. In a browser session, this retrieves the files the website saved on the device (Android: `Downloads`; iOS: the browser’s Downloads). - - The maximum combined download size per session is 100 MB. - -### Differences between App and Browser sessions - -Everything else is identical; only these behaviors differ: - -- **Starting point:** App sessions require uploading an app; browser sessions start directly from the Real Device Browser dashboard. -- **Upload destination on the device:** see the table above. -- **iOS documents in browser sessions** always appear under **Chrome** in the Files app — even when testing in Safari. This is by design, as only Chrome on iOS exposes a browsable file container. -- **Android `accept="image/*"` inputs** in browser sessions may open the photo picker; the default file input works across the board. - -### Automation - -Pass the **`uploadMedia`** capability — an array of uploaded media IDs (maximum 5) — to make the files available in your session. The capability is identical for **app** and **browser** (web) automation. Refer to the documentation for [Uploading Files and Media on Real Devices](appium-upload-filemedia.md). - -## Supported File Types - -The same formats and size limits apply to both **App** and **Browser** sessions: - -- **Images**: JPG, JPEG, PNG, GIF, BMP (Max: 10 MB) -- **Videos**: MP4 (Max: 50 MB) -- **Documents**: XLS, XLSX, DOC, DOCX, PDF, CSV, TXT, CRT, CER, GED, GPX, XML, PBIX, ZIP, JSON (Max: 15 MB) - -You can upload up to **5 files per session**. Files uploaded to or downloaded on the device exist only for the duration of the session and are wiped when the device is recycled. diff --git a/static/docs/upload-files-using-testmu.md b/static/docs/upload-files-using-testmu.md deleted file mode 100644 index 4376d6327..000000000 --- a/static/docs/upload-files-using-testmu.md +++ /dev/null @@ -1,146 +0,0 @@ -# Upload Files Using TestMu AI Selenium Grid - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -In Selenium automation testing, you may need to test the upload functionality on a web app or website. TestMu AI lets you test the upload feature on the cloud-based Selenium Grid across 3000+ browser and OS combinations. - -This document explains how to upload a file on the TestMu AI platform and access it via an automation test script. - -## How to Upload Files on TestMu AI Platform - -Upload files to the cloud-based Selenium Grid using the [API for uploading user-files](/api-doc/). - -1. Open [TestMu AI's API for uploading user-files](/api-doc/). -2. Click on the Lock icon to authorize your account. ] -3. Enter your TestMu AI username and access key to validate your credentials. Click the "Authorise" button to verify the credentials, and click Close to close the window. - -> Get your TestMu AI Username and Access Key from your [TestMu AI automation dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard). Click on the "key" icon in the top right corner. - -4. Click on the "Try it out" button. -5. Select the file using the "Choose File" button and click Execute to upload it on the TestMu AI platform. - -> **Note**: You can upload multiple files to lambda storage. A maximum of 150 files can be uploaded per organization. The file size limit is 20 MB per API call. If your total file sizes reach the limit, upload your files in multiple API calls. - -Once the file uploads successfully, you will see a response body with 200 response code, as shown in the image below: - -## How to Test the Uploaded File on TestMu AI Platform - -Test the uploaded file on the TestMu AI platform to perform automation testing on the cloud-based Selenium Grid. - -Add the [desired capability](https://www.testmuai.com/capabilities-generator/) into your automation script for file upload. - -For example: If you uploaded 3 files named "804402.png", "file_example_JPG_2500kB.jpg", and "Sample_File_To_Upload.txt", add the following desired capability: - -**Java** - -```java -HashSet ab = new HashSet<>(); -ab.add("Sample-Spreadsheet-10-rows.csv"); -DesiredCapabilities capabilities = new DesiredCapabilities(); -capabilities.setCapability("build","Upload"); -capabilities.setCapability("name","Upload Files"); -capabilities.setCapability("browserName", "Chrome"); -capabilities.setCapability("browserVersion","104"); -capabilities.setCapability("platformName", "Windows 10"); -capabilities.setCapability("lambda:userFiles",ab); -``` - -**C#** - -```csharp -//HashMap -string testName = callStack[0].ToString(); -InitCaps -String[] ltFile = new string[] { "file-name.csv" }; - -// Capabilities -capabilities.AddAdditionalOption("lambda:userFiles", ltFile); -``` - -**PHP** - -```javascript -$capabilities = array( -. -. -"lambda:userFiles" => [ -"804402.png", -"file_example_JPG_2500kB.jpg", -"Sample_File_To_Upload.txt" -]) -``` - -**Python** - -```py -capabilities = { -. -. -"lambda:userFiles" : [ -"804402.png", -"file_example_JPG_2500kB.jpg", -"Sample_File_To_Upload.txt" -] -} -``` - -**JavaScript** - -```js -var capabilities = { -. -. -"lambda:userFiles" : [ -"804402.png", -"file_example_JPG_2500kB.jpg", -"Sample_File_To_Upload.txt" -] -} -``` - -Now you can use the uploaded files in your automation test script. - -The following table shows the file path for uploaded files on both desktop and mobile operating systems. - -|Operating System | Default Download Path | -|------------- | --------------------------| -| Linux | /home/ltuser/Downloads | -| Windows | C:\\Users\ltuser\Downloads | -| macOS | /Users/ltuser/Downloads | -| Android | /storage/emulated/0/Android/data/com.android.chrome/files/Download/ | -| iOS | /Users/ltuser/Downloads/ios/media | - -For example, to upload a file on a web page via test script, use the following Java code (on Windows and macOS): - -```java -// If the file is 804402.png, and testing environment is Windows OS -WebElement addFile = driver.findElement(By.xpath(".//input[@type='file']")); -addFile.sendKeys("C:\\Users\\ltuser\\Downloads\\804402.png"); - -// If the file is 804402.png, and testing environment is macOS -WebElement addFile = driver.findElement(By.xpath(".//input[@type='file']")); -addFile.sendKeys("/Users/ltuser/Downloads/804402.png"); -``` - -# Download Files from TestMu AI Storage During Test Execution - -Retrieve files from TestMu AI cloud storage directly to the machine running your test during Selenium automation testing. - -## Prerequisites - -Verify the following before using the download hook. - -1. Upload files to TestMu AI storage using the [file upload API](https://www.testmuai.com/support/api-doc/). -2. Confirm the exact file names stored in lambda storage. - -## Download Hook Implementation - -Use the executeScript command to fetch files from lambda storage. - -### Basic Syntax - -``` -driver.executeScript("lambda-files-download=file_name") -``` - -You can now successfully upload files and use them with Selenium Automation testing on TestMu AI's cloud server. If you face any issues, reach out to us via our **24/7 chat support**, or mail us at [support@testmuai.com](mailto:support@testmuai.com). Happy Testing! diff --git a/static/docs/upload-media.md b/static/docs/upload-media.md deleted file mode 100644 index 121a216bd..000000000 --- a/static/docs/upload-media.md +++ /dev/null @@ -1,90 +0,0 @@ -# Uploading Files and Media on Real Devices - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI's file upload feature provides a convenient way to enhance your testing scenarios by allowing you to upload various media and non-media files directly to TestMu AI's cloud devices. In this section, we'll guide you through the process of uploading files, highlight the supported file types, and explain how to use the `uploadMedia` capability while running your test scripts. The same capability works for both **App** and **Browser** (web) automation — no separate capability to learn. - -## Objectives -By the end of this topic, you will be able to: - -1. Use Upload File and Media feature in Manual App testing. -2. Use Upload File and Media feature in App Automation. -3. Use Upload File and Media feature in Browser (web) Automation. - -## File Storage Paths on Devices - -Where an uploaded file lands on the device depends on whether it is an **App** or **Browser** session. Everything else about the upload flow is identical. - -| Category | Platform | App session location | Browser session location | File Type | -|-----------------|----------|--------------------------------------------------|-----------------------------------------------------------------|-------------------| -| Media Files | Android | Default gallery app, `/sdcard/Pictures` / `/sdcard/Movies` | Downloads (reachable from the browser file picker) | Images, Videos | -| | iOS | Camera Roll, `/private/var/mobile/Media/DCIM/` | Photo Library (reachable from the native *Photo Library* picker) | Images, Videos | -| Non-Media Files | Android | Default Downloads folder of the device | Downloads (reachable from the browser file picker) | Files | -| | iOS | App’s directory: Files app → On My iPhone → Your app's directory | Files app → On My iPhone → **Chrome** (even in Safari) | Files | - -### Supported File Types - -TestMu AI supports various file types for upload, ensuring flexibility in your testing scenarios. Below are the supported file types: - -- **Images**: JPG, JPEG, PNG, GIF, BMP (Maximum size: 10 MB) -- **Videos**: MP4 (Maximum size: 50 MB) -- **Files**: XLS, XLSX, DOC, DOCX, PDF, CSV, TXT, CRT, CER, GED, GPX, XML, PBIX, ZIP, JSON (Maximum size: 15 MB) - -## Upload File and Media feature in App & Browser Automation - -This section provides a comprehensive guide on leveraging this feature within automation tests. The steps are identical for **app automation** and **browser (web) automation** on real devices. It comprises two fundamental steps: - -- Uploading the files and obtaining the `media_url`. -- Using `media_url` into your tests using `uploadMedia` capability. - -### Step 1 : Uploading the files on TestMu AI Cloud - -#### Using REST API - -You can use the following curl command to upload any file `media` and `non-media` from your system to the TestMu AI cloud. - -{`curl --user "${YOUR_LAMBDATEST_USERNAME()}:${YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://api.lambdatest.com/mfs/v1.0/media/upload" -F "media_file=@"/Users/macuser/Downloads/image.jpeg"" -F "type=image" -F "custom_id=SampleImage"` -} - -**Request Parameters** -- `media_file`: This parameter denotes the media file to be uploaded from your local. -- `type`: This parameter denotes file type out of image,video and doc. -- `custom_id`: This parameter specifies a custom identifier for the media file. - -Below is a sample response demonstrating the return of the `media_url` parameter value: - -```bash -{ -"media_url": "lt://MEDIAb48ab11c599944ee9dcd26b3e2978d3c", -"name": "sample.csv", -"status": "success", -"custom_id": "Sample" -} -``` -#### Using App Automation Interface - -You can also utilize TestMu AI's user-friendly UI to upload the files on TestMu AI cloud and get the `media_url` using the upload button located at the top of the automation dashboard. - -### Step 2 : Setting Capability in Your Test Script - -Once the files are uploaded to TestMu AI's cloud, seamlessly integrate files into your automation tests via the capability. Set the **uploadMedia** capability to the **media_url** parameter returned in the API response. - - {`DesiredCapabilities desiredCapabilities = new DesiredCapabilities(); -desiredCapabilities.setCapability("uploadMedia", Arrays.asList("lt://MEDIAfcdb39b9602d474f825d6002416a3969", "lt://MEDIA8d13e569b3e140c18e82b066022518bd"));`} - - {`DesiredCapabilities desiredCapabilities = new DesiredCapabilities(); -desiredCapabilities.setCapability("uploadMedia", ["lt://MEDIAfcdb39b9602d474f825d6002416a3969", "lt://MEDIA8d13e569b3e140c18e82b066022518bd"]);`} - - {`desired_capabilities = { - "uploadMedia": ["lt://MEDIAf446d4170cd946aa9ec307d10cb679b9", "lt://MEDIA8d13e569b3e140c18e82b066022518bd"] -}`} - -- Each automation session permits a maximum of five file uploads. -- The `uploadMedia` capability is set the same way for **app** and **browser (web)** automation. -- In manual testing, iOS app needs to installed first to upload non-media files. -- For non-media files, make sure your iOS app's Info.plist file includes the UIFileSharingEnabled and LSSupportsOpeningDocumentsInPlace keys set to true. This configuration is necessary to enable your app's folder accessibility within the Files app. - -**Browser automation — differences to note** -- **iOS documents** uploaded in a browser session appear under **Chrome** in the Files app, even when testing in Safari. This is by design, as only Chrome on iOS exposes a browsable file container. Pick the file via *Choose File → On My iPhone → Chrome*. -- **iOS images and videos** appear in the **Photo Library** and are picked from the native *Photo Library* picker. -- On Android, all uploaded files land in **Downloads**, reachable from the browser file picker. Inputs using `accept="image/*"` may open the photo picker instead; the default file input works across the board. diff --git a/static/docs/use-cases-of-kane-ai.md b/static/docs/use-cases-of-kane-ai.md deleted file mode 100644 index 980b5a510..000000000 --- a/static/docs/use-cases-of-kane-ai.md +++ /dev/null @@ -1,55 +0,0 @@ -# Use Cases of KaneAI Across Business Domains - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -### Application Suites - -KaneAI accelerates testing and validation across complex application suites by auto-generating test cases, simulating user journeys, detecting integration failures, and continuously validating performance and reliability across modules and releases. - -### Corporate Legal Management - -KaneAI automates validation of legal workflows by testing contract lifecycle systems, clause extraction accuracy, compliance workflows, and document management reliability to help reduce legal risk and operational errors. - -### Customer Service, Support & CRM/CX - -KaneAI improves quality of CRM and CX platforms by autonomously testing customer journeys, chatbot accuracy, ticket workflows, integrations, and performance under high-volume scenarios to ensure seamless omnichannel experiences. - -### Digital Commerce - -KaneAI validates end-to-end eCommerce flows including browsing, search, cart, payments, offers, and order tracking by generating AI-driven test scenarios that detect checkout failures, latency issues, and conversion blockers. - -### Finance - -KaneAI strengthens financial systems by testing transaction integrity, reconciliation workflows, regulatory compliance, security vulnerabilities, and system resilience through AI-generated tests and real-world transaction simulations. - -### Human Resources / Recruiting - -KaneAI ensures HR and recruiting platforms perform reliably by testing applicant workflows, onboarding journeys, payroll accuracy, integrations with background checks, and system scalability during high hiring demand. - -### IT Operations / Service Desk - -KaneAI enhances ITSM and service desk platforms by validating incident, change, and asset workflows, chatbot accuracy, monitoring integrations, and system performance to improve uptime and operational efficiency. - -### Marketing - -KaneAI validates marketing automation platforms by testing campaign workflows, personalization engines, data integrations, email/SMS delivery, segmentation accuracy, and analytics reliability to maximize campaign performance. - -### Sales & Lead Management - -KaneAI ensures sales platforms function flawlessly by testing lead capture, routing, scoring, CRM synchronization, forecasting accuracy, and pipeline health across channels and integrations. - -### Security & Fraud Detection - -KaneAI proactively tests security and fraud systems by simulating attack vectors, anomaly scenarios, false-positive/false-negative conditions, and performance under threat load to strengthen detection accuracy and resilience. - -### Sourcing & Procurement - -KaneAI validates procurement workflows by testing vendor onboarding, purchase approval flows, contract compliance, invoice matching, and ERP integrations to reduce errors and cycle times. - -### Supply Chain - -KaneAI tests supply chain platforms by simulating demand spikes, inventory fluctuations, logistics delays, supplier failures, and integration accuracy to ensure system resilience and real-time visibility. - -### Workplace & Digital Workplace - -KaneAI strengthens digital workplace platforms by testing collaboration tools, access controls, identity flows, remote work scenarios, and performance across devices to ensure secure and seamless employee experiences. diff --git a/static/docs/user-management.md b/static/docs/user-management.md deleted file mode 100644 index bb317e5f5..000000000 --- a/static/docs/user-management.md +++ /dev/null @@ -1,61 +0,0 @@ -# User Management - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -With Organization Settings in TestMu AI, you have the ability to effortlessly invite your team members to your TestMu AI Account. TestMu AI offers a range of roles that can be assigned to users. - -At present, TestMu AI supports the following roles: - -**Admin:** This role grants full access to all the features and functionalities of TestMu AI, enabling the user to administer the account effectively. - -**User:** Users with this role have access to all the subscribed products and are able to execute and stop their own tests. However, certain Organization Settings, such as Team Management, Subscriptions & Invoices, Authentication & SSO, remain inaccessible. - -**Guest:** This role provides the ability to view test logs, analytics, and other reports. However, guests do not have the privilege to initiate test runs. - -By leveraging the flexibility of Organization Settings, you can seamlessly manage your team members and their respective roles within your TestMu AI Account. - -## Access Organization Settings - -To access the organization settings, head to **Settings** and select **Organization Settings** from the dropdown. - -This page shows a list of users in your organization with options to invite new users, update an user, change their password etc. - -## Invite Users To Your TestMu AI Account - -Enter **one or multiple email addresses** along with a **role** for these users and system would send **email invitations** for the invitee to accept invitations to join TestMu AI. - -They would be granted access to TestMu AI with the role you selected while inviting. - -## View Sent Invitations - -Click on **Pending Invitations** to see the list of invitations sent from your account. Default filter would show only the pending invitations. To see all statuses you can change to a different filter. - -## Managing Users - -As an administrator you can do the following: - -1. See the list of users. -2. View username & access token of a User. -3. Change password of your users. -4. Delete a user from your team. -5. Reset MFA of a user. - -### 1. List of Users - -Head to the **Organization Settings** tab to see a list of users. Under the **Members** tab, you can see a list of users in your account. - -### 2. View username & access token of a user - -As an Admin, you can view the **access token** of a user to be used in your CI/CD systems. Click on the **three dots** and select **View username & access token** to open a modal. - -### 3. Change password of a user - -As an Admin, you can change a user's password in your TestMu AI account. Click on the **three dots** and select **Change Password** would open a modal where you can set a new password. - -### 4. Delete a user - -As an Admin you can **delete** users from your TestMu AI account. Click on the **three dots** and select **Delete User** option which would ask for a reconfirmation. Confirming the same would remove the selected user from your account. - -### 5. Export - -You can **export** list of **users, service accounts and invitations** sent from your TestMu AI account. diff --git a/static/docs/userback-integration.md b/static/docs/userback-integration.md deleted file mode 100644 index 4df0a9ef4..000000000 --- a/static/docs/userback-integration.md +++ /dev/null @@ -1,67 +0,0 @@ -# Userback Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Userback is a platform that makes it easy for website and app teams to gather user feedback. It helps streamline the feedback process, allowing teams to collect and manage visual feedback from users directly on their websites or applications. This feedback can be in the form of screenshots, annotations, and comments, making it easier for teams to understand user suggestions or issues. - -TestMu AI's integration with Userback offers a smooth solution to track, log, and manage bugs found during [cross-browser testing](https://www.testmuai.com/). By integrating both platforms, developers can easily report bugs from TestMu AI directly to their Userback instance. This eliminates the need for manual tasks and issue management, making the process much more efficient and hassle-free. - -## Integrating Userback With TestMu AI - -**Step 1:** Sign in to your TestMu AI account. Don't have an account, register for free. - -**Step 2:** Select **Integrations** from the left sidebar and head to **Userback**. - -**Step 3:** It will route you to the installing console window, where you are required to enter your Userback **Email, Userback Host, and Userback API Token**. - -## Getting The Userback Email, Host, and API Token - -**Step 1:** Login to your Userback account. - -**Step 2:** Head to your **Profile Avatar** and select **Workspace Settings** from the dropdown. - -**Step 3:** Head to **API Token** from the left sidebar. - -**Step 4:** Click on **Create an API Token**. - -**Step 5:** Give a preferred name to your API token and click **Create**. - -**Step 6:** Copy your newly generated API token. - -To access your **Userback Host**, follow these steps: - -**Step 1:** Head to your **Profile Avatar** and select **Workspace Settings** from the dropdown. - -**Step 2:** Copy the **Workspace URL**. This will be your Userback Host. - -Enter the required credentials to integrate TestMu AI with Userback and click **Install**. - -You have now successfully integrated your TestMu AI account with Userback. - -## One-Click Bug Logging With Userback - -**Step 1:** In your TestMu AI Dashboard, head to **Real-Time Testing > Browser Testing** from the left navigation menu. - -**Step 2:** To perform **live-interactive testing** of your website or a web app, enter the web URL, select the testing type (Desktop/Mobile), select the browser version, OS, and resolution, and click **Start**. - -Upon clicking, a cloud-based virtual machine will fire up, running real browsers and operating systems where you can test websites and web apps for filing bugs. - -**Step 3:** Click on the **Bug icon** from the left in-session toolbar to take a screenshot of any bug you encounter. - -**Step 4:** After marking the bugs, click on the **Bug** icon in front of **Userback** under the integrations section. - -**Step 5:** In the Userback ticket, fill the bug details like **Project, Summary, and Description** and click **Mark As Bug** to log the bugs in Userback. - -**Step 6:** To view the logged issues, visit TestMu AI Dashboard and head to **Manage > Issue Tracker** from the left navigation bar. - -**Step 7:** Inside the **Issue Tracker**, you'll notice the logged bugs and their test environment details. Click on the **upper right arrow** in the square button to view your logged bugs in Userback instance. - -**Step 8:** Upon clicking, you will be redirected to the Userback dashboard to find your **logged bugs**. - -## Uninstalling The Userback Integration - -**Step 1:** Sign in to your TestMu AI account and visit **Integrations** from the left sidebar menu. - -**Step 2:** From My Integrations, click **Remove** under the **Userback** block. - -> That was all you need to know for TestMu AI + Userback Integration. Increase your productivity with our integrations. If you still have any questions for us, please feel free to let us know. Our experts are always available on **chat** to help you out with any roadblock regarding our product. Happy testing! diff --git a/static/docs/using-environment-variables-for-authentication-credentials.md b/static/docs/using-environment-variables-for-authentication-credentials.md deleted file mode 100644 index 8fad672a3..000000000 --- a/static/docs/using-environment-variables-for-authentication-credentials.md +++ /dev/null @@ -1,125 +0,0 @@ -# Using Environment Variables for Authentication Credentials - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -To provide a secure [cross-browser testing](https://www.testmuai.com) environment on-cloud, we recommend integrating TestMu AI Authentication Credentials in your test scripts. This allows our cloud servers to establish a secure connection between your local system and our [cloud-based Selenium Grid](https://www.testmuai.com/selenium-automation/). This document shows you how to set up your TestMu AI authentication credentials as environment variables for your respective operating system. This setup also helps authenticate your colleague's test cases when executed against an individual account. - -## Where to Find Authentication Credentials - -Get your Username and Access Key from the Automation Dashboard or your profile page. - -TestMu AI Authentication Credentials consist of two fields: - -1. TestMu AI Username -2. TestMu AI Access Key - ->You can find these authentication credentials by visiting the [Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/dashboard) or your [TestMu AI Profile](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). - -### Fetching Authentication Credentials from the Automation Tab - -1. Visit the Automation tab from the left navigation menu. -2. Click on the key icon from the top-right corner of your screen and copy the Access Key and Username. - -### Fetching Authentication Credentials from Your Profile - -1. Visit your TestMu AI Profile by clicking on the user icon on the top-right followed by clicking on Profile. - -2. You will find your Username & Access Key. - -## Configuring Environment Variables on Mac OSX/Linux Systems - -Export your credentials in the bash profile to make them available across terminal sessions. - -1. Open your terminal mode, and provide the command `vi ~/.bash_profile` & hit **Enter**. -2. Press the **i** button from your keyboard to perform text-insertion into your profile. -3. Feed TestMu AI Authentication credentials using the code below: - -{`$ export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -$ export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -4. Hit **Esc** once you have entered the above lines into your Profile. -5. Long-press the **Shift** button & hit **Z** twice (z z) to save your file and quit vi. -6. Enter `source ~/.bash_profile` into the terminal. - -## Configuring Environment Variables on Windows Operating Systems - -Add your credentials as System variables through the Windows Environment Variables dialog. - -1. Right-click on the **Computer** icon to access your Computer **Properties**. You can also follow the path `(Control Panel\All Control Panel Items\System)` into your computer address bar. -2. Go to **Advanced system settings** & click on **Environment Variables**. - -3. You will see a dialog prompt for **Environment Variables**. Click on **New** under the **System variables**. -4. Enter `LT_USERNAME` as the **Variable name** & provide your `TestMu AI Username` as the **Variable value**. -5. Hit **Ok** & repeat the steps to add `TestMu AI Access Key` as another **System Environment Variable**. - -## Incorporating Environment Variables in Selenium Automation Scripts - -Reference your environment variables in test scripts for each programming language. - -Now that we have set up our TestMu AI Authentication Credentials as System Environment Variables, we need to incorporate them in our [Selenium automation](https://www.testmuai.com/selenium-automation/) scripts for invoking your automation test suites on TestMu AI Selenium Grid. This section shows you how to reference environment variables for different programming languages. - -### Java - -- For **JUnit** & **TestNG**: - -``` -lambdatestUserName = System.getenv("LT_USERNAME"); -lambdatestAccessKey = System.getenv("LT_ACCESS_KEY"); -``` - -### Python - -- For **PyTest** & **unittest**: -``` -lambdatest_username = os.environ["LT_USERNAME"] -lambdatest_access_key = os.environ["LT_ACCESS_KEY"] -``` - -### NodeJS - -- For **Mocha-Chai** & **Jasmine-Chai**: - -``` -let username = process.env.LT_USERNAME, -accessKey = process.env.LT_ACCESS_KEY, -``` - -- For **Protractor-Jasmine**: - -``` -exports.config = { -lambdatestUser: process.env.LT_USERNAME, -lambdatestKey: process.env.LT_ACCESS_KEY -} -``` - -### Ruby - -- For **RSpec**: - -``` -username: ENV['LT_USERNAME'], -accessKey: ENV['LT_ACCESS_KEY'] -``` - -### C# - -- For **NUnit**: - -``` -var lambdatestUserName = -Environment.GetEnvironmentVariable("LT_USERNAME", EnvironmentVariableTarget.User); -var lambdatestAccessKey = -Environment.GetEnvironmentVariable("LT_ACCESS_KEY", EnvironmentVariableTarget.User); -``` - -### PHP - -- For **Behat**: - -``` -$user = getenv('LT_USERNAME') -$accesskey = getenv('LT_ACCESS_KEY') -``` - -> If you come across any issues during the setup, let us know. Our Support team is available **24x7**. You can also mail us at **support@testmuai.com**. diff --git a/static/docs/video-injection.md b/static/docs/video-injection.md deleted file mode 100644 index 197b75f54..000000000 --- a/static/docs/video-injection.md +++ /dev/null @@ -1,64 +0,0 @@ -# Video Injection - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Video Injection feature enables you to test video-based functionalities such as **live streaming, real-time video capture,** and **video processing** in your app across 10000+ real devices on the TestMu AI Real Device Cloud platform. - -This tool is ideal for testing features such as: - -- Real-time video streaming and playback -- Video-based verification (e.g., identity verification using live video) -- Video capture and upload workflows -- In-app video recording features - -## How It Works -TestMu AI utilizes **Sensor Instrumentation** to integrate with various mobile sensors, including the camera. When you enable Video Injection for a session, TestMu AI injects its camera code module into your app through Sensor Instrumentation. This module then mocks or overrides the iOS SDK used in your app. - -## Video Injection in App Automation - -This section explains how to use Video Injection with App automation. This section will also teach you how to use video injection for all the possible test cases for your app along with other useful information. - -### Step 1: Upload video to TestMu AI - -You can use the following curl command to upload any video of your choice to the TestMu AI cloud. Please note that we support only **MP4 format is only supported and the video can be 50 MB or less**. - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://api.lambdatest.com/mfs/v1.0/media/upload" -F "media_file=@"/Users/macuser/Downloads/video.mp4"" -F "type="video"" -F "custom_id="SampleVideo""`} - -Response of above cURL will be a JSON object containing the `media_url` of the format - lt://MEDIA123456789123456789 as shown below. - -```js -{ -"media_url":"lt://MEDIA123456789123456789", -"name":"Video_123.mp4", -"status":"success", -"custom_id":"SampleVideo" -} -``` - -### Step 2: Enable Video injection in the Appium Session - -You can use the appium capability to turn the video injection on in your application. Just add `"enableVideoInjection": True` in the desired capabilities while writing the automation script. By adding this capability the app undergoes Sensor Instrumentation, which was described earlier in the documentation. Also adding media url `"media": "lt://MEDIA123456789123456789"` in the capabilities to inject the video is now optional and can be passed in LambdaHook as mentioned in step 3. - - {`desired_capabilities = { - "enableVideoInjection": True, - "media": "lt://MEDIAf446d4170cd946aa9ec307d10cb679b9", #Add media_url here (optional) - }`} - - {`DesiredCapabilities desiredCapabilities = new DesiredCapabilities(); -desiredCapabilities.setCapability("enableVideoInjection", "true");`} - -### Step 3: Inject the video - -Refer to the code snippets given below to upload the video with the action `VideoInjection`. Please make sure that the video needs to be injected before the camera is triggered in your application. This is due to the fact that the camera only captures the video injected at the last. - - {`driver.execute_script("lambda-video-injection=media_url") #Add media_url here`} - - {`driver.execute_script("lambda-video-injection=media_url") //Add media_url here`} - -**Use Case**: This feature can be used when you want to test injecting more than one video in single test at different instances. The video injected through above process will replace the video which you put in previous steps. - -### Step 4: Capture the desired video through camera - -You need to first add a logic to inject the uploaded video in the script. Thereafter add the logic to click the camera button in the mobile app. This will show you the last injected video on your mobile app screen. - -> ?? Check the [documentation for Media Injection](/support/docs/camera-image-injection-on-real-devices/) feature in Manual App Testing on Real devices diff --git a/static/docs/view-lighthouse-performance-metrics.md b/static/docs/view-lighthouse-performance-metrics.md deleted file mode 100644 index e50edc51f..000000000 --- a/static/docs/view-lighthouse-performance-metrics.md +++ /dev/null @@ -1,78 +0,0 @@ -# View Lighthouse Performance Metrics Using Online Selenium Grid - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI integrates [**Lighthouse**](https://developers.google.com/web/tools/lighthouse) to view performance metrics (**Lighthouse Audits**) on its cloud-based Selenium Grid. You can view the scores of: - -* **Performance**, -* **Accessibility**, -* **Best Practices**, -* **SEO** -* **Progressive Web App** - -metrics with TestMu AI. TestMu AI also allows you to perform cross-browser testing on 3000+ combinations of real devices, browsers, and operating systems. - -> **Note:** TestMu AI currently supports **Lighthouse** on Chrome browsers for all Windows OS, macOS BigSur, macOS Mojave, and macOS Catalina. - -## Prerequisites - -Complete these steps before generating Lighthouse reports. - -1. Create a TestMu AI account. -2. Get your TestMu AI Username and Access Key from the [Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) or your TestMu AI Profile. - -## How to Include Lighthouse Performance Metrics in Selenium Automation - -Add the `performance` capability to your desired capabilities to enable Lighthouse metrics. - -You can add Lighthouse performance metrics for the website you want to test using the [Desired Capabilities](/docs/selenium-automation-capabilities/). Use the **`Performance`** feature of the **Desired Capabilities Generator**. - -For example, if you are using **TestNG** for Selenium Automation on TestMu AI, include the **Lighthouse** performance feature with the below code snippet: -```java -DesiredCapabilities caps = new DesiredCapabilities(); -. -. - -// To view performance metrics -caps.setCapability("performance", true); -``` -Below is the code snippet to trigger **Lighthouse** performance metrics on TestMu AI in **macOS Catalina** with **Google Chrome** browser version **86.0**. -```java -DesiredCapabilities caps = new DesiredCapabilities(); -caps.setCapability("browser", "Chrome"); -caps.setCapability("version", "86"); -caps.setCapability("platform", "macOS Catalina"); -caps.setCapability("build", "Lighthouse Performance Demo"); -caps.setCapability("name", "TestNG Test 3"); -caps.setCapability("network", true); -caps.setCapability("visual", true); -caps.setCapability("video", true); -caps.setCapability("console", true); -caps.setCapability("selenium_version", "4"); - -// To view performance metrics -caps.setCapability("performance", true); -``` -## How to View Lighthouse Performance Metrics on TestMu AI - -Access the Lighthouse performance report from the Automation Dashboard after running your tests. - -Once you have run your Selenium automation tests on TestMu AI, you can view the Lighthouse performance metrics on the website under test. - ->**Note:** The test duration should be at least 30 seconds to generate the **Lighthouse Performance Report**. - -Follow the below steps to view the **Lighthouse performance metrics** on TestMu AI: - -**1.** Go to the [Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com). Here you can see your recently executed Selenium automation tests. For this demo, we used a sample **TestNG** script from this [GitHub repository](https://github.com/LambdaTest/Java-TestNG-Selenium). - -**2.** Click on the test in which you activated the **Lighthouse performance** feature, as shown above. - -**3.** Click on the **Performance** tab. - -**4.** You can see the Lighthouse performance metrics on your screen for the website under test. For example, in the below image, the website under test is [https://lambdatest.github.io/sample-todo-app/](https://lambdatest.github.io/sample-todo-app/), which we used for our demo. - -**5.** Click on any of the metrics to view it in detail, along with the individual factor scores. For example, clicking on _Accessibility_ opens its details. - -Similarly, you can click on any of the performance metrics to view it in detail. Performance reports can also be downloaded in various formats. TestMu AI also helps you view the Passed Audits, individual metrics, detailed diagnosis, and improvement opportunities for that particular Lighthouse Audit. - -That's it folks! That's all about the Lighthouse Performance Metrics with TestMu AI. You can read more about the Lighthouse Audits and Performance Metrics from [web.dev](https://web.dev/lighthouse-performance/)page. If you have any doubt or questions, feel free to contact our experts at **24/7 chat support** or mail us at [support@testmuai.com](mailto:support@testmuai.com). Happy testing! :) diff --git a/static/docs/virtual-device-flutter-apps.md b/static/docs/virtual-device-flutter-apps.md deleted file mode 100644 index 10c27097f..000000000 --- a/static/docs/virtual-device-flutter-apps.md +++ /dev/null @@ -1,81 +0,0 @@ -# How to test flutter apps using Appium framework on Virtual Devices - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Google's Flutter is an open-source tool for developing native mobile apps. It enables developers to create cross-platform apps for both Android and iOS using a single codebase and programming language. - -You can now test Flutter apps on the TestMu AI Appium testing platform across 3000+ real Android and iOS devices. TestMu AI supports Appium's Flutter driver that lets you test Flutter apps using the Appium framework. To test Flutter apps, you will need to upload apps on TestMu AI cloud servers and then run your automated tests. - -## Prerequisites - -Before automating Flutter apps using Appium, make sure you have the following things configured. -- Your Flutter App must be compiled in `debug` or `profile` mode, as Appium Flutter Driver does not support Flutter App running in release mode. -- Add the below dependencies in Flutter app's `pubspec.yaml` - -```yaml -dev_dependencies: -test: Demo -flutter_test: -sdk: flutter -flutter_driver: -sdk: flutter -``` - -- In `main.dart` file, ensure the app has `enableFlutterDriverExtension()` enabled before `runApp`. - -```javascript -void main() { -enableFlutterDriverExtension(); -init(); -runApp(MyApp()); -} -``` - -- Set the `automationName` capability to `flutter` in Appium desired capabilities. - -```javascript -desiredCapabilities.setCapability("automationName", "flutter"); -``` - -Shown below is the test script for automating Flutter apps on TestMu AI platform. - -```javascript -import os - -from appium.webdriver import Remote -from appium_flutter_finder.flutter_finder import FlutterElement, FlutterFinder - -driver = Remote('http://LT_USERNAME:LT_ACCESS_KEY@mobile-hub.lambdatest.com/wd/hub/', dict( -platformName='Android', -automationName='flutter', -platformVersion='11', -deviceName='Google Pixel 4', -app='', -isRealMobile=false -)) - -finder = FlutterFinder() - -text_finder = finder.by_text('You have pushed the button this many times:') -text_element = FlutterElement(driver, text_finder) -print(text_element.text) - -key_finder = finder.by_value_key("next_route_key") -goto_next_route_element = FlutterElement(driver, key_finder) -print(goto_next_route_element.text) -goto_next_route_element.click() - -back_finder = finder.page_back() -back_element = FlutterElement(driver, back_finder) -back_element.click() - -tooltip_finder = finder.by_tooltip("Increment") -driver.execute_script('flutter:waitFor', tooltip_finder, 100) - -floating_button_element = FlutterElement(driver, tooltip_finder) -floating_button_element.click() - -counter_finder = finder.by_value_key("counter") -counter_element = FlutterElement(driver, counter_finder) -print(counter_element.text) -``` diff --git a/static/docs/visual-ui-testing.md b/static/docs/visual-ui-testing.md deleted file mode 100644 index e3b9fd96d..000000000 --- a/static/docs/visual-ui-testing.md +++ /dev/null @@ -1,27 +0,0 @@ -# Visual UI Testing - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - - Automated Screenshot Testing - Learn how to capture screenshots in bulk through different desktops and mobile devices running on various OS in a single go. - - Mark As Bug In Screenshot Testing - Mark as Bug is a feature that would help you highlight any UI observation from your test sessions on TestMu AI. - - Automated Screenshot API - Learn how to capture full-page screenshots of any URL by invoking an API call to TestMu AI cloud servers. - - Taking Screenshots of Pages Behind Login - Learn how to to automatically capture bulk, full-paged screenshots of any URL over different browsers. - - List Of Mobile And Desktop Browsers - The default list of Mobile and Desktop browsers for Automated Screenshot Testing - - Responsive Testing - Learn how to check the appearance of a website with RWD(Responsive Web Design). - - Mark As Bug In Responsive Testing - Mark as Bug is a feature that would help you highlight any UI observation from your test sessions on TestMu AI. - - Smart Visual Testing - Learn how to test and compare two image layouts to determine the differences between them quickly. diff --git a/static/docs/vividus-integration.md b/static/docs/vividus-integration.md deleted file mode 100644 index 4972bab8c..000000000 --- a/static/docs/vividus-integration.md +++ /dev/null @@ -1,102 +0,0 @@ -# Vividus Tests On TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Vividus is a codeless test automation tool, developed with the aim to help testers perform automation testing for any application, test, or scenario, using a single tool and common language. - -TestMu AI now integrates with the Vividus tool to help you execute automation tests on its cloud-based Selenium Grid. This TestMu AI-Vividus integration will help you perform cross-browser testing of your website, web app, or your mobile website, over 3000+ real desktop devices, mobile viewports, and operating system combinations on TestMu AI platform. This document is a comprehensive guide to help you understand how to integrate Vividus with your TestMu AI account. - -## How To Setup Your TestMu AI Account With Vividus? - -Once you have your project set up using the Vividus tool, here's a step by step guide to help you setup your TestMu AI account with Vividus tool: - -1. Once you have your test scenario ready, you need to add the TestMu AI plugin in the build.gradle -> dependencies, using the below command. - -```javascript -implementation(group: 'org.vividus', name: 'vividus-plugin-lambdatest', version: vividusVersion) -``` - -2. Next you need to add the TestMu AI cloud as the executing platform. To do so, open the configuration.properties file and add the below profile in it: - -```javascript -Next you need to add the TestMu AI cloud as the executing platform. To do so, open the configuration.properties file and add the below profile in it: -``` - -3. Now you need to specify you TestMu AI username and access key in the same configuration.properties file, using below syntax: - -```javascript -selenium.grid.username= -selenium.grid.password= -``` - -This will verify your credentials and connect your TestMu AI account with the Vividus tool successfully, based on your current plan. - -To get your Username and Access Key, go to your [TestMu AI automation dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/timeline/) and click on the "key" icon left to the help button, on the top right corner. - -Once done, you will be able to select your desired capabilities in their corresponding fields, like the platform, browser, browser version, resolution, etc. To know more about desired capabilities, refer to our [Capabilities Generator Guide](/docs/selenium-automation-capabilities/). - -That's it. Vividus tool has been integrated successfully with your TestMu AI account. Now you can execute your automated tests on TestMu AI's cloud-based Selenium Grid. - -## Running Tests From Your Vividus App On TestMu AI Platform - -For demo purpose, we have recorded a test scenario, which includes: - -```javascript -- Opening LambdaTest ToDo App -- Check the first item in the list -- Check the second item in the list -- Add a new item in the list, with text "Yey, Let's add it to list". -``` - -* Use the below syntax to pass the desired capabilities: - -```javascript -selenium.grid.capabilities.= -``` - -For example, we have used the below desired capabilities for demo purpose: - -```javascript -selenium.grid.capabilities.build=VividusDemo -selenium.grid.capabilities.visual=true -selenium.grid.capabilities.console=true -selenium.grid.capabilities.network=true -selenium.grid.capabilities.version=13.0 -selenium.grid.capabilities.platform=MacOS Big Sur -``` - -> Note: If no browser version or OS capability is passed, it will take the latest version of the browser available on TestMu AI platform, along with latest Windows OS version. - -* Now you need to select the desired browser, on which you want to run your tests. To do so, open the configuration.properties file, and change the name of the browser to the desired one, from Chrome. - -Syntax: - -```javascript -configuration.profiles=lambdatest/web,web/desktop/ -``` - -For example, let’s change the browser to Safari: - -```javascript -configuration.profiles=lambdatest/web,web/desktop/safari - -// Previously it was configuration.profiles=lambdatest/web,web/desktop/chrome -``` - -* To run your test, using the below command: - -```javascript -./gradlew runStories -``` - -* The test execution will then start, and you can see the same on the automation dashboard of your TestMu AI account. - -* Once the test is complete, you can view the test result on the automation dashboard by clicking on the test case. You can see and verify if the test has been passed or failed in the Automation Logs. - -For each test executed overthe TestMu AI platform, you can view logs, [mark bugs](/docs/mark-as-bug-in-automation-testing/), and do much more. - -## Running Tests From Vividus In Parallel On TestMu AI Platform - -Parallel execution is available out of the box and limited only with machine resources and cloud concurrency. In Vividus, concurrency is available at the story level, i.e., each story will use a new session. To configure how many threads you would like to use you need to change the value threads value in the suite.properties file, as shown below: - -> Congrats! You have successfully integrated the Vividus tool with your TestMu AI account. Now run your test cases over the TestMu AI's cloud-based Seleniunm Grid. If you come across any doubts or questions, reach out to us via our **24/7 chat support** or mail us at support@testmuai.com. Happy testing! 🙂 diff --git a/static/docs/voice-agent.md b/static/docs/voice-agent.md deleted file mode 100644 index aa9cd2a1a..000000000 --- a/static/docs/voice-agent.md +++ /dev/null @@ -1,60 +0,0 @@ -# Voice Agent Testing With TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -The Agent Testing Platform tests a voice agent the same way it tests a chat agent, except the conversation happens as audio instead of text. It holds a spoken conversation with your agent, transcribes the responses, and scores the interaction across the same 9 quality metrics. It covers voice assistants and voice-enabled support agents. - -A voice agent uses the same setup as a chat agent: upload the documents that describe it, generate scenarios, and run an evaluation. The difference is the medium, audio rather than text. - -## How Voice Agent Testing Works - -The platform conducts a voice-based conversation with your agent, following a generated scenario. The agent's voice responses are captured and transcribed, and the evaluation runs on the full audio conversation transcript. - -A voice agent is functionally identical to a chat agent, so every chat feature applies: workflow-based scenario generation, test suites, endpoint profiles, the playground, thresholds, go-live assessment, and scheduled runs. See how to [test a chat agent](/support/docs/chat-agent/) for the shared setup. - -## What Makes Voice Testing Different From Chat - -Voice testing differs from chat in the medium, not the metrics. - -- Conversations with the agent are conducted as audio (WAV) instead of text messages. -- The agent's voice responses are captured and transcribed for evaluation. -- Evaluation runs on the full audio conversation transcript. - -## Features - -A voice agent has all the features of a chat agent. - -- **Workflow-based test generation** with document upload and source integrations (Confluence, JIRA, GitHub). -- **Scenario management** with AI generation, validation criteria, personas, and special instructions. -- **Test suites** with test profile selection and run history. -- **Endpoint profiles** with Postman collection import. -- **Test profiles** for data-driven testing. -- **Playground** for interactive testing. -- **Evaluation execution** with metric selection and HyperExecute integration. -- **Metric threshold configuration.** -- **Go-live assessment** with a production readiness verdict. -- **Scheduled runs.** - -## Metrics - -Voice agents are evaluated on the same 9 quality metrics as chat agents, each scored on a 0 to 100% scale. - -| # | Metric | What it measures | -|---|--------|------------------| -| 1 | Bias Detection | Detects biased, discriminatory, or unfair responses | -| 2 | Hallucination Detection | Identifies false, fabricated, or unsupported information | -| 3 | Completeness | Whether the agent fully addresses the user's question | -| 4 | Context Awareness | How well the agent understands and maintains conversation context | -| 5 | Response Quality | Overall quality, clarity, and helpfulness of responses | -| 6 | Conversation Flow | Natural flow and coherence across multi-turn conversations | -| 7 | User Satisfaction | Estimated end-user satisfaction based on conversation quality | -| 8 | File Handling Quality | Quality of file upload and download interactions, if applicable | -| 9 | File Generation Accuracy | Accuracy of generated files against requirements, if applicable | - -Every evaluation result includes the overall score, per-metric scores with pass or fail badges, detailed analysis, the full conversation transcript, identified strengths, areas for improvement, actionable recommendations, and validation criteria results. - -## Related TestMu AI Guides - -- See how to [test a chat agent](/support/docs/chat-agent/), which shares the same setup and metrics. -- See how to [test phone agents](/support/docs/phone-agent/) for spoken agents over the telephone. -- See how the platform [runs an evaluation end to end](/support/docs/architecture-and-how-evaluation-works/). diff --git a/static/docs/vs-code-extension.md b/static/docs/vs-code-extension.md deleted file mode 100644 index d9c27d795..000000000 --- a/static/docs/vs-code-extension.md +++ /dev/null @@ -1,59 +0,0 @@ -# TestMu AI VS Code Extension - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI VS Code Extension enables you to perform real-time cross browser testing of your public and locally hosted websites and web applications right from VS Code. Using this extension, you can trigger real-time cross-browser testing from within VS Code Editor to test your web apps across 3000+ browsers, real devices, and operating systems. - -In this guide, learn how to configure TestMu AI VS Code Extension for real-time cross browser testing. - -## Installing The TestMu AI VS Code Extension - -**Note** -You can also install TestMu AI VS Code Extension from the Visual Studio Marketplace. - -1. Open VS Code and select the **Extensions** icon from the left sidebar. - -2. Search for *TestMu AI* in the search box and click **Install**. - -Once you install the extension, you'll notice a TestMu AI icon will be added on the left sidebar. - -## Authenticating With VS Code Extension - -1. Click the *TestMu AI* icon from the left sidebar and select **Realtime Testing**. - -2. It will ask you to enter your TestMu AI e-mail and an access key. You can get them from your [TestMu AI Profile](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile) page. - -* Enter your TestMu AI e-mail and press Enter. - -* Enter your TestMu AI access key and press Enter. - -After entering TestMu AI credentials, it will show you a Realtime Testing screen. You're all set to start your real-time cross browser testing! - -## Realtime Cross Browser Testing Using VS Code Extension - -1. To begin testing, enter the test URL, choose Desktop or Mobile, and select browser, browser version, operating system, and resolution. Click **START**. - -2. A confirmation dialog box will appear. Click **Open**. - -It will take you to a cloud-based virtual machine running real browsers and operating systems where you can perform real-time cross browser testing of your websites and web applications. - -For more details, head over to our guide on Real Time Browser Testing. - -## Testing Locally Hosted Websites Using VS Code Extension - -With TestMu AI VS Code Extension, it's simple to run real-time cross browser tests of your locally hosted websites without configuring the TestMu AI Tunnel binary from the Command Prompt. When you launch TestMu AI VS Code Extension, the tunnel starts automatically. - -**Note** -You can also trigger multiple tunnels based on your requirements by clicking **Start new tunnel** from the Tunnel dropdown menu - -1. To start testing, enter the localhost URL, choose Desktop or Mobile, and select browser, browser version, operating system, and resolution. Click **START**. - -2. A confirmation dialog box will appear. Click **Open**. - -It will take you to a cloud-based virtual machine running real browsers and operating systems where you can perform real-time cross browser testing of your locally hosted websites or privately hosted projects. - -For more details, head over to our guide on Testing Locally Hosted Pages. - -## Uninstalling The TestMu AI VS Code Extension - -To remove the TestMu AI VS Code Extension, select the **Extensions** icon from the left sidebar. Navigate to the settings icon and click **Uninstall**. diff --git a/static/docs/vsts-integration.md b/static/docs/vsts-integration.md deleted file mode 100644 index ce9b3af16..000000000 --- a/static/docs/vsts-integration.md +++ /dev/null @@ -1,96 +0,0 @@ -# Azure DevOps Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Azure DevOps, previously known as **VSTS: Visual Studio Team Services** – A software developed by Microsoft Corporations to provide an IDE(Integrated Development Environment) for facilitating management & development regarding a software project. Azure DevOps provides a set of specific, cloud-based collaboration tools for developers, testers & software architects. So you can plan smarter and ship even faster with the help of effective collaboration. With their **Azure Pipelines** you can deploy your development to any Git provider cloud. - -The TestMu AI Azure DevOps Integration allows you to create a work item directly in your project from TestMu AI platform. Push a bug, epic, task, story to your respective project anytime, even in the middle of your test session. The fields populated by you when marking as a bug through TestMu AI are displayed as information on the work item in Azure DevOps project for that testing instance. - -We recently added support for custom fields to our integration. From now on, all required custom fields in your Azure Devops account will be visible in your bug marking form on lambdatest. - -**Azure DevOps Integration with TestMu AI, like all of our other integrations to 3rd party applications, is available for freemium as well as premium plan.** - -## How To Integrate Azure DevOps With Your TestMu AI Account? - -**Step 1:** Login to your TestMu AI account. You would need to have Admin or User level access to see and install integrations. - -**Step 2:** Select **'Integration'** from the left navigation menu bar. This will guide you to a screen where you will find a list of 3rd party applications, available to integrate with your TestMu AI account. - -**Step 3:** Click on **'Add'** under the block that says 'Azure DevOps'. - -**Step 4:** As you hit **"INSTALL"**, you will be directed to a page where you have to provide your **Azure DevOps URL**, your **Azure DevOps Email** & your **Azure DevOps Access Token**. By doing so you will grant TestMu AI the access to your user-owned resources on Microsoft's Visual Studio Team Services account. This is necessary for authorization purpose between APIs of two different applications. - -> **Note:** For **Self-Hosted** Azure DevOps instance, you need to whitelisted IP address of your privately hosted projects. To get your IP whitelisted, please contact our customer support. - -* **What is my Azure DevOps URL?** - -**Azure DevOps URL** would be the domain name. Log into your Azure DevOps account and you will find it under your organization. - -**Step 5:** If you are already inside a project in your Azure DevOps account, then you will find it in the URL. For example: https://dev.azure.com/ **salmank0856** - -* **What is my Azure DevOps Email?** - -In this field you need to specify the **email id** through which you sign into your **Azure DevOps account**. -* **What is my Azure DevOps Access Token?** - -You can generate your Azure DevOps access token under **'Personal access tokens'** by clicking on setiings icon next to top right corner where your user avatar is displayed and click on **'+ New Token'**. - -* **What are access tokens?** - -Access tokens are strings with authorization key required to access an API. They are issued to the client server and are usually opaque. They are used for requesting access to protected, user-specific resources. Access tokens are vital from a security point of view & can be generated in different formats, depending upon security requirements specified on the resource server. - -**Step 6:** After you click on **'+ New Token'**, mention a **name** for the access token. Define the **scopes** for authorizing access. By doing so, you will be providing the level of access you want to provide to any 3rd party API. For TestMu AI integration with Azure DevOps, select the radio button for **'Full access'** & click on **'Create'**. - -* A token is generated, copy the token to your clipboard and make sure to store it safely with you. - -Access tokens are just as important as your passwords and shall not be misplaced into wrong hands. If by any chance you lose it in future or it gets shared to someone whom you don't trust, then you can always revoke the token and create a new one. - -For revoking a token, click on any personal access token and hit the **'Revoke'**. - -**Step 7:** Enter your Azure DevOps URL, E-mail and Access token in the provided field and & press **Install**. - -**Step 8:** Now, you will notice a success message as below. - -That's it! Go to Integrations again and you will be able to see Azure DevOps under 'My Integrations' section. You can now experience bug logging in a fly from any of your running test session in TestMu AI to your project directly by a single click. - -## How To Log Your First Bug Through Azure DevOps Integration? - -**Step 1:** Go for any of the test from the left navigation menu. For demo, we will be taking "**Real Time Testing**" option. - -**Step 2:** Present a URL of the web-app you need to test in the dialog box. After that, select any configuration for browser and operating system of your choice & hit '**Start**'. - -**Step 3:** After the VM is launched and operable. You can perform testing on your web-app for finding bugs. If a bug gets revealed, then you need to click on the **Bug icon** from the left panel for capturing a screenshot of the same. We have highlighted that option with red in the below image. - -**Step 4:** After a screenshot is captured, you can annotate any issue or a task with an **in-built image editor**. Once you are done highlighting the bug, click on the button that says **"Mark as Bug"**. - -**Step 5:** Once you click on **"Mark as Bug"** button a form would open up. Fill the fields as per your requirement. - -* You can choose out of your **'Projects'**, for logging your UI observation. -* You can select the appropriate **'Team'** to which you would like to share the UI observation. -* You can assign it to a colleague by populating the **'Member'** field. -* You can specify the number of **Iteration**. -* You can choose a the type of observation by selecting a **'Work Type'**. -* You can provide a **'Title'** when pushing the screenshot. -* You can also provide the steps to reproduce in the **'Repro Steps'** field. -* Specify the additional information in the **'System info'**. -* Select a **Priority** & **'Severity'** of the UI observation. -* Choose the concerned area from the list of **'Activity'** -* You also get to post a **'Description'** to help relate the cause of the issue or the task. - -**Step 6:** Click on **"Create Bug"** & observe it being successfully marked through a single click effort. Once you hit the button, you will get **prompt messages** on top of your Virtual Machine indicating the progress of bug logging. You will get a prompt message in few seconds mentioning "**Bug successfully marked**" indicating that the UI observation has been pushed to your project. - -**Step 7:** Visit your Azure DevOps project. You will be able to notice the logged issue right away by navigating to **Boards** from the left navigation menu and heading to the **Work Items**. - -**Step 8:** Click on the **Title** to open the details. All the data you provided through TestMu AI would already be presented in it. TestMu AI automatically includes test environment details and related screenshots in your work item on the respective project. - -## How To Remove Azure DevOps Integration? - -> You can work with one integration at a time. So if you would want to integrate to a similar 3rd party application, then you would have to **remove** your current integration. Here is how you can do that. - -**Step 1:** Login to your TestMu AI account. - -**Step 2:** Select '**Settings**' from the left navigation menu bar & click on '**Integrations**'. This will guide you to a screen where you will find 3rd party applications, available to integrate with your TestMu AI account. - -**Step 3:** Under '**My Integrations**',you'll now see '**Azure Devops**'. Click on '**REMOVE**'. You can find the remove button right next to **'Azure DevOps'.** - -That was all you need to know for TestMu AI + Azure DevOps Integration. Increase your productivity with our integrations. If you still have any questions for us, please feel free to let us know. Our experts are always **available on chat** to help you out with any roadblock regarding our product. Happy testing! diff --git a/static/docs/wd-with-selenium-running-wd-automation-scripts-on-testmu-selenium-grid.md b/static/docs/wd-with-selenium-running-wd-automation-scripts-on-testmu-selenium-grid.md deleted file mode 100644 index 16cf81604..000000000 --- a/static/docs/wd-with-selenium-running-wd-automation-scripts-on-testmu-selenium-grid.md +++ /dev/null @@ -1,88 +0,0 @@ -# Selenium With WD - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Run WD tests on the TestMu AI cloud grid. This guide covers setup, running a sample test, configuring capabilities, and testing locally hosted pages. - -**Unmaintained Framework** -The WD library is largely unmaintained. Consider migrating to [WebDriverIO](/support/docs/webdriverio-with-selenium-running-webdriverio-automation-scripts-on-testmu-selenium-grid/) or [Playwright](/support/docs/playwright-testing/). This guide is maintained for existing users. - -**Sample repo** -All the code used in this guide is available in the sample repository. - - View on GitHub - -## Prerequisites - -Complete these steps before running WD Selenium tests on TestMu AI. - -1. Create a [TestMu AI account](https://www.testmuai.com/register/?redirectTo=https://accounts.lambdatest.com/dashboard) and get your username and access key from the dashboard. -2. Install **NodeJS** v6 or newer from [nodejs.org](https://nodejs.org/en/). -3. Install **npm** from the [official npm website](https://www.npmjs.com/). -4. Download [Selenium JavaScript bindings](https://www.selenium.dev/downloads/) from the official website. - -## Step 1: Clone the Sample Project - -Clone the TestMu AI WD Selenium sample repository to your local machine. - -```bash -git clone https://github.com/LambdaTest/wd-selenium-sample -cd wd-selenium-sample -``` - -Install the required dependencies: -```bash -npm i -``` - -## Step 2: Set Your Credentials - -Set your TestMu AI username and access key as environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -## Step 3: Configure Your Test Capabilities - -Update the capabilities in `conf/single.conf.js` to define the browser and platform settings. - -```js -capabilities: [{ -browserName: 'chrome', -platform: 'Windows 10', -version: 'latest', -name: "single-test", -build: "wd-lambdatest-sample", -geoLocation : "US" -}] -``` - -Generate capabilities for your test requirements with the [Capabilities Generator](https://www.testmuai.com/capabilities-generator/). - -## Step 4: Run the Test - -Execute the test using the following command. - -**Single test:** -```bash -npm run single -``` - -**Parallel tests:** -```bash -npm run parallel -``` - -## Step 5: View Your Results - -After running the test, view your results on the [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build). - -The dashboard provides: -- Video recordings of each test session -- Screenshots captured at each step -- Console logs from the browser -- Network logs for debugging -- Detailed command logs diff --git a/static/docs/web-scanner-accessibility-scan.md b/static/docs/web-scanner-accessibility-scan.md deleted file mode 100644 index 95b600836..000000000 --- a/static/docs/web-scanner-accessibility-scan.md +++ /dev/null @@ -1,45 +0,0 @@ -# Starting an Accessibility Scan with Web Scanner - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Web Scanner exposes Accessibility scanning as an integrated workflow for site-wide URL-based scans. It is separate from Accessibility DevTools and from Accessibility Test Scheduling, even though the configuration concepts can overlap. - -When you are already in Web Scanner, this page walks through configuring an Accessibility scan and interpreting the main options (WCAG target, review and best-practice toggles, local testing, and login settings) before you start the run. Use it to align scan settings with your environment and reporting goals on that product surface. - -## When to use this - -Use this page when you are already in Web Scanner and want to run an Accessibility scan against a set of URLs from that product surface. - -## Prerequisites (onboarding) - -1. You can open the **Web Scanner** product and see **[Getting Started with Web Scanner](/support/docs/web-scanner-getting-started/)** flows without errors. -2. You have a **URL list** or plan to add URLs through the wizard, and permission to scan those hosts. -3. For private environments, install or enable **TestMu AI Tunnel** per [Testing Locally Hosted Pages](/support/docs/testing-locally-hosted-pages/) **before** starting a scan that depends on local routing. -4. For authenticated sites, collect **login steps** or credentials formats the wizard supports (see **Login Configurations** on this page). -5. Decide the **WCAG target** and whether **best-practice** or **needs-review** items belong in the first run. Narrower scopes make first triage faster. - -## Key configuration options - -- **WCAG Version** selects the standards target for the scan. -- **Review Issues** includes findings that may need manual validation. -- **Best Practices Issues** includes non-blocking improvement guidance. -- **Local Testing** lets you scan pre-production or internal environments through TestMu AI Tunnel. -- **Login Configurations** help Web Scanner access authenticated areas of the site. - -## Step-by-step - -1. Open the Web Scanner scan wizard and choose **Accessibility** as the scan type. -2. Review the default settings, including WCAG version and issue toggles. -3. Configure local testing or login flows if your target environment is private or protected. -4. Start the scan and wait for the build to complete. -5. Open the report to review issue counts, severity trends, page-level findings, and export options. - -## Product boundary - -Web Scanner is an integrated product surface for Accessibility scans. If you want browser-based manual analysis, use [Accessibility DevTools](/support/docs/accessibility-devtools/). If you want recurring Accessibility-native scheduling outside Web Scanner, use [Test Scheduling - Sitemap (Overview)](/support/docs/accessibility-test-scheduling/). - -## Related docs - -- [Getting Started with Web Scanner](/support/docs/web-scanner-getting-started/) -- [Test Scheduling - Sitemap (Overview)](/support/docs/accessibility-test-scheduling/) -- [Navigating the Dashboard](/support/docs/accessibility-testing-navigating-dashboard/) diff --git a/static/docs/web-scanner-adding-urls.md b/static/docs/web-scanner-adding-urls.md deleted file mode 100644 index 13afd7fc5..000000000 --- a/static/docs/web-scanner-adding-urls.md +++ /dev/null @@ -1,70 +0,0 @@ -# Adding URLs for Web Scanner - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Step 3: Add URLs to Your Scan - -After configuring the scan type (Step 2), the **Add URLs** section (Step 3) allows you to populate the test targets. You can choose from one of the following input methods. - -### 1. Add URLs Manually - -Use this method when you want full control over each scanned URL or you’re testing only a handful of critical paths. - -**Instructions:** -* Paste each URL in a new line or input box. -* Ensure proper formatting (`https://` required). - -**Best for:** -* Login pages, checkout flows, admin pages. - -### 2. Import via CSV File - -Use this method when you already maintain URL lists in a spreadsheet or text file. - -**CSV Format:** -One URL per line (no header). -```csv -[https://example.com](https://example.com) -[https://example.com/pricing](https://example.com/pricing) -[https://example.com/contact](https://example.com/contact) -``` - -**Instructions**: - -* Click on Import CSV. - -* Upload your .csv file. - -* URLs will auto-populate the scan list. - -Ideal for: - -* Large teams with pre-approved page lists. - -* Reusing URL sets across projects. - -### 3. Import via Sitemap -Use this method when: -* you want to extract URLs from an existing sitemap -* you’re scanning a dynamic, live website. - -**Instructions**: -* Paste the sitemap URL (XML format). -* Click Fetch Sitemap. -* The scanner will auto-detect URLs. -* You can selectively check/uncheck desired pages. - -Ideal for: -* Scanning large public websites. -* Ensuring complete coverage. - -### URL Limits and Validations -**Max URL Limit**: would depend upon your SmartUI (*for visual scan*) and Accessibility (*for accessibility scan*) plan. - -**All URLs must be valid and working.** - -**Duplicate URLs will be auto-filtered** - -**Reusability Tips** -* For frequently used URL sets, save them in CSVs and upload as needed. -* Create and save CSV url lists from sitemap and use in CSV mode for future. diff --git a/static/docs/web-scanner-advanced-features.md b/static/docs/web-scanner-advanced-features.md deleted file mode 100644 index 98e6bc5fb..000000000 --- a/static/docs/web-scanner-advanced-features.md +++ /dev/null @@ -1,306 +0,0 @@ -# Advanced Features in Web Scanner - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Overview - -Web Scanner provides advanced configuration options to enhance your scanning capabilities. These features allow you to test authenticated pages, access network-restricted resources, simulate different geographic locations, and scan locally hosted applications. - -The advanced features are accessible from the **Advanced Configurations** section when setting up your scan. You can configure: - -- **Authentication**: Test pages protected by Basic Auth or form-based authentication -- **Dedicated Proxy**: Access network-restricted resources through a whitelisted IP -- **Geolocation**: Simulate scans from different geographic locations -- **Tunnel**: Scan locally hosted web pages and applications - -## Authentication - -Web Scanner supports two types of authentication methods to scan pages that require login credentials: - -### Basic Authentication - -Basic Authentication is an HTTP authentication method where credentials are sent in the Authorization header. This method is useful for pages protected by HTTP Basic Auth. - -#### When to Use Basic Authentication - -- Pages protected by HTTP Basic Auth (e.g., `https://username:password@example.com`) -- Staging or development environments with basic authentication -- Pages that require simple username/password authentication - -#### Configuration Steps - -1. Navigate to **Advanced Configurations** in your scan setup -2. Click **Add login configurations** -3. In the **Configuration Overview** section: - - Enter a **Configuration name** (e.g., "Staging Environment") - - Select **Authentication type** as **Basic authentication** -4. In the **Login Information** section: - - Enter your **Username** - - Enter your **Password** -5. Click **Add & save configuration** - -> **Note:** It is recommended to use credentials of a dummy/test account for scanning purposes. Never use production credentials. - -### Form Authentication - -Form Authentication allows you to authenticate using login forms on your website. This method requires CSS selectors to identify the username and password fields. - -#### When to Use Form Authentication - -- Websites with custom login forms -- Pages that require multi-step authentication -- Applications with complex authentication flows - -#### Configuration Steps - -1. Navigate to **Advanced Configurations** in your scan setup -2. Click **Add login configurations** -3. In the **Configuration Overview** section: - - Enter a **Configuration name** (e.g., "Production Login") - - Select **Authentication type** as **Form authentication** -4. In the **Login Information** section: - - Enter the **Login page URL** (e.g., `https://www.yourwebsite.com/login`) - - Enter your **Username** - - Enter your **Password** -5. In the **CSS Selector** section: - - Enter the CSS selector for the username field (e.g., `#username`, `.login-email`, `input[name="email"]`) - - Enter the CSS selector for the password field (e.g., `#password`, `.login-password`, `input[type="password"]`) -6. Click **Add & save configuration** - -#### Finding CSS Selectors - -To find the CSS selector for form fields: - -1. Open your login page in a browser -2. Right-click on the username field and select **Inspect** -3. In the developer tools, right-click on the highlighted HTML element -4. Select **Copy** > **Copy selector** or **Copy CSS selector** -5. Use this selector in the Web Scanner configuration - -Common CSS selector patterns: -- ID selector: `#username`, `#password` -- Class selector: `.login-email`, `.login-password` -- Attribute selector: `input[name="email"]`, `input[type="password"]` -- Combined selector: `form.login-form input#username` - -#### Best Practices for Authentication - -- **Use Test Accounts**: Always use dedicated test accounts with minimal permissions -- **Secure Credentials**: Never share authentication configurations publicly -- **Verify Selectors**: Test your CSS selectors before running full scans -- **Handle Dynamic Content**: Ensure selectors work even if page structure changes slightly -- **Multiple Configurations**: You can create multiple authentication configurations for different environments - -## Dedicated Proxy - -Dedicated Proxy allows Web Scanner to access network-restricted resources through a whitelisted IP address. This is particularly useful for enterprise environments where resources are protected by IP whitelisting. - -For detailed information about dedicated proxy configuration and IP whitelisting, refer to the [Dedicated Proxy Documentation](/support/docs/dedicated-proxy). - -### When to Use Dedicated Proxy - -- Accessing resources behind corporate firewalls -- Testing applications with IP-based access restrictions -- Ensuring consistent IP address for compliance requirements -- Enterprise environments requiring network isolation - -### How Dedicated Proxy Works - -1. TestMu AI allocates a dedicated proxy server with a static IP address -2. Your network administrator whitelists this IP address -3. Web Scanner routes all scan requests through the dedicated proxy -4. The proxy accesses your network-restricted resources using the whitelisted IP - -### Configuration Steps - -1. Navigate to **Advanced Configurations** in your scan setup -2. Click on **Dedicated Proxy Configuration** -3. Select your dedicated proxy region from the dropdown -4. The proxy will be active for your scan session - -> **Note:** Dedicated Proxy is an enterprise feature. Contact your TestMu AI account manager to enable this feature for your organization. - -### Benefits of Dedicated Proxy - -- **Single IP Whitelisting**: Only one IP address needs to be whitelisted instead of multiple IP ranges -- **Consistent Access**: All scans use the same IP address for predictable access patterns -- **Network Security**: Maintains your network security policies while enabling cloud-based scanning -- **Compliance**: Meets enterprise security and compliance requirements - -### Prerequisites - -- Your organization must have a dedicated proxy enabled (enterprise feature) -- Network resources must be publicly discoverable but access-restricted -- The dedicated proxy IP must be whitelisted by your network administrators - -## Geolocation - -Geolocation allows you to simulate scans from different geographic locations. This is essential for testing region-specific content, compliance requirements, and localized user experiences. - -### When to Use Geolocation - -- Testing region-specific pricing or content -- Validating compliance with local regulations (GDPR, CCPA, etc.) -- Testing localized user experiences -- Verifying geo-targeted marketing campaigns -- Testing region-specific cookie consent banners - -### Configuration Steps - -Geolocation can be configured in two ways: - -**Method 1: Advanced Configurations** -1. Navigate to **Advanced Configurations** in your scan setup -2. Click on **Geolocation Selection** -3. Use the search bar to find and select your desired country or region -4. The scan will be executed with traffic routed through the selected location - -**Method 2: Scheduling Phase** -- Geolocation can also be configured by selecting the timezone during the scheduling phase. When you set up your scan schedule, you can choose a timezone that corresponds to your desired geographic location. - -For more information about geolocation capabilities and supported locations, refer to the [Selenium Geolocation Documentation](/support/docs/selenium-geolocation-capabilities). - -### Supported Locations - -Web Scanner supports geolocation simulation for numerous countries and regions worldwide. You can select from a comprehensive list including: - -- **Americas**: United States, Canada, Brazil, Mexico, Argentina, and more -- **Europe**: United Kingdom, Germany, France, Spain, Italy, and more -- **Asia-Pacific**: India, Japan, China, Australia, Singapore, and more -- **Middle East & Africa**: UAE, Saudi Arabia, South Africa, and more - -### Use Cases - -1. **Localized Content Testing**: Verify that region-specific content displays correctly -2. **Compliance Testing**: Ensure cookie consent banners and privacy notices appear for specific regions -3. **Pricing Validation**: Test that region-specific pricing is displayed accurately -4. **Language Testing**: Verify that the correct language is served based on geolocation -5. **Performance Testing**: Test how your site performs from different geographic locations - -### Important Notes - -- Geolocation affects the IP address used for scanning, not just the browser location -- Some features may not be available in all regions -- Geolocation cannot be used simultaneously with Tunnel in a single scan (use parallel scans if both are needed) - -## Tunnel - -Tunnel enables Web Scanner to access locally hosted web pages and applications. This is essential for testing development environments, staging servers, or applications behind firewalls. - -For comprehensive tunnel setup instructions, configuration options, and troubleshooting, refer to the [Testing Locally Hosted Pages Documentation](/support/docs/testing-locally-hosted-pages). - -### When to Use Tunnel - -- Testing locally hosted applications (localhost, 127.0.0.1) -- Scanning development or staging environments -- Accessing applications behind corporate firewalls -- Testing applications not publicly accessible - -### How Tunnel Works - -TestMu AI Tunnel creates a secure, encrypted connection between your local machine and the TestMu AI cloud infrastructure: - -1. You download and run the TestMu AI Tunnel binary on your local machine -2. The tunnel establishes a secure connection to TestMu AI's cloud -3. Web Scanner routes requests through the tunnel to access your local resources -4. All communication is encrypted and secure - -### Configuration Steps - -#### Step 1: Download Tunnel Binary - -Download the TestMu AI Tunnel binary for your platform: - -**Windows (64-bit)** -``` -https://downloads.lambdatest.com/tunnel/v3/windows/amd64/LT.exe -``` - -**Windows (32-bit)** -``` -https://downloads.lambdatest.com/tunnel/v3/windows/386/LT.exe -``` - -**macOS (Intel)** -``` -https://downloads.lambdatest.com/tunnel/v3/mac/amd64/LT -``` - -**macOS (Apple Silicon)** -``` -https://downloads.lambdatest.com/tunnel/v3/mac/arm64/LT -``` - -**Linux (64-bit)** -``` -https://downloads.lambdatest.com/tunnel/v3/linux/amd64/LT -``` - -**Linux (ARM64)** -``` -https://downloads.lambdatest.com/tunnel/v3/linux/arm64/LT -``` - -#### Step 2: Start the Tunnel - -```bash -LT.exe --user YOUR_LAMBDATEST_USERNAME --key YOUR_LAMBDATEST_ACCESS_KEY --tunnelName YOUR_TUNNEL_NAME -``` - -```bash -chmod +x LT -./LT --user YOUR_LAMBDATEST_USERNAME --key YOUR_LAMBDATEST_ACCESS_KEY --tunnelName YOUR_TUNNEL_NAME -``` - -Replace: -- `YOUR_LAMBDATEST_USERNAME`: Your TestMu AI username -- `YOUR_LAMBDATEST_ACCESS_KEY`: Your TestMu AI access key -- `YOUR_TUNNEL_NAME`: A unique name for your tunnel (e.g., "web-scanner-tunnel") - -#### Step 3: Configure Tunnel in Web Scanner - -1. Navigate to **Advanced Configurations** in your scan setup -2. Click on **Add a local testing setup** -3. Select your tunnel from the dropdown (it should show as "Active" if running) -4. The tunnel will be used to access your locally hosted pages - -### Tunnel Benefits - -- **Secure Connection**: All tunnel traffic is encrypted using TLS 1.2 -- **No Public Access Required**: Test applications that aren't publicly accessible -- **Local Development Testing**: Scan localhost and development servers -- **Firewall Bypass**: Access resources behind corporate firewalls -- **Cost Effective**: No need to deploy applications publicly for testing - -### Tunnel Best Practices - -1. **Keep Tunnel Running**: Ensure the tunnel is running before starting your scan -2. **Unique Tunnel Names**: Use descriptive, unique names for multiple tunnels -3. **Network Stability**: Ensure stable network connection for reliable tunnel operation -4. **Security**: Never share tunnel credentials or expose them in public repositories -5. **Resource Management**: Close tunnels when not in use to free up resources - -### Troubleshooting Tunnel Issues - -- **Connection Failures**: Verify your network allows outbound connections on port 443 -- **Tunnel Not Appearing**: Ensure the tunnel is running and you've used the correct credentials -- **Timeout Errors**: Check your local server is running and accessible -- **Firewall Issues**: Ensure your firewall allows the tunnel binary to make outbound connections - -## Combining Advanced Features - -You can combine multiple advanced features in a single scan: - -- **Authentication + Tunnel**: Test authenticated pages on locally hosted applications -- **Geolocation + Dedicated Proxy**: Test region-specific content through a dedicated proxy -- **Authentication + Geolocation**: Test authenticated pages from specific geographic locations - -> **Note:** Some feature combinations may have limitations. For example, Tunnel and Geolocation cannot be used simultaneously in a single scan. Use parallel scans if you need both features. - -## Additional Resources - -- [Web Scanner Overview](/support/docs/web-scanner-overview) -- [Starting a Visual UI Scan](/support/docs/web-scanner-visual-scan) -- [Testing Locally Hosted Pages](/support/docs/testing-locally-hosted-pages) -- [Dedicated Proxy Documentation](/support/docs/dedicated-proxy) -- [HTTP Basic Authentication](/support/docs/http-basic-authentication) diff --git a/static/docs/web-scanner-getting-started.md b/static/docs/web-scanner-getting-started.md deleted file mode 100644 index f12986590..000000000 --- a/static/docs/web-scanner-getting-started.md +++ /dev/null @@ -1,94 +0,0 @@ -# Getting Started with TestMu AI Web Scanner - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Accessing the Dashboard - -Navigate to: `https://webscanner.lambdatest.com/webscan/builds` - -If you use **Accessibility Test Scheduling** inside the main Accessibility product (sitemaps, crawlers, URL lists there) rather than this Web Scanner web app, follow [Test Scheduling](/support/docs/accessibility-test-scheduling/) instead. - -You will see a unified dashboard with the following primary controls: - -### Scan Type Tabs - -At the top left, you can switch between: -* Visual UI Scans -* Accessibility Scans - -Each tab displays its own scheduled scans, configurations, and build statuses. - -### Search and Filters - -Use the search bar to locate specific scans by name. Filters may be added in future iterations for build status, user, or date. - -### Login Configurations - -For your accessibility scans, login configurations allow users to: -* Add Basic Auth credentials -* Add form authentication (via CSS selectors) - -These configurations ensure that login-protected or session-gated URLs can still be scanned. - -### Schedule Scan Button - -The primary CTA on the dashboard, **Schedule Scan**, triggers the full scan creation wizard. - -## How to Create a New Scan - -Click the **Schedule Scan** button to launch the 4-step wizard. This process applies to both Visual and Accessibility scans. Platform-specific configuration steps appear based on your selection. - -### Step 1: Scan Overview - -* **Scan Name:** Add an identifiable title -* **Platform:** Choose between: - * Visual UI - * Accessibility - -### Step 2: Scan Type Configuration (Dynamic) - -Depending on the selected platform, you’ll either see Visual UI configuration or Accessibility configuration. You may use “Default“ settings or for more information follow the details provided: - -* [Starting a Visual Scan](/support/docs/web-scanner-visual-scan) -* [Starting an Accessibility Scan](/support/docs/web-scanner-accessibility-scan) - -### Step 3: Add URLs - -You can populate the scan target using three options: -* Manual Input: Paste multiple URLs -* CSV Upload: Upload a file with 1 URL per line -* Sitemap Import: Paste sitemap XML URL and select from extracted URLs - -The maximum limit is 1,000 URLs per scan. - -For more details you may refer to our guide on [Adding URLs for Web Scanner](/support/docs/web-scanner-adding-urls). - -### Step 4: Schedule the Scan - -Choose the frequency and timing for this scan: -* **Quick Scan:** One-time run -* **Recurring Scan:** - * Select days (e.g., Mon, Wed, Fri) - * Select time (e.g., 08:30 PM) - * Select timezone (e.g., UTC+5:30 Asia/Calcutta) -* Enable/disable **Run Initial Scan** (Visual UI only) - -## Post-Scan Scheduling - -Once scheduled, the scan appears in the appropriate tab of the dashboard. -* **Visual UI:** Scan redirects to SmartUI comparison build -* **Accessibility:** Scan redirects to detailed Accessibility report - -Each scan has an action menu (3-dots icon) with the following options: - -**Visual UI:** -* View History (SmartUI builds) -* Edit Scan Config (recurrence, time) -* Stop Recurring Scan -* Delete Scan - -**Accessibility:** -* Rename Scan -* Delete Scan - -Detailed controls are available after redirecting to the Accessibility dashboard. diff --git a/static/docs/web-scanner-overview.md b/static/docs/web-scanner-overview.md deleted file mode 100644 index ad4f1993b..000000000 --- a/static/docs/web-scanner-overview.md +++ /dev/null @@ -1,52 +0,0 @@ -# Web Scanner Overview - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## What is TestMu AI Web Scanner? - -TestMu AI Web Scanner is an AI-powered, browser-based scanning platform designed to help teams identify and resolve visual and accessibility issues across web applications. It enables both automated Visual UI Regression Testing and WCAG-compliant Accessibility Testing, allowing organizations to proactively catch visual bugs and accessibility violations across thousands of URLs at scale. - -This tool empowers QA teams, developers, and compliance managers to perform scheduled scans on public or locally-hosted sites, compare builds visually with pixel-perfect precision, and generate audit-ready accessibility reports. - -## Why Web Scanner? - -Manual testing of large web applications for layout shifts, broken UI, or WCAG non-compliance is slow and error-prone. TestMu AI Web Scanner solves this by enabling: - -* Automated visual comparisons across multiple browsers and resolutions powered by SmartUI -* Accessibility audits powered by dynamic WCAG checks -* Smart scheduling of one-time or recurring scans -* Support for authenticated and local testing pages -* Seamless redirection to SmartUI and Accessibility Dashboards - -## Supported Scan Types - -### 1. Visual UI Scan - -Visual scans detect pixel-based layout changes, design mismatches, missing elements, or unintended visual regressions by comparing screenshots taken at scheduled intervals. - -#### Key Capabilities - -- **Cross-Browser Validation**: Test UI across Chrome, Firefox, Edge, and Safari. -- **Responsive Testing**: Check layouts on 8 desktop sizes and 200+ mobile viewports (Android & iOS). -- **Custom Configurations**: Override defaults with your own SmartUI JSON settings. -- **Accurate Screenshots**: Use delays to capture pages with animations or dynamic content. -- **History & Comparison**: Access past scans to track changes, compare builds, and spot regressions. - -### 2. Accessibility Scan - -Accessibility scans audit your pages against WCAG 2.x standards, surfacing violations and recommendations to improve inclusivity. - -**Key Features:** -* WCAG 2.0/2.1 AA compliance and more -* "Needs Review" toggle for human-verifiable issues -* Best Practices flagging -* Deep scan reports with severity filters -* Smart filters for recurring violations -* Exportable reports - -## Ideal Users - -* **Frontend QA Teams:** Perform nightly visual regression scans across builds -* **Accessibility Engineers:** Validate compliance across thousands of URLs -* **Product Owners:** Monitor web releases for visual or WCAG health -* **Agencies:** Provide clients with before/after visual proof and compliance reports diff --git a/static/docs/web-scanner-scheduling-options.md b/static/docs/web-scanner-scheduling-options.md deleted file mode 100644 index 3ed6555c2..000000000 --- a/static/docs/web-scanner-scheduling-options.md +++ /dev/null @@ -1,49 +0,0 @@ -# Scheduling Options in Web Scanner - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Step 4: Schedule Scan - -Once scan configuration and URLs are complete, you’ll be prompted to set your scan frequency and runtime preferences. - -## Scheduling Modes - -### Quick Scan (One-Time) - -* Executes the scan immediately upon creation. -* No future runs will be scheduled. -* Useful for ad hoc validations, staging checks, or one-off deployments. - -### Recurring Scan - -* Repeats the scan based on chosen days and time. -* Ensures continuous monitoring of regressions or WCAG compliance. - -**Fields:** -* **Days:** Select one or more days (e.g., Mon, Wed, Fri). -* **Time:** Choose execution time (in 12-hour or 24-hour format). -* **Time Zone:** Select the appropriate time zone (e.g., UTC+5:30 Asia/Calcutta). - -### Run Initial Scan (Visual UI Only) - -* Applies only to Visual Scans and is enabled by default. -* Triggers a baseline SmartUI screenshot set right after scheduling. -* This is required for future builds to have a baseline to compare against. - -**Example Use Case:** -You're scheduling a scan for every Monday. With "Run Initial Scan" enabled, a baseline will be created immediately. This acts as the visual reference for next Monday’s screenshots. - -## Modify or Cancel Scheduled Scans - -### Visual Scans: - -* Access from the **Builds Dashboard**. -* Click the 3-dot menu and select **Edit**. -* Modify the scan name, schedule days, time, and time zone. -* **Options:** Edit Schedule, Stop Recurring Scan, Delete Scan. - -### Accessibility Scans: - -* Navigate to the **Scheduled Accessibility** section. -* **Options:** Rename, Delete. -* For advanced options like issue filtering and exporting, open the report in the Accessibility Dashboard. diff --git a/static/docs/web-scanner-visual-scan.md b/static/docs/web-scanner-visual-scan.md deleted file mode 100644 index 702be935c..000000000 --- a/static/docs/web-scanner-visual-scan.md +++ /dev/null @@ -1,104 +0,0 @@ -# Starting a Visual UI Scan with Web Scanner - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -## Purpose of Visual UI Scan - -Visual UI scans identify layout shifts, broken styles, font mismatches, and other design anomalies across web pages. It compares page screenshots taken during each run against a previously saved baseline (initial scan). - -Scans run across combinations of: -* **Browsers:** Chrome, Firefox, Safari, Edge -* **Resolutions:** A total of 30 viewport options are available, including 10 standard desktop, 10 Android, and 10 iOS device viewports. - -Results are redirected to the SmartUI Dashboard for detailed comparison. - -## Step 2 (If Visual Scan is Selected) - -Once **Visual UI** is selected in Step 1 of the scheduling wizard, the following configuration appears. - -### Default Settings - -* **Browsers:** Chrome, Firefox, Safari, Edge -* **Resolutions:** Desktop (1920x1080, 1366x768, 1024x768), Tablet, Mobile (Galaxy A12, Galaxy A31, iPhone 14 Pro) -* **Full Page Mode “ON“:** Captures the full length of your website regardless of viewport height limitations. -* **Delay “5 Seconds“:** To wait for animations or content to stabilize. - -### Advanced Options (Visual UI) - -Click the "Advanced” Tab to unlock additional configuration tabs. - -#### SmartUI Config Upload -You can create your own `.smartui.json` file for full control over: -* Browser selection -* Resolutions -* Delay setting -* The integrated JSON validator checks your configuration. - -#### Default SmartUI Config - -{` - "web": { - "browsers": [ - "chrome", - "firefox", - "safari", - "edge" - ], - "viewports": [ - [ - 1920 - ], - [ - 1366 - ], - [ - 1024 - ] - ] - }, - "mobile": { - "devices": [ - "iPhone 14", - "Galaxy S24" - ], - "fullPage": true, - "orientation": "portrait" - }, - "waitForTimeout": 1000, - "smartIgnore": true, - "enableJavaScript": false, - "ignoreHTTPSErrors": false, - "useGlobalCache": false -`} - -> **Note:** Using a SmartUI config bypasses manual browser, resolution, and delay selections, even in the "Edit Config" mode. - -#### Navigating Results for your Visual Scan - -## Dashboard Overview - -After running your tests, the **Visual Build** becomes available in the dashboard. - -### Build Status -Easily track the state of each screenshot: **Approved**, **Changes Found**, **New**, **Under Review**, and more. - -### Three-Dot Menu Options -Access additional functions for your build: - -1. **View History** - Browse older builds with complete stats for quick comparison. - -2. **Edit Scan Config** - Modify scan settings such as **schedule** and **viewports**. - -3. **Run Scan Now** - Run a new scan on demand and create a new build for comparison. - -4. **Delete Scan** - Remove a scan from the dashboard. - -5. **Stop Recurring Scan** - Halt any scheduled recurring scans instantly. - -**Access your SmartUI Dashboard**: -SmartUI detects pixel-based layout changes, design mismatches, missing elements, or unintended visual regressions by comparing screenshots taken at scheduled intervals. diff --git a/static/docs/web-testing-on-real-devices.md b/static/docs/web-testing-on-real-devices.md deleted file mode 100644 index 4f6d9c21e..000000000 --- a/static/docs/web-testing-on-real-devices.md +++ /dev/null @@ -1,101 +0,0 @@ -# Web Testing On Real Devices - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Mobile website testing involves testing websites and web apps across numerous mobile browsers like Chrome, Safari, Opera, etc to make sure your websites (and web apps) are rendering seamlessly across different mobile browsers and devices. TestMu AI real device cloud allows you to manually test websites and web apps on the range of real Android and iOS devices. - -This documentation covers how to perform web testing across real Android and iOS devices on the TestMu AI platform. - ->**Note**: Currently Web testing on real devices is in the Beta phase. To access this feature, contact TestMu AI Support. - -## Testing Websites And Web Apps On Real Devices - -For demonstration, we will perform real-time web testing on real Android devices. - -1. Sign into your TestMu AI account. From the left menu, navigate to Real Time Testing and click **Browser Testing** tab. - -2. Select Android trademark symbol and choose the test environments like BRAND, DEVICE/OS, and BROWSER on which you wish to run the test. After that, click **START**. - -3. A cloud-hosted real device environment will launch where you can perform live-interactive testing of websites for filing bugs. - -## Screenshots And Bug Logging - -One-click bug logging allows users to capture screenshots for future reference and log bugs while performing real-time mobile browser testing. - -1. To log a bug, click on the Bug icon. - -2. Once the screenshot has been captured, you can either download it by clicking the download option or use the built-in image editor tools to highlight any bugs or tasks. After that, click **MARK AS BUG**. - -3. An integration ticket will pop up. Provide the bugs/issue/task details and click **Create Issue**. Your bugs will be marked to your integrated bug-tracking tool. - -*TestMu AI offers 65+ integrations with your favourite tool for Bug tracking, CI/CD, Project Management, Codeless Automation, and more. Check out [TestMu AI integrations](https://www.testmuai.com/integrations).* - -## Video Recording Of A Test Session - -1. Click on the Video recorder icon from the left tool bar. - -2. The video recording of the current test session will start. To stop the test session recording, click again on the same Video recorder icon. - -## Viewing Gallery - -All the captured screenshots and test session video recordings are saved in the Gallery. - -1. To view the screenshots (or videos), click on the Gallery icon from the left tool bar. - -2. You can download these screenshots (or video recordings) to your local system. To download, click on the download option. - -## Accessing DevTools - -DevTools allows you to launch a web app within a mobile browser and do the live-debugging. - -1. To access the DevTools, click on **DevTools** icon. - -The DevTools will be launched and you can perform the live-debugging of your websites. - -## Geolocation Testing And Setting GPS Geolocation - -You can test mobile websites on different geolocations and GPS geolocations. - -1. Click on the location icon from the left tool bar and you'll notice a dropdown to select a geolocation. - -2. From the the dropdown, select the desired country on which you want to run the test. - -Based on the selected country, the cloud machine would subsequently be routed to a specified IP address. - -You can also test your websites based on the specific GPS Geolocation. - -1. Click on the location icon and select **Set Location**. - -2. Enter the location address (or latitude and longitude) and then click **Yes**. - -After few seconds, your GPS location will be set. - -## Device Controls - -Using the device controls, you can change the keyboard language, rotate between portrait and landscape modes, return to the previous page, and increase/decrease the volume. - -### Device Orientation - -From the left toolbar, click on the **Rotate** button. - -The device will be rotated to landscape mode or vice-versa. - -### Return To Previous Page, Increase And Decrease Volume - -Click on **Device Controls** icon and you'll notice the three icons for return to the previous page, increase volume and decrease volume. - -Here are following list of buttons with along with their respective action. - -| Action | Icon | Description | -|--------|------|-------------| -| Return | | Return to the previous page.| -| Increase | | Increase the volume.| -| Decrease | | Decrease the volume.| - -## Ending A Test Session - -1. Click on the End Session icon. - -2. A pop-up box will appear, select **Yes, End Session**. - ->In case you have any questions or need any additional information, drop them at our **24X7 Chat Support** or mail us directly at support@testmuai.com. diff --git a/static/docs/webdriverio-5-6-2-with-selenium-running-automation-scripts-on-testmu-selenium-grid.md b/static/docs/webdriverio-5-6-2-with-selenium-running-automation-scripts-on-testmu-selenium-grid.md deleted file mode 100644 index 9ab1d2a25..000000000 --- a/static/docs/webdriverio-5-6-2-with-selenium-running-automation-scripts-on-testmu-selenium-grid.md +++ /dev/null @@ -1,129 +0,0 @@ -# Selenium With WebDriverIO 5.6.2 - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Run WebDriverIO 5.6.2 tests on the TestMu AI cloud grid. This guide covers setup, running a sample test, configuring capabilities, and testing locally hosted pages. - -**Outdated Version** -This guide covers WebDriverIO 5.6.2. The current version is 8+. For the latest WebDriverIO guide, see [Selenium With WebDriverIO](/support/docs/webdriverio-with-selenium-running-webdriverio-automation-scripts-on-testmu-selenium-grid/). - -**Sample repo** -All the code used in this guide is available in the sample repository. - - View on GitHub - -## Prerequisites - -Complete these steps before running WebDriverIO 5.6.2 Selenium tests on TestMu AI. - -1. Create a [TestMu AI account](https://www.testmuai.com/register/?redirectTo=https://accounts.lambdatest.com/dashboard) and get your username and access key from the dashboard. -2. Install **NodeJS** v6 or newer from [nodejs.org](https://nodejs.org/en/). -3. Install **WebDriverIO**: `npm install webdriverio` -4. Download [Selenium JavaScript bindings](https://www.selenium.dev/downloads/) from the official website. - -## Step 1: Clone the Sample Project - -Clone the TestMu AI WebDriverIO Selenium sample repository to your local machine. - -```bash -git clone https://github.com/LambdaTest/webdriverio-selenium -cd webdriverio-selenium -``` - -Install the required dependencies: -```bash -npm i selenium-webdriver -npm install -``` - -## Step 2: Set Your Credentials - -Set your TestMu AI username and access key as environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -## Step 3: Configure Your Test Capabilities - -Update the capabilities in the config file to define the browser and platform settings. - -``` js -user= process.env.LT_USERNAME || "", -key= process.env.LT_ACCESS_KEY || "", - -exports.config = { - -updateJob: false, -user, -key, -specs: [ -'./tests/specs/single_test.js' -], -exclude: [], - -capabilities: [{ -browserName: 'chrome', -version:"64.0", -name:"Test webdriverio", -build:"build 1", -}], -sync: true, -logLevel: 'info', -coloredLogs: true, -screenshotPath: './errorShots/', -baseUrl: '', -waitforTimeout: 100000, -connectionRetryTimeout: 90000, -connectionRetryCount: 1, -path: '/wd/hub', -hostname: 'hub.lambdatest.com', -port: 80, - -framework: 'mocha', -mochaOpts: { -ui: 'bdd' -} -} -``` - -Generate capabilities for your test requirements with the [Capabilities Generator](https://www.testmuai.com/capabilities-generator/). - -## Step 4: Run the Test - -Execute the test using the following command. - -```bash -npm run single -``` - -## Step 5: View Your Results - -After running the test, view your results on the [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build). - -The dashboard provides: -- Video recordings of each test session -- Screenshots captured at each step -- Console logs from the browser -- Network logs for debugging -- Detailed command logs - -## Run WebDriverIO 5.6.2 Tests Using Agent Skills - -Use AI coding assistants to generate and run WebDriverIO 5.6.2 tests with the TestMu AI Agent Skill. - -The [webdriverio-skill](https://github.com/LambdaTest/agent-skills/tree/main/webdriverio-skill) is part of [TestMu AI Agent Skills](https://github.com/LambdaTest/agent-skills/) - structured packages that teach AI coding assistants how to write production-grade test automation. - -Install the skill: - -```bash -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/webdriverio-skill .claude/skills/ - -# For Cursor / Copilot -cp -r agent-skills/webdriverio-skill .cursor/skills/ -``` - -Install all available framework skills at once by cloning the repository directly into your tool's skills directory (e.g., `.claude/skills/`, `.cursor/skills/`). diff --git a/static/docs/webdriverio-on-hyperexecute-grid.md b/static/docs/webdriverio-on-hyperexecute-grid.md deleted file mode 100644 index 7fb9524f3..000000000 --- a/static/docs/webdriverio-on-hyperexecute-grid.md +++ /dev/null @@ -1,233 +0,0 @@ -# Running WebdriverIO Framework Tests on HyperExecute - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -WebdriverIO is a widely-used testing framework for Java applications, designed to simplify and enhance the testing process for developers. It provides a flexible and powerful platform for running test suites, enabling effective unit testing, integration testing, and end-to-end testing of Java applications - -HyperExecute is an AI Native Test Orchestration Cloud Platform that empowers you to run **end-to-end** tests **quickly** and **efficiently**. It provides Just-in-Time (JIT) testing infrastructure with fast execution **speeds**, **smart orchestration**, and **detailed logs**. - -This guide details how to execute your **WebdriverIO** framework tests on **HyperExecute** via two different methods: - -- [**Using Local System**](/support/docs/webdriverio-on-hyperexecute-grid/#1-testing-using-local-system) - You can use your own local machine to execute tests. -- [**Using Gitpod Platform**](/support/docs/webdriverio-on-hyperexecute-grid/#2-testing-using-gitpod) - Execute tests using GitPod. (Requires a [Gitpod](https://gitpod.io/login/) account) - -## 1. Testing Using Local System - -Follow the step-by-step guide to execute your test on HyperExecute. - -### Prerequisites - -To run the Tests on HyperExecute from your Local System, you are required: - -- Your TestMu AI [Username and Access key](/support/docs/hyperexecute-how-to-get-my-username-and-access-key/) -- [HyperExecute YAML](/support/docs/hyperexecute-yaml-version0.2/) file which contains all the necessary instructions. -- [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) in order to initiate a test execution Job . -- Setup the [Environmental Variable](/support/docs/hyperexecute-environment-variable-setup/) - -### Step 1: Configure Your Test Suite - -You can use your own project to configure and test it. For demo purposes, we are using the sample repository. - -**Sample repo** - -Download or Clone the code sample for the JUnit from the TestMu AI GitHub repository to run the tests on the HyperExecute. - - View on GitHub - -If you are using your own project, make sure you update the **Hub endpoint** in your tests file. - -By setting up the Hub endpoint, you establish the communication channel between your tests and the browser nodes, enabling effective test distribution and execution. - -Configure the desired capabilities based on your test requirements. For example: - -```yaml -capabilities: [{ -browserName: 'chrome', -platform: 'Windows 10', -version: 'latest', -name: "single-test", -build: "wd-lambdatest-sample", -geoLocation : "US" -}] -``` - -> You can also use **dev** and **beta** browser versions. To generate capabilities for your test requirements, you can use our inbuilt 🔗 Capabilities Generator. - -### Step 2: Setup the CLI in your Test Suite - -After cloning / downloading the sample repo, you need to setup the CLI and the environment variables. - -#### Download the HyperExecute CLI - -The CLI is used for triggering the tests on HyperExecute. It is recommend to download the CLI binary on the host system and keep it in the root directory of the suite to perform the tests on HyperExecute. - -You can download the CLI for your desired platform from the below mentioned links: - -| Platform | HyperExecute CLI | -| ---------| ---------------- | -| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe | -| MacOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute | -| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute | - -#### Setup Environment Variable - -Now, you need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in the [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). - -Run the below mentioned commands in your terminal to setup the CLI and the environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -### Step 3: Configure YAML in your Test Suite - -Configure your YAML file as per your use cases using **key value** pairs. - -In this sample YAML file, we have mentioned: - -- **version** of the YAML file -- **Timeouts** for executing your project -- **Mode of execution** is [Autosplit](/support/docs/hyperexecute-auto-split-strategy/). You can also opt for [Matrix](/support/docs/hyperexecute-matrix-multiplexing-strategy/) or [Hybrid](/support/docs/hyperexecute-hybrid-strategy/) mode. -- **Pre and Post** commands -- **Reports and Artefacts** that will be generated after the completion of tests -- and other necessary YAML Parameters - -```yaml ---- -version: 0.1 -globalTimeout: 150 -testSuiteTimeout: 150 -testSuiteStep: 150 - -runson: linux - -autosplit: true -retryOnFailure: true - -maxRetries: 1 -concurrency: 2 - -pre: -- npm install - -cacheKey: '{{ checksum "package-lock.json" }}' -cacheDirectories: -- node_modules - -mergeArtifacts: true - -uploadArtefacts: -[{ -"name": "Reports", -"path": ["Reports/*"] -}] - -report: true -partialReports: -type: json -location: Reports/ -frameworkName: extent - -testDiscovery: -type: automatic -args: -featureFilePaths: Features/ -frameWork: java -specificTags: ["@ToDoOne", "@ToDoTwo", "@ToDoThree"] - -testRunnerCommand: npx wdio wdio.conf.js --spec=$test - -jobLabel: [selenium-webdriverIO, linux, autosplit] -``` - -### Step 4: Execute your Test Suite - -> **NOTE :** In case of MacOS, if you get a permission denied warning while executing CLI, simply run **`chmod u+x ./hyperexecute`** to allow permission. In case you get a security popup, allow it from your **System Preferences** → **Security & Privacy** → **General tab**. - -Run the below command in your terminal at the root folder of the project: - -```bash -./hyperexecute --config RELATIVE_PATH_OF_YOUR_YAML_FILE -``` - -OR use this command if you have not exported your username and access key in the step 2. - - {`./hyperexecute --user ${ YOUR_LAMBDATEST_USERNAME()} --key ${ YOUR_LAMBDATEST_ACCESS_KEY()} --config RELATIVE_PATH_OF_YOUR_YAML_FILE `} - -### Step 5: Monitor the Test Execution - -Visit the [HyperExecute Dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) and check your Job status. - -### Step 6: Download Artifacts and Reports - -HyperExecute also facilitates the provision to download the [Artifacts](/support/docs/hyperexecute-artifacts/) and [Reports](/support/docs/hyperexecute-reports/) on your local machine. Click on the corresponding button to download your generated artifacts and reports. - -## 2. Testing Using Gitpod - -You can also use the Gitpod platform to execute our sample repository. It will fetch all the sample codebases and trigger the CLI to execute the tests. - -Follow the below steps to run Test using Gitpod: - -**Step 1:** Click '**Open in Gitpod**' button. You will be redirected to Login/Signup page. This button is configured to redirect you to the Gitpod platform where you will be able to execute our sample repository. - -**Step 2:** Login with TestMu AI credentials. Once logged in, a pop-up confirmation will appear, asking you to **'Proceed'** to the Gitpod editor in a new tab. The current tab will display the HyperExecute Dashboard. - -**Step 3:** Choose your preferred editor (we recommend VS Code Editor) - -**Step 4:** As you are running a sample project, Fetching of the Test Scripts, [HyperExecute YAML](/support/docs/deep-dive-into-hyperexecute-yaml/), [HyperExecute CLI](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) and Triggering your tests using the `Execution Command` will be automated. - -**Step 5:** Once you see the `Job Link` in the logs, you can visit the [HyperExecute dashboard](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/hyperexecute) to see the tests getting executed. - -You can also implement [Secret Keys](/support/docs/hyperexecute-how-to-save-and-manage-secrets/) in your YAML file. - -## Navigation in Automation Dashboard - -Every test run on the HyperExecute has a unique *jobId* associated with it. Each *jobId* can in turn constitute single (or multiple) *groupId*(s). You can visit [HyperExecute Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build) for checking the status of the test execution. - -You can seamlessly navigate between JobId's and taskId's. You need to click on the *testID* to navigate from the HyperExecute logs to the Automation Dashboard. - -The snapshot below shows the videos, logs and other meta data for that specific *test_ID* - -> -For any query or doubt, please feel free to contact us via **24×7 chat support** or you can also drop a mail to **support@testmuai.com**. -Happy testing! - -## Using the WebdriverIO Agent Skill with TestMu AI - -The [webdriverio-skill](https://github.com/LambdaTest/agent-skills/tree/main/webdriverio-skill) is a part of [TestMu AI Skills](https://github.com/LambdaTest/agent-skills/) that guide AI coding assistants in generating production-ready test automation. - -The webdriverio-skill package includes: - -``` -webdriverio-skill/ -├── SKILL.md -└── reference/ -├── playbook.md -└── advanced-patterns.md -``` - -It provides structured guidance for: - -* Project structure and setup -* Dependency configuration -* Local execution -* TestMu AI cloud execution -* Debugging patterns -* CI/CD integration - -### Installing WebdriverIO Agent Skill - -Install a WebdriverIO Agent Skill using the command below: - -``` -# Clone the repo and copy the skill you need -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/webdriverio-skill .claude/skills/ - -# Or for Cursor / Copilot -cp -r agent-skills/webdriverio-skill .cursor/skills/ -``` - -**Note**: If you prefer installing all available framework skills instead of only webdriverio-skill, clone the repository directly into your tool's skills directory (for example, .claude/skills/, .cursor/skills/, .gemini/skills/, or .agent/skills/). diff --git a/static/docs/webdriverio-with-selenium-running-webdriverio-automation-scripts-on-testmu-selenium-grid.md b/static/docs/webdriverio-with-selenium-running-webdriverio-automation-scripts-on-testmu-selenium-grid.md deleted file mode 100644 index 463147bee..000000000 --- a/static/docs/webdriverio-with-selenium-running-webdriverio-automation-scripts-on-testmu-selenium-grid.md +++ /dev/null @@ -1,142 +0,0 @@ -# Selenium With WebDriverIO - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Run WebDriverIO tests on the TestMu AI cloud grid. This guide covers setup, running a sample test, configuring capabilities, and testing locally hosted pages. - -**Sample repo** -All the code used in this guide is available in the sample repository. - - View on GitHub - -## Prerequisites - -Complete these steps before running WebDriverIO Selenium tests on TestMu AI. - -1. Create a [TestMu AI account](https://www.testmuai.com/register/?redirectTo=https://accounts.lambdatest.com/dashboard) and get your username and access key from the dashboard. -2. Install **NodeJS** v6 or newer from [nodejs.org](https://nodejs.org/en/). -3. Install **npm** from the [official npm website](https://www.npmjs.com/). -4. Download [Selenium JavaScript bindings](https://www.selenium.dev/downloads/) from the official website. - -## Step 1: Clone the Sample Project - -Clone the TestMu AI WebDriverIO Selenium sample repository to your local machine. - -```bash -git clone https://github.com/LambdaTest/webdriverio-selenium.git -cd webdriverio-selenium -``` - -Install the required dependencies: -```bash -npm i webdriverio -npm i selenium-webdriver -npm i -``` - -## Step 2: Set Your Credentials - -Set your TestMu AI username and access key as environment variables. - - {`export LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -export LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -## Step 3: Configure Your Test Capabilities - -Update the capabilities in `conf/single.conf.js` to define the browser and platform settings. - -```js -//single.conf.js -user= process.env.LT_USERNAME || "", -key= process.env.LT_ACCESS_KEY || "", - -exports.config = { - -updateJob: false, -user, -key, -specs: [ -'./tests/specs/single_test.js' -], -exclude: [], - -capabilities: [{ -browserName: 'chrome', -browserVersion: 'latest', -platformName: 'Windows 10', -name:"Test webdriverio", -build:"build 1", -}], -sync: true, -logLevel: 'info', -coloredLogs: true, -screenshotPath: './errorShots/', -baseUrl: '', -waitforTimeout: 100000, -connectionRetryTimeout: 90000, -connectionRetryCount: 1, -path: '/wd/hub', -hostname: 'hub.lambdatest.com', -port: 80, - -beforeSession: function (config, capabilities, specs) { -capabilities.name=specs[0].split(/(\\|\/)/g).pop() || undefined; -}, -after: function (result, capabilities, specs) { -driver.execute("lambda-status=".concat(result==0?"passed":"failed"),undefined); -}, - -framework: 'mocha', -mochaOpts: { -ui: 'bdd' -} -} -``` - -Generate capabilities for your test requirements with the [Capabilities Generator](https://www.testmuai.com/capabilities-generator/). - -## Step 4: Run the Test - -Execute the test using the following command. - -**Single test:** -```bash -npm run single -``` - -**Parallel tests:** -```bash -npm run parallel -``` - -## Step 5: View Your Results - -After running the test, view your results on the [TestMu AI Automation Dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/build). - -The dashboard provides: -- Video recordings of each test session -- Screenshots captured at each step -- Console logs from the browser -- Network logs for debugging -- Detailed command logs - -## Run WebDriverIO Tests Using Agent Skills - -Use AI coding assistants to generate and run WebDriverIO tests with the TestMu AI Agent Skill. - -The [webdriverio-skill](https://github.com/LambdaTest/agent-skills/tree/main/webdriverio-skill) is part of [TestMu AI Agent Skills](https://github.com/LambdaTest/agent-skills/) - structured packages that teach AI coding assistants how to write production-grade test automation. - -Install the skill: - -```bash -git clone https://github.com/LambdaTest/agent-skills.git -cp -r agent-skills/webdriverio-skill .claude/skills/ - -# For Cursor / Copilot -cp -r agent-skills/webdriverio-skill .cursor/skills/ -``` - -Install all available framework skills at once by cloning the repository directly into your tool's skills directory (e.g., `.claude/skills/`, `.cursor/skills/`). diff --git a/static/docs/webex-teams-integration.md b/static/docs/webex-teams-integration.md deleted file mode 100644 index c2e309ebf..000000000 --- a/static/docs/webex-teams-integration.md +++ /dev/null @@ -1,49 +0,0 @@ -# Webex Teams Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Cisco Webex Teams is a collaboration platform designed to help you and your team get more done faster. It combines messaging, file sharing, video conferencing, whiteboarding, and calling into one simple-to-use solution that keeps everyone connected and working smarter. - -TestMu AI and Webex Teams can be integrated to create detailed test cards in Webex Team Spaces for logged bugs in TestMu AI. This will notify you and your team members every time a new bug is pushed while performing [cross browser testing](https://www.testmuai.com) of your website from the TestMu AI platform to your Webex Teams instance. - -## Integrating Webex Teams With TestMu AI - -**Step 1:** Sign in to your TestMu AI account. Don't have an account, register for free. - -**Step 2:** Select **Integrations** from the left sidebar and then Webex. - -**Step 3:** Click on the **Install** button and you will be redirected to the login page for **Webex Teams**. - -**Step 4:** Enter your **Webex Teams** email address and click on **Sign In**. - -**Step 5:** Click on **Accept** to allow TestMu AI to gain access to your **Webex Teams account** to complete the integration. - -**Step 6:** After accepting, you will be redirected back to the TestMu AI dashboard. Now select your preferred **Space** from the dropdown. In this case, we will select the **Welcome Space** and click on **Confirm Space**. - -You have now successfully integrated TestMu AI with Webex Teams. - -## One-Click Bug Logging With Webex Teams - -**Step 1:** In your TestMu AI Dashboard, head to **Real-Time Testing > Browser Testing** from the left navigation menu. - -**Step 2:** To perform **live-interactive testing** of your website or a web app, enter the web URL, select the testing type (Desktop/Mobile), select the browser version, OS, and resolution, and click **Start**. - -Upon clicking, a cloud-based virtual machine will fire up, running real browsers and operating systems where you can test websites and web apps for filing bugs. - -**Step 3:** Click on the **Bug icon** from the left in-session toolbar to take a screenshot of any bug you encounter. - -**Step 4:** After marking the bugs, click on the **Bug icon** in front of TestMu AI under the integrations section. - -**Step 5:** In the Webex ticket, fill the bug details like **Assignee, Issue Type, Priority, Summary,** and **Description** and check the **Send to Webex** option. Then click **Mark As Bug** to log the bugs in Webex Teams. - -**Step 6:** To view your logged bugs, head to **Issue Tracker** under the **Manage** section on the left navigation menu. You'll notice the marked bugs along with the Test details. - -**Step 7:** You can find the test log details directly in your selected **Webex Teams Space** as shown below and share it with your team members. - -## Uninstalling The Webex Teams Integration - -**Step 1:** Sign in to your TestMu AI account and visit **Integrations** from the left sidebar menu. - -**Step 2:** From the **My Integrations** section, click the **remove option** under the Webex Teams block. - -> That was all you need to know for TestMu AI + Webex Teams Integration. Increase your productivity with our integrations. If you still have any questions for us, please feel free to let us know. Our experts are always available on **chat** to help you out with any roadblock regarding our product. Happy testing! diff --git a/static/docs/webhooks-integration.md b/static/docs/webhooks-integration.md deleted file mode 100644 index 33458ecb0..000000000 --- a/static/docs/webhooks-integration.md +++ /dev/null @@ -1,71 +0,0 @@ -# Webhooks Integration With TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Webhooks provide a way to receive real-time notifications about events in your TestMu AI account. Instead of polling for updates, webhooks push data to your specified endpoints whenever a build completes, enabling seamless integration with your existing workflows and tools. - -With the TestMu AI Webhooks Integration, you can: -- Receive instant build data notifications on your configured endpoints. -- Automate downstream processes based on test results. -- Integrate with custom dashboards, CI/CD pipelines, or notification systems. -- Keep QAs and developers in sync with real-time updates. - -## Prerequisites - -- You should have Admin or User level access to see and install integrations. -- You will require Webhook access for successful integration. -- A valid webhook URL endpoint that can receive POST requests. - -## Integrate Webhooks with your TestMu AI Account - -**Step 1:** Login to your TestMu AI account. You should have Admin or User level access to see and install integrations. - -**Step 2:** Click on **Settings** -> **Integrations** -> **Communication**. - -**Step 3:** Click on the **Connect** button of the `Webhooks` block. - -**Step 4:** In the **Manage URLs** section, enter your webhook URL in the input field provided. - -**Step 5:** Click the **Add URL** button to save your webhook endpoint. - -After successful addition, you will start receiving build data on every build completion. - -## Configure Notification Preferences - -Once the webhook is connected, you can customize which notifications you want to receive. Click on the **Webhooks** block to open the configuration modal. - -In the **Notification Preferences** section, you can enable or disable the following notification types: - -| Notification Type | Description | -|-------------------|-------------| -| **Build Completion Messages** | Receive notifications when a build completes | -| **Test Fail Messages** | Receive notifications when tests fail | -| **Test Pass Messages** | Receive notifications when tests pass | -| **Agent Testing Notifications** | Receive notifications for agent-based testing events | -| **HyperExecute Job Completion Messages** | Receive notifications when HyperExecute jobs complete | - -You can also configure the **Notification Time** to set a delay before receiving notifications after build completion. - -## Webhook Payload - -When a build completes, TestMu AI sends a POST request to your configured webhook URL with the build data. The payload includes information about: - -- Build status and results -- Test execution details -- Timestamps and duration -- Associated metadata - -## Use Cases - -1. **CI/CD Integration:** Trigger downstream jobs in your pipeline based on test results. -2. **Custom Dashboards:** Push test data to your internal monitoring and reporting tools. -3. **Team Notifications:** Forward build results to custom notification systems. -4. **Automated Reporting:** Generate and distribute test reports automatically upon build completion. - -## Remove Webhooks Integration - -**Step 1:** Login to your TestMu AI account. You should have Admin or User level access to see and install integrations. - -**Step 2:** Click on **Settings** -> **Integrations** -> **Communication**. - -**Step 3:** Click on the **Remove** button next to the webhook URL you want to delete. diff --git a/static/docs/webview-testing-in-android.md b/static/docs/webview-testing-in-android.md deleted file mode 100644 index c6800802d..000000000 --- a/static/docs/webview-testing-in-android.md +++ /dev/null @@ -1,65 +0,0 @@ -# WebView Testing In Android Online In Real Time - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -WebView, in Android, is the feature which allows any app to display a webpage as a part of its own activity, instead of opening it on a separate browser. This not only allows the app to retain the users within itself but also increases the user-experience multifold. As a result, WebView Testing is one of the most widely in-demand testing techniques around the world as of today. - -TestMu AI now allows you to perform your WebView Testing using its online cloud-based Selenium Grid, which gives you 3000+ combinations of browsers, operating systems, and mobile devices. You will be able to test WebViews of your app, interact with it in realtime, mark bugs, and much more, with just a click, on the TestMu AI platform. In this article, we will see how to perform WebView Testing in Android using Selenium Grid on TestMu AI. - ->Note: This is not our primary offering, as of now. Hence you need to launch WebView manually, using the steps mentioned in this doc. - -## How To Perform WebView Testing On TestMu AI - -**Step 1:** You will need an account on TestMu AI. If you already have an account on TestMu AI, you can directly skip to the next step. If you don't have an account yet, you can register for free. - -**Step 2:** Login to your TestMu AI account using this link. - -**Step 3:** Navigate to "Real Time Testing" from the left navigation menu. - -**Step 4:** Inorder to launch an Android device, select the _Mobile icon_ from the Device options, followed by _Android icon_ from the OS options. Then you can select the _Brand_, _Screen size along with Android version_ and _Browser_ as per your choice. For this demo purpose, we have selected **Samsung + Galaxy Note 10 with Android 9.0 + Chrome browser**, as shown in the image below: - ->Please note that we allow WebView Testing as of now in devices with Android version 9.0 and below. - -**Step 5:** Once you have selected the desired options for the virtual machine, click on **Start** to launch it. - -**Step 6:** Wait for the VM to be launched. It might take few seconds. - -**Step 7:** Once the VM has launched successfully, an Android device, as per the chosen configuration will open on your screen. - -**Step 8:** Then press the _home button_ at the bottom center of the device to go the device's home screen. - -**Step 9:** In order to perform the WebView Testing, on the home screen, press the _Android icon_ to open the device's WebView Browser Tester. - -**Step 10:** Once the **WebView Browser Tester** is open, you can perform your WebView Testing as per your requirement. - -## How To Mark Bug While Performing WebView Testing On TestMu AI - -**Step 1:** Open the WebView Browser Tester as per the steps given above. - -**Step 2:** Enter the desired URL upon which you want to perform the WebView Testing. - -**Step 3:** Once the webpage loads and you find a bug, select the **Mark as bug** option from the left toolbar. - -**Step 4:** A window will appear on your screen with several options to highlight the bug and mark it. - -**Step 5:** Scroll the are of the screen where you have found a bug. Highlight it using designs, texts, colors, etc from the options available on the left toolbar. Once you have completed the highlight, you can click on the **Mark as bug** button on the bottom right of the window. - -**Step 6:** A dialog box will appear on the screen asking you to fill required details about the bug, such as: - -* **Assignee:** Assign the person to whom you want to transfer the responsibility of the bug. -* **Issue Type:** Select the type of the issue, whether it is a bug or a task. -* **Priority:** Set the priority of the bug, from Lowest to Highest, as per your convenience. -* **Summary:** Set the summary of the bug as per your choice. It will help the assignee to know about the bug in one read. -* **Description:** This is an optional section, in which you can describe the bug in detail and include other relevant information, as per your choice. - -For this demo purpose, we have set the required details as shown in the image below. Once you have filled the details, press **Create Issue** at the bottom to create the issue. - -These steps will mark the bug successfully. You can a message appearing on your top right screen showing you that the bug has been marked successfully. - -To know more about the bug marking in TestMu AI, you can refer to [this article](/docs/mark-as-bug/). - -You can even integrate 3rd party apps in your account from our vast list of integrations to send the bug directly on your desired workspace. You can also track your issues using our [Issue Tracker](https://issues.lambdatest.com/) directly, and much more. - ->That's all to get started for performing WebView Testing in Android on TestMu AI. We are constantly adding more features and integrations to increase your productivity through our testing environment. If you still have any doubts or questions, please don't hesitate to contact us. Our experts are available on **24/7 chat support**. You can also drop us a mail at **[support@testmuai.com](mailto:support@testmuai.com)**. -> ->Happy testing! diff --git a/static/docs/what-is-browser-cloud.md b/static/docs/what-is-browser-cloud.md deleted file mode 100644 index eaaed725c..000000000 --- a/static/docs/what-is-browser-cloud.md +++ /dev/null @@ -1,68 +0,0 @@ -# Getting Started With TestMu AI Browser Cloud - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI is a scalable browser designed for AI agents to handle scraping, testing, and automation, without the overhead of managing infrastructure. - -Find all the guides, resources, and examples you need to build on TestMu AI Browser Cloud. - -## Get Started - - 🛠️ Set Up the SDK - Install the TestMu AI Browser SDK and configure your credentials. - - 🔌 Connect to a Session - Drive sessions with Puppeteer, Playwright, or Selenium. - - 🔍 Replay and Debug Sessions - Video replays, console logs, and full network capture for every session. - -## Features - - 🛡️ Stealth Mode - Built-in fingerprint masking, CAPTCHA solving, and ad blocking. - - 🍪 Session Persistence - Transfer cookies, local storage, and login state across sessions. - - 📁 Files API - Upload and download files between your machine and the cloud browser. - - 🧩 Chrome Extensions - Load Chrome extensions into your cloud browser sessions. - - 🔒 Localhost Tunnel - Access localhost and internal networks from cloud browsers. - -## Cookbook - -Ready-to-run examples to get you started fast. - - OpenAI Computer Use - Build a browser automation agent using Browser Cloud and OpenAI's Computer Use. - - Claude Computer Use - Build a browser automation agent using Browser Cloud and Claude's Computer Use. - - Gemini - Learn how Browser Cloud and Gemini power a fully autonomous browser agent. - - Playwright Starter - Build a starter project with Browser Cloud and Playwright. - - Puppeteer Starter - Build a starter project with Puppeteer and Browser Cloud. - - Quick Actions - Trigger browser tasks instantly. Scrape, screenshot, and PDF in one line. - -## Resources - - 📖 API Reference - Full reference for the Browser Cloud API. - - 🤖 AI Agent Skills - Pre-built skills to accelerate your agent development. - - ⚙️ Session Configuration - All session options and configuration parameters. diff --git a/static/docs/white-listing-proxy-with-testmu.md b/static/docs/white-listing-proxy-with-testmu.md deleted file mode 100644 index 3afbc9af2..000000000 --- a/static/docs/white-listing-proxy-with-testmu.md +++ /dev/null @@ -1,15 +0,0 @@ -# Whitelisting Proxy with TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -If you access the internet through a firewall-protected network, you may face issues connecting with TestMu AI. You may encounter error messages. - -In such scenarios, you need to whitelist the TestMu AI domain. Whitelisting tells your firewall-protected network to allow access for a set of IP addresses, IP range, or an entire domain that you trust. Whitelisting is a security procedure to help users access a website or web-app that their network security team considers non-malicious. - -Relay the below domain to your network security team and ask them to whitelist it so you can connect with TestMu AI from your firewall-protected network. - -| PROTOCOL | PORTS | SOURCE | DESTINATION | -|----------|-------------|------------------------|------------------| -| TCP | 80, 443, 22 | All TestMu AI Clients | *.lambdatest.com | - ->In case of any queries or special requests related to whitelisting, reach out via **chat support**. diff --git a/static/docs/why-we-need-kane-ai.md b/static/docs/why-we-need-kane-ai.md deleted file mode 100644 index 286fe79aa..000000000 --- a/static/docs/why-we-need-kane-ai.md +++ /dev/null @@ -1,35 +0,0 @@ -# Why do we need KaneAI? - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -KaneAI transforms the way teams approach test automation by bridging the gap between natural language requirements and executable test code. Built for modern development workflows, KaneAI empowers teams to plan, author, and evolve comprehensive test suites without the traditional barriers of complex scripting or framework expertise. - -## Current Testing Scenario Challenge -Traditional test automation often creates bottlenecks in development cycles. Teams face several persistent challenges: - -- **Technical Barriers:** Writing and maintaining test scripts requires specialized programming knowledge, limiting who can contribute to test creation. -- **Framework Complexity:** Managing multiple testing frameworks, languages, and integrations creates overhead and maintenance burden. -- **Scalability Issues:** As applications grow, test suites become increasingly difficult to organize, update, and execute consistently. -- **Security Concerns:** Handling authentication credentials and sensitive test data safely across different environments -Integration Friction: Connecting testing workflows with CI/CD pipelines often requires significant configuration and ongoing maintenance. - -## The KaneAI Solution -KaneAI addresses these challenges through an intelligent, unified platform that democratizes end-to-end test automation while maintaining enterprise-grade capabilities. - -- **Intelligent Input Processing:** KaneAI offers dual flexibility in test authoring by intelligently generating structured test cases by transforming various input formats such as text, JIRA tickets, PDFs, images, audio, videos, and spreadsheets, or manually author tests using conversational language for precise control. - -Whether you need rapid test generation from existing documentation or prefer hands-on test crafting, KaneAI's advanced processing capabilities streamline both approaches. - -- **Unified Test Management:** Consolidate your entire testing ecosystem within a single platform. KaneAI provides centralized control over test cases, execution plans, and results analysis, eliminating the fragmentation common in traditional testing tools. Teams benefit from: - - - Complete visibility into test coverage and execution status - - Streamlined collaboration between development, QA, and business teams - - Simplified reporting and metrics tracking across all test activities - -- **Multi-Language & Framework Support:** KaneAI supports all prominent frameworks and languages, offering the flexibility to export test code in the language and framework that you and your team prefers. - -- **Comprehensive API Testing:** Create robust API test suites with intelligent validation and seamless integration capabilities. KaneAI understands API contracts and automatically generates meaningful assertions, reducing the manual effort typically required for thorough API testing coverage. - -- **Secret Management:** Connecting testing workflows with CI/CD pipelines often requires significant configuration and ongoing maintenance. - -- **CI/CD Integration:** Connect seamlessly with your existing continuous integration and deployment pipelines. KaneAI's integration capabilities ensure automated test execution becomes a natural part of your development workflow, supporting consistent quality gates across multiple environments and deployment stages. diff --git a/static/docs/wordpress-plugin.md b/static/docs/wordpress-plugin.md deleted file mode 100644 index 8bc28a6dd..000000000 --- a/static/docs/wordpress-plugin.md +++ /dev/null @@ -1,31 +0,0 @@ -# WordPress Plugin - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - ->With TestMu AI plugin for WordPress, you can generate automated screenshots of your web pages and post directly from your WordPress platform. - -## How To Install & Use TestMu AI WordPress Plugin? - -**Step 1:** Login to your WordPress account. - -**Step 2:** Goto plugins and click on **Add New** plugin button. - -**Step 3:** Search for **TestMu AI** keyword in search bar and click on **install** button. - -**Step 4:** Click on **Activate** button and you are all set with TestMu AI plugin. - -**Step 5:** Now you can see **TestMu AI Screenshot** in plugin option on your WordPress platform. - -**Step 6:** Now go to TestMu AI plugin and add TestMu AI profile account details. - -**Step 7:** Go to profile section on your TestMu AI account and copy access key. - -**Step 8:** Add Email address and paste Access key in your WordPress platform. Click on Activate button. - -**Step 9:** You are all set with WordPress plugin. - -**Step 10:** Now you can Generate [Automated Screenshots](https://www.testmuai.com/automated-screenshot) of your web pages or posts with TestMu AI plugin. Go to All Posts and you will TestMu AI icon below every post. Click on that icon. - -**Step 11:** Now select the browser, browser version and mobile devices where you want to take screenshots and hit **Start** button. - -**Step 12:** Here all the screenshots are generated. diff --git a/static/docs/xcode-plugin-testmu.md b/static/docs/xcode-plugin-testmu.md deleted file mode 100644 index 470903091..000000000 --- a/static/docs/xcode-plugin-testmu.md +++ /dev/null @@ -1,63 +0,0 @@ -# LambdaMax XCode Plugin - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -TestMu AI have developed a new plugin to help developers execute tests on the TestMu AI Grid directly from the XCode (via LambdaMax). The all-new TestMu AI plugin will allow developers and testers to directly run automated tests based on XCUIT (XCode) framework on the TestMu AI real device cloud. - -XCUITest is a testing framework that can be used to automate UI tests for native iOS and macOS applications. The framework uses XCTest, an integrated testing framework developed by Apple. You can write automated UI tests using the XCUITest framework in Swift or Objective-C programming language. - -With this new plugin, testers and developers never have to leave XCode to execute a test on TestMu AI’s real device cloud. Follow these steps to install LambdaMax plugin: - -## How to Install LambdaMax Plugin - -1. On **Xcode** on your machine. - -2. Head to **Product** on the menu bar in the top-left corner. - -3. Hover on the **Scheme** option from the dropdown and click on the **Edit Scheme** from the dropdown list. - -4. In the pop-up window, click and expland the **Build** option from the sidebar. - -5. Head to the **Post Actions** option from the sidebar and copy the sample code. - -6. Click on **+** icon to add **"New Script Run Action"**. You can change the path and capabilities as per your preference, which will reflect directly on the TestMu AI Grid. - -Here is the sample code for the Xcode plugin: - -```bash -Xcode plugin: -Custom Xcode Script and Images attached below: - -#!/bin/sh - -# ci_post_xcodebuild.sh -# Debug/Release build name if required -# -# Created by Ankit Agarwal on 08/11/22. -# - -AppUploadResponse=$(curl -w --location --request POST 'https://{username}:{accessKey}@manual-api.lambdatest.com/app/upload/realDevice' --form 'name="{name}"' --form 'appFile=@"{filePathWithNameAndExtension}"' --form 'custom_id="{custom_id}"') - -appId=$(awk '{sub(/.*\"app_url\":\"/,"");sub(/\",\"url.*/,"");print}' <<< "$AppUploadResponse") - -echo "$appId" - -TestAppUploadResponse=$(curl -w --location --request POST 'https://{username}:{accessKey}@manual-api.lambdatest.com/app/upload/realDevice' --form 'name="{name}"' --form 'appFile=@"{filePathWithNameAndExtension}"' --form 'custom_id="{custom_id}"') - -testAppID=$(awk '{sub(/.*\"app_url\":\"/,"");sub(/\",\"url.*/,"");print}' <<< "$TestAppUploadResponse") - -echo "$testAppID" - -TestExeuctionPayload="{\"app\": \"$appId\", \"testSuite\": \"$testAppID\", \"device\": [ \"iPhone X-14\" ], \"tunnel\": false, \"network\": true, \"devicelog\": true, \"queueTimeout\": 3000, \"idleTimeout\": 200, \"build\": \"otrium1\", \"project\": \"testerror\" }" - -echo "$TestExeuctionPayload" - -XCUITTestExecution=$(curl -w --location --request POST 'https://{username}:{accessKey}@mobile-api.lambdatest.com/framework/v1/xcui/build' --header 'Content-Type: application/json' --data-raw "$TestExeuctionPayload") - -echo "$XCUITTestExecution" -``` ->**Note:** By clicking on the **Play** button in the top left corner, the script will automatically run on the TestMu AI Grid. - -7. In the above code, update the required TestMu AI credentials - `{Username:AccessKey}`. You can access your credentials under the [profile section](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). - -8. Along with this, you need to update the `{FilePathWithNameAndExtension}` with the path where your app and test suite is in local. diff --git a/static/docs/xcresult.md b/static/docs/xcresult.md deleted file mode 100644 index f26615ff5..000000000 --- a/static/docs/xcresult.md +++ /dev/null @@ -1,110 +0,0 @@ -# XCResult on TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Apple’s **Native XCResult Bundles (`.xcresult`)** are comprehensive test reports generated when you run XCUITest cases. These bundles include **test hierarchy, logs, stack traces, screenshots, and performance data**, which can be directly viewed in Xcode. They provide developers with rich debugging information, making it easier to analyze why a test passed or failed. - -On TestMu AI, you can now **generate and download `.xcresult` bundles** for your XCUI test sessions. You can access them via the **REST API**. - -## Prerequisites - -- Your TestMu AI [Username and Access Key](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/security). -- Access to an **iOS app (.ipa)** and an **XCUI Test app (.ipa)**. -- Xcode installed locally to view `.xcresult` bundles. - -## Flow for Adding XCUI Result Bundles - -### Step 1: Upload Your Application and Test Suite - -To begin testing, you need to upload both your iOS application (.ipa) file and your XCUI test suite (.ipa) file to TestMu AI. These files are required before executing tests. - -Detailed upload steps are available here: [Getting Started with XCUI Testing – Running Your First Test](/support/docs/getting-started-with-xcuitest/#running-your-first-test-a-step-by-step-guide) - -### Step 2: Execute Your Tests with Result Bundles - -To generate `.xcresult` bundles for your XCUI test executions, you must pass `"enableResultBundle": true` in your build request and use the new build endpoint: - -``` -POST https://mobile-api.lambdatest.com/mobile-automation/api/v1/xcuitest/builds -``` - -This endpoint initiates your test run and enables generation of the result bundle. - -| Parameter | Description | Values | -|--------------------|-------------------------------------------------------|--------------------------------| -| enableResultBundle | Enable generating result bundles for your XCUI build. | true/false (default: false) | - -Below is an example cURL command to execute your test with result bundles enabled: - -```bash -curl --location --request POST 'https://mobile-api.lambdatest.com/framework/v1/xcui/build' \ ---header 'Authorization: Basic BASIC_AUTH_TOKEN' \ ---header 'Content-Type: application/json' \ ---data-raw '{ -"app" : "APP_ID", -"testSuite": "TEST_SUITE_ID", -"device" : ["iPhone 11-14"], -"video" : true, -"queueTimeout": 10800, -"idleTimeout": 150, -"devicelog": true, -"network": false, -"build" : "Proverbial-XCUITest", -"enableResultBundle": true -}' -``` - -```bash -curl --location --request POST "https://mobile-api.lambdatest.com/framework/v1/xcui/build" \ ---header "Authorization: Basic BASIC_AUTH_TOKEN" \ ---header "Content-Type: application/json" \ ---data-raw "{ -"app" : "APP_ID", -"testSuite": "TEST_SUITE_ID", -"device" : ["iPhone 11-14"], -"video" : true, -"queueTimeout": 10800, -"idleTimeout": 150, -"devicelog": true, -"network": false, -"build" : "Proverbial-XCUITest", -"enableResultBundle": true -}" -``` - -### Step 3: Retrieve the Result Bundle - -Result bundles are generated at the Build level. To download the `.xcresult` bundle for a specific session, use the following GET endpoint: - -- In case of sharding, each shard execution is treated as a separate shards and generates its own `.xcresult` bundle. You will need to retrieve each shard's bundle individually. For more information, see [Sharding in HyperExecute](/support/docs/sharding-rd-hyperexec/). -- To view the `.xcresult` for a specific shard, you must pass the `shard:shardId` as a query parameter in your request. - -``` -GET https://mobile-api.lambdatest.com/mobile-automation/api/v1/framework/builds/{build-id}/xcresult -``` - -Replace `{build-id}` with the actual build ID. - -Example cURL command to download the result bundle: - -{`curl --location --request GET \\ -'https://mobile-api.lambdatest.com/mobile-automation/api/v1/framework/builds/{build-id}/xcresult' \\ ---header 'Authorization: Basic BASIC_AUTH_TOKEN' \\ ---output xcui-result-bundle.zip`} - -You will need your **BASIC_AUTH_TOKEN** (Base64 encoded `username:accesskey`) in the request header. -If you’re unsure how to generate it, follow the instructions here: [Executing the Test](/support/docs/getting-started-with-xcuitest/#step-3-executing-the-test). - -The response is a binary ZIP file containing the `.xcresult` bundle, which you can unzip and open directly in Xcode for detailed analysis. - -### Step 4: Report Structure - -The `.xcresult` bundle contains a comprehensive report of your XCUI test execution, including: - -- **Summary View**: Shows total tests executed, number passed, and number failed with a visual chart. -- **Errors Section**: Lists any critical errors or crashes encountered (e.g., test runner crashes with signal codes). -- **Tests Section**: Provides execution duration, device and OS version details, and per-device results. -- **Device & Configuration Matrix**: Displays which tests passed/failed on specific device configurations. -- **Individual Test Details**: Each test case shows its status (pass/fail), failure reason, logs, and any assertion errors. - -You can open the `.xcresult` bundle directly in Xcode to explore these details visually, enabling efficient debugging and analysis of your test runs. diff --git a/static/docs/xctestplan.md b/static/docs/xctestplan.md deleted file mode 100644 index 9dd82229d..000000000 --- a/static/docs/xctestplan.md +++ /dev/null @@ -1,139 +0,0 @@ -# XCTestPlan - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -XCTestPlan is a feature within Xcode used for building iOS, iPadOS, macOS, watchOS, tvOS, and audioOS applications. XCTestPlan helps manage how you run automated tests written with the XCTest framework. - -XCTestPlan improves the organization and efficiency of running automated tests within Xcode. It allows you to create targeted test runs and manage various testing scenarios effectively. - -## Prerequisites - -- Your TestMu AI [Username and Access key](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/security). -- Access to an **iOS** app (.ipa) and an **XCUI Test** app (.ipa file). - -If you do not have any **iOS** app (.ipa) and an **XCUI Test** app (.ipa) file, you can run your sample tests on TestMu AI by using our sample :link: [iOS app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_ios.ipa) and a sample :link: [XCUI Test](https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_ios_xcuitest.ipa). - -## Step 1: Upload Your Application - -To begin testing, upload your iOS application (.ipa file) to TestMu AI's servers. You'll use our **REST API** for this process. - -- **Authentication :** You'll need your TestMu AI Username and AccessKey. Combine them in the format `Username:AccessKey`. -- **Uploading the App :** Use **cURL command** to send a request to our API. The request should include the path to your application file (**appFile**). - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" --location --request POST 'https://manual-api.lambdatest.com/app/uploadFramework' --form 'appFile=@""' --form 'type="xcuit-ios"'`} - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" -X POST "https://manual-api.lambdatest.com/app/uploadFramework" -F "appFile=@""" -F "type="xcuit-ios""`} - -- Provide the path of your android application in the above URL in place of `` -- Response of above cURL will be a **JSON** object containing the `App URL` of the format - `lt://APP123456789123456789` and will be used in the last step. - -## Step 2: Upload Your Test Suite and XCTestRun File - -Upload your iOS test suite (.ipa) file to TestMu AI servers using our REST API. The following sample cURL command shows how to upload a test suite: - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" --location --request POST 'https://manual-api.lambdatest.com/app/uploadFramework' --form 'appFile=@""' --form 'type="xcuit-ios"'`} - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" --location --request POST "https://manual-api.lambdatest.com/app/uploadFramework" --form "appFile=@""" --form "type=\"xcuit-ios\""`} - -- Provide the path of your android application in the above URL in place of `` -- Response of above cURL will be a **JSON** object containing the `App URL` of the format - `lt://APP123456789123456789` and will be used in the last step. - -> This is a sample xctestplan file which is compatible with our sample application provided above. - -```js title="testLaunch.xctestplan" -{ -"configurations" : [ -{ -"id" : "1D354D3E-7B86-4201-A441-6D23AF535802", -"name" : "Configuration 1", -"options" : { -} -} -], -"defaultOptions" : { -}, -"testTargets" : [ -{ -// highlight-next-line -"skippedTests" : [ -// highlight-next-line -"LambdaUiKitIOSUITests\/testverifyAppLaunch()" -// highlight-next-line -], -"target" : { -"containerPath" : "container:LambdaUiKitIOS.xcodeproj", -"identifier" : "84946A3027745AFD000E2562", -"name" : "LambdaUiKitIOSUITests" -} -} -], -"version" : 1 -} -``` - -This will only be used for filtering tests using xctestplan. You can use the following capability in xcui/build api: -`"xctestplan" : "lt://APP1234567891234567890"` and use the following curl to upload xctestplan file. - -```bash -curl --location --request POST 'https://manual-api.lambdatest.com/app/xctestplan' \ ---header 'Cookie: cf_use_ob=0' \ ---header 'Authorization: Basic ' \ ---form 'appFile=@"/path/to/file"' \ ---form 'type="xctestplan"' -``` - -## Step 3: Executing The Test - -- You will need **base64 encoded authentication** in order to execute your Espresso automation test suite. Enter your `username:accesskey`�in **[Basic Authentication Header Generator](https://mixedanalytics.com/knowledge-base/api-connector-encode-credentials-to-base-64/)** to generate your auth token. - -Take note of the�base64�encoded authentication which needs to be added in the next step. - -{`${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}`} - -- Once you have uploaded your app and test suite, you can execute your test by running the following command: - -> Enter your **BASIC_AUTH_TOKEN**, **APP_ID** (generated in the first step) and **TEST_SUITE_ID** (generated in the second step) in the below command. - -```yaml -curl --location --request POST 'https://mobile-api.lambdatest.com/framework/v1/xcui/build' \ ---header 'Authorization: Basic BASIC_AUTH_TOKEN' \ ---header 'Content-Type: application/json' \ ---data-raw '{ -"app" : "lt://APP_ID", -"testSuite": "lt://TEST_SUITE_ID", -"xctestplan" : "lt://YOUR_XC_TEST_PLAN_ID" #only when you want to use XCTestPlan -"device" : ["iPhone 11-14"], -"video" : true, -"queueTimeout": 10800, -"idleTimeout": 150, -"devicelog": true, -"network": false, -"build" : "Proverbial-XCUITest" -}' -``` - -```yaml -curl --location --request POST "https://mobile-api.lambdatest.com/framework/v1/xcui/build" \ ---header "Authorization: Basic BASIC_AUTH_TOKEN" \ ---header "Content-Type: application/json" \ ---data-raw "{ -"app" : "lt://APP_ID", -"testSuite": "lt://TEST_SUITE_ID", -"xctestplan" : "lt://YOUR_XC_TEST_PLAN_ID" #only when you want to use XCTestPlan -"device" : ["iPhone 11-14"], -"video" : true, -"queueTimeout": 10800, -"idleTimeout": 150, -"devicelog": true, -"network": false, -"build" : "Proverbial-XCUITest" -}" -``` - -## Step 4: View Test Execution - -Once you have run your tests, you can view the test execution along with logs. You will be able to see the test cases passing or failing. You can view the same at TestMu AI Automation. - -> **NOTE:** You cannot use XCTestPlan and [Filters](/support/docs/speedup-xcui/) simultaneously - -To learn how to use XCTestPlan with sharding in XCUI, refer to the detailed guide in [Sharding for XCUI](/support/docs/sharding-rd-hyperexec/#xctestplan-in-sharding). diff --git a/static/docs/xcui-automation-on-emulators-simulators.md b/static/docs/xcui-automation-on-emulators-simulators.md deleted file mode 100644 index d01674783..000000000 --- a/static/docs/xcui-automation-on-emulators-simulators.md +++ /dev/null @@ -1,3 +0,0 @@ -# XCUI Automation using Emulators and Simulators on TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). diff --git a/static/docs/xcui-report.md b/static/docs/xcui-report.md deleted file mode 100644 index f9dacff77..000000000 --- a/static/docs/xcui-report.md +++ /dev/null @@ -1,37 +0,0 @@ -# XCUI XML Reports - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -XML reports provide a detailed summary of test execution, allowing you to better understand your test outcomes. This document aims to guide you through the process of retrieving XML reports for XCUI tests executed on the TestMu AI platform. - -## Objective - -### By the end of this document, you should be able to: - -1. Fetch XML reports for non-shard XCUI builds. - -2. Fetch XML reports for shard builds, both for individual shards and all shards collectively. - -## XML report APIs - -**Non-shard build :** -To fetch the XML report for a `non-shard` build, you can use the following cURL command: - -{`curl --location "https://mobile-api.lambdatest.com/mobile-automation/api/v1/framework/builds//report/?encoder=false" \ ---header 'Authorization: Basic '`} - -**Shard build (For single shard):** -To fetch the XML report for a `single shard` in a shard build,use: - -{`curl --location 'https://mobile-api.lambdatest.com/mobile-automation/api/v1/framework/jobs//report/?shard=&encoder=false' \ ---header 'Authorization: Basic '`} - -**Shard build (For all the shards):** -To fetch the XML reports for `all shards` in a shard build, use: - -{`curl --location 'https://mobile-api.lambdatest.com/mobile-automation/api/v1/framework/jobs//report/?encoder=false' \ ---header 'Authorization: Basic '`} - -- Authenticate the API using your TestMu AI username and access key, and replace `build_id`, `job_id` and `shard_id` for which you want to fetch report. -- It is recommended to run the sharding test(via HyperExecute CLI) in the verbose mode i.e. with the **--verbose** flag. This allows the shard ID(task ID) and build ID(Job ID) to be displayed in the logs and then they can be used to fetch the above reports. -- In case the report is not a valid XML format, the `encoder=true` parameter can be utilized to prevent the decoding of certain characters. Decoding is usually performed at the server's end to enhance the readability of the report. diff --git a/static/docs/xcui-supported-capabilities.md b/static/docs/xcui-supported-capabilities.md deleted file mode 100644 index d7985190a..000000000 --- a/static/docs/xcui-supported-capabilities.md +++ /dev/null @@ -1,33 +0,0 @@ -# XCUI Supported Capabilities - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -This document provide details about the features and capabilities supported for XCUI Framework on TestMu AI. - -> The capabilities listed in this document are also supported for [Virtual Devices](/support/docs/app-automation-on-emulators-simulators/) - -| Capability Name | Data Type | Description | -|------|-----------|-------------| -| app | String | Enter the app id generated while uploading the app to the platform. Example:`lt://APP123456789123456789` | -| testSuite | String | Enter the test suite id generated while uploading the test suite to the platform. Example: `lt://APP123456789123456789` | -| device | String | Enter the device name and os version in `DeviceName-OSVersion` format. Example: `"device" : ["Galaxy S20-10","Pixel 3 XL-9"]` | -| video | Boolean | When you pass the `video: true`, it will generate the video for all the tests that are executed. | -| queueTimeout | Integer | Enter the time in seconds after which you want your build to timeout from queue. Example: `queueTimeout: 300` | -| idleTimeout | Integer | Enter the time in seconds for maximum running time of a test in a build. Example: `idleTimeout: 120`| -| deviceLog | Boolean | To generate the device logs, pass the value as `deviceLog: true` | -| network | String | To generate the network logs, pass the value as `network: true`. | -| build | String | To set the Espresso build name. Example: `build: My Espresso Build`. | -| geoLocation | String | Set the geolocation [country code](/support/docs/appium-ip-geolocation/#supported-ip-geolocations) if you want to enable the same in your test. Example: `geoLocation: FR`| -| gpsLocation | Decimal | The capability should be defined to override the device’s default GPS coordinates and should be provided as a comma-separated string. Example: `"location": {"lat": "-50","long": "150.028333"}`.| -| tunnel, tunnelName | Boolean | To activate the tunnel mode, pass the value as `tunnel: true` and provide the name of your tunnel as `tunnelName: NewTunnel` | -| enableBluetooth | Boolean | Used to enable the bluetooth functionality during the session. Example: `enableBluetooth: true`| -| resignApp Not supported in **Virtual Devices** | Boolean | Set this to `false` if you want to to prevent the apps from being re-signed. The app should be built for enterprise distribution. | -| testRemarks | Boolean| Set to `true` to surface failure remarks at the top of the dashboard when a test fails. Has no effect on passing tests. **Default**: `false`. | -| retries | Integer |Defines the number of times a test should automatically retry if it fails. Maximum allowed value: 5.| -| region | String | Set the region for the test execution data center. Supported values: `US`, `EU`, `AP`. **Default:** Nearest data center. Example: `region: EU` | -| uploadMedia | String | Upload media files to the device for testing. Provide the media URL generated after [uploading the file](/support/docs/upload-media/). **Default:** `null`. Example: `uploadMedia: lt://MEDIA123456789` | - -Ensure that the latitude is between -90 and 90, and the longitude is between -180 and 180. Otherwise, an error will occur like "Invalid GPS location: Latitude must be between -90 and +90, Longitude must be between -180 and +180." - -**Important** -You can either use **GeoLocation** or **Tunnel** in single execute command. They both are mutually exclusive. diff --git a/static/docs/xcui-visual-regression.md b/static/docs/xcui-visual-regression.md deleted file mode 100644 index b6bb8317f..000000000 --- a/static/docs/xcui-visual-regression.md +++ /dev/null @@ -1,412 +0,0 @@ -# Getting Started With Visual Regression Testing Using XCUI On SmartUI Real Devices - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -XCUI (XCTest UI) is Apple's native UI testing framework used for testing iOS applications. It is part of the XCTest framework and allows developers to write UI automation tests for iOS apps. In this documentation, you will learn step-by-step how to perform regression testing and find Visual UI Regression bugs easily with the help of Smart Testing. - -## Prerequisites - -- Your TestMu AI [Username and Access key](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/security). -- Access to an **iOS** app (.ipa) and an **iOS Test** app (.ipa file). -- Go to [`LambdaTest SmartUI`](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) and login along with your credentials. - -**Sample repo** -If you do not have any **iOS** app (.ipa) and an **iOS Test** app (.ipa) file, you can run your sample tests on TestMu AI by using our sample :link: [XCUI app](https://prod-mobile-artefacts.lambdatest.com/assets/docs/prod-ios-app.ipa) and a sample :link: [XCUI Test App](https://prod-mobile-artefacts.lambdatest.com/assets/docs/prod-ios-test-app.ipa). - -Download or Clone the code sample from the TestMu AI GitHub repository to run the tests on the SmartUI. - View on GitHub - -## Step 1: Create a SmartUI Project - -The first step is to create a project with the application in which we will combine all your **builds** run on the project. -To create a SmartUI Project, follow these steps: - -1. Go to [Projects page](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/) -2. Click on the `new project` button -3. Select the platform as Real Device for executing your `XCUI` tests. -4. Add name of the project, approvers for the changes found, tags for any filter or easy navigation. -5. Click on the **Submit**. - -## Step 2: Set up your Authentication - -Now, you need to export your environment variables LT_USERNAME and LT_ACCESS_KEY that are available in the [TestMu AI Profile page](https://www.testmuai.com/login/?redirectTo=https://accounts.lambdatest.com/details/profile). - -Run the below mentioned commands in your terminal to setup the CLI and the environment variables. - - {`export LT_USERNAME=${ YOUR_LAMBDATEST_USERNAME()} -export LT_ACCESS_KEY=${ YOUR_LAMBDATEST_ACCESS_KEY()}`} - - {`set LT_USERNAME="${ YOUR_LAMBDATEST_USERNAME()}" -set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`} - -## Step 3: Update your App Configurations - -> It should be in your app configurations then only you can perform the XCUI integration. -- Update this [dependency](https://swiftpackageindex.com/LambdaTest/lambdatest-xcui) in your test classes. You can do it as follows: - -```java -import XCTest -import SmartuiXcui // importing the package -final class MyAppUITests: XCTestCase { -func testExample() throws { -// launch application -let app = XCUIApplication() -app.launch() -// take screenshot -let ltApp = LTApp() -try ltApp.screenshot(name: "screenshotName") -} -} -``` - -Now build your application. - -## Step 4: Upload your Application - -To begin testing, upload your iOS application (.ipa file) to TestMu AI's servers. You'll use our **REST API** for this process. - -- **Authentication :** You'll need your TestMu AI Username and AccessKey. Combine them in the format `Username:AccessKey`. -- **Uploading the App :** Use **cURL command** to send a request to our API. The request should include the path to your application file (**appFile**). - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" --location --request POST 'https://manual-api.lambdatest.com/app/uploadFramework' --form 'appFile=@""' --form 'type="xcuit-ios"'`} - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" --location --request POST "https://manual-api.lambdatest.com/app/uploadFramework" --form "appFile=@""" --form "type=\"xcuit-ios\""`} - -- Provide the path of your ios application in the above URL in place of `` -- Response of above cURL will be a **JSON** object containing the `App URL` of the format - `lt://APP123456789123456789` and will be used in the last step. - -## Step 5: Upload Your Test Suite - -Upload your XCUI test suite (.ipa) file to TestMu AI servers using our REST API. - -The following sample cURL command shows how to upload a test suite: - - {`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" --location --request POST 'https://manual-api.lambdatest.com/app/uploadFramework' --form 'appFile=@""' --form 'type="xcuit-ios"'`} - -{`curl -u "${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}" --location --request POST "https://manual-api.lambdatest.com/app/uploadFramework" --form "appFile=@""" --form "type=\"xcuit-ios\""`} - -- Provide the path of your ios application in the above URL in place of `` -- Response of above cURL will be a **JSON** object containing the `App URL` of the format - `lt://APP123456789123456789` and will be used in the last step. - -## Step 5: Executing The Test - -- You will need **base64 encoded authentication** in order to execute your Espresso automation test suite. Enter your `username:accesskey` in **[Basic Authentication Header Generator](https://mixedanalytics.com/knowledge-base/api-connector-encode-credentials-to-base-64/)** to generate your auth token. - -Take note of the base64 encoded authentication which needs to be added in the next step. - -{`${ YOUR_LAMBDATEST_USERNAME()}:${ YOUR_LAMBDATEST_ACCESS_KEY()}`} - -- Once you have uploaded your app and test suite, you can execute your test by running the following command: - -> Enter your **BASIC_AUTH_TOKEN**, **APP_ID** (generated in the fourth step) and **TEST_SUITE_ID** (generated in the fifth step) in the below command. - -```javascript -curl --location 'https://mobile-api.lambdatest.com/framework/v1/xcui/build' \ ---header 'Authorization: Basic BASIC_AUTH_TOKEN' \ ---header 'Content-Type: application/json' \ ---data '{ -"app" : "APP_ID", //enter your app-id -"testSuite": "TEST_SUITE_ID", //enter your test-suite id -"device" : ["iPhone 13-16"], //enter your device names -"smartUI.project": "XCUI-SmartUI-Project", -"build" : "Proverbial-XCUITest", -"video" : true, -"queueTimeout": 10800, -"idleTimeout": 150, -"devicelog": true, -"network": false, -"only-testing" : ["LambdaUiKitIOSUITestsLaunchTests"], -"smartUI.cropStatusBar" : true, -"smartUI.cropNavigationBar" : true -}' -``` - -> You can check the executed builds over at [TestMu AI SmartUI](https://www.testmuai.com/login/?redirectTo=https://smartui.lambdatest.com/). - -## How to use Sharding - -1. Firstly, create a folder on your local. -2. Download the HyperExecute CLI file and put it under this folder. You may download HyperExecute CLI from either of the following ways: -- Download it from our HyperExecute documentation page [here](/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/), or -- "Get Started" through HyperExecute [onboarding](https://www.testmuai.com/login/?redirectTo=https://hyperexecute.lambdatest.com/quickstart) and download the CLI. -3. Refer to the sample `.ipa` files and `.yaml` file given below. Put all of them under this folder. -- [ProverbialTest.ipa](https://prod-mobile-artefacts.lambdatest.com/assets/docs/ProverbialTest.ipa) -- [LambdaUiKitIOS.ipa](https://prod-mobile-artefacts.lambdatest.com/assets/docs/LambdaUiKitIOS.ipa) - -Refer to the sample `.yaml` file here - -```bash title="SampleYamlFile.yaml" -version: "0.2" -concurrency: 2 -runson: ios -autosplit: true -maxRetries: 2 -retryOnFailure: true -globalTimeout: 180 #MAXQUEUETIMEOUT - -framework: -name: "ios/xcui" -args: -buildName: "XCUIT" -video: true -networkLog: true -deviceLog: true - -# You can use either the appId (APP1234567) or provide the path of the application using appPath. - -#highlight-next-line -appPath: LambdaUiKitIOS.ipa -# We have used the appPath here instead of appId - -# You can use either the APP ID (APP1234567) or provide the path of the application. - -#highlight-next-line -testSuiteAppId: lt://APP10160202521675167637685231 -# We have used the testSuiteAppID here instead of testSuitePath - -deviceSelectionStrategy: all -devices: ["iPhone 12 Pro-14", "iPad Air (2019)-16"] - -smartUI: -project: "Espresso-SmartUI-Project" - -shards: -mappings: -- name: shard1 -strategy: "only-testing/skip-testing" -values: ["/"] -- name: shard2 -strategy: "only-testing/skip-testing" -values: ["/", "/"] -``` - -**When shards are added** - -If you are using the `deviceSelectionStrategy: all`, then in that case all the mentioned shards will be executed on all the devices. -**For example:** There are 2 shards and 2 devices mentioned, then the 2 shards will be executed on 2 devices (2 Shards * 2 Devices). - -If you are using the `deviceSelectionStrategy: any`, then in that case all the mentioned shards will be executed on any one device from the list provided. -**For example:** There are 2 shards and 2 devices mentioned, then the 2 shards will be executed on any device (2 shards * any device mentioned). - -**When shards aren't added** - -If you are using the `deviceSelectionStrategy: all`, then in that case all the tests will be executed on all the devices based on the concurrency. -**For example:** There are 6 tests and 10 devices mentioned, then the 6 tests will be auto distributed on the devices based on the max concurrency. - -If you are using the `deviceSelectionStrategy: any`, then in that case all the mentioned tests will be executed on any one device from the list provided. -**For example:** There are 6 tests and 10 devices mentioned, then the 6 tests will be executed on any one device. - -## Filters in Sharding -You can filter the Classes / Tests that you'd like to execute using filters. -Here's an example of the same. - -```java -filters: -attributes: -- type: className -values: ["LambdaUiKitIOSUITests","LambdaUiKitIOSUITestsLaunchTests"] -- type: testName -values: ["LambdaUiKitIOSUITests/testverifyAppLaunch"] -``` -This example will run only the 2 classes & one test as mentioned from the TestSuite. - -## XCTestPlan in Sharding -To implement the XCTestPlan in Sharding, add the `xctestplan` flag along with `app` and `testSuite` in the [framework](/support/docs/hyperexecute-yaml-version0.2/#framework) flag as shown below: - -```yaml -framework: -name: "ios/xcui" -args: -"app" : "lt://APP_ID", -"testSuite": "lt://TEST_SUITE_ID", -"xctestplan" : "lt://YOUR_XC_TEST_PLAN_ID" #only when you want to use XCTestPlan -``` - -> Refer the documentation to learn how to use [XCTestPlan feature](/support/docs/xctestplan/). - -## Smart Crop With SmartUI - -The all-new **Real Device mobile notification status bar and navigation bar crop** feature in SmartUI allows you to take your visual regression testing workflows to the next level. With Smart Crop, you can crop the status bar and navigation bar or footer from screenshots, enabling them to focus solely on the core UI elements during visual comparisons. - -By leveraging machine learning algorithms, it accurately detects and crops the status bar and navigation bar from screenshots. With precise image processing techniques, SmartUI precisely identifies the location of status bar elements. By excluding it from visual comparisons, the focus is solely on critical UI elements. - -| Original Screenshot | Cropped Screenshot | -|---------------------|--------------------| -| | | - -## Best Practices - -### Project and Build Naming - -- Use descriptive, consistent names for projects and builds -- Include app version or release info in build names -- Avoid special characters that might cause issues -- Use consistent naming conventions across test runs - -**Example:** -```json -{ -"smartUI.project": "MyApp-iOS-VisualTests", -"smartUI.build": "Release-1.0.0" -} -``` - -### Device Selection - -- Test on devices that match your user base -- Include multiple device configurations for comprehensive coverage -- Use device patterns for consistent testing (e.g., `iPhone 13-16`, `iPad Air (2019)-16`) - -### Smart Crop Configuration - -- Enable `cropStatusBar` to focus on core UI elements -- Enable `cropNavigationBar` for iOS devices -- Test cropped screenshots to ensure important content isn't removed - -### Test Organization - -- Use sharding for parallel test execution -- Group related tests in same build -- Use meaningful test names for better organization - -### App and Test Suite Management - -- Upload apps and test suites before execution -- Use app IDs (`lt://APP...`) for faster execution -- Keep app and test suite versions synchronized - -### App and Test Suite Management - -- Upload apps and test suites before execution -- Use app IDs (`lt://APP...`) for faster execution -- Keep app and test suite versions synchronized - -## Troubleshooting - -### Issue: Screenshots Not Captured - -**Symptoms**: Tests run but no screenshots appear in SmartUI dashboard - -**Possible Causes**: -- `visual: true` not set in request -- Incorrect project name -- Network connectivity issues -- Authentication issues - -**Solutions**: -1. Verify `visual: true` is set in API request (implicit for SmartUI) -2. Check project name matches exactly (case-sensitive): -```json -{ -"smartUI.project": "ExactProjectName" -} -``` - -3. Verify authentication token is correct: - - Generate base64 encoded token from username:accesskey - - Ensure token is included in Authorization header - -4. Check network connectivity to TestMu AI - -### Issue: "Project Not Found" Error - -**Symptoms**: Error indicating SmartUI project cannot be found - -**Possible Causes**: -- Project name typo or mismatch -- Project deleted -- Wrong account credentials - -**Solutions**: -1. Verify project exists in SmartUI dashboard -2. Copy project name directly from dashboard -3. Check credentials match the account with the project -4. Ensure project name is in API request - -### Issue: App Upload Fails - -**Symptoms**: App upload returns error or fails - -**Possible Causes**: -- Invalid IPA file -- File size too large -- Network issues -- Authentication problems - -**Solutions**: -1. Verify IPA file is valid and not corrupted -2. Check file size limits -3. Retry upload with stable network connection -4. Verify authentication credentials - -### Issue: Test Execution Fails - -**Symptoms**: Test suite execution fails or times out - -**Possible Causes**: -- Invalid test suite IPA -- Device not available -- Timeout settings too low -- Test suite errors - -**Solutions**: -1. Verify test suite IPA is valid -2. Check device availability -3. Increase `queueTimeout` and `idleTimeout`: -```json -{ -"queueTimeout": 10800, -"idleTimeout": 150 -} -``` - -4. Review device logs for test errors - -### Issue: Screenshots Show Incorrect Content - -**Symptoms**: Screenshots captured but show wrong screen or state - -**Possible Causes**: -- App state issues -- Timing problems -- Navigation issues - -**Solutions**: -1. Ensure app is in correct state before test execution -2. Add appropriate waits in test code -3. Verify test navigation flow - -### Issue: Sharding Not Working - -**Symptoms**: Shards not executing as expected - -**Possible Causes**: -- Incorrect shard configuration -- Device selection strategy mismatch -- Test class/name typos - -**Solutions**: -1. Verify shard configuration in YAML file -2. Check device selection strategy matches requirements -3. Verify test class names and test names are correct -4. Review HyperExecute logs for shard execution details - -### Getting Help - -If you encounter issues not covered here: - -- Review the [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) for detailed solutions -- Check [SmartUI Configuration Options](/support/docs/smartui-sdk-config-options) documentation -- See [Handling Dynamic Data](/support/docs/smartui-handle-dynamic-data) for dynamic content issues -- Visit [TestMu AI Support](https://www.testmuai.com/support) for additional resources -- Contact support at support@testmuai.com or use [24/7 Chat Support](https://www.testmuai.com/support) - -## Additional Resources - -- [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) -- [SmartUI Configuration Options](/support/docs/smartui-sdk-config-options) -- [Handling Dynamic Data](/support/docs/smartui-handle-dynamic-data) -- [Baseline Management](/support/docs/smartui-baseline-management) -- [Running Your First Project](/support/docs/smartui-running-your-first-project) -- [Appium Hooks Documentation](/support/docs/smartui-appium-hooks) -- [Espresso Visual Regression](/support/docs/espresso-visual-regression) diff --git a/static/docs/youtrack-integration.md b/static/docs/youtrack-integration.md deleted file mode 100644 index c5efbd171..000000000 --- a/static/docs/youtrack-integration.md +++ /dev/null @@ -1,85 +0,0 @@ -# YouTrack Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -YouTrack is a bug tracking and project management tool developed by JetBrains. It allows you to create agile boards, use reports and Gantt charts, dashboards, and time tracking. With YouTrack, you can find issues easily based on autocomplete queries and manipulate them in batches, customize all problem attributes, and create custom workflows. - -With TestMu AI and YouTrack integration, you can push bugs directly from TestMu AI to your YouTrack's project. Also, capture screenshots, annotate bugs, and share them with your teammates and colleagues. - -## How To Integrate YouTrack With Your TestMu AI Account? - -**Step 1**: Login to your TestMu AI account. To install integrations, you need to have admin or user level access. - -**Step 2**: Go to **'Integrations'** from the left menu bar. You will be routed to the integration page where you can find list of third-party applications. - -**Step 3**: From the **'Bug Tracker'** category, click on the **'YouTrack'** block. - -> -**Note:** If you already have an existing TestMu AI integration with any project management tool then you will be asked to uninstall the existing TestMu AI integration. - -**Step 4**: You will come across two different options to choose from - **Cloud** and **Self Hosted**. - -**Step 5:** When you select the **Cloud** option, you need to enter your YouTrack URL as shown below. - -**Step 6:** When you select the **Self Hosted** option, you need to enter YouTrack Host as shown below. - -> -* **What is YouTrack URL?** -YouTrack is the organization's domain name on your YouTrack Projects. It is included in the URL. - -* **What is an API Token?** -An API token is required to authenticate a user's identity from one server to another by retrieving client resources from the server where the connection is initiated. To ensure its security and data integrity, API tokens are important when an end user connects into an application via a two-step verification process. - -**Step 5**: To get the API token, log in to your YouTrack account and navigate to Profile on the top-right. - -**Step 6**: Click on the link that says _**'Update personal information and manage logins'**_. - -**Step 7**: Select the **'Authentication'** tab. - -**Step 8**: Click on **'New Token'**. - -**Step 9**: Enter the token name and click on **'Create'**. - -**Step 10**: Your new token will be generated. Click on **'Copy Token'**. - -**Step 11**: Enter the Youtrack Domain and copied API token in the provided field and then press **'Install'**. - -You have successfully integrated YouTrack with your TestMu AI account. Visit the **'Integrations'** again and you'll notice a green check on YouTrack block under the category _My Integrations_. - -## Logging First Bug Through YouTrack Integration - -For demonstration, we will log bugs while performing Real time testing. - -**Step 1**: Select the Real Time Testing from the left menu. - -**Step 2**: Enter the test URL, select browser, browser version, operating system and resolutions. Then press **'START'**. - -**Step 3**: A new virtual machine will fire up where you can perform live interactive testing of websites & web apps for filing bugs. To file a bug, click on the Bug icon to capture a screenshot. - -**Step 4**: Once the screenshot is captured, you can annotate any bugs or tasks by using in-built image editor tools. Then click on **'Mark As Bug'**. - -**Step 5**: You will get a YouTrack ticket where you are required to fill up your bug details as per your requirement and then click on **'Create Issue'**. - -> -* You can select the **Project**. -* You can choose the **Board** to manage your tasks. -* You can select the **Assignee** of the bug or task. -* You can specify the **Title** to your test. -* You can provide the **Description** of the issue. -* You can also choose **Task type**, **Task Priority**, **Task State**. - -Your bug will be marked successfully. - -**Step 6**: Visit your YouTrack agile board, you'll notice your marked bugs and tasks. - -Click on your Project, you'll find all the test environment details along with captured screenshots. - -## Uninstalling The YouTrack Integration - -**Step 1**: Login to your TestMu AI account. Visit Integrations and navigate to **'My Integrations'**. - -**Step 2**: Click on **'REMOVE'** adjacent to YouTrack block. Your YouTrack integration will be removed successfully. - -> -That's all about YouTrack integration with TestMu AI. In case, you have any questions or want any further integration with your favorite integration tools, please feel free to reach us at our **24*7 Chat Support** or email us at [support@testmuai.com](mailto:support@testmuai.com). - **Happy testing!** diff --git a/static/docs/zalenium-integration-with-testmu.md b/static/docs/zalenium-integration-with-testmu.md deleted file mode 100644 index fda0b193e..000000000 --- a/static/docs/zalenium-integration-with-testmu.md +++ /dev/null @@ -1,44 +0,0 @@ -# Zalenium Integration With TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Brought up by Zalando, Zalenium is open-source with the vision to eliminate the hassle of maintaining an in-house [Selenium Grid](https://www.testmuai.com/blog/why-selenium-grid-is-ideal-for-automated-browser-testing/). Zalenium does so by offering a Docker-based Selenium Grid on-cloud that gets auto-updated with the latest browsers and their driver versions. - -Zalenium offers capabilities to help you define your Desired Capabilities in your test automation suite. Zalenium also provides an intuitive dashboard representing all the detailed logs of your test script execution. Using Zalenium, you can even perform automation testing over your locally hosted web pages. However, you would only be able to perform automated cross browser testing over the browsers installed on your machine. - -TestMu AI integration with Zalenium will help you perform automated cross browser testing for your locally hosted web pages for over 3000+ real browser and browser versions, installed, and hosted on-cloud. - -## Prerequisites - -* Ensure you have a Zalenium Docker setup. -* **TestMu AI Authentication Credentials** - - Be aware of your TestMu AI authentication credentials i.e., your TestMu AI username, access key, and HubURL. You need to set them up as your environment variables. You can retrieve them from your **[TestMu AI automation dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/timeline/?viewType=build&page=1)** by clicking on the key icon near the help button. -* For Linux/Mac: -``` -$ export LT_USERNAME= {YOUR_LAMBDATEST_USERNAME} -$ export LT_ACCESS_KEY= {YOUR_LAMBDATEST_ACCESS_KEY} -``` -* For Windows: -``` -$ set LT_USERNAME= {YOUR_LAMBDATEST_USERNAME} -$ set LT_ACCESS_KEY= {YOUR_LAMBDATEST_ACCESS_KEY} -``` -## Specify TestMu AI Environment Variables Into Zalenium Docker - -Declare your TestMu AI environment variable into your respective Zalenium Docker over which you wish to perform automated cross browser testing. Below is an example code representing what you need to declare in your Zalenium Docker. - -``` -export LT_USERNAME= -export LT_ACCESS_KEY= -export LT_URL= # default value is "https://hub.lambdatest.com" -docker run --rm -ti --name zalenium -p 4444:4444 \ --e LT_USERNAME -e LT_ACCESS_KEY -e LT_URL \ --v /tmp/videos:/home/seluser/videos \ --v /var/run/docker.sock:/var/run/docker.sock \ ---privileged dosel/zalenium start --lambdaTestEnabled true -``` -Kudos! As simple as that, you have integrated Zalenium with the TestMu AI account. In case you have any questions, feel free to let us know by dropping an email at [support@testmuai.com][2], or just giving a shout to our **24/7 chat support**. Happy testing! 🙂 - -[1]: https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com -[2]: mailto:support%40lambdatest.com diff --git a/static/docs/zapier-integration-with-testmu.md b/static/docs/zapier-integration-with-testmu.md deleted file mode 100644 index 036b02577..000000000 --- a/static/docs/zapier-integration-with-testmu.md +++ /dev/null @@ -1,50 +0,0 @@ -# Zapier Integration With TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Zapier is a cloud-based platform that allows you to easily automate workflows between your apps and your services without having to build this integration yourself or hire someone to do it for you. Zapier works with Zaps, which you can create by combining multiple apps and services. These Zaps help you streamline your workflow and save time and effort. - -TestMu AI and Zapier enable you to create a smarter and more customized workflow, helping you save time to focus on the important things to ship quality products at light speed. - -TestMu AI is the only platform in the market with deep integration with Zapier to make automating workflows a piece of cake for you. - -## Prerequisites - -1. A TestMu AI account. If you don't have an account, [sign up for free](https://www.testmuai.com/register/?redirectTo=https://accounts.lambdatest.com/dashboard). -2. A Zapier account. - -## Integrating Zapier With TestMu AI - -1. Open your [Zapier dashboard](https://zapier.com/app/dashboard) after logging in to your Zapier account. - -2. Click on **+ Create Zap** on the top left-hand side of your screen to navigate to the Zapier editor. - -3. Select TestMu AI as the app that gets triggered and proceed to choosing an event. - -4. Select the event that will trigger the integrated alert and click on **Continue**. TestMu AI offers you three options in this field: - -- **New Build:** The integrations is triggered every time a new build is created. - -- **Marked New Issue:** The integration is triggered every time a new issue is created at TestMu AI. - -- **New Test:** The integration is triggered every time a new test is executed. - -5. Choose your TestMu AI account and click on **Continue** to navigate to testing your trigger. - -6. Test your TestMu AI trigger and click on **Continue** after the test is concluded. - -7. Proceed to configuring the action that gets triggered by the trigger. Select a tool from the myriad of option that Zapier provides. (We selected Gmail to demonstrate this entire process). - -8. Choose an event from the options that Zapier provides and click on **Continue.** - -9. Connect your Gmail account with Zapier and click on **Continue** to set up your event. - -10. Finish the required fields and click on **Continue.** - - **Note:** Zapier also allows you to use dynamic text fields from TestMu AI, such as the Test Details, Test ID, Test Browser and more. - -11. Verify your Email and click on **Test action** to test if your action and trigger are working correctly. You can also skip this test. - -12. Your Integration is ready and you can publish it to make it functional. After publishing, you will receive an email whenever a new build is created. - -You have successfully integrated Gmail with TestMu AI via Zapier. Zapier has many useful tools like Gmail, and you can use them all to optimize your testing process with TestMu AI. Happy testing! diff --git a/static/docs/zebrunner-integration.md b/static/docs/zebrunner-integration.md deleted file mode 100644 index 1068aeb2a..000000000 --- a/static/docs/zebrunner-integration.md +++ /dev/null @@ -1,36 +0,0 @@ -# Zebrunner Integration With TestMu AI’s Cloud Based Selenium Grid - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Zebrunner is a modern test reporting and analytics tool which offers very comprehensive automated test results and analysis with the help of a smart AI-based failure classification system. Through Zebrunner, you can easily detect failures with the help of rich artifacts, such as video, screens, logs, etc., and fix issues faster. - -Zebrunner now offers integration with TestMu AI to help you perform automated cross-browser testing taking advantage of the cloud-based Selenium Grid on over 2000 real browsers. TestMu AI [Selenium Grid](https://www.testmuai.com/blog/why-selenium-grid-is-ideal-for-automated-browser-testing/) integration to Zebrunner will help you run your Selenium automation scripts from Zebrunner instance to our cloud-based Selenium Grid very easily. Lets see how. - -## How To Integrate TestMu AI With Zebrunner Instance - -To integrate your TestMu AI account with the Zebrunner instance, go to the Integrations panel and activate the TestMu AI environment under Test Environment Provider category. - -The TestMu AI option will open and you will be needed to enter your TestMu AI credentials, including Hub URL, username and access key. Once you have entered the required information, click on Save button. - -As soon as you click on the Save button, your TestMu AI credentials will be verified. If the credentials verification fails, a red dot will appear against TestMu AI, else a green dot will appear stating that the integration is successful, as shown in the image below. - -## How To Execute Tests On TestMu AI From Zebrunner - -Now that you have integrated TestMu AI as the Test Environment Provide, you can execute your tests on the TestMu AI's cloud-based Selenium Grid. To do so: - -1. Select the project of your choice, and choose the environment as TestMu AI, from the list of active test environment providers. - -2. A list of options will appear now, as shown in the image below, stating the type of environment upon which you want to run your test. You can select any as per your choice. - -3. Fill the details of the test environment configuration on which you want to run your test. For example, if you select the option as _Web_, then you need to select the browser, browser version, operating system, and resolution. - -4. Once done, click on Launch, and the test execution will start immediately. - -## How To View Test Execution Details On TestMu AI Platform - -As soon as you execute the test from Zebrunner, the test execution will start on TestMu AI platform as well. To view the test execution details, open the [Automation dashboard](https://www.testmuai.com/login/?redirectTo=https://automation.lambdatest.com/). You will see your test details on the top, as shown below: - -Not only this, but you can also confirm the test environment configuration, view test execution video, console logs, network logs, exceptions, step by step screenshots, and much more. All you need to do is click on test build to open the Automation log view. - -> -Kudos! Now you can deploy your code in a reliable & faster manner at scale, using TestMu AI integration with Zebrunner, and perform automated cross browser testing with our cloud based online Selenium Grid. If you still have any questions, feel free to share them with us through our **24*7 Chat Portal** or by mailing to us on [**support@testmuai.com**](mailto:support@testmuai.com). Happy testing! diff --git a/static/docs/zenkit-integration.md b/static/docs/zenkit-integration.md deleted file mode 100644 index c162de56a..000000000 --- a/static/docs/zenkit-integration.md +++ /dev/null @@ -1,49 +0,0 @@ -# Zenkit Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Zenkit is a SaaS tool that helps teams collaborate on projects and build databases. It lets you track your information from the start, like brainstorming ideas all the way to finishing tasks together. Think of Zenkit as a flexible space to arrange and manage any work. - -TestMu AI's integration with Zenkit provides a seamless solution for tracking and managing bugs and issues encountered during [cross-browser testing](https://www.testmuai.com/online-browser-testing). With this integration, developers can effortlessly log bugs directly from the TestMu AI platform to their Zenkit instance, eliminating the need to maintain tasks and issues manually. - -## Integrating Zenkit With TestMu AI - -**Step 1:** Sign in to your TestMu AI account. Don't have an account, register for free. - -**Step 2:** Select **Integrations** from the left sidebar and head to **Zenkit**. - -**Step 3:** It will route you to the installing console window, where you are required to grant permission to TestMu AI to access your user-owned resources. - -**Step 4:** Click on the Install button to be redirected to the **Zenkit** dashboard. - -**Step 5:** Click on **Authorize**. - -You have now successfully integrated your TestMu AI account with Zenkit. - -## One-Click Bug Logging With Zenkit - -**Step 1:** In your TestMu AI Dashboard, head to **Real-Time Testing > Browser Testing** from the left navigation menu. - -**Step 2:** To perform **live-interactive testing** of your website or a web app, enter the web URL, select the testing type (Desktop/Mobile), select the browser version, OS, and resolution, and click **Start**. - -Upon clicking, a cloud-based virtual machine will fire up, running real browsers and operating systems where you can test websites and web apps for filing bugs. - -**Step 3:** Click on the **Bug icon** from the left in-session toolbar to take a screenshot of any bug you encounter. - -**Step 4:** After marking the bugs, click on the **Bug** icon in front of **Zenkit** under the integrations section. - -**Step 5:** In the Zenkit ticket, fill the bug details like **Workspace, Summary, Description, Workspace List,** and **Assignee**, and click **Mark As Bug** to log the bugs in Zenkit. - -**Step 6:** To view the logged issues, visit TestMu AI Dashboard and head to **Manage > Issue Tracker** from the left navigation bar. - -**Step 7:** Inside the **Issue Tracker**, you'll notice the logged bugs and their test environment details. Click on the **upper right arrow** in the square button to view your logged bugs in Zenkit instance. - -**Step 8:** Upon clicking, you will be redirected to the Zenkit dashboard to find your **logged bugs**. - -## Uninstalling The Zenkit Integration - -**Step 1:** Sign in to your TestMu AI account and visit **Integrations** from the left sidebar menu. - -**Step 2:** From My Integrations, click **Remove** under the **Zenkit** block. - -> That was all you need to know for TestMu AI + Zenkit Integration. Increase your productivity with our integrations. If you still have any questions for us, please feel free to let us know. Our experts are always available on **chat** to help you out with any roadblock regarding our product. Happy testing! diff --git a/static/docs/zephyr-scale-integration.md b/static/docs/zephyr-scale-integration.md deleted file mode 100644 index d9d7ee2c4..000000000 --- a/static/docs/zephyr-scale-integration.md +++ /dev/null @@ -1,142 +0,0 @@ -# Zephyr Scale Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Zephyr Scale is a test management platform that helps you plan, manage, and measure your tests inside Jira. It's designed from the ground up to support large test libraries and scaling teams. - -TestMu AI offers integration with Zephyr Scale that lets you perform automated browser testing on a cloud-based grid of 3000+ real browsers, devices, and OS combinations. By integrating Zephyr Scale with TestMu AI, you can view test execution results directly from TestMu AI platform to your Zephyr Scale instance. - -## Prerequisites - -1. Install the **Zephyr Scale - Test Management for Jira** application from the Atlassian Marketplace. - -2. A TestMu AI account. Don't have an account, sign up for free. - -3. A Jira Project key. - -4. A Zephyr Scale Access token. - -## Integrating TestMu AI with Zephyr Scale - -1. Clone the [TestMu AI Zephyr Scale GitHub repository](https://github.com/LambdaTest/junit-selenium-zephyr-scale-sample) and navigate to the cloned directory. - -```js -git clone https://github.com/LambdaTest/junit-selenium-zephyr-scale-sample -cd junit-selenium-zephyr-scale-sample -``` - -2. In the `PostTestResults.java` script, add the following String parameters. - -* File name of your test results. -* File path of the test result file. -* Jira Project key. -* Zephyr Scale Access token. - -```js -public class PostTestResults { - -public void callApi() { - -String fileName = "TEST-com.lambdatest.JUnitTodo.xml" ; //provide the file name of the test results -String filePath = "target/surefire-reports/TEST-com.lambdatest.JUnitTodo.xml"; //provide the file path of the test result file -String projectKey = "ZD"; //provide the Jira project key -String token = "abcdefghijklmnop123456"; // provide Zephyr Scale access token -``` -3. In the test script, you need to update test capabilities and add your TestMu AI Username and Access Key. You can get your TestMu AI Username and Access Key by navigating to your TestMu AI Profile section. - ->**Test Scenario**: The below JUnit Selenium script tests a sample to-do list app by marking couple items as done, adding a new item to the list and finally displaying the count of pending items as output. - -```js title="JUnitTodo.java" -package com.lambdatest; - -import org.junit.After; -import org.junit.Before; -import org.junit.Test; -import org.openqa.selenium.remote.DesiredCapabilities; -import org.openqa.selenium.remote.RemoteWebDriver; -import org.openqa.selenium.By; - -import java.net.MalformedURLException; -import java.net.URL; - -public class JUnitTodo { -String username = System.getenv("LT_USERNAME") == null ? "Your LT_USERNAME" : System.getenv("LT_USERNAME"); -String accessKey = System.getenv("LT_ACCESS_KEY") == null ? "Your LT_ACCESS_KEY" : System.getenv("LT_ACCESS_KEY"); -public static RemoteWebDriver driver = null; - -public String gridURL = "@hub.lambdatest.com/wd/hub"; -public String status = "failed"; -@Before -public void setUp() throws Exception { -DesiredCapabilities capabilities = new DesiredCapabilities(); -capabilities.setCapability("browserName", "chrome"); -capabilities.setCapability("version", "latest"); -capabilities.setCapability("platform", "Windows 10"); // If this cap isn't specified, it will just get the any available one -capabilities.setCapability("build", "LambdaTest-Zephyr Demo"); -capabilities.setCapability("name", "LambdaTest-Zephyr Tests"); - -try { -driver = new RemoteWebDriver(new URL("https://" + username + ":" + accessKey + gridURL), capabilities); -} catch (MalformedURLException e) { -System.out.println("Invalid grid URL"); -} catch (Exception e) { -System.out.println(e.getMessage()); -} -} - -@Test - -public void testSimple() throws Exception { -try { -//Change it to production page -driver.get("https://lambdatest.github.io/sample-todo-app/"); - -//Let's mark done first two items in the list. -driver.findElement(By.name("li1")).click(); -driver.findElement(By.name("li2")).click(); - -// Let's add an item in the list. -driver.findElement(By.id("sampletodotext")).sendKeys("Yey, Let's add it to list"); -driver.findElement(By.id("addbutton")).click(); - -// Let's check that the item we added is added in the list. -String enteredText = driver.findElementByXPath("/html/body/div/div/div/ul/li[6]/span").getText(); -if (enteredText.equals("Yey, Let's add it to list")) { -status = "passed"; -} -} catch (Exception e) { -System.out.println(e.getMessage()); -} -} -@After -public void tearDown() throws Exception { -if (driver != null) { -driver.executeScript("lambda-status=" + status); -driver.quit(); - -PostTestResults method = new PostTestResults(); -method.callApi(); -} -} -} -``` -**Important** -You need to invoke `method.callApi()` from the PostTestResults class in the `@After` hook of `JUnitTodo.java` file. - -## Running your Automation tests with TestMu AI and Zephyr Scale - -To run the test, pass the below command in the terminal. - -```js -mvn test -P single -``` - -Go to the TestMu AI's Automation Dashboard to view the test execution results. - -## Viewing the Test Results in Zephyr Scale - -To view test results in Zephyr Scale, head over to your Zephyr Scale project. The below screenshot shows the Test Case in Zephyr Scale that you executed on TestMu AI. - -You can also view the executed build by navigating to **Test Cycles**. - -To view the build details, click on the Zephyr project *Key* or *Name*. diff --git a/static/docs/zipboard-integration.md b/static/docs/zipboard-integration.md deleted file mode 100644 index df48017b9..000000000 --- a/static/docs/zipboard-integration.md +++ /dev/null @@ -1,43 +0,0 @@ -# zipBoard Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -ZipBoard is a visual feedback and bug tracking tool designed to streamline collaboration across teams during the web development process. By integrating TestMu AI with ZipBoard, teams can enhance their testing workflows, enabling seamless communication between developers, testers, and stakeholders. - -This integration allows you to capture real-time feedback and track issues directly within your test environments on TestMu AI's cloud platform, ensuring faster bug resolution and better product quality. In this documentation, we'll guide you through the steps to integrate ZipBoard with TestMu AI and optimize your test management workflow. - -## Integrate zipBoard from your TestMu AI Account - -**Step 1:** Login to your TestMu AI account. You should have Admin or User level access to see and install integrations. - -**Step 2:** Click on Settings -> Integrations -> Bug Tracker. - -**Step 3:** Click on **Connect** button of zipBoard block. - -**Step 4:** Now, provide your zipBoard API Token to establish integration with TestMu AI and click on **install** button. - -**Fetch your Bugherd API Token** -- Visit your zipBoard account -> **Edit Profile** tab. -- Click on the **+** icon to generate your API. Copy your API token to use it for authenticate it with TestMu AI integration. - -> This API key will be used to authenticate your zipBoard account to third-party apps. Please do not share it with anyone. If you believe your API key has been misplaced, you can always generate a new API key from zipBoard by clicking on the refresh icon in your profile settings. - -## Lodge your First Bug - -**Step 1:** Create a new projects on zipBoard if you haven't already. - -**Step 2:** Now, while testing your webapp or application at TestMu AI, click on the **mark as bug** button if you detect any bug. - -**Step 3:** Now update the comments of the bug and other details and click on Create Task button. - -**Step 4:** Go to your dashboard and check a ticket will be created for the same. - -## Uninstall zipBoard Integration - -**Step 1:** Login to your TestMu AI account. You should have Admin or User level access to see and install integrations. - -**Step 2:** Click on Settings -> Integrations -> Bug Tracker. - -**Step 3:** Click on the Remove button. - -> That was all you need to know for TestMu AI + zipBoard Integration. Increase your productivity with our integrations. If you still have any questions for us, please feel free to let us know. Our experts are always available on **chat** to help you out with any roadblock regarding our product. Happy testing! 🙂 diff --git a/static/docs/zoho-bugtracker-integration.md b/static/docs/zoho-bugtracker-integration.md deleted file mode 100644 index 5853ff12f..000000000 --- a/static/docs/zoho-bugtracker-integration.md +++ /dev/null @@ -1,52 +0,0 @@ -# Zoho BugTracker Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Zoho BugTracker is a robust cloud-based bug reporting platform that allows you to keep track of bugs and fix them rapidly. With its multitude of customizations and automation features, issue tracking is simple and effective. Additionally, Zoho BugTracker includes SLA management, time tracking, collaboration, analytics, document management, and automation. - -The TestMu AI integration with Zoho BugTracker allows you to log bugs while performing [cross browser testing](https://www.testmuai.com/) of your websites and web apps across over 3000+ browsers and OS combinations on the cloud. From the TestMu AI platform, you can easily log bugs to Zoho BugTracker dashboard. - -## Integrating Zoho BugTracker With TestMu AI - -1. Login to your TestMu AI account. Before you configure integrations, make sure you have administrator or user level access. - -2. Select **Integrations** from the left menu, then **ZOHO BUGTRACKER**. - -3. It will redirect to the below screen where you need to sign-in and activate your Zoho BugTracker account. - -4. Zoho BugTracker will ask you for permission to access data in your Zoho account after signing in. Simply click **Accept**. - -You now have TestMu AI integrated with Zoho BugTracker. - -## One-Click Bug Logging With Zoho BugTracker - -1. From the TestMu AI dashboard, select **Real Time Testing** from the left menu. - -2. Enter the test URL, select the browser, browser versions, operating system, and click **START**. - -It will redirect you to the cloud-based VM (virtual machine). - -3. If you come across any glitch or bugs, click the **Bug** icon to grab a screenshot of a web page from the left in-built tools. - -4. Using the in-built image editor tools, you can annotate bugs or task on a web page. Then click **Mark as Bug**. - -5. On clicking Mark as Bug, a Zoho BugTracker ticket will show up. Fill the details like Portal, Project, Title, and Description. Then click **CREATE ISSUE** - -Your bugs will now be marked to the Zoho BugTracker dashboard. - -7. Now visit TestMu AI dashboard. From the left menu, select **Issue Tracker**. - -8. On opening Issue Tracker, you'll notice the logged bugs along with the test environment details on the right. To view logged bugs in Zoho BugTracker, click **upper right arrow in square** button. - -It will route you to the Zoho BugTracker dashboard where you can find your marked bugs along with the test environment details. - -## Uninstalling The Zoho BugTracker Integration - -1. Login to your TestMu AI account and visit **Integrations** from the left menu. - -2. From **My Integrations**, click **REMOVE** under the ZOHO BUGTRACKER block. - -The Zoho BugTracker integration will be uninstalled. - -> -In case you have any questions or need further assistance regarding TestMu AI + Zoho BugTracker integration, please feel free to reach us at our **24*7 Chat Support** or email us at [support@testmuai.com](mailto:support@testmuai.com). diff --git a/static/docs/zoho-cliq-integration.md b/static/docs/zoho-cliq-integration.md deleted file mode 100644 index 8f295d1fa..000000000 --- a/static/docs/zoho-cliq-integration.md +++ /dev/null @@ -1,49 +0,0 @@ -# Zoho Cliq Integration - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Zoho Cliq is a team communication and collaboration platform designed to enhance productivity and streamline communication within organizations. It offers a range of features to facilitate real-time messaging, audio and video calls, file sharing, and integrations with other business applications. - -Integrating TestMu AI with Zoho Cliq, users can create detailed test cards in Zoho Cliq Channels for logged bugs in TestMu AI. This will notify you and your team members every time a new bug is pushed while performing [cross browser testing](https://www.testmuai.com) of your website from the TestMu AI platform to your Zoho Cliq instance. - -## Integrating Zoho Cliq With TestMu AI - -**Step 1:** Sign in to your TestMu AI account. Don't have an account, register for free. - -**Step 2:** Select **Integrations** from the left sidebar and then click on **Zoho Cliq**. - -**Step 3:** Click on the **Zoho Cliq** icon, and you will be presented with an instance window. Click on the **Install** button. - -**Step 4:** Upon clicking the **Install** button, you will be redirected to the **Zoho Accounts** page. Sign in to your **Zoho Account**. - -**Step 5:** Click the **Accept** button to allow TestMu AI to access data in your Zoho Account. - -**Step 6:** You will be redirected back to the TestMu AI dashboard. Select the **Zoho Cliq Channel** from the dropdown. In this case, we will select the **#announcements** channel and click **Confirm Channel**. - -You have now successfully integrated TestMu AI with Zoho Cliq. - -## One-Click Bug Logging With Zoho Cliq - -**Step 1:** In your TestMu AI Dashboard, head to **Real-Time Testing > Browser Testing** from the left navigation menu. - -**Step 2:** To perform **live-interactive testing** of your website or a web app, enter the web URL, select the testing type (Desktop/Mobile), select the browser version, OS, and resolution, and click **Start**. - -Upon clicking, a cloud-based virtual machine will fire up, running real browsers and operating systems where you can test websites and web apps for filing bugs. - -**Step 3:** Click on the **Bug icon** from the left in-session toolbar to take a screenshot of any bug you encounter. - -**Step 4:** After marking the bugs, click on the **Bug icon** in front of TestMu AI under the integrations section. - -**Step 5:** In the Zoho Cliq ticket, fill the bug details like **Assignee, Issue Type, Priority, Summary,** and **Description** and check the **Send to Zoho Cliq** option. Then click **Mark As Bug** to log the bugs in Webex Teams. - -**Step 6:** Head to **More Tools** in the TestMu AI dashboard and click on **Issue Tracker**. You'll notice the marked bugs along with the test details. - -**Step 7:** You can find the test log details directly in your selected **Zoho Cliq Channel**, as shown below, and share them with your team members. - -## Uninstalling The Zoho Cliq Integration - -**Step 1:** Sign in to your TestMu AI account and visit **Integrations** from the left sidebar menu. - -**Step 2:** From the **My Integrations** section, click the **remove option** under the Zoho Cliq block. - -> That was all you need to know for TestMu AI + Zoho Cliq Integration. Increase your productivity with our integrations. If you still have any questions for us, please feel free to let us know. Our experts are always available on **chat** to help you out with any roadblock regarding our product. Happy testing! diff --git a/static/docs/zohoproject-integration.md b/static/docs/zohoproject-integration.md deleted file mode 100644 index 77607683e..000000000 --- a/static/docs/zohoproject-integration.md +++ /dev/null @@ -1,55 +0,0 @@ -# Zoho Project Integration with TestMu AI - -> For the full site index for AI agents, see [llms.txt](https://www.testmuai.com/support/docs/llms.txt). - -Zoho Project is a powerful cloud-based project management solution that streamlines project tracking, bug management, and team collaboration through its robust customization and automation capabilities. - -The TestMu AI integration enables you to capture and log bugs directly from your test sessions to Zoho Project, complete with screenshots and test environment details, making your testing workflow more efficient. - -> Zoho Project Integration with TestMu AI is available for both freemium and premium plans. - -## Integrate Zoho Project With TestMu AI - -**Step 1:** Login to your TestMu AI account. You should have administrator or user level access. - -**Step 2:** Navigate to `Settings` -> `Integrations` -> `Bug Tracker`. - -**Step 3:** Click on **Install** on the Zoho Project block. - -**Step 4:** Select the region where your Zoho Project account is located. - -**Step 5:** Sign in to your Zoho Project account. - -**Step 6:** Click **Accept** when Zoho Project requests permission to access your Zoho account data. - -**Step 7:** Select your portal and then projects you want to integrate. - -**Step 8:** Click **Install**. TestMu AI will now be integrated with Zoho Project. - -## Log your First Bug - -**Step 1:** Ensure you have access to the appropriate Zoho Project workspace and project. - -**Step 2:** While testing on TestMu AI, if you encounter a bug, click on the **Mark as Bug** button available. - -**Step 3:** In the bug logging form, provide the necessary details. - -**Step 4:** Click on the **Create Issue** button. The bug will be logged into your Zoho Project, and you can view it directly from your Zoho Project dashboard. - -## Resync Zoho Project Integration - -To ensure that your Zoho Project integration stays up-to-date with the latest projects and users: - -**Step 1:** Navigate to **Settings** -> **Integrations** in your TestMu AI account. - -**Step 2:** Locate the Zoho Project integration and click on the **Resync** button. - -If your refresh token has expired, you will be prompted to reauthorize your Zoho Project integration. - -## Uninstall Zoho Project Integration - -**Step 1:** Navigate to `Settings` -> `Integrations` in your TestMu AI account. - -**Step 2:** Locate the Zoho Project integration and click on the **Remove** button to uninstall the integration. - -> That was all you need to know for TestMu AI + Zoho Project Integration. Increase your productivity with our integrations. If you still have any questions for us, please feel free to let us know. Our experts are always available on **chat** to help you out with any roadblock.